What’s New?
New language features:
- auto switch when TW language switched (opt-in on plugin settings)
- more settings, such as continuous listening option!
- Loads of more languages!
- minor BUG fixes
- and lots more!
… And custom VoiceCommands are possible!
Create a tiddler tagged $:/tags/VoiceCommand
Fill its text field with your desired actions
Use the field voice-commands to add a list of commands that should trigger the actions
Links etc… here in this thread please
The Link to the GitHub project page Sign in to GitHub · GitHub
The Link to the Demo page https://speech-to-text.finnsoftware.net/
The Link to the GitHub project page GitHub - flancast90/Speech-To-Text-in-TW5: An implementation of the Web Speech API for Jermolene's Tid
The Link to the Demo page https://speech-to-text.finnsoftware.net/
Thanks BTC, and especially Flanc…
This is seriously cool, I will try and contribute more moving forward.
I can’t believe how I could switch to the French language and even my poor French was recognised as I spoke it. Wow. As I said before to practice a new language and succeed at having the correct words transcribed is a revolutionary tool. No need to always talk to native speakers to practice 
Some thoughts
- I feel perhaps I need to identify easy ways to have tiddlers read back to us as a complement, I am sure there are similar tools for browsers, whether or not they can be built in like this transcription process. There was a plugin that reads text with a progress indication over the text as it reads, I will share when I locate it.
- Windows already has text to voice readers,
- I do look forward, if possible to inserting transcriptions into existing text, which I understand is on the list of features.
- The commands seem to be working well.
- I do not follow how to add multiple new keywords, I am just a little confused.
- Is it possible to capture the actual audio and save that in a tiddler as well?
- ie using voice not only to transcribe but also to trigger a recording.
Speculation
-
The reason I ask question 5 above, is along with 1. above we could build a wiki for managing reading and playing spoken word as a script or like karaoke. Podcasters, radio producers etc… may find this useful. Then editing content could be as simple as rearranging tiddlers, be it textural and/or audio read what you hear, and see what is up next.
-
If this was running on top of a node server we could imagine the operating system responding when detecting a new tiddler containing instructions.
-
I listen to a lot of spoken work podcasts. Perhaps one day I can just play a favorite quote into my microphone, to capture it in tiddlywiki, a bit like how I can copy and past a types transcript.
Regards
Your excited tiddlywikian
Tones
Hi @Tones,
To answer 4)
There’s the tiddler $:/config/speech-to-text/keywords that can be used to add multiple new keywords
Use its fields “ok-keywords” and “wiki-keywords” for that
There’s the rule that we’re using two keywords which are “Ok” and “Wiki”
But the $:/config/speech-to-text/keywords can be used to extend them
An example:
ok-keywords: bonjour Bonjour
wiki-keywords: madame Madame
BurningTreeC wrote:
The Link to the GitHub project page https://github.com/flancast90/Speech-To-Text-in-TW5
The Link to the Demo page https://speech-to-text.finnsoftware.net/
Thanks BTC & Flanc for the update beta.
I will check over next few days & comment back
Best wishes, TT
Ciao Flanc & BTC …
I agree with @Tones “Wow!”.
And the tool has already changed how I use some wiki.
ONE point I’d like to quickly make whilst the iron is hot. (Which I will fill in details for later with more precision.)
(FYI, I’m far less concerned with controlling the wiki outwith dictation. I’m also not that interested in whether it can “read-back” or record to an audio file.
First I need to get basic transcription working really well!
To give simple examples of, I think, main dictation focused needs …
1 - A way to verbally insert PUNCTUATION. (Without it, long dictation produces unwieldy texts.)
2 - A way to verbally INSERT “boilerplate text”.
For instance: "Ok Wiki insert exercise intro". Which might(?) be text in a Tiddler called “Exercise Intro” like “Lie on you back and rest a moment, scanning for how your back contacts the floor. Notice which parts touch it and which parts don’t.”
You get the idea?
What I don’t currently understand is how in dictation you’d execute a command to INSERT “boilerplate” text into the ongoing transcription?
3 - A way to CONCATENATE transcript fragments.
WHAT do I mean? In a test I dictated a long text in small segments. The result being that I ended up with …
- New Transcript
- New Transcript 1
… through to … - New Transcript 23
Manually joining those would be a fag. Maybe we need a way to click a button to CONCATENATE (latest last) all those new transcript fragments?
Just a thought, and likely YOU don’t need to devise that as a standard TW macro could likely do it?
BUT I did think it a real functional need for folk interested in serious dictation to be able to manipulate Transcripts, and this is one of the needs?
It is also fun to play with!
Thanks & best wishes
TT
@TiddlyTweeter:
To answer just your question about punctuation, I have tested the plugin with various punctuation words. The result is, by saying “comma”, you have a comma. The same works for question marks, periods, etc. Some of your other points, I felt were really good, as well. Hopefully @BurningTreeC and I can implement them in later versions of the plugin!
@TiddlyTweeter
at https://cloud.google.com/speech-to-text/docs/languages you can check if your chosen language supports spoken punctuation
the insertion of boilerplate text may be possible, we’ll have to think about it a little bit
concatenating transcript fragments could be done by a simple button. Maybe we should add $:/tags/TranscriptFragment to a new Transcript Tiddler so that filtering for transcript tiddlers gets easier?
Speech Recognition is not perfect, it sometimes doesn’t get the text fully right. That’s a limitation we cannot change
best wishes
Simon
Ciao Simon …
…Speech Recognition is not perfect, it sometimes doesn’t get the text fully right. That’s a limitation we cannot change
Right! For me it is not a big issue as the recognition (in the US & UK English I have tested most) is a zillion times better than other systems I have used in the past which were just ineffective and frustratingly useless.
Basically the recognition is high. It is a lot, lot quicker than typing, whatever the residual (cloud intelligence) issues.
Best wishes
Josiah
Simon wrote:
… the insertion of boilerplate text may be possible, we’ll have to think about it a little bit
Thank you! For my use cases it would be invaluable. And maybe for others too!
(Side note: I am a bit disappointed you are not getting more feedback. FYI, I posted notes about the your tool on Twitter and to two TW Telegram groups to try and get a bit more feedback.)
… concatenating transcript fragments could be done by a simple button. Maybe we should add $:/tags/TranscriptFragment to a new Transcript Tiddler so that filtering for transcript tiddlers gets easier?
Sounds like it could be useful to do that!
Best wishes
Josiah
@TiddlyTweeter
Thank you for the advertisement, I think it’s needed. I don’t know how many people are already using this plugin, feedback would be very welcome and would also be motivation for further development
best wishes,
Simon
Simon (BTC) wrote …
Thank you for the advertisement, I think it’s needed. I don’t know how many people are already using this plugin, feedback would be very welcome and would also be motivation for further development.
One thing I commented on Telegram that I think is TRUE, that might interest you …
-
… Speech input in TW could be a game changer as the TW UI system is ideal to leverage speech-to-text interface elements fully.
Best wishes
Josiah (@TiddlyTweeter)
Flanc …
I can confirm you can sing in the lyrics for VIP IN TRIP (Italiano [it-IT], Fabri Fibri) and the output is basically …
Federico, mi guardi su Wikipedia
Se sotto la voce di “rapper superfighissimo” c’è la mia faccia?
Ah, non c’è? E che faccia c’è, scusa?
Fabri Fibra è tanta roba, come il detto
Comandare è meglio che scopare, di chi era questa?
Io lo so, lo diceva (come non detto) … etc
https://www.youtube.com/watch?v=9CxSDqNrxPM
It needs a bit of Layout Massage after for new lines and missing punctuation & a couple of accents.
TT, researcher
Ciao Flanc
Following your comment I tried PUNCTUATING with “comma”.
It worked ONCE.
BUT I could not get it to work afterwards, though the cloud machine can …
… recognise the difference between a comma and a coma
Any idea why, & how to get it working reliably?
Best wishes, TT
@TiddlyTweeter,
It’s strange that punctuation only worked once. I can look at the code and try to replicate your situation, but if that is truly happening, it is most likely an API issue, and can’t be fixed through the plug-in.
Thanks!
1 Like
DUNNO! I tried looking at the Google Cloud docs for their implementation of the API for clues … but I get lost as it quickly gets too technical for my modest brain ;-(.
Leave it with me a few days. I can test on Chromebook later to see if it works better on it.
So … later
TT
flanc...@gmail.com wrote …