Clip: Emacs voice control - describing my setup during Linkarzu's livestream
| emacs, speech, speech-recognition
linkarzu wins the clip-off thanks to his fancy AI-powered clipping process! Here's a snippet from our conversation today. He'll be posting more clips, and I'm also going to edit the transcript of the conversation. Fun fun fun!
Additional notes:
- Substitution: I use
(add-hook 'substitute-post-replace-functions #'sacha-substitute-save-common-edits)to save the changes. sacha-substitute-save-common-edits Edit text has functions likesacha-subed-fix-common-errors-in-textthat apply those changes to my speech recognition results andsacha-subed-fix-common-errors-from-startthat apply them to already-prepared transcripts. - sachac/whisper.el has my fork of natrys/whisper.el to support voice commands. speech-input has a bunch of other things, too, and of course my Emacs config.
- I've been using phony to define more voice commands using a parser.
- Apparently I cannot spell when thinking out loud. It is Orgzly Revived, not ogrzly.
- This actually doesn't use generative AI in the sense of Gemini or Claude, except for the conversational French demo. I use a local speech recognition model (currently Orukeet) and local speech synthesis (Kokoro FastAPI), So depending on your feelings about large language models, the models might be small enough to be fine with you. Alternatively, there are classical approaches to both speech recognition and speech synthesis like kaldi and espeak-ng.
Transcript and screenshots
Sacha: It says Orgsley Revived. What is that even? But I can press Ctrl+; and it uses substitute.el.
Sacha: Orgzly Revived here. And what that does, in addition to the substitutions that happen throughout the buffer, so for example, if I'm adding a transcript, it will change that all throughout the transcript. I've also added some code because this is Emacs, I can just do that. that saves this in common errors or commonly misrecognized words. And then when I recognize text, whether it's before a voice command or if I'm just doing this or whatever, it will apply all of those common fixes. Common edits.
Sacha: Here we go. So it automatically adds it to this thing here, which gets saved automatically for me. And that's the change that I've just added. Linkarzu: Interesting. Pretty cool. Sacha: So it's really nice to have that kind of [dictionary] shortcut built into the way that I edit things. All I know is I just hit Ctrl+; and put in what it's supposed to be. And then it gets not only fixed in my current file, but in the future as well. Linkarzu: Yeah, I need to work on my transcript game.
Linkarzu: What other tricks do you want to show us there? What other things do you want to share about, brag about your Emacs workflow? Sacha: It's really hard to say. I told you how M-x doctor was the beginning of my experience, right? Linkarzu: Your journey, yep. Sacha: Yes, yes. So this is the doctor. You can say things like, "I'm feeling happy today," as I said, right? And then it uses that to generate its response. But I have this conversation mode. "Yes, especially when I get to tinker with Emacs." And then, uh... "What happened when you get to tinker with Emacs?" Did that...? Does the TTS come through? Yeah, it does. And the reason I did this was because after I showed off my vocal commands to Emacs Paris, they were like, oh, you know, what if you can have a conversation? So I was like, OK, let's come back full circle to the thing that got me curious about Emacs in the first place, and voice enable that. And then, of course, this is a very old AI. It's much more interesting if you apply it to the new models. So if I want to, for example, practice French. This is one of my voice commands here. Then I can... Doctor: "What would happen? I would appreciate it if you would continue." Sacha: Okay, here we are. Then I might say, "Je parle avec linkarzu." And then I think... There you go. It will eventually... Maybe? Thank you.
Gemini: "Ah, c'est super de discuter avec lui ! C'est toujours enrichissant de partager (to share) ses passions en direct. Quel sujet abordez-vous ensemble aujourd'hui?" Sacha: Alright, so this now uses a large language model. This one uses Gemini because gptel makes it super easy to plug in different models. So if I wanted to practice French, but I don't necessarily want to bug people because I take a while to think about my words and sometimes I'm looking stuff up, then at least I can simulate that kind of conversation. And there are a gazillion apps that will sell you a subscription to do this. But because it's here and it's plain text and I can control it, then I can more easily go through it afterwards and analyze the vocabulary or correct the things that it didn't get right, which I know is because of the speech recognition and not because of me or whatever, right? So it's so much fun to be able to tinker with Emacs to do what I wanted to do.
Sacha: Here I'm in live mode so my public tasks don't get mixed in with my private tasks. If I do an agenda, it should just be limited to the public stuff. Anyway, so the remind me created the note for me. It includes the audio in case the speech recognition didn't get anything right. And it includes a screenshot so that in case I'm doing this from a different application because my keyboard shortcut is global, I can still see the context and find that sort of stuff. So yeah.
Sacha: So this one here, for example, it says, remind me next month, et cetera, et cetera. And it has scheduled it for next month. Linkarzu: That's pretty cool. Sacha: I'm not actually going to pressure you to switch the Emacs. You can keep using Neovim and just borrow all the interesting ideas. org.nvim is there, I can adapt it to do this, you know?He hasn't released a clip of this one yet, but this exchange was one of my favourite parts of the main conversation:
[Linkarzu]: I don't know if like that's a particularity of Neovim users, we're all toxic and we just hate each other.
[Sacha]: Yeah, maybe you should just come over to the Emacs community. We have hot chocolate.
[Linkarzu]: If you go to the Emacs subreddit, it's like rainbows and butterflies. There's not a lot of hate. I always wonder why does that happen? It's not moderators, you know, it's just people. Is it just because it's old people? Like the professor, he doesn't want to fight any more. He's old enough. He retired already. He's just ready to pass to the next life. So what, what?
[Sacha]: Hypothesis number two, maybe your editor makes you grumpy.
That's at about 01:30:06 in this unlisted embed of the full conversation: