Sacha Chua - category - speech

Des commandes vocales pour ajouter des tâches sur Emacs

2026-10-07T19:52:45Z

Liens :

Orukeet pour la reconnaissance vocale parce qu'il est assez rapide et assez précis.
sachac/whisper.el
speech-input

0:00 Introduction

Je veux vous présenter ma petite expérience de la reconnaissance vocale pour dicter dans Emacs et pour les contrôler. J'utilise le modèle Orukeet pour la reconnaissance vocale parce qu'il est assez rapide et assez précis. Je l'intègre dans Emacs en utilisant mes bibliothèques sachac/whisper.el et speech-input. J'ai récemment créé des commandes vocales pour ajouter des éléments à ma liste de tâches. C'est étonnement utile parce que ça réduit la charge cognitive liée à la mémorisation des raccourcis clavier. Je vais vous montrer cela.

0:37 Ajouter des tâches

Je presse la touche zéro du pavé numérique avant et après avoir dit « Rappelle-moi demain de préparer une présentation pour Emacs Paris sur les commandes vocales. » La fonction crée une tâche dans ma boîte de réception. La tâche est planifiée pour demain. Il y a un lien vers l'enregistrement

1:09 D'autres dates

« Rappelez-moi demain de préparer une présentation pour Emacs Paris sur les commandes vocales », la capture d'écran et le contexte de tâche. Ça m'aident si j'ai été mal comprise ou si j'ai besoin de contexte. De la même manière, je peux dire « Rappelle-moi la semaine prochaine de faire la paperasse pour l'école. » « Rappelle-moi aujourd'hui de créer une vidéo sur mes commandes vocales. »

1:59 Basculer à une tâche

Je peux aussi créer une tâche et déclencher le suivi du temps (ou clock in). Par exemple, je peux dire « Commence à implémenter la génération des questions. »

2:15 Retrouver le fil de mes pensées

Si je perds le fil de mes pensées : « j'en étais où ? » Ça m'affiche les tâches récentes.

2:27 Je faisais quoi ?

« Je faisais quoi ? » Ça bascule sur la tâche actuelle. Voilà des commandes vocales pour ajouter des tâches.

View Org source for this post

You can e-mail me at sacha@sachachua.com.

Emacs Carnival Oct 2026: Exploring keys and voice input in Emacs

2026-10-07T12:18:31Z

Making more use of the keyboard

October's Emacs Carnival theme "Are keyboards and M-x here to stay?" is a good nudge to think about the voice interface I've been experimenting with and where I want to take it.

Like Chris Maiorana, I love the way Emacs's M-x (execute-extended-command) allows me to call any command by name. It means that I can call commands or come up with my own commands without worrying about assigning them to keyboard shortcuts or making sense of where to place them in a hierarchical menu. With flexible completion via the vertico and orderless packages, I don’t even need to type in the full name of the command or remember the words in the right order. If I still can’t remember the command, it’s easy enough to use defalias to make my own nickname for it.

Figure 1: I can filter M-x to show just my custom commands

I tend to limit keyboard shortcuts to only my most common commands. Everything else is hard to practise enough so that it sticks in my muscle memory. Even with marginalia showing me keybindings when I use M-x and which-key showing me hints when I pause, it doesn’t sink in.

Figure 2: Marginalia shows keybindings

Maybe I could use key-quiz to drill a small number of keyboard shortcuts I’m actively working on learning, or modify help-quick-sections to show me reminders when I use C-h C-q (help-quick-toggle).

Getting one keyboard shortcut to do the right thing in different contexts is great because it means fewer shortcuts to memorize. Sometimes I can do that by binding the right functions to different mode maps with bind-key, like the way my sacha-org-insert-link-dwim uses the right link syntax in Markdown or OddMuse too. I also like the context-sensitive keyboard shortcuts I get from Embark. I find myself using a handful of those regularly, and C-. (embark-act) starts a bunch of shortcuts across different types of things: i to insert, w to copy, and so on. For example, I know C-x C-f (find-file) will let me select a file to open. Instead of opening it, I can use C-. i to insert the filename into the buffer.

Simpler keyboard shortcuts help. I rarely use the numpad on my laptop keyboard, so those keys can be mapped to functions that don't need to be followed by lots of typing. For example, 0 is a large button. I use it to start and stop speech recognition. Actually, I bind f9 in Emacs to sacha-whisper-run-at-point, and I bind numpad-0 as a keyboard shortcut in KDE (my window manager) instead of just in Emacs so that it's available globally. I'm still on X11 instead of Wayland. The shell script xdotool-emacs focuses Emacs briefly and sends the keys, and the shortcut is set to /home/sacha/bin/xdotool-emacs key --clearmodifiers F9. I do that instead of using emacsclient because emacsclient starts a different buffer, and I might want to keep the context. My sacha-whisper-maybe-type function detects if Emacs is not the focused application and types the text if it isn't, which means I can use it as keyboard input for oher applications while still taking advantage of the corrections and commands I've defined in Emacs. An easy-to-press global shortcut means I can use it from anywhere, even when my foreground app is the browser or another application.

An additional modifier opens up more possibilities for keyboard shortcuts near the regular typing position. My Windows key acts like a Super modifier. I've set it up as a one-shot modifier with Kanata so that I can either hold it down to access more shortcuts or tap it to treat it like a leader key.

There are quite a few commands for which neither M-x nor keyboard shortcuts feel like the right fit: things I want to do faster or more fluidly than M-x, but which I don't do often enough to memorize a good shortcut for. I've been experimenting a bit with voice input, and I really like it.

Some reasons why I'm curious about speech recognition

I could use more practice speaking, since I don't talk to a lot of people in real life.
- English: Videos and livestreams can be a way to share interesting things and start conversations. I want to organize my thoughts, improve my breathing, and reduce my stutters and false starts. That will make the audio more pleasant to listen to. Dictating to my computer gives me more practice. It also makes things easier to livestream if I can reuse my speech as input.
- French: I like challenging my brain and I want to help the kiddo learn it too. Improving my speaking will also make it easier for me to listen to myself and close the feedback loop. Maybe I can even use speech recognition to give me feedback and increase motivation while speaking, like keeping track of words or phrases I want to use. When I use them in conversation, the buffer can cross them off the list, which lets me focus on the words remaining.
  
  Figure 3: Screenshot showing balance and target words
Dictation can capture ideas quickly, with more natural wording, especially since typing sometimes gets messed up on my computer.
Voice interfaces might let me reduce some kinds of task friction or work hands-free. For example, I like setting timers by voice on my phone while cooking. It might be interesting to be able to dictate a draft while knitting or hand-sewing.
Brain fog is something I will likely need to deal with. I got a preview of it thanks to the sleep deprivation of early parenting. I know that brain fog is probably going to happen around menopause, it will probably also be part of aging, and it's a common symptom of long COVID which I might still end up getting despite my precautions. I could get into an accident and hurt my hands. I like Emacs and I want to be able to keep using it even if cognitive or physical changes make it harder to remember or use lots of keyboard shortcuts. If I start playing around with this now, the capabilities and the habits will be there when I need it.
Other people are already dealing with those kinds of physical or cognitive constraints, so things I figure out might be useful for other people.
A global shortcut for voice notes and commands let me quickly dash off a reminder before I take off. I don't have to find Emacs and go through my Org capture templates.
It's fun. I like saying "Prepare for take-off" and having Emacs get ready for a spontaneous stream by updating YouTube using my current Org Mode subtree title and description. I like seeing if my dictionary of substitutions can fix the commonly misrecognized words. Saying "I command you to…" to get Emacs to do a M-x command is totally inefficient but also playful, and silly things stick in my brain better thanks to the mnemonic advantage of humor. My brain likes that kind of novelty.

For context, I use Linux on a Lenovo P52 laptop (manufactured 2018, 64 GB RAM but hardly any GPU), and I have an Android phone. I like writing regular text, Emacs Lisp, and some Python and JavaScript. I'm slowly starting to have more focused time now that the kiddo is in virtual school, although it's still broken up by recess and lunch. I will probably have an increasing amount of focused time over the next few years.

Quick benefits from voice commands

I love the way voice reduces context shifts. For example, some of my most-used voice commands are:

remind me [today/tomorrow/next week/next month/someday] to…: capture and schedule the task; include an Org link and screenshot for context, and link to the audio recording in case the text was misrecognized

Figure 4: Captured tasks have an audio recording, a screenshot, and a link
switch to…: capture the task and immediately clock into it
what was I doing?: jump to the current task
where was I?: show recently-clocked tasks

Figure 5: This is the same as C-u M-x org-clock-goto

I don't have to think about which keys I've configured in org-capture-templates. I just use a natural phrase.

I've defined "open …" to open things from my bookmarked links stored in an Org file. Because my numpad-0 keybinding works from anywhere, I can even use it to open sites or capture notes while looking at my browser.

Because Emacs can interact with lots of other applications, I can even experiment with controlling those applications by voice. For example, I use obs-websocket.el to control OBS for streaming or recording a video. Using "Let's go live" and "Over and out" to start or stop streaming or "Start recording" and "Stop recording" to manage the recording means I can do that without needing to find the right window. I stream with a 10-second delay so that there's a little bit of time to catch any private information that's been accidentally shared on the screen. My new "panic" command lets me discard that delay and interrupt the stream by using an OBS keyboard shortcut to kill the stream immediately, even if I'm probably not going to remember that exact keyboard shortcut when I'm stressed. It should be a little easier to remember that than a keyboard shortcut I hardly ever use or a menu item that is a really small target (and my Emacs might not even be my foreground app).

For regular dictation, I'm experimenting with having it automatically place each sentence on a new line or list item. This makes it easy to reorganize in Org. I can refill it into a paragraph when I'm done.

I still have to deal with the limits of my memory, of course. I have a "what can I say" command that opens up an Org file with my notes. I keep this as a manual note instead of just dumping the contents of my sacha-whisper-commands variable so that I can prioritize the things I'm trying to build as habits.

Here's the part of my config that sets up some speech commands.

I might want more complex grammar someday for things that regular expressions can't easily handle. That's something other projects have done a good job of figuring out.

Prior work on voice control of Emacs: Voicemacs, Emacslisten

There's been a lot of good work on voice input over the years, especially since a number of Emacs users have a hard time with or would like to avoid repetitive stress injury. In addition to the innumerable projects that let you use Whisper or similar speech recognition engines to dictate prose, there are also a few projects that use a constrained vocabulary to more reliably control Emacs. Two projects in this area are based on the Talon voice control system: Voicemacs and Emacslisten.

The Talon community hub defines some Emacs commands. I think it simulates keyboard input and assumes that you have flx or some other fuzzy completion matching so that input like M-x t-d-on-e runs toggle-debug-on-error, as defined in community/apps/emacs/emacs_commands.csv. I seem to be having a hard time with getting orderless to do that with orderless-flex, so I'm just going to remove that column of the CSV from my emacs_commands.csv.
Emacslisten uses Emacs Lisp macros to define the command vocabulary. Cursorfree defines a whole bunch of them, using hatty for voice navigation within a buffer.
Voicemacs defines its commands in Python in the user's Talon config (see example). It feels fairly complex, like it assumes it's being cloned as your ~/.talon/user directory. It lets Emacs and Talon communicate via RPC to send additional context. It also extends some things like dired and company to make them more voice-friendly.

I'm new to both projects, so I'm not sure if I'm reading this right, but I think I might be able to use Emacslisten to define voice commands with Emacs Lisp, and then maybe someday use Voicemacs to provide richer context for some of those commands. For example, Voicemacs can list snippets, but I'm not quite sure how to use that yet.

It looks like I can use Talon for commands and my whisper.el setup for regular dictation at the same time. The Pipewire virtual sinks help with that, I think, since the programs don't have to have an exclusive lock on the audio device. This means that I can explore the kinds of vocabularies that people have set up while also taking advantage of greater accuracy and ease of correction in my whisper dictation workflow. I think I can use the Talon process socket to turn off recognition when I want to dictate using my whisper.el setup, and then I can turn it back on afterwards.

A note about the underlying speech recognition: Talon is cross-platform and free, but not open source. Subscribing to Talon (CAD 36/month) lets you use a Conformer + Whisper hybrid engine that uses whisper-large for more accuracy, but that's a bit on the expensive side for me. Dragonfly is a free open source engine that can use Kaldi or CMU Pocket Sphinx for speech recognition, but I'm not sure what the performance is like.

Interestingly, because Talon simulates keystrokes, using the commands from the community set means keys sometimes arrive out of order on my computer. "split solo" is supposed to run M-x delete-other-windows, but it sometimes ends up sending M-x delete-other-wnidows instead. This is the same sort of annoyance that gets in the way of typing quickly on my computer. Keystrokes get processed out of order when the CPU gets busy, which is part of the reason why I find dictation to be more reliable than typing sometimes. I should fix that someday, but my attempts so far aren't working. (I wonder if it has something to do with the GPU under Linux; maybe I should drop back down to CPU only since it turns out I can't use the GPU for much anyway.) Quick fix, let's try adding a brief delay in my settings.talon:

settings():
    # Adds a small pause (in ms) between each character typed
    insert_wait = 5

    # Ensures modifiers (Shift/Ctrl) don't un-latch too fast
    key_wait = 1.0

I think Voicemacs uses a sevver and and Emacslisten uses emacsclient to communicate with Emacs directly, so they probably won't run into this keystroke problem.

It might be interesting to see if I can port some of my simpler commands over to Emacslisten or straight Talon so that they're always active. Then I can slowly work on getting used to the other commands people find useful, or incorporating them into my setup.

Imagining a hybrid approach: keyboard, voice, even mouse/stylus?

I love shifting fluidly from one form of input to another, depending on what makes sense at that moment.

For example, I often start my thoughts by drawing on a light dotted grid using Noteful on my iPad. That lets me quickly cluster ideas spatially, drawing connections and highlighting words as I figure out what I want to say.

2026-10-07-01 Experimenting with voice input in Emacs - rough braindump.jpeg

Then I use dictation and typing to sketch the outline, jumping around as thoughts occur to me. If I'm livestreaming at the same time, that helps me demonstrate things along the way and lets me benefit from people's questions. Dictation lets me explain things to people and capture the text at the same time.

I can move around with the keyboard, but it would be really cool to use voice for quick navigation between sections of my outline using either line numbers or words. I've implemented a "line" command that lets me jump to a line using the last two digits of its line number, since I probably won't have more than 100 lines visible on the screen at a time. For example, this line is currently 119, so referring to it as "nineteen" would work. I'd like to add more operations for editing the text. I think it would be great to be able to say "kill line 33" to delete that line or "move lines 16 to 20 after 27" to have it move those lines there.

I now have a sacha-whisper-one-sentence-per-line variable that splits up dictation if non-nil, so we'll see where those take me. Orukeet does a reasonable job of punctuating my sentences if I don't pause too much between phrases. My M-q re-wraps paragraphs between one sentence per line, one paragraph per line, and wrapping to fill-column, and it's probably the sort of thing that more voice commands would make even more convenient.

Maybe I can even use that spatial sense of a layout by touching buttons in an Emacs-served webpage on my tablet to direct my dictation to different places. Org-draw uses a similar Emacs web server approach to let you draw on another device and embed the results into an Org file, and there are also simpler HTTP servers (web-server, simple-httpd), so it's probably quite doable. I'm imagining something that takes the top-level items from my current nested list and makes them nice big buttons, so I could have something like "Introduction", "Keyboard", "Reasons", "Experiment", "Hybrid", and "Conclusion", and clicking on those could trigger a POST request to the server inside Emacs which moves my point accordingly. I might even have buttons for various commands. It sounds pretty straightforward to implement.

When my brain comes up with lots of interesting ideas, my org-capture voice commands let me choose which rabbit holes to postpone and which ones I want to dive into.

If I get interrupted by life, I can write on my phone while waiting for the kiddo. Orgzly Revived and Syncthing let me pick up where I left off.

As I get thoughts down, I can use keyboard shortcuts to rearrange things and then convert the list of sentences into paragraphs. I can easily turn text into links using my favourites.

I can imagine combining the audio clips from the text properties with the videos and screenshots to create a quick narrated clip that I can edit (using Emacs, of course), or publishing the narration so that people can listen to it if they like.

This combination of drawing + audio + text works better for me than any of those components alone. Visuals give me anchors for my thoughts. Audio helps with the braindump. Text permits precise, searchable details we can copy. Like George Jones, I find they all have their uses and their strengths.

When I try to dictate longer sections without that kind of visual support, I often lose my train of thought. I want to see where I've been and where I want to go. I don't trust LLMs to clean up that kind of rough transcript; the output doesn't quite resonate with me. I like keeping speech recognition constrained to sentences I can immediately check or reorganize, and I have the recordings as text properties so I can go back to that segment if I need to (at least during the current editing section). That way, I'm never left scratching my head and wondering what I meant.

I love that local speech recognition is getting better and better. I'm curious about whether small language models can do better turn detection to help compensate for my intra-phrase delays when I'm thinking of a word, and whether decision models can automatically classify a sentence into its place in my outline. I think it would be cool if embeddings or other language models could suggest related words or resources as I go along. Lots of interesting possibilities even with the technology that's already available.

Thanks to George Jones for hosting this month's Emacs Carnival on input alternatives. Check out the page for more posts!

View Org source for this post

You can e-mail me at sacha@sachachua.com.

Experimenting with Orukeet for speech recognition

2026-09-12T13:39:22Z

Watch on YouTube

This is a quick demonstration of using Orukeet as a speech recognition model inside Emacs. I'm recording this in real time on a Lenovo P52 laptop using only CPU. I can replay just one part of my recording. I can switch languages in the middle of recording too. Par exemple, maintenant je parle français. Seems promising.

Links/notes:

oruk/orukeet · Hugging Face
file:///home/sacha/vendor/orukeet/orukeet-server.py - I generated this with Google Gemini for a quick POC. It keeps the model in memory for slightly faster processing per segment.
Run Orukeet locally: Python speech recognition | oruk
- https://oruk.ai/examples/orukeet-local.py - Python script you can run from the command line to transcribe files.
file:///home/sacha/vendor/orukeet/orukeet-local.py
sachac/whisper.el - my fork of natrys/whisper.el
sacha-whisper-continue - This lets me cue transcriptions of previous segments while continuing to record new ones. I like it because I tend to pause while thinking out loud. I can define the chunk that will get transcribed so that I don't end up with incomplete chunks or lost context.

View Org source for this post

You can e-mail me at sacha@sachachua.com.

Categorizing Emacs News items by voice in Org Mode

2026-03-24T01:19:29Z

I'm having fun exploring which things might actually be easier to do by voice than by typing. For example, after I wrote some code to expand yasnippets by voice, I realized that it was easier to:

press my shortcut,
say "okay, define interactive function",
and then press my shortcut again,

than to:

mentally say it,
get the first initials,
type in "dfi",
and press Tab to expand.

Another area where I do this kind of mental translation for keyboard shortcuts is when I categorize dozens of Emacs-related links each week for Emacs News. I used to do this by hand. Then I wrote a function to try to guess the category based on regular expressions (my-emacs-news-guess-category in emacs-news/index.org, which is large). Then I set up a menu that lets me press numbers corresponding to the most frequent categories and use tab completion for the rest. 1 is Emacs Lisp, 2 is Emacs development, 3 is Emacs configuration, 4 is appearance, 5 is navigation, and so on. It's not very efficient, but some of it has at least gotten into muscle memory, which is also part of why it's hard to change the mapping. I don't come across that many links for Emacs development or Spacemacs, and I could probably change them to something else, but… Anyway.

Figure 1: Screenshot of my menu for categorizing links

I wanted to see if I could categorize links by voice instead. I might not always be able to count on being able to type a lot, and it's always fun to experiment with other modes of input. Here's a demonstration showing how Emacs can automatically open the URLs, wait for voice input, and categorize the links using a reasonably close match. The *Messages* buffer displays the recognized output to help with debugging.

Download the video

Screencast with audio: categorizing links by voice

This is how it works:

It starts an ffmpeg recording process.
It starts Silero voice activity detection.
When it detects that speech has ended, it use curl to send the WAV to an OpenAI-compatible server (in my case, Speaches with the Systran/faster-whisper-base.en model) for transcription, along with a prompt to try to influence the recognition.
It compares the result with the candidates using string-distance for an approximate match. It calls the code to move the current item to the right category, creating the category if needed.

Since this doesn't always result in the right match, I added an Undo command. I also have a Delete command for removing the current item, Scroll Up and Scroll Down, and a way to quit.

Initial thoughts

I used it to categorize lots of links in this week's Emacs News, and I think it's promising. I loved the way my hands didn't have to hover over the number keys or move between those and the characters. Using voice activity detection meant that I could just keep dictating categories instead of pressing keyboard shortcuts or using the foot pedal I recently dusted off. There's a slight delay, of course, but I think it's worth it. If this settles down and becomes a solid part of my workflow, I might even be able to knit or hand-sew while doing this step, or simply do some stretching exercises.

What about using streaming speech recognition? I've written some code to use streaming speech recognition, but the performance wasn't good enough when I tried it on my laptop (Lenovo P52 released in 2018, no configured GPU under Linux). The streaming server dropped audio segments in order to try to catch up. I'd rather have everything transcribed at the level of the model I want, even if I have to wait a little while. I also tried using the Web Speech API in Google Chrome for real-time speech transcription, but it's a little finicky. I'm happy with the performance I get from either manually queueing speech segments or using VAD and then using batch speech recognition with a model that's kept in memory (which is why I use a local server instead of a command-line tool). Come to think of it, I should try this with a higher-quality model like medium or large, just in case the latency turns out to be not that much more for this use case.

What about external voice control systems like Talon Voice or Cursorless? They seem like neat ideas and lots of people use them. I think hacking something into Emacs with full access to its internals could be lots of fun too.

A lot of people have experimented with voice input for Emacs over the years. It could be fun to pick up ideas for commands and grammars. Some examples:

Using Python to Code by Voice - YouTube (2013)
jgarvin/mandimus: Use speech recognition to command your computer and Emacs. · GitHub
ErikPrantare/cursorfree.el: Edit and navigate from anywhere in the buffer · GitHub
~lepisma/emacs-speech-input - uses the idea of a voice cursor, uses an LLM to execute editing instructions

What about automating myself out of this loop? I've considered training a classifier or sending the list to a large language model to categorize links in order to set more reasonable defaults, but I think I'd still want manual control, since the fun is in getting a sense of all the cool things that people are tinkering around with in the Emacs community. I found that with voice control, it was easier for me to say the category than to look for the category it suggested and then say "Okay" to accept the default. If I display the suggested category in a buffer with very large text (and possibly category-specific background colours), then I can quickly glance at it or use my peripheral vision. But yeah, it's probably easier to look at a page and say "Org Mode" than to look at the page, look at the default text, see if it matches Org Mode, and then say okay if it is.

Ideas for next steps

I wonder how to line up several categories. I could probably rattle off a few without waiting for the next one to load, and just pause when I'm not sure. Maybe while there's a reasonably good match within the first 1-3 words, I'll take candidates from the front of the queue. Or I could delimit it with another easily-recognized word, like "next".

I want to make a more synchronous version of this idea so that I can have a speech-enabled drop-in replacement that I can use as my y-or-n-p while still being able to type y or n. This probably involves using sit-for and polling to see if it's done. And then I can use that to play Twenty Questions, but also to do more serious stuff. It would also be nice to have replacements for read-string and completing-read, since those block Emacs until the user enters something.

I might take a side-trip into a conversational interface for M-x doctor and M-x dunnet, because why not. Naturally, it also makes sense to voice-enable agent-shell and gptel interactions.

I'd like to figure out a number- or word-based completion mechanism so that I can control Reddit link replacement as well, since I want to select from a list of links from the page. Maybe something similar to the way voicemacs adds numbers to helm and company or how flexi-choose.el works.

I'm also thinking about how I can shift seamlessly between typing and speaking, like when I want to edit a link title. Maybe I can check if I'm in the minibuffer and what kind of minibuffer I'm in, perhaps like the way Embark does.

It would be really cool to define speech commands by reusing the keymap structure that menus also use. This is how to define a menu in Emacs Lisp:

(easy-menu-define words-menu global-map
  "Menu for word navigation commands."
  '("Words"
     ["Forward word" forward-word]
     ["Backward word" backward-word]))

and this is how to set just one binding:

(keymap-set-after my-menu ""
  '("Drink" . drink-command) 'eat)

That makes sense to reuse for speech commands. I'd also like to be able to specify aliases while hiding them or collapsing them for a "What can I say" help view… Also, if keymaps work, then maybe minor modes or transient maps could work? This sort of feels like it should be the voice equivalent of a transient map.

The code so far

(defun my-emacs-news-categorize-with-voice (&optional skip-browse)
  (interactive (list current-prefix-arg))
  (unless skip-browse
    (my-spookfox-browse))
  (speech-input-cancel-recording)
  (let ((default (if (fboundp 'my-emacs-news-guess-category) (my-emacs-news-guess-category))))
    (speech-input-from-list
     (if default
         (format "Category (%s): " default)
       "Category: ")
     '(("Org Mode" "Org" "Org Mode")
       "Other"
       "Emacs Lisp"
       "Coding"
       ("Emacs configuration" "Config" "Configuration")
       ("Appearance" "Appearance")
       ("Default" "Okay" "Default")
       "Community"
       "AI"
       "Writing"
       ("Reddit" "Read it" "Reddit")
       "Shells"
       "Navigation"
       "Fun"
       ("Dired" "Directory" "Dir ed")
       ("Mail, news, and chat" "News" "Mail" "Chat")
       "Multimedia"
       "Scroll down"
       "Scroll up"
       "Web"
       "Delete"
       "Skip"
       "Undo"
       ("Quit" "Quit" "Cancel" "All done"))
     (lambda (result text)
       (message "Recognized %s original %s" result text)
       (pcase result
         ("Undo"
          (undo)
          (my-emacs-news-categorize-with-voice t))
         ("Skip"
          (forward-line)
          (my-emacs-news-categorize-with-voice))
         ("Quit"
          (message "All done.")
          (speech-input-cancel-recording))
         ("Reddit"
          (my-emacs-news-replace-reddit-link)
          (my-emacs-news-categorize-with-voice t))
         ("Scroll down"
          (my-spookfox-scroll-down)
          (my-emacs-news-categorize-with-voice t))
         ("Scroll up"
          (my-spookfox-scroll-up)
          (my-emacs-news-categorize-with-voice t))
         ("Delete"
          (delete-line)
          (undo-boundary)
          (my-emacs-news-categorize-with-voice))
         ("Default"
          (my-org-move-current-item-to-category
           (concat default ":"))
          (undo-boundary)
          (my-emacs-news-categorize-with-voice))
         (_
          (my-org-move-current-item-to-category
           (concat result ":"))
          (undo-boundary)
          (my-emacs-news-categorize-with-voice))))
     t)))

It uses Spookfox to control Firefox from Emacs:

(defun my-spookfox-scroll-down ()
  (interactive)
  (spookfox-js-injection-eval-in-active-tab "window.scrollBy(0, document.documentElement.clientHeight);" t))

(defun my-spookfox-scroll-up ()
  (interactive)
  (spookfox-js-injection-eval-in-active-tab "window.scrollBy(0, -document.documentElement.clientHeight);"))

(defun my-spookfox-background-tab (url &rest args)
  "Open URL as a background tab."
  (if spookfox--connected-clients
      (spookfox-tabs--request (cl-first spookfox--connected-clients) "OPEN_TAB" `(:url ,url))
    (browse-url url)))

It also uses these functions for categorizing Org Mode items:

(defun my-org-move-current-item-to-category (category)
    "Move current list item under CATEGORY earlier in the list.
  CATEGORY can be a string or a list of the form (text indent regexp).
  Point should be on the next line to process, even if a new category
  has been inserted."
    (interactive (list (completing-read "Category: " (my-org-get-list-categories))))
    (when category
      (let* ((col (current-column))
             (item (point-at-bol))
             (struct (org-list-struct))
             (category-text (if (stringp category) category (elt category 0)))
             (category-indent (if (stringp category) 2 (+ 2 (elt category 1))))
             (category-regexp (if (stringp category) category (elt category 2)))
             (end (elt (car (last struct)) 6))
             (pos (point))
             s)
        (setq s (org-remove-indentation (buffer-substring-no-properties item (org-list-get-item-end item struct))))
        (save-excursion
          (if (string= category-text "x")
              (org-list-send-item item 'delete struct)
            (goto-char (caar struct))
            (if (re-search-forward (concat "^ *- +" category-regexp) end t)
                (progn
                  ;; needs a patch to ol.el to check if stringp
                  (org-list-send-item item (point-at-bol) struct)
                  (org-move-item-down)
                  (org-indent-item))
              (goto-char end)
              (org-list-insert-item
               (point-at-bol)
               struct (org-list-prevs-alist struct))
              (let ((old-struct (copy-tree struct)))
                (org-list-set-ind (point-at-bol) struct 0)
                (org-list-struct-fix-bul struct (org-list-prevs-alist struct))
                (org-list-struct-apply-struct struct old-struct))
              (goto-char (point-at-eol))
              (insert category-text)
              (org-list-send-item item 'end struct)
              (org-indent-item)
              (org-indent-item))
            (recenter))))))

(defun my-org-guess-list-category (&optional categories)
  (interactive)
  (require 'cl-lib)
  (unless categories
    (setq categories
          (my-helm-org-list-categories-init-candidates)))
  (let* ((beg (line-beginning-position))
         (end (line-end-position))
         (string (buffer-substring-no-properties beg end))
         (found
          (cl-member string
                     categories
                     :test
                     (lambda (string cat-entry)
                       (unless (string= (car cat-entry) "x")
                         (string-match (regexp-quote (downcase (car cat-entry)))
                                       string))))))
    (when (car found)
      (my-org-move-current-item-to-category
       (cdr (car found)))
      t)))

For the speech-input functions, experimental code is at https://codeberg.org/sachac/speech-input .

View Org source for this post

You can comment on Mastodon or e-mail me at sacha@sachachua.com.

Using speech recognition for on-the-fly translations in Emacs and faking in-buffer completion for the results

2026-03-31T16:31:15Z

[2026-03-31 Tue]: I added the recognized text so that I can confirm what was translated. I also moved my-type-with-hint to learn-lang-type-with-hint.

When I'm writing a journal entry in French, I sometimes want to translate a phrase that I can't look up word by word using a dictionary. Instead of switching to a browser, I can use an Emacs function to prompt me for text and either insert or display the translation. The plz library makes HTTP requests slightly neater.

(defun my-lang-en-to-fr (text &optional display-only)
  (interactive (list (read-string "Text: ") current-prefix-arg))
  (let* ((url "https://translation.googleapis.com/language/translate/v2")
         (params `(("key" . ,(getenv "GOOGLE_API_KEY"))
                   ("q" . ,text)
                   ("source" . "en")
                   ("target" . "fr")
                   ("format" . "text")))
         (query-string (mapconcat
                        (lambda (pair)
                          (format "%s=%s"
                                  (url-hexify-string (car pair))
                                  (url-hexify-string (cdr pair))))
                        params
                        "&"))
         (full-url (concat url "?" query-string)))
    (let* ((response (plz 'get full-url :as #'json-read))
           (data (alist-get 'data response))
           (translations (alist-get 'translations data))
           (first-translation (car translations))
           (translated-text (alist-get 'translatedText first-translation)))
      (when (called-interactively-p 'any)
        (if display-only
            (message "%s" translated-text)
          (insert translated-text)))
      translated-text)))

I think it would be even nicer if I could use speech synthesis, so I can keep it a little more separate from my typing thoughts. I want to be able to say "Okay, translate …" or "Okay, … in French" to get a translation. I've been using my fork of natrys/whisper.el for speech recognition in English, and I like it a lot. By adding a function to whisper-after-transcription-hook, I can modify the intermediate results before they're inserted into the buffer.

(defun my-whisper-translate ()
  (goto-char (point-min))
  (let ((case-fold-search t))
    (when (re-search-forward "okay[,\\.]? translate[,\\.]? \\(.+\\)\\|okay[,\\.]? \\(.+?\\) in French" nil t)
      (let* ((s (or (match-string 1) (match-string 2)))
             (translation (save-match-data (my-lang-en-to-fr s))))
        (replace-match
         (propertize translation
                     'type-hint translation
                     'type-original s
                     'help-echo s))))))

(with-eval-after-load 'whisper
  (add-hook 'whisper-after-transcription-hook 'my-whisper-translate 70))

But that's too easy. I want to actually type things myself so that I get more practice. Something like an autocomplete suggestion would be handy as a way of showing me a hint at the cursor. The usual completion-at-point functions are too eager to insert things if there's only one candidate, so we'll just fake it with an overlay. This code works only with my whisper.el fork because it supports using a list of functions for whisper-insert-text-at-point.

(defun my-whisper-maybe-type-with-hints (text)
  "Add this function to `whisper-insert-text-at-point'."
  (let* ((hint (and text (org-find-text-property-in-string 'type-hint text)))
         (original (and text (org-find-text-property-in-string 'type-original text))))
    (if hint
        (progn
          (learn-lang-type-with-hint hint original)
          nil)
      text)))

Here's a demonstration of me saying "Okay, this is a test, in French.":

Download the video

Screencast of using speech recognition to translate into French and provide a hint when typing

Since we're faking in-buffer completion here, maybe we can still get away with considering this as an entry for Emacs Carnival February 2026: Completion ? =)

This is part of my Emacs configuration.

View Org source for this post

You can e-mail me at sacha@sachachua.com.

Using whisper.el to convert speech to text and save it to the currently clocked task in Org Mode or elsewhere

2026-01-13T18:38:54Z

[2026-10-02 Fri]: I now use Orukeet as the speech recognition model because it's faster.
[2026-01-30 Fri]: Major change: I switched to my fork of natrys/whisper.el so that I can specify functions that change the window configuration etc.
[2026-01-13 Tue]: Change main function to sacha-whisper-run, use seq-reduce to go through the functions.
[2026-01-09 Fri]: Added code for automatically capturing screenshots, saving text, working with a list of functions.
[2026-01-08 Thu]: Added demo, fixed some bugs.
[2026-01-04 Sun]: Added note about difference from MELPA package, fixed :vc

I want to get my thoughts into the computer quickly, and talking might be a good way to do some of that. whisper.el gives me a convenient way to interact with a server compatible with OpenAI Whisper's API. (Experimenting with Orukeet for speech recognition) Note: This is not the same whisper package as the one on MELPA. Here is how I have it set up for reasonable performance on my Lenovo P52 with just the CPU, no GPU.

I've bound to the command whisper-run in Emacs, but I use my window manager's custom shortcuts to map to execute /home/sacha/bin/xdotool-emacs key --clearmodifiers F9, which sends to Emacs. (file:///home/sacha/bin/xdotool-emacs) That makes it a shortcut that works even in other applications.

I press to start recording, talk, and then press to stop recording. By default, it inserts the text into the buffer at the current point. I've set whisper-return-cursor-to-start to nil so that I can keep going.

(use-package whisper
  :vc (:url "https://github.com/natrys/whisper.el")
  :load-path "~/vendor/whisper.el"
  :config
  (setq whisper--mode-line-recording-indicator "⏺")
  (setq whisper-quantize "q4_0")
  (setq whisper-install-directory "~/vendor")
  (setq whisper--install-path (concat
     (expand-file-name (file-name-as-directory whisper-install-directory))
     "whisper.cpp/"))
  (setq whisper-return-cursor-to-start nil)
  (setq whisper--ffmpeg-input-device "VirtualMicSink.monitor")
  (setq whisper-language "en")
  (setq whisper-recording-timeout 3000)
  (setq whisper-before-transcription-hook nil)
  (setq whisper-use-threads (1- (num-processors)))
  (setq whisper-transcription-buffer-name-function 'whisper--simple-transcription-buffer-name)
  (add-hook 'whisper-after-transcription-hook 'sacha-subed-fix-common-errors-from-start -100)
  :bind
  (("" . whisper-run)
   ("C-" . sacha-whisper-run)
   ("S-" . whisper-run)
   ("S-" . sacha-whisper-replay)
   ("M-" . sacha-whisper-toggle-language)))

[2026-10-02 Fri]: "Remind me to…" is now handled by https://codeberg.org/sachac/.emacs.d/src/branch/gh-pages/lisp/.user-lisp-autoloads.el.

Disk space is inexpensive and backups are great, so let's save each file using the timestamp.

(defvar sacha-whisper-dir "~/recordings/whisper/")
;;;###autoload
(defun sacha-whisper-set-temp-filename ()
  (setq whisper--temp-file (expand-file-name
                            (format-time-string "%Y-%m-%d-%H-%M-%S.wav")
                            sacha-whisper-dir)))

(with-eval-after-load 'whisper
  (add-hook 'whisper-before-transcription-hook #'sacha-whisper-set-temp-filename))

I can replay the most recently recorded snippet in case it's missed something and I've forgotten what I just said.

;;;###autoload
(defun sacha-whisper-replay (&optional file)
  "Replay the last temporary recording."
  (interactive (list
                (when current-prefix-arg
                  (read-file-name "File: " sacha-whisper-dir))))
  (setq whisper--temp-file (or file whisper--temp-file))
  (mpv-play whisper--temp-file))

;;;###autoload
(defun sacha-whisper-insert-retry (&optional file)
  (interactive (list
                (when current-prefix-arg
                  (read-file-name "File: " sacha-whisper-dir))))
  (whisper--cleanup-transcription)
  (setq whisper--marker (point-marker)
        whisper--temp-file (or file whisper--temp-file))
  (whisper--transcribe-audio))

Update [2026-10-02 Fri]: Il peut aussi comprendre le français. I used to have code to set the language in case the auto-recognition doesn't get it right, but now I'm using Orukeet as the speech recognition engine now, so it doesn't take a language setting.

;;;###autoload
(defun sacha-whisper-reset (text)
  (setq sacha-whisper-skip-annotation nil)
  (remove-hook 'whisper-insert-text-at-point #'sacha-whisper-org-save-to-clocked-task)
  text)

;; Only works with my tweaks to whisper.el
;; https://github.com/sachac/whisper.el/tree/whisper-insert-text-at-point-function
(with-eval-after-load 'whisper
  (setq whisper-insert-text-at-point
        '(sacha-whisper-scratch-that
          sacha-whisper-handle-commands
          sacha-whisper-save-text
          sacha-learn-lang-notice-bingo-words
;          sacha-whisper-save-to-file
          sacha-save-to-kill-ring-after-current
          sacha-whisper-maybe-expand-snippet
          sacha-speech-input-quantified-track
          sacha-whisper-maybe-type
          sacha-whisper-maybe-type-with-hints
          sacha-whisper-insert
          sacha-whisper-reset)))

(defvar sacha-whisper-last-annotation nil "Last annotation so we can skip duplicates.")
(defvar sacha-whisper-skip-annotation nil)
(defvar sacha-whisper-target-markers nil "List of markers to send text to.")

;;;###autoload
(defun sacha-whisper-insert (text)
  (let ((markers
         (cond
          ((null sacha-whisper-target-markers)
           (list whisper--marker)) ; current point where whisper was started
          ((listp sacha-whisper-target-markers)
           sacha-whisper-target-markers)
          ((markerp sacha-whisper-target-markers)
           (list sacha-whisper-target-markers))))
        (orig-point (point))
        (orig-buffer (current-buffer)))
    (when text
      (mapcar (lambda (marker)
                (with-current-buffer (marker-buffer marker)
                  (save-restriction
                    (widen)
                    (when (markerp marker) (goto-char marker))
                    (when (and (derived-mode-p 'org-mode) (org-at-drawer-p))
                      (insert "\n"))
                    (whisper--insert-text
                     (concat
                      (if (looking-back "[ \t\n]\\|^")
                          ""
                        " ")
                      (string-trim text)))
                    ;; Move the marker forward here
                    (move-marker marker (point)))))
              markers)
      (when sacha-whisper-target-markers
        (goto-char orig-point))
      nil)))

;;;###autoload
(defun sacha-whisper-maybe-type (text)
  "If Emacs is not the focused app, simulate typing TEXT.
Add this function to `whisper-insert-text-at-point'."
  (when text
    (if (frame-focus-state)
        text
      (make-process :name "xdotool" :command
                    (list "xdotool" "type"
                          text))
      nil)))

;;;###autoload
(defun sacha-whisper-clear-markers ()
  (interactive)
  (setq sacha-whisper-target-markers nil))

;;;###autoload
(defun sacha-whisper-use-current-point (&optional add)
  (interactive (list current-prefix-arg))
  (if add
      (push (point-marker) sacha-whisper-target-markers)
    (setq sacha-whisper-target-markers (list (point-marker)))))

;;;###autoload
(defun sacha-whisper-run-at-point (&optional add)
  (interactive (list current-prefix-arg))
  (sacha-whisper-clear-markers)
  (whisper-run))

(keymap-global-set "" #'sacha-whisper-run-at-point)
(keymap-global-set "" #'whisper-run)

;;;###autoload
(defun sacha-whisper-jump-to-marker ()
  (interactive)
  (with-current-buffer (marker-buffer (car sacha-whisper-target-markers))
    (goto-char (car sacha-whisper-target-markers))))

;;;###autoload
(defun sacha-whisper-use-currently-clocked-task (&optional add)
  (interactive (list current-prefix-arg))
  (save-window-excursion
    (save-restriction
      (save-excursion
        (org-clock-goto)
        (org-end-of-meta-data)
        (org-end-of-subtree)
        (if add
            (push (point-marker) sacha-whisper-target-markers)
          (setq sacha-whisper-target-markers (list (point-marker))))))))

;;;###autoload
(defun sacha-whisper-run (&optional skip-annotation)
  (interactive (list current-prefix-arg))
  (require 'whisper)
  (add-hook 'whisper-insert-text-at-point #'sacha-whisper-org-save-to-clocked-task -10)
  (whisper-run)
  (when skip-annotation
    (setq sacha-whisper-skip-annotation t)))

;;;###autoload
(defun sacha-whisper-save-text (text)
  "Save TEXT beside `whisper--temp-file'."
  (when text
    (let ((link (org-store-link nil)))
      (with-temp-file (concat (file-name-sans-extension whisper--temp-file) ".txt")
        (when link
          (insert link "\n"))
        (insert text)))
    text))

;;;###autoload
(defun sacha-whisper-org-save-to-clocked-task (text)
  (when text
    (save-window-excursion
      (with-current-buffer (if (markerp whisper--marker) (marker-buffer whisper--marker) (current-buffer))
        (when (markerp whisper--marker) (goto-char whisper--marker))
        ;; Take a screenshot maybe
        (let* ((link (and (not sacha-whisper-skip-annotation)
                          (org-store-link nil)))
               (region (and (region-active-p) (buffer-substring (region-beginning) (region-end))))
               (screenshot-filename
                (when (or
                       (null link)
                       (not (string= sacha-whisper-last-annotation link))
                       (not (frame-focus-state))) ; not in focus, take a screenshot
                  (sacha-screenshot-current-screen (concat (file-name-sans-extension whisper--temp-file) ".png")))))
          (if (org-clocking-p)
              (save-window-excursion
                (save-restriction
                  (save-excursion
                    (org-clock-goto)
                    (org-end-of-subtree)
                    (unless (bolp)
                      (insert "\n"))
                    (insert "\n")
                    (if (and link (not (string= sacha-whisper-last-annotation link)))
                        (insert
                         (if screenshot-filename
                             (concat "(" (org-link-make-string
                                          (concat "file:" screenshot-filename)
                                          "screenshot") ") ")
                           "")
                         link
                         "\n")
                      (when screenshot-filename
                        (insert (org-link-make-string
                                 (concat "file:" screenshot-filename)
                                 "screenshot")
                                "\n")))
                    (when region
                      (insert "#+begin_example\n" region "\n#+end_example\n"))
                    (insert text "\n")
                    (setq sacha-whisper-last-annotation link)))
                (run-at-time 0.5 nil (lambda (text) (message "Added clock note: %s" text)) text))
            ;; No clocked task, prompt for a place to capture it
            (kill-new text)
            (setq org-capture-initial text)
            (call-interactively 'org-capture)
            ;; Delay the window configuration
            (let ((config (current-window-configuration)))
              (run-at-time 0.5 nil
                           (lambda (text config)
                             (set-window-configuration config)
                             (message "Copied: %s" text))
                           text config))))))))

(with-eval-after-load 'org
  (add-hook 'org-clock-in-hook #'sacha-whisper-org-clear-saved-annotation))

;;;###autoload
(defun sacha-whisper-org-clear-saved-annotation ()
  (setq sacha-whisper-org-last-annotation nil))

Here's an idea for a function that saves the recognized text with a timestamp.

(defvar sacha-whisper-notes "~/sync/stream/narration.org")
;;;###autoload
(defun sacha-whisper-save-to-file (text)
  (when text
    (let ((link (org-store-link nil)))
      (with-current-buffer (find-file-noselect sacha-whisper-notes)
        (goto-char (point-max))
        (insert "\n\n" (format-time-string "%H:%M ") text "\n" (if link (concat link "\n") ""))
        (save-buffer)
        (run-at-time 0.5 nil (lambda (text) (message "Saved to file: %s" text)) text)))
    text))

Let's add it to the kill ring, but not at the most recent slot. That way, it's easy for me to get to, but it doesn't overwrite anything that I've copied.

;;;###autoload
(defun sacha-save-to-kill-ring-after-current (text)
  "Save TEXT to the kill ring, but not at the top spot."
  (when text
    (if kill-ring
        (let ((temp (pop kill-ring)))
          (kill-new text)
          (push temp kill-ring))
      (kill-new text)))
  text)

And now I can redo things if needed:

;;;###autoload
(defun sacha-whisper-redo ()
  (interactive)
  (setq whisper--marker (point-marker))
  (whisper--transcribe-audio))

I think I've just figured out my Pipewire setup so that I can record audio in OBS while also being able to do speech to text, without the audio stuttering. qpwgraph was super helpful for visualizing the Pipewire connections and fixing them.


flatpak run com.github.wwmm.easyeffects -q 2>/dev/null
systemctl --user restart pipewire pipewire-pulse wireplumber
sleep 5
nohup flatpak run com.github.wwmm.easyeffects --gapplication-service >/dev/null 2>&1 &

pactl unload-module module-null-sink 2>/dev/null
pactl unload-module module-loopback 2>/dev/null
pactl unload-module module-remap-source 2>/dev/null

pactl load-module module-null-sink \
  sink_name="VirtualMicSink" \
  sink_properties=device.description="VirtualMic_backend"

pactl load-module module-remap-source \
  source_name="VirtualMic" \
  master="VirtualMicSink.monitor" \
  source_properties="device.description='Actual_Virtual_Mic' device.icon_name='audio-input-microphone' device.class='filter'"

pactl load-module module-null-sink \
  sink_name="CombinedSink" \
  sink_properties=device.description=CombinedSink


if pactl list short sources | grep -i pci-0000; then
  pactl load-module module-loopback \
    source="alsa_input.pci-0000_00_1f.3.analog-stereo" \
    sink="VirtualMicSink" \
    latency_msec=200 \
    adjust_time=3 \
    source_output_properties="node.description='SysToVMic' node.name='SysToVMic' media.name='SysVToMic'" \
    sink_input_properties="node.description='SysToVMic' node.name='SysToVMic' media.role='filter'"
  pactl load-module module-loopback \
    source="alsa_input.pci-0000_00_1f.3.analog-stereo" \
    sink="CombinedSink" \
    latency_msec=200 \
    adjust_time=3 \
    source_output_properties="node.description='SysToCombined' node.name='SysToCombined' media.name='SysVToMic'" \
    sink_input_properties="node.description='SysToCombined' node.name='SysToCombined' media.role='filter'"
fi

if pactl list short sources | grep -i USB_Advanced; then
    # YETI=alsa_input.usb-C-Media_Electronics_Inc._USB_Advanced_Audio_Device-00.analog-stereo
    YETI_OUT=alsa_output.usb-C-Media_Electronics_Inc._USB_Advanced_Audio_Device-00.analog-stereo
    YETI=easyeffects_sink.monitor
  # pactl load-module module-loopback \
  #   source="$YETI" \
  #   sink="VirtualMicSink" \
  #   latency_msec=200 \
  #   adjust_time=3 \
  #   source_output_properties="node.description='YetiToVMic' node.name='YetiToVMic' media.name='YetiToVMic'" \
  #   sink_input_properties="node.description='YetiToVMic' node.name='YetiToVMic' media.role='filter'"
  pactl load-module module-loopback \
    source="$YETI" \
    sink="VirtualMicSink" \
    latency_msec=200 \
    adjust_time=3 \
    source_output_properties="node.description='YetiToVMic' node.name='YetiToVMic' media.name='YetiToVMic'" \
    sink_input_properties="node.description='YetiToVMic' node.name='YetiToVMic' media.role='filter'"
  pactl load-module module-loopback \
    source="$YETI" \
    sink="CombinedSink" \
    latency_msec=200 \
    adjust_time=3 \
    source_output_properties="node.description='SysToCombined' node.name='SysToCombined' media.name='SysVToMic'" \
    sink_input_properties="node.description='SysToCombined' node.name='SysToCombined' media.role='filter'"
  pactl load-module module-null-sink sink_name=MeetSink sink_properties=device.description="Google_Meet_Audio"
  pactl load-module module-loopback source="$YETI" sink=MeetSink

  pactl load-module module-combine-sink sink_name=SnippetBridge slaves="MeetSink,$YETI_OUT" label="Snippet Bridge"
fi

pactl load-module module-null-sink \
  sink_name="ExtraSink1" sink_properties=device.description=ExtraSink1

pactl load-module module-loopback \
  source="ExtraSink1.monitor" \
  sink="CombinedSink" \
  source_output_properties="node.description='ExtraSink1ToCombined' node.name='ExtraSink1ToCombined' media.name='ExtraSink1ToCombined'" \
  sink_input_properties="node.description='ExtraSink1ToCombined' node.name='ExtraSink1ToCombined' media.role='filter'" \
  latency_msec=200 adjust_time=3

pactl set-default-source VirtualMic

Here's a demo:

Screencast of using whisper.el to do speech-to-text into the current buffer, clocked-in task, or other function

Transcript

Controlling my Android phone by voice

2025-01-30T21:27:02Z

[2025-01-30 Thu]: Fix timestamp format in toggle recording task.

I want to be able to use voice control to do things on my phone while I'm busy washing dishes, putting things away, knitting, or just keeping my hands warm. It'll also be handy to have a way to get things out of my head when the kiddo is koala-ing me. I've been using my Google Pixel 8's voice interface to set timers, send text messages, and do quick web searches. Building on my recent thoughts on wearable computing, I decided to spend some more time investigating the Google Assistant and Voice Access features in Android and setting up other voice shortcuts.

Tasker routines

I switched back to Google Assistant from Gemini so that I could run Tasker routines. I also found out that I needed to switch the language from English/Canada to English/US in order for my Tasker scripts to run instead of Google Assistant treating them as web searches. Once that was sorted out, I could run Tasker tasks with "Hey Google, run {task-name} in Tasker" and parameterize them with "Hey Google, run {task-name} with {parameter} in Tasker."

Voice Access

Learning how to use Voice Access to navigate, click, and type on my phone was straightforward. "Scroll down" works for webpages, while "scroll right" works for the e-books I have in Libby. Tapping items by text usually works. When it doesn't, I can use "show labels", "show numbers", or "show grid." The speech-to-text of "type …" isn't as good as Whisper, so I probably won't use it for a lot of dictation, but it's fine for quick notes. I can keep recording in the background so that I have the raw audio in case I want to review it or grab the WhisperX transcripts instead.

For some reason, saying "Hey Google, voice access" to start up voice access has been leaving the Assistant dialog on the screen, which makes it difficult to interact with the screen I'm looking at. I added a Tasker routine to start voice access, wait a second, and tap on the screen to dismiss the Assistant dialog.

Start Voice.tsk.xml - Import via Taskernet

Start Voice.tsk.xml