Using speech recognition for on-the-fly translations in Emacs and faking in-buffer completion for the results
Posted: - Modified: | audio, speech-recognition, emacs, speech: I added the recognized text so that I can confirm what was translated. I also moved my-type-with-hint to learn-lang-type-with-hint.
When I'm writing a journal entry in French, I sometimes want to translate a phrase that I can't look up word by word using a dictionary. Instead of switching to a browser, I can use an Emacs function to prompt me for text and either insert or display the translation. The plz library makes HTTP requests slightly neater.
(defun my-lang-en-to-fr (text &optional display-only)
(interactive (list (read-string "Text: ") current-prefix-arg))
(let* ((url "https://translation.googleapis.com/language/translate/v2")
(params `(("key" . ,(getenv "GOOGLE_API_KEY"))
("q" . ,text)
("source" . "en")
("target" . "fr")
("format" . "text")))
(query-string (mapconcat
(lambda (pair)
(format "%s=%s"
(url-hexify-string (car pair))
(url-hexify-string (cdr pair))))
params
"&"))
(full-url (concat url "?" query-string)))
(let* ((response (plz 'get full-url :as #'json-read))
(data (alist-get 'data response))
(translations (alist-get 'translations data))
(first-translation (car translations))
(translated-text (alist-get 'translatedText first-translation)))
(when (called-interactively-p 'any)
(if display-only
(message "%s" translated-text)
(insert translated-text)))
translated-text)))
I think it would be even nicer if I could use speech synthesis, so I can keep it a little more separate from my typing thoughts. I want to be able to say "Okay, translate …" or "Okay, … in French" to get a translation. I've been using my fork of natrys/whisper.el for speech recognition in English, and I like it a lot. By adding a function to whisper-after-transcription-hook, I can modify the intermediate results before they're inserted into the buffer.
(defun my-whisper-translate ()
(goto-char (point-min))
(let ((case-fold-search t))
(when (re-search-forward "okay[,\\.]? translate[,\\.]? \\(.+\\)\\|okay[,\\.]? \\(.+?\\) in French" nil t)
(let* ((s (or (match-string 1) (match-string 2)))
(translation (save-match-data (my-lang-en-to-fr s))))
(replace-match
(propertize translation
'type-hint translation
'type-original s
'help-echo s))))))
(with-eval-after-load 'whisper
(add-hook 'whisper-after-transcription-hook 'my-whisper-translate 70))
But that's too easy. I want to actually type things myself so that I get more practice. Something like an autocomplete suggestion would be handy as a way of showing me a hint at the cursor. The usual completion-at-point functions are too eager to insert things if there's only one candidate, so we'll just fake it with an overlay. This code works only with my whisper.el fork because it supports using a list of functions for whisper-insert-text-at-point.
(defun my-whisper-maybe-type-with-hints (text)
"Add this function to `whisper-insert-text-at-point'."
(let* ((hint (and text (org-find-text-property-in-string 'type-hint text)))
(original (and text (org-find-text-property-in-string 'type-original text))))
(if hint
(progn
(learn-lang-type-with-hint hint original)
nil)
text)))
Here's a demonstration of me saying "Okay, this is a test, in French.":
Since we're faking in-buffer completion here, maybe we can still get away with considering this as an entry for Emacs Carnival February 2026: Completion ? =)