Categories: geek » emacs

View topic page - RSS - Atom - Subscribe via email
Recommended links:

Obscure Emacs package appreciation: backup-walker

| emacs

The Emacs Carnival theme for September is obscure packages, which made me think of how the backup-walker package saved me from having to write some code all over again. Something went wrong when I was editing my config in Org Mode. I probably accidentally deleted a subtree due to over-enthusiastic speed commands. (… Maybe I should make my k shortcut for my-org-cut-subtree-or-list-item only work in my Inbox.org and news.org files.) Chunks of my literate Emacs configuration were gone, including the code that defined my-org-insert-link-dwim. Before I noticed, I'd already exported my (now slightly shorter) Emacs configuration file with org-babel-tangle and restarted Emacs. I couldn't recover the definition from memory using symbol-function. I couldn't use vundo to browse the Emacs undo tree. As usual, I'd been neglecting to commit my config changes to Git, so I couldn't restore a previous version. Oops.

Well, not the first time I've needed to rewrite code from scratch because of a brain hiccup. I started to reimplement the function. Then I remembered that I had other backups. I have a 2 TB SSD in my laptop, and I had configured Emacs to neatly save numbered backups in a separate directory, keeping all the versions without deleting any of the old ones.

(setq backup-directory-alist '(("\\.env$" . nil)
                               ("." . "~/.config/emacs/backups")))
(with-eval-after-load 'tramp
  (setq tramp-backup-directory-alist nil))
(setq delete-old-versions -1)
(setq version-control t)
(setq auto-save-file-name-transforms '((".*" "~/.config/emacs/auto-save-list/" t)))

At the moment, there are about 12,633 files adding up to 3 GB. Totally worth it for peace of mind. I could probably use grep to search for the function, but it wasn't easy to see what changed between versions.

I had learned about backup-walker in the process of writing about Thinking about time travel with the Emacs text editor, Org Mode, and backups. So I used backup-walker to flip through my file's numbered backups in much the same way that git-timemachine lets you flip through Git versions of a file. After M-x backup-walker-start, I tapped p to go through the previous backups. The diff it showed me made it easy to check with C-s (isearch-forward) if this was the version I was looking for. When I found the change, I pressed RET to load the version with the function in it. Once I found it, it was easy to restore that section. I also restored a couple of other sections that I'd accidentally deleted too, like the custom plain text publishing backend I use to export Emacs News with less punctuation. It took maybe 5 minutes to figure this out. Hooray for backup-walker!

Note that the backup-walker diff was the other way around from what I expected. It goes "diff new old" instead of "diff old new", so the green regions marked with + indicate stuff that was removed by the newer version (compared to the one a little older than it) and the red regions marked with - indicate stuff that was added. This could be useful if you think backwards in time, kind of like the Emacs Antinews file, but my mind doesn't quite work that way. I wanted it to look like a regular diff, with the additions in newer versions marked with +. Emacs being Emacs, I changed it. Here's an example showing what it looks like now:

2025-09-17_13-46-12.png
Figure 1: backup-walker diffs going the direction I want them to: additions (+) marked in green, deletions (-) in red

The following code makes it behave the way I expect:

(defun my-backup-walker-refresh ()
  (let* ((index (cdr (assq :index backup-walker-data-alist)))
         (suffixes (cdr (assq :backup-suffix-list backup-walker-data-alist)))
         (prefix (cdr (assq :backup-prefix backup-walker-data-alist)))
         (right-file (concat prefix (nth index suffixes)))
         (right-version (format "%i" (backup-walker-get-version right-file)))
         diff-buff left-file left-version)
    (if (eq index 0)
        (setq left-file (cdr (assq :original-file backup-walker-data-alist))
              left-version "orig")
      (setq left-file (concat prefix (nth (1- index) suffixes))
            left-version (format "%i" (backup-walker-get-version left-file))))
    ;; we change this to go the other way here
    (setq diff-buf (diff-no-select right-file left-file nil 'noasync))
    (setq buffer-read-only nil)
    (delete-region (point-min) (point-max))
    (insert-buffer diff-buf)
    (set-buffer-modified-p nil)
    (setq buffer-read-only t)
    (force-mode-line-update)
    (setq header-line-format
          (concat (format "{{ ~%s~ → ~%s~ }} "
                          (propertize left-version 'face 'font-lock-variable-name-face)
                          (propertize right-version 'face 'font-lock-variable-name-face))
                  (if (nth (1+ index) suffixes)
                      (concat (propertize "<p>" 'face 'italic)
                              " ~"
                              (propertize (int-to-string
                                           (backup-walker-get-version (nth (1+ index) suffixes)))
                                          'face 'font-lock-keyword-face)
                              "~ ")
                    "")
                  (if (eq index 0)
                      ""
                    (concat (propertize "<n>" 'face 'italic)
                            " ~"
                            (propertize (int-to-string (backup-walker-get-version (nth (1- index) suffixes)))
                                        'face 'font-lock-keyword-face)
                            "~ "))
                  (propertize "<return>" 'face 'italic)
                  " open ~"
                  (propertize (propertize (int-to-string (backup-walker-get-version right-file))
                                          'face 'font-lock-keyword-face))
                  "~"))
    (kill-buffer diff-buf)))
(with-eval-after-load 'backup-walker
  (advice-add 'backup-walker-refresh :override #'my-backup-walker-refresh))

backup-walker is not actually a real package in the sense of M-x package-install, but fortunately, recent Emacs makes it easier to install from a repository. I needed to install it from https://github.com/lewang/backup-walker. It was written so long ago that I needed to defalias some functions that were removed in Emacs 26.1. Here's the use-package snippet from my configuration:

(use-package backup-walker
  :vc (:url "https://github.com/lewang/backup-walker")
  :commands backup-walker-start
  :init
  (defalias 'string-to-int 'string-to-number)  ; removed in 26.1
  (defalias 'display-buffer-other-window 'display-buffer))

So there's an obscure package recommendation: backup-walker. It hasn't been updated for more than a decade, and it's not even installable the regular way, but it's still handy.

I can imagine all sorts of ways this workflow could be even better. It might be nice to dust off backup-walker off, switch out the obsolete functions, add an option for the diff direction, and maybe sort things out so that you can reverse the diff, split hunks, and apply hunks to your original file. And maybe a way to walk the backup history for changes in a specific region? I suppose someone could make a spiffy Transient-based user interface to modernize it. But it's fine, it works. Maybe there's a more modern equivalent, but I didn't see anything in a quick search of M-x list-packages / N (package-menu-filter-by-name-or-description) for "backup~, except maybe vc-backup.1 Is there a general-purpose VC equivalent to git-timemachine? That might be useful.

I should really be saving things in proper version control, but this was a good backup. That reminds me: I should backup my backup backups. I had initially excluded my ~/.config directory from borgbackup because of the extra bits and bobs that I wouldn't need when restoring from backup (like all the Emacs packages I'd just re-download). But my file backups… Yeah, that's worth it. I changed my --exclude-from to --patterns-from and changing my borg-patterns file to look like this:

+ /home/sacha/.config/emacs/backups
- /home/sacha/.config/*
# ... other rules

May backup-walker save you from a future oops!

Footnotes

1

vc-backup: The original repo is missing, but you can read it via ELPA's copy. Update: It's over on Codeberg now, and presumably the info on ELPA will be updated soon.

This is part of my Emacs configuration.
View org source for this post

Getting a Google Docs draft ready for Mailchimp via Emacs and Org Mode

Posted: - Modified: | emacs, org

: Got it to include the dates in the TOC as well

I've been volunteering to help with the Bike Brigade newsletter. I like that there are people who are out there helping improve food security by delivering food bank hampers to recipients. Collecting information for the newsletter also helps me feel more appreciation for the lively Toronto biking scene, even though I still can't make it out to most events. The general workflow is:

  1. collect info
  2. draft the newsletter somewhere other volunteers can give feedback on
  3. convert the newsletter to Mailchimp
  4. send a test message
  5. make any edits requested
  6. schedule the email campaign

We have the Mailchimp Essentials plan, so I can't just export HTML for the whole newsletter. Someday I should experiment with services that might let me generate the whole newsletter from Emacs. That would be neat. Anyway, with Mailchimp's block-based editor, at least I can paste in HTML code for the text/buttons. That way, I don't have to change colours or define links by hand.

The logistics volunteers coordinate via Slack, so a Slack Canvas seemed like a good way to draft the newsletter. I've previously written about my workflow for copying blocks from a Slack Canvas and then using Emacs to transform the rich text, including recolouring the links in the section with light text on a dark background. However, copying rich text from a Slack Canvas turned out to be unreliable. Sometimes it would copy what I wanted, and sometimes nothing would get copied. There was no way to export HTML from the Slack Canvas, either.

I switched to using Google Docs for the drafts. It was a little less convenient to add items from Slack messages and I couldn't easily right-click to download the images that I pasted in. It was more reliable in terms of copying, but only if I used xclip to save the clipboard into a file instead of trying to do the whole thing in memory.

I finally got to spend a little time automating a new workflow. This time I exported the Google Doc as a zip that had the HTML file and all the images in a subdirectory. The HTML source is not very pleasant to work with. It has lots of extra markup I don't need. Here's what an entry looks like:

2025-09-17_09-22-35.png
Figure 1: Exported HTML for an entry

Things I wanted to do with the HTML:

  • Remove the google.com/url redirection for the links. Mailchimp will add its own redirection for click-tracking, but at least the links can look simpler when I paste them in.
  • Remove all the extra classes and styles.
  • Turn [ call to action ] into fancier Mailchimp buttons.

Also, instead of transforming one block at a time, I decided to make an Org Mode document with all the different blocks I needed. That way, I could copy and paste things in quick succession.

Here's what the result looks like. It makes a table of contents, adds the sign-up block, and adds the different links and blocks I need to paste into Mailchimp.

2025-09-17_10-03-27.png
Figure 2: Screenshot of newsletter Org file with blocks for easy copying

I need to copy and paste the image filenames into the upload dialog on Mailchimp, so I use my custom Org Mode link type for copying to the clipboard. For the HTML code, I use #+begin_src html ... #+end_src instead of #+begin_export html ... #+end_export so that I can use Embark and embark-org to quickly copy the contents of the source block. (That doesn't work for export blocks yet.) I have C-. bound to embark-act, the source block is detected by the functions that embark-org.el added to embark-target-finders, and the c binding in embark-org-src-block-map calls embark-org-copy-block-contents. So all I need to do is C-. c in a block to copy its contents.

Here's the code to process the newsletter draft
(defun my-brigade-process-latest-newsletter-draft (date)
  "Create an Org file with the HTML for different blocks."
  (interactive (list (if current-prefix-arg (org-read-date nil t nil "Date: ")
                       (org-read-date nil t "+Sun"))))
  (when (stringp date) (setq date (date-to-time date)))
  (let ((default-directory "~/Downloads/newsletter")
        file
        dom
        sections)
    (call-process "unzip" nil nil nil "-o" (my-latest-file "~/Downloads" "\\.zip$"))
    (setq file (my-latest-file default-directory))
    (with-temp-buffer
      (insert-file-contents-literally file)
      (goto-char (point-min))
      (setq dom (my-brigade-simplify-html (libxml-parse-html-region (point-min) (point-max))))
      (my-brigade-save-newsletter-images dom)
      (setq sections
            (my-html-group-by-tag
             'h1
             (dom-children
              (dom-by-tag
               dom 'body)))))
    (with-current-buffer (get-buffer-create "*newsletter*")
      (erase-buffer)
      (org-mode)
      (insert
       (format-time-string "%B %-e, %Y" date) "\n"
       "* In this e-mail\n#+begin_src html\n"
       "<p>Hi Bike Brigaders! Here’s what's happening this week, with quick signup links. In this e-mail:</p>"
       (replace-regexp-in-string
        "<li>" "\n<li>"
        (with-temp-buffer
          (svg-print
           (apply 'dom-node
                  'ul nil
                  (append
                   (my-brigade-toc-items (assoc-default "Bike Brigade" sections 'string=))
                   (my-brigade-toc-items (assoc-default "In our community" sections 'string=)))))
          (buffer-string)))
       "\n<br />\n"
       (my-brigade-copy-signup-block date)
       "\n#+end_src\n\n")
      (dolist (sec '("Bike Brigade" "In our community"))
        (insert "* " sec "\n"
                (mapconcat
                 (lambda (group)
                   (let* ((item (apply 'dom-node 'div nil
                                       (append
                                        (list (dom-node 'h2 nil (car group)))
                                        (cdr group))))
                          (image (my-brigade-image (car group))))
                     (format "** %s\n\n%s\n%s\n\n#+begin_src html\n%s\n#+end_src\n\n"
                             (car group)
                             (if image (org-link-make-string (concat "copy:" image)) "")
                             (or (my-html-last-link-href item) "")
                             (my-transform-html
                              (delq nil
                                    (list
                                     'my-transform-html-remove-images
                                     'my-transform-html-remove-italics
                                     'my-brigade-simplify-html
                                     'my-brigade-format-buttons
                                     (when (string= sec "In our community")
                                       'my-brigade-recolor-recursively)))
                              item))))
                 (my-html-group-by-tag 'h2 (cdr (assoc sec sections 'string=)))
                 "")))
      (insert "* Other updates\n"
              (format "#+begin_src html\n<h2>Other updates</h2>%s\n#+end_src\n\n"
                      (my-transform-html
                       '(my-transform-html-remove-images
                         my-transform-html-remove-italics
                         my-brigade-simplify-html)
                       (car (cdr (assoc "Other updates" sections 'string=))))))
      (goto-char (point-min))
      (display-buffer (current-buffer)))))

(defun my-brigade-toc-items (section-children)
  "Return a list of <li /> nodes."
  (mapcar
   (lambda (group)
     (let* ((text (dom-texts (cadr group)))
            (regexp (format "^%s \\([A-Za-z]+ [0-9]+\\)"
                            (regexp-opt '("Mon" "Tue" "Wed" "Thu" "Fri" "Sat" "Sun"))))
            (match (when (string-match regexp text) (match-string 1 text))))
       (dom-node 'li nil
                 (if match
                     (format "%s: %s" match (car group))
                   (car group)))))
   (my-html-group-by-tag 'h2 section-children)))

(defun my-html-group-by-tag (tag dom-list)
  "Use TAG to divide DOM-LIST into sections. Return an alist of (section . children)."
  (let (section-name current-section results)
    (dolist (node dom-list)
      (if (and (eq (dom-tag node) tag)
               (not (string= (string-trim (dom-texts node)) "")))
          (progn
            (when current-section
              (push (cons section-name (nreverse current-section))  results)
              (setq current-section nil))
            (setq section-name (string-trim (dom-texts node))))
        (when section-name
          (push node current-section))))
    (when current-section
      (push (cons section-name (reverse current-section))  results)
      (setq current-section nil))
    (nreverse results)))

(defun my-html-last-link-href (node)
  "Return the last link HREF in NODE."
  (dom-attr (car (last (dom-by-tag node 'a))) 'href))

(defun my-brigade-image (heading)
  "Find the latest image related to HEADING."
  (car
   (nreverse
    (directory-files my-brigade-newsletter-images-directory
                        t (regexp-quote (my-brigade-newsletter-heading-to-image-file-name heading))))))

Some of the functions it uses are in my config, particularly the section on Transforming HTML clipboard contents with Emacs to smooth out Mailchimp annoyances: dates, images, comments, colours.

Along the way, I learned that svg-print is a good way to turn document object models back into HTML.

When I saw two more events and one additional link that I wanted to include, I was glad I already had this code sorted out. It made it easy to paste the images and details into the Google Doc, reformat it slightly, and get the info through the process so that it ended up in the newsletter with a usefully-named image and correctly-coloured links.

I think this is a good combination of Google Docs for getting other people's feedback and letting them edit, and Org Mode for keeping myself sane as I turn it into whatever Mailchimp wants.

My next step for improving this workflow might be to check out other e-mail providers in case I can get Emacs to make the whole template. That way, I don't have to keep switching between applications and using the mouse to duplicate blocks and edit the code.

This is part of my Emacs configuration.
View org source for this post

2025-09-15 Emacs news

| emacs, emacs-news

There were lots of Emacs-related discussions on Hacker News, mainly because of two posts about Emacs's extensibility: an Org Mode example and a completion example. This guide on connecting to databases from Org Mode Babel blocks started a few discussions. There were a couple of Emacs Carnival posts on obscure packages, too. Also, if you want to present at EmacsConf 2025 (great way to meet like-minded folks), please send your proposal in by this Friday (Sept 19). Thanks!

Links from reddit.com/r/emacs, r/orgmode, r/spacemacs, r/planetemacs, Mastodon #emacs, Bluesky #emacs, Hacker News, lobste.rs, programming.dev, lemmy.world, lemmy.ml, planet.emacslife.com, YouTube, the Emacs NEWS file, Emacs Calendar, and emacs-devel. Thanks to Andrés Ramírez for emacs-devel links. Do you have an Emacs-related link or announcement? Please e-mail me at sacha@sachachua.com. Thank you!

View org source for this post

Emacs and dom.el: quick notes on parsing HTML and turning DOMs back into HTML

| elisp

libxml-parse-html-region turns HTML into a DOM (document object model). There's also xml-parse-file and xml-parse-region. xml-parse-string actually parses the character data at point and returns it as a string instead of parsing a string as a parameter. If you have a string and you want to parse it, insert it into a temporary buffer and use libxml-parse-html-region or xml-parse-region.

(let ((s "<span>Hello world</span>")
      dom)
  (setq dom
        (with-temp-buffer
          (insert s)
          (libxml-parse-html-region))))
(html nil (body nil (span nil Hello world)))

Then you can use functions like dom-by-tag, dom-search, dom-attr, dom-children, etc. If you need to make a deep copy of the DOM, you can use copy-tree.

Turning the DOM back into HTML can be a little tricky. By default, dom-print escapes & in attributes, which could mess up things like href:

  (with-temp-buffer
    (dom-print (dom-node 'a '((href . "https://example.com?a=b&c=d"))))
     (buffer-string))
  <a href="https://example.com?a=b&amp;c=d" />

shr-dom-print handles & correctly, but it adds spaces in between elements. Also, you need to escape HTML entities in text, maybe with org-html-encode-plain-text.

  (with-temp-buffer
    (shr-dom-print
      (dom-node 'p nil
                (dom-node 'span nil "hello")
                (dom-node 'span nil "world")
                (dom-node 'a '((href . "https://example.com?a=b&c=d"))
                          (org-html-encode-plain-text "text & stuff"))))
    (buffer-string))
  <p> <span>hello</span> <span>world</span> <a href="https://example.com?a=b&c=d">text &amp; stuff</a></p>

svg-print does the right thing when it comes to href and tags, but you need to escape HTML entities yourself as usual.

(with-temp-buffer
  (svg-print
   (dom-node 'p nil
             (dom-node 'span nil "hello")
             (dom-node 'span nil "world")
             (dom-node 'a '((href . "https://example.com?a=b&c=d"))
                       (org-html-encode-plain-text "text & stuff"))))
  (buffer-string))
  <p><span>hello</span><span>world</span><a href="https://example.com?a=b&c=d">text &amp; stuff</a></p>

Looks like I'll be using svg-print for more than just SVGs.

Relevant Emacs info pages:

View org source for this post

2025-09-08 Emacs news

| emacs, emacs-news

There was lots of conversation around Why Rewriting Emacs Is Hard this week, with threads on Reddit, HN, and lobste.rs. Also, if you want to present at EmacsConf 2025 (great way to meet like-minded folks), please send your proposal in by next Friday (Sept 19). Thanks!

Links from reddit.com/r/emacs, r/orgmode, r/spacemacs, r/planetemacs, Mastodon #emacs, Bluesky #emacs, Hacker News, lobste.rs, programming.dev, lemmy.world, lemmy.ml, planet.emacslife.com, YouTube, the Emacs NEWS file, Emacs Calendar, and emacs-devel. Thanks to Andrés Ramírez for emacs-devel links. Do you have an Emacs-related link or announcement? Please e-mail me at sacha@sachachua.com. Thank you!

View org source for this post

2025-09-01 Emacs news

| emacs, emacs-news

People continue to share interesting Emacs elevator pitches. The Emacs Carnival September theme is "Obscure packages." Recommend a hidden gem!

Links from reddit.com/r/emacs, r/orgmode, r/spacemacs, r/planetemacs, Mastodon #emacs, Bluesky #emacs, Hacker News, lobste.rs, programming.dev, lemmy.world, lemmy.ml, planet.emacslife.com, YouTube, the Emacs NEWS file, Emacs Calendar, and emacs-devel. Thanks to Andrés Ramírez for emacs-devel links. Do you have an Emacs-related link or announcement? Please e-mail me at sacha@sachachua.com. Thank you!

View org source for this post

Emacs elevator pitch: tinkerers unite

| emacs, community

This is for the Emacs Carnival 2025-08: Your Elevator Pitch for Emacs hosted by Jeremy Friesen. Emacs is a text editor, but people have made it so much more.

Text and links from sketch

Emacs elevator pitch https://sach.ac/2025-08-31-01

That's the theme for the August Emacs Carnival.

I don't spend much time in elevators these days, and I didn't talk much to strangers even during the before-times.

So let's imagine this is more of, say, a meetup. (Someday I'll get back to going to those.) Could be tech, could be something else. Could be online, could be in person.

I don't have to convince everyone. I don't even have to convince a single person. My goal is to listen for the tinkerers: the ones who like to ask "Why?" and "What if?" and who try things out. They're interesting.

I can find them by:

  • watching talks (sketchnotes are a great thank-you gift)
  • eavesdropping or asking questions
  • sharing what I'm tinkering with

No: Why would anyone do that? Yes: Have you thought of trying xyz?

For tinkerers, the juice might be worth the squeeze. Emacs can be challenging, but it can also pay off. It can even be fun.

Even when I find a fellow tinkerer, the conversation isn't "Have you tried Emacs? You should try Emacs." It'll probably be more like:

"I'd love to keep hearing about your experiments. Do you have something I can subscribe to or follow?" (Side-quest: Try to convince them to blog.)

and then the conversation can unfold over time:

  • "Ooh, I like that idea. Here's my take on it."
  • "How did you do that? What's that?!"
  • "Oh, yeah, Emacs. It's very programmable, so I can get it to do all sorts of stuff for me. Wanna see?"

…and sometimes they fall into the rabbit hole themselves, as tinkerers often do. But even if they don't try Emacs (or don't stick with it), cross-pollination is great. And sometimes Emacs changes their life.

To get a sense of the kinds of things someone has gotten Emacs to do, check out Alvaro Ramirez's post. I have a list like that at emacs. On the topic of cross-pollination, I like Jeremy Friesen's EmacsConf 2023 talk on Mentoring VS-Coders as an Emacsian (or How to show not tell people about the wonders of Emacs).

It's always fun to come across a fellow tinkerer. I love seeing what people come up with. Emacs works out really well for tinkerers. It's not just about taking advantage of the technical capabilities (and you can do a surprising amount with text, images, and interaction), it's also about being part of a great community that's in it long-term. Good stuff.

Feel free to use this sketch under the Creative Commons Attribution License.

View org source for this post