Categories: geek » emacs

RSS - Atom - Subscribe via email

Defining generic and mode-specific Emacs Lisp functions with cl-defmethod

| emacs, elisp

2022-01-27: Added example function description.
2022-01-02: Changed quote to function in the defalias.

I recently took over the maintenance of subed, an Emacs mode for editing subtitles. One of the things on my TODO list was to figure out how to handle generic and format-specific functions instead of relying on defalias. For example, there are SubRip files (.srt), WebVTT files (.vtt), and Advanced SubStation Alpha (.ass). I also want to add support for Audacity labels and other formats.

There are some functions that will work across all of them once you have the appropriate format-specific functions in place, and there are some functions that have to be very different depending on the format that you're working with. Now, how do you do those things in Emacs Lisp? There are several ways of making general functions and specific functions.

For example, the forward-paragraph and backward-paragraph commands use variables to figure out the paragraph separators, so buffer-local variables can change the behaviour.

However, I needed a bit more than regular expressions. An approach taken in some packages like smartparens is to have buffer-local variables have the actual functions to be called, like sp-forward-bound-fn and sp-backward-bound-fn.

(defvar-local sp-forward-bound-fn nil
  "Function to restrict the forward search")

(defun sp--get-forward-bound ()
  "Get the bound to limit the forward search for looking for pairs.
If it returns nil, the original bound passed to the search
function will be considered."
  (and sp-forward-bound-fn (funcall sp-forward-bound-fn)))

Since there were so many functions, I figured that might be a little bit unwieldy. In Org mode, custom export backends are structs that have an alist that maps the different types of things to the functions that will be called, overriding the functions that are defined in the parent export backend.

(cl-defstruct (org-export-backend (:constructor org-export-create-backend)
          (:copier nil))
  name parent transcoders options filters blocks menu)

(defun org-export-get-all-transcoders (backend)
  "Return full translation table for BACKEND.

BACKEND is an export back-end, as return by, e.g,,
`org-export-create-backend'.  Return value is an alist where
keys are element or object types, as symbols, and values are
transcoders.

Unlike to `org-export-backend-transcoders', this function
also returns transcoders inherited from parent back-ends,
if any."
  (when (symbolp backend) (setq backend (org-export-get-backend backend)))
  (when backend
    (let ((transcoders (org-export-backend-transcoders backend))
          parent)
      (while (setq parent (org-export-backend-parent backend))
        (setq backend (org-export-get-backend parent))
        (setq transcoders
              (append transcoders (org-export-backend-transcoders backend))))
      transcoders)))

The export code looked a little bit complicated, though. I wanted to see if there was a different way of doing things, and I came across cl-defmethod. Actually, the first time I tried to implement this, I was focused on the fact that cl-defmethod could call different things depending on the class that you give it. So initially I had created a couple of classes: subed-backend class, and then subclasses such as subed-vtt-backend. This allowed me to store the backend as a buffer-local variable and differentiate based on that.

(require 'eieio)

(defclass subed-backend ()
  ((regexp-timestamp :initarg :regexp-timestamp
                     :initform ""
                     :type string
                     :custom string
                     :documentation "Regexp matching a timestamp.")
   (regexp-separator :initarg :regexp-separator
                     :initform ""
                     :type string
                     :custom string
                     :documentation "Regexp matching the separator between subtitles."))
  "A class for data and functions specific to a subtitle format.")

(defclass subed-vtt-backend (subed-backend) nil
  "A class for WebVTT subtitle files.")

(cl-defmethod subed--timestamp-to-msecs ((backend subed-vtt-backend) time-string)
  "Find HH:MM:SS,MS pattern in TIME-STRING and convert it to milliseconds.
Return nil if TIME-STRING doesn't match the pattern.
Use the format-specific function for BACKEND."
  (save-match-data
    (when (string-match (oref backend regexp-timestamp) time-string)
      (let ((hours (string-to-number (match-string 1 time-string)))
            (mins  (string-to-number (match-string 2 time-string)))
            (secs  (string-to-number (match-string 3 time-string)))
            (msecs (string-to-number (subed--right-pad (match-string 4 time-string) 3 ?0))))
        (+ (* (truncate hours) 3600000)
           (* (truncate mins) 60000)
           (* (truncate secs) 1000)
           (truncate msecs))))))

Then I found out that you can use major-mode as a context specifier for cl-defmethod, so you can call different specific functions depending on the major mode that your buffer is in. It doesn't seem to be mentioned in the elisp manual, so at some point I should figure out how to suggest mentioning it. Anyway, now I have some functions that get called if the buffer is in subed-vtt-mode and some functions that get called if the buffer is in subed-srt-mode.

The catch is that cl-defmethod can't define interactive functions. So if I'm defining a command, an interactive function that can be called with M-x, then I will need to have a regular function that calls the function defined with cl-defmethod. This resulted in a bit of duplicated code, so I have a macro that defines the method and then defines the possibly interactive command that calls that method. I didn't want to think about whether something was interactive or not, so my macro just always creates those two functions. One is a cl-defmethod that I can override for a specific major mode, and one is the function that actually calls it, which may may not be interactive. It doesn't handle &rest args, but I don't have any in subed.el at this time.

(defmacro subed-define-generic-function (name args &rest body)
  "Declare an object method and provide the old way of calling it."
  (declare (indent 2))
  (let (is-interactive
        doc)
    (when (stringp (car body))
      (setq doc (pop body)))
    (setq is-interactive (eq (caar body) 'interactive))
    `(progn
       (cl-defgeneric ,(intern (concat "subed--" (symbol-name name)))
           ,args
         ,doc
         ,@(if is-interactive
               (cdr body)
             body))
       ,(if is-interactive
            `(defun ,(intern (concat "subed-" (symbol-name name))) ,args
               ,(concat doc "\n\nThis function calls the generic function `"
                        (concat "subed--" (symbol-name name)) "' for the actual implementation.")
               ,(car body)
               (,(intern (concat "subed--" (symbol-name name)))
                ,@(delq nil (mapcar (lambda (a)
                                      (unless (string-match "^&" (symbol-name a))
                                        a))
                                    args))))
          `(defalias (quote ,(intern (concat "subed-" (symbol-name name))))
             (function ,(intern (concat "subed--" (symbol-name name))))
             ,doc)))))

For example, the function:

(subed-define-generic-function timestamp-to-msecs (time-string)
  "Find timestamp pattern in TIME-STRING and convert it to milliseconds.
Return nil if TIME-STRING doesn't match the pattern.")

expands to:

(progn
  (cl-defgeneric subed--timestamp-to-msecs
      (time-string)
    "Find timestamp pattern in TIME-STRING and convert it to milliseconds.
Return nil if TIME-STRING doesn't match the pattern.")
  (defalias 'subed-timestamp-to-msecs 'subed--timestamp-to-msecs "Find timestamp pattern in TIME-STRING and convert it to milliseconds.
Return nil if TIME-STRING doesn't match the pattern."))

and the interactive command defined with:

(subed-define-generic-function forward-subtitle-end ()
  "Move point to end of next subtitle.
Return point or nil if there is no next subtitle."
  (interactive)
  (when (subed-forward-subtitle-id)
    (subed-jump-to-subtitle-end)))

expands to:

(progn
  (cl-defgeneric subed--forward-subtitle-end nil "Move point to end of next subtitle.
Return point or nil if there is no next subtitle."
                 (when
                     (subed-forward-subtitle-id)
                   (subed-jump-to-subtitle-end)))
  (defun subed-forward-subtitle-end nil "Move point to end of next subtitle.
Return point or nil if there is no next subtitle.

This function calls the generic function `subed--forward-subtitle-end' for the actual implementation."
         (interactive)
         (subed--forward-subtitle-end)))

Then I can define a specific one with:

(cl-defmethod subed--timestamp-to-msecs (time-string &context (major-mode subed-srt-mode))
  "Find HH:MM:SS,MS pattern in TIME-STRING and convert it to milliseconds.
Return nil if TIME-STRING doesn't match the pattern.
Use the format-specific function for MAJOR-MODE."
  (save-match-data
    (when (string-match subed--regexp-timestamp time-string)
      (let ((hours (string-to-number (match-string 1 time-string)))
            (mins  (string-to-number (match-string 2 time-string)))
            (secs  (string-to-number (match-string 3 time-string)))
            (msecs (string-to-number (subed--right-pad (match-string 4 time-string) 3 ?0))))
        (+ (* (truncate hours) 3600000)
           (* (truncate mins) 60000)
           (* (truncate secs) 1000)
           (truncate msecs))))))

The upside is that it's easy to either override or extend a function's behavior. For example, after I sort subtitles, I want to renumber them if I'm in an SRT buffer because SRT subtitles have numeric IDs. This doesn't happen in any of the other modes. So I can just define that this bit of code runs after the regular code that runs.

(cl-defmethod subed--sort :after (&context (major-mode subed-srt-mode))
  "Renumber after sorting. Format-specific for MAJOR-MODE."
  (subed-srt--regenerate-ids))

The downside is that going to the function's definition and stepping through it is a little more complicated because it's hidden behind this macro and the cl-defmethod infrastructure. I think that if you describe-function the right function, the internal version with the --, then it will list the different implementations of it. I added a note to the regular function's docstring to make it a little easier.

Here's what M-x describe-function subed-forward-subtitle-end looks like:

describe-function.svg

Figure 1: Describing a generic function

I'm going to give this derived-mode branch a try for a little while by subtitling some more EmacsConf talks before I merge it into the main branch. This is my first time working with cl-defmethod, and it looks pretty interesting.

View or add comments

2021-12-27 Emacs news

| emacs, emacs-news

Links from reddit.com/r/emacs, r/orgmode, r/spacemacs, r/planetemacs, Hacker News, planet.emacslife.com, YouTube, the Emacs NEWS file, Emacs Calendar, emacs-devel, and lemmy/c/emacs.

View or add comments

2021-12-20 Emacs news

| emacs, emacs-news

Links from reddit.com/r/emacs, r/orgmode, r/spacemacs, r/planetemacs, Hacker News, planet.emacslife.com, YouTube, the Emacs NEWS file, Emacs Calendar, emacs-devel, and lemmy/c/emacs.

View or add comments

Why I Love Emacs - from Bob Oliver

| emacs, org

Sometimes I post updates from people who don't have their own blog. Here's one from Bob Oliver. - Sacha

This short article sets out why I, as an Emacs newbie, really, really love this software. But before I get into that I would like to explain my voyage (Note: absence of the 'journey' word) to Emacs.

Many moons ago, back in the late seventies / early eighties I was a Cobol programmer, a job I loved. As it is with life, circumstances change and I moved away from Data Processing, as we called it in olden days. This meant I had to get my programming fix using my Sinclair Spectrum, which I programmed using their version of BASIC. I learned how to build my own, very simple games, and spent many hours playing my games and programming more. Then the children came along, the Sinclair went into the loft (attic for non-UK readers) and I had little or no time for hobbies.

Years later, with family grown and flown the nest, the Raspberry Pi was released and revised my love of programming. I took to learning C and Python - though remain very much at the beginner stage. All very enjoyable. This sparked a notion that I might be able to build an app and enhance my future pension prospects. To this end I installed xCode on my MacBook and also tried VS-Code. Needless to say I have not achieved proficiency and have since removed those products from my MacBook.

I still wanted to enhance my knowledge of C, Python and Bash, and so was really pleased when the Raspberry Pi foundation released Raspberry O/S Desktop for Mac (apologies if this name is not technically correct). This enabled me to re-purpose an old MacBook (circa 2009 and no longer supported) as a Linux machine, which got me interesting in learning all things Linux. This led to me installing Emacs as my code editor. Through reading all things Emacs I discovered org-mode and now Emacs is my text editor of choice.

As probably most new users to Emacs, I found it a bit confusing at first, but did as recommended stuck with it, and I am really glad I did.

What do I use Emacs for?

A very good question. Short answer is code and text editor.

  1. Writing, compiling, testing and running C programs.
  2. Writing, testing and running Bash scripts.
  3. Writing, testing and running Python programs.
  4. Compiling my, not so, daily journal.
  5. Using org-mode as my word processor of choice.

The key reason for using org-mode for my journal, was portability and long term accessibility. I had used various electronic journals before, each with their own proprietary file standards, making me concerned that my journal would not be available to my children long after I have gone. Also as Linux, and hence org-mode, use plain text files I can edit with any text editor on any platform, so can be assured that I can move the files as and when I change computers. Also as plain text files, they are readily searchable, so I can recall memories easily.

Finding Emacs and org-mode is probably one of the best things I have done since I retired from full-time employment.

What next:

  1. Maintain my journal writing.
  2. Write up my poems in org-mode - I have several going back to my teenage years.
  3. Develop my writing skills and maybe write a novel.
  4. Learn how to send and recieve mail through Emacs - I have yet to find a guide that is not too technical / complicated for me.

SO MY MESSAGE IS JOIN THE EMACS AND ORG SOCIETY - YOU WON'T REGRET IT.

Bob Oliver Essex, England.

View or add comments

Thinking about Emacs community maintenance

| emacs

Building on Bastien Guerry's talk on maintainers at the recent EmacsConf, I thought I'd reflect a little bit on what I do to help out with the Emacs community because, as Bastien pointed out in his Q&A, that's also kind of maintenance. If I think about what I like to do and why, and what I don't like to do and why not, I'm more likely to keep enjoying things and not burn out. If I think about what I do, I might be able to find ways to do them more effectively or other things that complement them efficiently. And hey, maybe those notes will prompt other people to think about what they do, what they like and dislike, and how to grow.

I like helping with Emacs because it tickles my brain. I like playing around with Emacs and getting it to do interesting things, and I get a lot of reward for a little effort. I do this because it's intrinsically fun. Hacking on cool things with Emacs is interesting because I can come up with different things that a text editor really shouldn't be able to do, and yet it does. I also like coming across blog posts by other people who have written about their ideas, or maybe they're building on something that I've shared, or maybe they've come up with ideas of their own. It's great when other people share, as I get to learn from them too. And of course, you know, of course, comments and emails and all that stuff are also very nice.

There are a few things that aren't a particularly good fit for me or my life at the moment, so I try to do as few of them as possible.

  • Time-sensitive commitments: meetings, coordinating with other people, hosting video chats, scheduled livestreaming… It would be nice to help out, but I also need to hang out with the kiddo. I don't like the feeling of being pulled two different ways, so that's something I've been minimizing.
  • Since I don't have much computer time, I'm still a little too impatient to listen to or watch videos, and I like my quiet time too much to listen to them when I'm washing the dishes or tidying up. I don't have a lot of discretionary time, so I'd prefer to speed-read through captions.
  • Talking is a little tough. After a day with a little kid, sometimes I don't particularly want to talk to anyone. I'm experimenting with recording myself thinking out loud so that I can get ideas down quickly, but I'm not quite at the point of doing livestreams yet.
  • Keeping things in my head is hard, too, like if I'm working on a big task that's hard to break down into small chunks, or something that requires a lot of these small chunks built up over a period of of days. My discretionary time is really one- or two-hour chunks (or less) and pretty fragmented fragmented–one day here, another day there–because I've got other things that I need to do in the meantime. So I tend to focus on short things instead.
  • I'm also not really keen on things that build up more technical debt. There's a whole list of things that I will get to do some day when I can actually sit down and focus on things. In the meantime, I just want to get through this part. =)
  • I'm kinda iffy on e-mail. I tend to respond slowly, and I usually redirect people to other places like #emacs or Reddit if they're asking for help. Don't have much time at night, and I don't want to spend it going through my inbox.
  • Emacs geeking becomes less fun if things aren't smooth at home: grumpy kiddo, clutter building up, too many oops moments.

Anyway, so the core of what I do:

  • Emacs News: I put that together every week. I browse Reddit posts as a mental break from hanging out with the kiddo. I pick up other sources, too. People email me interesting things to include. I've got a few shortcuts for filing things. Putting it all together takes about 30-60 minutes, especially if I get carried away incorporating cool stuff into my config. I do it because it's a lot of fun to see all the cool things that people are working on, and it takes advantage of speed-reading. Plus I can do most of the information-gathering in the scattered moments of my day, even if I don't have a chunk of focused time to come up with and write about an interesting Emacs hack. And it seems to be a great way to link the community up, because there's just so much stuff that's going on.
  • Code: I share my config and I add to it once in a while. Lately I've been breaking things out into smaller modules that perhaps someday could turn into packages. For the most part, I stick things into my config. Some of those functions turn into blog posts as well.
  • Blog posts: They're a great way to remember, share, and get ideas and comments from other people. It's particularly fun when other people build on those ideas and then I get to use their even awesomer version.
  • EmacsConf: This feels like it takes a surprising amount of time. I like working behind the scenes. I like doing the schedule because it tickles my brain when I can put the talks in some kind of flow. I like doing captions because I can work with text. I do a lot of the publishing on the wiki, and I've written a lot of automation for it. It's a good part of my evil plan, because it gets stuff out of people's heads and configurations and into a form that other people can look at and maybe talk to them about it. It's handy for connecting people.
  • Calendar: This is basically an iCal aggregator that gets updates from various meetup groups and includes some manually-created entries. It goes into a bunch of places, like Emacs News, #emacs, EmacsWiki:Usergroups, and Twitter. I think that's a nice little way to help people find virtual events, and then people don't feel so isolated. I find that a lot of people are the only Emacs users they know in real life, so it's fantastic that there are a lot of virtual meetups these days. Then they can find other people who can share ideas. I think that one of the best ways to learn Emacs is by looking over someone's shoulder, because then you can you can see things that the other person might forget to point out to you, and you can ask questions about those. Meetups are great for that sort of thing, and that's why I do the calendar as well.

I don't have as much time for things like sketches or videos at the moment, but I like to do them once in a while. Sketches make it easy to map things out, untangle my thoughts, or plan ideas. Some of my beginner maps are still floating around. Videos are handy for showing interesting things. Sometimes people ask questions about things that aren't actually the focus of the video, which is great. Hanging out in #emacs is nice because I learn from other people's questions and answers.

It would be fantastic if someone could do emacs-devel summaries which I could just link to in Emacs News. The traffic is a bit too heavy on that list for me to get a sense of how to summarize it, and I have a hard time keeping track of things. I'd also love it if someone else did video encoding and other detail-oriented work, so if someone wants to take that on for EmacsConf, feel free. Someday it would be nice to have enough free time to go to virtual meetups or do Emacs Chats again, and it would be nice to see more people organizing those as well. And more blog posts, too! I'd also love to have more playtime to experiment with interesting packages and features, too. Anyway, someday!

In the meantime, I've been experimenting with tweaking my workflow. Lately I've been actually plugging into an external monitor instead of trying to code on my laptop. I've also got to work on the sleep/discretionary time trade-off. If I stay up too much, the next day gets a bit rough, and rough is hard with a kiddo. It'll be interesting to smoothen my workflow for posts, especially if I can get the hang of including images and screencasts, because then I can post more things quickly, which means I can squeeze them into the time that I have.

So those are basically the things I do for Emacs community maintenance. I'm kinda in a holding pattern while I'm still dealing with all the things that come with life with a small child, but kids grow faster than one might expect. I'll be able to get back to longer posts someday, but right now, this is good. Anyway, that's what I do because it's fun.

Hmm… Now that I think about it, the ideal things for me to work on:

  • are self-directed and low-risk, so that when I'm not working on them, I can fully focus on playing with A- instead of getting pulled back and forth by coordination or dealing with mistakes
  • can be broken down into small tasks (maybe 30min to an hour) and written about, or can fit into 5 minutes here and 5 minutes there, or can build on things I can do during non-screentime
  • aren't error-prone, since I'll be doing them late at night and possibly a bit sleepily (I tried early morning, but A- wakes up a bit unpredictably)
  • are text-based (easier to edit, update, share, and search), so I don't need to fiddle with video/audio for now
  • can result in compounding benefits, like automating a process or connecting people

So I might swap some of the things that don't fit that profile for things that do. I think that mostly means working on smaller chunks and writing more (possibly using autogenerated transcripts to split writing into recording and editing), and handing over some of the tasks I don't like so much so that other people can pick those tasks up if they want.

It might also be worth channeling people's goodwill into better things, too. For example, if people happen to want to thank me for Emacs News or other things, perhaps I can convince people to write a blog post and/or Reddit post sharing something Emacs-related (interesting thing you learned recently? a package that more people should know about? your story of how you got into it and what you've learned along the way?), since that can go out into the wider world and expand the conversation.

There's plenty of room to make things even more fun and more effective, so let's see. =)

View or add comments

Adding little nudges to help on the EmacsConf wiki

| emacs, emacsconf

A number of people helped capture the talks for EmacsConf 2021, which was fantastic because we were able to stream all of the first day's talks with open captions and most of the second day's talks too. Right now, in fact, there are only two talks left that haven't been captioned. After the conference, a couple of other people volunteered to help out as well. Whee!

I want to figure out a good way to help people work on the things that they're interested in without necessarily burdening them with too much work, too little work, too much coordination, not enough coordination. Before the conference, one of the perks we had offered was that captioners got early access to the videos. I had a password-protected directory on a web server and an index that I made using Emacs Lisp to display the the talks that still need to be captioned. People e-mailed me to call dibs on the talk they wanted to caption, and that was how we avoided duplicating work. Now that all the videos are public, of course, people can just go to the regular wiki.

The other thing to think about is that in addition to captioning the two remaining talks (not essential, but it would be nice), there are also different levels of things that we can do. It would be nice to have chapter markers for some of the longer Q&A sessions. It would be fantastic to cross reference those with the questions and answers so that so that people can jump to the section they're interested in. It'd be incredible if somebody actually wrote down the answers. And it'd be even more awesome if people actually captioned the Q&A sessions as well, which were in many cases much longer than the actual sessions. So this is a fair bit of work, but people can probably pick a level that matches their interest and time available.

I'm not entirely sure how to coordinate this especially since I've got limited computer time. So my goal is to have something where volunteers can basically just wander around looking for talks that they're interested in and see ways to help out, or see a list of things that could use some work. So for example, while they're browsing the maintainers talk, they might say, "Oh, this one needs some chapter markers. I want to help with that. How do I do that? How do I get started?" And then they go down that path. On the other hand, you might have somebody sitting down saying, "I've got an hour and I want to go help out. What can I do?"

I don't want to keep data in many different places. I wonder if I can use the wiki for a lot of this coordination. Now that the videos are public, I've started tagging the pages that need extra help, like long Q&A session that need chapter markers.

With a little bit more work, I think people will be able to follow the instructions from there, especially if they've done this kind of captioning before, or email us to ask for help and then we can get them started.

I also thought about using Etherpad to do that kind of coordination where people would put their name next to a thing to reserve it, but then that's one more step. I don't know. At the moment, editing the wiki is a bit of an involved process. Worst-case scenario (best-case, actually, if we have lots of people wanting to help? =) ), people can call dibs by emailing us at and one of us organizers will add a little note there in the volunteer attribute. It's probably a good start, so we'll see where we can take it.

View or add comments

2021-12-13 Emacs news

| emacs, emacs-news

Links from reddit.com/r/emacs, r/orgmode, r/spacemacs, r/planetemacs, Hacker News, planet.emacslife.com, YouTube, the Emacs NEWS file, Emacs Calendar, emacs-devel, and lemmy/c/emacs.

View or add comments