Categories: geek » emacs » org

View topic page - RSS - Atom - Subscribe via email

Run source blocks in an Org Mode subtree by custom ID

| emacs, org

I like the way Org Mode lets me logically group functions into headings. If I give the heading a CUSTOM_ID property (which is also handy for exporting to HTML, as it turns into an link anchor), I can use that property to find the subtree. Then I can use org-babel-execute-subtree to execute all source blocks in that subtree, which means I can mix scripting languages if I want to.

Here's the code:

(defun my-org-execute-subtree-by-custom-id (id &optional filename)
  "Prompt for a CUSTOM_ID value and execute the subtree with that ID.
If called with \\[universal-argument], prompt for a file, and then prompt for the ID."
  (interactive (if current-prefix-arg
                   (let ((file (read-file-name "Filename: ")))
                     (list
                      (with-current-buffer (find-file-noselect file)
                        (completing-read
                         "Custom ID: "
                         (org-property-values "CUSTOM_ID")))
                      file))
                 (list
                  (completing-read "Custom ID: " (org-property-values "CUSTOM_ID")))))
  (with-current-buffer (if filename (find-file-noselect filename) (current-buffer))
    (let ((pos (org-find-property "CUSTOM_ID" id)))
      (if pos
          (org-babel-execute-subtree)
        (if filename(error "Could not find %s in %s" id filename)
          (error "Could not find %s" id))))))

For example, in Using Org Mode, Emacs Lisp, and TRAMP to parse meetup calendar entries and generate a crontab, I have a Emacs Lisp source block that generates a crontab on a different computer, and a shell source block that installs it on that computer.

Technical notes: org-babel-execute-subtree narrows to the current subtree, so if I want anything from the rest of the buffer, I need to widen the focus again. Also, it's wrapped in a save-restriction and a save-excursion, so someday I might want to figure out how to handle the cases where I want to change what I'm looking at.

elisp: links in Org Mode let me call functions by clicking on them or following them with C-c C-o (org-open-at-point). This means I can make links that execute subtrees that might even be in a different file. For example, I can define links like these:

  • [[elisp:(my-org-execute-subtree-by-custom-id "update" "~/sync/emacs-calendar/README.org")][Update Emacs calendar]]
  • [[elisp:(my-org-execute-subtree-by-custom-id "crontab" "~/sync/emacs-calendar/README.org")][Update Emacs meetup crontab]]

That could be a good starting point for a dashboard.

Related: Execute a single named Org Babel source block

This is part of my Emacs configuration.
View org source for this post

Using Org Mode, Emacs Lisp, and TRAMP to parse meetup calendar entries and generate a crontab

| org, emacs

Times and time zones trip me up. Even with calendar notifications, I still fumble scheduled events. Automation helps me avoid embarrassing hiccups.

We run BigBlueButton as a self-hosted web conferencing server for EmacsConf. It needs at least 8 GB of RAM when active. When it's dormant, it fits on a 1 GB RAM virtual private server. It's easy enough to scale the server up and down as needed. Using the server for Emacs meetups in between EmacsConfs gives people a way to get together, and it also means I can regularly test the infrastructure. That makes scaling it up for EmacsConf less nerve-wracking.

I have some code that processes various Emacs meetup iCalendar files (often with repeating entries) and combines them into one iCal file that people can subscribe to calendar, as well as Org files in different timezones that they can include in their org-agenda-files. The code I use to parse the iCal seems to handle time zones and daylight savings time just fine. I set it up so that the Org files have simple non-repeating entries, which makes them easy to parse. I can use the Org file to determine the scheduled jobs to run with cron on a home server (named xu4) that's up all the time.

This code parses the Org file for schedule information, then generates pairs of crontab entries. The first entry scales the BigBlueButton server up 1 hour before the event using my bbb-testing script, and the second entry scales the server down 6 hours after the event using my bbb-dormant script (more info). That gives organizers time to test it before the event starts, and it gives people plenty of time to chat. A shared CPU 8 GB RAM Linode costs USD 0.072 per hour, so that's USD 0.50 per meetup hosted.

Using #+begin_src emacs-lisp :file "/ssh:xu4:~/bbb.crontab" :results file as the header for my code block and using an SSH agent for authentication lets me use TRAMP to write the file directly to the server. (See Results of Evaluation (The Org Manual))

(let* ((file "/home/sacha/sync/emacs-calendar/emacs-calendar-toronto.org")
       (time-format "%M %H %d %m")
       (bbb-meetups "OrgMeetup\\|Emacs Berlin\\|Emacs APAC")
       (scale-up "/home/sacha/bin/bbb-testing")
       (scale-down "/home/sacha/bin/bbb-dormant"))
  (mapconcat
   (lambda (o)
     (let ((start-time (format-time-string time-format (- (car o) 3600 )))
           (end-time (format-time-string time-format (+ (car o) (* 6 3600)))))
       (format "# %s\n%s * %s\n%s * %s\n"
               (cdr o)
               start-time
               scale-up
               end-time
               scale-down)))
   (delq nil
         (with-temp-buffer
           (insert-file-contents file)
           (org-mode)
           (goto-char (point-min))
           (org-map-entries
            (lambda ()
              (when (and
                     (string-match bbb-meetups (org-entry-get (point) "ITEM"))
                     (re-search-forward org-tr-regexp (save-excursion (org-end-of-subtree)) t))
                (let ((time (match-string 0)))
                  (cons (org-time-string-to-seconds time)
                        (format "%s - %s" (org-entry-get (point) "ITEM") time)))))
            "LEVEL=1")))
   "\n"))

The code makes entries that look like this:

# OrgMeetup (virtual) - <2025-06-11 Wed 12:00>--<2025-06-11 Wed 14:00>
00 11 11 06 * /home/sacha/bin/bbb-testing
00 18 11 06 * /home/sacha/bin/bbb-dormant

# Emacs Berlin (hybrid, in English) - <2025-06-25 Wed 12:30>--<2025-06-25 Wed 14:30>
30 11 25 06 * /home/sacha/bin/bbb-testing
30 18 25 06 * /home/sacha/bin/bbb-dormant

# Emacs APAC: Emacs APAC meetup (virtual) - <2025-06-28 Sat 04:30>--<2025-06-28 Sat 06:00>
30 03 28 06 * /home/sacha/bin/bbb-testing
30 10 28 06 * /home/sacha/bin/bbb-dormant

This works because meetups don't currently overlap. If there were, I'll need to tweak the code so that the server isn't downscaled in the middle of a meetup. It'll be a good problem to have.

I need to load the crontab entries by using crontab bbb.crontab. Again, I can tell Org Mode to run this on the xu4 home server. This time I use the :dir argument to specify the default directory, like this:

#+begin_src sh :dir "/ssh:xu4:~" :results silent
crontab bbb.crontab
#+end_src

Then cron can take care of things automatically, and I'll just get the e-mail notifications from Linode telling me that the server has been resized. This has already come in handy, like when I thought of Emacs APAC as being on Saturday, but it was actually on Friday my time.

I have another Emacs Lisp block that I use to retrieve all the info and update the list of meetups. I can add (goto-char (org-find-property "CUSTOM_ID" "crontab")) to find this section and use org-babel-execute-subtree to execute all the code blocks. That makes it an automatic part of my process for updating the Emacs Calendar and Emacs News. Here's the code that does the calendar part (Org source):

(defun my-prepare-calendar-for-export ()
  (interactive)
  (with-current-buffer (find-file-noselect "~/sync/emacs-calendar/README.org")
    (save-restriction
      (widen)
      (goto-char (point-min))
      (re-search-forward "#\\+NAME: event-summary")
      (org-ctrl-c-ctrl-c)
      (org-export-to-file 'html "README.html")
      ;; (unless my-laptop-p (my-schedule-announcements-for-upcoming-emacs-meetups))
      ;; update the crontab
      (goto-char (org-find-property "CUSTOM_ID" "crontab"))
      (org-babel-execute-subtree)
      (when my-laptop-p
        (org-babel-goto-named-result "event-summary")
        (re-search-forward "^- ")
        (goto-char (match-beginning 0))
        (let ((events (org-babel-read-result)))
          (oddmuse-edit "EmacsWiki" "Usergroups")
          (goto-char (point-min))
          (delete-region (progn (re-search-forward "== Upcoming events ==\n\n") (match-end 0))
                         (progn (re-search-forward "^$") (match-beginning 0)))
          (save-excursion (insert (mapconcat (lambda (s) (concat "* " s "\n")) events ""))))))))
(my-prepare-calendar-for-export)

I used a similar technique to generate the EmacsConf crontabs for automatically switching to the next talk. For that one, I used Emacs Lisp to write the files directly instead of using the :file header argument for Org Mode source blocks. That made it easier to loop over multiple files.

Hmm. Come to think of it, the technique of "go to a specific subtree and then execute it" is pretty powerful. In the past, I've found it handy to execute source blocks by name. Executing a subtree by custom ID is even more useful because I can easily mix source blocks in different languages or include other information. I think that's worth adding a my-org-execute-subtree-by-custom-id function to my Emacs configuration. Combined with an elisp: link, I can make links that execute functional blocks that might even be in different files. That could be a good starting point for a dashboard.

I love the way Emacs can easily work with files and scripts in different languages on different computers, and how it can help me with times and time zones too. This code should help me avoid brain hiccups and calendar mixups so that people can just enjoy getting together. Now I don't have to worry about whether I remembered to set up cron entries and if I did the math right for the times. We'll see how it holds up!

View org source for this post

Org Mode: Cutting the current list item (including nested lists) with a speed command

| emacs, org

Defining shortcuts in org-speed-commands is handy because you can use these single-key shortcuts at the beginning of a subtree. With a little modification, they'll also work at the beginning of list items.

(defun my-org-use-speed-commands-for-headings-and-lists ()
  "Activate speed commands on list items too."
  (or (and (looking-at org-outline-regexp) (looking-back "^\**" nil))
      (save-excursion (and (looking-at (org-item-re)) (looking-back "^[ \t]*" nil)))))
(setq org-use-speed-commands 'my-org-use-speed-commands-for-headings-and-lists)

I want k to be an org-speed-commands that cuts the current subtree or list item. This is handy when I'm cleaning up the Mastodon toots in my weekly review or getting rid of outline items that I no longer need. By default, k is mapped to org-cut-subtree, but it's easy to override.

(defun my-org-cut-subtree-or-list-item (&optional n)
  "Cut current subtree or list item."
  (cond
   ((and (looking-at org-outline-regexp) (looking-back "^\**" nil))
    (org-cut-subtree n))
   ((looking-at (org-item-re))
    (kill-region (org-beginning-of-item) (org-end-of-item)))))
(with-eval-after-load 'org
  (setf (alist-get "k" org-speed-commands nil nil #'string=)
        #'my-org-cut-subtree-or-list-item))

So now, if I put my cursor before "1." below and press k:

- this
  1. is a
    - nested
  2. list
- with levels

it will turn into:

  • this
    1. list
  • with levels

You can find out a little more about Org Mode speed commands in the Org manual: (info "(org) Speed Keys").

This is part of my Emacs configuration.
View org source for this post

Tweaking my 11ty blog to link to the Mastodon post defined in an Org Mode property

| 11ty, mastodon, org

One of the things I like about blogging from Org Mode in Emacs is that it's easy to add properties to the section that I'm working on and then use those property values elsewhere. For example, I've modified Emacs to simplify tooting a link to my blog post and saving the Mastodon status URL in the EXPORT_MASTODON property. Then I can use that in my 11ty static site generation process to include a link to the Mastodon thread as a comment option.

First, I need to export the property and include it in the front matter. I use .11tydata.json files to store the details for each blog post. I modified ox-11ty.el so that I could specify functions to change the front matter (org-11ty-front-matter-functions, org-11ty--front-matter):

(defvar org-11ty-front-matter-functions nil
  "Functions to call with the current front matter plist and info.")
(defun org-11ty--front-matter (info)
  "Return front matter for INFO."
  (let* ((date (plist-get info :date))
         (title (plist-get info :title))
         (modified (plist-get info :modified))
         (permalink (plist-get info :permalink))
         (categories (plist-get info :categories))
         (collections (plist-get info :collections))
         (extra (if (plist-get info :extra) (json-parse-string
                                             (plist-get info :extra)
                                             :object-type 'plist))))
    (seq-reduce
     (lambda (prev val)
       (funcall val prev info))
     org-11ty-front-matter-functions
     (append
      extra
      (list :permalink permalink
            :date (if (listp date) (car date) date)
            :modified (if (listp modified) (car modified) modified)
            :title (if (listp title) (car title) title)
            :categories (if (stringp categories) (split-string categories) categories)
            :tags (if (stringp collections) (split-string collections) collections))))))

Then I added the EXPORT_MASTODON Org property as part of the front matter. This took a little figuring out because I needed to pass it as one of org-export-backend-options, where the parameter is defined as MASTODON but the actual property needs to be called EXPORT_MASTODON.

(defun my-org-11ty-add-mastodon-to-front-matter (front-matter info)
  (plist-put front-matter :mastodon (plist-get info :mastodon)))
(with-eval-after-load 'ox-11ty
  (cl-pushnew
   '(:mastodon "MASTODON" nil nil)
   (org-export-backend-options (org-export-get-backend '11ty)))
  (add-hook 'org-11ty-front-matter-functions #'my-org-11ty-add-mastodon-to-front-matter))

Then I added the Mastodon field as an option to my comments.cjs shortcode. This was a little tricky because I'm not sure I'm passing the data correctly to the shortcode (sometimes it ends up as item.data, sometimes it's item.data.data, …?), but with ?., I can just throw all the possibilities in there and it'll eventually find the right one.

const pluginRss = require('@11ty/eleventy-plugin-rss');
module.exports = function(eleventyConfig) {
  function getCommentChoices(data, ref) {
    const mastodonUrl = data.mastodon || data.page?.mastodon || data.data?.mastodon;
    const mastodon = mastodonUrl && `<a href="${mastodonUrl}" target="_blank" rel="noopener noreferrer">comment on Mastodon</a>`;
    const url = ref.absoluteUrl(data.url || data.permalink || data.data?.url || data.data?.permalink, data.metadata?.url || data.data?.metadata?.url);
    const subject = encodeURIComponent('Comment on ' + url);
    const body = encodeURIComponent("Name you want to be credited by (if any): \nMessage: \nCan I share your comment so other people can learn from it? Yes/No\n");
    const email = `<a href="mailto:sacha@sachachua.com?subject=${subject}&body=${body}">e-mail me at sacha@sachachua.com</a>`;
    const disqusLink = url + '#comment';
    const disqusForm = data.metadata?.disqusShortname && `<div id="disqus_thread"></div>
<script>
 var disqus_config = function () {
   this.page.url = "${url}";
   this.page.identifier = "${data.id || ''} ${data.metadata?.url || ''}?p=${ data.id || data.permalink || this.page?.url}";
   this.page.disqusTitle = "${ data.title }"
   this.page.postId = "${ data.id || data.permalink || this.page?.url }"
 };
 (function() { // DON'T EDIT BELOW THIS LINE
   var d = document, s = d.createElement('script');
   s.src = 'https://${ data.metadata?.disqusShortname }.disqus.com/embed.js';
   s.setAttribute('data-timestamp', +new Date());
   (d.head || d.body).appendChild(s);
 })();
</script>
<noscript>Disqus requires Javascript, but you can still e-mail me if you want!</noscript>`;
    return { mastodon, disqusLink, disqusForm, email };
  }
  eleventyConfig.addShortcode('comments', function(data, linksOnly=false) {
    const { mastodon, disqusForm, disqusLink, email } = getCommentChoices(data, this);
    if (linksOnly) {
      return `You can ${mastodon ? mastodon + ', ' : ''}<a href="${disqusLink}">comment with Disqus (JS required)</a>${mastodon ? ',' : ''} or ${email}.`;
    } else {
      return `<div id="comment"></div>
You can ${mastodon ? mastodon + ', ' : ''}comment with Disqus (JS required)${mastodon ? ', ' : ''} or you can ${email}.
${disqusForm || ''}`;}
  });
}

I included it in my post.cjs shortcode:

module.exports = eleventyConfig =>
eleventyConfig.addShortcode('post', async function(item, index, includeComments) {
  let comments = '<div class="comments">' + (includeComments ? this.comments(item) : this.comments(item, true)) + '</div>';
  let categoryList = item.categories || item.data && item.data.categories;
  let categoriesFooter = '', categories = '';
  if (categoryList && categoryList.length > 0) {
    categoriesFooter = `<div class="footer-categories">More posts about ${this.categoryList(categoryList)}</div>`;
    categories = `| <span class="categories">${this.categoryList(categoryList)}</span>`;
  }

  return  `<article class="post" id="index${index}" data-url="${item.url || item.permalink || ''}">
<header><h2 data-pagefind-meta="title"><a href="${item.url || item.permalink || ''}">${item.title || item.data && item.data.title}</a></h2>
${this.timeInfo(item)}${categories}
</header>
<div class="entry">
${await (item.templateContent || item.layoutContent || item.data?.content || item.content || item.inputContent)}
</div>
${comments}
${categoriesFooter}
</article>`;
});

I also included it in my RSS item template to make it easier for people to send me comments without having to dig through my website for contact info.

const posthtml = require("posthtml");
const urls = require("posthtml-urls");

module.exports = (eleventyConfig) => {
  eleventyConfig.addAsyncShortcode('rssItem', async function(item) {
    let content = item.templateContent.replace(/--/g, '&#45;&#45;');
    if (this.transformWithHtmlBase) {
      content = await this.transformWithHtmlBase(content);
    }
    return `<item>
    <title>${item.data.title}</title>
    <link>${this.absoluteUrl(item.url, item.data.metadata.url)}</link>
    <dc:creator><![CDATA[${item.data.metadata.author.name}]]></dc:creator>
    <pubDate>${item.date.toUTCString()}</pubDate>
    ${item.data.categories?.map((cat) => `<category>${cat}</category>`).join("\n") || ''}
    <guid isPermaLink="false">${this.guid(item)}</guid>
    <description><![CDATA[${content}
<p>${this.comments(item, true)}</p>]]></description>
    </item>`;
  });
};

The new workflow I'm trying out seems to be working:

  1. Keep npx eleventy --serve running in the background, using .eleventyignore to make rebuilds reasonably fast.
  2. Export the subtree with C-c e s 1 1, which uses org-export-dispatch to call my-org-11ty-export with the subtree.
  3. After about 10 seconds, use my-org-11ty-copy-just-this-post and verify.
  4. Use my-mastodon-11ty-toot-post to compose a toot. Edit the toot and post it.
  5. Check that the EXPORT_MASTODON property has been set.
  6. Export the subtree again, this time with the front matter.
  7. Publish my whole blog.

Next, I'm thinking of modifying my-mastodon-11ty-toot-post so that it includes a list of links to blog posts I might be building on or responding to, and possibly the handles of people related to those blog posts or topics. Hmm…

View org source for this post

mastodon.el: Copy toot content as Org Mode

| mastodon, org, emacs

Sometimes I want to copy a toot and include it in my Org Mode notes, like when I post a thought and then want to flesh it out into a blog post. This code defines my-mastodon-org-copy-toot-content, which converts the toot text to Org Mode format using Pandoc and puts it in the kill ring so I can yank it somewhere else.

(defun my-mastodon-toot-at-url (&optional url)
  "Return JSON toot object at URL.
If URL is nil, return JSON toot object at point."
  (if url
      (let* ((search (format "%s/api/v2/search" mastodon-instance-url))
             (params `(("q" . ,url)
                       ("resolve" . "t"))) ; webfinger
             (response (mastodon-http--get-json search params :silent)))
        (car (alist-get 'statuses response)))
    (mastodon-toot--base-toot-or-item-json)))

(defun my-mastodon-org-copy-toot-content (&optional url)
  "Copy the current toot's content as Org Mode.
Use pandoc to convert.

When called with \\[universal-argument], prompt for a URL."
  (interactive (list
                (when current-prefix-arg
                  (read-string "URL: "))))

  (let ((toot (my-mastodon-toot-at-url url)))
    (with-temp-buffer
      (insert (alist-get 'content toot))
      (call-process-region nil nil "pandoc" t t nil "-f" "html" "-t" "org")
      (kill-new
       (concat
        (org-link-make-string
         (alist-get 'url toot)
         (concat "@" (alist-get 'acct (alist-get 'account toot))))
        ":\n\n#+begin_quote\n"
        (string-trim (buffer-string)) "\n#+end_quote\n"))
      (message "Copied."))))
This is part of my Emacs configuration.
View org source for this post

Org Mode: Merge top-level items in an item list

| org

I usually summarize Mastodon links, move them to my Emacs News Org file, and then categorize them. Today I accidentically categorized the links while they were still in my Mastodon buffer, so I had two lists with categories. I wanted to write some Emacs Lisp to merge sublists based on the top-level items. I could sort the list alphabetically with C-c ^ (org-sort) and then delete the redundant top-level item lines, but it's fun to tinker with Emacs Lisp.

Example input:

  • Topic A:
    • Item 1
    • Item 2
      • Item 2.1
  • Topic B:
    • Item 3
  • Topic A:
    • Item 4
      • Item 4.1

Example output:

  • Topic B:
    • Item 3
  • Topic A:
    • Item 1
    • Item 2
      • Item 2.1
    • Item 4
      • Item 4.1

The sorting doesn't particularly matter to me, but I want the things under Topic A to be combined. Someday it might be nice to recursively merge other entries (ex: if there's another "Topic A: - Item 2" subitem like "Item 2.2"), but I don't need that yet.

Anyway, we can parse the list with org-list-to-lisp (which can even delete the original list) and recreate it with org-list-to-org, so then it's a matter of transforming the data structure.

(defun my-org-merge-list-entries-at-point ()
  "Merge entries in a nested Org Mode list at point that have the same top-level item text."
  (interactive)
  (save-excursion
    (let* ((list-indentation (save-excursion
                               (goto-char (caar (org-list-struct)))
                               (current-indentation)))
           (list-struct (org-list-to-lisp t))
           (merged-list (my-org-merge-list-entries list-struct)))
      (insert (org-ascii--indent-string (org-list-to-org merged-list) list-indentation)
              "\n"))))

(defun my-org-merge-list-entries (list-struct)
  "Merge an Org list based on its top-level headings"
  (cons (car list-struct)
        (mapcar
         (lambda (g)
           (list
            (car g)
            (let ((list-type (car (car (cdr (car (cdr g))))))
                  (entries (seq-mapcat #'cdar (mapcar #'cdr (cdr g)))))
              (apply #'append (list list-type) entries nil))))
         (seq-group-by #'car (cdr list-struct)))))

A couple of test cases:

(ert-deftest my-org-merge-list-entries ()
  (should
   (equal
    (my-org-merge-list-entries
     '(unordered ("Topic B:" (unordered ("Item 3")))))
    '(unordered ("Topic B:" (unordered ("Item 3"))))))
  (should
   (equal
    (my-org-merge-list-entries
     '(unordered ("Topic B:" (unordered ("Item 3")))
                 ("Topic A:"
                  (unordered ("Item 1")
                             ("Item 2"
                              (unordered ("Item 2.1")))))
                 ("Topic A:"
                  (unordered
                   ("Item 4" (unordered ("Item 4.1")))))))
    '(unordered
      ("Topic B:" (unordered ("Item 3")))
      ("Topic A:"
       (unordered ("Item 1")
                  ("Item 2" (unordered ("Item 2.1")))
                  ("Item 4" (unordered ("Item 4.1")))))))))
View org source for this post

Remove open Org Mode clock entries

| emacs, org

Pedro pointed out that I had some incomplete clock entries in my Emacs configuration. org-resolve-clocks prompts you for what to do with each open clock entry in your Org agenda files and whatever Org Mode files you have open.

If you don't feel like cancelling each clock with C, I also wrote this function to delete all open clocks in the current file.

(defun my-org-delete-open-clocks ()
  (interactive)
  (flush-lines
   (rx
    line-start
    (zero-or-more space)
    "CLOCK:"
    (one-or-more space)
    (regexp org-ts-regexp-inactive)
    (zero-or-more space)
    line-end)))
This is part of my Emacs configuration.
View org source for this post