Org Mode: calculating table sums using tag hierarchies

| org, elisp

While collecting posts for Emacs News, I came across this question about adding up Org Mode table data by tag hierarchy, which might be interesting if you want to add things up in different combinations. I haven't needed to do something like that myself, but I got curious about it. It turns out that you can define a tag hierarchy like this:

#+STARTUP: noptag
#+TAGS:
#+TAGS: [ GT1 : tagA tagC tagD ]
#+TAGS: [ GT2 : tagB tagE ]
#+TAGS: [ GT3 : tagB tagC tagD ]

The first two lines remove any other tags you've defined in your config aside from those in org-tag-persistent-alist, but can be omitted if you want to also include other tags you've defined in org-tag-alist. Note that it doesn't have to be a strict tree. Tags can belong to more than one tag group.

EduMerco wanted to know how to use those tag groups to sum up rows in a table. I added a #+NAME header to the table so that I could refer to it with :var source=source later on.

#+NAME: source
| tag  | Q1 | Q2 |
|------+----+----|
| tagA |  9 |    |
| tagB |  4 |  2 |
| tagC |  1 |  4 |
| tagD |    |  5 |
| tagE |    |  6 |
(defun my-sum-tag-groups (source &optional groups)
  "Sum up the rows in SOURCE by GROUPS.
If GROUPS is nil, use `org-tag-groups-alist'."
  (setq groups (or groups org-tag-groups-alist))
  (cons
   (car source)
   (mapcar
    (lambda (tag-group)
      (let ((tags (org--tags-expand-group (list (car tag-group))
                                          groups nil)))
        (cons (car tag-group)
              (seq-map-indexed
               (lambda (colname i)
                 (apply '+
                        (mapcar (lambda (tag)
                                  (let ((val (or (elt (assoc-default tag source) i) "0")))
                                    (if (stringp val)
                                        (string-to-number val)
                                      (or val 0))))
                                tags)))
               (cdr (car source))))))
    groups)))

Then that can be used with the following code:

#+begin_src emacs-lisp :var source=source :colnames no :results table
(my-sum-tag-groups source)
#+end_src

to result in:

tag Q1 Q2
GT1 10 9
GT2 4 8
GT3 5 11

Because org--tags-expand-group takes the groups as a parameter, you could use it to sum things by different groups. The #+TAGS: directives above set org-tag-groups-alist to:

(("GT1" "tagA" "tagC" "tagD")
 ("GT2" "tagB" "tagE")
 ("GT3" "tagB" "tagC" "tagD"))

Following the same format, we could do something like this:

(my-sum-tag-groups source '(("Main" "- Subgroup 1" "- Subgroup 2")
                            ("- Subgroup 1" "tagA" "tagB")
                            ("- Subgroup 2" "tagC" "tagD")
                            ))
tag Q1 Q2
Main 14 11
- Subgroup 1 13 2
- Subgroup 2 1 9

I haven't specifically needed to add tag groups in tables myself, but I suspect the recursive expansion in org--tags-expand-group might come in handy even in a non-Org context. Hmm…

View org source for this post

2025-09-29 Emacs news

Posted: - Modified: | emacs, emacs-news

: Fixed title for RDF editor

Very niche, but I'm happy to see that nethack-el is actively being worked on again. I remember having a lot of fun with that. =)

Also, the theme for October's Emacs Carnival is maintenance. Check out the posts for September's theme of obscure packages, too!

Links from reddit.com/r/emacs, r/orgmode, r/spacemacs, Mastodon #emacs, Bluesky #emacs, Hacker News, lobste.rs, programming.dev, lemmy.world, lemmy.ml, planet.emacslife.com, YouTube, the Emacs NEWS file, Emacs Calendar, and emacs-devel. Thanks to Andrés Ramírez for emacs-devel links. Do you have an Emacs-related link or announcement? Please e-mail me at sacha@sachachua.com. Thank you!

View org source for this post

Org Mode: a LaTeX letter that includes PDFs and hyperlinked page numbers

| org

I messed up on one of my tax forms, so I needed to send the tax agency a single document that included the amended tax return and the supporting slips, with my name, social insurance number, and reference number on every page. It turned out to be rather complicated trying to get calculated \pageref to work with \includepdf, so I just used \hyperlink with hard-coded page numbers. I also needed to use qpdf --decrypt input.pdf output.pdf to decrypt a PDF I downloaded from one of my banks before I could include it with \includepdf.

Here's what I wanted to do with this Org Mode / LaTeX example:

  • Coloured header on all pages with info and page numbers
  • Including PDFs
  • Hyperlinks to specific pages
* Letter
#+DATE: 2025-09-24
#+LATEX_CLASS: letter
#+OPTIONS: toc:nil ^:nil title:nil
#+LATEX_HEADER: \usepackage[margin=1in]{geometry}
#+LATEX_HEADER: \hypersetup{hidelinks}
#+LATEX_HEADER: \usepackage{pdfpages}
#+LATEX_HEADER: \usepackage{fancyhdr}
#+LATEX_HEADER: \usepackage{lastpage}
#+LATEX_HEADER: \usepackage{xcolor}
#+LATEX_HEADER: \signature{FULL NAME GOES HERE}
#+LATEX_HEADER: \fancypagestyle{plain}{
#+LATEX_HEADER: \fancyhf{}
#+LATEX_HEADER: \fancyhead[L]{\color{teal}\hyperlink{page.1}{HEADER INFO}}
#+LATEX_HEADER: \fancyhead[R]{\color{teal}\thepage\ of \pageref{LastPage}}
#+LATEX_HEADER: }
#+LATEX_HEADER: \pagestyle{plain}
#+LATEX_HEADER: \makeatletter
#+LATEX_HEADER: \let\ps@empty\ps@plain
#+LATEX_HEADER: \let\ps@firstpage\ps@plain
#+LATEX_HEADER: \makeatother
#+LATEX_HEADER: \renewcommand{\headrulewidth}{0pt}
#+LATEX_HEADER: \newcommand{\pdf}[1]{\includepdf[link,pages=-, scale=.8]{#1}}
#+LATEX_HEADER: \newcommand{\pages}[2]{\hyperlink{page.#1}{#1}-\hyperlink{page.#2}{#2}}
#+LATEX: \begin{letter}{}
#+LATEX: \opening{Dear person I am writing to:}

Text of the letter goes here.
Please find attached:

| Pages                             | |
| @@latex:\pages{2}{10}@@           | Description of filename1.pdf |
| @@latex:\hyperlink{page.5}{5}@@ | Can link to a specific page |
| @@latex:\pages{11}{15}@@           | Description of filename2.pdf |

#+LATEX:\closing{Best regards,}

#+LATEX: \end{letter}

#+LATEX: \pdf{filename1.pdf}
#+LATEX: \pdf{filename2.pdf}

After filling it in, I exported it with C-c C-e (org-export) C-s (to limit it to the subtree) l p (to export a PDF via LaTeX).

Not the end of the world. At least I learned a little more LaTeX and Org Mode along the way!

View org source for this post

Visual vocabulary practice - ABCs

| drawing

I've been giving myself more time to just enjoy drawing: not trying to untangle a thought, just wandering around and seeing where the lines and colours take me. While looking for examples of sketchnotes for self-facilitation for my post on finding the shape of my thoughts, I came across Sketchnotes: Changing The Way You See Your Thoughts — Creative Soul of Denise Nicole. I liked the ABC exercise near the bottom. For fun, I copied the same words and tried my own spin on things.

Text from sketch

Visual vocabulary practice - ABCs 2025-09-21-04

  • anchor
  • banner
  • calculate
  • DNA
  • energy
  • freeze
  • guitar
  • height
  • Instagram
  • judge
  • kid
  • ladder
  • meeting (A+ drew the details)
  • network
  • obstacle
  • planning
  • quote
  • repeat
  • scroll
  • think
  • universe
  • volley
  • weight
  • x-ray
  • yo-yo
  • zoo
  • extra: jar
  • extra: light

When A+ saw what I was doing, she asked me to swap out my meeting icon from "people around a table" to her online meetings at virtual school. She even added details: "This is the kid with the Minecraft background, this is the kid with big headphones…" I enjoyed watching her in this state of playful focus. I wonder what else I can draw that she might have fun taking over. "Meeting" is my favourite one in this set, but since that's mostly A+'s, my next favourite is "freeze." Canada gets even colder than -20C, but for me, -20C is definitely stay inside weather.

I've also been enjoying Kamo's books, like How to Draw Cute Doodles and Illustrations. Her style reminds me of the Illustration School series by Sachiko Umoto, which I also liked. I think I tend towards simple and approachable rather than realistic or technically impressive. Learning how to draw concrete things might help me get better at drawing abstract things. It's fun to slow down and pay attention to more details, too. Turns out I'd been drawing guitar holes in the wrong place all this time. Now I know!

Related posts:

Here are some other resources that might be helpful:

View org source for this post

EmacsConf infrastructure upgrades

| emacsconf

With the EmacsConf call for proposals now closed, I have a little time before EmacsConf speakers send in their pre-recorded videos come in for captioning. I decided to dust off the infrastructure to see what makes sense to upgrade.

We use Etherpad for collaborative note-taking during EmacsConf. It's straightforward to use and pretty reliable. Conference participants can use it to share notes, questions, and links. They can also use IRC to ask questions, and volunteers copy those questions into the pad for the talk. Hosts and speakers can keep an eye on the pad for questions. We send speakers a copy of their talk's pad after the conference, and we post that along with other follow-up questions on the conference wiki. Here's an example: Writing academic papers in Org-Roam.

A native Emacs solution for collaborative notes would be even better. CRDT was great for experimenting with real-time collaboration within Emacs, but I'm not sure it can handle a ton of simultaneous connections and I don't want to find out in the middle of the conference. Also, requiring Emacs would leave out the people who only have a web browser handy. It would be super cool if we had something with Emacs, web, and IRC interfaces, but for now, Etherpad will do.

We started by using Wikimedia's instance, and then we moved to hosting our own. For the past two years, we've used Etherpad 1.9.7. Etherpad is currently at version 2.5.0. There are some performance improvements, bugfixes, and security fixes, so I think it'll be worth upgrading to that. I don't know of any specific issues or upgrades, but it's a good idea to stay closer to the latest release than to get too far out of date.

I switched our roles/pad/tasks/main.yml to use systemli.etherpad from ansible-galaxy. I also figured out how to set up a Vagrant virtual machine that I could destroy and reconfigure with vagrant destroy; vagrant up --provision. Here's my Vagrant file for that:

# -*- mode: ruby -*-
Vagrant.configure("2") do |config|
  config.vm.box = "debian/bookworm64"
  config.vm.define "pad" do |pad|
  end
  config.vm.provider "virtualbox" do |vb|
    vb.memory = "2048"
  end
  config.vm.network "private_network", ip: "192.168.56.2"
  config.vm.provision "ansible" do |ansible|
    ansible.playbook = "../../vagrant-playbook.yml"
  end
end

The playbook it refers to has this:

- name: Pre-flight checks and package installation
  hosts: pad
  become: true
  gather_facts: false
  pre_tasks:
    - name: Ensure ntpdate is installed for time sync
      ansible.builtin.apt:
        name: ntpdate
        state: present
        update_cache: yes
    - name: Synchronize system clock
      ansible.builtin.command: ntpdate pool.ntp.org
      changed_when: true
    - name: Ensure ACL package is installed
      ansible.builtin.apt:
        name: acl
        state: present
- name: Load vars
  hosts: pad
  tags: always
  tasks:
    - include_vars:
        file: vagrant-vars.yml
- name: Set up pad proxy
  hosts: pad
  tags: proxy
  roles:
    - pad-proxy
- name: Set up pad
  hosts: pad
  tags: pad
  roles:
    - pad

I had used Vagrant in 2013, and it felt good to have the time to set up more testing infrastructure. I liked being able to test my pad and pad-proxy roles against a local virtual machine. I could figure out whatever tweaks I needed without messing up the production instance that we use for some meetups in between conferences.

Our production instance is on Debian 10 (Buster). That has reached its end of life for security updates, so apt-cache update doesn't work on it any more, and those steps in my Ansible playbook fail. I'm waiting for Amin Bandali to work on upgrading that server, since he has other stuff running on it. By setting update_cache variable that I override in my inventory.yml and referring to it with update_cache: "" in my task, I can conditionally disable the apt-cache update steps. That let me run the playbook against the production server, and now we're on Etherpad 2.x.

I recently updated our BigBlueButton instance to version 3.0.12. We've decided to stay with Icecast 2.4.4-1 for doing the livestreaming. We'll probably also keep OBS 29.1.2 and ffmpeg 6.0.1 instead of upgrading. With no must-have new features and other organizers' limited availability, it's better to keep those parts stable. This year, we'll continue using whisperx to help with the first draft of captions, but we'll probably try large-v3 instead of large-v2 by default. Some people find that large-v3's performance is better, some people find it's worse, so we'll see. Now that I know about whisperx's --initial_prompt option, I might be able to nudge it to the vocabulary and punctuation style we like.

Since the bones seem pretty solid, I'm looking forward to refamiliarizing myself with the Emacs Lisp code I wrote to help run the conference. I saved a bunch of improvement ideas from last year, and I can't wait to turn them into code. That's probably going to be lots of fun!

View org source for this post

Emacs: Cycle through different paragraph formats: all on one line, wrapped, max one sentence per line, one sentence per line

Posted: - Modified: | emacs

: Add move-to-left-margin to work around bug when using fill-paragraph-semlf at the end of a paragraph.

I came across Schauderbasis - reformat paragraph via @EFLS@mastodon.social. Now I want M-q to cycle through different ways of wrapping text:

  • all on one line
  • according to fill-column
  • at most one sentence per line (although still wrapping at fill-column)
  • at most one sentence per line (don't even try to keep it within fill-column).

Screencast cycling through different paragraph formats

Now that semantic linefeeds are part of core Emacs (as of 2025-06-14), the code for cycling through different paragraph formats can be pretty short. Most of it is actually just the logic for cycling through different commands. That might come in handy elsewhere. There's an unfill package as well, but since the code for unfilling a paragraph is very simple, I'll just include that part.

Note that fill-paragraph-semlf pays attention to sentence-end-double-space, and it doesn't handle comments yet. I also have some code to check if I'm in a comment and skip those filling methods if needed.

This might encourage me to write shorter sentences. I can move sentences around with M-Shift-up and M-Shift-down in Org Mode, which is pretty handy. Also, one sentence per line makes diffs easier to read. But wrapped text is annoying to edit in Orgzly Revived on my phone, because the wrapping makes a very ragged edge on a narrow screen. I might unwrap things that I want to edit there. With a little bit of tweaking to skip source blocks, I can narrow to the subtree, select my whole buffer, and cycle the formatting however I like.

(defvar my-repeat-counter '()
  "How often `my-repeat-next' was called in a row using the same command.
This is an alist of (cat count list) so we can use it for different functions.")

(defun my-unfill-paragraph ()
  "Replace newline chars in current paragraph by single spaces.
This command does the inverse of `fill-paragraph'."
  (interactive)
  (let ((fill-column most-positive-fixnum))
    (fill-paragraph)))

(defun my-fill-paragraph-semlf-long ()
  (interactive)
  (let ((fill-column most-positive-fixnum))
    (fill-paragraph-semlf)))

(defun my-repeat-next (category &optional element-list reset)
  "Return the next element for CATEGORY.
Initialize with ELEMENT-LIST if this is the first time."
  (let* ((counter
          (or (assoc category my-repeat-counter)
              (progn
                (push (list category -1 element-list)
                      my-repeat-counter)
                (assoc category my-repeat-counter)))))
    (setf (elt (cdr counter) 0)
          (mod
           (if reset 0 (1+ (elt (cdr counter) 0)))
           (length (elt (cdr counter) 1))))
    (elt (elt (cdr counter) 1) (elt (cdr counter) 0))))

(defun my-in-prefixed-comment-p ()
  (or (member 'font-lock-comment-delimiter-face (face-at-point nil t))
      (member 'font-lock-comment-face (face-at-point nil t))
      (save-excursion
        (beginning-of-line)
        (comment-search-forward (line-end-position) t))))

;; It might be nice to figure out what state we're
;; in and then cycle to the next one if we're just
;; working with a single paragraph. In the
;; meantime, just going by repeats is fine.
(defun my-reformat-paragraph-or-region ()
  "Cycles the paragraph between three states: filled/unfilled/fill-sentences.
If a region is selected, handle all paragraphs within that region."
  (interactive)
  (let ((func (my-repeat-next 'my-reformat-paragraph
                              '(fill-paragraph my-unfill-paragraph fill-paragraph-semlf
                                               my-fill-paragraph-semlf-long)
                              (not (eq this-command last-command))))
        (deactivate-mark nil))
    (if (region-active-p)
        (save-restriction
          (save-excursion
            (narrow-to-region (region-beginning) (region-end))
            (goto-char (point-min))
            (while (not (eobp))
              (skip-syntax-forward " ")
              (let ((elem (and (derived-mode-p 'org-mode)
                               (org-element-context))))
                (cond
                 ((eq (org-element-type elem) 'headline)
                  (org-forward-paragraph))
                 ((member (org-element-type elem)
                          '(src-block export-block headline property-drawer))
                  (goto-char
                   (org-element-end (org-element-context))))
                 (t
                  (funcall func)
                  (if fill-forward-paragraph-function
                      (funcall fill-forward-paragraph-function)
                    (forward-paragraph))))))))
      (save-excursion
        (move-to-left-margin)
        (funcall func)))))

(keymap-global-set "M-q" #'my-reformat-paragraph-or-region)

Sometimes I use writeroom-mode to make the lines look even narrower, with lots of margin on the side.

Related:

This is part of my Emacs configuration.
View org source for this post

2025-09-22 Emacs news

| emacs, emacs-news

If you want to write functions that let you pick values with completion, check out Manuel and Corwin's posts for simple examples, or chmouel's post for a yasnippet version. Enjoy!

Links from reddit.com/r/emacs, r/orgmode, r/spacemacs, r/planetemacs, Mastodon #emacs, Bluesky #emacs, Hacker News, lobste.rs, programming.dev, lemmy.world, lemmy.ml, planet.emacslife.com, YouTube, the Emacs NEWS file, Emacs Calendar, and emacs-devel. Thanks to Andrés Ramírez for emacs-devel links. Do you have an Emacs-related link or announcement? Please e-mail me at sacha@sachachua.com. Thank you!

View org source for this post