Categories: geek » emacs

RSS - Atom - Subscribe via email

Archiving public toots on my blog

| mastodon, emacs, org

I want to compile my global microblog posts into weekly posts so that they're archived on my blog. It might make sense to make them list items so that I can move them around easily.

my-mastodon-insert-my-statuses-since
(defun my-mastodon-insert-my-statuses-since (date)
  (interactive (list (org-read-date "Since date: ")))
  (insert
   (format "#+begin_toot_archive\n%s\n#+end_toot_archive\n"
           (mapconcat
            (lambda (o)
              (format "- %s\n  #+begin_quote\n  #+begin_export html\n%s\n  #+end_export\n  #+end_quote\n\n"
                      (org-link-make-string (assoc-default 'url o) (assoc-default 'created_at o))
                      (org-ascii--indent-string (assoc-default 'content o) 2))
              ;; (format "#+begin_quote\n#+begin_export html\n%s\n#+end_export\n#+end_quote\n\n%s\n\n"
              ;;        (assoc-default 'content o)
              ;;        (org-link-make-string (assoc-default 'url o) (assoc-default 'created_at o)))
              )
            (seq-filter
             (lambda (o)
               (string= (assoc-default 'visibility o) "public"))
             (my-mastodon-fetch-posts-after
              (format "accounts/%s/statuses?count=40&exclude_reblogs=t&exclude_replies=t" (mastodon-auth--get-account-id))
              date))
            ""))))

Here's a little thing I used to convert a two-level list into my collapsible sections:

my-org-convert-list-to-collapsible-details
(defun my-org-convert-list-to-collapsible-details ()
  (interactive)
  (let ((list (org-list-to-lisp t)))
    (mapc (lambda (o)
            (when (stringp (car o))
              (insert
               (format
                "#+begin_my_details %s :open t\n%s#+end_my_details\n"
                (car o)
                (mapconcat
                 (lambda (s)
                   (concat "- " (string-trim (org-ascii--indent-string (car s) 2)) "\n"))
                 (cdr (cadr o)))))))
          (cdr list))))

And here are my toots from the past week, roughly categorized into collapsible sections:

EmacsConf
  • 2024-09-17T21:55:39.065Z CFP, draft schedule

    The #EmacsConf call for proposals (https://emacsconf.org/2024/cfp/) target date is this Friday (Sept 20), so I've started drafting the schedule. Thanks to my SVG schedule visualization code and function for checking availability constraints from last year ( https://sachachua.com/blog/2023/09/emacsconf-backstage-scheduling-with-svgs/), it took only about an hour to sketch this out: https://emacsconf.org/2024/organizers-notebook/#draft-schedule . We can run two tracks simultaneously and I can also slightly reduce the buffer between talks, so there's plenty of space for more #Emacs talks if people want to propose them or nudge people to propose them. =)

  • 2024-09-15T17:07:41.935Z diversity

    I feel complicated feelings about #EmacsConf and diversity. On one hand, yes, I would love to have a mix of speakers that reflects the mix of interesting stories and people I come across in the #Emacs community. (I wouldn't get rid of or discourage anyone; I just want more! :) )

    On the other hand, preparing and giving a presentation is a lot of work, and I have first-hand appreciation of how difficult it can be to find time to think - much less predict a specific time to have a conversation. (I'm only just beginning to be able to have some thinking time that isn't accompanied by the guilt of letting my kiddo binge-watch YouTube videos or the uncertainties of sacrificing my sleep, and I still rarely schedule anything for myself.)

    In addition, there are little risks that other people might not even have on their radar. All it takes is one person developing a parasocial relationship or fixation, or someone getting grumpy about someone's pronouns or personal characteristics or opinions, and then deciding to go and ruin someone's day (or life)... I'd hate to encourage someone to put themselves out there and end up with that happening to them, even if it's not at all their fault or mine.

    So yeah, it's a little hard for me to reach out. I can deal with impostor syndrome making people feel like they might not have much to say (share what you're learning! We're all figuring things out together), but I'm not so sure about the other concerns. While I'd like to think that in the Emacs community we often have a convivial atmosphere, sometimes it gets weird here too.

    I'm not sure what to do here aside from thinking out loud. I wish I could wave a magic wand and solve some structural issues that could make things more equitable, but that's waaay above my paygrade. I can keep working on figuring out how to make use of fragmented time, and maybe that will help other people too. I like working on the captions for EmacsConf; they help me a lot, too. I can experiment with workflows for sharing what I'm learning in a way that doesn't require a lot of focus time, speech fluency (I occasionally stutter and have to redo things), or a powerful computer. (Emacs is totally my nonlinear video editor.) I can make an indirect request for more people to consider proposing a talk for https://emacsconf.org/2024/cfp/ (target date is Sept 20, but I think the other organizers are considering extending it too), even with all the caveats my anxious brain suggests. (I know, I'm terrible at sales. :) ) And really, EmacsConf isn't important in the grand scheme of things, it's just a fun excuse to get together and connect with other people who like this stuff too. :)

    I wonder how this can be better. Thoughts?

Emacs
  • 2024-09-17T15:07:41.153Z consult-omni

    All right, I just got consult-omni and a Google custom search JSON API key set up so that I can call consult-omni-google, type keywords, pick the correct match, and insert it as an Org Mode link (or linkify the current region). I can think of more tweaks (embark-act on the current word or region to linkify it), but this is already pretty neat.

  • 2024-09-16T14:05:06.263Z - user-init-file

    Is there already an interactive #emacs command for opening user-init-file? I think that could be handy for newbies if we could just tell them to use "M-x visit-user-init-file" or even "Select 'Open init file' from the menu", although I suppose by the time we ask them to fiddle with the init file to add stuff to it, it's fine to encourage them to be comfortable with C-h v user-init-file and then maybe even teach them about M-x ffap at that point. Hmm...

  • 2024-09-16T13:51:12.091Z - casual-symbol-overlay

    Trying out casual-symbol-overlay (http://yummymelon.com/devnull/announcing-casual-symbol-overlay.html) by hooking it into embark-act, which I've bound to `C-.`:

    ```emacs-lisp
    (use-package casual-symbol-overlay
    :if my-laptop-p
    :init
    (with-eval-after-load 'embark
    (keymap-set embark-symbol-map "z" #'casual-symbol-overlay-tmenu)))
    ```

  • 2024-09-11T17:12:03.791Z no nested lists for Org Babel

    TIL that #OrgMode Babel only takes the top level of nested lists passed in via :var (https://orgmode.org/manual/Environment-of-a-Code-Block.html - Note that only the top-level list items are passed along. Nested list items are ignored.) When I try the manual example on my computer, I do indeed get only the top-level list items, unlike the nested data from https://mail.gnu.org/archive/html/emacs-orgmode/2020-10/msg00536.html . Of course, now that makes me want nested lists for both input and output...

Mastodon
Moving to P52
  • 2024-09-18T00:03:09.981Z WhisperX

    Now that I have word-level timestamps from WhisperX (https://sachachua.com/blog/2024/09/using-whisperx-to-get-word-level-timestamps-for-audio-editing-with-emacs-and-subed-record/), I think I'll be able to write an elisp equivalent of the merging/splitting strategies of https://github.com/jianfch/stable-ts?tab=readme-ov-file#regrouping-words to merge subtitles considering gap, duration, length, and maximum number of words.

  • 2024-09-15T17:08:17.757Z upgrade

    (reposting, forgot to make it public)

    I installed a 2TB Crucial T500 NVMe into my Lenovo P52 so that I can try dual-booting into Linux, since it was hard to figure out how I could get all my usual conveniences in WSL.

    A preliminary test with a fresh Kubuntu install showed that my 11ty static blog generation takes about the same time as it does on the X230T, which is a little surprising considering the newer processor and the faster SSD, but maybe I'll have to look for speed gains elsewhere there. I think whisper.cpp is a lot more usable on this computer though, so I'm looking forward to taking advantage of that. The P52 might also make video editing possible, and it might support more modern monitors. It is a fair bit larger and heavier, though. I might end up still using both.

    Anyway, I decided to redo the install by cloning my previous SSD. I want to see if I can skip the step of setting all those things up (although I'll need to redo the Syncthing config, of course). I don't have the extra parts that would let me install the 2.5" SSD from my X230T directly into the P52, but W- has a drive dock that works off USB 2.0. Slow and steady, but that's fine, I can run things overnight. I woke up today to find out that dd doesn't handle extended partitions and needs me to dd them one by one. That's cool, I'll just have that running in the background today.

    If the clone doesn't work or if it's too much trouble to take the clone and give it its own identity, I'll probably wipe it and do another install. Since the X230T is on Kubuntu, I think I'll keep it on Kubuntu as well, to minimize the things I need to keep in my head as I switch between computers. My home directory is in a separate partition, so I can keep it if I want to try something different.

    Now I just have to wait a few hours for these dd commands...

  • 2024-09-11T14:14:55.602Z static blog

    My "plugins is not iterable" issue got fixed when I downgraded `@11ty/eleventy` from `@beta` to `@2.0.1`. Yay, that's one thing off my list!

Other tech stuff
Parenting
  • 2024-09-17T12:28:29.082Z emotion check-in

    I appreciate my kiddo's grade 3 teacher. =) She's currently doing the morning check-in of emotions (how's everyone feeling) using 9 images of Grogu with different facial expressions, which gets the kids (1) laughing, (2) interpreting facial expressions that aren't explicitly labeled, and (3) figuring out what they're feeling.

  • 2024-09-12T12:50:02.014Z pull system

    The kiddo is 8 and I'm developing a better understanding of what "fiercely independent" means. One of the things I'm working on learning is how to shut up and trust the process. =) I've started thinking of it like the pull system of Lean manufacturing principles. Things work out better when I wait for her to ask a question (to pull from me) because at that point, she's ready to hear the answer.

As it turns out, org-list-to-org uses the Org export mechanism, so it quietly discards things like #+begin_export html blocks. I decided to hard-code assumptions about the list's structure instead, which works for now.

View org source for this post

Using WhisperX to get word-level timestamps for audio editing with Emacs and subed-record

Posted: - Modified: | emacs, subed

[2024-10-14 Mon]: Actually, WhisperX makes a JSON with word-level timing data, so let's use that instead.

I'm gradually shifting more things to this Lenovo P52 to take advantage of its newer processor, 64 GB of RAM, and 2 TB drive. (Whee!) One of the things I'm curious about is how I can make better use of multimedia. I couldn't get whisper.cpp to work on my Lenovo X230T, so I mostly relied on the automatic transcripts from Google Recorder (with timestamps generated by aeneas) or cloud-based transcription services like Deepgram.

I have a lot of silences in my voice notes when I think out loud. whisper.cpp got stuck in loops during silent parts, but WhisperX handles them perfectly. WhisperX is also fast enough for me to handle audio files locally instead of relying on Deepgram. With the default model, I can process the files faster than real-time:

File length Transcription time
42s 17s
7m48s 1m41s

I used this command to get word-level timing data.

~/vendor/whisperx/.venv/bin/whisperx --compute_type int8 --highlight_words True --print_progress True "$1"

Among other things, it makes a text file that looks like this:

I often need to... I sometimes need to replace or navigate by symbols.
Casual symbol overlays a new package that adds those shortcuts so that I don't have to remember the other keywords for them.

and a JSON file that looks like this:

{"segments": [{"start": 0.427, "end": 7.751, "text": " I often need to... I sometimes need to replace or navigate by symbols.", "words": [{"word": "I", "start": 0.427, "end": 0.507, "score": 0.994}, {"word": "often", "start": 0.587, "end": 0.887, "score": 0.856}, {"word": "need", "start": 0.987, "end": 1.227, "score": 0.851}, {"word": "to...", "start": 1.267, "end": 1.508, "score": 0.738}, {"word": "I", "start": 4.329, "end": 4.429, "score": 0.778}, ...]}, ...]}

Sometimes I just want the text so that I can use an audio braindump as the starting point for a blog post or for notes. WhisperX is way more accurate than Google Recorder, so that will probably be easier once I update my workflow for that.

Sometimes I want to make an edited audio file that sounds smooth so that I can use it in a podcast, a video, or some audio notes. For that, I'd like word-level timing data so that I can cut out words or sections. Aeneas didn't give me word-level timestamps, but WhisperX does, so I can get the time information before I start editing. I can extract the word timestamps from the JSON like this:

(defun my-subed-word-tsv-from-whisperx-json (file)
  (interactive "FJSON: ")
  (let* ((json-array-type 'list)
         (json-object-type 'alist)
         (data (json-read-file file))
         (filename (concat (file-name-sans-extension file) ".tsv"))
         (base (seq-mapcat
                (lambda (segment)
                  (seq-map (lambda (word)
                             (let-alist word
                               (list nil
                                     (and .start (* 1000 .start))
                                     (and .end (* 1000 .end))
                                     .word)))
                           (alist-get 'words segment)))
                (alist-get 'segments data)))
         (current base)
         (last-end 0))
     ;; numbers at the end of a sentence sometimes don't end up with times
     ;; so we need to fix them
    (while current
      (unless (elt (car current) 1)           ; start
        (setf (elt (car current) 1) (1+ last-end)))
      (unless (elt (car current) 2)
        (setf (elt (car current) 2) (1- (elt (cadr current) 1))))
      (setq
       last-end (elt (car current) 2)
       current (cdr current)))
    (subed-create-file
     filename
     base
     t
     'subed-tsv-mode)
    (find-file filename)))

Here's my old code for parsing the highlighted VTT or SRT files that underline each word:

(defun my-subed-load-word-data-from-whisperx-highlights (file)
  "Return a list of word cues from FILE.
FILE should be a VTT or SRT file produced by whisperx with the
--highlight_words True option."
  (seq-keep (lambda (sub)
              (when (string-match "<u>\\(.+?\\)</u>" (elt sub 3))
                (setf (elt sub 3) (match-string 1 (elt sub 3)))
                sub))
            (subed-parse-file file)))

(defun my-subed-word-tsv-from-whisperx-highlights (file)
  (interactive "FVTT: ")
  (with-current-buffer (find-file-noselect (concat (file-name-nondirectory file) ".tsv"))
    (erase-buffer)
    (subed-tsv-mode)
    (subed-auto-insert)
    (mapc (lambda (sub) (apply #'subed-append-subtitle nil (cdr sub)))
          (my-subed-load-word-data-from-whisperx-highlights file))
    (switch-to-buffer (current-buffer))))

I like to use the TSV format for this one because it's easy to scan down the right side. Incidentally, this format is compatible with Audacity labels, so I could import that there if I wanted. I like Emacs much more, though. I'm used to having all my keyboard shortcuts at hand.

0.427000	0.507000	I
0.587000	0.887000	often
0.987000	1.227000	need
1.267000	1.508000	to...
4.329000	4.429000	I
4.469000	4.869000	sometimes
4.950000	5.170000	need
5.210000	5.410000	to
5.530000	6.090000	replace

Once I've deleted the words I don't want to include, I can merge subtitles for phrases so that I can keep the pauses between words. A quick heuristic is to merge subtitles if they don't have much of a pause between them.

(defvar my-subed-merge-close-subtitles-threshold 500)
(defun my-subed-merge-close-subtitles (threshold)
  "Merge subtitles with the following one if there is less than THRESHOLD msecs gap between them."
  (interactive (list (read-number "Threshold in msecs: " my-subed-merge-close-subtitles-threshold)))
  (goto-char (point-min))
  (while (not (eobp))
    (let ((end (subed-subtitle-msecs-stop))
          (next-start (save-excursion
                        (and (subed-forward-subtitle-time-start)
                             (subed-subtitle-msecs-stop)))))
      (if (and end next-start (< (- next-start end) threshold))
          (subed-merge-with-next)
        (or (subed-forward-subtitle-end) (goto-char (point-max)))))))

Then I can use subed-waveform-show-all to tweak the start and end timestamps. Here I switch to another file I've been editing…

2024-09-17-12-06-12.svg
Figure 1: Screenshot of subed-waveform

After that, I can use subed-record to compile the audio into an .opus file that sounds reasonably smooth.

I sometimes need to replace or navigate by symbols. casual-symbol-overlay is a package that adds a transient menu so that I don't have to remember the keyboard shortcuts for them. I've added it to my embark-symbol-keymap so I can call it with embark-act. That way it's just a C-. z away.

I want to make lots of quick audio notes that I can shuffle and listen to in order to remember things I'm learning about Emacs (might even come up with some kind of spaced repetition system), and I'd like to make more videos someday too. I think WhisperX, subed, and Org Mode will be fun parts of my workflow.

This is part of my Emacs configuration.
View org source for this post

Highlight the current line while still being able to easily customize/describe underlying faces

| emacs

I use global-hl-line-mode to highlight the current line.

(global-hl-line-mode 1)

However, I don't want hl-line to interfere with the default face suggested by customize-face, which is returned by face-at-point.

(defun my-suggest-other-faces (func &rest args)
  (if global-hl-line-mode
      (progn
        (global-hl-line-mode -1)
        (prog1 (apply func args)
          (global-hl-line-mode 1)))
    (apply func args)))
(advice-add #'face-at-point :around #'my-suggest-other-faces)

Now I can use customize-face and describe-face without hl-line interfering all the time.

This is part of my Emacs configuration.

Collecting Emacs News from Mastodon

| emacs, mastodon

One of the things I like about browsing Mastodon in Emacs using mastodon.el is that I can modify my workflow to make things easier. For example, I often come across links that I'd like to save for Emacs News. I want to boost the post and save it to an Org file, and I can do that with a single keystroke. It uses the my-mastodon-store-link function defined elsewhere in my config.

(use-package org
  :config
  (add-to-list
   'org-capture-templates
   '("w" "Emacs News" entry (file+headline "~/sync/orgzly/news.org" "Collect Emacs News")
     "* %a  :news:

#+begin_quote
%:text
#+end_quote

"
     :prepend t :immediate-finish t)))
(defun my-mastodon-save-toot-for-emacs-news ()
  (interactive)
  ;; boost if not already boosted
  (unless (get-text-property
           (car
            (mastodon-tl--find-property-range 'byline (point)))
           'boosted-p)
    (mastodon-toot--toggle-boost-or-favourite 'boost))
  ;; store a link and capture the note
  (org-capture nil "w"))

(use-package mastodon
  :bind (:map mastodon-mode-map ("w" . my-mastodon-save-toot-for-emacs-news)))

This puts a bunch of notes in my ~/sync/orgzly/news.org file. Then I can use my-emacs-news-summarize-mastodon-items to summarize a bunch of items I've captured from Mastodon, taking the title from the first link and including a link to the toot using the author's handle. This is what it looks like:

output-2024-09-16-13:14:38.gif
Figure 1: Quick screencast summarizing Mastodon toots

Here's the code that makes that happen:

(defun my-emacs-news-summarize-mastodon-items ()
  (interactive)
  (while (not (eobp))
    (let* ((info (my-mastodon-get-note-info))
           (title (when (car (plist-get info :links))
                    (my-page-title (car (plist-get info :links)))))
           (summary (read-string
                     (if title
                         (format "Summary (%s): " title)
                       "Summary: ")
                     title)))
      (org-cut-subtree)
      (unless (string= summary "")
        (insert "- " (org-link-make-string
                      (or (car (plist-get info :links))
                          (plist-get info :url))
                      summary)
                (if (and (car (plist-get info :links))
                         (plist-get info :handle))
                    (concat " (" (org-link-make-string (plist-get info :url)
                                                       (plist-get info :handle))
                            ")")
                  "")
                "\n")))))
(defun my-mastodon-get-note-info ()
  "Return (:handle ... :url ... :links ... :text) for the current subtree."
  (let ((url (org-entry-get (point) "ITEM"))
        beg end
        handle)
    (save-excursion
      (org-back-to-heading)
      (org-end-of-meta-data)
      (setq beg (point))
      (setq end (org-end-of-subtree))
      (when (string-match "https://\\(.+?\\)/\\(@.+?\\)/" url)
        (setq handle (concat
                      (match-string 2 url) "@" (match-string 1 url))))

      (list
       :handle handle
       :url (if (string-match org-link-bracket-re url) (match-string 1 url) url)
       :links (mapcar (lambda (o) (org-element-property :raw-link o))
                      (my-org-get-links-in-region beg end))
       :text (string-trim (buffer-substring-no-properties beg end))))))

(ert-deftest my-mastodon-get-note-info ()
 (should
  (equal
   (with-temp-buffer
     (insert "** SOMEDAY https://mastodon.online/@jcastp/111762105597746747         :news:
:PROPERTIES:
:CREATED:  [2024-01-22 Mon 05:51]
:END:

jcastp@mastodon.online - I've shared my emacs config: https://codeberg.org/jcastp/emacs.d

After years of reading other's configs, copying really useful snippets, and tinkering a little bit myself, I wanted to give something back, although I'm still an amateur (and it shows, but I want to improve!)

If you can find there something you can use, then I'm happy to be useful to the community.

#emacs
")
     (org-mode)
     (my-mastodon-get-note-info))
   '(:handle "@jcastp@mastodon.online"
             :url
             "https://mastodon.online/@jcastp/111762105597746747"
             :links
             ("https://codeberg.org/jcastp/emacs.d")
             :text
             "jcastp@mastodon.online - I've shared my emacs config: https://codeberg.org/jcastp/emacs.d\n\nAfter years of reading other's configs, copying really useful snippets, and tinkering a little bit myself, I wanted to give something back, although I'm still an amateur (and it shows, but I want to improve!)\n\nIf you can find there something you can use, then I'm happy to be useful to the community.\n\n#emacs"))))

It turns headings into something like this:

which I can then copy into my Emacs News Org Mode file and categorize with some keyboard shortcuts.

This works particularly well with my combined Mastodon timelines, because then I can look through all the #emacs posts from mastodon.social, emacs.ch, and social.sachachua.com in one go.

This is part of my Emacs configuration.

2024-09-16 Emacs news

Posted: - Modified: | emacs, emacs-news

[2024-09-17 Tue]: Fixed link to buffer navigation tweaks, although emacswiki is still down

Links from reddit.com/r/emacs, r/orgmode, r/spacemacs, r/planetemacs, Mastodon #emacs, Hacker News, lobste.rs, programming.dev, lemmy.world, lemmy.ml, communick.news, planet.emacslife.com, YouTube, the Emacs NEWS file, Emacs Calendar, and emacs-devel. Thanks to Andrés Ramírez for emacs-devel links. Do you have an Emacs-related link or announcement? Please e-mail me at sacha@sachachua.com. Thank you!

View org source for this post

Combining Mastodon timelines using mastodon.el

Posted: - Modified: | emacs, mastodon
  • [2024-10-07 Mon]: Added screenshot.
  • [2024-09-16 Mon]: Read JSON arrays as lists to be compatible with the latest mastodon.el.
2024-10-07-10-37-39.svg
Figure 1: Screenshot of combined timeline in mastodon.el

I like checking out the #emacs hashtag when I put together Emacs News. In the past, I usually browsed the hashtag timeline on emacs.ch, which also picked up updates from other people that emacs.ch was following. Now that I've moved to @sacha@social.sachachua.com and emacs.ch is winding down, I wanted to see if there was a way for me to see a combined view using mastodon.social's API feed (paging by max_id as needed). I haven't enabled public timeline feeds on my server, so I also need to reuse the OAuth mechanics from mastodon.el.

First, let's start by making a unified timeline. By digging around in mastodon-tl.el, I found that I could easily create a timeline view by passing it a vector of toot JSONs.

(defun my-mastodon-fetch-posts-after (base-url after-date)
  "Page backwards through BASE-URL using max_id for all the posts after AFTER-DATE."
  (require 'plz)
  (require 'mastodon-http)
  (let ((results [])
        (url base-url)
        (use-mastodon-el (not (string-match "^http" base-url)))
        (json-array-type 'list)
        page filtered)
    (while url
      (setq page (if use-mastodon-el
                     (mastodon-http--get-json (mastodon-http--api url) nil :silent)
                   (seq-map (lambda (o)
                              (cons (cons 'external t) o))
                            (plz 'get url :as #'json-read)))
            filtered (seq-filter (lambda (o) (string< after-date (assoc-default 'created_at o)))
                                 page))
      (if filtered
          (progn
            (setq results (seq-concatenate 'vector filtered results)
                  url (concat base-url (if (string-match "\\?" base-url) "&" "?")
                              "max_id=" (number-to-string (1- (string-to-number (assoc-default 'id (elt (last page) 0)))))))
            (message "%s %s" (assoc-default 'created_at (elt (last page) 0)) url))
        (setq url nil)))
    results))

(defun my-mastodon-combined-tag-timeline (later-than tag servers)
  "Display items after LATER-THAN about TAG from SERVERS and the current mastodon.el account."
  (interactive (list
                (org-read-date nil nil nil nil nil "-Mon")
                "#emacs"
                '("mastodon.social" "emacs.ch" "fosstodon.org")))
  (require 'mastodon)
  (require 'mastodon-tl)
  (require 'mastodon-toot)
  (let* ((limit 40)
         (sources (cons (format "timelines/tag/emacs?count=%d" limit)
                        (mapcar (lambda (s)
                                  (format "https://%s/api/v1/timelines/tag/emacs?count=%d" s limit))
                                servers)))
         (combined
          (sort
           (seq-reduce (lambda (prev val)
                         (seq-union prev
                                    (my-mastodon-fetch-posts-after val later-than)
                                    (lambda (a b) (string= (assoc-default 'uri a)
                                                           (assoc-default 'uri b)))))
                       sources [])
           (lambda (a b)
             (string< (assoc-default 'created_at b)
                      (assoc-default 'created_at a))))))
    (with-current-buffer (get-buffer-create "*Combined*")
      (let ((inhibit-read-only t))
        (erase-buffer)
        (mastodon-tl--timeline combined)
        (mastodon-mode))
      (setq mastodon-tl--buffer-spec `(account ,(cons mastodon-active-user mastodon-instance-url) buffer-name ,(buffer-name)))
      (display-buffer (current-buffer)))))

The tricky thing is that boosting and replying in mastodon.el both use the toot IDs instead of the toot URLs, so they only work for toots that came in via my current mastodon.el account. Toots from other timelines might not have been fetched by my server yet. Adding an external property lets me find that in the item_json text property in the timeline buffer. For those toots, I can use (mastodon-url-lookup (mastodon-toot--toot-url)) to open the toot in a new buffer that does allow boosting or replying, which is probably enough for my purposes.

(defun my-mastodon-lookup-toot ()
  (interactive)
  (mastodon-url-lookup (mastodon-toot--toot-url)))

When I go through Emacs News, I have a shortcut that boosts a post and saves it to as an Org Mode capture with a link to the toot. I sometimes want to reply, too. So I just need to intervene before boosting and replying. Boosting and favoriting both use mastodon-toot--action, which looks up the base-item-id text property. Replying looks up the item-json property and gets the id from it.

(defun my-text-property-update-at-point (pos prop value)
  (let ((start (previous-single-property-change (or pos (point)) prop))
        (end (next-single-property-change (or pos (point)) prop)))
    (put-text-property (or start (point-min))
                       (or end (point-max))
                       prop value)))

(defun my-mastodon-update-external-item-id (&rest _)
  (when (mastodon-tl--field 'external (mastodon-tl--property 'item-json))
    ;; ask the server to resolve it
    (let* ((response (mastodon-http--get-json (format "%s/api/v2/search" mastodon-instance-url)
                                              `(("q" . ,(mastodon-toot--toot-url))
                                                ("resolve" . "t"))))
           (id (alist-get 'id (seq-first (assoc-default 'statuses response))))
           (inhibit-read-only t)
           (json (get-text-property (point) 'item-json)))
      (when (and id json)
        (my-text-property-update-at-point (point) 'base-item-id id)
        (my-text-property-update-at-point (point) 'item-json
                                          (progn
                                            (setf (alist-get 'id json) id)
                                            (setf (alist-get 'external json) nil)
                                            json))))))

So now all I need to do is make sure that this is called before the relevant mastodon.el functions if I'm looking at an external toot.

(with-eval-after-load 'mastodon-tl
  (advice-add #'mastodon-toot--action :before #'my-mastodon-update-external-item-id)
  (advice-add #'mastodon-toot--reply :before #'my-mastodon-update-external-item-id)
  (advice-add #'mastodon-tl--thread :before #'my-mastodon-update-external-item-id))

The only thing is that I need to press RET after loading a thread with T (mastodon-tl--thread) for some reason, but that's okay. Now I can boost and save posts with my usual Emacs News shortcut, and I can reply easily too.

I'm curious: how many toots would I be missing if I looked at only one instance's hashtag? Let's look at the #emacs hashtag toots on 2024-09-12:

Mastodon comparison
(defun my-three-way-comparison (seq1 seq2 seq3 &optional test-fn)
  `(("1" ,@(seq-difference seq1 (seq-union seq2 seq3 test-fn) test-fn))
    ("2" ,@(seq-difference seq2 (seq-union seq1 seq3 test-fn) test-fn))
    ("3" ,@(seq-difference seq3 (seq-union seq1 seq2 test-fn) test-fn))
    ("1&2" ,@(seq-difference (seq-intersection seq1 seq2 test-fn) seq3 test-fn))
    ("1&3" ,@(seq-difference (seq-intersection seq1 seq3 test-fn) seq2 test-fn))
    ("2&3" ,@(seq-difference (seq-intersection seq2 seq3 test-fn) seq1 test-fn))
    ("1&2&3" ,@(seq-intersection (seq-intersection seq2 seq3 test-fn) seq1 test-fn))))
(defun my-three-way-comparison-report (label1 seq1 label2 seq2 label3 seq3 &optional test-fn)
  (let ((list (my-three-way-comparison seq1 seq2 seq3)))
    `((,(format "%s only" label1) ,@(assoc-default "1" list #'string=))
      (,(format "%s only" label2) ,@(assoc-default "2" list #'string=))
      (,(format "%s only" label3) ,@(assoc-default "3" list #'string=))
      (,(format "%s & %s" label1 label2) ,@(assoc-default "1&2" list #'string=))
      (,(format "%s & %s" label1 label3) ,@(assoc-default "1&3" list #'string=))
      (,(format "%s & %s" label2 label3) ,@(assoc-default "2&3" list #'string=))
      ("all" ,@(assoc-default "1&2&3" list #'string=)))))

(assert (equal (my-three-way-comparison '("A" "A&B" "A&C" "A&B&C" "A1")
                                        '("B" "A&B" "A&B&C" "B&C")
                                        '("C" "A&C" "A&B&C" "B&C"))
               '(("1" "A" "A1")
                 ("2" "B")
                 ("3" "C")
                 ("1&2" "A&B")
                 ("1&3" "A&C")
                 ("2&3" "B&C")
                 ("1&2&3" "A&B&C"))))

(let* ((later-than "2024-09-12")
       (earlier-than "2024-09-13")
       (results
        (mapcar (lambda (o)
                  (cons (car o)
                        (seq-map (lambda (o) (assoc-default 'uri o))
                                 (seq-filter (lambda (toot)
                                               (string< (assoc-default 'created_at toot)
                                                        earlier-than))
                                             (my-mastodon-fetch-posts-after
                                              (format "%stimelines/tag/emacs?count=40" (cdr o))
                                              later-than)))))
                `((mastodon-social . "https://mastodon.social/api/v1/")
                  (emacs-ch . "https://emacs.ch/api/v1/")
                  (my-instance . ""))))
       (intersections
        (let-alist results
          (my-three-way-comparison-report
           "mastodon.social"
           .mastodon-social
           "emacs.ch"
           .emacs-ch
           "my instance"
           .my-instance
           #'string=))))
  (mapcar
   (lambda (row)
     (list (elt row 0) (length (cdr row))
           (string-join
            (seq-map-indexed (lambda (o i)
                               (org-link-make-string o (number-to-string (1+ i))))
                             (cdr row))
            " ")))
   intersections))
mastodon.social only 3 1 2 3
emacs.ch only 1 1
my instance only 0  
mastodon.social & emacs.ch 9 1 2 3 4 5 6 7 8 9
mastodon.social & my instance 0  
emacs.ch & my instance 1 1
all 11 1 2 3 4 5 6 7 8 9 10 11

Here's an Euler diagram visualizing it.

2024-09-13-12-45-10.png
Figure 2: #emacs posts on 2024-09-12 - an Euler diagram showing the table above

I love that I can tinker with mastodon.el to get it to combine the timelines. (I'm crossing the streams!) Yay Emacs!

This is part of my Emacs configuration.
View org source for this post

2024-09-09 Emacs news

| emacs, emacs-news

Links from reddit.com/r/emacs, r/orgmode, r/spacemacs, r/planetemacs, Mastodon #emacs, Hacker News, lobste.rs, programming.dev, lemmy.world, lemmy.ml, communick.news, planet.emacslife.com, YouTube, the Emacs NEWS file, Emacs Calendar, and emacs-devel. Thanks to Andrés Ramírez for emacs-devel links. Do you have an Emacs-related link or announcement? Please e-mail me at sacha@sachachua.com. Thank you!