I want to compile my global microblog posts into weekly posts so that they're archived on my blog. It might make sense to make them list items so that I can move them around easily.

(defun my-mastodon-insert-my-statuses-since (date)
  (interactive (list (org-read-date "Since date: ")))
  (insert
   (format "#+begin_toot_archive\n%s\n#+end_toot_archive\n"
           (mapconcat
            (lambda (o)
              (format "- %s\n  #+begin_quote\n  #+begin_export html\n%s\n  #+end_export\n  #+end_quote\n\n"
                      (org-link-make-string (assoc-default 'url o) (assoc-default 'created_at o))
                      (org-ascii--indent-string (assoc-default 'content o) 2))
              ;; (format "#+begin_quote\n#+begin_export html\n%s\n#+end_export\n#+end_quote\n\n%s\n\n"
              ;;        (assoc-default 'content o)
              ;;        (org-link-make-string (assoc-default 'url o) (assoc-default 'created_at o)))
              )
            (seq-filter
             (lambda (o)
               (string= (assoc-default 'visibility o) "public"))
             (my-mastodon-fetch-posts-after
              (format "accounts/%s/statuses?count=40&exclude_reblogs=t&exclude_replies=t" (mastodon-auth--get-account-id))
              date))
            ""))))

Here's a little thing I used to convert a two-level list into my collapsible sections:

(defun my-org-convert-list-to-collapsible-details ()
  (interactive)
  (let ((list (org-list-to-lisp t)))
    (mapc (lambda (o)
            (when (stringp (car o))
              (insert
               (format
                "#+begin_my_details %s :open t\n%s#+end_my_details\n"
                (car o)
                (mapconcat
                 (lambda (s)
                   (concat "- " (string-trim (org-ascii--indent-string (car s) 2)) "\n"))
                 (cdr (cadr o)))))))
          (cdr list))))

And here are my toots from the past week, roughly categorized into collapsible sections:

As it turns out, org-list-to-org uses the Org export mechanism, so it quietly discards things like #+begin_export html blocks. I decided to hard-code assumptions about the list's structure instead, which works for now.

I'm gradually shifting more things to this Lenovo P52 to take advantage of its newer processor, 64 GB of RAM, and 2 TB drive. (Whee!) One of the things I'm curious about is how I can make better use of multimedia. I couldn't get whisper.cpp to work on my Lenovo X230T, so I mostly relied on the automatic transcripts from Google Recorder (with timestamps generated by aeneas) or cloud-based transcription services like Deepgram.

I have a lot of silences in my voice notes when I think out loud. whisper.cpp got stuck in loops during silent parts, but WhisperX handles them perfectly. WhisperX is also fast enough for me to handle audio files locally instead of relying on Deepgram. With the default model, I can process the files faster than real-time:

I used this command to get word-level timing data. It makes VTT and SRT files that underline the specific word:

~/vendor/whisperx/.venv/bin/whisperx --compute_type int8 --highlight_words True --print_progress True "$1"

The resulting VTT file looks like this:

WEBVTT

00:00.427 --> 00:00.507
<u>I</u> often need to... I sometimes need to replace or navigate by symbols.

00:00.507 --> 00:00.587
I often need to... I sometimes need to replace or navigate by symbols.

00:00.587 --> 00:00.887
I <u>often</u> need to... I sometimes need to replace or navigate by symbols.

00:00.887 --> 00:00.987
I often need to... I sometimes need to replace or navigate by symbols.

Sometimes I just want the text so that I can use an audio braindump as the starting point for a blog post or for notes. WhisperX is way more accurate than Google Recorder, so that will probably be easier once I update my workflow for that.

Sometimes I want to make an edited audio file that sounds smooth so that I can use it in a podcast, a video, or some audio notes. For that, I'd like word-level timing data so that I can cut out words or sections. Aeneas didn't give me word-level timestamps, but WhisperX does, so I can get the time information before I start editing. I can extract the word timestamps from the underlined text like this:

(defun my-subed-load-word-data-from-whisperx-highlights (file)
  "Return a list of word cues from FILE.
FILE should be a VTT or SRT file produced by whisperx with the
--highlight_words True option."
  (seq-keep (lambda (sub)
              (when (string-match "<u>\\(.+?\\)</u>" (elt sub 3))
                (setf (elt sub 3) (match-string 1 (elt sub 3)))
                sub))
            (subed-parse-file file)))

(defun my-subed-word-tsv-from-whisperx-highlights (file)
  (interactive "FVTT: ")
  (with-current-buffer (find-file-noselect (concat (file-name-nondirectory file) ".tsv"))
    (erase-buffer)
    (subed-tsv-mode)
    (subed-auto-insert)
    (mapc (lambda (sub) (apply #'subed-append-subtitle nil (cdr sub)))
          (my-subed-load-word-data-from-whisperx-highlights file))
    (switch-to-buffer (current-buffer))))

I like to use the TSV format for this one because it's easy to scan down the right side. Incidentally, this format is compatible with Audacity labels, so I could import that there if I wanted. I like Emacs much more, though. I'm used to having all my keyboard shortcuts at hand.

0.427000	0.507000	I
0.587000	0.887000	often
0.987000	1.227000	need
1.267000	1.508000	to...
4.329000	4.429000	I
4.469000	4.869000	sometimes
4.950000	5.170000	need
5.210000	5.410000	to
5.530000	6.090000	replace
6.270000	6.370000	or
6.490000	6.971000	navigate

Once I've deleted the words I don't want to include, I can merge subtitles for phrases so that I can keep the pauses between words. A quick heuristic is to merge subtitles if they don't have much of a pause between them.

(defvar my-subed-merge-close-subtitles-threshold 500)
(defun my-subed-merge-close-subtitles (threshold)
  "Merge subtitles with the following one if there is less than THRESHOLD msecs gap between them."
  (interactive (list (read-number "Threshold in msecs " my-subed-merge-close-subtitles-threshold)))
  (while (not (eobp))
    (let ((end (subed-subtitle-msecs-stop))
          (next-start (save-excursion
                        (and (subed-forward-subtitle-time-start)
                             (subed-subtitle-msecs-stop)))))
      (if (and end next-start (< (- next-start end) threshold))
          (subed-merge-with-next)
        (or (subed-forward-subtitle-end) (goto-char (point-max)))))))

Then I can use subed-waveform-show-all to tweak the start and end timestamps.

After that, I can use subed-record to compile the audio into an .opus file that sounds reasonably smooth.

I sometimes need to replace or navigate by symbols. casual-symbol-overlay is a package that adds a transient menu so that I don't have to remember the keyboard shortcuts for them. I've added it to my embark-symbol-keymap so I can call it with embark-act. That way it's just a C-. z away.

I want to make lots of quick audio notes that I can shuffle and listen to in order to remember things I'm learning about Emacs (might even come up with some kind of spaced repetition system), and I'd like to make more videos someday too. I think WhisperX, subed, and Org Mode will be fun parts of my workflow.

I use global-hl-line-mode to highlight the current line.

(global-hl-line-mode 1)

However, I don't want hl-line to interfere with the default face suggested by customize-face, which is returned by face-at-point.

(defun my-suggest-other-faces (func &rest args)
  (if global-hl-line-mode
      (progn
        (global-hl-line-mode -1)
        (prog1 (apply func args)
          (global-hl-line-mode 1)))
    (apply func args)))
(advice-add #'face-at-point :around #'my-suggest-other-faces)

Now I can use customize-face and describe-face without hl-line interfering all the time.

One of the things I like about browsing Mastodon in Emacs using mastodon.el is that I can modify my workflow to make things easier. For example, I often come across links that I'd like to save for Emacs News. I want to boost the post and save it to an Org file, and I can do that with a single keystroke. It uses the my-mastodon-store-link function defined elsewhere in my config.

(use-package org
  :config
  (add-to-list
   'org-capture-templates
   '("w" "Emacs News" entry (file+headline "~/sync/orgzly/news.org" "Collect Emacs News")
     "* %a  :news:

#+begin_quote
%:text
#+end_quote

"
     :prepend t :immediate-finish t)))
(defun my-mastodon-save-toot-for-emacs-news ()
  (interactive)
  ;; boost if not already boosted
  (unless (get-text-property
           (car
            (mastodon-tl--find-property-range 'byline (point)))
           'boosted-p)
    (mastodon-toot--toggle-boost-or-favourite 'boost))
  ;; store a link and capture the note
  (org-capture nil "w"))

(use-package mastodon
  :bind (:map mastodon-mode-map ("w" . my-mastodon-save-toot-for-emacs-news)))

This puts a bunch of notes in my ~/sync/orgzly/news.org file. Then I can use my-emacs-news-summarize-mastodon-items to summarize a bunch of items I've captured from Mastodon, taking the title from the first link and including a link to the toot using the author's handle. This is what it looks like:

Here's the code that makes that happen:

(defun my-emacs-news-summarize-mastodon-items ()
  (interactive)
  (while (not (eobp))
    (let* ((info (my-mastodon-get-note-info))
           (title (when (car (plist-get info :links))
                    (my-page-title (car (plist-get info :links)))))
           (summary (read-string
                     (if title
                         (format "Summary (%s): " title)
                       "Summary: ")
                     title)))
      (org-cut-subtree)
      (unless (string= summary "")
        (insert "- " (org-link-make-string
                      (or (car (plist-get info :links))
                          (plist-get info :url))
                      summary)
                (if (and (car (plist-get info :links))
                         (plist-get info :handle))
                    (concat " (" (org-link-make-string (plist-get info :url)
                                                       (plist-get info :handle))
                            ")")
                  "")
                "\n")))))
(defun my-mastodon-get-note-info ()
  "Return (:handle ... :url ... :links ... :text) for the current subtree."
  (let ((url (org-entry-get (point) "ITEM"))
        beg end
        handle)
    (save-excursion
      (org-back-to-heading)
      (org-end-of-meta-data)
      (setq beg (point))
      (setq end (org-end-of-subtree))
      (when (string-match "https://\\(.+?\\)/\\(@.+?\\)/" url)
        (setq handle (concat
                      (match-string 2 url) "@" (match-string 1 url))))

      (list
       :handle handle
       :url (if (string-match org-link-bracket-re url) (match-string 1 url) url)
       :links (mapcar (lambda (o) (org-element-property :raw-link o))
                      (my-org-get-links-in-region beg end))
       :text (string-trim (buffer-substring-no-properties beg end))))))

(ert-deftest my-mastodon-get-note-info ()
 (should
  (equal
   (with-temp-buffer
     (insert "** SOMEDAY https://mastodon.online/@jcastp/111762105597746747         :news:
:PROPERTIES:
:CREATED:  [2024-01-22 Mon 05:51]
:END:

jcastp@mastodon.online - I've shared my emacs config: https://codeberg.org/jcastp/emacs.d

After years of reading other's configs, copying really useful snippets, and tinkering a little bit myself, I wanted to give something back, although I'm still an amateur (and it shows, but I want to improve!)

If you can find there something you can use, then I'm happy to be useful to the community.

#emacs
")
     (org-mode)
     (my-mastodon-get-note-info))
   '(:handle "@jcastp@mastodon.online"
             :url
             "https://mastodon.online/@jcastp/111762105597746747"
             :links
             ("https://codeberg.org/jcastp/emacs.d")
             :text
             "jcastp@mastodon.online - I've shared my emacs config: https://codeberg.org/jcastp/emacs.d\n\nAfter years of reading other's configs, copying really useful snippets, and tinkering a little bit myself, I wanted to give something back, although I'm still an amateur (and it shows, but I want to improve!)\n\nIf you can find there something you can use, then I'm happy to be useful to the community.\n\n#emacs"))))

It turns headings into something like this:

• Insert a screenshot image in latex-mode with the help of org-download (@mmonga@mathstodon.xyz)

which I can then copy into my Emacs News Org Mode file and categorize with some keyboard shortcuts.

This works particularly well with my combined Mastodon timelines, because then I can look through all the #emacs posts from mastodon.social, emacs.ch, and social.sachachua.com in one go.

• Emacs 30:
  • Emacs 30.0.91 pretest is available (Reddit, Irreal, lemmy.ml)
  • Why Emacs Is the Best: 10 Key Advantages – Tristan's Zettelkasten (@tristanC@functional.cafe) - includes notes on upgrading to Emacs 30
• Upcoming events:
  • Emacs APAC: Announcing Emacs Asia-Pacific (APAC) virtual meetup, Saturday, September 28, 2024
  • M-x Research: TBA https://m-x-research.github.io/ Wed Sep 18 0800 America/Vancouver - 1000 America/Chicago - 1100 America/Toronto - 1500 Etc/GMT - 1700 Europe/Berlin - 2030 Asia/Kolkata - 2300 Asia/Singapore
  • Emacs Berlin (virtual, in English) https://emacs-berlin.org/ Wed Sep 25 0930 America/Vancouver - 1130 America/Chicago - 1230 America/Toronto - 1630 Etc/GMT - 1830 Europe/Berlin - 2200 Asia/Kolkata – Thu Sep 26 0030 Asia/Singapore
  • EmacsSF: Emacs Hangout: Potpourri (Hybrid) https://www.meetup.com/emacs-sf/events/303385779/ Thu Sep 26 1800 America/Vancouver - 2000 America/Chicago - 2100 America/Toronto – Fri Sep 27 0100 Etc/GMT - 0300 Europe/Berlin - 0630 Asia/Kolkata - 0900 Asia/Singapore
  • Emacs APAC (virtual) https://emacs-apac.gitlab.io/ Sat Sep 28 0130 America/Vancouver - 0330 America/Chicago - 0430 America/Toronto - 0830 Etc/GMT - 1030 Europe/Berlin - 1400 Asia/Kolkata - 1630 Asia/Singapore
• Beginner:
  • The Emacs Newbie Project (@summeremacs@fashionsocial.host, emacs-devel)
• Emacs configuration:
  • Emacs and kmonad (keyboard manager with multi-tap functionality and s-exps config)
  • Minimal ~/.emacs.d - Emacs Starter Kit with Better Defaults and Optimized Startup (Release 1.1.0) (Reddit)
  • rexim's Emacs Config
  • waymondo's Emacs Config
  • galahad's Emacs Config
  • funk443's Emacs Config
  • jokem59/personal-configs · GitHub
  • ektaynot's Doom Emacs Config
• Emacs Lisp:
  • Alvaro Ramirez: Spiffing up those echo messages (Reddit) - displaying SVGs in the echo area
  • Lightning Talk: Latest from GNU Emacs (05:35) Andrea Corallo, European Lisp Symposium
  • Ep585, Emacs Lisp, Find and Replace in a Dir, and Undo All (01:25:55)
• Appearance:
  • How to show row (line) numbers in Emacs - TuxeduTV
  • How to change theme in Emacs - TuxeduTV
  • Protesilaos Stavrou: Emacs: show-font version 0.1.0
  • Protesilaos Stavrou: Emacs: cursory version 1.1.0
  • [auto-dark] 0.13 - Support multiple themes (breaking-changes) (Reddit) - multiple themes, changed default way, added examples
  • The Modus Operandi theme on Emacs 30 is so elegant
  • El paquete adaptive-wrap para Emacs (@GK_S@mastodon.social)
• Navigation:
  • beginning-of-buffer and end-of-buffer tweaks from e3d3
  • pivaldi/nefle: Emacs package to search the nearest file from a given file-system location (@pi@emacs.ch)
  • La función Dedicated Windows en Emacs (@GK_S@mastodon.social)
  • Emacs 30.1 highlight - intuitive tab line (Reddit)
• TRAMP:
  • Short video on TRAMP (Reddit)
• Dired:
  • Protesilaos Stavrou: Emacs: dired-preview version 0.3.0
• Writing:
  • Chris Maiorana: Introducing a Word Diff Counter in Emacs
  • Silently save personal dictionary after inserting a new word: (setq ispell-silently-savep t)
• Org Mode:
  • Why and How I use "Org Mode" for my writing and more (2022, Reddit, Irreal)
  • Insert a screenshot image in latex-mode with the help of org-download (@mmonga@mathstodon.xyz)
  • Conditional tangling in org mode based on tags · GitHub (@zondo@mathstodon.xyz)
  • How I use :dbconnection in org files (@eslr@mastodon.social)
  • A quick hack to convert org/org-roam links from ID to filename with optional pandoc conversion to other formats
  • Irreal: Babel and Python
  • Guide to creating a multilingual website with org-publish (@len@toot.si)
  • Org-zettel-ref-mode release 0.3.1 (Reddit) - More elegant names for overviews, improved sync
  • Conversion script: Logseq to org roam (Reddit)
• Denote:
  • Protesilaos Stavrou: Emacs: consult-denote version 0.2.0
• Coding:
  • Announcing Casual Symbol Overlay (Reddit, Irreal)
  • csr-maker.el: Make CSR requests for SSL certs (@trevdev@fosstodon.org)
  • Unlock the Power of Tree-sitter in Emacs with Combobulate (13:29)
  • Magnus: Followup on secrets in my work notes
  • emacs-blame Update 0.2, New feat: support highlight all lines with the same commit information. (Reddit)
  • ~akagi/vc-pijul (@fd9a@fosstodon.org)
  • A function that checks if a file is committed to Git
• Shells:
  • #175 - Add menus for eat - akib/emacs-eat - Codeberg.org (@rudi@hachyderm.io)
  • Save shell history per project.el (@tristanC@functional.cafe)
• Mail, news, and chat:
  • Andrea: Adding utilities to open urls to emacs-slack
  • Sacha Chua: Combining Mastodon timelines using mastodon.el
• Evil mode:
  • Sanel Zukan: evil-mode in terminal without Alt/Meta
• EXWM:
  • Firefox in my EXWM (YouTube 01:04, Reddit) - includes some commands
• Fun:
  • Emacs on the Scene (Reddit, Irreal)
• AI:
  • meain/yap: Your Assistant for Prompting LLMs (@meain@toot.community)
  • Speech Dictation Mode for Emacs (Reddit, HN)
• Community:
  • Weekly Tips, Tricks, &c. Thread
  • Trying Emacs as a Neovim User // Just Chatting (01:35:29)
  • Part Four of My Battles with Emacs
  • Genuine Question, aren't some things better in other apps? (long discussion on Reddit)
• Other:
  • eee.el: Extended Emacs With External Tui application (Reddit)
  • Extending ekg with structured data
  • Anand Tamariya: The Oval Editor (Emacs) (YouTube 02:06, Reddit)- combine text with formatting and drawings
  • I got Emacs to run in iTerm on Mac!
• Emacs development:
  • emacs-devel:
    • The role of the last_marked array in understanding rare crashes during GC
  • Allow 'kill-region' to kill the last word when there is no region (actually kill-region-dwim)
  • Provide a modified xref backend for TeX buffers
  • New commands unix-word-rubout, unix-filename-rubout
  • ; * etc/NEWS: Announce the improvements in w32 GUI dialogs.
  • Make the grep buffer editable
  • Improve accuracy of character categories
  • Update Emacs sources for Unicode 16.0
  • Allow to disable symbolic links check in Dired
  • Add Ediff feature for copying all differences
  • Allow comments to organizer in icalendar event replies (Bug#72831)
• New packages:
  • astute: A minor mode to redisplay `smart' typography (MELPA)
  • codex-theme: Codex theme, a simple high contrast theme (MELPA)
  • elysium: Automatically apply LLM-created code-suggestions (MELPA)
  • geiser-overlay: Overlay Scheme evaluation results (MELPA)
  • glass-tty-theme: Reverse video-like theme for the Glass TTY VT220 font (MELPA)
  • kanagawa-themes: Elegant theme inspired by The Great Wave off Kanagawa (MELPA)
  • on: Hooks for faster startup (MELPA)
  • org-node-fakeroam: Stand-ins for org-roam-autosync-mode (MELPA)
  • py-vterm-interaction: A mode for Python REPL using vterm (MELPA)
  • show-font: Show font features in a buffer (GNU ELPA)

Links from reddit.com/r/emacs, r/orgmode, r/spacemacs, r/planetemacs, Mastodon #emacs, Hacker News, lobste.rs, programming.dev, lemmy.world, lemmy.ml, communick.news, planet.emacslife.com, YouTube, the Emacs NEWS file, Emacs Calendar, and emacs-devel. Thanks to Andrés Ramírez for emacs-devel links. Do you have an Emacs-related link or announcement? Please e-mail me at sacha@sachachua.com. Thank you!

I like checking out the #emacs hashtag when I put together Emacs News. In the past, I usually browsed the hashtag timeline on emacs.ch, which also picked up updates from other people that emacs.ch was following. Now that I've moved to @sacha@social.sachachua.com and emacs.ch is winding down, I wanted to see if there was a way for me to see a combined view using mastodon.social's API feed (paging by max_id as needed). I haven't enabled public timeline feeds on my server, so I also need to reuse the OAuth mechanics from mastodon.el.

First, let's start by making a unified timeline. By digging around in mastodon-tl.el, I found that I could easily create a timeline view by passing it a vector of toot JSONs.

(defun my-mastodon-fetch-posts-after (base-url after-date)
  "Page backwards through BASE-URL using max_id for all the posts after AFTER-DATE."
  (require 'plz)
  (require 'mastodon-http)
  (let ((results [])
        (url base-url)
        (use-mastodon-el (not (string-match "^http" base-url)))
        (json-array-type 'list)
        page filtered)
    (while url
      (setq page (if use-mastodon-el
                     (mastodon-http--get-json (mastodon-http--api url) nil :silent)
                   (seq-map (lambda (o)
                              (cons (cons 'external t) o))
                            (plz 'get url :as #'json-read)))
            filtered (seq-filter (lambda (o) (string< after-date (assoc-default 'created_at o)))
                                 page))
      (if filtered
          (progn
            (setq results (seq-concatenate 'vector filtered results)
                  url (concat base-url (if (string-match "\\?" base-url) "&" "?")
                              "max_id=" (number-to-string (1- (string-to-number (assoc-default 'id (elt (last page) 0)))))))
            (message "%s %s" (assoc-default 'created_at (elt (last page) 0)) url))
        (setq url nil)))
    results))

(defun my-mastodon-combined-tag-timeline (later-than tag servers)
  "Display items after LATER-THAN about TAG from SERVERS and the current mastodon.el account."
  (interactive (list
                (org-read-date nil nil nil nil nil "-Mon")
                "#emacs"
                '("mastodon.social" "emacs.ch")))
  (require 'mastodon)
  (require 'mastodon-tl)
  (require 'mastodon-toot)
  (let* ((limit 40)
         (sources (cons (format "timelines/tag/emacs?count=%d" limit)
                        (mapcar (lambda (s)
                                  (format "https://%s/api/v1/timelines/tag/emacs?count=%d" s limit))
                                servers)))
         (combined
          (sort
           (seq-reduce (lambda (prev val)
                         (seq-union prev
                                    (my-mastodon-fetch-posts-after val later-than)
                                    (lambda (a b) (string= (assoc-default 'uri a)
                                                           (assoc-default 'uri b)))))
                       sources [])
           (lambda (a b)
             (string< (assoc-default 'created_at b)
                      (assoc-default 'created_at a))))))
    (with-current-buffer (get-buffer-create "*Combined*")
      (let ((inhibit-read-only t))
        (erase-buffer)
        (mastodon-tl--timeline combined)
        (mastodon-mode))
      (setq mastodon-tl--buffer-spec `(account ,(cons mastodon-active-user mastodon-instance-url) buffer-name ,(buffer-name)))
      (display-buffer (current-buffer)))))

The tricky thing is that boosting and replying in mastodon.el both use the toot IDs instead of the toot URLs, so they only work for toots that came in via my current mastodon.el account. Toots from other timelines might not have been fetched by my server yet. Adding an external property lets me find that in the item_json text property in the timeline buffer. For those toots, I can use (mastodon-url-lookup (mastodon-toot--toot-url)) to open the toot in a new buffer that does allow boosting or replying, which is probably enough for my purposes.

(defun my-mastodon-lookup-toot ()
  (interactive)
  (mastodon-url-lookup (mastodon-toot--toot-url)))

When I go through Emacs News, I have a shortcut that boosts a post and saves it to as an Org Mode capture with a link to the toot. I sometimes want to reply, too. So I just need to intervene before boosting and replying. Boosting and favoriting both use mastodon-toot--action, which looks up the base-item-id text property. Replying looks up the item-json property and gets the id from it.

(defun my-text-property-update-at-point (pos prop value)
  (let ((start (previous-single-property-change (or pos (point)) prop))
        (end (next-single-property-change (or pos (point)) prop)))
    (put-text-property (or start (point-min))
                       (or end (point-max))
                       prop value)))

(defun my-mastodon-update-external-item-id (&rest _)
  (when (mastodon-tl--field 'external (mastodon-tl--property 'item-json))
    ;; ask the server to resolve it
    (let* ((response (mastodon-http--get-json (format "%s/api/v2/search" mastodon-instance-url)
                                              `(("q" . ,(mastodon-toot--toot-url))
                                                ("resolve" . "t"))))
           (id (alist-get 'id (seq-first (assoc-default 'statuses response))))
           (inhibit-read-only t)
           (json (get-text-property (point) 'item-json)))
      (when (and id json)
        (my-text-property-update-at-point (point) 'base-item-id id)
        (my-text-property-update-at-point (point) 'item-json
                                          (progn
                                            (setf (alist-get 'id json) id)
                                            (setf (alist-get 'external json) nil)
                                            json))))))

So now all I need to do is make sure that this is called before the relevant mastodon.el functions if I'm looking at an external toot.

(with-eval-after-load 'mastodon-tl
  (advice-add #'mastodon-toot--action :before #'my-mastodon-update-external-item-id)
  (advice-add #'mastodon-toot--reply :before #'my-mastodon-update-external-item-id)
  (advice-add #'mastodon-tl--thread :before #'my-mastodon-update-external-item-id))

The only thing is that I need to press RET after loading a thread with T (mastodon-tl--thread) for some reason, but that's okay. Now I can boost and save posts with my usual Emacs News shortcut, and I can reply easily too.

I'm curious: how many toots would I be missing if I looked at only one instance's hashtag? Let's look at the #emacs hashtag toots on 2024-09-12:

(defun my-three-way-comparison (seq1 seq2 seq3 &optional test-fn)
  `(("1" ,@(seq-difference seq1 (seq-union seq2 seq3 test-fn) test-fn))
    ("2" ,@(seq-difference seq2 (seq-union seq1 seq3 test-fn) test-fn))
    ("3" ,@(seq-difference seq3 (seq-union seq1 seq2 test-fn) test-fn))
    ("1&2" ,@(seq-difference (seq-intersection seq1 seq2 test-fn) seq3 test-fn))
    ("1&3" ,@(seq-difference (seq-intersection seq1 seq3 test-fn) seq2 test-fn))
    ("2&3" ,@(seq-difference (seq-intersection seq2 seq3 test-fn) seq1 test-fn))
    ("1&2&3" ,@(seq-intersection (seq-intersection seq2 seq3 test-fn) seq1 test-fn))))
(defun my-three-way-comparison-report (label1 seq1 label2 seq2 label3 seq3 &optional test-fn)
  (let ((list (my-three-way-comparison seq1 seq2 seq3)))
    `((,(format "%s only" label1) ,@(assoc-default "1" list #'string=))
      (,(format "%s only" label2) ,@(assoc-default "2" list #'string=))
      (,(format "%s only" label3) ,@(assoc-default "3" list #'string=))
      (,(format "%s & %s" label1 label2) ,@(assoc-default "1&2" list #'string=))
      (,(format "%s & %s" label1 label3) ,@(assoc-default "1&3" list #'string=))
      (,(format "%s & %s" label2 label3) ,@(assoc-default "2&3" list #'string=))
      ("all" ,@(assoc-default "1&2&3" list #'string=)))))

(assert (equal (my-three-way-comparison '("A" "A&B" "A&C" "A&B&C" "A1")
                                        '("B" "A&B" "A&B&C" "B&C")
                                        '("C" "A&C" "A&B&C" "B&C"))
               '(("1" "A" "A1")
                 ("2" "B")
                 ("3" "C")
                 ("1&2" "A&B")
                 ("1&3" "A&C")
                 ("2&3" "B&C")
                 ("1&2&3" "A&B&C"))))

(let* ((later-than "2024-09-12")
       (earlier-than "2024-09-13")
       (results
        (mapcar (lambda (o)
                  (cons (car o)
                        (seq-map (lambda (o) (assoc-default 'uri o))
                                 (seq-filter (lambda (toot)
                                               (string< (assoc-default 'created_at toot)
                                                        earlier-than))
                                             (my-mastodon-fetch-posts-after
                                              (format "%stimelines/tag/emacs?count=40" (cdr o))
                                              later-than)))))
                `((mastodon-social . "https://mastodon.social/api/v1/")
                  (emacs-ch . "https://emacs.ch/api/v1/")
                  (my-instance . ""))))
       (intersections
        (let-alist results
          (my-three-way-comparison-report
           "mastodon.social"
           .mastodon-social
           "emacs.ch"
           .emacs-ch
           "my instance"
           .my-instance
           #'string=))))
  (mapcar
   (lambda (row)
     (list (elt row 0) (length (cdr row))
           (string-join
            (seq-map-indexed (lambda (o i)
                               (org-link-make-string o (number-to-string (1+ i))))
                             (cdr row))
            " ")))
   intersections))

Here's an Euler diagram visualizing it.

I love that I can tinker with mastodon.el to get it to combine the timelines. (I'm crossing the streams!) Yay Emacs!

tldr: I'm now at @sacha@social.sachachua.com .

Since the emacs.ch Mastodon instance is winding down, I decided to follow @technomancy's recommendation of GoToSocial as a Mastodon-compatibly lightweight ActivityPub surver. I set it up as a Docker container because I still don't trust myself when it comes to security, and I put it behind an Nginx reverse proxy because I already have my web server on port 443.

The issues I ran across were mostly user error. Instead of copying my Nginx config from one of my other subsites, I should probably have just started with the sample Nginx configuration from the documentation. Eventually I figured out that I needed the Host line and I didn't need the location /.well-known/ thing from my other config. I did add add_header 'Access-Control-Allow-Origin' '*'; to allow cross-origin resource sharing (CORS), though, so that I could use semaphore.social as needed.

I also used:

docker network list
docker network inspect gotosocial_gotosocial

to find the IP addresses I needed for my config.yaml:

log-level: "error"
host: "social.sachachua.com"
landing-page-user: "sacha"
letsencrypt-enabled: false
port: 8080
bind-address: "172.22.0.2"
trusted-proxies:
  - "127.0.0.1/32"
  - "172.22.0.1"
  - "172.22.0.2"

I don't have a lot of space left on my Linode virtual private server, so I set up GoToSocial in a 10 GB block storage volume (of which it is now using 2.5GB). I have backups enabled, but the automatic backups don't cover block storage. I think I can work around that with a crontab entry that copies the sqlite* files from gotosocial/data into a backup directory in a regular partition.

I like using mastodon.el to go through the #emacs hashtag to find interesting posts and conversations for Emacs News. Since I have a single-user instance, I might not find as many #emacs posts on my own server compared to emacs.ch or mastodon.social #emacs. GoToSocial doesn't support relays yet. Until I feel reasonably confident that I'm catching a pretty good selection of #emacs posts from the people I follow and the instances I federate with, I might also check against those other timelines.

I think I've gotten @sacha@sachachua.com pointing to @sacha@social.sachachua.com, so if you'd like to follow me on Mastodon or another ActivityPub-compatible service, you can add either ID. I usually toot links to my Emacs News posts. I'd like to do a little more microblogging to share what I'm learning and figuring out, since I notice my blog posts tend to get sidetracked by yak-shaving. =)