Combining Mastodon timelines using mastodon.el
Posted: - Modified: | emacs: Read JSON arrays as lists to be compatible with the latest mastodon.el.
I like checking out the #emacs hashtag when I put together Emacs News. In the past, I usually browsed the hashtag timeline on emacs.ch, which also picked up updates from other people that emacs.ch was following. Now that I've moved to @sacha@social.sachachua.com and emacs.ch is winding down, I wanted to see if there was a way for me to see a combined view using mastodon.social's API feed (paging by
max_id as needed). I haven't enabled public
timeline feeds on my server, so I also need to reuse the OAuth mechanics from mastodon.el.
First, let's start by making a unified timeline. By digging around in mastodon-tl.el, I found that I could easily create a timeline view by passing it a vector of toot JSONs.
(defun my-mastodon-fetch-posts-after (base-url after-date) "Page backwards through BASE-URL using max_id for all the posts after AFTER-DATE." (require 'plz) (require 'mastodon-http) (let ((results []) (url base-url) (use-mastodon-el (not (string-match "^http" base-url))) (json-array-type 'list) page filtered) (while url (setq page (if use-mastodon-el (mastodon-http--get-json (mastodon-http--api url) nil :silent) (seq-map (lambda (o) (cons (cons 'external t) o)) (plz 'get url :as #'json-read))) filtered (seq-filter (lambda (o) (string< after-date (assoc-default 'created_at o))) page)) (if filtered (progn (setq results (seq-concatenate 'vector filtered results) url (concat base-url (if (string-match "\\?" base-url) "&" "?") "max_id=" (number-to-string (1- (string-to-number (assoc-default 'id (elt (last page) 0))))))) (message "%s %s" (assoc-default 'created_at (elt (last page) 0)) url)) (setq url nil))) results)) (defun my-mastodon-combined-tag-timeline (later-than tag servers) "Display items after LATER-THAN about TAG from SERVERS and the current mastodon.el account." (interactive (list (org-read-date nil nil nil nil nil "-Mon") "#emacs" '("mastodon.social" "emacs.ch"))) (require 'mastodon) (require 'mastodon-tl) (require 'mastodon-toot) (let* ((limit 40) (sources (cons (format "timelines/tag/emacs?count=%d" limit) (mapcar (lambda (s) (format "https://%s/api/v1/timelines/tag/emacs?count=%d" s limit)) servers))) (combined (sort (seq-reduce (lambda (prev val) (seq-union prev (my-mastodon-fetch-posts-after val later-than) (lambda (a b) (string= (assoc-default 'uri a) (assoc-default 'uri b))))) sources []) (lambda (a b) (string< (assoc-default 'created_at b) (assoc-default 'created_at a)))))) (with-current-buffer (get-buffer-create "*Combined*") (let ((inhibit-read-only t)) (erase-buffer) (mastodon-tl--timeline combined) (mastodon-mode)) (setq mastodon-tl--buffer-spec `(account ,(cons mastodon-active-user mastodon-instance-url) buffer-name ,(buffer-name))) (display-buffer (current-buffer)))))
The tricky thing is that boosting and replying in
mastodon.el both use the toot IDs instead of the
toot URLs, so they only work for toots that came
in via my current mastodon.el account. Toots from
other timelines might not have been fetched by my
server yet. Adding an external property lets me
find that in the item_json text property in the
timeline buffer. For those toots, I can use
(mastodon-url-lookup (mastodon-toot--toot-url))
to open the toot in a new buffer that does allow
boosting or replying, which is probably enough for
my purposes.
(defun my-mastodon-lookup-toot () (interactive) (mastodon-url-lookup (mastodon-toot--toot-url)))
When I go through Emacs News, I have a shortcut
that boosts a post and saves it to as an Org Mode
capture with a link to the toot. I sometimes want
to reply, too. So I just need to intervene before
boosting and replying. Boosting and favoriting
both use mastodon-toot--action, which looks up
the base-item-id text property. Replying looks
up the item-json property and gets the id from
it.
(defun my-text-property-update-at-point (pos prop value) (let ((start (previous-single-property-change (or pos (point)) prop)) (end (next-single-property-change (or pos (point)) prop))) (put-text-property (or start (point-min)) (or end (point-max)) prop value))) (defun my-mastodon-update-external-item-id (&rest _) (when (mastodon-tl--field 'external (mastodon-tl--property 'item-json)) ;; ask the server to resolve it (let* ((response (mastodon-http--get-json (format "%s/api/v2/search" mastodon-instance-url) `(("q" . ,(mastodon-toot--toot-url)) ("resolve" . "t")))) (id (alist-get 'id (seq-first (assoc-default 'statuses response)))) (inhibit-read-only t) (json (get-text-property (point) 'item-json))) (when (and id json) (my-text-property-update-at-point (point) 'base-item-id id) (my-text-property-update-at-point (point) 'item-json (progn (setf (alist-get 'id json) id) (setf (alist-get 'external json) nil) json))))))
So now all I need to do is make sure that this is called before the relevant mastodon.el functions if I'm looking at an external toot.
(with-eval-after-load 'mastodon-tl (advice-add #'mastodon-toot--action :before #'my-mastodon-update-external-item-id) (advice-add #'mastodon-toot--reply :before #'my-mastodon-update-external-item-id) (advice-add #'mastodon-tl--thread :before #'my-mastodon-update-external-item-id))
The only thing is that I need to press RET after loading a thread with T (mastodon-tl--thread) for some reason, but that's okay. Now I can boost and save posts with my usual Emacs News shortcut, and I can reply easily too.
I'm curious: how many toots would I be missing if I looked at only one instance's hashtag? Let's look at the #emacs hashtag toots on 2024-09-12:
Mastodon comparison
(defun my-three-way-comparison (seq1 seq2 seq3 &optional test-fn) `(("1" ,@(seq-difference seq1 (seq-union seq2 seq3 test-fn) test-fn)) ("2" ,@(seq-difference seq2 (seq-union seq1 seq3 test-fn) test-fn)) ("3" ,@(seq-difference seq3 (seq-union seq1 seq2 test-fn) test-fn)) ("1&2" ,@(seq-difference (seq-intersection seq1 seq2 test-fn) seq3 test-fn)) ("1&3" ,@(seq-difference (seq-intersection seq1 seq3 test-fn) seq2 test-fn)) ("2&3" ,@(seq-difference (seq-intersection seq2 seq3 test-fn) seq1 test-fn)) ("1&2&3" ,@(seq-intersection (seq-intersection seq2 seq3 test-fn) seq1 test-fn)))) (defun my-three-way-comparison-report (label1 seq1 label2 seq2 label3 seq3 &optional test-fn) (let ((list (my-three-way-comparison seq1 seq2 seq3))) `((,(format "%s only" label1) ,@(assoc-default "1" list #'string=)) (,(format "%s only" label2) ,@(assoc-default "2" list #'string=)) (,(format "%s only" label3) ,@(assoc-default "3" list #'string=)) (,(format "%s & %s" label1 label2) ,@(assoc-default "1&2" list #'string=)) (,(format "%s & %s" label1 label3) ,@(assoc-default "1&3" list #'string=)) (,(format "%s & %s" label2 label3) ,@(assoc-default "2&3" list #'string=)) ("all" ,@(assoc-default "1&2&3" list #'string=))))) (assert (equal (my-three-way-comparison '("A" "A&B" "A&C" "A&B&C" "A1") '("B" "A&B" "A&B&C" "B&C") '("C" "A&C" "A&B&C" "B&C")) '(("1" "A" "A1") ("2" "B") ("3" "C") ("1&2" "A&B") ("1&3" "A&C") ("2&3" "B&C") ("1&2&3" "A&B&C")))) (let* ((later-than "2024-09-12") (earlier-than "2024-09-13") (results (mapcar (lambda (o) (cons (car o) (seq-map (lambda (o) (assoc-default 'uri o)) (seq-filter (lambda (toot) (string< (assoc-default 'created_at toot) earlier-than)) (my-mastodon-fetch-posts-after (format "%stimelines/tag/emacs?count=40" (cdr o)) later-than))))) `((mastodon-social . "https://mastodon.social/api/v1/") (emacs-ch . "https://emacs.ch/api/v1/") (my-instance . "")))) (intersections (let-alist results (my-three-way-comparison-report "mastodon.social" .mastodon-social "emacs.ch" .emacs-ch "my instance" .my-instance #'string=)))) (mapcar (lambda (row) (list (elt row 0) (length (cdr row)) (string-join (seq-map-indexed (lambda (o i) (org-link-make-string o (number-to-string (1+ i)))) (cdr row)) " "))) intersections))
| mastodon.social only | 3 | 1 2 3 |
| emacs.ch only | 1 | 1 |
| my instance only | 0 | |
| mastodon.social & emacs.ch | 9 | 1 2 3 4 5 6 7 8 9 |
| mastodon.social & my instance | 0 | |
| emacs.ch & my instance | 1 | 1 |
| all | 11 | 1 2 3 4 5 6 7 8 9 10 11 |
Here's an Euler diagram visualizing it.
I love that I can tinker with mastodon.el to get it to combine the timelines. (I'm crossing the streams!) Yay Emacs!