#EmacsConf backstage: automatically updating talk status from the crontab

Oct 26, 2023| emacs, emacsconf, org

Now that I've figured out how to automatically play EmacsConf talks with crontab, I want to update our approach to using TRAMP and timers to run two tracks semi-automatically.

When a talk starts playing, I would like to:

announce it on IRC so that the talk details split up the chat log and make it easier to read
publish the media files to https://media.emacsconf.org/2023 (currently password-protected while testing) so that people can view them
update the talk wiki page with the media files and the captions
update the Youtube and Toobnix videos so that they're public: this is manual for the moment, since I haven't put time into automating it yet

I have code for most of this, and we've done it successfully for the past couple of years. I just need to refamiliarize myself with how to do it and how to set it up for testing during the dry run, and modify it to work with the new crontab-based system.

Setting things up

Last year, I set up the server with the ability to act as the controller, so that it wasn't limited to my laptop. The organizers notebook says I put it in the orga@ user's account, and I probably ran it under a screen session. I've submitted a talk for this year's conference, so I can use a test video for that one. First, I need to update the Ansible configuration for publishing and editing:

ansible-playbook -i inventory.yml prod-playbook.yml --tags publish,edit

I needed to add a version attribute to the git repo checkout in the Ansible playbook, since we'd switched from master to main. I also needed to set emacs_version to 29.1 since I started using seq-keep in my Emacs Lisp functions. For testing, I set emacsconf-publishing-phase to conference.

Act on TODO state changes

org-after-todo-state-change-hook makes it easy to automatically run functions when the TODO state changes. I add this hook that runs a list of functions and passes the talk information so that I don't have to parse the talk info in each function.

emacsconf-org-after-todo-state-change: Run all the hooks in ‘emacsconf-todo-hooks’.

(defun emacsconf-org-after-todo-state-change ()
  "Run all the hooks in `emacsconf-todo-hooks'.
If an `emacsconf-todo-hooks' entry is a list, run it only for the
tracks with the ID in the cdr of that list."
  (let* ((talk (emacsconf-get-talk-info-for-subtree))
         (track (emacsconf-get-track (plist-get talk :track))))
    (mapc
     (lambda (hook-entry)
       (cond
        ((symbolp hook-entry) (funcall hook-entry talk))
        ((member (plist-get track :id) (cdr hook-entry))
         (funcall (car hook-entry) talk))))
     emacsconf-todo-hooks)))

This can be enabled and disabled with the following functions.

emacsconf-add-org-after-todo-state-change-hook: Add FUNC to ‘org-after-todo-stage-change-hook’.

(defun emacsconf-add-org-after-todo-state-change-hook ()
  "Add FUNC to `org-after-todo-stage-change-hook'."
  (interactive)
  (with-current-buffer (find-buffer-visiting emacsconf-org-file)
    (add-hook 'org-after-todo-state-change-hook #'emacsconf-org-after-todo-state-change nil t)))

emacsconf-remove-org-after-todo-state-change-hook: Remove FUNC from ‘org-after-todo-stage-change-hook’.

(defun emacsconf-remove-org-after-todo-state-change-hook ()
  "Remove FUNC from `org-after-todo-stage-change-hook'."
  (interactive)
  (with-current-buffer (find-buffer-visiting emacsconf-org-file)
    (remove-hook 'org-after-todo-state-change-hook
                 #'emacsconf-org-after-todo-state-change  t)))

Announce on IRC

This is still much the same as last year.

emacsconf-erc-announce-on-change: Announce talk.

(defun emacsconf-erc-announce-on-change (talk)
  "Announce talk."
  (let ((func
         (pcase org-state
           ("PLAYING" #'erc-cmd-NOWPLAYING)
           ("CLOSED_Q" #'erc-cmd-NOWCLOSEDQ)
           ("OPEN_Q" #'erc-cmd-NOWOPENQ)
           ("UNSTREAMED_Q" #'erc-cmd-NOWUNSTREAMEDQ)
           ("TO_ARCHIVE" #'erc-cmd-NOWDONE))))
    (when func
      (funcall func talk))))

Here's a sample command that announces that the talk is now playing.

erc-cmd-NOWPLAYING: Set the channel topics to announce TALK.

(defun erc-cmd-NOWPLAYING (talk)
  "Set the channel topics to announce TALK."
  (interactive (list (emacsconf-complete-talk-info)))
  (when (stringp talk)
    (setq talk (or (emacsconf-find-talk-info talk) (error "Could not find talk %s" talk))))
  ;; Announce it in the track's channel
  (if (emacsconf-erc-recently-announced (format "---- %s:" (plist-get talk :slug)))
      (message "Recently announced, skipping")
    (when (plist-get talk :track)
      (emacsconf-erc-with-channels (list (concat "#" (plist-get talk :channel)))
        (erc-cmd-TOPIC
         (format
          "%s: %s (%s) pad: %s Q&A: %s | %s"
          (plist-get talk :slug)
          (plist-get talk :title)
          (plist-get talk :speakers)
          (plist-get talk :pad-url)
          (plist-get talk :qa-info)
          (car (assoc-default
                (concat "#" (plist-get talk :channel))
                emacsconf-topic-templates))))
        (erc-send-message (format "---- %s: %s - %s ----"
                                  (plist-get talk :slug)
                                  (plist-get talk :title)
                                  (plist-get talk :speakers-with-pronouns)))
        (erc-send-message
         (concat "Add your notes/questions to the pad: " (plist-get talk :pad-url)))
        (cond
         ((string-match "live" (or (plist-get talk :q-and-a) ""))
          (erc-send-message (concat "Live Q&A: " (plist-get talk :bbb-redirect))))
         ((plist-get talk :irc)
          (erc-send-message (format "or discuss the talk on IRC (nick: %s)"
                                    (plist-get talk :irc)))))))
    ;; Short announcement in #emacsconf
    (emacsconf-erc-with-channels (list emacsconf-erc-hallway emacsconf-erc-org)
      (erc-send-message (format "-- %s track: %s: %s (watch: %s, pad: %s, channel: #%s)"
                                (plist-get talk :track)
                                (plist-get talk :slug)
                                (plist-get talk :title)
                                (plist-get talk :watch-url)
                                (plist-get talk :pad-url)
                                (plist-get talk :channel))))))

Because the commands change the topic, I need to op the user in all the Emacsconf channels.

erc-cmd-OPALL

(defun erc-cmd-OPALL (&optional nick)
  (emacsconf-erc-with-channels (mapcar 'car emacsconf-topic-templates)
    (if nick
        (erc-cmd-OP nick)
      (erc-cmd-OPME))))

This code is in emacsconf-erc.el.

Publish media files

We used to scramble to upload all the videos in the days or weeks after the conference, since the presentations were live. Since we switched to encouraging speakers to upload videos before the conference, we've been able to release the videos pretty much as soon as the talk starts playing. This code in emacsconf-publish.el takes care of copying the files from the backstage to the public media directory, republishing the index, and republishing the playlist. That way, people who come in late or who want to refer to the video can easily get the full video right away.

emacsconf-publish-media-files-on-change: Publish the files and update the index.

(defun emacsconf-publish-media-files-on-change (talk)
  "Publish the files and update the index."
  (interactive (list (emacsconf-complete-talk-info)))
  (let ((org-state (if (boundp 'org-state) org-state (plist-get talk :status))))
    (if (plist-get talk :public)
        ;; Copy main extension files from backstage to public
        (let ((files (directory-files emacsconf-backstage-dir nil
                                      (concat "^"
                                              (regexp-quote (plist-get talk :file-prefix))
                                              (regexp-opt emacsconf-main-extensions)))))
          (mapc (lambda (file)
                  (when (and
                         (not (file-exists-p (expand-file-name file emacsconf-public-media-directory)))
                         (or (not (string-match "--main.vtt$" file))
                             (plist-get talk :captions-edited)))
                    (copy-file (expand-file-name file emacsconf-backstage-dir)
                               (expand-file-name file emacsconf-public-media-directory) t)))
                files))
      ;; Remove files from public
      (let ((files (directory-files emacsconf-public-media-directory nil
                                    (concat "^"
                                            (regexp-quote (plist-get talk :file-prefix)
                                                          )))))
        (mapc (lambda (file)
                (delete-file (expand-file-name file emacsconf-public-media-directory)))
              files)))
    (emacsconf-publish-public-index)
    (emacsconf-publish-playlist
     (expand-file-name "index.m3u" emacsconf-public-media-directory)
     (concat emacsconf-name " " emacsconf-year)
     (emacsconf-public-talks (emacsconf-get-talk-info)))))

The :public property is automatically added by this function based on the TODO status or the time:

emacsconf-add-talk-status: Add status label and public info.

(defun emacsconf-add-talk-status (o)
  "Add status label and public info."
  (plist-put o :status-label
             (or (assoc-default (plist-get o :status)
                                emacsconf-status-types 'string= "")
                 (plist-get o :status)))
  (when (or
         (member (plist-get o :status)
                 (split-string "PLAYING CLOSED_Q OPEN_Q UNSTREAMED_Q TO_ARCHIVE TO_EXTRACT TO_FOLLOW_UP DONE"))
         (time-less-p (plist-get o :start-time)
                      (current-time)))
    (plist-put o :public t))
  o)

Update the wiki page

This function updates the schedule page and the page for the talk. It's also in emacsconf-publish.el.

emacsconf-publish-update-talk: Publish the schedule page and the page for this talk.

(defun emacsconf-publish-update-talk (talk)
  "Publish the schedule page and the page for this talk."
  (interactive (list (emacsconf-complete-talk-info)))
  (when (stringp talk) (setq talk (emacsconf-resolve-talk talk)))
  (when (functionp 'emacsconf-upcoming-insert-or-update)
    (emacsconf-upcoming-insert-or-update))
  (emacsconf-publish-with-wiki-change
    (let ((info (emacsconf-publish-prepare-for-display (emacsconf-get-talk-info))))
      (emacsconf-publish-before-page talk info)
      (emacsconf-publish-after-page talk info)
      (emacsconf-publish-schedule info))))

It uses the publishing functions I described in this post on adding talks to the wiki.

This macro commits changes when emacsconf-publish-autocommit-wiki is t, so I need to set that also.

emacsconf-publish-with-wiki-change

(defmacro emacsconf-publish-with-wiki-change (&rest body)
  (declare (indent 0) (debug t))
  `(progn
     ,@body
     (emacsconf-publish-commit-and-push-wiki-maybe
      ,emacsconf-publish-autocommit-wiki
      (and (stringp ,(car body)) ,(car body)))))

Make the videos public on YouTube and Toobnix

This is low-priority, but it might be nice to figure out. The easiest way is probably to use open the Youtube/Toobnix URLs on my computer and then use either Tampermonkey or Spookfox to set the talk to public. Someday!

Update the talk status on the server

Last year, I experimented with having the shell scripts automatically update the status of the talk from TO_STREAM to PLAYING and from PLAYING to CLOSED_Q. Since I've moved the talk-running into track-specific crontabs, now I need to sudo back to the orga user and set XDG_RUNTIME_DIR in order to use emacsclient. I can call this with sudo -u orga talk $slug $status in the roles/obs/templates/handle-session script.

Here's the Ansible template for roles/prerec/templates/talk. It uses getent to look up the user ID.

#!/bin/bash
# 
# How to use: talk slug from-status-regexp to-status
# or talk slug to-status

SLUG="$1"
FROM_STATUS="$2"
TO_STATUS="$3"
XDG_RUNTIME_DIR=/run/user/

if [ "x$TO_STATUS" == "x" ]; then
    FROM_STATUS=.
    TO_STATUS="$2"
fi
cd 
#echo "Pulling conf.org..."
#git pull
echo "Updating status..."
XDG_RUNTIME_DIR=$XDG_RUNTIME_DIR emacsclient --eval "(emacsconf-with-todo-hooks (emacsconf-update-talk-status \"$SLUG\" \"$FROM_STATUS\" \"$TO_STATUS\"))" -a emacs
#echo "Committing and pushing in the background"
#git commit -m "Update task status for $SLUG from $FROM_STATUS to $TO_STATUS" conf.org
#git push &

Testing notes

Looks like everything works fine when I run it from the crontab: the talk status is updated, the media files are published, the wiki is updated, and the talks are announced on IRC. Backup plan A is to manually control the talk status using Emacs on the server. Backup plan B is to control the talk status using Emacs on my laptop. Backup plan C is to call the individual functions instead of relying on the todo state change functions. I think it'll all work out, although I'll probably want to do another dry run at some point to make sure. Slowly getting there…

You can comment with Disqus or you can e-mail me at sacha@sachachua.com.