#EmacsConf backstage: coordinating captioning volunteers using a backstage area
| emacs, emacsconf
One of the benefits of volunteering for or speaking at EmacsConf is that you get early access to the talks. We upload the files to a password-protected web server. It's nothing fancy, just a directory that has basic authentication and a static index.html generated by Emacs Lisp.
I organize the talks by TODO status so that people can easily see
which talks are ready to be captioned. Since captioning can take a
little while, we reserve the talks we want to caption. That way,
people don't accidentally work on a talk that someone else is already
captioning. When people e-mail me to reserve a talk, I move the talk
from TO_ASSIGN to TO_CAPTION and add their name in the
:CAPTIONER: property.
Publishing the backstage index is done by an Emacs Lisp function that smooshes the information together and then writes the files directly to the server using TRAMP.
emacsconf-publish-backstage-index
(defun emacsconf-publish-backstage-index (&optional filename) (interactive) (setq filename (or filename (expand-file-name "index.html" emacsconf-backstage-dir))) (let ((info (or emacsconf-schedule-draft (emacsconf-publish-prepare-for-display (emacsconf-get-talk-info)))) (emacsconf-main-extensions (append emacsconf-main-extensions emacsconf-publish-backstage-extensions))) (with-temp-file filename (let* ((talks (mapcar (lambda (o) (append (list :captions-edited t) o)) (seq-filter (lambda (o) (plist-get o :speakers)) (emacsconf-active-talks (emacsconf-filter-talks info))))) (by-status (seq-group-by (lambda (o) (plist-get o :status)) talks)) (files (directory-files emacsconf-backstage-dir))) (insert "<html><head><meta charset=\"UTF-8\"><link rel=\"stylesheet\" href=\"/style.css\" /></head><body>" (if (file-exists-p (expand-file-name "include-in-index.html" emacsconf-cache-dir)) (with-temp-buffer (insert-file-contents (expand-file-name "include-in-index.html" emacsconf-cache-dir)) (buffer-string)) "") "<p>Schedule by status: (gray: waiting, light yellow: processing, yellow: to assign, light green: captioning, green: captioned and ready)<br />Updated by conf.org and the wiki repository</br />" (let* ((emacsconf-schedule-svg-modify-functions '(emacsconf-schedule-svg-color-by-status)) (img (emacsconf-schedule-svg 800 200 info))) (with-temp-buffer (mapc (lambda (node) (dom-set-attribute node 'href (concat "#" (dom-attr node 'data-slug)))) (dom-by-tag img 'a)) (svg-print img) (buffer-string))) "</p>" (if (eq emacsconf-backstage-phase 'prerec) (format "<p>Waiting for %d talks (~%d minutes) out of %d total</p>" (length (assoc-default "WAITING_FOR_PREREC" by-status)) (emacsconf-sum :time (assoc-default "WAITING_FOR_PREREC" by-status)) (length talks)) "") "<ul>" (mapconcat (lambda (status) (concat "<li>" (if (string= status "TO_ASSIGN") "TO_ASSIGN (waiting for volunteers)" status) ": " (mapconcat (lambda (o) (format "<a href=\"#%s\">%s</a>" (plist-get o :slug) (plist-get o :slug))) (assoc-default status by-status) ", ") "</li>")) (pcase emacsconf-backstage-phase ('prerec '("TO_PROCESS" "PROCESSING" "TO_ASSIGN" "TO_CAPTION" "TO_STREAM")) ('harvest '("TO_ARCHIVE" "TO_REVIEW_QA" "TO_INDEX_QA" "TO_CAPTION_QA"))) "") "</ul>" (pcase emacsconf-backstage-phase ('prerec (concat (emacsconf-publish-backstage-processing by-status files) (emacsconf-publish-backstage-to-assign by-status files) (emacsconf-publish-backstage-to-caption by-status files) (emacsconf-publish-backstage-to-stream by-status files))) ('harvest (let ((stages '(("TO_REVIEW_QA" . "Please review the --bbb-webcams.webm file and/or the --bbb-webcams.vtt and tell us (emacsconf-submit@gnu.org) if a Q&A session can be published or if it needs to be trimmed (lots of silence at the end of the recording, accidentally included sensitive information, etc.).") ("TO_INDEX_QA" . "Please review the --answers.webm and --answers.vtt files to make chapter markers so that people can jump to specific parts of the Q&A session. The <a href=\"https://emacsconf.org/harvesting/\">harvesting page on the wiki</a> has some notes on the process. That way, it's easier for people to see the questions and answers without needing to listen to everything again. You can see <a href=\"https://emacsconf.org/2022/talks/asmblox\">asmblox</a> for an example of the Q&A chapter markers.") ("TO_CAPTION_QA" . "Please edit the --answers.vtt for the Q&A talk you're interested in, correcting misrecognized words and cleaning it up so that it's nice to use as closed captions. All talks should now have large-model VTTs to make it easier to edit.")))) (mapconcat (lambda (stage) (let ((status (car stage))) (format "<h1>%s: %d talk(s) (%d minutes)</h1>%s<ol class=\"videos\">%s</ol>" status (length (assoc-default status by-status)) (emacsconf-sum :video-time (assoc-default status by-status)) (cdr stage) (mapconcat (lambda (f) (format "<li><a name=\"%s\"></a><strong><a href=\"%s%s\">%s</a></strong><br />%s (id:%s)<br />%s</li>" (plist-get f :slug) emacsconf-base-url (plist-get f :url) (plist-get f :title) (plist-get f :speakers-with-pronouns) (plist-get f :slug) (emacsconf-publish-index-card (append (list :video-note (unless (file-exists-p (expand-file-name (concat (plist-get f :file-prefix) "--bbb-webcams.webm") emacsconf-cache-dir)) "<div>No Q&A video for this talk</div>") :video-file (cond ((file-exists-p (expand-file-name (concat (plist-get f :file-prefix) "--answers.webm") emacsconf-cache-dir)) (concat (plist-get f :file-prefix) "--answers.webm")) ((file-exists-p (expand-file-name (concat (plist-get f :file-prefix) "--bbb-webcams.webm") emacsconf-cache-dir)) (concat (plist-get f :file-prefix) "--bbb-webcams.webm")) (t t)) ;; omit video :video-id "-qanda" :extra (concat (emacsconf-surround "QA note: " (plist-get f :qa-note) "<br />") (format "Q&A archiving: <a href=\"%s-%s.txt\">IRC: %s-%s</a>" (format-time-string "%Y-%m-%d" (plist-get f :start-time)) (plist-get (emacsconf-get-track f) :channel) (format-time-string "%Y-%m-%d" (plist-get f :start-time)) (plist-get (emacsconf-get-track f) :channel)) (emacsconf-surround ", <a href=\"" (if (file-exists-p (expand-file-name (concat (plist-get f :file-prefix) "--pad.txt") emacsconf-cache-dir)) (concat (plist-get f :file-prefix) "--pad.txt")) "\">Etherpad (Markdown)</a>" "") (emacsconf-surround ", <a href=\"" (plist-get f :bbb-playback) "\">BBB playback</a>" "") (emacsconf-surround ", <a href=\"" (if (file-exists-p (expand-file-name (concat (plist-get f :file-prefix) "--bbb.txt") emacsconf-cache-dir)) (concat (plist-get f :file-prefix) "--bbb.txt")) "\">BBB text chat</a>" "") (emacsconf-surround ", <a href=\"" (if (file-exists-p (expand-file-name (concat (plist-get f :file-prefix) "--bbb-webcams.opus") emacsconf-cache-dir)) (concat (plist-get f :file-prefix) "--bbb-webcams.opus")) "\">BBB audio only</a>" "")) :files (emacsconf-publish-talk-files f files)) f) emacsconf-main-extensions))) (assoc-default status by-status) "\n")))) stages "\n")))) (if (file-exists-p (expand-file-name "include-in-index-footer.html" emacsconf-cache-dir)) (with-temp-buffer (insert-file-contents (expand-file-name "include-in-index-footer.html" emacsconf-cache-dir)) (buffer-string)) "") "</body></html>")))))
For example, the section for talks that are waiting for volunteers is handled by the function emacsconf-publish-backstage-to-assign.
emacsconf-publish-backstage-to-assign
(defun emacsconf-publish-backstage-to-assign (by-status files) (let ((list (assoc-default "TO_ASSIGN" by-status))) (format "<h1>%s talk(s) to be captioned, waiting for volunteers (%d minutes)</h1><p>You can e-mail <a href=\"mailto:sacha@sachachua.com\">sacha@sachachua.com</a> to call dibs on editing the captions for one of these talks. We use OpenAI Whisper to provide auto-generated VTT that you can use as a starting point, but you can also write the captions from scratch if you like. If you're writing them from scratch, you can choose to include timing information, or we can probably figure them out afterwards with a forced alignment tool. More info: <a href=\"https://emacsconf.org/captioning/\">captioning tips</a></p><ul class=\"videos\">%s</ul>" (length list) (emacsconf-sum :video-time list) (mapconcat (lambda (f) (setq f (append f (list :extra (if (plist-get f :caption-note) (concat "<div class=\"caption-note\">" (plist-get f :caption-note) "</div>") "") :files (emacsconf-publish-talk-files f files)))) (format "<li><a name=\"%s\"></a><strong><a href=\"%s\">%s</a></strong><br />%s (id:%s)<br />%s</li>" (plist-get f :slug) (plist-get f :absolute-url) (plist-get f :title) (plist-get f :speakers) (plist-get f :slug) (emacsconf-publish-index-card f))) list "\n"))))
Each talk has a little card that includes its video and links to files.
emacsconf-publish-index-card: Format an HTML card for TALK, linking the files in EXTENSIONS.
(defun emacsconf-publish-index-card (talk &optional extensions) "Format an HTML card for TALK, linking the files in EXTENSIONS." (let* ((file-prefix (plist-get talk :file-prefix)) (video-file (plist-get talk :video-file)) (video (and file-prefix (emacsconf-publish-index-card-video (or (plist-get talk :video-id) (concat (plist-get talk :slug) "-mainVideo")) video-file talk extensions)))) ;; Add extra information to the talk (setq talk (append talk (list :time-info (emacsconf-surround "Duration: " (plist-get talk :video-duration) " minutes" "") :video-html (or (plist-get video :video) "") :audio-html (or (plist-get video :audio) "") :chapter-list (or (plist-get video :chapter-list) "") :resources (or (plist-get video :resources) "") :extra (or (plist-get talk :extra) "") :speaker-info (or (plist-get talk :speakers-with-pronouns) "")))) (emacsconf-replace-plist-in-string talk "<div class=\"vid\">${video-html}${audio-html}<div>${extra}</div>${time-info}${resources}${chapter-list}</div>")))
So it's all built on templates and a little bit of code to make the appropriate lists. You can find this code in emacsconf-publish.el.
You can comment with Disqus or you can e-mail me at sacha@sachachua.com.