#EmacsConf backstage: adding a talk to the wiki

Sep 26, 2023| emacsconf, emacs

The EmacsConf 2023 call for participation has finished, hooray! We've sent out acceptances and added talks to the wiki. We experimented with early acceptances this year, which was really nice because it gives people quick feedback and allows people to get started on their videos early. That meant that I needed to be able to easily add talks to the wiki throughout the call for participation. We use templates and an Ikiwiki directive to make it easier to consistently format talk pages. This post covers adding a talk to the wiki, and these functions are in the emacsconf-el repository.

Overview

Amin Bandali picked Ikiwiki for the wiki for emacsconf.org. I think it's because Ikiwiki works with plain text in a Git repository, which fits nicely with our workflow. I can use Emacs Lisp to generate files that are included in other files, and I can also use Emacs Lisp to generate the starting point for files that may be manually edited later on.

We organize conference pages by year. Under the directory for this year (2023/), we have:

talks

talk descriptions that can be manually edited

talks/SLUG.md: the description for the talk. Includes ../info/SLUG-nav.md, ../info/SLUG-before.md, and ../info/SLUG-after.md

info

automatically-generated files that are included before and after the talk description, and navigation links between talks

info/SLUG-nav.md: navigation links
info/SLUG-before.md: navigation links
info/SLUG-after.md: navigation links

schedule-details.md

list of talks

organizers-notebook/index.org

public planning notebook

The filenames and URLs for each talk are based on the ID for a talk. I store that in the SLUG property to make it easy to differentiate from CUSTOM_ID, since CUSTOM_ID is useful for lots of other things in Org Mode. I usually assign the slugs when I add the talks to our private conf.org file, although sometimes people suggest a specific ID.

Templating

Publishing to wiki pages and replying to e-mails are easier if I can substitute text into readable templates. There are a number of templating functions for Emacs Lisp, like the built-in tempo.el or s-lex-format from s.el. I ended up writing something that works with plists instead, since we use property lists (plists) all over the emacsconf-el library.

emacsconf-replace-plist-in-string: Replace ${keyword} from ATTRS in STRING.

(defun emacsconf-replace-plist-in-string (attrs string)
  "Replace ${keyword} from ATTRS in STRING."
  (let ((a attrs) name val)
    (while a
      (setq name (pop a) val (pop a))
      (when (stringp val)
        (setq string
              (replace-regexp-in-string (regexp-quote (concat "${" (substring (symbol-name name) 1) "}"))
                                        (or val "")
                                        string t t))))
    string))

It is also handy to be able to add text around another string only if the string is non-nil, and to provide a different string to use if it isn't specified..

emacsconf-surround: Concat BEFORE, TEXT, and AFTER if TEXT is specified, or return ALTERNATIVE.

(defun emacsconf-surround (before text after &optional alternative)
  "Concat BEFORE, TEXT, and AFTER if TEXT is specified, or return ALTERNATIVE."
  (if (and text (not (string= text "")))
      (concat (or before "") text (or after ""))
    alternative))

Getting the talk information

To get the data to fill in the template, we can run a bunch of different functions. This lets us add or remove functions when we need to. We pass the previous result to the next function in order to accumulate properties or modify them. The result is a property list for the current talk.

emacsconf-get-talk-info-for-subtree: Run ‘emacsconf-talk-info-functions’ to extract the info for this entry.

(defun emacsconf-get-talk-info-for-subtree ()
  "Run `emacsconf-talk-info-functions' to extract the info for this entry."
  (seq-reduce (lambda (prev val) (save-excursion (save-restriction (funcall val prev))))
              emacsconf-talk-info-functions
              nil))

emacsconf-talk-info-functions: Functions to collect information.

(defvar emacsconf-talk-info-functions
  '(emacsconf-get-talk-info-from-properties
    emacsconf-get-talk-categories
    emacsconf-get-talk-abstract-from-subtree
    emacsconf-get-talk-logbook
    emacsconf-add-talk-status
    emacsconf-add-checkin-time
    emacsconf-add-timezone-conversions
    emacsconf-add-speakers-with-pronouns
    emacsconf-add-live-info)
  "Functions to collect information.")

Getting the talk abstract

I add a *** Talk abstract subheading to the talk and put the rest of the submission under a *** Talk details subheading. This allows me to extract the text of the Talk abstract heading (or whatever matches emacsconf-abstract-heading-regexp, which is set to "abstract".).

emacsconf-get-subtree-entry: Return the text for the subtree matching HEADING-REGEXP.

(defun emacsconf-get-subtree-entry (heading-regexp)
  "Return the text for the subtree matching HEADING-REGEXP."
  (car
   (delq nil
         (org-map-entries
          (lambda ()
            (when (string-match heading-regexp (org-entry-get (point) "ITEM"))
              (org-get-entry)))
          nil 'tree))))

emacsconf-get-talk-abstract-from-subtree: Add the abstract from a subheading.

(defun emacsconf-get-talk-abstract-from-subtree (o)
  "Add the abstract from a subheading.
The subheading should match `emacsconf-abstract-heading-regexp'."
  (plist-put o :abstract (substring-no-properties (or (emacsconf-get-subtree-entry "abstract") ""))))

I include emacsconf-get-talk-abstract-from-subtree in emacsconf-talk-info-functions so that it retrieves that information when I call emacsconf-get-talk-info-for-subtree.

Publishing the talk page

We add accepted talks to the wiki so that people can see what kinds of talks will be at EmacsConf 2023. To add the talk to the wiki, I use emacsconf-publish-add-talk. It'll create the talk page without overwriting anything that's already there and redo the automatically-generated info pages that provide navigation, status, and so on.

emacsconf-publish-add-talk: Add the current talk to the wiki.

(defun emacsconf-publish-add-talk ()
  "Add the current talk to the wiki."
  (interactive)
  (emacsconf-publish-talk-page (emacsconf-get-talk-info-for-subtree))
  (emacsconf-publish-info-pages)
  (magit-status-setup-buffer emacsconf-directory))

The talk page includes the description and other resources.

emacsconf-publish-talk-page: Draft the talk page for O unless the page already exists or FORCE is non-nil.

(defun emacsconf-publish-talk-page (o &optional force)
  "Draft the talk page for O unless the page already exists or FORCE is non-nil."
  (interactive (list (emacsconf-get-talk-info-for-subtree)
                     (> (prefix-numeric-value current-prefix-arg) 1)))
  (let ((filename (expand-file-name (format "%s.md" (plist-get o :slug))
                                    (expand-file-name "talks" (expand-file-name emacsconf-year emacsconf-directory)))))
    (unless (file-directory-p (expand-file-name "talks" (expand-file-name emacsconf-year emacsconf-directory)))
      (mkdir (expand-file-name "talks" (expand-file-name emacsconf-year emacsconf-directory))))
    (when (or force (null (file-exists-p filename)))
      (with-temp-file filename
        (insert
         (emacsconf-replace-plist-in-string
          (emacsconf-convert-talk-abstract-to-markdown
           (append o (list
                      :speaker-info (emacsconf-publish-format-speaker-info o)
                      :meta "!meta"
                      :categories (if (plist-get o :categories)
                                      (mapconcat (lambda (o) (format "[[!taglink %s]]" o))
                                                 (plist-get o :categories)
                                                 " ")
                                    ""))))
          "[[${meta} title=\"${title}\"]]
[[${meta} copyright=\"Copyright &copy; ${year} ${speakers}\"]]
[[!inline pages=\"internal(${year}/info/${slug}-nav)\" raw=\"yes\"]]

<!-- Initially generated with emacsconf-publish-talk-page and then left alone for manual editing -->
<!-- You can manually edit this file to update the abstract, add links, etc. --->\n

# ${title}
${speaker-info}

[[!inline pages=\"internal(${year}/info/${slug}-before)\" raw=\"yes\"]]

${abstract-md}

[[!inline pages=\"internal(${year}/info/${slug}-after)\" raw=\"yes\"]]

[[!inline pages=\"internal(${year}/info/${slug}-nav)\" raw=\"yes\"]]

${categories}
"))))))

Ikiwiki uses Markdown, so we can take advantage of Org's Markdown export.

emacsconf-convert-talk-abstract-to-markdown: Set the :abstract-md property to a Markdown version of the abstract.

(defun emacsconf-convert-talk-abstract-to-markdown (o)
  "Set the :abstract-md property to a Markdown version of the abstract."
  (plist-put o :abstract-md (org-export-string-as (or (plist-get o :abstract) "") 'md t)))

Publishing the list of talks

The list of talks at https://emacsconf.org/2023/talks/ is grouped by track.

emacsconf-publish-schedule: Generate the schedule or program.

(defun emacsconf-publish-schedule (&optional info)
  "Generate the schedule or program."
  (interactive)
  (emacsconf-publish-schedule-svg-snippets)
  (setq info (or info (emacsconf-publish-prepare-for-display info)))
  (with-temp-file (expand-file-name "schedule-details.md"
                                    (expand-file-name emacsconf-year emacsconf-directory))
    (insert
     (if (member emacsconf-publishing-phase '(cfp program))
         (let ((sorted (emacsconf-publish-prepare-for-display (or info (emacsconf-get-talk-info)))))
           (mapconcat
            (lambda (track)
              (concat
               "Jump to: "
               ;; links to other tracks
               (string-join (seq-keep (lambda (track-link)
                                        (unless (string= (plist-get track-link :id)
                                                         (plist-get track :id))
                                          (format "<a href=\"#%s\">%s</a>"
                                                  (plist-get track-link :id)
                                                  (plist-get track-link :name))))
                                      emacsconf-tracks)
                            " | ")
               "\n\n"
               (let ((track-talks (seq-filter (lambda (o) (string= (plist-get o :track)
                                                                   (plist-get track :name)))
                                              sorted)))
                 (format
                  "<h1 id=\"%s\" class=\"sched-track %s\">%s (%d talks)</h1>\n%s"
                  (plist-get track :id)
                  (plist-get track :name)
                  (plist-get track :name)
                  (length track-talks)
                  (emacsconf-publish-format-main-schedule track-talks)))))
            emacsconf-tracks "\n\n"))
       (emacsconf-publish-format-interleaved-schedule info))))
  (when (member emacsconf-publishing-phase '(cfp program))
    (with-temp-file (expand-file-name
                     "draft-schedule.md"
                     (expand-file-name emacsconf-year emacsconf-directory))
      (insert
       "[[!sidebar content=\"\"]]\n\n" 
       "This is a *DRAFT* schedule.\n"
       (let ((emacsconf-publishing-phase 'schedule))
         (emacsconf-publish-format-interleaved-schedule info))))))

The emacsconf-format-main-schedule function displays the information for the talks in each track. It's pretty straightforward, but I put it in a function because I call it from a number of places.

emacsconf-publish-format-main-schedule: Include the schedule information for INFO.

(defun emacsconf-publish-format-main-schedule (info)
  "Include the schedule information for INFO."
  (mapconcat #'emacsconf-publish-sched-directive info "\n"))

We define an Ikiwiki sched directive that conditionally displays things depending on what we specify, so it's easy to add more information during the schedule or conference phase. This is templates/sched.md in the EmacsConf wiki git repository:

<div data-start="<TMPL_VAR startutc>" data-end="<TMPL_VAR endutc>" class="sched-entry <TMPL_IF track>track-<TMPL_VAR track></TMPL_IF track>">
<div class="sched-meta">
<TMPL_IF start>
<span class="sched-time"><span class="sched-start"><TMPL_VAR start></span>
<TMPL_IF end> - <span class="sched-end"><TMPL_VAR end></span></TMPL_IF end>
</span></TMPL_IF start>
<TMPL_IF track> <span class="sched-track <TMPL_VAR track>"><TMPL_IF watch><a href="<TMPL_VAR watch>"></TMPL_IF><TMPL_VAR track><TMPL_IF watch></a></TMPL_IF></span></TMPL_IF track>
<TMPL_IF pad> <span class="sched-pad"><a href="<TMPL_VAR pad>">Etherpad</a></TMPL_IF pad>
<TMPL_IF q-and-a> <span class="sched-q-and-a">Q&amp;A: <TMPL_VAR q-and-a></span> </TMPL_IF q-and-a>
</div>
<div class="sched-title"><a href="<TMPL_VAR url>"><TMPL_VAR title></a></div>
<div class="sched-speakers"><TMPL_VAR speakers> <TMPL_IF note>- <TMPL_VAR note></TMPL_IF note></div>
<TMPL_IF resources>
<ul class="resources">
<TMPL_VAR resources>
</ul>
</TMPL_IF resources>
<TMPL_IF time><span class="sched-duration><TMPL_VAR time></span> minutes</TMPL_IF time>
<TMPL_IF slug> <span class="sched-slug">id:<TMPL_VAR slug></span></TMPL_IF slug>
</div>

This Emacs Lisp function converts a talk into that directive.

emacsconf-publish-sched-directive: Format the schedule directive with info for O.

(defun emacsconf-publish-sched-directive (o)
  "Format the schedule directive with info for O."
  (format "[[!template id=sched%s]]"
          (let ((result "")
                (attrs (append
                        (pcase emacsconf-publishing-phase
                          ('program
                           (list
                            :time (plist-get o :time)))
                          ((or 'schedule 'conference)
                           (list
                            :status (pcase (plist-get o :status)
                                      ("CAPTIONED" "captioned")
                                      ("PREREC_RECEIVED" "received")
                                      ("DONE" "done")
                                      ("STARTED" "now playing")
                                      (_ nil))
                            :time (plist-get o :time)
                            :q-and-a (plist-get o :qa-link) 
                            :pad (and emacsconf-publish-include-pads (plist-get o :pad-url))
                            :startutc (format-time-string "%FT%T%z" (plist-get o :start-time) t)
                            :endutc (format-time-string "%FT%T%z" (plist-get o :end-time) t)
                            :start (format-time-string "%-l:%M" (plist-get o :start-time) emacsconf-timezone)
                            :end (format-time-string "%-l:%M" (plist-get o :end-time) emacsconf-timezone)))
                          ('resources
                           (list
                            :pad nil
                            :channel nil
                            :resources (mapconcat (lambda (s) (concat "<li>" s "</li>"))
                                                  (emacsconf-link-file-formats-as-list
                                                   (append o
                                                           (list :base-url (format "%s%s/" emacsconf-media-base-url emacsconf-year)))
                                                   (append emacsconf-main-extensions (list "--answers.webm" "--answers.opus" "--answers.vtt")))
                                                  ""))))
                        (list
                         :title (plist-get o :title)
                         :url (concat "/" (plist-get o :url))
                         :speakers (plist-get o :speakers)
                         :track (if (member emacsconf-publishing-phase '(schedule conference)) (plist-get o :track))
                         :watch (plist-get o :watch-url)
                         :slug (plist-get o :slug)
                         :note
                         (string-join
                          (delq nil
                                (list
                                 (when (plist-get o :captions-edited)
                                   "captioned")
                                 (when (and (plist-get o :public)
                                            (or (plist-get o :toobnix-url)
                                                (plist-get o :video-file)))
                                   "video posted")))
                          ", ")
                         )
                        )))
            (while attrs
              (let ((field (pop attrs))
                    (val (pop attrs)))
                (when val
                  (setq result (concat result " " (substring (symbol-name field) 1) "=\"\"\"" val "\"\"\"")))))
            result)))

Publishing auto-generated navigation

It's nice to be able to navigate between talks without going back to the schedule page each time. This is handled by just keeping two extra copies of the list: one with the first talk popped off, and one with an extra element added to the beginning. Then we can use the heads of those lists for next/previous links.

emacsconf-publish-nav-pages: Generate links to the next and previous talks.

(defun emacsconf-publish-nav-pages (&optional talks)
  "Generate links to the next and previous talks.
During the schedule and conference phase, the talks are sorted by time.
Otherwise, they're sorted by track and then schedule."
  (interactive (list (emacsconf-publish-prepare-for-display (or emacsconf-schedule-draft (emacsconf-get-talk-info)))))
  (let* ((next-talks (cdr talks))
         (prev-talks (cons nil talks))
         (label (if (member emacsconf-publishing-phase '(schedule conference))
                    "time"
                  "track")))
    (unless (file-directory-p (expand-file-name "info" (expand-file-name emacsconf-year emacsconf-directory)))
      (mkdir (expand-file-name "info" (expand-file-name emacsconf-year emacsconf-directory))))
    (while talks
      (let* ((o (pop talks))
             (next-talk (emacsconf-format-talk-link (pop next-talks)))
             (prev-talk (emacsconf-format-talk-link (pop prev-talks))))
        (with-temp-file (expand-file-name (format "%s-nav.md" (plist-get o :slug))
                                          (expand-file-name "info" (expand-file-name emacsconf-year emacsconf-directory)))
          (insert (concat "\n<div class=\"talk-nav\">
Back to the [[talks]]  \n"
                          (if prev-talk (format "Previous by %s: %s  \n" label prev-talk) "")
                          (if next-talk (format "Next by %s: %s  \n" label next-talk) "")
                          (if (plist-get o :track) ; tagging doesn't work here because ikiwiki will list the nav page
                              (format "Track: <span class=\"sched-track %s\">%s</span>  \n" (plist-get o :track) (plist-get o :track))
                            "")
                          "</div>
")))))))

Before the talk description

We include some details about the schedule in the talk page, before the description.

emacsconf-publish-before-page: Generate the page that has the info included before the abstract.

(defun emacsconf-publish-before-page (talk &optional info)
  "Generate the page that has the info included before the abstract.
This includes the intro note, the schedule, and talk resources."
  (interactive (list (emacsconf-complete-talk-info)))
  (setq info (or info (emacsconf-publish-prepare-for-display (emacsconf-get-talk-info))))
  (with-temp-file (expand-file-name (format "%s-before.md" (plist-get talk :slug))
                                    (expand-file-name "info" (expand-file-name emacsconf-year emacsconf-directory)))
    
    (insert "<!-- Automatically generated by emacsconf-publish-before-page -->\n")
    (insert (emacsconf-surround "" (plist-get talk :intro-note) "\n\n" ""))
    (let ((is-live (emacsconf-talk-live-p talk)))
      (when is-live (emacsconf-publish-captions-in-wiki talk))
      (when (member emacsconf-publishing-phase '(schedule conference))
        (insert (emacsconf-publish-format-talk-schedule-image talk info)))
      (insert (emacsconf-publish-format-talk-schedule-info talk) "\n\n")
      (insert
       (if (plist-get talk :public) (emacsconf-wiki-talk-resources talk) "")
       "\n# Description\n"))
    (insert "<!-- End of emacsconf-publish-before-page -->")))

emacsconf-publish-format-talk-schedule-info: Format schedule information for O.

(defun emacsconf-publish-format-talk-schedule-info (o)
  "Format schedule information for O."
  (let ((friendly (concat "/" emacsconf-year "/talks/" (plist-get o :slug) ))
        (timestamp (org-timestamp-from-string (plist-get o :scheduled))))
    (emacsconf-replace-plist-in-string
     (append o
             (list
              :format
              (concat (or (plist-get o :video-time)
                          (plist-get o :time))
                      "-min talk"
                      (if (plist-get o :q-and-a)
                          (format " followed by %s Q&A%s"
                                  (plist-get o :q-and-a)
                                  (if (eq emacsconf-publishing-phase 'conference)
                                      (format " (%s)"
                                              (if (string-match "live" (plist-get o :q-and-a))
                                                  (if (eq 'after (emacsconf-bbb-status o))
                                                      "done"
                                                    (format "<https://emacsconf.org/current/%s/room>" (plist-get o :slug)))
                                                (emacsconf-publish-webchat-link o)))
                                    ""))
                        ""))
              :pad-info
              (if emacsconf-publish-include-pads
                  (format "Etherpad: <https://pad.emacsconf.org/%s-%s>  \n" emacsconf-year (plist-get o :slug))
                "")
              :irc-info
              (format "Discuss on IRC: [#%s](%s)  \n" (plist-get o :channel)
                      (plist-get o :webchat-url))
              :status-info
              (if (member emacsconf-publishing-phase '(cfp program schedule conference)) (format "Status: %s  \n" (plist-get o :status-label)) "")
              :schedule-info
              (if (and (member emacsconf-publishing-phase '(schedule conference))
                       (not (emacsconf-talk-all-done-p o))
                       (not (string= (plist-get o :status) "CANCELLED")))
                  (let ((start (org-timestamp-to-time (org-timestamp-split-range timestamp)))
                        (end (org-timestamp-to-time (org-timestamp-split-range timestamp t))))
                    (format
                     "<div>Times in different timezones:</div><div class=\"times\" start=\"%s\" end=\"%s\"><div class=\"conf-time\">%s</div><div class=\"others\"><div>which is the same as:</div>%s</div></div><div><a href=\"/%s/watch/%s/\">Find out how to watch and participate</a></div>"
                     (format-time-string "%Y-%m-%dT%H:%M:%SZ" start t)
                     (format-time-string "%Y-%m-%dT%H:%M:%SZ" end t)
                     (emacsconf-timezone-string o emacsconf-timezone)
                     (string-join (emacsconf-timezone-strings
                                   o
                                   (seq-remove (lambda (zone) (string= emacsconf-timezone zone))
                                               emacsconf-timezones)) "<br />")
                     emacsconf-year
                     (plist-get (emacsconf-get-track (plist-get o :track)) :id)))
                "")))
     (concat
      "[[!toc  ]]
Format: ${format}  
${pad-info}${irc-info}${status-info}${schedule-info}\n" 
      (if (plist-get o :alternate-apac)
          (format "[[!inline pages=\"internal(%s/inline-alternate)\" raw=\"yes\"]]  \n" emacsconf-year)
        "")
      "\n"))))

After the talk description

After the talk description, we include a footer that makes it easier for people to e-mail questions using either the PUBLIC_EMAIL property of the talk or the emacsconf-org-private e-mail address.

emacsconf-publish-format-email-questions-and-comments: Invite people to e-mail either the public contact for TALK or the private list.

(defun emacsconf-publish-format-email-questions-and-comments (talk)
  "Invite people to e-mail either the public contact for TALK or the private list."
  (format "Questions or comments? Please e-mail %s"
          (emacsconf-publish-format-public-email talk
                                         (or
                                          (and (string= (plist-get talk :public-email) "t")
                                               (plist-get talk :email))
                                          (plist-get talk :public-email)
                                          "emacsconf-org-private@gnu.org"))))

emacsconf-publish-after-page: Generate the page with info included after the abstract.

(defun emacsconf-publish-after-page (talk &optional info)
  "Generate the page with info included after the abstract.
This includes captions, contact, and an invitation to participate."
  (interactive (list (emacsconf-complete-talk-info)))
  ;; Contact information
  (with-temp-file (expand-file-name (format "%s-after.md" (plist-get talk :slug))
                                    (expand-file-name "info" (expand-file-name emacsconf-year emacsconf-directory)))
    (insert
     "<!-- Automatically generated by emacsconf-publish-after-page -->\n"
     "\n\n"
     ;; main transcript
     (if (plist-get talk :public) (emacsconf-publish-format-captions talk) "")
     (emacsconf-publish-format-email-questions-and-comments talk) "\n"
     (if (eq emacsconf-publishing-phase 'cfp)
         (format "\n----\nGot an idea for an EmacsConf talk or session? We'd love to hear from you! Check out the [[Call for Participation|/%s/cfp]] for details.\n" emacsconf-year)
       "")
     "\n\n<!-- End of emacsconf-publish-after-page -->\n")))

Whenever the schedule changes

This function makes it easier to regenerate all those dynamic pages that need to be updated whenever the schedule changes.

emacsconf-publish-info-pages: Populate year/info/*-nav, -before, and -after files.

(defun emacsconf-publish-info-pages (&optional info)
  "Populate year/info/*-nav, -before, and -after files."
  (interactive (list nil))
  (setq info (or info (emacsconf-publish-prepare-for-display info)))
  (emacsconf-publish-with-wiki-change
    (emacsconf-publish-nav-pages info)
    (emacsconf-publish-schedule info)
    (mapc (lambda (o)
            (emacsconf-publish-before-page o info)
            (emacsconf-publish-after-page o info))
          info)))

Summary

So once the review period has passed and we're ready to accept the talk, I change the status to WAITING_FOR_PREREC and find room for it in the schedule. Then I use emacsconf-publish-add-talk to add the talk description to the wiki. I review the files it generated, tweak hyperlinks as needed, add the pages to the Git repository, and push the commit to the server. If I rearrange talks or change times, I just need to run emacsconf-publish-info-pages and all the dynamically-generated pages will be updated.

You can comment with Disqus or you can e-mail me at sacha@sachachua.com.