EmacsConf backstage: scheduling with SVGs
| emacs, emacsconfThe EmacsConf 2023 call for participation deadline is coming up next Friday (Sept 15), so I'm getting ready to draft the schedule.
Here's a quick overview of how I experiment with schedules for EmacsConf. I have all the talk details in Org subtrees. There's a SLUG property that has the talk ID, a TIME property that says how long a talk is, and a Q_AND_A property that says what kind of Q&A the speaker wants: live, IRC, Etherpad, or after the event. Some talks have fixed starting times, like the opening remarks. Others start when the Q&A for the previous session ends. I generate an SVG so that I can quickly see how the schedule looks. Are there big gaps? Did I follow everyone's availability constraints? Does the schedule flow reasonably logically?
Code to define a schedule
(require 'emacsconf-schedule) (setq emacsconf-schedule-tracks '((:label "Saturday" :start "2023-12-02 9:00" :end "2023-12-02 18:00" :tracks ("General")) (:label "Sunday" :start "2023-12-03 9:00" :end "2023-12-03 18:00" :tracks ("General")))) (setq emacsconf-schedule-default-buffer-minutes 10 emacsconf-schedule-default-buffer-minutes-for-live-q-and-a 30 emacsconf-schedule-break-time 10 emacsconf-schedule-lunch-time 60 emacsconf-schedule-max-time 30 emacsconf-schedule-strategies '(emacsconf-schedule-allocate-buffer-time emacsconf-schedule-set-all-tracks-to-general)) (setq emacsconf-schedule-plan '(("GEN Saturday, Dec 2" :start "2023-12-02 09:00") sat-open adventure writing one uni lunch hyperdrive ref mentor flat (hn :start "15:30") (web :start "16:00") ("GEN Sunday, Dec 3" :start "2023-12-03 09:00") sun-open windows extending lunch lspocaml sharing emacsen voice llm)) (setq emacsconf-schedule-draft (emacsconf-schedule-prepare (emacsconf-schedule-inflate-sexp emacsconf-schedule-plan))) (emacsconf-schedule-validate emacsconf-schedule-draft) (let ((emacsconf-schedule-svg-modify-functions '(emacsconf-schedule-svg-color-by-status))) (with-temp-file "schedule.svg" (svg-print (emacsconf-schedule-svg 800 200 emacsconf-schedule-draft)))) (clear-image-cache)
Common functions are in emacsconf-schedule.el in the emacsconf-el repository, while conference-specific code is in our private conf.org file.
Motivation
I was working on the schedule for EmacsConf 2022 when I ran into a problem. I wanted more talks than could fit into the time that we had, even if I scheduled it as tightly as possible with just a few minutes of transition between talks. This had been the scheduling strategy we used in previous years, squishing all the talks together with just enough time to let people know where they could go to ask questions. We had experimented with an alternate track for Q&A during EmacsConf 2021, so that had given us a little more space for discussion, but it wasn't going to be enough to fit in all the talks we wanted for EmacsConf 2022.
Could I convince the other organizers to take on the extra work needed for a multi-track conference? I knew it would be a lot of extra technical risk, and we'd need another host and streamer too. Could I make the schedule easy to understand on the wiki if we had two tracks? The Org Mode table I'd been using for previous years was not going to be enough. I was getting lost in the text.
Visualizing the schedule as an SVG
I'd been playing around with SVGs in Emacs. It was pretty straightforward to write a function that used time for the X axis and displayed different tracks. That showed us how full the conference was going to be if I tried to pack everything into two days.
emacsconf-schedule-svg-track: Draw the actual rectangles and text for the talks.
(defun emacsconf-schedule-svg-track (svg base-x base-y width height start-time end-time info) "Draw the actual rectangles and text for the talks." (let ((scale (/ width (float-time (time-subtract end-time start-time))))) (mapc (lambda (o) (let* ((offset (floor (* scale (float-time (time-subtract (plist-get o :start-time) start-time))))) (size (floor (* scale (float-time (time-subtract (plist-get o :end-time) (plist-get o :start-time)))))) (x (+ base-x offset)) (y base-y) (node (dom-node 'rect (list (cons 'x x) (cons 'y y) (cons 'opacity "0.8") (cons 'width size) (cons 'height (1- height)) (cons 'stroke "black") (cons 'stroke-dasharray (if (string-match "live" (or (plist-get o :q-and-a) "live")) "" "5,5,5")) (cons 'fill (cond ((string-match "BREAK\\|LUNCH" (plist-get o :title)) "white") ((plist-get o :invalid) "red") ((string-match "EST" (or (plist-get o :availability) "")) "lightgray") (t "lightgreen")))))) (parent (dom-node 'a (list (cons 'href (concat (if emacsconf-use-absolute-url emacsconf-base-url "/") (plist-get o :url))) (cons 'title (plist-get o :title)) (cons 'data-slug (plist-get o :slug))) (dom-node 'title nil (concat (format-time-string "%l:%M-" (plist-get o :start-time) emacsconf-timezone) (format-time-string "%l:%M " (plist-get o :end-time) emacsconf-timezone) (plist-get o :title))) node (dom-node 'g `((transform . ,(format "translate(%d,%d)" (+ x size -2) (+ y height -2)))) (dom-node 'text (list (cons 'fill "black") (cons 'x 0) (cons 'y 0) (cons 'font-size 10) (cons 'transform "rotate(-90)")) (svg--encode-text (or (plist-get o :slug) (plist-get o :title)))))))) (run-hook-with-args 'emacsconf-schedule-svg-modify-functions o node parent) (dom-append-child svg parent))) info)))
emacsconf-schedule-svg-day: Add the time scale and the talks on a given day.
(defun emacsconf-schedule-svg-day (elem label width height start end tracks) "Add the time scale and the talks on a given day." (let* ((label-margin 15) (track-height (/ (- height (* 2 label-margin)) (length tracks))) (x 0) (y label-margin) (scale (/ width (float-time (time-subtract end start)))) (time start)) (dom-append-child elem (dom-node 'title nil (concat "Schedule for " label))) (svg-rectangle elem 0 0 width height :fill "white") (svg-text elem label :x 3 :y (- label-margin 3) :fill "black" :font-size "10") (mapc (lambda (track) (emacsconf-schedule-svg-track elem x y width track-height start end track) (setq y (+ y track-height))) tracks) ;; draw grid (while (time-less-p time end) (let ((x (* (float-time (time-subtract time start)) scale))) (dom-append-child elem (dom-node 'g `((transform . ,(format "translate(%d,%d)" x label-margin))) (dom-node 'line `((stroke . "darkgray") (x1 . 0) (y1 . 0) (x2 . 0) (y2 . ,(- height label-margin label-margin)))) (dom-node 'text `((fill . "black") (x . 0) (y . ,(- height 2 label-margin)) (font-size . 10) (text-anchor . "left")) (svg--encode-text (format-time-string "%-l %p" time emacsconf-timezone))))) (setq time (time-add time (seconds-to-time 3600))))) elem))
emacsconf-schedule-svg-days: Display multiple days.
(defun emacsconf-schedule-svg-days (width height days) "Display multiple days." (let ((svg (svg-create width height)) (day-height (/ height (length days))) (y 0)) (dom-append-child svg (dom-node 'title nil "Graphical view of the schedule")) (mapc (lambda (day) (let ((group (dom-node 'g `((transform . ,(format "translate(0,%d)" y)))))) (dom-append-child svg group) (emacsconf-schedule-svg-day group (plist-get day :label) width day-height (date-to-time (plist-get day :start)) (date-to-time (plist-get day :end)) (plist-get day :tracks))) (setq y (+ y day-height))) days) svg))
emacsconf-schedule-svg: Make the schedule SVG for INFO.
(defun emacsconf-schedule-svg (width height &optional info) "Make the schedule SVG for INFO." (setq info (emacsconf-prepare-for-display (or info (emacsconf-get-talk-info)))) (let ((days (seq-group-by (lambda (o) (format-time-string "%Y-%m-%d" (plist-get o :start-time) emacsconf-timezone)) (sort (seq-filter (lambda (o) (or (plist-get o :slug) (plist-get o :include-in-info))) info) #'emacsconf-sort-by-scheduled)))) (emacsconf-schedule-svg-days width height (mapcar (lambda (o) (let ((start (concat (car o) "T" emacsconf-schedule-start-time emacsconf-timezone-offset)) (end (concat (car o) "T" emacsconf-schedule-end-time emacsconf-timezone-offset))) (list :label (format-time-string "%A" (date-to-time (car o))) :start start :end end :tracks (emacsconf-by-track (cdr o))))) days))))
With that, I was able to show the other organizers what a two-track conference could look like.
Defining a schedule with two tracks
(emacsconf-schedule-test filename (emacsconf-time-constraints '(("LUNCH" "11:30" "13:30"))) (emacsconf-schedule-default-buffer-minutes 15) (emacsconf-schedule-default-buffer-minutes-for-live-q-and-a 25) (arranged (emacsconf-schedule-inflate-sexp '(("GEN Saturday, December 3" . "2022-12-03 09:00") "Saturday opening remarks" survey orgyear rolodex break links buttons lunch hyperorg realestate health break jupyter workflows ("Saturday closing remarks" . "2022-12-03 17:00") ("GEN Sunday, December 4" . "2022-12-04 09:00") "Sunday opening remarks" journalism handwritten break school science lunch meetups buddy community orgvm indieweb fanfare ("Sunday closing remarks" . "2022-12-04 17:00") ("DEV Saturday, December 3" . "2022-12-03 10:00") localizing treesitter lspbridge lunch sqlite mail eev python break wayland (haskell . "2022-12-03 16:00") ("DEV Sunday, December 4" . "2022-12-04 10:00") justl eshell detached rde lunch tramp async break asmblox dbus maint )) ) (emacsconf-schedule-break-time 10) (emacsconf-schedule-lunch-time 60) (emacsconf-schedule-max-time 30) (emacsconf-schedule-tweaked-allocations '(("indieweb" . 20) ("maint" . 20) ("workflows" . 20))) (emacsconf-scheduling-strategies '(emacsconf-schedule-override-breaks emacsconf-schedule-allocate-buffer-time )) (tracks '((:label "Saturday" :start "2022-12-03 9:00" :end "2022-12-03 18:00" :tracks (("^GEN Sat" "^GEN Sun") ("^DEV Sat" "^DEV Sun"))) (:label "Sunday" :start "2022-12-04 9:00" :end "2022-12-04 18:00" :tracks (("^GEN Sun" "^DEV Sat") ("^DEV Sun"))))))
When the other organizers saw the two schedules, they were strongly in favour of the two-track option. Yay!
Changing scheduling strategies
As I played around with the schedule, I wanted a quick way to test different scheduling strategies, such as changing the length of Q&A sessions for live web conferences versus IRC/Etherpad/email Q&A. Putting those into variables allowed me to easily override them with a let form, and I used a list of functions to calculate or modify the schedule from the Org trees.
emacsconf-schedule-allocate-buffer-time: Allocate buffer time based on whether INFO has live Q&A.
(defun emacsconf-schedule-allocate-buffer-time (info) "Allocate buffer time based on whether INFO has live Q&A. Uses `emacsconf-schedule-default-buffer-minutes' and `emacsconf-schedule-default-buffer-minutes-for-live-q-and-a'." (mapcar (lambda (o) (when (plist-get o :slug) (unless (plist-get o :buffer) (plist-put o :buffer (number-to-string (if (string-match "live" (or (plist-get o :q-and-a) "live")) emacsconf-schedule-default-buffer-minutes-for-live-q-and-a emacsconf-schedule-default-buffer-minutes))))) o) info))
This is actually applied by emacsconf-schedule-prepare, which runs through the list of strategies defined in emacsconf-schedule-strategies.
emacsconf-schedule-prepare: Apply ‘emacsconf-schedule-strategies’ to INFO to determine the schedule.
(defun emacsconf-schedule-prepare (&optional info) "Apply `emacsconf-schedule-strategies' to INFO to determine the schedule." (emacsconf-schedule-based-on-info (seq-reduce (lambda (prev val) (funcall val prev)) emacsconf-schedule-strategies (or info (emacsconf-get-talk-info)))))
Type of Q&A
I wanted to see which sessions had live Q&A via web conference and which ones had IRC/Etherpad/asynchronous Q&A. I set the talk outlines so that dashed lines mean asynchronous Q&A and solid lines mean live. Live Q&As take a little more work on our end because the host starts it up and reads questions, but they're more interactive. This is handled by
(cons 'stroke-dasharray (if (string-match "live" (or (plist-get o :q-and-a) "live")) "" "5,5,5"))
in the emacsconf-schedule-svg-track function.
Moving talks around
I wanted to be able to quickly reorganize talks by moving their IDs around in a list instead of just using the order of the subtrees in my Org Mode file. I wrote a function that took a list of symbols, looked up each of the talks, and returned a list of talk info property lists. I also added the ability to override some things about talks, such as whether something started at a fixed time.
emacsconf-schedule-inflate-sexp: Takes a list of talk IDs and returns a list that includes the scheduling info.
(defun emacsconf-schedule-inflate-sexp (sequence &optional info include-time) "Takes a list of talk IDs and returns a list that includes the scheduling info. Pairs with `emacsconf-schedule-dump-sexp'." (setq info (or info (emacsconf-get-talk-info))) (let ((by-assoc (mapcar (lambda (o) (cons (intern (plist-get o :slug)) o)) (emacsconf-filter-talks info))) date) (mapcar (lambda (seq) (unless (listp seq) (setq seq (list seq))) (if include-time (error "Not yet implemented") (let ((start-prop (or (plist-get (cdr seq) :start) (and (stringp (cdr seq)) (cdr seq)))) (time-prop (or (plist-get (cdr seq) :time) ; this is duration in minutes (and (numberp (cdr seq)) (cdr seq)))) (track-prop (plist-get (cdr seq) :track))) (append ;; overriding (when start-prop (if (string-match "-" start-prop) (setq date (format-time-string "%Y-%m-%d" (date-to-time start-prop))) (setq start-prop (concat date " " start-prop))) (list :scheduled (format-time-string (cdr org-time-stamp-formats) (date-to-time start-prop) emacsconf-timezone) :start-time (date-to-time start-prop) :fixed-time t)) (when track-prop (list :track track-prop)) (when time-prop (list :time (if (numberp time-prop) (number-to-string time-prop) time-prop))) ;; base entity (cond ((eq (car seq) 'lunch) (list :title "LUNCH" :time (number-to-string emacsconf-schedule-lunch-time))) ((eq (car seq) 'break) (list :title "BREAK" :time (number-to-string emacsconf-schedule-break-time))) ((symbolp (car seq)) (assoc-default (car seq) by-assoc)) ((stringp (car seq)) (or (seq-find (lambda (o) (string= (plist-get o :title) (car seq))) info) (list :title (car seq)))) (t (error "Unknown %s" (prin1-to-string seq)))))))) sequence)))
That allowed me to specify a test schedule like this:
(setq emacsconf-schedule-plan '(("GEN Saturday, Dec 2" :start "2023-12-02 09:00") sat-open adventure writing one uni lunch hyperdrive ref mentor flat (hn :start "15:30") (web :start "16:00") ("GEN Sunday, Dec 3" :start "2023-12-03 09:00") sun-open windows extending lunch lspocaml sharing emacsen voice llm)) (setq emacsconf-schedule-draft (emacsconf-schedule-prepare (emacsconf-schedule-inflate-sexp emacsconf-schedule-plan)))
Validating the schedule
Now that I could see the schedule visually, it made sense to add some validation. As mentioned in my post about timezones, I wanted to validate live Q&A against the speaker's availability and colour sessions red if they were outside the times I'd written down.
I also wanted to arrange the schedule so that live Q&A sessions didn't start at the same time, giving me a little time to switch between sessions in case I needed to help out. I wrote a function that checked if the Q&A for a session started within five minutes of the previous one. This turned out to be pretty useful, since I ended up mostly taking care of playing the videos and opening the browsers for both streams. With that in place, I could just move the talks around until everything fit well together.
emacsconf-schedule-validate-live-q-and-a-sessions-are-staggered: Try to avoid overlapping the start of live Q&A sessions.
(defun emacsconf-schedule-validate-live-q-and-a-sessions-are-staggered (schedule) "Try to avoid overlapping the start of live Q&A sessions. Return nil if there are no errors." (when emacsconf-schedule-validate-live-q-and-a-sessions-buffer (let (last-end) (delq nil (mapcar (lambda (o) (prog1 (when (and last-end (time-less-p (plist-get o :end-time) (time-add last-end (seconds-to-time (* emacsconf-schedule-validate-live-q-and-a-sessions-buffer 60))))) (plist-put o :invalid (format "%s live Q&A starts at %s within %d minutes of previous live Q&A at %s" (plist-get o :slug) (format-time-string "%m-%d %-l:%M" (plist-get o :end-time)) emacsconf-schedule-validate-live-q-and-a-sessions-buffer (format-time-string "%m-%d %-l:%M" last-end))) (plist-get o :invalid)) (setq last-end (plist-get o :end-time)))) (sort (seq-filter (lambda (o) (string-match "live" (or (plist-get o :q-and-a) ""))) schedule) (lambda (a b) (time-less-p (plist-get a :end-time) (plist-get b :end-time))) ))))))
Publishing SVGs in the wiki
When the schedule settled down, it made perfect sense to include the image on the schedule page as well as on each talk page. I wanted people to be able to click on a rectangle and load the talk page. That meant including the SVG in the markup and allowing the attributes in ikiwiki's HTML sanitizer. In our htmlscrubber.pm, I needed to add svg rect text g title line to allow, and add version xmlns x y x1 y1 x2 y2 fill font-size font-weight stroke stroke-width stroke-dasharray transform opacity to the default attributes.
For the public-facing pages, I wanted to colour the talks based on
track. I specified the colours in emacsconf-tracks (peachpuff and
skyblue) and used them in emacsconf-schedule-svg-color-by-track.
emacsconf-schedule-svg-color-by-track: Color sessions based on track.
(defun emacsconf-schedule-svg-color-by-track (o node &optional parent) "Color sessions based on track." (let ((track (emacsconf-get-track (plist-get o :track)))) (when track (dom-set-attribute node 'fill (plist-get track :color)))))
I wanted talk pages to highlight the talk on the schedule so that people could easily find other sessions that conflict. Because a number of people in the Emacs community browse with Javascript turned off, I used Emacs Lisp to generate a copy of the SVG with the current talk highlighted.
emacsconf-publish-format-talk-page-schedule: Add the schedule image for TALK based on INFO.
(defun emacsconf-publish-format-talk-page-schedule (talk info) "Add the schedule image for TALK based on INFO." (concat "\nThe following image shows where the talk is in the schedule for " (format-time-string "%a %Y-%m-%d" (plist-get talk :start-time) emacsconf-timezone) ". Solid lines show talks with Q&A via BigBlueButton. Dashed lines show talks with Q&A via IRC or Etherpad." (format "<div class=\"schedule-in-context schedule-svg-container\" data-slug=\"%s\">\n" (plist-get talk :slug)) (let* ((width 800) (height 150) (talk-date (format-time-string "%Y-%m-%d" (plist-get talk :start-time) emacsconf-timezone)) (start (date-to-time (concat talk-date "T" emacsconf-schedule-start-time emacsconf-timezone-offset))) (end (date-to-time (concat talk-date "T" emacsconf-schedule-end-time emacsconf-timezone-offset))) svg) (with-temp-buffer (setq svg (emacsconf-schedule-svg-day (svg-create width height) (format-time-string "%A" (plist-get talk :start-time) emacsconf-timezone) width height start end (emacsconf-by-track (seq-filter (lambda (o) (string= (format-time-string "%Y-%m-%d" (plist-get o :start-time) emacsconf-timezone) talk-date)) info)))) (mapc (lambda (node) (let ((rect (car (dom-by-tag node 'rect)))) (if (string= (dom-attr node 'data-slug) (plist-get talk :slug)) (progn (dom-set-attribute rect 'opacity "0.8") (dom-set-attribute rect 'stroke-width "3") (dom-set-attribute (car (dom-by-tag node 'text)) 'font-weight "bold")) (dom-set-attribute rect 'opacity "0.5")))) (dom-by-tag svg 'a)) (svg-print svg) (buffer-string))) "\n</div>\n" "\n" (emacsconf-format-talk-schedule-info talk) "\n\n"))
Colouring talks based on status
As the conference approached, I wanted to colour talks based on their status, so I could see how much of our schedule already had videos and even how many had been captioned. That was just a matter of adding a modifier function to change the SVG rectangle colour depending on the TODO status.
emacsconf-schedule-svg-color-by-status: Set talk color based on status.
(defun emacsconf-schedule-svg-color-by-status (o node &optional _) "Set talk color based on status. Processing: palegoldenrod, Waiting to be assigned a captioner: yellow, Captioning in progress: lightgreen, Ready to stream: green, Other status: gray" (unless (plist-get o :invalid) (dom-set-attribute node 'fill (pcase (plist-get o :status) ((rx (or "TO_PROCESS" "PROCESSING" "TO_AUTOCAP")) "palegoldenrod") ((rx (or "TO_ASSIGN")) "yellow") ((rx (or "TO_CAPTION")) "lightgreen") ((rx (or "TO_STREAM")) "green") (_ "gray")))))
Summary
Common functions are in emacsconf-schedule.el in the emacsconf-el repository, while conference-specific code is in our private conf.org file.
Some information is much easier to work with graphically than in plain text. Seeing the schedules as images made it easier to spot gaps or errors, tinker with parameters, and communicate with other people. Writing Emacs Lisp functions to modify my data made it easier to try out different things without rearranging the text in my Org file. Because Emacs can display images inside the editor, it was easy for me to make changes and see the images update right away. Using SVG also made it possible to export the image and make it interactive. Handy technique!