Categories: emacsconf

View topic page - RSS - Atom - Subscribe via email

EmacsConf 2025 notes

| emacs, emacsconf

Intended audience: This is a long post (~ 8,400 words). It's mostly for me, but I hope it might also be interesting if you like to use Emacs to do complicated things or if you also happen to organize online conferences.

: Added a note about streaming from virtual desktops.

The videos have been uploaded and thank-you notes have been sent, so now I get to write quick notes on EmacsConf 2025 and reuse some of my code from last year's post.

Stats

While organizing EmacsConf 2025, I thought it was going to be a smaller conference compared to last year because of lots of last-minute cancellations. Now that I can finally add things up, I see how it all worked out:

2024 2025 Type
31 25 Presentations
10.7 11.3 Presentation duration (hours)
21 11 Q&A web conferences
7.8 5.2 Q&A duration (hours)
18.5 16.5 Total

EmacsConf 2025 was actually a little longer than 2024 in total presentation time, although that's probably because we had more live talks which included answering questions. It was almost as long overall including live Q&A in BigBlueButton rooms, but we did end an hour or so earlier each day.

Looking at the livestreaming data, I see that we had fewer participants compared to the previous year. Here are the stats from Icecast, the program we use for streaming:

  • Saturday:
    • gen: 107 peak + 7 lowres (compared to 191 in 2024)
    • dev: 97 peak + 7 lowres (compared to 305 in 2024)
  • Sunday: I forgot to copy Sunday stats, whoops! I think there were about 70 people on the general stream. Idea: Automate this next time.

The YouTube livestream also had fewer participants at the time of the stream, but that's okay. Here are the stats from YouTube:

2024 peak 2025 peak YouTube livestream
46 23 Gen Sat AM
24 7 Gen Sat PM
15 8 Dev Sat AM
20 14 Dev Sat PM
28 14 Gen Sun AM
26 11 Gen Sun PM

Fewer people attended compared to last year, but it's still an amazing get-together from the perspective of being able to get a bunch of Emacs geeks in a (virtual) room. People asked a lot of questions over IRC and the Etherpads, and the speakers shared lots of extra details that we captured in the Q&A sessions. I'm so glad people were able to connect with each other.

Based on the e-mails I got from speakers about their presentations and the regrets from people who couldn't make it to EmacsConf, it seemed that people were a lot busier in 2025 compared to 2024. There were also a lot more stressors in people's lives. But it was still a good get-together, and it'll continue to be useful going forward.

At the moment, the EmacsConf 2025 videos have about 20k views total on YouTube. (media.emacsconf.org doesn't do any tracking.) Here are the most popular ones so far:

While I was looking at the viewing stats on YouTube, I noticed that people are still looking at videos all the way back to 2013 (like Emacs Live - Sam Aaron and Emacs Lisp Development - John Wiegley), and many videos have thousands of views. Here are some of the most popular ones from past conferences:

Views aren't everything, of course, but maybe they let us imagine a little about how many people these speakers might have been able to reach. How wonderful it is that people can spend a few days putting together their insights and then have that resonate with other people through time. Speakers are giving us long-term gifts.

Timeline

Of course, the process of preparing all of that started long before the days of the conference. We posted the call for proposals towards the end of June, like we usually do, because we wanted to give people plenty of time to start thinking about their presentations. We did early acceptances again this year, and we basically accepted everything, so people could start working on their videos almost right away. Here's the general timeline:

CFP 2025-06-27 Fri
CFP deadline 2025-09-19 Fri
Speaker notifications 2025-09-26 Fri
Publish schedule 2025-10-24 Fri (oops, forgot to do this elsewhere)
Video submission target date 2025-10-31 Fri
EmacsConf 2025-12-06 Sat - 2025-12-07 Sun
Live talks, Q&A videos posted 2025-12-17
Q&A notes posted 2025-12-28
These notes 2026-01-01
submissions_plot.png
Figure 1: EmacsConf 2025: cumulative proposals and video uploads

This graph shows that we got more proposals earlier this year (solid blue line: 2025) compared to last year (gray: 2024), although there were fewer last-minute ones and more cancellations this year. Some people were very organized. (Video submissions in August!) Some people sent theirs in later because they first had to figure out all the details of what they proposed, which is totally relatable for anyone who's found themselves shaving Emacs yaks before.

I really appreciated the code that I wrote to create SVG previews of the schedule. That made it much easier to see how the schedule changed as I added or removed talks. I started stressing out about the gap between the proposals and the uploaded videos (orange) in November. Compared to last year, the submissions slowly trickled in. The size of the gap between the blue line (cumulative proposals) and the orange line (cumulative videos uploaded) was much like my stress level, because I was wondering how I'd rearrange things if most of the talks had to cancel at the last minute or if we were dealing with a mostly-live schedule. Balancing EmacsConf anxiety with family challenges resulted in all sorts of oopses in my personal life. (I even accidentally shrank my daughter's favourite T-shirt.) It helped to remind myself that we started off as a single-track single-day conference with mostly live sessions, that it's totally all right to go back to that, and that we have a wonderful (and very patient) community. I think speakers were getting pretty stressed too. I reassured speakers that Oct 31 was a soft target date, not a hard deadline. I didn't want to cancel talks just because life got busy for people.

It worked out reasonably well. We had enough capacity to process and caption the videos as they came in, even the last-minute uploads. Many speakers did their own captions. We ended up having five live talks. Live talks are generally a bit more stressful for me because I worry about technical issues or other things getting in the way, but all those talks went well, thank goodness.

For some reason, Linode doesn't seem to show me detailed accrued charges any more. I think it used to show them before, which is why I managed to write last year's notes in December. If I skip the budget section of these notes, maybe I can post them earlier, and then just follow up once the invoices are out.

I think the timeline worked out mostly all right this year. I don't think moving the target date for videos earlier would have made much of a difference, since speakers would probably still be influenced by the actual date of the conference. It's hard to stave off that feeling of pre-conference panic: Do we have enough speakers? Do we have enough videos? But these graphs might help me remember that it's been like that before and it has still worked out, so it might just be part of my job as an organizer to embrace the uncertainty. Most conferences do live talks, anyway.

Emacs Lisp code for making the table

(append '(("slug" "submitted" "uploaded" "cancelled") hline)
        (sort (delq nil
                    (org-map-entries
                     (lambda ()
                       (list
                        (org-entry-get (point) "SLUG")
                        (org-entry-get (point) "DATE_SUBMITTED")
                        (org-entry-get (point) "DATE_UPLOADED")
                        (org-entry-get (point) "DATE_CANCELLED")))
                     "DATE_SUBMITTED={.}"))
              :key (lambda (o) (format "%s - %s" (elt o 3) (elt o 2)))
              :lessp #'string<) nil)

Python code for plotting the graph
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
import matplotlib.dates as mdates
import io

all_dates = pd.date_range(start='2025-06-01', end='2025-12-09', freq='D')
def convert(rows):
    df = pd.DataFrame(rows, columns=['slug', 'proposal_date', 'upload_date', 'cancelled_date'])
    df = df.map(lambda x: pd.NA if (x == [] or x == "nil") else x)
    df['proposal_date'] = pd.to_datetime(df['proposal_date']).apply(lambda d: d.replace(year=2025))
    df['upload_date'] = pd.to_datetime(df['upload_date']).apply(lambda d: d.replace(year=2025))
    df['cancelled_date'] = pd.to_datetime(df['cancelled_date']).apply(lambda d: d.replace(year=2025))
    prop_counts = df.groupby('proposal_date').size().reindex(all_dates, fill_value=0).cumsum()
    additions = df[df['proposal_date'].notna()].groupby('proposal_date').size().reindex(all_dates, fill_value=0)
    subtractions = df[df['cancelled_date'].notna()].groupby('cancelled_date').size().reindex(all_dates, fill_value=0)
    upload_counts = df[df['upload_date'].notna()].groupby('upload_date').size().reindex(all_dates, fill_value=0).cumsum()
    prop_counts = (additions - subtractions).cumsum()
    return prop_counts, upload_counts

prop_counts, upload_counts = convert(current_data)
prop_counts_2024, upload_counts_2024 = convert(previous_data)

plot_df = pd.DataFrame({
    'Date': all_dates,
    '2025 Proposals': prop_counts.values,
    '2025 Videos': upload_counts.values,
})
plot_df_melted = plot_df.melt(id_vars='Date', var_name='Type', value_name='Count')
sns.set_theme(style="whitegrid")
plt.figure(figsize=(12, 6))
plt.plot(all_dates, prop_counts_2024, color='gray', label='2024 cumulative proposals')
plt.plot(all_dates, upload_counts_2024, color='gray', alpha=0.5, label='2024 cumulative uploads')
ax = sns.lineplot(data=plot_df_melted, x='Date', y='Count', hue='Type', linewidth=2.5)
ax.xaxis.set_major_locator(mdates.MonthLocator())
ax.xaxis.set_major_formatter(mdates.DateFormatter('%b'))
plt.title('EmacsConf 2025: cumulative proposals and video uploads', fontsize=16)
plt.xlabel('Date', fontsize=12)
plt.ylabel('Cumulative Count', fontsize=12)
plt.tight_layout()
plt.savefig('submissions_plot.png')

Managing conference information in Org Mode and Emacs

Organizing the conference meant keeping track of lots of e-mails and lots of tasks over a long period of time. To handle all of the moving parts, I relied on Org Mode in Emacs to manage all the conference-related information. This year, I switched to mainly using the general organizers notebook, with the year-specific one mostly just used for the draft schedule. I added some shortcuts to jump to headings in the main notebook or in the annual notebook (emacsconf-main-org-notebook-heading and emacsconf-current-org-notebook-heading emacsconf.el).

As usual, we used lots of structured entry properties to store all the talk information. Most of my functions from last year worked out fine, aside from the occasional odd bug when I forgot what property I was supposed to use. (:ROOM:, not :BBB:…)

The validation functions I thought about last year would have been nice to have, since I ran into a minor case-sensitivity issue with the Q&A value for one of the talks (live versus Live). This happened last year too and it should have been caught by the case-fold-search I thought I added to my configuration, but since that didn't seem to kick in, I should probably also deal with it on the input side. Idea: I want to validate that certain fields like QA_TYPE have only a specified set of values. I also probably need a validation function to make sure that newly-added talks have all the files that the streaming setup expects, like an overlay for OBS, and one that checks if I've set any properties outside the expected list.

Communication

After some revision, the call for participation went out on emacs-tangents, Emacs News, emacsconf-discuss, emacsconf-org, Reddit, lobste.rs, and System Crafters. (Thanks!) People also mentioned it in various meetups, and there were other threads leading up to it (Reddit, HN, lemmy.world, @fsf)

Once the speakers confirmed the tentative schedules worked for them, I published the schedule on emacsconf.org on Oct 24, as sceduled. But I forgot to announce the schedule on emacsconf-discuss and other places, though. I probably felt a little uncertain about announcing the schedule because it was in such flux. There was even a point where we almost cancelled the whole thing. I also got too busy to reach out to podcasts. I remembered to post it to foss.events, though! Idea: I can announce things and trust that it'll all settle down. I can spend some time neatening up our general organizers-notebook so that there's more of a step-by-step-process. Maybe org-clone-subtree-with-time-shift will be handy.

There were also a few posts and threads afterwards:

For communicating with speakers and volunteers, I reused the mail merge from previous years, with a few more tweaks. I added a template for thanking volunteers. I added some more code for mailing all the volunteers with a specific tag. Some speakers were very used to the process and did everything pretty independently. Other speakers needed a bit more facilitation.

I could get better at coordinating with people who want to help. In the early days, there wasn't that much to help with. A few people volunteered to help with captions for specific videos, so I waited for their edits and focused on other tasks first. I didn't want to pressure them with deadlines or preempt them by working on those when I'd gotten through my other tasks, but it was hard to leave those tasks dangling. They ended up sending in the edits shortly before the conference, which still gave me enough time to coordinate with speakers to review the captions. It was hard to wait, but I appreciate the work they contributed. In late November and early December, there were so many tasks to juggle, but I probably could have e-mailed people once a week with a summary of the things that could be done. Idea: I can practice letting people know what I'm working on and what tasks remain, and I can find a way to ask for help that fits us. If I clean up the design of the backstage area, that might make it easier for people to get started. If I improve my code for comparing captions, I can continue editing subtitles as a backup if I want to take a break from my other work, and merge in other people's contributions when they send them.

I set up Mumble for backstage coordination with other organizers, but we didn't use it much because most of the time, we were on air with someone, so we didn't want to interrupt each other.

There was a question about our guidelines for conduct and someone's out-of-conference postings. I forwarded the matter to our private mailing list so that other volunteers could think about it, because at that point in time, my brain was fried. That seemed to have been resolved. Too bad there's no edebug-on-entry for people so we can figure things out with more clarity. I'm glad other people are around to help navigate situations!

Schedule

Like last year, we had two days of talks, with two tracks on the first day, and about 15-20 minutes between each talk. We had a couple of last-minute switches to live Q&A sessions, which was totally fine. That's why I set up all the rooms in advance.

A few talks ended up being a bit longer than their proposed length. With enough of a heads-up, I could adjust the schedule (especially as other talks got cancelled), but sometimes it was difficult to keep track of changes as I shuffled talks around. I wrote some code to calculate the differences, and I appreciate how the speakers patiently dealt with my flurries of e-mails.

Scheduling some more buffer time between talks might be good. In general, the Q&A time felt like a good length, though, and it was nice that people had the option of continuing with the speaker in the web conference room. So it was actually more like we had two or three or four tracks going on at the same time.

Having two tracks allowed us to accept all the talks. I'm glad I kept the dev track to a single day, though. I ended up managing things all by myself on Sunday afternoon, and that was hard enough with one track. Fortunately, speakers were comfortable navigating the Etherpad questions themselves and reading the questions out loud. Sometimes I stepped in during natural pauses to read the next question on the list (assuming I managed to find the right window among all the ones open on my screen).

Idea: If I get better at setting up one window per ongoing Q&A session and using the volume controls to spatialize each so that I can distinguish left/middle/right conversations, it might be easier for me to keep track of all of those. I wasn't quite sure how to arrange all the windows I wanted to pay attention even with an external screen (1920x1200 + my laptop's 1920x1080). I wonder if I'd trust EXWM to handle tiling all the different web browser windows, while keeping the VNC windows floating so that they don't mess with the size of the stream. Maybe I can borrow my husband's ultrawide monitor. Maybe I can see if I can use one of those fancy macropads for quick shortcuts, like switching to a specified set of windows and unmuting myself. Maybe I can figure out how to funnel streaming captions into IRC channels or another text form (also a requested feature) so that I can monitor them that way and quickly skim the history… Or I can focus on making it easier for people to volunteer (and reminding myself that it's okay to ask for their help!), since then I don't have to split my attention all those different ways and I get to learn from other people's awesomeness.

If we decide to go with one track next year, we might have to prioritize talks, which is hard especially if some of the accepted speakers end up cancelling anyway. Alternatively, a better way might be to make things easier for last-minute volunteers to join and read the questions. They don't even need any special setup or anything; they can just join the BigBlueButton web conference session and read from the pad. Idea: A single Etherpad with all the direct BBB URLs and pad links will make this easier for last-minute volunteers.

Also like last year, we added an open mic session to fill in the time from a last-minute cancellation, and that went well. It might be nice to build that into the schedule earlier instead of waiting for a cancellation so that people can plan for it. We moved the closing remarks earlier as well so that people didn't have to stay up so late in other timezones.

I am super, super thankful we had a crontab automatically switching between talks, because that meant one less time-sensitive thing I had to pay attention to. I didn't worry about cutting people off too early because people could continue off-stream, although this generally worked better for Q&A sessions done over BigBlueButton or on Etherpad rather than IRC. I also improved the code for generating a test schedule so that I could test the automatic switching.

It was probably a good thing that I didn't automatically write the next session time to the Etherpad, though, because people had already started adding notes and questions to the pad before the conference, so I couldn't automatically update them as I changed the schedule. Idea: I can write a function to copy the heads-up for the next talk, or I can add the text to a checklist so that I can easily copy and paste it. Since I was managing the check-ins for both streams as well as doing the occasional bit of hosting, a checklist combining all the info for all the streams might be nice. I didn't really take advantage of the editability of Etherpad, so maybe putting it into HTML instead will allow me to make it easier to skim or use. (Copy icons, etc.)

Like before, I offset the start of the dev track to give ourselves a little more time to warm up, and I started Sunday morning with more asynchronous Q&A instead of web conferences. Not much in terms of bandwidth issues this year.

We got everyone's time constraints correctly this year, hooray! Doing timezone conversions in Emacs means I don't have to calculate things myself, and the code for checking the time constraints of scheduled sessions worked with this year's shifting schedules too. Idea: It would be great to mail iCalendar files (.ics) to each speaker so that they can easily add their talk (including check-in time, URLs, and mod codes) to their calendar. Bonus points if we can get it to update previous copies if I've made a change.

This is what the schedule looked like:

(with-temp-file (expand-file-name "schedule.svg" emacsconf-cache-dir)
  (let ((emacsconf-use-absolute-url t))
    (svg-print
     (emacsconf-schedule-svg
      800 300
      (emacsconf-publish-prepare-for-display
       (emacsconf-get-talk-info))))))

I added a vertical view to the 2025 organizers notebook, which was easier to read. I also added some code to handle cancelling a talk and keeping track of rescheduled talks.

Graphical view of the scheduleSchedule for SaturdaySaturday 9:00- 9:10 Saturday opening remarkssat-open 9:10- 9:20 Making Org-Babel reactiveorg-babel 9:30- 9:55 Emacs as a fully-fledged reference managerreference10:15-10:40 org-gmail: A deep integration of Gmail into your Org Modegmail10:50-11:15 Reading and writing emails in GNU Emacs with Gnusgnus11:25-11:45 LaTeX export in org-mode: the overhaullatex 1:00- 1:25 Basic Calc functionality for engineering or electronicscalc 1:35- 2:15 Blee-LCNT: An Emacs-centered content production and self-publication frameworkblee-lcnt 2:35- 2:40 GNU Emacs Greader (Gnamù Reader) mode is the best Emacs mode in existencegreader 2:50- 3:40 Open sessionopen-mic 4:00- 4:10 Saturday closing remarks / open sessionsat-close 9:30- 9:55 One year progress update Schemacs (formerly Gypsum)schemacs10:15-10:35 Juicemacs: exploring speculative JIT compilation for ELisp in Javajuicemacs10:45-11:10 Swanky Python: Interactive development for Pythonswanky11:20-11:40 Interactive Python programming in Emacspython 1:00- 1:25 Emacs, editors, and LLM driven workflowsllm 1:45- 2:05 Emacs and private AI: a great matchprivate-ai 2:25- 2:55 Common Lisp images communicating like-a-human through shared Emacs slime and eevcommonlisp 3:05- 3:30 Modern Emacs/Elisp hardware/software accelerated graphicsgraphics9 AM10 AM11 AM12 PM1 PM2 PM3 PM4 PM5 PMSchedule for SundaySunday 9:00- 9:10 Sunday opening remarkssun-open 9:10- 9:30 Some problems of modernizing Emacsmodern 9:40-10:15 An introduction to the Emacs Readerreader10:35-10:45 Weightlifting tracking with Emacs on Androidweights11:05-11:25 corfu+yasnippet: Easier than I thoughtcompletion 1:00- 1:25 Zettelkasten for regular Emacs hackerszettelkasten 1:45- 2:15 Questions and answers to help you fly with Hyperbolehyperboleqa 2:15- 2:35 Gardening in Emacs: A Windows user's tale of tending, tweaking, and triumphgardening 2:45- 3:20 Bookclub tapasbookclub-tapas 3:40- 3:50 Sunday closing remarkssun-close9 AM10 AM11 AM12 PM1 PM2 PM3 PM4 PM5 PM

Idea: I still haven't gotten around to localizing times on the watch pages. That could be nice.

Recorded introductions

Recording all the introductions beforehand was extremely helpful. I used subed-record.el to record and compile the audio without having to edit out the oopses manually. Recording the intros also gave me something to do other than worry about missing videos.

A few speakers helped correct the pronunciations of their names, which was nice. I probably could have picked up the right pronunciation for one of them if I had remembered to check his video first, as he had not only uploaded a video early but even said his name in it. Next time, I can go check that first.

This year, all the intros played the properly corrected files along with their subtitles. The process is improving!

Recorded videos

As usual, most speakers sent in pre-recorded videos this year, although it took them a little bit longer to do them because life got busy for everyone. Just like last year, speakers uploaded their files via PsiTransfer. I picked up some people's videos late because I hadn't checked. I've modified the upload instructions to ask the speakers to email me when they've uploaded their file, since I'm not sure that PsiTransfer can automatically send email when a new file has been uploaded. We set up a proper domain name, so this year, people didn't get confused by trying to FTP to it.

I had to redo some of the re-encodings and ask one speaker to reupload is video, but we managed to catch those errors before they streamed.

I normalized all the audio myself this year. I used Audacity to normalize it to -16 LUFS. I didn't listen to everything in full, but the results seemed to have been alright. Idea: Audio left/right difference was not an issue this year, but in the future, I might still consider mixing the audio down to mono.

All the files properly loaded from the cache directory instead of getting shadowed by files in other directories, since I reused the process from last year instead of switching mid-way.

People didn't complain about colour smearing, but it looks like the Calc talk had some. Ah! For some reason, MPV was back on 0.35, so now I've modified our Ansible playbook so that it insists on 0.38. I don't think there were any issues about switching between dark mode or light mode.

There was one last-minute upload where I wasn't sure whether there were supposed to be captions in the first part. When I toggled the captions to try to reload them once I copied over the updated VTT, I think I accidentally left them toggled them off, but fortunately the speaker let me know in IRC.

For the opening video, I made the text a bit more generic by removing the year references so that I can theoretically reuse the audio next year. That might save me some time.

I modified our mpv.conf to display the time remaining in the lower right-hand corner. This was a fantastic idea that made it easy to give the speaker a heads-up that their recorded talk was about to finish and that we were about to switch over to the live Q&A session. Because sometimes we weren't able to spare enough attention to host a session, I usually just added the time of the next session to the Etherpad to give speakers a heads-up that the stream would be switching away from their Q&A session. Then I didn't need to make a fancy Javascript timer either.

Captioning

The next step in the process was to caption each uploaded video. While speech recognition tools give us a great headstart, there's really no way around the work of getting technical topics right.

We used WhisperX for speech-to-text again. This time, I used the --initial_prompt argument to try to get it to spell things properly: "Transcribe this talk about Emacs. It may mention Emacs keywords such as Org Mode, Org Roam, Magit, gptel, or chatgpt-shell, or tech keywords such as LLMs. Format function names and keyboard shortcut sequences according to Emacs conventions using Markdown syntax. For example: control h becomes \`C-h\`.". It did not actually do the keyboard shortcut sequences. (Emacs documentation is an infinitesimal fraction of the training data, probably!) It generally did a good job of recognizing Emacs rather than EMAX and capitalizing things. Idea: I can experiment with few-shot prompting techniques or just expand my-subed-fix-common-errors to detect the patterns.

This year, we used Anush V's sub-seg tool to split the subtitles into reasonably short, logically split phrases. I used to manually split these so that the subtitles flowed more smoothly, or if I was pressed for time, I left it at just the length-based splits that WhisperX uses. sub-seg was much nicer as a starting point, so I wrote a shell script to run it more automatically. It still had a few run-on captions and there were a couple of files that confused it, so I manually split those.

Shell script for calling sub-seg

#!/bin/bash ~/vendor/sub-seg/.venv/bin/python3 ~/vendor/sub-seg/src/prediction.py "$1" tmp/subseg-predict.txt ~/vendor/sub-seg.venv/bin/python3 ~/vendor/sub-seg/src/postprocess_to_single_lines.py "\(1" /tmp/subseg-predict.txt /tmp/subseg-cleaned.txt grep -v -e "^\)" /tmp/subseg-cleaned.txt > "${1%.*}–backstage–split.txt"

I felt that it was also a good idea to correct the timing before posting the subtitles for other people to work on because otherwise it would be more difficult for any volunteers to be able to replay parts of the subtitles that needed extra attention. WhisperX often gave me overlapping caption timestamps and it didn't always have word timestamps I could use instead. I used Aeneas to re-align the text with the audio so that we could get better timestamps. As usual, Aeneas got confused by silences or non-speech audio, so I used my subed-word-data.el code to visualize the word timing and my subed-align.el code to realign regions. Idea: I should probably be able to use the word data to realign things semi-automatically, or at least flag things that might need more tweaking. There were a few chunks that weren't recognized at all, but I was able to spot them and fix them when I was fixing the timestamps. Making this more automated will make it much easier to share captioning work with other volunteers. Alternatively, I can also post the talks for captioning before I fix the timestamps, because I can fix them while other people are editing the text.

While I was correcting the timestamps, it was pretty tempting for me to just go ahead and fix whatever errors I encountered along the way. From there on, it seemed like only a little bit more work was needed in order to get it ready for the speaker to review, and besides, doing subtitles is one of the things I enjoy about EmacsConf. So the end result is that for most of the talks, by the time I finished getting it to the point where I felt like someone who was new to captioning could take over, it was often quite close to being done. I did actually manage to successfully accept some people's help. The code that I wrote for showing me the word differences between two files (sometimes the speaker's script or the subtitles that were edited by another volunteer) was very useful for last-minute merges.

Working on captions is one of my favourite parts. Well-edited captions are totally a nice-to-have, but I like it when people can watch the videos (or skim them) and not worry about not being able to hear or understand what someone said. I enjoy sitting down and spending time with people's presentations, turning them into text that we can search. Captioning helps me feel connected with some of the things I love the most about the Emacs community.

I really appreciated how a number of speakers and volunteers helped with quality control by watching other videos in the backstage area. Idea: I can spend some time improving the design of the backstage area to make it more pleasant and to make it easier for people to find something to work on if they have a little time.

Of course, some talks couldn't be captioned beforehand because they were live. For live conferences and for many of the Q&A sessions, we used BigBlueButton.

BigBlueButton web conference

We used BigBlueButton again this year instead of switching over to something like Galene. I moved the node from bandali's Linode account to mine so that I wouldn't feel as guilty bringing it up and down throughout the year for various Emacs meetups. That also meant that the actual setup was fairly stable since it had survived a lot of meetups before EmacsConf, and I didn't have to spend as much time before the event worrying about it. I still worried a little, as after the November OrgMeetup, I noticed a technical issue that I couldn't nail down. Fortunately, that happened after the meetup had already ended. I also used my meetup crontab to schedule test sessions with some speakers so that they could record their presentation or doublecheck their sharing setup.

I resized the BigBlueButton early Friday afternoon this year so that speakers could have a few more hours to test their setups if they wanted to, especially since one speaker had mentioned having a hard time sharing his screen on Wayland.

I thought I had --allow_auto_disk_resize true in the bbb-prod shell-script I used to resize the node, but I don't think linode-cli actually resized the disk (maybe I ran a different shell script), and I forgot to double-check that before the conference. Fortunately, we had just enough space for all the recordings. I noticed only when I was waiting for the recordings to finish processing, since some of them were failing. Idea: Next time, I should manually check that the disk has resized, and I can probably tweak my checklist too.

Like before, we used moderator codes so that speakers could let themselves into the room and get everything set up just in case. Idea: I wonder if there's a way to specify the moderator code in the URL so that I can link to that directly. Maybe I can modify the web interface to fill that in with a little bit of Javascript.

Oddly, the moderator codes didn't let people start or stop recording, but fortunately I was able to log in and take care of all of that. I mostly remembered to start the recording, except for one instance where I started recording a little bit late.

Code for reporting BBB usage
(emacsconf-extract-bbb-report
 (seq-remove (lambda (o)
               (string-match "backups" o))
             (directory-files-recursively emacsconf-extract-bbb-raw-dir "events.xml")))
2023 2024 2025  
62 107 35 Max number of simultaneous users
6 7 5 Max number of simultaneous meetings
27 25 14 Max number of people in one meeting
84 102 55 Total unique users
36 40 27 Total unique talking

I might be able to get away with a Linode 8 GB node (USD 0.072/hour) instead of a Linode 16 GB node (USD 0.144/hour), but keeping it at the current size is probably okay.

I'll go into more detail in the budget section, but the total was USD 14.02 to host BigBlueButton ourselves in December, and between USD 5 to USD 8 / month to run it the rest of the time (including upsizing it for meetups), so that was much less than what it would have cost for BigBlueButton-specific hosting.

Checking speakers in

To keep the live talks and Q&A sessions flowing smoothly, there's a bit of a scramble behind the scenes. We asked speakers to check in at least half an hour before they need to go live, and most of them were able to find their way to the BigBlueButton room using the provided moderator codes. I mostly handled the check-ins, with some help from Corwin when two people needed to be checked in at the same time. We generally didn't have any tech issues, although a couple of people were missing during their Q&A sessions. (We just shrugged and continued; too much to take care of to worry about anything!)

Check-ins are usually good opportunities to chat with speakers, but because I needed to pay attention to the other streams as well as check in other people, it felt a bit more rushed and less enjoyable. I missed having those opportunities to connect, but I'm glad speakers were able to handle so much on their own.

Hosting

Corwin did a lot of the on-stream hosting with a little help from Amin on Saturday. I occasionally jumped in and asked the speakers questions from the pad, but when I was busy checking people in or managing other technical issues cropping up, the speakers also read the questions out themselves so that I could match things up in the transcript afterwards.

No crashes this time, I think. Hooray!

Infrastructure

I used September and October to review all the infrastructure and see what made sense to upgrade. The BigBlueButton instance is on a virtual private server that's dedicated to it, since it doesn't like to share with any other services. Since I set it up from scratch following the recommended configuration, the server uses a recent version of Ubuntu. Some of the other servers use outdated Debian distributions that no longer get security updates. They have other services on them, so I'll leave them to bandali to upgrade when he's ready.

Streaming

I am so, so glad that we had our VNC+OBS setup this year, with two separate user accounts using OBS to stream each track from a virtual desktop on a remote server. We switched to this setup in 2022 so that I could handle multiple tracks without worrying about last-minute coordination or people's laptop specs. If we had had our original setup in 2019, with hosts streaming from their personal computers, I think we'd have been dead in the water. Instead, our scripts took care of most of the on-screen actions, so I just needed to rearrange the layout. (Idea: If I can figure out how to get BigBlueButton to use our preferred layout, that would make it even better.)

I used MPV to monitor the streams in little thumbnails. I set those windows to always be on top, and I set the audio so that the general track was on my left side and the development track was o my right. I used some shortcuts to jump between streams reliably, taking advantage of how I bound the Windows key on my laptop to the Super modifier. I used KDE's custom keyboard shortcuts to set Super-g to raise all of my general-track windows and Super-d to do the same for all of the development-track ones. Those were set to commands like this:

/home/sacha/bin/focus_or_launch "emacsconf-gen - TigerVNC"; /home/sacha/bin/focus_or_launch gen.webm; /home/sacha/bin/focus_or_launch "gen.*Konsole"

Code for focus_or_launch

#!/bin/bash

############# GLOBVAR/PREP ###############

Executable="\(1" ExecutableBase="\)(basename "$Executable")" Launchcommand="$2" Usage="\ Usage: $(basename $0) command [launchcommand] [exclude] E.g.: $(basename $0) google-chrome\ " ExcludePattern="$3"

################ MAIN ####################

if ; then MostRecentWID="$(printf "%d" $(wmctrl -xl | grep "$ExecutableBase" | tail -1 2> /dev/null | awk '{print \(1}'))" else MostRecentWID="\)(printf "%d" $(wmctrl -xl | grep -ve "$ExcludePattern" | grep "$ExecutableBase" | tail -1 2> /dev/null | awk '{print $1}'))" fi

if ; then if ; then "$Executable" > /dev/null 2>&1 & else $LaunchCommand > /dev/null 2>&1 & fi disown else if xdotool search –class "\(ExecutableBase" | grep -q "^\)(xdotool getactivewindow)$"; then xdotool sleep 0.050 key "ctrl+alt+l" else

xdotool windowactivate "$MostRecentWID" 2>&1 | grep failed \ && xdotool search –class "$ExecutableBase" windowactivate %@ fi fi

(Idea: It would be great to easily assign specific windows to shortcuts on my numeric keypad or on a macropad. Maybe I can rename a window or manually update a list that a script can read…)

To get the video streams out to viewers, we used Icecast (a streaming media server) on a Linode 64GB 16 core shared CPU server again this year, and that seemed to work out. I briefly contemplated using a more modern setup like nginx-rtmp, Ant Media Server, or SRS if we wanted HLS (wider support), adaptive bitrate streaming, or lower latency, but I decided against it because I didn't want to add more complexity. Good thing too. This year would not have been a good time to experiment with something new.

Like before, watching the stream directly using mpv, vlc, or ffplay was smoother than watching it through the web-based viewers. One of the participants suggested adding more detailed instructions for VLC so that people can enjoy it even without using the command-line, so we did.

The 480p stream and the YouTube stream were all right, although I forgot to start one of the YouTube streams until a bit later. Idea: Next time, I can set all the streams to autostart.

Corwin had an extra server lying around, so I used that to restream to Toobnix just in case. That seems to have worked, although I didn't have the brainspace to check on it.

I totally forgot about displaying random packages on the waiting screen. Idea: Maybe next year I can add a fortune.txt to the cache directory with various one-line Emacs tips.

I might be able to drop the specifications down to 32GB 8 core if we wanted to.

Publishing

People appreciated being able to get videos and transcripts as soon as each talk aired. Publishing the video files and transcripts generally worked out smoothly, aside from a few times when I needed to manually update the git repository. I modified the code to add more links to the Org files and to walk me through publishing videos to YouTube.

I uploaded all the videos to YouTube and scheduled them so that I didn't have to manage that during the conference. I did not get around to uploading them to Toobnix until after the conference since dealing with lots of duplicated updates is annoying. I've been writing a few more functions to work with the Toobnix API, so it might be a little easier to do things like update descriptions or subtitles next year.

Etherpad

As usual, we used Etherpad to collect questions from conference partcipants, and many speakers answered questions there as well.

I used the systemli.etherpad Ansible role to upgrade to Etherpad 2.5. This included some security fixes, so that was a relief.

I added pronouns and pronunciations to the Etherpad to make it easier for hosts to remember just in case.

I did not get to use emacsconf-erc-copy to copy IRC to Etherpad because I was too busy juggling everything else, so I just updated things afterwards.

Idea: Just in case, it might be good to have a backup plan in case I need to switch Etherpad to authenticated users or read-only use. Maybe I can prepare questions beforehand, just in case we get some serious griefing.

IRC

The plain-text chat channels on IRC continued to be a great place to discuss things, with lots of discussions, comments, and encouraging feedback. My automated IRC system continued to do a good job of posting the talk links, and the announcements made it easier to split up the log by talk.

Planning for social challenges is harder than planning for technical ones, though. libera.chat has been dealing with spam attacks recently. Someone's also been griefing #emacs and other channels via the chat.emacsconf.org web interface, so we decided to keep chat.emacsconf.org dormant until the conference itself. If this is an issue next year, we might need to figure out moderation. I'd prefer to not require everyone to register accounts or be granted voice permissions, so we'll see.

Extracting the Q&A

We recorded all the Q&A sessions so that we could post them afterwards. As mentioned, I only started the recording late once. Progress! I generally started it a few minutes early. As I got more confident about paying attention to the start of a session and rearranging the layout on screen, I also got better at starting the recording shortly before turning it over to the speaker. That made it easier to trim the recordings afterwards.

It took me a little longer to get to the Q&A sessions this year. Like last year, getting the recordings from BigBlueButton was fairly easy because I could get a single processed video instead of combining the audio, screenshare, and webcam video myself. This year the single-file video downloads were .m4v files, so I needed to modify things slightly. I think my workflow last year assumed I started with .webm files. Anyway, after some re-encoding, I got it processed. Now I've modified the /usr/local/bigbluebutton/core/scripts/video.yml to enable webm. I wonder if it got wiped after my panic-reinstall after November's OrgMeetup. Idea: I should add the config to my Ansible so that it stays throughout reinstalls and upgrades.

I wrote a emacsconf-subed-copy-current-chapter-text function so that I can more easily paste answers into the Q&A… which turned out to be a superset of the emacsconf-extract-subed-copy-section-text I mentioned in last year's notes and totally forgot about. This tells me that I need to add it to my EmacsConf - organizers-notebook notes so that I actually know what to do. I did not use my completion functions to add section headings based on comments either. Apparently this was emacsconf-extract-insert-note-with-question-heading, which I do have a note about in my organizer notebook.

Audio mixing was reasonable. I normalized the audio in Audacity, and I manually fixed some sections where some participants' audio volumes were lower.

Budget

Hosting a conference online continues to be pretty manageable. Here are our costs for the year (including taxes where applicable):

Code for calculating BBB node costs
(append
'(("Node" "Jan" "Feb" "Mar" "Apr" "May" "Jun" "Jul" "Aug" "Sep" "Oct" "Nov" "Dec" "Total") hline)
(mapcar
 (lambda (group)
   (let ((amounts (mapcar
          (lambda (file)
            (format "%.2f"
                    (apply '+
                           (seq-keep (lambda (row)
                                       (when (string-match (car group) (car row))
                                         (string-to-number (elt row 8))))
                                     (cdr (pcsv-parse-file file))))))
          (directory-files (format "~/proj/emacsconf/2025/invoices/%s/" (cdr group))
                           t "\\.csv"))))
   (append (list (car group))
           amounts
           (list (format "%.2f" (apply '+ (mapcar 'string-to-number amounts)))))))
 '(("meet" . "sacha")
   ("front0" . "bandali")
   ("live0" . "bandali")))
   nil
   )
Node Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Total
meet 2.17 7.55 6.78 6.74 7.13 6.95 7.19 7.27 6.75 7.19 7.56 14.02 87.30
front0 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 18.79 73.79
live0 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 32.89 87.89

Grand total for 2025: USD 248.98

Server configuration during the conference:

Node Specs Hours upscaled CPU load
meet 16GB 8core shared 52 peak 172% CPU (100% is 1 CPU)
front 32GB 8core shared 47 peak 47% CPU (100% is 1 CPU)
live 64GB 16core shared 48 peak 440% CPU (100% is 1 CPU)
res 46GB 12core   peak 60% total CPU (100% is 12 CPUs); each OBS ~3.5 CPUs), mem 7GB used
media 3GB 1core    

These hosting costs are a little higher than 2024 because we now pay for hosting the BigBlueButton server (meet) year-round. That's ~ 5 USD/month for a Linode 1 GB instance and an extra USD 2-3 / month that lets us provide a platform for OrgMeetup, Emacs APAC, and Emacs Berlin by alternating between a 1 GB Linode and an 8 GB Linode as needed. The meetups had previously been using the free Jitsi service, but sometimes that has some limitations. In 2023 and most of 2024, the BigBlueButton server had been provided by FossHost, which has since shut down. This time, maintaining the server year-round meant that we didn't have to do any last-minute scrambles to install and configure the machine, which I appreciated. I could potentially get the cost down further if I use Linode's custom images to create a node from a saved image right before a meetup, but I think that trades about USD 2 of savings/month for much more technical risk, so I'm fine with just leaving it running downscaled.

If we need to cut costs, live0 might be more of a candidate because I think we'll be able to use Ansible scripts to recreate the Icecast setup. I think we're fine, though. Also, based on the CPU peak loads, we might be able to get away with lower specs during the conference (maybe meet: 8 GB, front: 8 GB, live: 32 GB), for an estimated savings of USD 27.76, with the risk of having to worry about it if we hit the limit. So it's probably not a big deal for now.

I think people's donations through the Working Together program can cover the costs for this year, just like last year. (Thanks!) I just have to do some paperwork.

In addition to these servers, Ry P provided res.emacsconf.org for OBS streaming over VNC sessions. The Free Software Foundation also provided media.emacsconf.org for serving media files, and yang3 provided eu.media.emacsconf.org.

If other people are considering running an online conference, the hosting part is surprisingly manageable, at least for our tiny audience size of about 100 peak simultaneous viewers and 35 web conference participants. It was nice to make sure that everyone can watch without ads or Javascript.

Behind the scenes: In terms of keeping an eye on performance limits, we're usually more CPU-bound than memory- or disk-bound, so we had plenty of room this year. Now I have some Org Babel source blocks to automatically collect the stats from different servers. Here's what that Org block looks like:

#+begin_src sh :dir /ssh:res:/ :results verbatim :wrap example
top -b -n 1 | head
#+end_src

I should write something similar to grab the Icecast stats from http://live0.emacsconf.org:8001/status.xsl periodically. Curl will do the trick, of course, but maybe I can get Emacs to add a row to an Org Mode table.

Tracking my time

As part of my general interest in time-tracking, I tracked EmacsConf-related time separately from my general Emacs-related time.

Calculations
(append (list '("Year" "Month" "Hours") 'hline)
(seq-mapcat
 (lambda (year)
   (mapcar
    (lambda (group)
      (list
       year
       (car group)
       (format "%.1f"
               (apply '+
                      (mapcar (lambda (o) (/ (alist-get 'duration o) 3600.0))
                              (cdr group))))))
    (seq-group-by (lambda (o)
                    (substring (alist-get 'date o) 5 7))
                  (quantified-records
                   (concat year "-01-01")
                   (concat year "-12-31")
                   "&filter_string=emacsconf&split=keep&order=oldest"))))
 '("2024" "2025")))
Year Month Hours
2024 01 2.5
2024 07 2.8
2024 08 0.6
2024 09 7.0
2024 10 14.4
2024 11 30.7
2024 12 46.2
2025 01 9.7
2025 02 1.4
2025 04 0.9
2025 06 0.8
2025 07 6.0
2025 08 3.6
2025 09 10.9
2025 10 1.2
2025 11 21.0
2025 12 64.3
import pandas as pd
import matplotlib.pyplot as plt
df = pd.DataFrame(data[1:], columns=data[0])
df = pd.pivot_table(df, columns=['Month'], index=['Year'], values='Hours', aggfunc='sum', fill_value=0)
df = df.reindex(columns=range(1, 13), fill_value=0)
df['Total'] = df.sum(axis=1)
df = df.applymap(lambda x: f"{x:.1f}" if x != 0 else "")
return df
Year 1 2 3 4 5 6 7 8 9 10 11 12 Total
2024 2.5           2.8 0.6 7.0 14.4 30.7 46.2 104.2
2025 9.7 1.4   0.9   0.8 6.0 3.6 10.9 1.2 21.0 64.3 119.8

This year, I spent more time doing the reencoding and captions in December, since most of the submissions came in around that time. I didn't even listen to all the videos in real time. I used my shortcuts to skim the captions and jump around. It would have been nice to be able to spread out the work a little bit more instead of squeezing most of it into the first week of December, since that would have made it easier to coordinate with other volunteers without feeling like I might have to do more last-minute scrambles if they were busier than expected.

I was a little bit nervous about making sure that the infrastructure was all going to be okay for the conference as well as double checking the videos for possible encoding issues or audio issues. Stress tends to make me gloss over things or make small mistakes, which meant I had to slow down even more to double-check things and add more things to our process documentation.

I'm glad it all worked out. Even if I switched that time to working on Emacs stuff myself, I don't think I would have been able to put together all the kinds of wonderful tips and experiences that other people shared in the conference, so that was worthwhile.

(Here's a longer-term analysis time going back to 2012.)

Thanks

  • Thank you to all the speakers, volunteers, and participants, and to all those other people in our lives who make it possible through time and support.
  • Thanks to other volunteers: 
    • Corwin and Amin for helping with the organization
    • JC Helary, Triko, and James Endres Howell for help reviewing CFPs
    • Amitav Krishna, Rodion Goritskov, jay_bird, and indra for captions
    • yang3 for the EU mirror we're setting up
    • Bhavin Gandhi, Michael Kokosenski, Iain Young, Jamie Cullen, Ihor Radchenko (yantar92), FlowyCoder for other help
  • Thanks to the Free Software Foundation for the mailing lists, the media server, and of course, GNU Emacs.
  • Thanks to Ry P for the server that we're using for OBS streaming and processing videos.
  • Thanks to the many users and contributers and project teams that create all the awesome free software we use, especially:
  • Thanks to shoshin for the music.
  • Thanks to people who donated via the FSF Working Together program: Scott Ranby, Jonathan Mitchell, and 8 other anonymous donors!

Overall

I've already started hearing from people who enjoyed the conference and picked up lots of good tips from it. Wonderful!

EmacsConf 2025 was a bit more challenging this year. The world has gotten a lot busier. Return-to-work mandates, job market turmoil, health challenges, bigger societal changes… It's harder for people to find time. For example, the maintainers of Emacs and Org are too busy working on useful updates and bugfixes to rehash the news for us, so maybe I'll get back to doing Emacs News Highlights next year. I'm super lucky in that I can stand outside most of all of that and make this space where we can take a break and chat about Emacs. I really appreciated having this nice, cozy little place where people could get together and bump into other people who might be interested in similar things, either at the event itself or in the discussions afterwards. I'm glad we started with a large schedule and let things settle down. I'd rather have that slack instead of making speakers feel stressed or guilty.

I love the way that working on the various parts of the conference gives me an excuse to tinker with Emacs and figure out how to use it for more things.

As you can tell from the other sections in this post, I tend to have so much fun doing this that I often forget to check if I've already written the same functions before. When I do remember, I feel really good about being able to build on this accumulation of little improvements.

I didn't feel like I really got to attend EmacsConf this year, but that's not really surprising because I don't usually get to. I think the only time I've actually been able to take proper notes during EmacsConf itself was back in 2013. It's okay, I get to spend a little time with presentations before anyone else does, and there's always the time afterwards. Plus I can always reach out to the speakers myself. It might be nice to be able to just sit and enjoy it and ask questions. Maybe someday when I've automated enough to make this something that I can run easily even on my own.

So let's quickly sketch out some possible scenarios:

  • I might need to do it on my own next year, in case other organizers get pulled away at the last minute: I think it's possible, especially if I can plan for some emergency moderation or last-minute volunteers. I had a good experience, despite the stress of juggling things live on stream. (One time, one of the speakers had a question for me, and he had to repeat it a few times before I found the right tab and unmuted.) Still, I think it would be great to do it again next year.
  • Some of the other organizers might be more available: It would be nice to have people on screen handling the hosting. If people can help out with verifying the encoding, normalizing the audio, and shepherding videos through the process, that might let me free up some time to coordinate captions with other volunteers even for later submissions.
  • I might get better at asking for help and making it easier for people to get involved: That would be pretty awesome. Sometimes it's hard for me to think about what and how, so if some people can take the initiative, that could be good.

This is good. It means that I can probably say yes to EmacsConf even if it's just me and whoever wants to share what they've been learning about Emacs this year. That's the basic level. We're still here, amazing! Anything else people can add to that is a welcome bonus. We'll make stone soup together.

EmacsConf doesn't have to be snazzy. We don't need to try to out-market VS Code or whatever other editors emerge over the next 10 years. We can just keep having fun and doing awesome things with Emacs. I love the way that this online conference lets people participate from all over the world. We like to focus on facilitating sharing and then capturing the videos, questions, answers so that people can keep learning from them afterwards. I'm looking forward to more of that next year.

I'd love to hear what people thought of it and see how we can make things better together. I'd love to swap notes with organizers of other conferences, too. What's it like behind the scenes?

My next steps are:

  • Extract part of this into a report for the emacsconf.org website, like EmacsConf - 2024 - EmacsConf 2024 Report.
  • Announce the resources and report on emacsconf-discuss.
  • Catch up on the other parts of my life I've been postponing.
  • Start thinking about EmacsConf 2026 =)
View org source for this post

EmacsConf infrastructure upgrades

| emacsconf

With the EmacsConf call for proposals now closed, I have a little time before EmacsConf speakers send in their pre-recorded videos come in for captioning. I decided to dust off the infrastructure to see what makes sense to upgrade.

We use Etherpad for collaborative note-taking during EmacsConf. It's straightforward to use and pretty reliable. Conference participants can use it to share notes, questions, and links. They can also use IRC to ask questions, and volunteers copy those questions into the pad for the talk. Hosts and speakers can keep an eye on the pad for questions. We send speakers a copy of their talk's pad after the conference, and we post that along with other follow-up questions on the conference wiki. Here's an example: Writing academic papers in Org-Roam.

A native Emacs solution for collaborative notes would be even better. CRDT was great for experimenting with real-time collaboration within Emacs, but I'm not sure it can handle a ton of simultaneous connections and I don't want to find out in the middle of the conference. Also, requiring Emacs would leave out the people who only have a web browser handy. It would be super cool if we had something with Emacs, web, and IRC interfaces, but for now, Etherpad will do.

We started by using Wikimedia's instance, and then we moved to hosting our own. For the past two years, we've used Etherpad 1.9.7. Etherpad is currently at version 2.5.0. There are some performance improvements, bugfixes, and security fixes, so I think it'll be worth upgrading to that. I don't know of any specific issues or upgrades, but it's a good idea to stay closer to the latest release than to get too far out of date.

I switched our roles/pad/tasks/main.yml to use systemli.etherpad from ansible-galaxy. I also figured out how to set up a Vagrant virtual machine that I could destroy and reconfigure with vagrant destroy; vagrant up --provision. Here's my Vagrant file for that:

# -*- mode: ruby -*-
Vagrant.configure("2") do |config|
  config.vm.box = "debian/bookworm64"
  config.vm.define "pad" do |pad|
  end
  config.vm.provider "virtualbox" do |vb|
    vb.memory = "2048"
  end
  config.vm.network "private_network", ip: "192.168.56.2"
  config.vm.provision "ansible" do |ansible|
    ansible.playbook = "../../vagrant-playbook.yml"
  end
end

The playbook it refers to has this:

- name: Pre-flight checks and package installation
  hosts: pad
  become: true
  gather_facts: false
  pre_tasks:
    - name: Ensure ntpdate is installed for time sync
      ansible.builtin.apt:
        name: ntpdate
        state: present
        update_cache: yes
    - name: Synchronize system clock
      ansible.builtin.command: ntpdate pool.ntp.org
      changed_when: true
    - name: Ensure ACL package is installed
      ansible.builtin.apt:
        name: acl
        state: present
- name: Load vars
  hosts: pad
  tags: always
  tasks:
    - include_vars:
        file: vagrant-vars.yml
- name: Set up pad proxy
  hosts: pad
  tags: proxy
  roles:
    - pad-proxy
- name: Set up pad
  hosts: pad
  tags: pad
  roles:
    - pad

I had used Vagrant in 2013, and it felt good to have the time to set up more testing infrastructure. I liked being able to test my pad and pad-proxy roles against a local virtual machine. I could figure out whatever tweaks I needed without messing up the production instance that we use for some meetups in between conferences.

Our production instance is on Debian 10 (Buster). That has reached its end of life for security updates, so apt-cache update doesn't work on it any more, and those steps in my Ansible playbook fail. I'm waiting for Amin Bandali to work on upgrading that server, since he has other stuff running on it. By setting update_cache variable that I override in my inventory.yml and referring to it with update_cache: "" in my task, I can conditionally disable the apt-cache update steps. That let me run the playbook against the production server, and now we're on Etherpad 2.x.

I recently updated our BigBlueButton instance to version 3.0.12. We've decided to stay with Icecast 2.4.4-1 for doing the livestreaming. We'll probably also keep OBS 29.1.2 and ffmpeg 6.0.1 instead of upgrading. With no must-have new features and other organizers' limited availability, it's better to keep those parts stable. This year, we'll continue using whisperx to help with the first draft of captions, but we'll probably try large-v3 instead of large-v2 by default. Some people find that large-v3's performance is better, some people find it's worse, so we'll see. Now that I know about whisperx's --initial_prompt option, I might be able to nudge it to the vocabulary and punctuation style we like.

Since the bones seem pretty solid, I'm looking forward to refamiliarizing myself with the Emacs Lisp code I wrote to help run the conference. I saved a bunch of improvement ideas from last year, and I can't wait to turn them into code. That's probably going to be lots of fun!

View org source for this post

Scaling a BigBlueButton server down to a 1 GB node between uses

| geek, tech, emacsconf, emacs

Now that we've survived EmacsConf, I've been looking into running a BigBlueButton server so that various Emacs meetups can use it if they like instead of relying on Jitsi or other free video-conferencing services. (I spent some time looking into Galene, but I'm not quite sure that's ready for our uses yet, like this issue that LibrePlanet ran into with recording.)

BigBlueButton requires a server with at least 4 CPU cores and 8 GB of RAM to even start up, and it doesn't like to share with other services. This costs about USD 48+tax/month on Linode or USD 576+tax/year, which is not an efficient use of funds yet. I could delete it after each instance, but I've been having a hard time properly restoring it from backup after deploying to a new IP address. bbb-conf --setip doesn't seem to catch everything, so I was still getting curl errors related to verifying the certificate.

A reasonable in-between is to run it on Linode's lowest plan (1 core, 1GB RAM; USD 60+tax for the year) in between meetups, and then spin things up for maybe 6-12 hours around each meetup. If I go with the 4-core 8 GB setup, that would be an extra USD 0.43 - 0.86 USD per meetup, which is eminently doable. I could even go with the recommended configuration of 8 cores and 16 GB memory on a dedicated CPU plan (USD 0.216/hour, so USD 1.30 to 2.59 per meetup). This was the approach that we used while preparing for EmacsConf. Since I didn't have a lot of programming time, I scaled the node up to 4 core / 8GB RAM whenever I had time to work on it, and I scaled it down to 1GB at the end of each of my working sessions. I scaled it up to dedicated 8 core / 16 GB RAM for EmacsConf, during which we used roughly half of the CPU capacity in order to host a max of 107 simultaneous users over 7 meetings.

I reviewed my BigBlueButton setup notes in the EmacsConf organizers notebook and the 2024 notebook and set up a Linode instance under my account, so that I can handle the billing and also so that Amin Bandali doesn't get spammed by all the notifications (up, down, up, down…). And then I'll be able to just scale it up when EmacsConf comes around again, which is nice.

Anyway, BBB refuses to install on a machine with fewer than 4 cores or 8 GB RAM, but once you set it up, it'll valiantly thrash around even on an underpowered server, which makes working with the server over ssh a lot slower. Besides, that's not friendly to other people using the same server. I wanted to configure the services so that they would only run on a server of the correct size. It turns out that systemd will let you specify either ConditionMemory and ConditionCPUs in the unit configuration file, and that you can use files ending in .conf in a directory named like yourservicename.service.d to override part of the configuration. Clear examples were hard to find, so I wanted to share these notes.

Since ConditionMemory is specified in bytes (ex: 8000000000), I found ConditionCPUs to be easier to read.

I used this command to check if I'd gotten the syntax right:

systemd-analyze condition 'ConditionCPUs=>=4'

and then I wrote this script to set up the overrides:

CPUS_REQUIRED=4
for ID in coturn.service redis-server.service bigbluebutton.target multi-user.target bbb-graphql-server.service bbb-rap-resque-worker.service bbb-webrtc-sfu.service bbb-fsesl-akka.service bbb-webrtc-recorder.service bbb-pads.service bbb-export-annotations.service bbb-web.service freeswitch.service etherpad.service bbb-rap-starter.service bbb-rap-caption-inbox.service freeswitch.service bbb-apps-akka.service bbb-graphql-actions postgresql@14-main.service; do
    mkdir -p /etc/systemd/system/$ID.d
    printf "[Unit]\nConditionCPUs=>=$CPUS_REQUIRED\n" > /etc/systemd/system/$ID.d/require-cpu.conf
done
systemctl daemon-reload
systemd-analyze verify bigbluebutton.target

It seems to work. When I use linode-cli to resize to the testing size, BigBlueButton works:

#!/bin/bash
source /home/sacha/.profile
PATH=/home/sacha/.local/bin/:$PATH
linode-cli linodes resize $BBB_ID --type g6-standard-4 --allow_auto_disk_resize false
sleep 4m
linode-cli linodes boot $BBB_ID
sleep 3m
ssh root@bbb.emacsverse.org "bbb-conf --restart; cd ~/greenlight-v3; docker compose restart"
notify-send "Should be ready"

And when I resize it down to a 1 GB nanode, BigBlueButton doesn't get started and the VPS is nice and responsive when I SSH in.

#!/bin/bash
source /home/sacha/.profile
PATH=/home/sacha/.local/bin/:$PATH
echo Powering off
linode-cli linodes shutdown $BBB_ID
sleep 60
echo "Resizing BBB node to nanode, dormant"
linode-cli linodes resize $BBB_ID --type g6-nanode-1 --allow_auto_disk_resize false

So now I'm going to coordinate with Ihor Radchenko about when he might want to try this out for OrgMeetup, and I can talk to other meetup organizers to figure out times. People will probably want to test things before announcing it to their meetup groups, so we just need to schedule that. It's BigBlueButton 3.0. I'm not 100% confident in the setup. We had some technical issues with some EmacsConf speakers even though we did a tech check with them before we went live with their session. Not sure what happened there.

I'm still a little nervous about accidentally forgetting to downscale the server and running up a bill, but I've scheduled downscaling with the at command before, so that's helpful. If it turns out to be something we want to do regularly, I might even be able to use a cronjob from my other server so that it happens even if my laptop is off, and maybe set up a backup nginx server with a friendly message (and maybe a list of upcoming meetups) in case people connect before it's been scaled up. Anyway, I think that's a totally good use of part of the Google Open Source Peer Bonus I received last year.

As an aside, you can change a room's friendly_id to something actually friendly. In the Rails console (docker exec -it greenlight-v3 bundle exec rails console), you could do something like this:

Room.find_by(friendly_id: "CURRENT_ROOM_ID").update_attribute(:friendly_id, "NEW_CUSTOM_ID")

Anyway, let me know if you organize an Emacs meetup and want to give this BigBlueButton instance a try!

View org source for this post

EmacsConf 2024 notes

Posted: - Modified: | emacs, emacsconf

The videos have been uploaded, thank-you notes have been sent, and the kiddo has decided to play a little Minecraft on her own, so now I get to write some quick notes on EmacsConf 2024.

Stats

Talks 31
Hours 10.7
Q&A web conferences 21
Hours 7.8
  • Saturday:
    • gen: 177 peak + 14 peak lowres
    • dev: 226 peak + 79 peak lowres
  • Sunday:
    • gen: 89 peak + 10 peak lowres

Server configuration:

meet 16GB 8core dedicated peak 409% CPU (100% is 1 CPU), average 69.4%
front 32GB 8core shared peak 70.66% CPU (100% is 1 CPU)
live 64GB 16core shared peak 552% CPU (100% is 1 CPU) average 144%
res 46GB 12core peak 81.54% total CPU (100% is 12 CPUs); each OBS ~250%), mem 7GB used
media 3GB 1core  

YouTube livestream stats:

Shift Peak Avg
Gen Sat AM 46 28
Gen Sat PM 24 16
Dev Sat AM 15 7
Dev Sat PM 20 12
Gen Sun AM 28 17
Gen Sun PM 26 18

Timeline

Call for proposals [2024-06-30 Sun]
CFP deadline [2024-09-20 Fri]
Speaker notifications [2024-09-27 Fri]
Publish schedule [2024-10-25 Fri]
Video target date [2024-11-08 Fri]
EmacsConf [2024-12-07 Sat]-[2024-12-07 Sat]

We did early acceptances again this year. That was nice. I wasn't sure about committing longer periods of time early in the scheduling process, so I usually tried to nudge people to plan a 20-minute video with the option of possibly doing more, and I okayed longer talks once we figured out what the schedule looked like.

There were 82 days between the call for proposals and the CFP deadline, another 49 days from that to the video target date, and 29 days between the video target date and EmacsConf. It felt like there was a good amount of time for proposals and videos. Six videos came in before or on the target date. The rest trickled in afterwards, which was fine because we wanted to keep things low-pressure for the speakers. We had enough capacity to process and caption the videos as they came in.

Data

We continued to use an Org file to store the talk information. It would be great to add some validation functions:

  • Check permissions and ownership for files
  • Check case sensitivity for Q&A type detection
  • Check BBB redirect pages to make sure they exist
  • Check transcripts for ` because that messes up formatting; consider escaping for the wiki
  • Check files are public and readable
  • Check captioned by comment vs caption status vs captioner

Speakers uploaded their files via PsiTransfer again. I didn't get around to setting up the FTP server. I should probably rename ftp-upload.emacsconf.org to upload.emacsconf.org so that people don't get confused.

Communication

As usual, we announced the EmacsConf call for proposals on emacs-tangents, Emacs News, emacsconf-discuss, emacsconf-org, https://reddit.com/r/emacs. System Crafters, Irreal, and Emacs APAC, mentioned it, and people also posted about EmacsConf on Mastodon, X, BlueSky, and Facebook. @len@toot.si suggested submitting EmacsConf to https://foss.events, so I did. There was some other EmacsConf-related discussions in r/emacs. 200ok and Ardeo organized an in-person meetup in Switzerland, and emacs.si got together in Ljubljana.

For communicating with speakers and volunteers, I used lots of mail merge (emacsconf-mail.el). Most of the templates only needed a little tweaking from last year's code. I added a function to help me double-check delivery, since the batches that I tried to send via async sometimes ran into errors.

Next time, I think it could be interesting to add more blog posts and Mastodon toots.

Also, maybe it would be good to get in touch with podcasts like

to give a heads up on EmacsConf before it happens and also let them know when videos are available.

We continued to use Mumble for backstage coordination. It worked out well.

Schedule

The schedule worked out to two days of talks, with two tracks on the first day, and about 15-20 minutes between each talk. We were able to adapt to late submissions, last-minute cancellations, and last-minute switches from Q&A to live.

We added an open mic session on Sunday to fill in the time from a last-minute cancellation. That worked out nicely and it might be a good idea to schedule in that time next year. It was also good to move some of the usual closing remarks earlier. We were able to wrap up in a timely manner, which was great for some hosts and participants because they didn't have to stay up so late.

Sunday was single-track, so it was nice and relaxed. I was a little worried that people might get bored if the current talk wasn't relevant to their interests, but everyone managed just fine. I probably should have remembered that Emacs people are good at turning extra time into more configuration tweaks.

Most of the scheduling was determined by people's time constraints, so I didn't worry too much about making the talks flow logically. I accidentally forgot to note down one speaker's time constraints, but he caught it when we e-mailed the draft schedule and I was able to move things around for a better time for him.

There was a tiny bit of technical confusion because the automated schedule publishing on res had case-sensitive matching (case-fold-search was set to nil), so if a talk was set to "Live" Q&A, it didn't announce it as a live talk because it was looking for live. Whoops. I've added that configuration setting to my emacsconf-stream-config.el, so the ansible scripts should get it next time.

I asked Leo and Corwin if they wanted to manually control the talks this year. They opted to leave it automatically managed by crontab so that they wouldn't have to worry as much about timekeeping. It worked reliably. Hooray for automation! The only scheduling hiccup was because I turned off the crontab so that we could do Saturday closing remarks when we wanted to and I forgot to reenable autopilot the next day. We noticed when the opening remarks didn't start right on the dot, and I got everything back on track.

Like last year, I scheduled the dev track to start a little later than the gen track. That made for a less frantic morning. Also, this year we scheduled Sunday morning to start with more IRC Q&A instead of live Q&A. We didn't notice any bandwidth issues on Sunday morning this time.

It would be nice to have Javascript countdowns in some kind of web interface to make it easier for hosts, especially if we can update it with the actual time the current video will end in MPV.

I can also update the emacsconf-stream.el code to make it easier to automatically count down to the next talk or to a specific talk.

We have Javascript showing local time on the individual talk pages, but it would be nice to localize the times on all the schedule/watch pages too.

Most of my stuff (scheduling, publishing, etc.) is handled by automation with just a little bit of manual nudging every so often, so it might be possible to organize an event that's more friendly to Europe/APAC timezones.

Recorded videos

As usual, we strongly encouraged speakers to record videos to lower everyone's stress levels and allow for captioning by volunteers, so that's what most speakers did. We were able to handle a few last-minute submissions as well as a live talk. Getting videos also meant we could publish them as each talk went live, including automatically putting the videos and transcripts on the wiki.

We didn't have obvious video encoding cut-offs, so re-encoding in a screen was a reliable way to avoid interruptions this year. Also, no one complained about tiny text or low resolution, so the talk preparation instructions seem to be working out.

Automatically normalizing the audio with ffmpeg-normalize didn't work out, so Leo Vivier did a last-minute scramble to normalize the audio the day before the conference. Maybe that's something that volunteers can help with during the lead-up to the conference, or maybe I can finally figure out how to fit that into my process. I don't have much time or patience to listen to things, but it would be nice to get that sorted out early.

Next year we can try remixing the audio to mono. One of the talks had some audio moving around, which was a little distracting. Also, some people listen to the talks in one ear, so it would be good to drop things down to mono for them.

We think 60fps videos stressed the res server a bit, resulting in dropped frames. Next year, we can downsample those to 30fps and add a note to the talk preparation instructions. The hosts also suggested looking into setting up streaming from each host's computer instead of using our shared VNC sessions.

There was some colour smearing and weirdness when we played some videos with mpv on res. Upgrading MPV to v0.38 fixed it.

Some people requested dark mode (light text on dark background), so maybe we can experiment with recommending that next year.

I did a last-minute change to the shell scripts to load resources from the cache directory instead of the assets/stream directory, but I didn't get all of the file references, so sometimes the test videos played or the introductions didn't have captions. On the plus side, I learned how to use j in MPV to reload a subtitle file.

Sometimes we needed to play the videos manually. If we get the hang of starting MPV in a screen or tmux session, it might be easier for hosts to check how much time is left, or to restart a video at a specific point if needed. Leo said he'll work on figuring out the configuration and the Lua scripts.

I uploaded all the videos to YouTube and scheduled them. That was nice because then I didn't have to keep updating things during the conference. It turns out that Toobnix also has a way to schedule uploads. I just need to upload it as unlisted first, and then choose Scheduled from the visibility. I wonder if peertube-cli can be extended to schedule things. Anyway, since I didn't know about that during the conference, I just used emacsconf-publish-upload-talk function to upload videos.

It was fun playing Interview with an Emacs Enthusiast in 2023 [Colorized] - YouTube at lunch. I put together some captions for it after the conference, so maybe we can play it with captions next year.

Recorded introductions

We record introductions so that hosts don't have to worry about how to say things on air. I should probably send the intro check e-mail earlier–maybe on the original video target date, even if speakers haven't submitted their videos yet. This will reduce the last-minute scramble to correct intros.

When I switched the shell scripts to use the cache directory, I forgot to get it to do the intros from that directory as well, so some of the uncorrected intros were played.

I forgot to copy the intro VTTs to the cache directory. This should be handled by the subed-record process for creating intros, so it'll be all sorted out next year.

Captioning

We used WhisperX for speech-to-text this year. It did a great job at preparing the first drafts of captions that our wonderful army of volunteer captioners could then edit. WhisperX's built-in voice activity detection cut down a lot on the hallucinations that OpenAI Whisper had during periods of silence in last year's captions, and there was only one instance of WhisperX missing a chunk of text from a speaker that I needed to manually fill in. I upgraded to a Lenovo P52 with 64GB RAM, so I was able to handle last-minute caption processing on my computer. It might be handy to have a smaller model ready for those last-minute requests, or have something ready to go for the commercial APIs.

The timestamps were a little bit off. It was really helpful that speakers and volunteers used the backstage area to check video quality. I used Aeneas to re-align the text, but Aeneas was also confused by silences. I've added some code to subed so that I can realign regions of subtitles using Aeneas or WhisperX timestamps, and I also wrote some code to skim timestamps for easy verification.

Anush V experimented with using machine learning for subtitle segmentation, so that might be something to explore going forward.

BigBlueButton web conference

This year we set up a new BigBlueButton web conferencing server. The server with our previous BigBlueButton instance had been donated by a defunct nonprofit, so it finally got removed on October 27. After investigating whether Jitsi or Galene might be a good fit for EmacsConf, we decided to continue with BigBlueButton. There were some concerns about non-free Mongo for BBB versions >= 2.3 and < 3, so I installed BBB 3.0. This was hard to get working on a Docker on the existing res server. We decided it was worth spinning up an additional Linode virtual private server. It turned out that BBB refused to run on anything smaller than 8GB/4core, so I scaled up to that during testing, scaled back down to 1GB/1core in between, and scaled up to 16GB/8core dedicated during the conference.

I'm still not 100% sure I set everything up correctly or that everything was stable. Maybe next year BBB 3.0 will be better-tested, someone more sysad-y can doublecheck the setup, or we can try Galene.

One of the benefits of upgrading to BBB 3.0 was that we could use the smart layout feature to drag the webcam thumbnails to the side of the shared screen. This made shared screens much easier to read. I haven't automated this yet, but it was easy enough for us to do via the shared VNC session.

On the plus side, it was pretty straightforward to use the Rails console to create all the rooms. We used moderator access codes to give all the speakers moderator access. Mysteriously, superadmins didn't automatically have moderator access to all the rooms even if they were logged in, so we needed to add host access by hand so that they could start the recordings.

Since we self-hosted and were budgeting more for the full-scale node, I didn't feel comfortable scaling it up to production size until a few days before the conference. I sent the access codes with the check-in e-mails to give speakers time to try things out.

Compared to last year's stats:

  2023 2024
Max number of simultaneous users 62 107
Max number of simultaneous meetings 6 7
Max number of people in one meeting 27 25
Total unique people 84 102
Total unique talking 36 40

(Max number of simultaneous users wasn't deduplicated, since we need that number for server load planning)

Tech checks and hosting

FlowyCoder did a great job getting everyone checked in, especially once I figured out the right checklist to use. We used people's emergency contact information a couple of times.

Corwin and Leo were able to jump in and out of the different streams for hosting. Sometimes they were both in the same Q&A session, which made it more conversational especially when they were covering for technical issues. We had a couple of crashes even though the tech checks went fine, so that was weird. Maybe something's up with BBB 3.0 or how I set it up.

Next time, we can consider asking speakers what kind of facilitation style they like. A chatty host? Someone who focuses on reading the questions and then gets out of the way? Speakers reading their own questions and the host focusing on timekeeping/troubleshooting?

Streaming

I experimented with setting up the live0 streaming node as a 64GB 32core dedicated CPU server, but that was overkill, so we went back down to 64GB 16core and it still didn't approach the CPU limits.

The 480p stream seemed stable, hooray! I had set it up last year to automatically kick in as soon as I started streaming to Icecast, and that worked out. I think I changed a loop to be while true instead of making it try 5 times, so that probably helped.

I couldn't get Toobnix livestreaming to work this year. On the plus side, that meant that I could use OBS to directly stream to YouTube instead of trying to set up multicasting. I set up one YouTube livestreaming event for each shift and added the RTMP keys to our shift checklists so that I could update the settings before starting the stream. That was pretty straightforward.

This year, I wrote a little randomizer function to display things on the countdown screen. At first I just dumped in https://www.gnu.org/fun/jokes/gnuemacs.acro.exp.en.html, but some of those were not quite what I was looking for. (… Probably should've read them all first!) Then I added random packages from GNU ELPA and NonGNU ELPA, and that was more fun. I might add MELPA next time too. The code for dumping random packages is probably worth putting into a different blog post, since it's the sort of thing people might like to add to their dashboards or screensavers.

I ran into some C-s annoyances in screen even with flow control turned off, so it might be a good idea to switch to tmux instead of screen.

Next year, I think it might be a good idea to make intro images for each talk. Then we can use that as the opening slide in BigBlueButton (unless they're already sharing something else) as well as a video thumbnail.

Publishing

The automated process for publishing talks and transcripts to the wiki occasionally needed nudging when someone else had committed a change to the wiki. I thought I had a git pull in there somewhere, but maybe I need to look at it some more.

I forgot to switch the conference publishing phase and enable the inclusion of Etherpads, but fortunately Ihor noticed. I did some last-minute hacking to add them in, and then I remembered the variables I needed to set. Just need to add it to our process documentation.

Etherpad

We used Etherpad 1.9.7 to collect Q&A again this year. I didn't upgrade to Etherpad v2.x because I couldn't figure out how to get it running within the time I set aside for it, but maybe that's something for next year.

I wrote some Elisp to copy the current ERC line (unwrapped) for easier pasting into Etherpad. That worked out really well, and it let me keep up with copying questions from IRC to the pad in between other bits of running around. (emacsconf-erc-copy in emacsconf-erc.el)

Next year, I'll add pronouns and pronunciations to the Etherpad template so that hosts can refer to them easily.

If I rejig the template to move the next/previous links so that notes can be added to the end, I might be able to use the Etherpad API to add text from IRC.

IRC

We remembered to give the libera.chat people a heads-up before the conference, so we didn't run into usage limits for https://chat.emacsconf.org. Yay!

Aside from writing emacsconf-erc-copy (emacsconf-erc.el) to make it easier to add text from IRC to the Etherpad, I didn't tinker much with the IRC setup for this year. It continued to be a solid platform for discussion.

I think a keyboard shortcut for inserting a talk's URL could be handy and should be pretty easy to add to my Embark keymap.

Extracting the Q&A

We sometimes forgot to start the recording for the Q&A until a few minutes into the talk. I considered extracting the Q&A recordings from the Icecast dump or YouTube stream recordings in order to get those first few minutes, but decided it wasn't worth it since people could generally figure out the answers.

Getting the recordings off BigBlueButton was easier this year because I configured it with video as an additional processing format, so we could grab one file per session instead of combining the different streams with ffmpeg.

I did a quick pass of the Q&A transcripts and chat logs to see if people mentioned anything that they might want to take out. I also copied IRC messages and the pads, and I copied over the answers from the transcripts using the new emacsconf-extract-subed-copy-section-text function.

Audio mixing was uneven. It might be nice to figure out separate audio recordings just in case (#12302, bigbluebutton-dev). We ended up not tinkering with the audio for the Q&A, so next time, I can probably upload them without waiting to see if anyone wants to fiddle with the audio.

Trimming the Q&A was pretty straightforward. I added a subed-crop-media-file function to subed so that I can trim files easily.

Thanks to my completion functions for adding section headings based on comments, it was easy to index the Q&A this year. I didn't even put it up backstage for people to work on.

Nudged by @ctietze, I'm experimenting with adding sticky videos if Javascript is enabled so that it's easier to navigate using the transcript. There's still a bit of tinkering to do, but it's a start.

I added some conference-related variables to a .dir-locals.el file so that I can more easily update things even for past conferences. This is mostly related to publishing the captions on the wiki pages, which I do with Emacs Lisp.

Budget and donations

Costs (USD, not including 13% tax):

52.54 Extra costs for hosting in December
3.11 Extra costs for BBB testing in November
120 Hosting costs year-round (two Linode nanodes)

Total of USD 175.65 + tax, or USD 198.48 for 2024.

The Free Software Foundation also provided media.emacsconf.org for serving media files. Ry P provided res.emacsconf.org for OBS streaming over VNC sessions.

Amin Bandali was away during the conference weekend and no one else knew how to get the list of donors and current donation stats from the FSF Working Together program on short notice. Next time, we can get that sorted out beforehand so that we can thank donors properly.

Documentation and time

I think my biggest challenge was having less time to prepare for EmacsConf this year because the kiddo wanted more of my attention. In many ways, the automation that I'd been gradually building up paid off. We were able to pull together EmacsConf even though I had limited focus time.

Here's my Emacs-related time data (including Emacs News and tweaking my config):

Year Jan Feb March April May June July Aug Sept Oct Nov Dec Total
2023 23.4 15.9 16.2 11.2 4.4 11.5 6.5 13.3 36.6 86.6 93.2 113.0 432
2024 71.2 12.0 5.6 6.6 3.3 9.6 11.0 4.7 36.0 40.3 52.3 67.7 320

(and here's a longer-term analysis going back to 2012.)

I spent 92.6 hours total in October and November 2024 doing Emacs-related things, compared to 179.8 hours the previous year – so, around half the time. Part of the 2023 total was related to preparing my presentation for EmacsConf, so I was much more familiar with my scripts then. Apparently, there was still a lot more that I needed to document. As I scrambled to get EmacsConf sorted out, I captured quick tasks/notes for the things I need to add to our organizers notebook. Now I get to go through all those notes in my inbox. Maybe next year will be even smoother.

On the plus side, all the process-related improvements meant that the other volunteers could jump in pretty much whenever they wanted, including during the conference itself. I didn't want to impose firm commitments on people or bug them too much by e-mail, so we kept things very chill in terms of scheduling and planning. If people were available, we had stuff people could help with. If people were busy, that was fine, we could manage. This was nice, especially when I applied the same sort of chill approach to myself.

I'd like to eventually get to the point of being able to mostly follow my checklists and notes from the start of the conference planning process to the end. I've been moving notes from year-specific organizer notebooks to the main organizers' notebook. I plan to keep that one as the main file for notes and processes, and then to have specific dates and notes in the yearly ones.

Thanks

  • Thank you to all the speakers, volunteers, and participants, and to all those other people in our lives who make it possible through time and support.
  • Thanks to Leo Vivier and Corwin Brust for hosting the sessions, and to FlowyCoder for checking people in.
  • Thanks to our proposal review volunteers James Howell, JC Helary, and others for helping with the early acceptance process.
  • Thanks to our captioning volunteers: Mark Lewin, Rodrigo Morales, Anush, annona, and James Howell, and some speakers who captioned their own talks.
  • Thanks to Leo Vivier for fiddling with the audio to get things nicely synced.
  • Thanks to volunteers who kept the mailing lists free from spam.
  • Thanks to Bhavin Gandhi, Christopher Howard, Joseph Turner, and screwlisp for quality-checking.
  • Thanks to shoshin for the music.
  • Thanks to Amin Bandali for help with infrastructure and communication.
  • Thanks to Ry P for the server that we're using for OBS streaming and for processing videos.
  • Thanks to the Free Software Foundation for Emacs itself, the mailing lists, the media.emacsconf.org server, and handling donations on our behalf through the FSF Working Together program. https://www.fsf.org/working-together/fund
  • Thanks to the many users and contributers and project teams that create all the awesome free software we use, especially: BigBlueButton, Etherpad, Icecast, OBS, TheLounge, libera.chat, ffmpeg, OpenAI Whisper, WhisperX, the aeneas forced alignment tool, PsiTransfer, subed, and many, many other tools and services we used to prepare and host this years conference
  • Thanks to everyone!

Overall

Good experience. Lots of fun. I'd love to do it again next year. EmacsConf feels like a nice, cozy get-together where people share the cool things they've been working on and thinking about. People had fun! They said:

  • "emacsconf is absolutely knocking it out of the park when it comes to conference logistics"
  • "I think this conference has defined the terms for a successful online conference."
  • "EmacsConf is one of the big highlights of my year every year. Thank you a ton for running this 😊"

It's one of the highlights of my year too. =) Looking forward to the next one!

In the meantime, y'all can stay connected via Emacs News, meetups (online and in person), Planet Emacslife, and now emacs.tv. Enjoy!

p.s. I'd love to learn from other people's conference blog posts, EmacsConf or otherwise. I'm particularly interested in virtual conferences and how we can tinker with them to make them even better. I'm having a hard time finding posts; please feel free to send me links to ones you've liked or written!

View org source for this post

EmacsConf backstage: Makefile targets

Posted: - Modified: | emacsconf

[2024-11-16 Sat]: Removed highlight_words from whisperx call.

We like to use pre-recorded videos at EmacsConf to minimize technical risks. This also means we can caption them beforehand, stream them with open captions, and publish them as soon as the talk goes live.

Here's the process:

  1. Speakers upload their videos in whatever format they like. We use PsiTransfer to accept the uploaded files.
  2. We rename the files to have the talk title and speaker name in the filename, like emacsconf-2024-emacs30--emacs-30-highlights--philip-kaludercic--original.mov.
  3. We use FFmpeg to reencode them to WEBM so that everything is available in a free format, and we replace the --original.* part with --reencoded.webm. We copy this to --main.webm as a starting point.
  4. We extract the audio and save it to --reencoded.opus.
  5. We use ffmpeg-normalize to normalize the audio and save it to --normalized.opus.
  6. We use WhisperX to get a reasonable starting point for captions, which we save to --reencoded.vtt. I remove the underlines and the tsv and srt files.
  7. Someone edits the captions. We save edited captions as --main.vtt.
  8. --normalized.opus and --main.vtt get combined into --main.webm.

I've been slowly learning how to set up Makefile rules to automate more and more of this. Let's go through parts of the roles/prerec/templates/Makefile.

Make the reencoded webm from the original MP4, MOV, MKV, or WEBM

Here's the rule that makes a --reencoded.webm based on the original mp4, mov, mkv, or webm.

VIDEO_EXTS = mp4 mkv webm mov
source_patterns = $(foreach ext,$(VIDEO_EXTS),$(1)--original.$(ext))
emacsconf-%--reencoded.webm: SOURCES = $(call source_patterns, emacsconf-$*)
emacsconf-%--reencoded.webm:
  $(eval SOURCE := $(lastword $(sort $(wildcard $(SOURCES)))))
  @if [ -z "$(SOURCE)" ]; then \
    echo "No source file found for $@"; \
    echo "Tried: $(SOURCES)"; \
    exit 1; \
  fi
  @echo "Using source: $(SOURCE)"
  ./reencode-in-screen.sh "$(SOURCE)"

Reencoding can take a while and it's prone to me accidentally breaking it, so we stick it in a GNU screen so that I don't accidentally quit it. This is reencode-in-screen.sh:

#!/bin/bash
ORIGINAL=$1
BASE="${ORIGINAL%--original.*}"
REENCODED="${BASE}--reencoded.webm"
SLUG=$(echo "$ORIGINAL" | perl -ne '/^emacsconf-[0-9]*-(.*?)--/ && print $1')
LOCK=".lock-$SLUG"

if [ ! -f "$REENCODED" ]; then
    if [  -f "$LOCK" ]; then
        echo "$LOCK already exists, waiting for it"
    else
        touch "$LOCK"
        screen -dmS reencode-$SLUG /bin/bash -c "reencode.sh \"$ORIGINAL\" \"$REENCODED\" && thumbnail.sh \"$MAIN\" && rm \"$LOCK\""
        echo "Processing $REENCODED in reencode-$SLUG"
    fi
fi

which calls roles/prerec/templates/reencode.sh. Here's the templatized version from Ansible:

#!/usr/bin/env bash

set -euo pipefail

# Defaults
q={{ reencode_quality }}
cpu={{ reencode_cpu }}
time_limit=""
print_only=false
limit_resolution={{ res_y }}
limit_fps={{ fps }}

while getopts :q:c:t:s OPT; do
    case $OPT in
        q|+q)
            q="$OPTARG"
            ;;
        c|+c)
            cpu="$OPTARG"
            ;;
        t|+t)
            time_limit="-to $OPTARG"
            ;;
        s)
            print_only=true
            ;;
        *)
            echo "usage: `basename $0` [+-q ARG] [+-c ARG} [--] ARGS..."
            exit 2
    esac
done
shift `expr $OPTIND - 1`
OPTIND=1

input="$1"
output="${2:-$(echo $input | sed 's/--original.*/--reencoded.webm/')}"

command="$(cat<<EOF
ffmpeg -y -i "$input" $time_limit \
       -vf "scale='-1':'min($limit_resolution,ih)',
            fps='$limit_fps'" \
       -c:v libvpx-vp9 -b:v 0 -crf $q -an \
       -row-mt 1 -tile-columns 2 -tile-rows 2 -cpu-used $cpu -g 240 \
       -pass 1 -f webm -threads $cpu /dev/null &&
    ffmpeg -y -i "$input" $time_limit \
           -vf "scale='-1':'min($limit_resolution,ih)',
                fps='$limit_fps'" \
               -c:v libvpx-vp9 -b:v 0 -crf $q -c:a libopus \
               -row-mt 1 -tile-columns 2 -tile-rows 2 -cpu-used $cpu \
               -pass 2 -threads $cpu -- "$output"
EOF
)"

if [ $print_only == true ]; then
    echo "$command"
else
    eval "$command"
fi

Process the audio and captions

Processing the audio is relatively straightforward.

emacsconf-%--reencoded.opus: emacsconf-%--reencoded.webm
  ffmpeg -i "$<" -c:a copy "$@"

emacsconf-%--normalized.opus: emacsconf-%--reencoded.opus
  ffmpeg-normalize "$<" -ofmt opus -c:a libopus -o "$@"

emacsconf-%--reencoded.vtt: emacsconf-%--reencoded.opus
  whisperx --model large-v2 --align_model WAV2VEC2_ASR_LARGE_LV60K_960H --compute_type int8 --print_progress True --max_line_width 50 --segment_resolution chunk --max_line_count 1 --language en "$<"

After this, we need to manually process the --reencoded.vtt and then eventually save the edited version as --main.vtt.

Combine the video, audio, and subtitles

The next part of the Makefile creates the --main.webm from the reencoded, normalized, and edited files, or from just the --reencoded.webm if that's all that's available.

emacsconf-%--main.webm: emacsconf-%--reencoded.webm emacsconf-%--normalized.opus emacsconf-%--main.vtt
  ffmpeg -i emacsconf-$*--reencoded.webm -i emacsconf-$*--normalized.opus -i emacsconf-$*--main.vtt \
    -map 0:v -map 1:a -c:v copy -c:a copy \
    -map 2 -c:s webvtt -y \
    $@

emacsconf-%--main.webm: emacsconf-%--reencoded.webm
  cp "$<" "$@"

This works because the Makefile picks the most specific set of dependencies.

Making all the files based on the original ones that are available

Finally, we need some rules to make various things. We do this with a wildcard match for all the original files, and then we make a list without the --original.*. After that, we can just use addsuffix to add the different file endings.

PRERECS_ORIGINAL := $(wildcard emacsconf-*--original.*)
PREFIXES := $(shell for f in $(PRERECS_ORIGINAL); do echo "$${f%--original.*}"; done)
PRERECS_REENCODED := $(addsuffix --reencoded.webm, $(PREFIXES))
PRERECS_OPUS := $(addsuffix --reencoded.opus, $(PREFIXES))
PRERECS_NORMAL := $(addsuffix --normalized.opus, $(PREFIXES))
PRERECS_MAIN := $(addsuffix --main.webm, $(PREFIXES))
PRERECS_CAPTIONS := $(addsuffix --reencoded.vtt, $(PREFIXES))

all: reencoded opus normal main
reencoded: $(PRERECS_REENCODED)
opus: $(PRERECS_OPUS)
normal: $(PRERECS_NORMAL)
captions: $(PRERECS_CAPTIONS)
main: $(PRERECS_MAIN)

I sometimes do the captions on my computer, so I've left them out of the all target.

Seems to be doing all right so far. It's nice having the Makefile figure out what's changed and what needs to be updated.

View org source for this post

EmacsConf backstage: making lots of intro videos with subed-record

| emacsconf, subed, emacs

Summary (735 words): Emacs is a handy audio/video editor. subed-record can combine multiple audio files and images to create multiple output videos.

Watch on YouTube

It's nice to feel like you're saying someone's name correctly. We ask EmacsConf speakers to introduce themselves in the first few seconds of their video, but people often forget to do that, so that's okay. We started recording introductions for EmacsConf 2022 so that stream hosts don't have to worry about figuring out pronunciation while they're live. Here's how I used subed-record to turn my recordings into lots of little videos.

First, I generated the title images by using Emacs Lisp to replace text in a template SVG and then using Inkscape to convert the SVG into a PNG. Each image showed information for the previous talk as well as the upcoming talk. (emacsconf-stream-generate-in-between-pages)

emacsconf.svg.png
Figure 1: Sample title image

Then I generated the text for each talk based on the title, the speaker names, pronunciation notes, pronouns, and type of Q&A. Each introduction generally followed the pattern, "Next we have title by speakers. Details about Q&A." (emacsconf-pad-expand-intro and emacsconf-subed-intro-subtitles below)

00:00:00.000 --> 00:00:00.999
#+OUTPUT: sat-open.webm
[[file:/home/sacha/proj/emacsconf/2023/assets/in-between/sat-open.svg.png]]
Next, we have "Saturday opening remarks".

00:00:05.000 --> 00:00:04.999
#+OUTPUT: adventure.webm
[[file:/home/sacha/proj/emacsconf/2023/assets/in-between/adventure.svg.png]]
Next, we have "An Org-Mode based text adventure game for learning the basics of Emacs, inside Emacs, written in Emacs Lisp", by Chung-hong Chan. He will answer questions via Etherpad.

I copied the text into an Org note in my inbox, which Syncthing copied over to the Orgzly Revived app on my Android phone. I used Google Recorder to record the audio. I exported the m4a audio file and a rough transcript, copied them back via Syncthing, and used subed-record to edit the audio into a clean audio file without oopses.

Each intro had a set of captions that started with a NOTE comment. The NOTE comment specified the following:

  • #+AUDIO:: the audio source to use for the timestamped captions that follow
  • [[file:...]]: the title image I generated for each talk. When subed-record-compile-video sees a comment with a link to an image, video, or animated GIF, it takes that visual and uses it for the span of time until the next visual.
  • #+OUTPUT: the file to create.
NOTE #+OUTPUT: hyperdrive.webm
[[file:/home/sacha/proj/emacsconf/2023/assets/in-between/hyperdrive.svg.png]]
#+AUDIO: intros-2023-11-21-cleaned.opus

00:00:15.680 --> 00:00:17.599
Next, we have "hyperdrive.el:

00:00:17.600 --> 00:00:21.879
Peer-to-peer filesystem in Emacs", by Joseph Turner

00:00:21.880 --> 00:00:25.279
and Protesilaos Stavrou (also known as Prot).

00:00:25.280 --> 00:00:27.979
Joseph will answer questions via BigBlueButton,

00:00:27.980 --> 00:00:31.080
and Prot might be able to join depending on the weather.

00:00:31.081 --> 00:00:33.439
You can join using the URL from the talk page

00:00:33.440 --> 00:00:36.320
or ask questions through Etherpad or IRC.

NOTE
#+OUTPUT: steno.webm
[[file:/home/sacha/proj/emacsconf/2023/assets/in-between/steno.svg.png]]
#+AUDIO: intros-2023-11-19-cleaned.opus

00:03:23.260 --> 00:03:25.480
Next, we have "Programming with steno",

00:03:25.481 --> 00:03:27.700
by Daniel Alejandro Tapia.

NOTE
#+AUDIO: intro-2023-11-29-cleaned.opus

00:00:13.620 --> 00:00:16.580
You can ask your questions via Etherpad and IRC.

00:00:16.581 --> 00:00:18.079
We'll send them to the speaker

00:00:18.080 --> 00:00:19.919
and post the answers in the talk page

00:00:19.920 --> 00:00:21.320
after the conference.

I could then call subed-record-compile-video to create the videos for all the intros, or mark a region with C-SPC and then subed-record-compile-video only the intros inside that region.

Sample intro

Using Emacs to edit the audio and compile videos worked out really well because it made it easy to change things.

  • Changing pronunciation or titles: For EmacsConf 2023, I got the recordings sorted out in time for the speakers to correct my pronunciation if they wanted to. Some speakers also changed their talk titles midway. If I wanted to redo an intro, I just had to rerecord that part, run it through my subed-record audio cleaning process, add an #+AUDIO: comment specifying which file I want to take the audio from, paste it into my main intros.vtt, and recompile the video.
  • Cancelling talks: One of the talks got cancelled, so I needed to update the images for the talk before it and the talk after it. I regenerated the title images and recompiled the videos. I didn't even need to figure out which talk needed to be updated - it was easy enough to just recompile all of them.
  • Changing type of Q&A: For example, some speakers needed to switch from answering questions live to answering them after the conference. I could just delete the old instructions, paste in the instructions from elsewhere in my intros.vtt (making sure to set #+AUDIO to the file if it came from a different take), and recompile the video.

And of course, all the videos were captioned. Bonus!

So that's how using Emacs to edit and compile simple videos saved me a lot of time. I don't know how I'd handle this otherwise. 47 video projects that might all need to be updated if, say, I changed the template? Yikes. Much better to work with text. Here are the technical details.

Generating the title images

I used Inkscape to add IDs to our template SVG so that I could edit them with Emacs Lisp. From emacsconf-stream.el:

emacsconf-stream-generate-in-between-pages: Generate the title images.
(defun emacsconf-stream-generate-in-between-pages (&optional info)
  "Generate the title images."
  (interactive)
  (setq info (or emacsconf-schedule-draft (emacsconf-publish-prepare-for-display (emacsconf-filter-talks (or info (emacsconf-get-talk-info))))))
  (let* ((by-track (seq-group-by (lambda (o) (plist-get o :track)) info))
         (dir (expand-file-name "in-between" emacsconf-stream-asset-dir))
         (template (expand-file-name "template.svg" dir)))
    (unless (file-directory-p dir)
      (make-directory dir t))
    (mapc (lambda (track)
            (let (prev)
              (mapc (lambda (talk)
                      (let ((dom (xml-parse-file template)))
                        (mapc (lambda (entry)
                                (let ((prefix (car entry)))
                                  (emacsconf-stream-svg-set-text dom (concat prefix "title")
                                                 (plist-get (cdr entry) :title))
                                  (emacsconf-stream-svg-set-text dom (concat prefix "speakers")
                                                 (plist-get (cdr entry) :speakers))
                                  (emacsconf-stream-svg-set-text dom (concat prefix "url")
                                                 (and (cdr entry) (concat emacsconf-base-url (plist-get (cdr entry) :url))))
                                  (emacsconf-stream-svg-set-text
                                   dom
                                   (concat prefix "qa")
                                   (pcase (plist-get (cdr entry) :q-and-a)
                                     ((rx "live") "Live Q&A after talk")
                                     ((rx "pad") "Etherpad")
                                     ((rx "IRC") "IRC Q&A after talk")
                                     (_ "")))))
                              (list (cons "previous-" prev)
                                    (cons "current-" talk)))
                        (with-temp-file (expand-file-name (concat (plist-get talk :slug) ".svg") dir)
                          (dom-print dom))
                        (shell-command
                         (concat "inkscape --export-type=png -w 1280 -h 720 --export-background-opacity=0 "
                                 (shell-quote-argument (expand-file-name (concat (plist-get talk :slug) ".svg")
                                                                         dir)))))
                      (setq prev talk))
                    (emacsconf-filter-talks (cdr track)))))
          by-track)))

emacsconf-stream-svg-set-text: Update DOM to set the tspan in the element with ID to TEXT.
(defun emacsconf-stream-svg-set-text (dom id text)
  "Update DOM to set the tspan in the element with ID to TEXT.
If the element doesn't have a tspan child, use the element itself."
  (if (or (null text) (string= text ""))
      (let ((node (dom-by-id dom id)))
        (when node
          (dom-set-attribute node 'style "visibility: hidden")
          (dom-set-attribute (dom-child-by-tag node 'tspan) 'style "fill: none; stroke: none")))
    (setq text (svg--encode-text text))
    (let ((node (or (dom-child-by-tag
                     (car (dom-by-id dom id))
                     'tspan)
                    (dom-by-id dom id))))
      (cond
       ((null node)
        (error "Could not find node %s" id))                      ; skip
       ((= (length node) 2)
        (nconc node (list text)))
       (t (setf (elt node 2) text))))))

Generating the script

From emacsconf-pad.el:

emacsconf-pad-expand-intro: Make an intro for TALK.
(defun emacsconf-pad-expand-intro (talk)
  "Make an intro for TALK."
  (cond
   ((null (plist-get talk :speakers))
    (format "Next, we have \"%s\"." (plist-get talk :title)))
   ((plist-get talk :intro-note)
    (plist-get talk :intro-note))
   (t
    (let ((pronoun (pcase (plist-get talk :pronouns)
                     ((rx "she") "She")
                     ((rx "\"ou\"" "Ou"))
                     ((or 'nil "nil" (rx string-start "he") (rx "him")) "He")
                     ((rx "they") "They")
                     (_ (or (plist-get talk :pronouns) "")))))
      (format "Next, we have \"%s\", by %s%s.%s"
              (plist-get talk :title)
              (replace-regexp-in-string ", \\([^,]+\\)$"
                                        ", and \\1"
                                        (plist-get talk :speakers))
              (emacsconf-surround " (" (plist-get talk :pronunciation) ")" "")
              (pcase (plist-get talk :q-and-a)
                ((or 'nil "") "")
                ((rx "after") " You can ask questions via Etherpad and IRC. We'll send them to the speaker, and we'll post the answers on the talk page afterwards.")
                ((rx "live")
                 (format " %s will answer questions via BigBlueButton. You can join using the URL from the talk page or ask questions through Etherpad or IRC."
                         pronoun
                         ))
                ((rx "pad")
                 (format " %s will answer questions via Etherpad."
                         pronoun
                         ))
                ((rx "IRC")
                 (format " %s will answer questions via IRC in the #%s channel."
                         pronoun
                         (plist-get talk :channel)))))))))

And from emacsconf-subed.el:

emacsconf-subed-intro-subtitles: Create the introduction as subtitles.
(defun emacsconf-subed-intro-subtitles ()
  "Create the introduction as subtitles."
  (interactive)
  (subed-auto-insert)
  (let ((emacsconf-publishing-phase 'conference))
    (mapc
     (lambda (sub) (apply #'subed-append-subtitle nil (cdr sub)))
     (seq-map-indexed
      (lambda (talk i)
        (list
         nil
         (* i 5000)
         (1- (* i 5000))
         (format "#+OUTPUT: %s.webm\n[[file:%s]]\n%s"
                 (plist-get talk :slug)
                 (expand-file-name
                  (concat (plist-get talk :slug) ".svg.png")
                  (expand-file-name "in-between" emacsconf-stream-asset-dir))
                 (emacsconf-pad-expand-intro talk))))
      (emacsconf-publish-prepare-for-display (emacsconf-get-talk-info))))))

View org source for this post

EmacsConf backstage: Trimming the BigBlueButton recordings based on YouTube duration

| emacsconf, emacs, youtube, video

I wanted to get the Q&A sessions up quickly after the conference, so I uploaded them to YouTube and added them to the EmacsConf 2023 playlist. I used YouTube's video editor to roughly guess where to trim them based on the waveforms. I needed to actually trim the source videos, though, so that our copies would be up to date and I could use those for the Toobnix uploads.

My first task was to figure out which videos needed to be trimmed to match the YouTube edits. First, I retrieved the video details using the API and the code that I added to emacsconf-extract.el.

(setq emacsconf-extract-youtube-api-video-details (emacsconf-extract-youtube-get-video-details emacsconf-extract-youtube-api-playlist-items))

Then I made a table comparing the file duration with the YouTube duration, showing rows only if the difference was more than 3 minutes.

(append
 '(("type" "slug" "file duration" "youtube duration" "diff"))
 (let ((threshold-secs (* 3 60))) ; don't sweat small differences
   (seq-mapcat
    (lambda (talk)
      (seq-keep
       (lambda (row)
         (when (plist-get talk (cadr row))
           (let* ((video (emacsconf-extract-youtube-find-url-video-in-list
                          (plist-get talk (cadr row))
                          emacsconf-extract-youtube-api-video-details))
                  (video-duration (if (and video (emacsconf-extract-youtube-duration-msecs video))
                                      (/ (emacsconf-extract-youtube-duration-msecs video) 1000.0)))
                  (file-duration (ceiling
                                  (/ (compile-media-get-file-duration-ms (emacsconf-talk-file talk (format "--%s.webm" (car row))))
                                     1000.0))))
             (when (and video-duration (> (abs (- file-duration video-duration)) threshold-secs))
               (list (car row)
                     (plist-get talk :slug)
                     (and file-duration (format-seconds "%h:%z%.2m:%.2s" file-duration))
                     (and video-duration (format-seconds "%h:%z%.2m:%.2s" video-duration))
                     (emacsconf-format-seconds
                      (abs (- file-duration video-duration))))))))
       '(("main" :youtube-url)
         ("answers" :qa-youtube-url))))
    (emacsconf-publish-prepare-for-display (emacsconf-get-talk-info)))))

Then I got the commands to trim the videos.

 (mapconcat (lambda (row)
              (let ((talk (emacsconf-resolve-talk (elt row 1))))
                (format "ffmpeg -y -i %s--%s.webm -t %s -c copy %s--%s--trimmed.webm"
                        (plist-get talk :file-prefix)
                        (car row)
                        (concat (elt row 3) ".000")
                        (plist-get talk :file-prefix)
                        (car row))))
            (cdr to-trim)
            "\n"))

After quickly checking the results, I copied them over to the original videos, updated the video data in my conf.org, and republished the info pages in the wiki.

The time I spent on figuring out how to talk to the YouTube API feels like it's paying off.