EmacsConf backstage: Figuring out our maximum number of simultaneous BigBlueButton users
| emacsconfUpdate: fix total number of unique users; I flipped the assoc so that the car is the user ID and the cdr is the name
A few people have generously donated money to EmacsConf, so now we're thinking of how to use that money effectively to scale EmacsConf up or help people be happier.
One of the things I'd like to improve is our BigBlueButton web conferencing setup, since fiddling with the screen layout was a little annoying this year. We're using BigBlueButton 2.2, which was released in 2020. The current version is 2.7 and has a few improvements that I think would be very useful.
- Better layouts mean that webcams can be on the left side, leaving more space for the presentation, which means a more pleasant viewing experience and less manual fiddling with the sizes of things.
- Built-in timers could help speakers and hosts easily stay on track.
- A unified WEBM export (instead of separate videos for webcams and screensharing) means less post-processing with ffmpeg, and probably a better layout too.
- The option to share system audio when using a Chromium-based browser means easier multimedia presentations, since setting up audio loopbacks can be tricky.
We'd love to use those improvements at the next EmacsConf, and they might be handy for the handful of other Emacs meetups who use our BigBlueButton setup from time to time. I think reducing the mental load from managing screen layouts might be an important step towards making it possible to have a third track.
The current BigBlueButton is a 6-core 3.4GHz virtual machine with 8 GB RAM. During EmacsConf 2023, the CPU load stayed at around 35%, with 4 GB memory used. It idles at 3% CPU and about 3 GB RAM. We have ssh access to an account with sudo, but no higher-level access in case that breaks or in case we mess up upgrading the underlying Ubuntu distribution too, which we should because it's reached its support end-of-life.
BigBlueButton's website recommends installing 2.7 on a clean, dedicated system instead of trying to do the upgrade in place. It requires a major version upgrade to at least Ubuntu 20.04, and it recommends 16 GB memory and 8 CPU cores.
System administration isn't my current cup of tea, and the other organizers might be busy.
Some choices we're thinking about are:
- Continue with our current 2.2 setup, just hack better layouts into it with Tampermonkey or something: probably not a very good choice from the perspective of being a good citizen of the Internet, since the system's out of date
- Try to upgrade in place and hope we don't break anything: one of the other organizers is willing to add this to his maybe-do list
- Install 2.7 on a new node, try to migrate to it to figure out the process, and then maybe consider spinning up a new node during EmacsConf, adding it to our hosting costs budget
- Pay for BigBlueButton hosting: might be worth it if no one wants to take on the responsibility for managing BBB ourselves
- Switch to hosted Jitsi: recording might be trickier
Commercial BigBlueButton hosts tend to charge based on the number of simultaneous users and the number of rooms.
It's been nice having one room per group of speakers because then we can e-mail speakers their personal URL for testing and checking in, the scripts can join the correct room automatically, we never have to worry about time, and all the recordings are split up. In previous years, we rotated among a set of five rooms, but then we needed to keep track of who was using which rooms. I think going with multiple rooms makes sense.
So it mostly comes down to the number of simultaneous users. I rsynced
/var/bbb/recording/raw and cross-referenced each talk with its BBB
meeting using slugs I'd added to the meeting title, disambiguating
them as needed. Then I could use the following function from
emacsconf-extract.el:
Report on simultaneous users
(defun emacsconf-extract-bbb-report () (let* ((max 0) (participant-count 0) (meeting-count 0) (max-meetings 0) (max-participants 0) meeting-participants (meeting-events (sort (seq-mapcat (lambda (talk) (when (plist-get talk :bbb-meeting-id) (let ((dom (xml-parse-file (emacsconf-extract-bbb-raw-events-file-name talk))) participants talking meeting-events) (mapc (lambda (o) (pcase (dom-attr o 'eventname) ("ParticipantJoinEvent" (cl-pushnew (cons (dom-text (dom-by-tag o 'userId)) (dom-text (dom-by-tag o 'name))) participants) (push (cons (string-to-number (dom-text (dom-by-tag o 'timestampUTC))) (dom-attr o 'eventname)) meeting-events)) ("ParticipantLeftEvent" (when (string= (dom-attr o 'module) "PARTICIPANT") (push (cons (string-to-number (dom-text (dom-by-tag o 'timestampUTC))) (dom-attr o 'eventname)) meeting-events))) ("ParticipantTalkingEvent" (cl-pushnew (assoc-default (dom-text (dom-by-tag o 'participant)) participants) talking)) ((or "CreatePresentationPodEvent" "EndAndKickAllEvent") (push (cons (string-to-number (dom-text (dom-by-tag o 'timestampUTC))) (dom-attr o 'eventname)) meeting-events)))) (dom-search dom (lambda (o) (dom-attr o 'eventname)))) (cl-pushnew (list :slug (plist-get talk :slug) :participants participants :talking talking) meeting-participants) meeting-events))) (emacsconf-get-talk-info)) (lambda (a b) (< (car a) (car b)))))) (dolist (event meeting-events) (pcase (cdr event) ("CreatePresentationPodEvent" (cl-incf meeting-count) (when (> meeting-count max-meetings) (setq max-meetings meeting-count))) ("ParticipantJoinEvent" (cl-incf participant-count) (when (> participant-count max-participants) (setq max-participants participant-count))) ("ParticipantLeftEvent" (cl-decf participant-count)) ("EndAndKickAllEvent" (cl-decf meeting-count)))) `((,(length meeting-participants) "Number of meetings analyzed") (,max-participants "Max number of simultaneous users") (,max-meetings "Max number of simultaneous meetings") (,(apply 'max (mapcar (lambda (o) (length (plist-get o :participants))) meeting-participants)) "Max number of people in one meeting") (,(length (seq-uniq (seq-mapcat (lambda (o) (mapcar #'cdr (plist-get o :participants))) meeting-participants))) "Total unique users") (,(length (seq-uniq (seq-mapcat (lambda (o) (plist-get o :talking)) meeting-participants))) "Total unique talking"))))
| 31 | Number of meetings analyzed |
| 62 | Max number of simultaneous users |
| 6 | Max number of simultaneous meetings |
| 27 | Max number of people in one meeting |
| 84 | Total unique users |
| 36 | Total unique talking |
The number of simultaneous users is pretty manageable. Most people watch the stream, which we broadcast via Icecast, so those numbers aren't reflected here. I think we tended to have between 100-200 viewers on Icecast.
For that kind of usage, some hosting options are:
BigBlueButton hosting:
Host Monthly Concurrent users Notes BiggerBlueButton USD 40 150 I'd need to check if we can have more than 10 created rooms if only at most 10 are used concurrently Web Hosting Zone USD 49 100 Myna Parrot USD 60 75 USD 150/month + USD 15 setup fee if we want to use our own URL BigBlueButton.host USD 85 80 BigBlueMeeting USD 125 100 BBB On Demand 8 vCPU 32 GB RAM: USD 1.20/hour, USD 0.05/hour when stopped: USD 86 for 3 days BBB On Demand 100 USD 2.40/hour: USD 173 for 3 days Virtual private server: We'd need to set up and manage this ourselves. We could probably run it for one week before to give speakers time to do their tech-checks and one week after to give me time to pull the recordings. The other servers are on Linode, so it might make sense to keep it there too and manage it all in one place.
Type Monthly dedicated 8 GB 4-core USD 72 USD 0.108/hour, so USD 36 if we run it for two weeks dedicated CPU 16 GB 8-core USD 144 USD 0.216/hour, so USD 72 if we run it for two weeks
It would be nice if we could just do the upgrade and get it back onto our current server (also, fixing up our current server with a proper SMTP setup so that it could send out things like password reminder emails), although the current BigBlueButton server was donated by a defunct organization so it might be a good idea to have a backup plan for it anyway.
It would also be nice to add it to our Ansible configuration so that we could install BigBlueButton that way, maybe based on ansible-role-bigbluebutton. But again, not my current cup of tea, so it will need to wait until someone can step up to do it or I get around to it.
The Free Software Foundation feels strongly about software as a service substitute. They're okay with virtual private servers, but I'm not sure how far their moral objection goes when it comes to using and paying for free/libre/opensource software as a service, like BigBlueButton. I'm personally okay with paying for services, especially if they're based on free software. Since EmacsConf is committed to using free software and not requiring people to use non-free software, that might be something the other organizers can weigh in on. If someone feels strongly enough about it, maybe they'll work on it. I think it can be hard enough for people to find the time for stuff they like, so if no one particularly likes doing this sort of stuff, I'm okay with scaling down or paying for something that's ready to go.
Anyway, at least we have the numbers for decisions!
