category - emacs :: Sacha Chua

While putting together some notes on how to use subed.el with auto-generated YouTube captions, I decided to get subed-word-data working with the Youtube VTT format. The sample file I downloaded from one of my YouTube videos had a cue whose text started with a blank line, so I ended up redoing the way subed-vtt.el parsed cues. Twice, actually. The first time, I changed it to handle multiple blocks in cue text (separated by blank lines). Then I came across this test suite for WebVTT parsing and found out that the timing line for a cue doesn't have to be preceded by a blank line, so then I needed to change the VTT parsing again.

Fortunately, I inherited a large Buttercup test suite from subed.el's original author. I've been adding to it over the years. As I learned more about how I wanted subed.el to behave, I added more cases. Then I worked on shifting the code to behave the way I wanted it to. At first, I didn't quite understand what I wanted the code to do, but as I pinned down more test cases, I was able to figure it out.

This was the first time I used toggle-window-dedicated (C-x w d) extensively. This function keeps the same buffer displayed in the window instead of letting Emacs Lisp functions replace it. First, I set up a large window for my subed-vtt.el. I split the other side into a smaller window for my test-subed-vtt.el, another window for a temporary VTT file, and a window for output from whatever command I'm running. I set most of the windows to dedicated except for my temporary output window. I also enabled winner-mode just in case I messed things up so that I could restore with winner-undo. Dedicating the windows meant I didn't have to keep fussing around with my windows and buffers to get things back to where I wanted them to be. I did use C-x o (other-window) a lot, so maybe it'll be worth getting the hang of ace-window.

Figure 1: Dedicated windows for the code, the test file, and a VTT buffer for interactive testing; a spare window for other output

Sometimes I wanted to focus on one of those small windows. prot/window-single-toggle was helpful for maximizing a window and then returning to the previous configuration.

I mostly evaluated or edebugged my code and then used my-buttercup-run-dwim to run a suite in my test-subed-vtt.el. sp-backward-up-sexp and sp-narrow-to-sexp were also helpful for navigating the suites and focusing on whatever I was working on.

I'm looking forward to exploring the other test cases from that repository. It feels good to get better as a coder.

I just finished reading Mathematica: a Secret World of Imagination and Curiosity, by David Bessis. The idea of training your intuition echoed in my mind as I wrote test cases and changed the code. I started to look forward to the gaps between my understanding of the spec and my understanding of the test cases, and the gaps between the test cases and my implementation. It felt almost like a conversation. Sometimes it was hard to translate an idea into code. I felt myself getting muddled and turned around. Whenever I noticed that, I just had to back up and start from something I understood, figure out what I was uncertain about, and then go from there.

I like the way that subed.el gives me tools for thinking about text, times, and metadata together. That abstraction has been helpful for editing audio and making videos. The more solid I can make it, the easier it will be to imagine other things that use those ideas.

View org source for this post

You can e-mail me at sacha@sachachua.com.

2025-01-27 Emacs news

Jan 27, 2025| emacs, emacs-news

Upcoming events (iCal file, Org):
- Emacs Berlin (hybrid, in English) https://emacs-berlin.org/ Wed Jan 29 0930 America/Vancouver - 1130 America/Chicago - 1230 America/Toronto - 1730 Etc/GMT - 1830 Europe/Berlin - 2300 Asia/Kolkata – Thu Jan 30 0130 Asia/Singapore
- M-x Research: TBA https://m-x-research.github.io/ Fri Feb 7 0800 America/Vancouver - 1000 America/Chicago - 1100 America/Toronto - 1600 Etc/GMT - 1700 Europe/Berlin - 2130 Asia/Kolkata – Sat Feb 8 0000 Asia/Singapore
Emacs configuration:
Emacs Lisp:
Appearance:
Navigation:
Org Mode:
Denote:
- Tips for getting Denote and Howm to work together (@jbaty@social.lol)
Completion:
- Enhancing icomplete-vertical-mode in Emacs: A Follow-Up (Reddit)
- consult-dir–zoxide-dirs
Coding:
Shells:
- [eshell-smart feature proposal] Preserve previous prompt
- Thanos Apollo: Emacs As Your Terminal Emulator [Video] (YouTube 07:45)
Web:
- Lars Ingebrigtsen: Thinking About Opening URLs in Emacs, and Wayland, and Stuff
Mail, news, and chat:
Multimedia:
AI:
- Clojure visual-tools 30 - Workflow Demos 4: LLMs in Emacs for Clojure (01:39:50)
Community:
Other:
Emacs development:
- emacs-devel:
  - Improving prefix map names - suggestion for package maintainers: name prefix maps
- In Elisp manual document new functions for changing window layouts
- Display separators on tty menus with display table entries
- Add 'treesit-forward-comment' with 'forward-comment-function'
- ibuffer: Display column titles in header line
- ibuffer: New defcustom `ibuffer-human-readable-size'
New packages:
- kql-mode: Major mode for highlighting KQL (MELPA)
- locs-and-refs: Define locations and references for text files and buffers (MELPA)
- minuet: Code completion using LLM (MELPA)
- typespec-ts-mode: Major mode for TypeSpec (using tree-sitter) (MELPA)
- xmltokf: Functional wrappers around xmltok (MELPA)

Links from reddit.com/r/emacs, r/orgmode, r/spacemacs, r/planetemacs, Mastodon #emacs, Bluesky #emacs, Hacker News, lobste.rs, programming.dev, lemmy.world, lemmy.ml, communick.news, planet.emacslife.com, YouTube, the Emacs NEWS file, Emacs Calendar, and emacs-devel. Thanks to Andrés Ramírez for emacs-devel links. Do you have an Emacs-related link or announcement? Please e-mail me at sacha@sachachua.com. Thank you!

View org source for this post

You can e-mail me at sacha@sachachua.com.

Scaling a BigBlueButton server down to a 1 GB node between uses

Jan 23, 2025| geek, tech, emacsconf, emacs

Now that we've survived EmacsConf, I've been looking into running a BigBlueButton server so that various Emacs meetups can use it if they like instead of relying on Jitsi or other free video-conferencing services. (I spent some time looking into Galene, but I'm not quite sure that's ready for our uses yet, like this issue that LibrePlanet ran into with recording.)

BigBlueButton requires a server with at least 4 CPU cores and 8 GB of RAM to even start up, and it doesn't like to share with other services. This costs about USD 48+tax/month on Linode or USD 576+tax/year, which is not an efficient use of funds yet. I could delete it after each instance, but I've been having a hard time properly restoring it from backup after deploying to a new IP address. bbb-conf --setip doesn't seem to catch everything, so I was still getting curl errors related to verifying the certificate.

A reasonable in-between is to run it on Linode's lowest plan (1 core, 1GB RAM; USD 60+tax for the year) in between meetups, and then spin things up for maybe 6-12 hours around each meetup. If I go with the 4-core 8 GB setup, that would be an extra USD 0.43 - 0.86 USD per meetup, which is eminently doable. I could even go with the recommended configuration of 8 cores and 16 GB memory on a dedicated CPU plan (USD 0.216/hour, so USD 1.30 to 2.59 per meetup). This was the approach that we used while preparing for EmacsConf. Since I didn't have a lot of programming time, I scaled the node up to 4 core / 8GB RAM whenever I had time to work on it, and I scaled it down to 1GB at the end of each of my working sessions. I scaled it up to dedicated 8 core / 16 GB RAM for EmacsConf, during which we used roughly half of the CPU capacity in order to host a max of 107 simultaneous users over 7 meetings.

I reviewed my BigBlueButton setup notes in the EmacsConf organizers notebook and the 2024 notebook and set up a Linode instance under my account, so that I can handle the billing and also so that Amin Bandali doesn't get spammed by all the notifications (up, down, up, down…). And then I'll be able to just scale it up when EmacsConf comes around again, which is nice.

Anyway, BBB refuses to install on a machine with fewer than 4 cores or 8 GB RAM, but once you set it up, it'll valiantly thrash around even on an underpowered server, which makes working with the server over ssh a lot slower. Besides, that's not friendly to other people using the same server. I wanted to configure the services so that they would only run on a server of the correct size. It turns out that systemd will let you specify either ConditionMemory and ConditionCPUs in the unit configuration file, and that you can use files ending in .conf in a directory named like yourservicename.service.d to override part of the configuration. Clear examples were hard to find, so I wanted to share these notes.

Since ConditionMemory is specified in bytes (ex: 8000000000), I found ConditionCPUs to be easier to read.

I used this command to check if I'd gotten the syntax right:

systemd-analyze condition 'ConditionCPUs=>=4'

and then I wrote this script to set up the overrides:

CPUS_REQUIRED=4
for ID in coturn.service redis-server.service bigbluebutton.target multi-user.target bbb-graphql-server.service bbb-rap-resque-worker.service bbb-webrtc-sfu.service bbb-fsesl-akka.service bbb-webrtc-recorder.service bbb-pads.service bbb-export-annotations.service bbb-web.service freeswitch.service etherpad.service bbb-rap-starter.service bbb-rap-caption-inbox.service freeswitch.service bbb-apps-akka.service bbb-graphql-actions postgresql@14-main.service; do
    mkdir -p /etc/systemd/system/$ID.d
    printf "[Unit]\nConditionCPUs=>=$CPUS_REQUIRED\n" > /etc/systemd/system/$ID.d/require-cpu.conf
done
systemctl daemon-reload
systemd-analyze verify bigbluebutton.target

It seems to work. When I use linode-cli to resize to the testing size, BigBlueButton works:

#!/bin/bash
source /home/sacha/.profile
PATH=/home/sacha/.local/bin/:$PATH
linode-cli linodes resize $BBB_ID --type g6-standard-4 --allow_auto_disk_resize false
sleep 4m
linode-cli linodes boot $BBB_ID
sleep 3m
ssh root@bbb.emacsverse.org "bbb-conf --restart; cd ~/greenlight-v3; docker compose restart"
notify-send "Should be ready"

And when I resize it down to a 1 GB nanode, BigBlueButton doesn't get started and the VPS is nice and responsive when I SSH in.

#!/bin/bash
source /home/sacha/.profile
PATH=/home/sacha/.local/bin/:$PATH
echo Powering off
linode-cli linodes shutdown $BBB_ID
sleep 60
echo "Resizing BBB node to nanode, dormant"
linode-cli linodes resize $BBB_ID --type g6-nanode-1 --allow_auto_disk_resize false

So now I'm going to coordinate with Ihor Radchenko about when he might want to try this out for OrgMeetup, and I can talk to other meetup organizers to figure out times. People will probably want to test things before announcing it to their meetup groups, so we just need to schedule that. It's BigBlueButton 3.0. I'm not 100% confident in the setup. We had some technical issues with some EmacsConf speakers even though we did a tech check with them before we went live with their session. Not sure what happened there.

I'm still a little nervous about accidentally forgetting to downscale the server and running up a bill, but I've scheduled downscaling with the at command before, so that's helpful. If it turns out to be something we want to do regularly, I might even be able to use a cronjob from my other server so that it happens even if my laptop is off, and maybe set up a backup nginx server with a friendly message (and maybe a list of upcoming meetups) in case people connect before it's been scaled up. Anyway, I think that's a totally good use of part of the Google Open Source Peer Bonus I received last year.

As an aside, you can change a room's friendly_id to something actually friendly. In the Rails console (docker exec -it greenlight-v3 bundle exec rails console), you could do something like this:

Room.find_by(friendly_id: "CURRENT_ROOM_ID").update_attribute(:friendly_id, "NEW_CUSTOM_ID")

Anyway, let me know if you organize an Emacs meetup and want to give this BigBlueButton instance a try!

View org source for this post

You can e-mail me at sacha@sachachua.com.

Categories: geek » emacs