Categories: video

RSS - Atom - Subscribe via email

#YayEmacs 9: Trimming/adding silences to get to a target; subed-record-sum-time

| subed, yay-emacs, emacs, video

New in this video: subed-record-sum-time, #+PAD_LEFT and #+PAD_RIGHT

I like the constraints of a one-minute video, so I added a subed-record-sum-time command. That way, when I edit the video using Emacs, I can check how long the result will be. First, I split the subtitles, align it with the audio to fix the timestamps, and double check the times. Then I can skip my oopses. Sometimes WhisperX doesn't catch them, so I also look at waveforms and characters per second. I already talk quickly, so I'm not going to speed that up but I can trim the pauses in between phrases which is easy to do with waveforms. Sometimes, after reviewing a draft, I realize I need a little more time. If the original audio has some silence, I can just copy and paste it. If not, I can pad left or pad right to add some silence. I can try the flow of some sections and compile the video when I'm ready. Emacs can do almost anything. Yay Emacs!

You can watch this on YouTube, download the video, or download the audio.

Play by play:

  • I like the constraints of a one-minute video, so I added a subed-record-sum-time command. That way, when I edit the video using Emacs, I can check how long the result will be.
    • subed-record uses subtitles and directives in comments in a VTT subtitle file to edit audio and video. subed-record-sum-time calculates the resulting duration and displays it in the minibuffer.
  • First, I split the subtitles, align it with the audio to fix the timestamps, and double check the times.
    • I'm experimenting with an algorithmic way to combine the breaks from my script with the text from the transcript. subed-align calls the aeneas forced alignment tool to match up the text with the timestamps. I use subed-waveform-show-all to show all the waveforms.
  • Then I can skip my oopses.
    • Adding a NOTE #+SKIP comment before a subtitle makes subed-record-compile-video and subed-record-compile-flow skip that part of the audio.
  • Sometimes WhisperX doesn't catch them,
    • WhisperX sometimes doesn't transcribe my false starts if I repeat things quickly.
  • so I also look at waveforms
    • subed-waveform-show-all adds waveforms for all the subtitles. If I notice there's a pause or a repeated shape in the waveform, or if I listen and notice the repetition, I can confirm by middle-clicking on the waveform to sample part of it.
  • and characters per second.
    • Low characters per second is sometimes a sign that the timestamps are incorrect or there's a repetition that wasn't transcribed.
  • I already talk quickly, so I'm not going to speed that up
    • Also, I already sound like a chipmunk; mechanically speeding up my recording to fit in a certain time will make that worse =)
  • but I can trim the pauses in between phrases which is easy to do with waveforms.
    • left-click to set the start, right-click to set the stop. If I want to adjust the previous/next one at the same time, I would use shift-left-click or shift-right-click, but here I want to skip the gaps between phrases, so I adjust the current subtitle without making the previous/next one longer.
  • Sometimes, after reviewing a draft, I realize I need a little more time.
    • I can specify visuals like a video, animated GIF, or an image by adding a [[file:...]] link in the comment for a subtitle. That visual will be used until the next visual is specified in a comment on a different subtitle. subed-record-compile-video can automatically speed up video clips to fit in the time for the current audio segment, which is the set of subtitles before the next visual is defined. After I compile and review the video, sometimes I notice that something goes by too quickly.
  • If the original audio has some silence, I can just copy and paste it.
    • This can sometimes feel more natural than adding in complete silence.
  • If not, I can pad left or pad right to add some silence.
    • I added a new feature so that I could specify something like #+PAD_RIGHT: 1.5 in a comment to add 1.5 seconds of silence after the audio specified by that subtitle.
  • I can try the flow of some sections
    • I can select a region and then use M-x subed-record-compile-try-flow to play the audio or C-u M-x subed-record-compile-try-flow to play the audio+video for that region.
  • and compile the video when I'm ready.
    • subed-record-compile-video compiles the video to the file specified in #+OUTPUT: filename. ffmpeg is very arcane, so I'm glad I can simplify my use of it with Emacs Lisp.
  • Emacs can do almost anything. Yay Emacs!
    • Non-linear audio and video editing is actually pretty fun in a text editor, especially when I can just use M-x vundo to navigate my undo history.

Links:

Related:

View org source for this post
View or add comments (Disqus), or e-mail me at sacha@sachachua.com

Editing videos with Emacs and subed-record.el

| emacs, subed, video

I want to document more of my Minecraft adventures with A+. Video is a natural way to do this. It builds on her familiarity with the tutorials and streams she enjoys watching. I set up OBS on her laptop and plugged in my Blue Yeti microphone. We did our first interview yesterday. I edited and subtitled it (because why not!), uploaded it as an unlisted YouTube video, and shared it with her dad, sister, and cousins.

I did the video editing in Emacs with subed-record. First, I used WhisperX to transcribe the video, and I used subed-align to fix the timestamps with aeneas. I normalized the audio with Audacity and I exported the .opus file for use in subed-record.el. Then I added NOTE #+SKIP before times I wanted to remove, like when she asked for a retake. Here's what that subtitle markup looks like: 

WEBVTT

NOTE #+SKIP

00:00:00.000 --> 00:00:16.679
And then I'll record in my side also
and we'll just put it in somehow.
Somehow. Okay. We can edit that, right?
Yeah, we'll learn how to edit things.
It'll be great.

NOTE
Introduction
#+AUDIO: cuberventures-001.opus
[[file:intro.webm]]
#+OUTPUT: cuberventures-001-fxnt-create-2-windmill-home-cafe-trains-hotel-half-underwater.webm

00:00:16.680 --> 00:00:19.399
Okay, so now we're here with <username>.

00:00:19.400 --> 00:00:23.039
I want to find out what you like about Minecraft and

00:00:23.040 --> 00:00:26.079
all the cool things that you have been building lately.

This was a little different from my usual video creation workflow, where I record the audio and the video separately. When I wrote subed-record.el, I assumed I'd edit the audio first, choose images/GIFs/videos that were already ready to go, and then combine those visuals with sections of audio, speeding things up or slowing things down as needed. Now I wanted to apply the same edits to the video as I did to the audio. A+ did a great job of looking at stuff in Minecraft while talking about them, so I wanted to keep her narration in sync. I added some code to allow me to specify a same-edits keyword for the visuals. That meant that I would use the same selection list that I used for cutting the audio. Here's what that subtitle markup looks like: 

NOTE
[[file:2024-12-31 10-35-14.mkv]]
#+OPTIONS: same-edits

00:00:43.860 --> 00:00:45.941
Shall we take a tour of my world?

00:00:45.942 --> 00:00:50.079
Sure, let's tell people which mod pack this is.

00:00:50.080 --> 00:00:55.639
This is FXNT Create 2, also known as FoxyNoTail Create 2.

NOTE Windmill

00:00:55.640 --> 00:00:58.239
I've got this little bit of path leading to the interview

00:00:58.240 --> 00:01:01.839
room. This is my unfinished windmill. I've been meaning to

This workflow lets me cut out segments in the middle of the video, like this: 

00:17:30.200 --> 00:17:33.119
great start for a tour. I'm looking forward to seeing what

00:17:33.120 --> 00:17:34.112
you will build next.

NOTE #+SKIP

00:17:34.113 --> 00:18:02.379
Do you have any last words before
we try to figure out this video editing thing?
Yeah. We'll cut that last part out.
Let's just do a retake on that last part.
Someday. Out here. Okay. There you go.
This is a beautiful view.

00:18:02.380 --> 00:18:08.119
The last things I want to say about this world is there'll be

I also wanted to start the video with a segment from my recording, so we could see her avatar on screen during the introduction. She kept her computer on first-person POV instead of changing the camera. I used mpv to figure out the timestamps for the start and end of the part that I wanted to use, then I used ffmpeg to cut that clip. I added a comment with a link to that video in order to use it before the main video. That's the [[file:intro.webm]] in the first section's comments.

After testing a small section of the transcript by selecting a region and using subed-record-compile-video, I deselected the region and used subed-record-compile-video to produce the whole video.

I also modified subed-record-compile-subtitles to include the other non-directive comments, so I can include the section headings in the raw VTT file and have them turn up in the exported version. Then I can use the new subed-section-comments-as-chapters command to copy those as chapters for the YouTube description.

We're not going to share that particular video yet, but I'm looking forward to trying that technique with videos about stuff I'm figuring out in Minecraft or Emacs. It's also tempting me to think about ways to specify transitions like crossfades and other fancy effects like overlays.

I like using the transcript as the starting point for video editing. It just makes sense to me to work with it as text. I also like this experiment with documenting more of our Minecraft experimentation. It seems to get her talking and encourages her to build more. I'm looking forward to learning more about Minecraft and making videos too.

We did another video today using the new shortcuts I've just set up for toggling OBS recording. This time we didn't even need to do any editing. I used Org Export to make her a little HTML file that had the two videos on it, so she can review it any time. Onward!

View org source for this post
View or add comments (Disqus), or e-mail me at sacha@sachachua.com

Animating SVG topic maps with Inkscape, Emacs, FFmpeg, and Reveal.js

| emacs, drawing, org, ffmpeg, video

tldr (2167 words): I can make animating presentation maps easier by writing my own functions for the Emacs text editor. In this post, I show how I can animate an SVG element by element. I can also add IDs to the path and use CSS to build up an SVG with temporary highlighting in a Reveal.js presentation.

2024-01-10-01 Animating SVG topic maps with Inkscape, Emacs, FFmpeg, and Reveal.js #drawing #emacs #video #workflow.png
2024-01-10-01 Animating SVG topic maps with Inkscape, Emacs, FFmpeg, and Reveal.js #drawing #emacs #video #workflow.png

Text from the sketch
  • PNG: Inkscape: trace
  • Supernote (e-ink)
  • iPad: Adobe Fresco

Convert PDF to SVG with Inkscape (Cairo option) or pdftocairo)

  • PNG / Supernote PDF: Combined shapes. Process
    1. Break apart, fracture overlaps
    2. Recombine
    3. Set IDs
    4. Sort paths -> Animation style 1
  • Adobe Fresco: individual elements in order; landscape feels natural

Animation styles

  • Animation style 1: Display elements one after another
  • Animation style 2: Display elements one after another, and also show/hide highlights
    • Table: slide ID, IDs to add, temporary highlights -> Reveal.js: CSS with transitions

Ideas for next steps:

  • Explore graphviz & other diagramming tools
  • Frame-by-frame SVGs
    • on include
    • write to files
  • FFmpeg crossfade
  • Recording Reveal.js presentations
  • Use OCR results?

I often have a hard time organizing my thoughts into a linear sequence. Sketches are nice because they let me jump around and still show the connections between ideas. For presentations, I'd like to walk people through these sketches by highlighting different areas. For example, I might highlight the current topic or show the previous topics that are connected to the current one. Of course, this is something Emacs can help with. Before we dive into it, here are quick previews of the kinds of animation I'm talking about:

animation-loop.gif
Figure 1: Animation style 1: based on drawing order

Animation style 2: building up a map with temporary highlights

Getting the sketches: PDFs are not all the same

Let's start with getting the sketches. I usually export my sketches as PNGs from my Supernote A5X. But if I know that I'm going to animate a sketch, I can export it as a PDF. I've recently been experimenting with Adobe Fresco on the iPad, which can also export to PDF. The PDF I get from Fresco is easier to animate, but I prefer to draw on the Supernote because it's an e-ink device (and because the kiddo usually uses the iPad).

If I start with a PNG, I could use Inkscape to trace the PNG and turn it into an SVG. I think Inkscape uses autotrace behind the scenes. I don't usually put my highlights on a separate layer, so autotrace will make odd shapes.

It's a lot easier if you start off with vector graphics in the first place. I can export a vector PDF from the SuperNote A5X and either import it into Inkscape using the Cairo option or use the command-line pdftocairo tool.

I've been looking into using Adobe Fresco, which is a free app available for the iPad. Fresco's PDF export can be converted to an SVG using Inkscape or PDF to Cairo. What I like about the output of this app is that it gives me individual elements as their own paths and they're listed in order of drawing. This makes it really easy to animate by just going through the paths in order.

Animation style 1: displaying paths in order

Here's a sample SVG file that pdfcairo creates from an Adobe Fresco PDF export: 

pdftocairo -svg ~/Downloads/subed-audio.pdf ~/Downloads/subed-audio.svg
Sample SVG
subed-audio.svg

Adobe Fresco also includes built-in time-lapse, but since I often like to move things around or tidy things up, it's easier to just work with the final image, export it as a PDF, and convert it to an SVG.

I can make a very simple animation by setting the opacity of all the paths to 0, then looping through the elements to set the opacity back to 1 and write that version of the SVG to a separate file. From how-can-i-generate-png-frames-that-step-through-the-highlights:

my-animate-svg-paths: Add one path at a time. Save the resulting SVGs to OUTPUT-DIR.
(defun my-animate-svg-paths (filename output-dir)
  "Add one path at a time. Save the resulting SVGs to OUTPUT-DIR."
  (unless (file-directory-p output-dir)
    (make-directory output-dir t))
  (let* ((dom (xml-parse-file filename))
         (paths (seq-filter (lambda (e) (dom-attr e 'style))
                            (dom-by-tag dom 'path)))
         (total (length paths))
         (frame-num (length paths))
         result)
    (dolist (elem paths)
      (dom-set-attribute elem 'style
                         (concat
                          (dom-attr elem 'style)
                          ";mix-blend-mode:darken")))
    (with-temp-file (expand-file-name (format "frame-%03d.svg" (1+ frame-num)) output-dir)
      (xml-print dom))
    (dolist (elem paths)
      (dom-set-attribute elem 'style
                         (concat
                          (dom-attr elem 'style)
                          ";fill-opacity:0")))
    (dolist (elem paths)
      (with-temp-file (expand-file-name
                       (format "frame-%03d.svg"
                               (- total frame-num))
                       output-dir)
        (message "%03d" frame-num)
        (dom-set-attribute elem 'style
                           (concat (dom-attr elem 'style)
                                   ";fill-opacity:1"))
        (push (list (format "frame-%03d.svg"
                            (1+ (- total frame-num)))
                    (dom-attr elem 'id))
              result)
        (setq frame-num (1- frame-num))
        (xml-print dom)))
    (reverse result)))

Here's how I call it: 

(my-animate-svg-paths "~/Downloads/subed-audio.svg" "/tmp/subed-audio/frames" t)

Then I can use FFmpeg to combine all of those frames into a video: 

ffmpeg -i frame-%03d.svg -vf palettegen -y palette.png
ffmpeg -framerate 30 -i frame-%03d.svg -i palette.png -lavfi "paletteuse" -loop 0 -y animation-loop.gif
animation-loop.gif
Figure 2: Animating SVG paths based on drawing order

Neither Supernote nor Adobe Fresco give me the original stroke information. These are filled shapes, so I can't animate something drawing it. But having different elements appear in sequence is fine for my purposes. If you happen to know how to get stroke information out of Supernote .note files or of an iPad app that exports nice single-line SVGs that have stroke direction, I would love to hear about it.

Identifying paths from Supernote sketches

When I export a PDF from Supernote and convert it to an SVG, each color is a combined shape with all the elements. If I want to animate parts of the image, I have to break it up and recombine selected elements (Inkscape's Ctrl-k shortcut) so that the holes in shapes are properly handled. This is a bit of a tedious process and it usually ends up with elements in a pretty random order. Since I have to reorder elements by hand, I don't really want to animate the sketch letter-by-letter. Instead, I combine them into larger chunks like topics or paragraphs.

The following code takes the PDF, converts it to an SVG, recolours highlights, and then breaks up paths into elements:

my-sketch-convert-pdf-and-break-up-paths: Convert PDF to SVG and break up paths.
(defun my-sketch-convert-pdf-and-break-up-paths (pdf-file &optional rotate)
  "Convert PDF to SVG and break up paths."
  (interactive (list (read-file-name
                      (format "PDF (%s): "
                              (my-latest-file "~/Dropbox/Supernote/EXPORT/" "pdf"))
                      "~/Dropbox/Supernote/EXPORT/"
                      (my-latest-file "~/Dropbox/Supernote/EXPORT/" "pdf")
                      t
                      nil
                      (lambda (s) (string-match "pdf" s)))))
  (unless (file-exists-p (concat (file-name-sans-extension pdf-file) ".svg"))
    (call-process "pdftocairo" nil nil nil "-svg" (expand-file-name pdf-file)
                  (expand-file-name (concat (file-name-sans-extension pdf-file) ".svg"))))
  (let ((dom (xml-parse-file (expand-file-name (concat (file-name-sans-extension pdf-file) ".svg"))))
        highlights)
    (setq highlights (dom-node 'g '((id . "highlights"))))
    (dom-append-child dom highlights)
    (dolist (path (dom-by-tag dom 'path))
      ;;  recolor and move
      (unless (string-match (regexp-quote "rgb(0%,0%,0%)") (or (dom-attr path 'style) ""))
        (dom-remove-node dom path)
        (dom-append-child highlights path)
        (dom-set-attribute
         path 'style
         (replace-regexp-in-string
          (regexp-quote "rgb(78.822327%,78.822327%,78.822327%)")
          "#f6f396"
          (or (dom-attr path 'style) ""))))
      (let ((parent (dom-parent dom path)))
        ;; break apart
        (when (dom-attr path 'd)
          (dolist (part (split-string (dom-attr path 'd) "M " t " +"))
            (dom-append-child
             parent
             (dom-node 'path `((style . ,(dom-attr path 'style))
                               (d . ,(concat "M " part))))))
          (dom-remove-node dom path))))
    ;; remove the use
    (dolist (use (dom-by-tag dom 'use))
      (dom-remove-node dom use))
    (dolist (use (dom-by-tag dom 'image))
      (dom-remove-node dom use))
    ;; move the first g down
    (let ((g (car (dom-by-id dom "surface1"))))
      (setf (cddar dom)
            (seq-remove (lambda (o)
                          (and (listp o) (string= (dom-attr o 'id) "surface1")))
                        (dom-children dom)))
      (dom-append-child dom g)
      (when rotate
        (let* ((old-width (dom-attr dom 'width))
               (old-height (dom-attr dom 'height))
               (view-box (mapcar 'string-to-number (split-string (dom-attr dom 'viewBox))))
               (rotate (format "rotate(90) translate(0 %s)" (- (elt view-box 3)))))
          (dom-set-attribute dom 'width old-height)
          (dom-set-attribute dom 'height old-width)
          (dom-set-attribute dom 'viewBox (format "0 0 %d %d" (elt view-box 3) (elt view-box 2)))
          (dom-set-attribute highlights 'transform rotate)
          (dom-set-attribute g 'transform rotate))))
    (with-temp-file (expand-file-name (concat (file-name-sans-extension pdf-file) "-split.svg"))
      (svg-print (car dom)))))

2023-10-split.svg
Figure 3: Image after splitting up into elements

You can see how the spaces inside letters like "o" end up being black. Selecting and combining those paths fixes that.

Combining paths in Inkscape

If there were shapes that were touching, then I need to draw lines and fracture the shapes in order to break them apart.

Fracturing shapes and checking the highlights

The end result should be an SVG with the different chunks that I might want to animate, but I need to identify the paths first. You can assign object IDs in Inkscape, but this is a bit of an annoying process since I haven't figured out a keyboard-friendly way to set object IDs. I usually find it easier to just set up an Autokey shortcut (or AutoHotkey in Windows) to click on the ID text box so that I can type something in.

Autokey script for clicking 
import time
x, y = mouse.get_location()
# Use the coordinates of the ID text field on your screen; xev can help
mouse.click_absolute(3152, 639, 1)
time.sleep(1)
keyboard.send_keys("<ctrl>+a")
mouse.move_cursor(x, y)

Then I can select each element, press the shortcut key, and type an ID into the textbox. I might use "t-…" to indicate the text for a map section, "h-…" to indicate a highlight, and arrows by specifying their start and end.

Setting IDs in Inkscape

To simplify things, I wrote a function in Emacs that will go through the different groups that I've made, show each path in a different color and with a reasonable guess at a bounding box, and prompt me for an ID. This way, I can quickly assign IDs to all of the paths. The completion is mostly there to make sure I don't accidentally reuse an ID, although it can try to combine paths if I specify the ID. It saves the paths after each change so that I can start and stop as needed. Identifying paths in Emacs is usually much nicer than identifying them in Inkscape.

Identifying paths inside Emacs

my-svg-identify-paths: Prompt for IDs for each path in FILENAME.
(defun my-svg-identify-paths (filename)
  "Prompt for IDs for each path in FILENAME."
  (interactive (list (read-file-name "SVG: " nil nil
                                     (lambda (f) (string-match "\\.svg$" f)))))
  (let* ((dom (car (xml-parse-file filename)))
         (paths (dom-by-tag dom 'path))
         (vertico-count 3)
         (ids (seq-keep (lambda (path)
                          (unless (string-match "path[0-9]+" (or (dom-attr path 'id) "path0"))
                            (dom-attr path 'id)))
                        paths))
         (edges (window-inside-pixel-edges (get-buffer-window)))
         id)
    (my-svg-display "*image*" dom nil t)
    (dolist (path paths)
      (when (string-match "path[0-9]+" (or (dom-attr path 'id) "path0"))
        ;; display the image with an outline
        (unwind-protect
            (progn
              (my-svg-display "*image*" dom (dom-attr path 'id) t)
              (setq id (completing-read
                        (format "ID (%s): " (dom-attr path 'id))
                        ids))
              ;; already exists, merge with existing element
              (if-let ((old (dom-by-id dom id)))
                  (progn
                    (dom-set-attribute
                     old
                     'd
                     (concat (dom-attr (dom-by-id dom id) 'd)
                             " "
                             ;; change relative to absolute
                             (replace-regexp-in-string "^m" "M"
                                                       (dom-attr path 'd))))
                    (dom-remove-node dom path)
                    (setq id nil))
                (dom-set-attribute path 'id id)
                (add-to-list 'ids id))))
        ;; save the image just in case we get interrupted halfway through
        (with-temp-file filename
          (svg-print dom))))))

Sorting and animating the paths by IDs

Then I can animate SVGs by specifying the IDs. I can reorder the paths in the SVG itself so that I can animate it group by group, like the way that the Adobe Fresco SVGs were animated element by element.

Reordering paths 
(my-svg-reorder-paths "~/proj/2023-12-audio-workflow/map.svg"
                      '("t-start" "h-audio" "h-capture" "t-but" "t-mic" "h-mic"
                        "t-reviewing" "h-reviewing"
                        "t-words" "h-words" "t-workflow" "h-workflow"
                        "t-lapel" "h-lapel"
                        "mic-recorder" "t-recorder" "h-recorder"
                        "t-syncthing" "h-sync"
                        "t-keywords" "h-keywords" "t-keyword-types"
                        "t-lines" "h-lines"
                        "t-align" "h-align"
                        "arrow"
                        "t-org" "h-org" "t-todo" "h-todo" "h-linked"
                        "t-jump" "h-jump"
                        "t-waveform" "h-waveform"
                        "t-someday"
                        "h-sections"
                        "t-speech-recognition" "h-speech-recognition"
                        "t-ai" "h-ai"
                        "t-summary"
                        "extra")
                      "~/proj/2023-12-audio-workflow/map-output.svg")
(my-animate-svg-paths "~/proj/2023-12-audio-workflow/map-output.svg"
                      "~/proj/2023-12-audio-workflow/frames/")
Table of filenames after reordering paths and animating the image
frame-001.svg t-start
frame-002.svg h-audio
frame-003.svg h-capture
frame-004.svg t-but
frame-005.svg t-mic
frame-006.svg h-mic
frame-007.svg t-reviewing
frame-008.svg h-reviewing
frame-009.svg t-words
frame-010.svg h-words
frame-011.svg t-workflow
frame-012.svg h-workflow
frame-013.svg t-lapel
frame-014.svg h-lapel
frame-015.svg mic-recorder
frame-016.svg t-recorder
frame-017.svg h-recorder
frame-018.svg t-syncthing
frame-019.svg h-sync
frame-020.svg t-keywords
frame-021.svg h-keywords
frame-022.svg t-keyword-types
frame-023.svg t-lines
frame-024.svg h-lines
frame-025.svg t-align
frame-026.svg h-align
frame-027.svg arrow
frame-028.svg t-org
frame-029.svg h-org
frame-030.svg t-todo
frame-031.svg h-todo
frame-032.svg h-linked
frame-033.svg t-jump
frame-034.svg h-jump
frame-035.svg t-waveform
frame-036.svg h-waveform
frame-037.svg t-someday
frame-038.svg h-sections
frame-039.svg t-speech-recognition
frame-040.svg h-speech-recognition
frame-041.svg t-ai
frame-042.svg h-ai
frame-043.svg t-summary
frame-044.svg extra

The table of filenames makes it easy to use specific frames as part of a presentation or video.

Here is the result as a video: 

(let ((compile-media-output-video-width 1280)
      (compile-media-output-video-height 720))
  (my-ffmpeg-animate-images
   (directory-files "~/proj/2023-12-audio-workflow/frames/" t "\\.svg$")
   (expand-file-name "~/proj/2023-12-audio-workflow/frames/animation.webm")
   4))

Animation of SVG by paths

The way it works is that the my-svg-reorder-paths function removes and readds elements following the list of IDs specified, so everything's ready to go for step-by-step animation. Here's the code:

my-svg-reorder-paths: Sort paths in FILENAME.
(defun my-svg-reorder-paths (filename &optional ids output-filename)
  "Sort paths in FILENAME."
  (interactive (list (read-file-name "SVG: " nil nil (lambda (f) (string-match "\\.svg$" f)))
                     nil (read-file-name "Output: ")))
  (let* ((dom (car (xml-parse-file filename)))
         (paths (dom-by-tag dom 'path))
         (parent (dom-parent dom (car paths)))
         (ids-left
          (nreverse (seq-keep (lambda (path)
                                (unless (string-match "path[0-9]+" (or (dom-attr path 'id) "path0"))
                                  (dom-attr path 'id)))
                              paths)))
         list)
    (when (called-interactively-p)
      (while ids-left
        (my-svg-display "*image*" dom (car ids-left))
        (let ((current (completing-read
                        (format "ID (%s): "
                                (car ids-left))
                        ids-left nil nil nil nil (car ids-left)))
              node)
          (add-to-list 'ids current)
          (setq ids-left (seq-remove (lambda (o) (string= o current)) ids-left)))))
    (if ids ;; reorganize under the first path's parent
        (progn
          (dolist (id ids)
            (if-let ((node (car (dom-by-id dom id))))
                (progn
                  (dom-remove-node dom node)
                  (dom-append-child parent node))
              (message "Could not find %s" id)))
          (with-temp-file (or output-filename filename)
            (svg-print dom))))
    (nreverse (seq-keep (lambda (path)
                          (unless (string-match "path[0-9]+" (or (dom-attr path 'id) "path0"))
                            (dom-attr path 'id)))
                        (dom-by-tag dom 'path)))))

Animation style 2: Building up a map with temporary highlights

I can also use CSS rules to transition between opacity values for more complex animations. For my EmacsConf 2023 presentation, I wanted to make a self-paced, narrated presentation so that people could follow hyperlinks, read the source code, and explore. I wanted to include a map so that I could try to make sense of everything. For this map, I wanted to highlight the previous sections that were connected to the topic for the current section.

I used a custom Org link to include the full contents of the SVG instead of just including it with an img tag. 

#+ATTR_HTML: :class r-stretch
my-include:~/proj/emacsconf-2023-emacsconf/map.svg?wrap=export html

my-include-export: Export PATH to FORMAT using the specified wrap parameter.
(defun my-include-export (path _ format _)
  "Export PATH to FORMAT using the specified wrap parameter."
  (let (params body start end)
    (when (string-match "^\\(.*+?\\)\\(?:::\\|\\?\\)\\(.*+\\)" path)
      (setq params (save-match-data (org-protocol-convert-query-to-plist (match-string 2 path)))
            path (match-string 1 path)))
    (with-temp-buffer
      (insert-file-contents-literally path)
      (when (string-match "\\.org$" path)
        (org-mode))
      (if (plist-get params :name)
          (when (org-babel-find-named-block (plist-get params :name))
            (goto-char (org-babel-find-named-block (plist-get params :name)))
            (let ((block (org-element-context)))
              (setq start (org-element-begin block)
                    end (org-element-end block))))
        (goto-char (point-min))
        (when (plist-get params :from-regexp)
          (re-search-forward (url-unhex-string (plist-get params :from-regexp)))
          (goto-char (match-beginning 0)))
        (setq start (point))
        (setq end (point-max))
        (when (plist-get params :to-regexp)
          (re-search-forward (url-unhex-string (plist-get params :to-regexp)))
          (setq end (match-beginning 0))))
      (setq body (buffer-substring start end)))
    (with-temp-buffer
      (when (plist-get params :wrap)
        (let* ((wrap (plist-get params :wrap))
               block args)
          (when (string-match "\\<\\(\\S-+\\)\\( +.*\\)?" wrap)
            (setq block (match-string 1 wrap))
            (setq args (match-string 2 wrap))
            (setq body (format "#+BEGIN_%s%s\n%s\n#+END_%s\n"
                               block (or args "")
                               body
                               block)))))
      (when (plist-get params :summary)
        (setq body (format "#+begin_my_details %s\n%s\n#+end_my_details\n"
                           (plist-get params :summary)
                           body)))
      (insert body)
      (org-export-as format nil nil t))))

I wanted to be able to specify the entire sequence using a table in the Org Mode source for my presentation. Each row had the slide ID, a list of highlights in the form prev1,prev2;current, and a comma-separated list of elements to add to the full-opacity view.

Slide Highlight Additional elements
props-map h-email;h-properties t-email,email-properties,t-properties
file-prefixes h-properties;h-filename t-filename,properties-filename
renaming h-filename;h-renaming t-renaming,filename-renaming
shell-scripts h-renaming;h-shell-scripts renaming-shell-scripts,t-shell-scripts
availability h-properties;h-timezone t-timezone,properties-timezone
schedule h-timezone;h-schedule t-schedule,timezone-schedule
emailing-speakers h-timezone,h-mail-merge;h-emailing-speakers schedule-emailing-speakers,t-emailing-speakers
template h-properties;h-template t-template,properties-template
wiki h-template;h-wiki t-wiki,template-wiki,schedule-wiki
pad h-template;h-pad template-pad,t-pad
mail-merge h-template;h-mail-merge t-mail-merge,template-mail-merge,schedule-mail-merge,emailing-speakers-mail-merge
bbb h-bbb t-bbb
checkin h-mail-merge;h-checkin t-checkin,bbb-checkin
redirect h-bbb;h-redirect t-redirect,bbb-redirect
shortcuts h-email;h-shortcuts t-shortcuts,email-shortcuts
logbook h-shortcuts;h-logbook shortcuts-logbook,t-logbook
captions h-captions t-captions,captions-wiki
tramp h-captions;h-tramp t-tramp,captions-tramp
crontab h-tramp;h-crontab tramp-crontab,bbb-crontab,t-crontab
transitions h-crontab;h-transitions shell-scripts-transitions,t-transitions,shortcuts-transitions,transitions-crontab
irc h-transitions;h-irc t-irc,transitions-irc

Reveal.js adds a "current" class to the slide, so I can use that as a trigger for the transition. I have a bit of Emacs Lisp code that generates some very messy CSS, in which I specify the ID of the slide, followed by all of the elements that need their opacity set to 1, and also specifying the highlights that will be shown in an animated way.

my-reveal-svg-progression-css: Make the CSS.
(defun my-reveal-svg-progression-css (map-progression &optional highlight-duration)
  "Make the CSS.
map-progression should be a list of lists with the following format:
((\"slide-id\" \"prev1,prev2;cur1\" \"id-to-add1,id-to-add2\") ...)."
  (setq highlight-duration (or highlight-duration 2))
  (let (full)
    (format
     "<style>%s</style>"
     (mapconcat
      (lambda (slide)
        (setq full (append (split-string (elt slide 2) ",") full))
        (format "#slide-%s.present path { opacity: 0.2 }
%s { opacity: 1 !important }
%s"
                (car slide)
                (mapconcat (lambda (id) (format "#slide-%s.present #%s" (car slide) id))
                           full
                           ", ")
                (my-reveal-svg-highlight-different-colors slide)))
      map-progression
      "\n"))))
I call it from my Org file like this: 

#+NAME: progression-css
#+begin_src emacs-lisp :exports code :var map-progression=progression :var highlight-duration=2 :results silent
(my-reveal-svg-progression-css map-progression highlight-duration)
#+end_src

Here's an excerpt showing the kind of code it makes: 

<style>#slide-props-map.present path { opacity: 0.2 }
#slide-props-map.present #t-email, #slide-props-map.present #email-properties, #slide-props-map.present #t-properties { opacity: 1 !important }
#slide-props-map.present #h-email { fill: #c6c6c6; opacity: 1 !important; transition: fill 0.5s; transition-delay: 0.0s }#slide-props-map.present #h-properties { fill: #f6f396; opacity: 1 !important; transition: fill 0.5s; transition-delay: 0.5s }
#slide-file-prefixes.present path { opacity: 0.2 }
#slide-file-prefixes.present #t-filename, #slide-file-prefixes.present #properties-filename, #slide-file-prefixes.present #t-email, #slide-file-prefixes.present #email-properties, #slide-file-prefixes.present #t-properties { opacity: 1 !important }
#slide-file-prefixes.present #h-properties { fill: #c6c6c6; opacity: 1 !important; transition: fill 0.5s; transition-delay: 0.0s }#slide-file-prefixes.present #h-filename { fill: #f6f396; opacity: 1 !important; transition: fill 0.5s; transition-delay: 0.5s }
...</style>

Since it's automatically generated, I don't have to worry about it once I've gotten it to work. It's all hidden in a results drawer. So this CSS highlights specific parts of the SVG with a transition, and the highlight changes over the course of a second or two. It highlights the previous names and then the current one. The topics I'd already discussed would be in black, and the topics that I had yet to discuss would be in very light gray. This could give people a sense of the progress through the presentation.

Code for making the CSS 
(defun my-reveal-svg-animation (slide)
  (string-join
   (seq-map-indexed
    (lambda (step-ids i)
      (format "%s { fill: #f6f396; transition: fill %ds; transition-delay: %ds }"
              (mapconcat
               (lambda (id) (format "#slide-%s.present #%s" (car slide) id))
               (split-string step-ids ",")
               ", ")
              highlight-duration
              (* i highlight-duration)))
    (split-string (elt slide 1) ";"))
   "\n"))

(defun my-reveal-svg-highlight-different-colors (slide)
  (let* ((colors '("#f6f396" "#c6c6c6")) ; reverse
         (steps (split-string (elt slide 1) ";"))
         (step-length 0.5))
    (string-join
     (seq-map-indexed
      (lambda (step-ids i)
        (format "%s { fill: %s; opacity: 1 !important; transition: fill %.1fs; transition-delay: %.1fs }"
                (mapconcat
                 (lambda (id) (format "#slide-%s.present #%s" (car slide) id))
                 (split-string step-ids ",")
                 ", ")
                (elt colors (- (length steps) i 1))
                step-length
                (* i 0.5)))
      steps))))

(defun my-reveal-svg-progression-css (map-progression &optional highlight-duration)
  "Make the CSS.
map-progression should be a list of lists with the following format:
((\"slide-id\" \"prev1,prev2;cur1\" \"id-to-add1,id-to-add2\") ...)."
  (setq highlight-duration (or highlight-duration 2))
  (let (full)
    (format
     "<style>%s</style>"
     (mapconcat
      (lambda (slide)
        (setq full (append (split-string (elt slide 2) ",") full))
        (format "#slide-%s.present path { opacity: 0.2 }
%s { opacity: 1 !important }
%s"
                (car slide)
                (mapconcat (lambda (id) (format "#slide-%s.present #%s" (car slide) id))
                           full
                           ", ")
                (my-reveal-svg-highlight-different-colors slide)))
      map-progression
      "\n"))))

As a result, as I go through my presentation, the image appears to build up incrementally, which is the effect that I was going for. I can test this by exporting only my map slides: 

(save-excursion
  (goto-char (org-babel-find-named-block "progression-css"))
  (org-babel-execute-src-block))
(let ((org-tags-exclude-from-inheritance "map")
      (org-export-select-tags '("map")))
   (oer-reveal-export-to-html))

Ideas for next steps

  • Graphviz, mermaid-js, and other diagramming tools can make SVGs. I should be able to adapt my code to animate those diagrams by adding other elements in addition to path. Then I'll be able to make diagrams even more easily.
  • Since SVGs can contain CSS, I could make an SVG equivalent of the CSS rules I used for the presentation, maybe calling a function with a Lisp expression that specifies the operations (ex: ("frame-001.svg" "h-foo" opacity 1)). Then I could write frames to SVGs.
  • FFmpeg has a crossfade filter. With a little bit of figuring out, I should be able to make the same kind of animation in a webm form that I can include in my regular videos instead of using Reveal.js and CSS transitions.
  • I've also been thinking about automating the recording of my Reveal.js presentations. For my EmacsConf talk, I opened my presentation, started the recording with the system audio and the screen, and then let it autoplay the presentation. I checked on it periodically to avoid the screensaver/energy saving things from kicking in and so that I could stop the recording when it's finished. If I want to make this take less work, one option is to use ffmpeg's "-t" argument to specify the expected duration of the presentation so that I don't have to manually stop it. I'm also thinking about using Puppeteer to open the presentation, check when it's fully loaded, and start the process to record it - maybe even polling to see whether it's finished. I haven't gotten around to it yet. Anyhow, those are some ideas to explore next time.
  • As for animation, I'm still curious about the possibility of finding a way to access the raw stroke information if it's even available from my Supernote A5X (difficult because it's a proprietary data format) or finding an app for the iPad that exports single line SVGs that use stroke information instead of fill. That would only be if I wanted to do those even fancier animations that look like the whole thing is being drawn for you. I was trying to figure out if I could green screen the Adobe Fresco timelapse videos so that even if I have a pre-sketch to figure out spacing and remind me what to draw, I can just export the finished elements. But there's too much anti-aliasing and I haven't figured out how to do it cleanly yet. Maybe some other day.
  • I use Google Cloud Vision's text detection engine to convert my handwriting to text. It can give me bounding polygons for words or paragraphs. I might be able to figure out which curves are entirely within a word's bounding polygon and combine those automatically.
  • It would be pretty cool if I could combine the words recognized by Google Cloud Vision with the word-level timestamps from speech recognition so that I could get word-synced sketchnote animations with maybe a little manual intervention.

Anyway, those are some workflows for animating sketches with Inkscape and Emacs. Yay Emacs!

View org source for this post
View or add comments (Disqus), or e-mail me at sacha@sachachua.com

Quick notes on livestreaming to YouTube with FFmpeg on a Lenovo X230T

| video, youtube, streaming, ffmpeg, yay-emacs

[2024-01-05 Fri]: Updated scripts

2024-01-03-03 Quick thoughts on livestreaming #sharing #video #streaming #community.png
2024-01-03-03 Quick thoughts on livestreaming #sharing #video #streaming #community.png

Text from the sketch

Quick thoughts on livestreaming

Why:

  • work out loud
  • share tips
  • share more
  • spark conversations
  • (also get questions about things)

Doable with ffmpeg on my X230T:

  • streaming from my laptop
  • lapel mic + system audio,
  • second screen for monitoring

Ideas for next time:

  • Overall notes in Emacs with outline, org-timer timestamped notes; capture to this file
  • Elisp to start/stop the stream → find old code
  • Use the Yeti? Better sound
  • tee to a local recording
  • grab screenshot from SuperNote mirror?

Live streaming info density:

  • High: Emacs News review, package/workflow demo
  • Narrating a blog post to make it a video
  • Categorizing Emacs News, exploring packages
  • Low: Figuring things out

YouTube can do closed captions for livestreams, although accuracy is low. Videos take a while to be ready to download.

Experimenting with working out loud

I wanted to write a report on EmacsConf 2023 so that we could share it with speakers, volunteers, participants, donors, related organizations like the Free Software Foundation, and other communities. I experimented with livestreaming via YouTube while I worked on the conference highlights.

It's a little over an hour long and probably very boring, but it was nice of people to drop by and say hello.

The main parts are:

  • 0:00: reading through other conference reports for inspiration
  • 6:54: writing an overview of the talks
  • 13:10: adding quotes for specific talks
  • 25:00: writing about the overall conference
  • 32:00: squeezing in more highlights
  • 49:00: fiddling with the formatting and the export

It mostly worked out, aside from a brief moment of "uhhh, I'm looking at our private conf.org file on stream". Fortunately, the e-mail addresses that were showed were the public ones.

Technical details

Setup:

  • I set up environment variables and screen resolution: 

      # From pacmd list-sources | egrep '^\s+name'
  LAPEL=alsa_input.usb-Jieli_Technology_USB_Composite_Device_433035383239312E-00.mono-fallback #
  YETI=alsa_input.usb-Blue_Microphones_Yeti_Stereo_Microphone_REV8-00.analog-stereo
  SYSTEM=alsa_output.pci-0000_00_1b.0.analog-stereo.monitor
  # MIC=$LAPEL
  # AUDIO_WEIGHTS="1 1"
  MIC=$YETI
  AUDIO_WEIGHTS="0.5 0.5"
  OFFSET=+1920,430
  SIZE=1280x720
  SCREEN=LVDS-1  # from xrandr
  xrandr --output $SCREEN --mode 1280x720
  • I switch to a larger size and a light theme. I also turn consult previews off to minimize the risk of leaking data through buffer previews.
    my-emacsconf-prepare-for-screenshots: Set the resolution, change to a light theme, and make the text bigger.
    (defun my-emacsconf-prepare-for-screenshots ()
  (interactive)
  (shell-command "xrandr --output LVDS-1 --mode 1280x720")
  (modus-themes-load-theme 'modus-operandi)
  (my-hl-sexp-update-overlay)
  (set-face-attribute 'default nil :height 170)
  (keycast-mode))

Testing: 

ffmpeg -f x11grab -video_size $SIZE -i :0.0$OFFSET -y /tmp/test.png; display /tmp/test.png
ffmpeg -f pulse -i $MIC -f pulse -i $SYSTEM -filter_complex amix=inputs=2:weights=$AUDIO_WEIGHTS:duration=longest:normalize=0 -y /tmp/test.mp3; mpv /tmp/test.mp3
DATE=$(date "+%Y-%m-%d-%H-%M-%S")
ffmpeg -f x11grab -framerate 30 -video_size $SIZE -i :0.0$OFFSET -f pulse -i $MIC -f pulse -i $SYSTEM -filter_complex "amix=inputs=2:weights=$AUDIO_WEIGHTS:duration=longest:normalize=0" -c:v libx264 -preset fast -maxrate 690k -bufsize 2000k -g 60 -vf format=yuv420p -c:a aac -b:a 96k -y -flags +global_header "/home/sacha/recordings/$DATE.flv" -f flv

Streaming: 

DATE=$(date "+%Y-%m-%d-%H-%M-%S")
ffmpeg -f x11grab -framerate 30 -video_size $SIZE -i :0.0$OFFSET -f pulse -i $MIC -f pulse -i $SYSTEM -filter_complex "amix=inputs=2:weights=$AUDIO_WEIGHTS:duration=longest:normalize=0[audio]" -c:v libx264 -preset fast -maxrate 690k -bufsize 2000k -g 60 -vf format=yuv420p -c:a aac -b:a 96k -y -f tee -map 0:v -map '[audio]' -flags +global_header  "/home/sacha/recordings/$DATE.flv|[f=flv]rtmp://a.rtmp.youtube.com/live2/$YOUTUBE_KEY"

To restore my previous setup:

my-emacsconf-back-to-normal: Go back to a more regular setup.
(defun my-emacsconf-back-to-normal ()
  (interactive)
  (shell-command "xrandr --output LVDS-1 --mode 1366x768")
  (modus-themes-load-theme 'modus-vivendi)
  (my-hl-sexp-update-overlay)
  (set-face-attribute 'default nil :height 115)
  (keycast-mode -1))

Ideas for next steps

I can think of a few workflow tweaks that might be fun:

  • a stream notes buffer on the right side of the screen for context information, timestamped notes to make editing/review easier (maybe using org-timer), etc. I experimented with some streaming-related code in my config, so I can dust that off and see what that's like. I also want to have an org-capture template for it so that I can add notes from anywhere.
  • a quick way to add a screenshot from my Supernote to my Org files

I think I'll try going through an informal presentation or Emacs News as my next livestream experiment, since that's probably higher information density.

View org source for this post
View or add comments (Disqus), or e-mail me at sacha@sachachua.com

EmacsConf backstage: Trimming the BigBlueButton recordings based on YouTube duration

| emacsconf, emacs, youtube, video

I wanted to get the Q&A sessions up quickly after the conference, so I uploaded them to YouTube and added them to the EmacsConf 2023 playlist. I used YouTube's video editor to roughly guess where to trim them based on the waveforms. I needed to actually trim the source videos, though, so that our copies would be up to date and I could use those for the Toobnix uploads.

My first task was to figure out which videos needed to be trimmed to match the YouTube edits. First, I retrieved the video details using the API and the code that I added to emacsconf-extract.el. 

(setq emacsconf-extract-youtube-api-video-details (emacsconf-extract-youtube-get-video-details emacsconf-extract-youtube-api-playlist-items))

Then I made a table comparing the file duration with the YouTube duration, showing rows only if the difference was more than 3 minutes. 

(append
 '(("type" "slug" "file duration" "youtube duration" "diff"))
 (let ((threshold-secs (* 3 60))) ; don't sweat small differences
   (seq-mapcat
    (lambda (talk)
      (seq-keep
       (lambda (row)
         (when (plist-get talk (cadr row))
           (let* ((video (emacsconf-extract-youtube-find-url-video-in-list
                          (plist-get talk (cadr row))
                          emacsconf-extract-youtube-api-video-details))
                  (video-duration (if (and video (emacsconf-extract-youtube-duration-msecs video))
                                      (/ (emacsconf-extract-youtube-duration-msecs video) 1000.0)))
                  (file-duration (ceiling
                                  (/ (compile-media-get-file-duration-ms (emacsconf-talk-file talk (format "--%s.webm" (car row))))
                                     1000.0))))
             (when (and video-duration (> (abs (- file-duration video-duration)) threshold-secs))
               (list (car row)
                     (plist-get talk :slug)
                     (and file-duration (format-seconds "%h:%z%.2m:%.2s" file-duration))
                     (and video-duration (format-seconds "%h:%z%.2m:%.2s" video-duration))
                     (emacsconf-format-seconds
                      (abs (- file-duration video-duration))))))))
       '(("main" :youtube-url)
         ("answers" :qa-youtube-url))))
    (emacsconf-publish-prepare-for-display (emacsconf-get-talk-info)))))

Then I got the commands to trim the videos. 

 (mapconcat (lambda (row)
              (let ((talk (emacsconf-resolve-talk (elt row 1))))
                (format "ffmpeg -y -i %s--%s.webm -t %s -c copy %s--%s--trimmed.webm"
                        (plist-get talk :file-prefix)
                        (car row)
                        (concat (elt row 3) ".000")
                        (plist-get talk :file-prefix)
                        (car row))))
            (cdr to-trim)
            "\n"))

After quickly checking the results, I copied them over to the original videos, updated the video data in my conf.org, and republished the info pages in the wiki.

The time I spent on figuring out how to talk to the YouTube API feels like it's paying off.

View or add comments (Disqus), or e-mail me at sacha@sachachua.com

EmacsConf backstage: Using Spookfox to publish YouTube and Toobnix video drafts

| emacsconf, emacs, spookfox, youtube, video

I ran into quota limits when uploading videos to YouTube with a command-line tool, so I uploaded videos by selecting up to 15 videos at a time using the web-based interface. Each video was a draft, though, and I was having a hard time updating its visibility through the API. I think it eventually worked, but in the meantime, I used this very hacky hack to look for the "Edit Draft" button and click through the screens to publish them.

emacsconf-extract-youtube-publish-video-drafts-with-spookfox: Look for drafts and publish them.
(defun emacsconf-extract-youtube-publish-video-drafts-with-spookfox ()
  "Look for drafts and publish them."
  (while (not (eq (spookfox-js-injection-eval-in-active-tab
                   "document.querySelector('.edit-draft-button div') != null" t) :false))
    (progn
      (spookfox-js-injection-eval-in-active-tab
       "document.querySelector('.edit-draft-button div').click()" t)
      (sleep-for 2)
      (spookfox-js-injection-eval-in-active-tab
       "document.querySelector('#step-title-3').click()" t)
      (when (spookfox-js-injection-eval-in-active-tab
             "document.querySelector('tp-yt-paper-radio-button[name=\"PUBLIC\"] #radioLabel').click()" t)
        (spookfox-js-injection-eval-in-active-tab
         "document.querySelector('#done-button').click()" t)
        (while (not (eq  (spookfox-js-injection-eval-in-active-tab
                          "document.querySelector('#close-button .label') == null" t)
                         :false))
          (sleep-for 1))

        (spookfox-js-injection-eval-in-active-tab
         "document.querySelector('#close-button .label').click()" t)
        (sleep-for 1)))))

Another example of a hacky Spookfox workaround was publishing the unlisted videos. I couldn't figure out how to properly authenticate with the Toobnix (Peertube) API to change the visibility of videos. Peertube uses AngularJS components in the front end, so using .click() on the input elements didn't seem to trigger anything. I found out that I needed to use .dispatchEvent(new Event('input')) to tell the dropdown for the visibility to display the options. source

emacsconf-extract-toobnix-publish-video-from-edit-page: Messy hack to set a video to public and store the URL.
(defun emacsconf-extract-toobnix-publish-video-from-edit-page ()
  "Messy hack to set a video to public and store the URL."
  (interactive)
  (spookfox-js-injection-eval-in-active-tab "document.querySelector('label[for=privacy]').scrollIntoView(); document.querySelector('label[for=privacy]').closest('.form-group').querySelector('input').dispatchEvent(new Event('input'));" t)
  (sit-for 1)
  (spookfox-js-injection-eval-in-active-tab "document.querySelector('span[title=\"Anyone can see this video\"]').click()" t)
  (sit-for 1)
  (spookfox-js-injection-eval-in-active-tab "document.querySelector('button.orange-button').click()" t)(sit-for 3)
  (emacsconf-extract-store-url)
  (shell-command "xdotool key Alt+Tab sleep 1 key Ctrl+w Alt+Tab"))

It's a little nicer using Spookfox to automate browser interactions than using xdotool, since I can get data out of it too. I could also have used Puppeteer from either Python or NodeJS, but it's nice staying with Emacs Lisp. Spookfox has some Javascript limitations (can't close windows, etc.), so I might still use bits of xdotool or Puppeteer to work around that. Still, it's nice to now have an idea of how to talk to AngularJS components.

View or add comments (Disqus), or e-mail me at sacha@sachachua.com

EmacsConf backstage: Making a (play)list, checking it twice

| emacs, emacsconf, spookfox, youtube, video

I wanted the EmacsConf 2023 Youtube and Toobnix playlists to mostly reflect the schedule of the conference by track, with talks followed by their Q&A sessions (if recorded).

The list

I used Emacs Lisp to generate a list of videos in the order I wanted. That Sunday closing remarks aren't actually in the playlists because they're combined with the Q&A for my session on how we run Emacsconf.

emacsconf-extract-check-playlists: Return a table for checking playlist order.
(defun emacsconf-extract-check-playlists ()
  "Return a table for checking playlist order."
  (let ((pos 0))
    (seq-mapcat (lambda (o)
                  (delq
                   nil
                   (list
                    (when (emacsconf-talk-file o "--main.webm")
                      (cl-incf pos)
                      (list pos
                            (plist-get o :title)
                            (org-link-make-string
                             (plist-get o :youtube-url)
                             "YouTube")
                            (org-link-make-string
                             (plist-get o :toobnix-url)
                             "Toobnix")))
                    (when (emacsconf-talk-file o "--answers.webm")
                      (cl-incf pos)
                      (list pos (concat "Q&A: " (plist-get o :title))
                            (org-link-make-string
                             (plist-get o :qa-youtube-url)
                             "YouTube")
                            (org-link-make-string
                             (plist-get o :qa-toobnix-url)
                             "Toobnix"))))))
                (emacsconf-publish-prepare-for-display (emacsconf-get-talk-info)))))

1 An Org-Mode based text adventure game for learning the basics of Emacs, inside Emacs, written in Emacs Lisp YouTube Toobnix
2 Authoring and presenting university courses with Emacs and a full libre software stack YouTube Toobnix
3 Q&A: Authoring and presenting university courses with Emacs and a full libre software stack YouTube Toobnix
4 Teaching computer and data science with literate programming tools YouTube Toobnix
5 Q&A: Teaching computer and data science with literate programming tools YouTube Toobnix
6 Who needs Excel? Managing your students qualifications with org-table YouTube Toobnix
7 one.el: the static site generator for Emacs Lisp Programmers YouTube Toobnix
8 Q&A: one.el: the static site generator for Emacs Lisp Programmers YouTube Toobnix
9 Emacs turbo-charges my writing YouTube Toobnix
10 Q&A: Emacs turbo-charges my writing YouTube Toobnix
11 Why Nabokov would use Org-Mode if he were writing today YouTube Toobnix
12 Q&A: Why Nabokov would use Org-Mode if he were writing today YouTube Toobnix
13 Collaborative data processing and documenting using org-babel YouTube Toobnix
14 How I play TTRPGs in Emacs YouTube Toobnix
15 Q&A: How I play TTRPGs in Emacs YouTube Toobnix
16 Org-Mode workflow: informal reference tracking YouTube Toobnix
17 (Un)entangling projects and repos YouTube Toobnix
18 Emacs development updates YouTube Toobnix
19 Emacs core development: how it works YouTube Toobnix
20 Top 10 ways Hyperbole amps up Emacs YouTube Toobnix
21 Using Koutline for stream of thought journaling YouTube Toobnix
22 Parallel text replacement YouTube Toobnix
23 Q&A: Parallel text replacement YouTube Toobnix
24 Eat and Eat powered Eshell, fast featureful terminal inside Emacs YouTube Toobnix
25 The browser in a buffer YouTube Toobnix
26 Speedcubing in Emacs YouTube Toobnix
27 Emacs MultiMedia System (EMMS) YouTube Toobnix
28 Q&A: Emacs MultiMedia System (EMMS) YouTube Toobnix
29 Programming with steno YouTube Toobnix
30 Mentoring VS-Coders as an Emacsian (or How to show not tell people about the wonders of Emacs) YouTube Toobnix
31 Q&A: Mentoring VS-Coders as an Emacsian (or How to show not tell people about the wonders of Emacs) YouTube Toobnix
32 Emacs saves the Web (maybe) YouTube Toobnix
33 Q&A: Emacs saves the Web (maybe) YouTube Toobnix
34 Sharing Emacs is Caring Emacs: Emacs education and why I embraced video YouTube Toobnix
35 Q&A: Sharing Emacs is Caring Emacs: Emacs education and why I embraced video YouTube Toobnix
36 MatplotLLM, iterative natural language data visualization in org-babel YouTube Toobnix
37 Enhancing productivity with voice computing YouTube Toobnix
38 Q&A: Enhancing productivity with voice computing YouTube Toobnix
39 LLM clients in Emacs, functionality and standardization YouTube Toobnix
40 Q&A: LLM clients in Emacs, functionality and standardization YouTube Toobnix
41 Improving compiler diagnostics with overlays YouTube Toobnix
42 Q&A: Improving compiler diagnostics with overlays YouTube Toobnix
43 Editor Integrated REPL Driven Development for all languages YouTube Toobnix
44 REPLs in strange places: Lua, LaTeX, LPeg, LPegRex, TikZ YouTube Toobnix
45 Literate Documentation with Emacs and Org Mode YouTube Toobnix
46 Q&A: Literate Documentation with Emacs and Org Mode YouTube Toobnix
47 Windows into Freedom YouTube Toobnix
48 Bringing joy to Scheme programming YouTube Toobnix
49 Q&A: Bringing joy to Scheme programming YouTube Toobnix
50 GNU Emacs: A World of Possibilities YouTube Toobnix
51 Q&A: GNU Emacs: A World of Possibilities YouTube Toobnix
52 A modern Emacs look-and-feel without pain YouTube Toobnix
53 The Emacsen family, the design of an Emacs and the importance of Lisp YouTube Toobnix
54 Q&A: The Emacsen family, the design of an Emacs and the importance of Lisp YouTube Toobnix
55 emacs-gc-stats: Does garbage collection actually slow down Emacs? YouTube Toobnix
56 Q&A: emacs-gc-stats: Does garbage collection actually slow down Emacs? YouTube Toobnix
57 hyperdrive.el: Peer-to-peer filesystem in Emacs YouTube Toobnix
58 Q&A: hyperdrive.el: Peer-to-peer filesystem in Emacs YouTube Toobnix
59 Writing a language server in OCaml for Emacs, fun, and profit YouTube Toobnix
60 Q&A: Writing a language server in OCaml for Emacs, fun, and profit YouTube Toobnix
61 What I learned by writing test cases for GNU Hyperbole YouTube Toobnix
62 Q&A: What I learned by writing test cases for GNU Hyperbole YouTube Toobnix
63 EmacsConf.org: How we use Org Mode and TRAMP to organize and run a multi-track conference YouTube Toobnix
64 Q&A: EmacsConf.org: How we use Org Mode and TRAMP to organize and run a multi-track conference YouTube Toobnix
65 Saturday opening remarks YouTube Toobnix
66 Saturday closing remarks YouTube Toobnix
67 Sunday opening remarks YouTube Toobnix
68 Sunday closing remarks YouTube Toobnix

YouTube

I bulk-added the Youtube videos to the playlist. The videos were not in order because I uploaded some late submissions and forgotten videos, which then got added to the end of the list.

I tried using the API to sort the playlist. This got it most of the way there, and then I sorted the rest by hand.

emacsconf-extract-youtube-api-sort-playlist: Try to roughly sort the playlist.
(defun emacsconf-extract-youtube-api-sort-playlist (&optional dry-run-only)
  "Try to roughly sort the playlist."
  (interactive)
  (setq emacsconf-extract-youtube-api-playlist (seq-find (lambda (o) (let-alist o (string= .snippet.title (concat emacsconf-name " " emacsconf-year))))
                                        (assoc-default 'items emacsconf-extract-youtube-api-playlists)))
  (setq emacsconf-extract-youtube-api-playlist-items
        (emacsconf-extract-youtube-api-paginated-request (concat "https://youtube.googleapis.com/youtube/v3/playlistItems?part=snippet,contentDetails,status&forMine=true&order=date&maxResults=100&playlistId="
                                                (url-hexify-string (assoc-default 'id emacsconf-extract-youtube-api-playlist)))))
  (let* ((playlist-info emacsconf-extract-youtube-api-playlists)
         (playlist-items emacsconf-extract-youtube-api-playlist-items)
         (info (emacsconf-publish-prepare-for-display (emacsconf-get-talk-info)))
         (slugs (seq-map (lambda (o) (plist-get o :slug)) info))
         (position (1- (length playlist-items)))
         result)
    ;; sort items
    (mapc (lambda (talk)
            (when (plist-get talk :qa-youtube-id)
              ;; move the q & a
              (let ((video-object (emacsconf-extract-youtube-find-url-video-in-list
                                   (plist-get talk :qa-youtube-url)
                                   playlist-items)))
                (let-alist video-object
                  (cond
                   ((null video-object)
                    (message "Could not find video for %s" (plist-get talk :slug)))
                   ;; not in the right position, try to move it
                   ((< .snippet.position position)
                    (let ((video-id .id)
                          (playlist-id .snippet.playlistId)
                          (resource-id .snippet.resourceId))
                      (message "Trying to move %s Q&A to %d from %d" (plist-get talk :slug) position .snippet.position)
                      (add-to-list 'result (list (plist-get talk :slug) "answers" .snippet.position position))
                      (unless dry-run-only
                        (plz 'put "https://www.googleapis.com/youtube/v3/playlistItems?part=snippet"
                          :headers `(("Authorization" . ,(url-oauth-auth "https://youtube.googleapis.com/youtube/v3/"))
                                     ("Accept" . "application/json")
                                     ("Content-Type" . "application/json"))
                          :body (json-encode
                                 `((id . ,video-id)
                                   (snippet
                                    (playlistId . ,playlist-id)
                                    (resourceId . ,resource-id)
                                    (position . ,position))))))))))
                (setq position (1- position))))
            ;; move the talk if needed
            (let ((video-object
                   (emacsconf-extract-youtube-find-url-video-in-list
                    (plist-get talk :youtube-url)
                    playlist-items)))
              (let-alist video-object
                (cond
                 ((null video-object)
                  (message "Could not find video for %s" (plist-get talk :slug)))
                 ;; not in the right position, try to move it
                 ((< .snippet.position position)
                  (let ((video-id .id)
                        (playlist-id .snippet.playlistId)
                        (resource-id .snippet.resourceId))
                    (message "Trying to move %s to %d from %d" (plist-get talk :slug) position .snippet.position)
                    (add-to-list 'result (list (plist-get talk :slug) "main" .snippet.position position))
                    (unless dry-run-only
                      (plz 'put "https://www.googleapis.com/youtube/v3/playlistItems?part=snippet"
                        :headers `(("Authorization" . ,(url-oauth-auth "https://youtube.googleapis.com/youtube/v3/"))
                                   ("Accept" . "application/json")
                                   ("Content-Type" . "application/json"))
                        :body (json-encode
                               `((id . ,video-id)
                                 (snippet
                                  (playlistId . ,playlist-id)
                                  (resourceId . ,resource-id)
                                  (position . ,position))))))
                    ))))
              (setq position (1- position))))
          (nreverse info))
    result))

I needed to sort some of the videos manually. Trying to scroll by dragging items to the top of the currently-displayed section of the list was slow, and dropping the item near the top of the list so that I could pick it up again after paging up was a little disorienting. Fortunately, keyboard scrolling with page-up and page-down worked even while dragging an item, so that was what I ended up doing: select the item and then page-up while dragging.

YouTube doesn't display numbers for the playlist positions, but this will add them. The numbers don't dynamically update when the list is reordered, so I just re-ran the code after moving things around.

emacsconf-extract-youtube-spookfox-add-playlist-numbers: Number the playlist for easier checking.
(defun emacsconf-extract-youtube-spookfox-add-playlist-numbers ()
  "Number the playlist for easier checking.
Related: `emacsconf-extract-check-playlists'."
  (interactive)
  (spookfox-js-injection-eval-in-active-tab "[...document.querySelectorAll('ytd-playlist-video-renderer')].forEach((o, i) => { o.querySelector('.number')?.remove(); let div = document.createElement('div'); div.classList.add('number'); div.textContent = i; o.prepend(div) }))" t))

2023-12-11_12-57-25.png
Figure 1: Adding numbers to the Youtube playlist

In retrospect, I could probably have just cleared the playlist and then added the videos using the in the right order instead of fiddling with inserting things.

Toobnix (Peertube)

Toobnix (Peertube) doesn't seem to have a way to bulk-add videos to a playlist (or even to bulk-set their visibility). I started trying to figure out how to use the API, but I got stuck because my token didn't seem to let me access unlisted videos or do other things that required proper authentication. Anyway, I came up with this messy hack to open the videos in sequence and add them to the playlist using Spookfox. 

(defun emacsconf-extract-toobnix-set-up-playlist ()
  (interactive)
  (mapcar
   (lambda (o)
     (when (plist-get o :toobnix-url)
       (browse-url (plist-get o :toobnix-url))
       (read-key "press a key when page is loaded")
       (spookfox-js-injection-eval-in-active-tab "document.querySelector('.action-button-save').click()" t)
       (spookfox-js-injection-eval-in-active-tab "document.querySelector('my-peertube-checkbox').click()" t)
       (read-key "press a key when saved to playlist"))
     (when (plist-get o :qa-toobnix-url)
       (browse-url (plist-get o :qa-toobnix-url))
       (read-key "press a key when page is loaded")
       (spookfox-js-injection-eval-in-active-tab "document.querySelector('.action-button-save').click()" t)
       (spookfox-js-injection-eval-in-active-tab "document.querySelector('my-peertube-checkbox').click()" t)
       (read-key "press a key when saved to playlist")))
   (emacsconf-publish-prepare-for-display (emacsconf-get-talk-info))))

Maybe next year, I might be able to figure out how to use the APIs to do this stuff automatically.

This code is in emacsconf-extract.el.

View or add comments (Disqus), or e-mail me at sacha@sachachua.com