2024-01-21 Yay Emacs: copy link, Spookfox + Org Babel, choosing what to hack on, SVG highlighting, ical

| yay-emacs, emacs

<2024-01-21 Sun 07:30-08:15>

Very rough notes, just gotta get them out there! =)

  • 7:24:43 AM Hello and thanks for joining me!
  • Meta information:
  • Do you have any Yay Emacs moments this week?
  • 7:31:23 AM trying vdo.ninja to get the video from the X230T to the Surface Book and using OBS from the Surface Book
  • 7:32:12 AM Org Mode custom link: copy to clipboard
  • 7:34:49 AM Running the current Org Mode Babel Javascript block from Emacs using Spookfox
  • 7:45:42 AM Emacs tweaks: Choosing what to hack on (also, I figured out dynamically highlighting SVG diagrams from Graphviz!)
  • 8:14:16 AM What are you looking forward to exploring next week?
    • Me, maybe:
      • using soundex and a list of commonly misrecognized words to correct transcripts
      • Map of Emacs communities for possible 5-minute lightning talk at LibrePlanet
      • smarter refiling: tag target, active thoughts/posts/projects, card/pile sorting
      • (Update: None of this happened, but that's cool! Figured out how to add closed captions for audio on my site and how to synchronize sketch highlights with the audio)

Also, my Evil Plan for Yay Emacs!

I'd love to hear your comments via YouTube live chat, Mastodon (I'm @sachac@emacs.ch), or e-mail (sacha@sachachua.com).

Thanks to:

Other notes:

Next livestream: Sat 2024-01-28 7:30 AM EST, https://yayemacs.com and https://youtube.com/@sachactube/live . I'll probably talk about learning more about Emacs Lisp functions using apropos-function, eldoc, marginalia, helpful, and elisp-demos. See you then!

View org source for this post

Updating Planet Venus so that planet.emacslife.com can handle mix-blend-mode in my SVGs

| geek, python

I wanted to easily turn segments from my Yay Emacs livestreams or recorded narration into closed-caption audio and dynamic highlighting of my SVG sketches and text transcripts. That way, people could easily jump around to sections they're interested in.

Not everyone has Javascript turned on, so I wanted to start with something that made sense even in RSS feeds like the one on Planet Emacslife (which strips out <style> and <script>) and was progressively enhanced with captions and highlighting if you saw it on my site.

2024-01-25_08-37-11.png
Figure 1: My SVGs were broken: black fill, no mix-blend-mode

The blog aggregator I'm using, Planet Venus, hasn't been updated in 14 years. It even uses Python 2. I considered switching to a different aggregator, so I started checking out different community planets. Most of the other planets listed in this HN thread about aggregators looked like they were using the same Planet Venus aggregator, although these were some planets that used something else:

I decided I'd stick with Planet Venus for now, since I could probably figure out how to get the attributes sorted out.

I found planet/vendor/feedparser.py by digging around. Adding mix-blend-mode to the list of attributes there was not enough to get it working. I started exploring pdb for interactive Python debugging inside Emacs, although I think dap is an option too. I wrote a short bit of code to test things out:

import sys,os
sys.path.insert(1, os.path.join(os.path.dirname(__file__), 'planet/vendor'))
from feedparser import _sanitizeHTML
assert 'strong' in _sanitizeHTML('<strong>Hello</strong>', 'utf-8', 'text/html')
assert 'mix-blend-mode' in _sanitizeHTML('<svg><path style="fill: red;mix-blend-mode:darken"></path></svg>', 'utf-8', 'text/html')

It was pretty easy to use pdb to start stepping through and into functions, although I didn't dig into it deeply because I figured it out another way. While looking through the pull requests for the Venus repository, I came across this pull request to add data- attributes which was helpful because it pointed me to planet/vendor/html5lib/sanitizer.py. Once I added mix-blend-mode to that one, things worked. Here's my Github branch.

2024-01-25_08-51-33.png
Figure 2: planet.emacslife.com now lets me use mix-blend-mode

On a somewhat related note, I also had to patching shr to handle SVG images with viewBox attributes. I guess SVGs aren't that common yet, but I'm looking forward to playing around with them more, so I might as well make things better (at least when it comes to things I can actually tweak). mix-blend-mode on SVG elements says it's not supported in Safari or a bunch of mobile browsers, but it seems to be working on my phone, so maybe that's cool now. Using mix-blend-mode means I don't have to do something complicated when it comes to animating highlights while still keeping text visible, and improving SVG support is the right thing to do. Onward!

View org source for this post

Patching elfeed and shr to handle SVG images with viewBox attributes

Posted: - Modified: | emacs

I want to use more SVG sketches in my blog posts. I would like them to look reasonable in elfeed. When I first checked them out, though, the images were a little too big to be comfortable.

To test things quickly, I figured out how to use Elfeed to display the first entry from a local file.

(with-temp-buffer (get-buffer-create "*elfeed*")
  (elfeed-show-entry
   (car
    (elfeed-entries-from-rss
     "file:///tmp/test/index.xml"
     (xml-parse-file "/tmp/test/index.xml")))))
2024-01-26_08-34-03.png
Figure 1: The image is a little too big to be comfortable

I'd just been using the default width and height from the pdftocairo import, but changing the width and height seemed like a good first step. I could fix this when I convert the file, export it from Org Mode as a my-include link, or transform it when processing it in the 11ty static site generator. Let's start by changing it in the SVG file itself.

(defun my-svg-resize (file width height)
  "Resize FILE to WIDTH and HEIGHT in pixels, keeping aspect ratio."
  (interactive "FSVG: \nnWidth: \nnHeight: ")
  (let* ((dom (xml-parse-file file))
         (orig-height (string-to-number (dom-attr dom 'height)))
         (orig-width (string-to-number (dom-attr dom 'width)))
         (aspect-ratio (/ (* 1.0 orig-width) orig-height))
         new-width new-height)
    (setq new-width width
          new-height (/ new-width aspect-ratio))
    (when (> new-height height)
      (setq new-height height
            new-width (* new-height aspect-ratio)))
    (dom-set-attribute dom 'width (format "%dpx" new-width))
    (dom-set-attribute dom 'height (format "%dpx" new-height))
    (with-temp-file file
      (xml-print dom))))

Even when I changed the width and height of the SVG image, the image didn't follow suit. Mysterious.

2024-01-26_08-34-32.png
Figure 2: SVG image cut off

After a bit of digging around using Edebug, I found out that elfeed uses shr, which uses libxml-parse-html-region, and that converts attributes to lowercase. This is generally what you want to do for HTML, since HTML tags and attributes are case-insensitive, but SVG tags are case-sensitive. It looks like other implementations work around this by special-casing attributes. libxml-parse-html-region is C code that calls a library function, so it's hard to tinker with, but I can at least fix the behaviour at the level of shr. I took the list of SVG attributes to correct case for and wrote this code to fix the attribute cases.

List of atttributes to correct
(defconst shr-correct-attribute-case
  '((attributename . attributeName)
    (attributetype . attributeType)
    (basefrequency . baseFrequency)
    (baseprofile . baseProfile)
    (calcmode . calcMode)
    (clippathunits . clipPathUnits)
    (diffuseconstant . diffuseConstant)
    (edgemode . edgeMode)
    (filterunits . filterUnits)
    (glyphref . glyphRef)
    (gradienttransform . gradientTransform)
    (gradientunits . gradientUnits)
    (kernelmatrix . kernelMatrix)
    (kernelunitlength . kernelUnitLength)
    (keypoints . keyPoints)
    (keysplines . keySplines)
    (keytimes . keyTimes)
    (lengthadjust . lengthAdjust)
    (limitingconeangle . limitingConeAngle)
    (markerheight . markerHeight)
    (markerunits . markerUnits)
    (markerwidth . markerWidth)
    (maskcontentunits . maskContentUnits)
    (maskunits . maskUnits)
    (numoctaves . numOctaves)
    (pathlength . pathLength)
    (patterncontentunits . patternContentUnits)
    (patterntransform . patternTransform)
    (patternunits . patternUnits)
    (pointsatx . pointsAtX)
    (pointsaty . pointsAtY)
    (pointsatz . pointsAtZ)
    (preservealpha . preserveAlpha)
    (preserveaspectratio . preserveAspectRatio)
    (primitiveunits . primitiveUnits)
    (refx . refX)
    (refy . refY)
    (repeatcount . repeatCount)
    (repeatdur . repeatDur)
    (requiredextensions . requiredExtensions)
    (requiredfeatures . requiredFeatures)
    (specularconstant . specularConstant)
    (specularexponent . specularExponent)
    (spreadmethod . spreadMethod)
    (startoffset . startOffset)
    (stddeviation . stdDeviation)
    (stitchtiles . stitchTiles)
    (surfacescale . surfaceScale)
    (systemlanguage . systemLanguage)
    (tablevalues . tableValues)
    (targetx . targetX)
    (targety . targetY)
    (textlength . textLength)
    (viewbox . viewBox)
    (viewtarget . viewTarget)
    (xchannelselector . xChannelSelector)
    (ychannelselector . yChannelSelector)
    (zoomandpan . zoomAndPan))
  "Attributes for correcting the case in SVG and MathML.
Based on https://html.spec.whatwg.org/multipage/parsing.html#parsing-main-inforeign .")

This recursive function does the actual replacements.

(defun shr-correct-dom-case (dom)
  "Correct the case for SVG segments."
  (dolist (attr (dom-attributes dom))
    (when-let ((rep (assoc-default (car attr) shr-correct-attribute-case)))
      (setcar attr rep)))
  (dolist (child (dom-children dom))
    (shr-correct-dom-case child))
  dom)

Then we can replace shr-tag-svg with this:

(defun shr-tag-svg (dom)
  (when (and (image-type-available-p 'svg)
             (not shr-inhibit-images)
             (dom-attr dom 'width)
             (dom-attr dom 'height))
    (funcall shr-put-image-function
             (list (shr-dom-to-xml (shr-correct-dom-case dom) 'utf-8)
                   'image/svg+xml)
             "SVG Image")))

The result is that the image now respects width, height, and viewBox:

2024-01-26_08-33-31.png
Figure 3: Fixed image

Here is a small test for shr-correct-dom-case:

(ert-deftest shr-correct-dom-case ()
  (let ((case-fold-search nil))
    (should
     (string-match
      "viewBox"
      (shr-dom-to-xml
       (shr-correct-dom-case
        (with-temp-buffer
          (insert "<svg viewBox=\"0 0 100 100\"></svg>")
          (libxml-parse-html-region (point-min) (point-max)))))))))

And another test involving displaying an image:

(with-current-buffer (get-buffer-create "*test*")
  (erase-buffer)
  (insert "<svg width=\"100px\" height=\"100px\" viewBox=\"0 0 200 200\"><circle cx=\"100\" cy=\"100\" r=\"100\"/></svg>\n")
  (shr-insert-document (libxml-parse-html-region (point-min) (point-max)))
  (display-buffer (current-buffer)))
2024-01-26_09-09-48.png
Figure 4: Correct output: full circle

shr.el is in Emacs core, so I'll need to turn this into a patch and send it to emacs-devel at some point.

View org source for this post

My Evil Plan for Yay Emacs!

| yay-emacs, emacs, planning

Here's a clip from my 2024-01-21 Yay Emacs livestream about my goals for Yay Emacs and the built-in payoffs that I think will help me keep doing it.

  • 00:00.000: Getting more ideas into blog posts and workflow demos: Now, also, I have an evil plan. My evil plan is that this is a good way for me to get ideas and convert them into blog posts and code and then do the workflow demos, because it's sometimes really difficult to see how to use something from just the code.
  • 00:20.340: I have fun tickling my brain: This process is fun. Tweaking Emacs is fun for me.
  • 00:24.840: I learn from other people's comments, questions: Also, if I do this out loud, other people can help out with questions and comments, like the way that you're all doing now, which is great. Fantastic.
  • 00:35.200: Other people pick up ideas: Of course, those are all very selfish reasons. So I'm hoping other people are getting something out of this too. (Hello, 19 people who are watching, and also for some reason, the hundreds of people who check these videos out afterwards. Great, fantastic.) I'm hoping you pick up some ideas from the crazy things that we like to play with in terms of Emacs.
  • 00:57.440: We bounce ideas around and make lots of progress: My medium-term plan there is then to start seeing how those ideas get transformed when they get bounced off other people and other people bounce ideas back. Because that's one of the fun things about Emacs, right? Everything is so personalizable that seeing how one workflow idea gets transformed into somebody else's life, you learn something from that process. I'm really looking forward to how bouncing ideas around will work here.
  • 01:32.200: More people share more: Especially if we can find little things that make doing things more fun or they make it take less effort–then maybe more people will share more things, and then I get to learn from that also,
  • 01:46.940: Building up an archive: which is fantastic because long term, this can help build up an archive. Then people can go into that archive and find things without necessarily waiting for me. I don't become the bottleneck. People can just go in there and find… "Oh, you remember that time that I saw this interesting idea about SVG highlighting or whatever." You can just go in there and try to find more information.
  • 02:11.640: More people join and thrive in the Emacs community: So that's great. Then ideally, as people find the things that resonate with them, the cool demos that say this text editor can be extended to do audio editing and animation and all that crazy stuff, then more people will come and join and share what they're learning, and then move on to building stuff maybe for themselves and for other people, and then it'll be even more amazing.
  • 02:42.780: I could be a voice in people's heads: And lastly, this is kind of odd, but having listened in the background to so many of the kiddo's current viewing habits, her favorite YouTube channels like J Perm or Cubehead or Tingman for Rubik's cube videos or Eyecraftmc for Minecraft, I'm beginning to appreciate kind of the value of having these mental models of other people in your head. I can imagine how they talk and all that stuff. I am looking forward to watching more Emacs videos, which I haven't done in a while because usually for Emacs News, I'm just skimming through the transcripts super quickly on account of (A), lots of videos and (B), not much time. So this idea of getting other people's voices into your head, or possibly becoming a voice in somebody's head, I think there might be something interesting there. Of course, the buzz these days is, "oh no, AI voice cloning, this is a safety issue and all of that stuff." But I think there are positive uses for this as well, in the sense that… As qzump says, you know, they are like, "I have no Emacs friends and you're speaking to my soul." A lot of us are doing this in isolation. We don't normally meet other people. So the more voices we can have in our heads of actual people who enjoy doing these things, the less weird we feel. Or actually, more like… the more weird we feel, but in a good way, like there's a tribe, right? If sharing more ideas in a multimedia sort of way, like with either audio narration with images or this webcam thing that we're trying (my goodness, I have to actually dress up) helps people build these mental models in their head… Hey, one of the nice things about this webcam thing is I can make hand gestures. Cool, cool. Might be interesting. If, while you're hacking on Emacs, you can imagine me cheering you on and saying, "That's fantastic. Have you thought about writing a blog post about that so that we get that into Planet Emacs Life and, and then into Emacs News? Please share what you're learning." It'll be great. So, yeah, maybe that's a thing. So that's my evil plan for Yay Emacs.
View org source for this post

2024-01-22 Emacs news

| emacs, emacs-news

Links from reddit.com/r/emacs, r/orgmode, r/spacemacs, r/planetemacs, Hacker News, lobste.rs, kbin, programming.dev, planet.emacslife.com, YouTube, the Emacs NEWS file, Emacs Calendar, and emacs-devel. Thanks to Andrés Ramírez for emacs-devel links. Do you have an Emacs-related link or announcement? Please e-mail me at sacha@sachachua.com. Thank you!

View org source for this post

Yay Emacs: Using elisp: links in Org Mode to note the time and display messages on stream

| yay-emacs, org

I like adding chapters to my videos so that people can jump to sections. I can figure out the sections by reading the transcript, adding NOTE comments, and extracting the times for those with my-youtube-copy-chapters. It could be nice to capture the times on the fly. org-timer could let me insert relative timestamps, but I think it might need some tweaking to synchronize that with when the stream starts according to YouTube. I've set up a capture, too, so I can take notes with timestamps.

It turns out that I don't have a lot of mental bandwidth when I'm on stream, so it's hard to remember keyboard shortcuts. (Maybe if I practise using the hydra I set up…) Fortunately, Org Mode's elisp: link type makes it easy to set up executable shortcuts. For example, I can add links like [[elisp:my-stream-message-link][TODO]] to my livestream plans like this:

2024-01-20-elisp-links.svg
Figure 1: Shortcuts with elisp: links

I can then click on the links or use C-c C-o (org-open-link-at-point) to run the function. When I follow the TODO link in the first item, Emacs displays a clock and a message based on the rest of the line after the link.

2024-01-20-message.svg
Figure 2: Displaying a clock and a message

In the background, the code also sets the description of the link to the wall-clock time.

2024-01-20-time.svg
Figure 3: Link description updated with the time

If I start the livestream with a clock displayed on screen, I can use that to translate wall-clock times to relative time offsets. I'll probably figure out some Elisp to translate the times automatically at some point, maybe based on something like org-timer-change-times-in-region.

I figured it might be fun to add a QR code automatically if we detect a URL, taking advantage of that qrencode package I started playing around with.

2024-01-20-qr.svg
Figure 4: With a QR code

You can also use elisp: links for more complicated Emacs Lisp functions, like this: elisp:(progn ... ...).

Here's the code that makes it happen. It's based on emacsconf-stream.el.

(defvar my-stream-message-buffer "*Yay Emacs*")
(defvar my-stream-message-timer nil)

(defun my-stream-message-link ()
  (interactive)
  (save-excursion
    (when (and (derived-mode-p 'org-mode)
               (eq (org-element-type (org-element-context)) 'link))
      (my-stream-update-todo-description-with-time)
      (goto-char (org-element-end (org-element-context)))
      (my-stream-message (org-export-string-as (buffer-substring (point) (line-end-position)) 'ascii t)))))
(defun my-stream-update-todo-description-with-time ()
  (when (and (derived-mode-p 'org-mode)
             (eq (org-element-type (org-element-context)) 'link))
    (my-org-update-link-description (format-time-string "%-I:%M:%S %p"))))

(defun my-stream-message (&optional message)
  (interactive "MMessage: ")
  ;; update the description of the link at point to be the current time, if any
  (switch-to-buffer (get-buffer-create my-stream-message-buffer))
  (erase-buffer)
  (delete-other-windows)
  (when (string= message "") (setq message nil))
  (face-remap-add-relative 'default :height 200)
  (insert
   "Yay Emacs! - Sacha Chua (sacha@sachachua.com)\n"
   (propertize
    "date"
    'stream-time (lambda () (format-time-string "%Y-%m-%d %H:%M:%S %Z (%z)")))
   "\n\n"
   message)
  ;; has a URL? Let's QR encode it!
  (when-let ((url (save-excursion
                    (when (re-search-backward ffap-url-regexp nil t)
                      (thing-at-point-url-at-point)))))
    (insert (propertize (qrencode url) 'face '(:height 50)) "\n"))
  (insert  "\nYayEmacs.com\n")
  (when (timerp my-stream-message-timer) (cancel-timer my-stream-message-timer))
  (my-stream-update-time)
  (setq my-stream-message-timer (run-at-time t 1 #'my-stream-update-time))
  (goto-char (point-min)))

(defun my-stream-update-time ()
  "Update the displayed time."
  (if (get-buffer my-stream-message-buffer)
      (when (get-buffer-window my-stream-message-buffer)
        (with-current-buffer my-stream-message-buffer
          (save-excursion
            (goto-char (point-min))
            (let (match)
              (while (setq match (text-property-search-forward 'stream-time))
                (goto-char (prop-match-beginning match))
                (add-text-properties
                 (prop-match-beginning match)
                 (prop-match-end match)
                 (list 'display
                       (funcall (get-text-property
                                 (prop-match-beginning match)
                                 'stream-time))))
                (goto-char (prop-match-end match)))))))
    (when (timerp my-stream-message-timer)
      (cancel-timer my-stream-message-timer))))

Let's see if that makes it easy enough for me to remember to actually do it!

View org source for this post

Running the current Org Mode Babel Javascript block from Emacs using Spookfox

| emacs, org, spookfox

I often want to send Javascript from Emacs to the web browser. It's handy for testing code snippets or working with data on pages that require Javascript or authentication. I could start Google Chrome or Mozilla Firefox with their remote debugging protocols, copy the websocket URLs, and talk to the browser through something like Puppeteer, but it's so much easier to use the Spookfox extension for Mozilla to execute code in the active tab. spookfox-js-injection-eval-in-active-tab lets you evaluate Javascript and get the results back in Emacs Lisp.

I wanted to be able to execute code even more easily. This code lets me add a :spookfox t parameter to Org Babel Javascript blocks so that I can run the block in my Firefox active tab. For example, if I have (spookfox-init) set up, Spookfox connected, and https://planet.emacslife.com in my active tab, I can use it with the following code:

#+begin_src js :eval never-export :spookfox t :exports results
[...document.querySelectorAll('.post > h2')].slice(0,5).map((o) => '- ' + o.textContent.trim().replace(/[ \n]+/g, ' ') + '\n').join('')
#+end_src
  • Mario Jason Braganza: Updated to Emacs 29.2
  • Irreal: Zamansky: Learning Elisp #16
  • Tim Heaney: Lisp syntax
  • Erik L. Arneson: Many Posts of Interest for January 2024
  • William Denton: Basic citations in Org (Part 4)

Evaluating a Javascript block with :spookfox t

To do this, we wrap some advice around the org-babel-execute:js function that's called by org-babel-execute-src-block.

(defun my-org-babel-execute:js-spookfox (old-fn body params)
  "Maybe execute Spookfox."
  (if (assq :spookfox params)
      (spookfox-js-injection-eval-in-active-tab
       body t)
    (funcall old-fn body params)))
(with-eval-after-load 'ob-js
  (advice-add 'org-babel-execute:js :around #'my-org-babel-execute:js-spookfox))

I can also run the block in Spookfox without adding the parameter if I make an interactive function:

(defun my-spookfox-eval-org-block ()
  (interactive)
  (let ((block (org-element-context)))
    (when (and (eq (org-element-type block) 'src-block)
               (string= (org-element-property :language block) "js"))
      (spookfox-js-injection-eval-in-active-tab
       (nth 2 (org-src--contents-area block))
       t))))

I can add that as an Embark context action:

(with-eval-after-load 'embark-org
  (define-key embark-org-src-block-map "f" #'my-spookfox-eval-org-block))

In Javascript buffers, I want the ability to send the current line, region, or buffer too, just like nodejs-repl does.

(defun my-spookfox-send-region (start end)
  (interactive "r")
  (spookfox-js-injection-eval-in-active-tab (buffer-substring start end) t))

(defun my-spookfox-send-buffer ()
  (interactive)
  (my-spookfox-send-region (point-min) (point-max)))

(defun my-spookfox-send-line ()
  (interactive)
  (my-spookfox-send-region (line-beginning-position) (line-end-position)))

(defun my-spookfox-send-last-expression ()
  (interactive)
  (my-spookfox-send-region (save-excursion (nodejs-repl--beginning-of-expression)) (point)))

(defvar-keymap my-js-spookfox-minor-mode-map
  :doc "Send parts of the buffer to Spookfox."
  "C-x C-e" 'my-spookfox-send-last-expression
  "C-c C-j" 'my-spookfox-send-line
  "C-c C-r" 'my-spookfox-send-region
  "C-c C-c" 'my-spookfox-send-buffer)

(define-minor-mode my-js-spookfox-minor-mode "Send code to Spookfox.")

I usually edit Javascript files with js2-mode, so I can use my-js-spookfox-minor-mode in addition to that.

I can turn the minor mode on automatically for :spookfox t source blocks. There's no org-babel-edit-prep:js yet, I think, so we need to define it instead of advising it.

(defun org-babel-edit-prep:js (info)
  (when (assq :spookfox (nth 2 info))
    (my-js-spookfox-minor-mode 1)))

Let's try it out by sending the last line repeatedly:

Sending the current line

I used to do this kind of interaction with Skewer, which also has some extra stuff for evaluating CSS and HTML. Skewer hasn't been updated in a while, but maybe I should also check that out again to see if I can get it working.

Anyway, now it's just a little bit easier to tinker with Javascript!

View org source for this post
This is part of my Emacs configuration.