Through blogging, we discover our thoughts and other people

| connecting, blogging, writing
Text and links from sketch

Through blogging, we discover our thoughts and other people.

Henrik Karlsson's "Advice for a friend who wants to start a blog" nudged me to explore two threads of thought:

Writing helps you refine your thoughts:

  • This reminds me of Sonke Ahrens's How to Take Smart Notes and David Bessis's Mathematica.
    • Everything drives toward writing; writing is how to clarify your thoughts
    • Writing helps you improve your intuition, which feeds your writing.
  • I want to understand:
    • What's possible?
    • What's easier? what's harder (for now?)
    • How can we make things. easier? How can we make more things doable?
  • and also:
    • What am I thinking?
    • what do I want to try?
  • I want to get better at this through practice.

Writing helps you find your tribe:

  • Definitely - and the more idiosyncratic my posts are, the more amazing it is when someone resonates with it, even years later.
  • (I was amused to see him trace his tweet's flow through Stian Håklev, who reached out for a conversation in 2010 about peer-to-peer education because he read my blog.)
  • I deliberately boost my tribe's information flow:
  • I want to get better at this by
    • following my curiosity
    • improving search and serendipity
    • connecting people & ideas with community infrastructure and resources

Both sides: Because it's fun and leads to more awesomeness.

View org source for this post

Playing Minecraft parkour together

| minecraft

A+ eight years old and she won't always want to play with me, so while I can, we play together. Lately, she's been interested in Minecraft parkour maps.

I remember when parkour was too frustrating for us to do. In the beginning, I felt my stomach flip every time I fell. I kept having to tell myself that I wasn't the one actually falling. When we first played through Path of the Jedi, we spent a long, long time on the stage with the parkour challenge until we found out that we could bounce on the leaves to reset the timers. That was too hard for her before, so I ended up moving her character through that stage.

Now, she enjoys the challenge. She zips ahead through the different levels. Whenever she falls, she just gets back up and does it again. "It's frustration practice," she tells me cheerfully, my words echoing from her mouth. I'm getting slightly better at it too. My stomach no longer flip-flops, and I can sometimes get those double-slime jumps on the first try. It still takes me multiple tries to get through a puzzle that she does in one go, though. Still, I have the most patient tour guide and teacher. She often stays a step or two ahead of me so that she can point out where to go. Sometimes I'm the one who spots the thing we need to do to progress. If there's a particularly tricky jump that I'm having a hard time with and she's already far ahead, she lets her character AFK while she stands beside me and cheers me on. I still tend to yelp when I fall down, which she finds amusing.

We started playing with Parkour Spiral 2, and we've been slowly working our way through other ones released by Hielke Maps. I liked the multiple paths in Parkour Town, the small, self-contained puzzles of Parkour Paradise, and the colours of Parkour Egg. We might not play Parkour Volcano though. We're not yet good at timed parkour, and we ended up cheating our way past the lava-is-rising stage in Parkour Pyramid. Anyway, once we get through those puzzles, there are tons of other maps.

I like how she works hard on figuring out puzzles and is so satisfied when she makes a hard jump. I'm glad I can semi-sort-of keep up with her, or at least trail along behind her at a pace that isn't too frustrating for her. It's fun figuring out tougher levels together.

I'm glad that she talks about the cycle of trying and failing and trying and learning, and how feelings play into it: building that frustration tolerance, trying hard things, feeling the satisfaction of accomplishment. She still gets frustrated in other contexts, but I'm confident it'll transfer. I'm learning, too.

View org source for this post

Improving subed-vtt parsing; using dedicated windows in Emacs; training my intuition

| emacs, subed

While putting together some notes on how to use subed.el with auto-generated YouTube captions, I decided to get subed-word-data working with the Youtube VTT format. The sample file I downloaded from one of my YouTube videos had a cue whose text started with a blank line, so I ended up redoing the way subed-vtt.el parsed cues. Twice, actually. The first time, I changed it to handle multiple blocks in cue text (separated by blank lines). Then I came across this test suite for WebVTT parsing and found out that the timing line for a cue doesn't have to be preceded by a blank line, so then I needed to change the VTT parsing again.

Fortunately, I inherited a large Buttercup test suite from subed.el's original author. I've been adding to it over the years. As I learned more about how I wanted subed.el to behave, I added more cases. Then I worked on shifting the code to behave the way I wanted it to. At first, I didn't quite understand what I wanted the code to do, but as I pinned down more test cases, I was able to figure it out.

This was the first time I used toggle-window-dedicated (C-x w d) extensively. This function keeps the same buffer displayed in the window instead of letting Emacs Lisp functions replace it. First, I set up a large window for my subed-vtt.el. I split the other side into a smaller window for my test-subed-vtt.el, another window for a temporary VTT file, and a window for output from whatever command I'm running. I set most of the windows to dedicated except for my temporary output window. I also enabled winner-mode just in case I messed things up so that I could restore with winner-undo. Dedicating the windows meant I didn't have to keep fussing around with my windows and buffers to get things back to where I wanted them to be. I did use C-x o (other-window) a lot, so maybe it'll be worth getting the hang of ace-window.

2025-01-28_12-40-04.png
Figure 1: Dedicated windows for the code, the test file, and a VTT buffer for interactive testing; a spare window for other output

Sometimes I wanted to focus on one of those small windows. prot/window-single-toggle was helpful for maximizing a window and then returning to the previous configuration.

I mostly evaluated or edebugged my code and then used my-buttercup-run-dwim to run a suite in my test-subed-vtt.el. sp-backward-up-sexp and sp-narrow-to-sexp were also helpful for navigating the suites and focusing on whatever I was working on.

I'm looking forward to exploring the other test cases from that repository. It feels good to get better as a coder.

I just finished reading Mathematica: a Secret World of Imagination and Curiosity, by David Bessis. The idea of training your intuition echoed in my mind as I wrote test cases and changed the code. I started to look forward to the gaps between my understanding of the spec and my understanding of the test cases, and the gaps between the test cases and my implementation. It felt almost like a conversation. Sometimes it was hard to translate an idea into code. I felt myself getting muddled and turned around. Whenever I noticed that, I just had to back up and start from something I understood, figure out what I was uncertain about, and then go from there.

I like the way that subed.el gives me tools for thinking about text, times, and metadata together. That abstraction has been helpful for editing audio and making videos. The more solid I can make it, the easier it will be to imagine other things that use those ideas.

View org source for this post

2025-01-27 Emacs news

| emacs, emacs-news

Links from reddit.com/r/emacs, r/orgmode, r/spacemacs, r/planetemacs, Mastodon #emacs, Bluesky #emacs, Hacker News, lobste.rs, programming.dev, lemmy.world, lemmy.ml, communick.news, planet.emacslife.com, YouTube, the Emacs NEWS file, Emacs Calendar, and emacs-devel. Thanks to Andrés Ramírez for emacs-devel links. Do you have an Emacs-related link or announcement? Please e-mail me at sacha@sachachua.com. Thank you!

View org source for this post

Scaling a BigBlueButton server down to a 1 GB node between uses

| geek, tech, emacsconf, emacs

Now that we've survived EmacsConf, I've been looking into running a BigBlueButton server so that various Emacs meetups can use it if they like instead of relying on Jitsi or other free video-conferencing services. (I spent some time looking into Galene, but I'm not quite sure that's ready for our uses yet, like this issue that LibrePlanet ran into with recording.)

BigBlueButton requires a server with at least 4 CPU cores and 8 GB of RAM to even start up, and it doesn't like to share with other services. This costs about USD 48+tax/month on Linode or USD 576+tax/year, which is not an efficient use of funds yet. I could delete it after each instance, but I've been having a hard time properly restoring it from backup after deploying to a new IP address. bbb-conf --setip doesn't seem to catch everything, so I was still getting curl errors related to verifying the certificate.

A reasonable in-between is to run it on Linode's lowest plan (1 core, 1GB RAM; USD 60+tax for the year) in between meetups, and then spin things up for maybe 6-12 hours around each meetup. If I go with the 4-core 8 GB setup, that would be an extra USD 0.43 - 0.86 USD per meetup, which is eminently doable. I could even go with the recommended configuration of 8 cores and 16 GB memory on a dedicated CPU plan (USD 0.216/hour, so USD 1.30 to 2.59 per meetup). This was the approach that we used while preparing for EmacsConf. Since I didn't have a lot of programming time, I scaled the node up to 4 core / 8GB RAM whenever I had time to work on it, and I scaled it down to 1GB at the end of each of my working sessions. I scaled it up to dedicated 8 core / 16 GB RAM for EmacsConf, during which we used roughly half of the CPU capacity in order to host a max of 107 simultaneous users over 7 meetings.

I reviewed my BigBlueButton setup notes in the EmacsConf organizers notebook and the 2024 notebook and set up a Linode instance under my account, so that I can handle the billing and also so that Amin Bandali doesn't get spammed by all the notifications (up, down, up, down…). And then I'll be able to just scale it up when EmacsConf comes around again, which is nice.

Anyway, BBB refuses to install on a machine with fewer than 4 cores or 8 GB RAM, but once you set it up, it'll valiantly thrash around even on an underpowered server, which makes working with the server over ssh a lot slower. Besides, that's not friendly to other people using the same server. I wanted to configure the services so that they would only run on a server of the correct size. It turns out that systemd will let you specify either ConditionMemory and ConditionCPUs in the unit configuration file, and that you can use files ending in .conf in a directory named like yourservicename.service.d to override part of the configuration. Clear examples were hard to find, so I wanted to share these notes.

Since ConditionMemory is specified in bytes (ex: 8000000000), I found ConditionCPUs to be easier to read.

I used this command to check if I'd gotten the syntax right:

systemd-analyze condition 'ConditionCPUs=>=4'

and then I wrote this script to set up the overrides:

CPUS_REQUIRED=4
for ID in coturn.service redis-server.service bigbluebutton.target multi-user.target bbb-graphql-server.service haproxy.service bbb-rap-resque-worker.service bbb-webrtc-sfu.service bbb-fsesl-akka.service bbb-webrtc-recorder.service bbb-pads.service bbb-export-annotations.service bbb-web.service freeswitch.service etherpad.service bbb-rap-starter.service bbb-rap-caption-inbox.service freeswitch.service bbb-apps-akka.service bbb-graphql-actions postgresql@14-main.service; do
    mkdir -p /etc/systemd/system/$ID.d
    printf "[Unit]\nConditionCPUs=>=$CPUS_REQUIRED\n" > /etc/systemd/system/$ID.d/require-cpu.conf
done
systemctl daemon-reload
systemd-analyze verify bigbluebutton.target

It seems to work. When I use linode-cli to resize to the testing size, BigBlueButton works:

linode-cli linodes resize $BBB_ID --type g6-standard-4 --allow_auto_disk_resize false
sleep 5m
linode-cli linodes boot $BBB_ID
ssh root@bbb.emacsverse.org "cd ~/greenlight-v3; docker compose restart"
notify-send "Should be ready"

And when I resize it down to a 1 GB nanode, BigBlueButton doesn't get started and the VPS is nice and responsive when I SSH in.

echo Powering off
linode-cli linodes shutdown $BBB_ID
sleep 30
echo "Resizing BBB node to nanode, dormant"
linode-cli linodes resize $BBB_ID --type g6-nanode-1 --allow_auto_disk_resize false

So now I'm going to coordinate with Ihor Radchenko about when he might want to try this out for OrgMeetup, and I can talk to other meetup organizers to figure out times. People will probably want to test things before announcing it to their meetup groups, so we just need to schedule that. It's BigBlueButton 3.0. I'm not 100% confident in the setup. We had some technical issues with some EmacsConf speakers even though we did a tech check with them before we went live with their session. Not sure what happened there.

I'm still a little nervous about accidentally forgetting to downscale the server and running up a bill, but I've scheduled downscaling with the at command before, so that's helpful. If it turns out to be something we want to do regularly, I might even be able to use a cronjob from my other server so that it happens even if my laptop is off, and maybe set up a backup nginx server with a friendly message (and maybe a list of upcoming meetups) in case people connect before it's been scaled up. Anyway, I think that's a totally good use of part of the Google Open Source Peer Bonus I received last year.

As an aside, you can change a room's friendly_id to something actually friendly. In the Rails console (docker exec -it greenlight-v3 bundle exec rails console), you could do something like this:

Room.find_by(friendly_id: "CURRENT_ROOM_ID").update_attribute(:friendly_id, "NEW_CUSTOM_ID")

Anyway, let me know if you organize an Emacs meetup and want to give this BigBlueButton instance a try!

View org source for this post

Controlling my Android phone by voice

Posted: - Modified: | speech, android, speechtotext

[2025-01-30 Thu]: Fix timestamp format in toggle recording task.

I want to be able to use voice control to do things on my phone while I'm busy washing dishes, putting things away, knitting, or just keeping my hands warm. It'll also be handy to have a way to get things out of my head when the kiddo is koala-ing me. I've been using my Google Pixel 8's voice interface to set timers, send text messages, and do quick web searches. Building on my recent thoughts on wearable computing, I decided to spend some more time investigating the Google Assistant and Voice Access features in Android and setting up other voice shortcuts.

Tasker routines

I switched back to Google Assistant from Gemini so that I could run Tasker routines. I also found out that I needed to switch the language from English/Canada to English/US in order for my Tasker scripts to run instead of Google Assistant treating them as web searches. Once that was sorted out, I could run Tasker tasks with "Hey Google, run {task-name} in Tasker" and parameterize them with "Hey Google, run {task-name} with {parameter} in Tasker."

Voice Access

Learning how to use Voice Access to navigate, click, and type on my phone was straightforward. "Scroll down" works for webpages, while "scroll right" works for the e-books I have in Libby. Tapping items by text usually works. When it doesn't, I can use "show labels", "show numbers", or "show grid." The speech-to-text of "type …" isn't as good as Whisper, so I probably won't use it for a lot of dictation, but it's fine for quick notes. I can keep recording in the background so that I have the raw audio in case I want to review it or grab the WhisperX transcripts instead.

For some reason, saying "Hey Google, voice access" to start up voice access has been leaving the Assistant dialog on the screen, which makes it difficult to interact with the screen I'm looking at. I added a Tasker routine to start voice access, wait a second, and tap on the screen to dismiss the Assistant dialog.

Start Voice.tsk.xml - Import via Taskernet

Start Voice.tsk.xml
<TaskerData sr="" dvi="1" tv="6.3.13">
  <Task sr="task24">
    <cdate>1737565479418</cdate>
    <edate>1737566416661</edate>
    <id>24</id>
    <nme>Start Voice</nme>
    <pri>1000</pri>
    <Share sr="Share">
      <b>false</b>
      <d>Start voice access and dismiss the assistant dialog</d>
      <g>Accessibility,AutoInput</g>
      <p>true</p>
      <t></t>
    </Share>
    <Action sr="act0" ve="7">
      <code>20</code>
      <App sr="arg0">
        <appClass>com.google.android.apps.accessibility.voiceaccess.LauncherActivity</appClass>
        <appPkg>com.google.android.apps.accessibility.voiceaccess</appPkg>
        <label>Voice Access</label>
      </App>
      <Str sr="arg1" ve="3"/>
      <Int sr="arg2" val="0"/>
      <Int sr="arg3" val="0"/>
    </Action>
    <Action sr="act1" ve="7">
      <code>30</code>
      <Int sr="arg0" val="0"/>
      <Int sr="arg1" val="1"/>
      <Int sr="arg2" val="0"/>
      <Int sr="arg3" val="0"/>
      <Int sr="arg4" val="0"/>
    </Action>
    <Action sr="act2" ve="7">
      <code>107361459</code>
      <Bundle sr="arg0">
        <Vals sr="val">
          <EnableDisableAccessibilityService>&lt;null&gt;</EnableDisableAccessibilityService>
          <EnableDisableAccessibilityService-type>java.lang.String</EnableDisableAccessibilityService-type>
          <Password>&lt;null&gt;</Password>
          <Password-type>java.lang.String</Password-type>
          <com.twofortyfouram.locale.intent.extra.BLURB>Actions To Perform: click(point,564\,1045)
Not In AutoInput: true
Not In Tasker: true
Separator: ,
Check Millis: 1000</com.twofortyfouram.locale.intent.extra.BLURB>
          <com.twofortyfouram.locale.intent.extra.BLURB-type>java.lang.String</com.twofortyfouram.locale.intent.extra.BLURB-type>
          <net.dinglisch.android.tasker.JSON_ENCODED_KEYS>parameters</net.dinglisch.android.tasker.JSON_ENCODED_KEYS>
          <net.dinglisch.android.tasker.JSON_ENCODED_KEYS-type>java.lang.String</net.dinglisch.android.tasker.JSON_ENCODED_KEYS-type>
          <net.dinglisch.android.tasker.RELEVANT_VARIABLES>&lt;StringArray sr=""&gt;&lt;_array_net.dinglisch.android.tasker.RELEVANT_VARIABLES0&gt;%ailastbounds
Last Bounds
Bounds (left,top,right,bottom) of the item that the action last interacted with&lt;/_array_net.dinglisch.android.tasker.RELEVANT_VARIABLES0&gt;&lt;_array_net.dinglisch.android.tasker.RELEVANT_VARIABLES1&gt;%ailastcoordinates
Last Coordinates
Center coordinates (x,y) of the item that the action last interacted with&lt;/_array_net.dinglisch.android.tasker.RELEVANT_VARIABLES1&gt;&lt;_array_net.dinglisch.android.tasker.RELEVANT_VARIABLES2&gt;%err
Error Code
Only available if you select &amp;lt;b&amp;gt;Continue Task After Error&amp;lt;/b&amp;gt; and the action ends in error&lt;/_array_net.dinglisch.android.tasker.RELEVANT_VARIABLES2&gt;&lt;_array_net.dinglisch.android.tasker.RELEVANT_VARIABLES3&gt;%errmsg
Error Message
Only available if you select &amp;lt;b&amp;gt;Continue Task After Error&amp;lt;/b&amp;gt; and the action ends in error&lt;/_array_net.dinglisch.android.tasker.RELEVANT_VARIABLES3&gt;&lt;/StringArray&gt;</net.dinglisch.android.tasker.RELEVANT_VARIABLES>
          <net.dinglisch.android.tasker.RELEVANT_VARIABLES-type>[Ljava.lang.String;</net.dinglisch.android.tasker.RELEVANT_VARIABLES-type>
          <net.dinglisch.android.tasker.extras.VARIABLE_REPLACE_KEYS>parameters plugininstanceid plugintypeid </net.dinglisch.android.tasker.extras.VARIABLE_REPLACE_KEYS>
          <net.dinglisch.android.tasker.extras.VARIABLE_REPLACE_KEYS-type>java.lang.String</net.dinglisch.android.tasker.extras.VARIABLE_REPLACE_KEYS-type>
          <net.dinglisch.android.tasker.subbundled>true</net.dinglisch.android.tasker.subbundled>
          <net.dinglisch.android.tasker.subbundled-type>java.lang.Boolean</net.dinglisch.android.tasker.subbundled-type>
          <parameters>{"_action":"click(point,564\\,1045)","_additionalOptions":{"checkMs":"1000","separator":",","withCoordinates":false},"_whenToPerformAction":{"notInAutoInput":true,"notInTasker":true},"generatedValues":{}}</parameters>
          <parameters-type>java.lang.String</parameters-type>
          <plugininstanceid>b46b8afc-c840-40ad-9283-3946c57a1018</plugininstanceid>
          <plugininstanceid-type>java.lang.String</plugininstanceid-type>
          <plugintypeid>com.joaomgcd.autoinput.intent.IntentActionv2</plugintypeid>
          <plugintypeid-type>java.lang.String</plugintypeid-type>
        </Vals>
      </Bundle>
      <Str sr="arg1" ve="3">com.joaomgcd.autoinput</Str>
      <Str sr="arg2" ve="3">com.joaomgcd.autoinput.activity.ActivityConfigActionv2</Str>
      <Int sr="arg3" val="60"/>
      <Int sr="arg4" val="1"/>
    </Action>
  </Task>
</TaskerData>

I can use "Hey Google, read aloud" to read a webpage. I can use "Hey Google, skip ahead 2 minutes" or "Hey Google, rewind 30 seconds." Not sure how I can navigate by text, though. It would be nice to get an overview of headings and then jump to the one I want, or search for text and continue from there.

Autoplay an emacs.tv video

I wanted to be able to play random emacs.tv videos without needing to touch my phone. I added autoplay support to the web interface so that you can open https://emacs.tv?autoplay=1 and have it autoplay videos when you select the next random one by clicking on the site logo, "Lucky pick", or the dice icon. The first video doesn't autoplay because YouTube requires user interaction in order to autoplay unmuted videos, but I can work around that with a Tasker script that loads the URL, waits a few seconds, and clicks on the heading with AutoInput.

Emacs TV.tsk.xml - Import via Taskernet

Emacs TV.tsk.xml
<TaskerData sr="" dvi="1" tv="6.3.13">
  <Task sr="task18">
    <cdate>1737558964554</cdate>
    <edate>1737562488128</edate>
    <id>18</id>
    <nme>Emacs TV</nme>
    <pri>1000</pri>
    <Share sr="Share">
      <b>false</b>
      <d>Play random Emacs video</d>
      <g>Watch</g>
      <p>true</p>
      <t></t>
    </Share>
    <Action sr="act0" ve="7">
      <code>104</code>
      <Str sr="arg0" ve="3">https://emacs.tv?autoplay=1</Str>
      <App sr="arg1"/>
      <Int sr="arg2" val="0"/>
      <Str sr="arg3" ve="3"/>
    </Action>
    <Action sr="act1" ve="7">
      <code>30</code>
      <Int sr="arg0" val="0"/>
      <Int sr="arg1" val="3"/>
      <Int sr="arg2" val="0"/>
      <Int sr="arg3" val="0"/>
      <Int sr="arg4" val="0"/>
    </Action>
    <Action sr="act2" ve="7">
      <code>107361459</code>
      <Bundle sr="arg0">
        <Vals sr="val">
          <EnableDisableAccessibilityService>&lt;null&gt;</EnableDisableAccessibilityService>
          <EnableDisableAccessibilityService-type>java.lang.String</EnableDisableAccessibilityService-type>
          <Password>&lt;null&gt;</Password>
          <Password-type>java.lang.String</Password-type>
          <com.twofortyfouram.locale.intent.extra.BLURB>Actions To Perform: click(point,229\,417)
Not In AutoInput: true
Not In Tasker: true
Separator: ,
Check Millis: 1000</com.twofortyfouram.locale.intent.extra.BLURB>
          <com.twofortyfouram.locale.intent.extra.BLURB-type>java.lang.String</com.twofortyfouram.locale.intent.extra.BLURB-type>
          <net.dinglisch.android.tasker.JSON_ENCODED_KEYS>parameters</net.dinglisch.android.tasker.JSON_ENCODED_KEYS>
          <net.dinglisch.android.tasker.JSON_ENCODED_KEYS-type>java.lang.String</net.dinglisch.android.tasker.JSON_ENCODED_KEYS-type>
          <net.dinglisch.android.tasker.RELEVANT_VARIABLES>&lt;StringArray sr=""&gt;&lt;_array_net.dinglisch.android.tasker.RELEVANT_VARIABLES0&gt;%ailastbounds
Last Bounds
Bounds (left,top,right,bottom) of the item that the action last interacted with&lt;/_array_net.dinglisch.android.tasker.RELEVANT_VARIABLES0&gt;&lt;_array_net.dinglisch.android.tasker.RELEVANT_VARIABLES1&gt;%ailastcoordinates
Last Coordinates
Center coordinates (x,y) of the item that the action last interacted with&lt;/_array_net.dinglisch.android.tasker.RELEVANT_VARIABLES1&gt;&lt;_array_net.dinglisch.android.tasker.RELEVANT_VARIABLES2&gt;%err
Error Code
Only available if you select &amp;lt;b&amp;gt;Continue Task After Error&amp;lt;/b&amp;gt; and the action ends in error&lt;/_array_net.dinglisch.android.tasker.RELEVANT_VARIABLES2&gt;&lt;_array_net.dinglisch.android.tasker.RELEVANT_VARIABLES3&gt;%errmsg
Error Message
Only available if you select &amp;lt;b&amp;gt;Continue Task After Error&amp;lt;/b&amp;gt; and the action ends in error&lt;/_array_net.dinglisch.android.tasker.RELEVANT_VARIABLES3&gt;&lt;/StringArray&gt;</net.dinglisch.android.tasker.RELEVANT_VARIABLES>
          <net.dinglisch.android.tasker.RELEVANT_VARIABLES-type>[Ljava.lang.String;</net.dinglisch.android.tasker.RELEVANT_VARIABLES-type>
          <net.dinglisch.android.tasker.extras.VARIABLE_REPLACE_KEYS>parameters plugininstanceid plugintypeid </net.dinglisch.android.tasker.extras.VARIABLE_REPLACE_KEYS>
          <net.dinglisch.android.tasker.extras.VARIABLE_REPLACE_KEYS-type>java.lang.String</net.dinglisch.android.tasker.extras.VARIABLE_REPLACE_KEYS-type>
          <net.dinglisch.android.tasker.subbundled>true</net.dinglisch.android.tasker.subbundled>
          <net.dinglisch.android.tasker.subbundled-type>java.lang.Boolean</net.dinglisch.android.tasker.subbundled-type>
          <parameters>{"_action":"click(point,229\\,417)","_additionalOptions":{"checkMs":"1000","separator":",","withCoordinates":false},"_whenToPerformAction":{"notInAutoInput":true,"notInTasker":true},"generatedValues":{}}</parameters>
          <parameters-type>java.lang.String</parameters-type>
          <plugininstanceid>45ce7a83-47e5-48fb-8c3e-20655e668353</plugininstanceid>
          <plugininstanceid-type>java.lang.String</plugininstanceid-type>
          <plugintypeid>com.joaomgcd.autoinput.intent.IntentActionv2</plugintypeid>
          <plugintypeid-type>java.lang.String</plugintypeid-type>
        </Vals>
      </Bundle>
      <Str sr="arg1" ve="3">com.joaomgcd.autoinput</Str>
      <Str sr="arg2" ve="3">com.joaomgcd.autoinput.activity.ActivityConfigActionv2</Str>
      <Int sr="arg3" val="60"/>
      <Int sr="arg4" val="1"/>
    </Action>
  </Task>
</TaskerData>

Then I set up a Google Assistant routine with the triggers "teach me" or "Emacs TV" and the action "run Emacs TV in Tasker. Now I can say "Hey Google, teach me" and it'll play a random Emacs video for me. I can repeat "Hey Google, teach me" to get a different video, and I can pause with "Hey Google, pause video".

This was actually my second approach. The first time I tried to implement this, I thought about using Voice Access to interact with the buttons. Strangely, I couldn't get Voice Access to click on the header links or the buttons even when I had aria-label, role="button", and tabindex attributes set on them. As a hacky workaround, I made the site logo pick a new random video when clicked, so I can at least use it as a large touch target when I use "display grid" in Voice Access. ("Tap 5" will load the next video.)

There doesn't seem to be a way to add custom voice access commands to a webpage in a way that hooks into Android Voice Access and iOS Voice Control, but maybe I'm just missing something obvious when it comes to ARIA attributes.

Open my Org agenda and scroll through it

There were some words that I couldn't get Google Assistant or Voice Access to understand, like "open Orgzly Revived". Fortunately, "Open Revived" worked just fine.

I wanted to be able to see my Org Agenda. After some fiddling around (see the resources in this section), I figured out this AutoShare intent that runs an agenda search:

orgzly-revived-search.intent

orgzly-revived-search.intent
{
  "target": "Activity",
  "appname": "Orgzly Revived",
  "action": "android.intent.action.MAIN",
  "package": "com.orgzlyrevived",
  "class": "com.orgzly.android.ui.main.MainActivity",
  "extras": [
    {
      "type": "String",
      "key": "com.orgzly.intent.extra.QUERY_STRING",
      "name": "Query"
    }
  ],
  "name": "Search",
  "id": "Orgzly-search"
}

Then I defined a Tasker task called "Search Orgzly Revived":

Download Search Orgzly Revived.tsk.xml

Search Orgzly Revived.tsk.xml
<TaskerData sr="" dvi="1" tv="6.3.13">
  <Task sr="task16">
    <cdate>1676823952566</cdate>
    <edate>1737567565538</edate>
    <id>16</id>
    <nme>Search Orgzly Revived</nme>
    <pri>100</pri>
    <Share sr="Share">
      <b>false</b>
      <d>Search Orgzly Revived</d>
      <g>Work,Well-Being</g>
      <p>false</p>
      <t></t>
    </Share>
    <Action sr="act0" ve="7">
      <code>18</code>
      <App sr="arg0">
        <appClass>com.orgzly.android.ui.LauncherActivity</appClass>
        <appPkg>com.orgzlyrevived</appPkg>
        <label>Orgzly Revived</label>
      </App>
      <Int sr="arg1" val="0"/>
    </Action>
    <Action sr="act1" ve="7">
      <code>547</code>
      <Str sr="arg0" ve="3">%extra</Str>
      <Str sr="arg1" ve="3">com.orgzly.intent.extra.QUERY_STRING:%par1</Str>
      <Int sr="arg2" val="0"/>
      <Int sr="arg3" val="0"/>
      <Int sr="arg4" val="0"/>
      <Int sr="arg5" val="3"/>
      <Int sr="arg6" val="1"/>
    </Action>
    <Action sr="act2" ve="7">
      <code>877</code>
      <Str sr="arg0" ve="3">android.intent.action.MAIN</Str>
      <Int sr="arg1" val="0"/>
      <Str sr="arg2" ve="3"/>
      <Str sr="arg3" ve="3"/>
      <Str sr="arg4" ve="3">%extra</Str>
      <Str sr="arg5" ve="3"/>
      <Str sr="arg6" ve="3"/>
      <Str sr="arg7" ve="3">com.orgzlyrevived</Str>
      <Str sr="arg8" ve="3">com.orgzly.android.ui.main.MainActivity</Str>
      <Int sr="arg9" val="1"/>
    </Action>
    <Img sr="icn" ve="2">
      <nme>mw_action_today</nme>
    </Img>
  </Task>
</TaskerData>

I made a Google Assistant routine that uses "show my agenda" as the trigger and "run search orgzly revived in Tasker" as the action. After a quick "Hey Google, show my agenda; Hey Google, voice access", I can use "scroll down" to page through the list. "Back" gets me to the list of notebooks, and "inbox" opens my inbox.

Resources:

Add and open notes in Orgzly Revived

When I'm looking at an Orgzly Revived notebook with Voice Access turned on, "plus" starts a new note. Anything that isn't a label gets typed, so I can just start saying the title of my note (or use "type …"). If I want to add the content, I have to use "hide keyboard", "tap content", and then "type …"). "Tap scheduled time; Tomorrow" works if the scheduled time widget is visible, so I just need to use "scroll down" if the title is long. "Tap done; one" saves it.

Adding a note could be simpler - maybe a Tasker task that prompts me for text and adds it. I could use Tasker to prepend to my Inbox.org and then reload it in Orgzly. It would be more elegant to figure out the intent for adding a note, though. Maybe in the Orgzly Android intent receiver documentation?

When I'm looking at the Orgzly notebook and I say part of the text in a note without a link, it opens the note. If the note has a link, it seems to open the link directly. Tapping by numbers also goes to the link, but tapping by grid opens the note.

I'd love to speech-enable this someday so that I can hear Orgzly Revived step through my agenda and use my voice to mark things as cancelled/done, schedule them for today/tomorrow/next week, or add extra notes to the body.

Add items to OurGroceries

W+ and I use the OurGroceries app. As it turns out, "Hey Google, ask OurGroceries to add milk" still works. Also, Voice Access works fine with OurGroceries. I can say "Plus", dictate an item, and tap "Add." I configured the cross-off action to be swipes instead of taps to minimize accidental crossing-off at the store, so I can say "swipe right on apples" to mark that as done.

Track time

I added a Tasker task to update my personal time-tracking system, and I added some Google Assistant routines for common categories like writing or routines. I can also use "run track with {category} in Tasker" to track a less-common category. The kiddo likes to get picked up and hugged a lot, so I added a "Hey Google, koala time" routine to clock into childcare in a more fun way. I have to enunciate that one clearly or it'll get turned into "Call into …", which doesn't work.

Toggle recording

Since I was tinkering around with Tasker a lot, I decided to try moving my voice recording into it. I want to save timestamped recordings into my ~/sync/recordings directory so that they're automatically synchronized with Syncthing, and then they can feed into my WhisperX workflow. This feels a little more responsive and reliable than Fossify Voice Recorder, actually, since that one tended to become unresponsive from time to time.

Download Toggle Recording.tsk.xml - Import via Taskernet

Toggle Recording.tsk.xml
<TaskerData sr="" dvi="1" tv="6.3.13">
  <Task sr="task12">
    <cdate>1737504717303</cdate>
    <edate>1738272248919</edate>
    <id>12</id>
    <nme>Toggle Recording</nme>
    <pri>100</pri>
    <Share sr="Share">
      <b>false</b>
      <d>Toggle recording on and off; save timestamped file to sync/recordings</d>
      <g>Sound</g>
      <p>true</p>
      <t></t>
    </Share>
    <Action sr="act0" ve="7">
      <code>37</code>
      <ConditionList sr="if">
        <Condition sr="c0" ve="3">
          <lhs>%RECORDING</lhs>
          <op>12</op>
          <rhs></rhs>
        </Condition>
      </ConditionList>
    </Action>
    <Action sr="act1" ve="7">
      <code>549</code>
      <Str sr="arg0" ve="3">%RECORDING</Str>
      <Int sr="arg1" val="0"/>
      <Int sr="arg2" val="0"/>
      <Int sr="arg3" val="0"/>
    </Action>
    <Action sr="act10" ve="7">
      <code>166160670</code>
      <Bundle sr="arg0">
        <Vals sr="val">
          <ActionIconString1>&lt;null&gt;</ActionIconString1>
          <ActionIconString1-type>java.lang.String</ActionIconString1-type>
          <ActionIconString2>&lt;null&gt;</ActionIconString2>
          <ActionIconString2-type>java.lang.String</ActionIconString2-type>
          <ActionIconString3>&lt;null&gt;</ActionIconString3>
          <ActionIconString3-type>java.lang.String</ActionIconString3-type>
          <ActionIconString4>&lt;null&gt;</ActionIconString4>
          <ActionIconString4-type>java.lang.String</ActionIconString4-type>
          <ActionIconString5>&lt;null&gt;</ActionIconString5>
          <ActionIconString5-type>java.lang.String</ActionIconString5-type>
          <AppendTexts>false</AppendTexts>
          <AppendTexts-type>java.lang.Boolean</AppendTexts-type>
          <BackgroundColor>&lt;null&gt;</BackgroundColor>
          <BackgroundColor-type>java.lang.String</BackgroundColor-type>
          <BadgeType>&lt;null&gt;</BadgeType>
          <BadgeType-type>java.lang.String</BadgeType-type>
          <Button1UnlockScreen>false</Button1UnlockScreen>
          <Button1UnlockScreen-type>java.lang.Boolean</Button1UnlockScreen-type>
          <Button2UnlockScreen>false</Button2UnlockScreen>
          <Button2UnlockScreen-type>java.lang.Boolean</Button2UnlockScreen-type>
          <Button3UnlockScreen>false</Button3UnlockScreen>
          <Button3UnlockScreen-type>java.lang.Boolean</Button3UnlockScreen-type>
          <Button4UnlockScreen>false</Button4UnlockScreen>
          <Button4UnlockScreen-type>java.lang.Boolean</Button4UnlockScreen-type>
          <Button5UnlockScreen>false</Button5UnlockScreen>
          <Button5UnlockScreen-type>java.lang.Boolean</Button5UnlockScreen-type>
          <ChronometerCountDown>false</ChronometerCountDown>
          <ChronometerCountDown-type>java.lang.Boolean</ChronometerCountDown-type>
          <Colorize>false</Colorize>
          <Colorize-type>java.lang.Boolean</Colorize-type>
          <DismissOnTouchVariable>&lt;null&gt;</DismissOnTouchVariable>
          <DismissOnTouchVariable-type>java.lang.String</DismissOnTouchVariable-type>
          <ExtraInfo>&lt;null&gt;</ExtraInfo>
          <ExtraInfo-type>java.lang.String</ExtraInfo-type>
          <GroupAlertBehaviour>&lt;null&gt;</GroupAlertBehaviour>
          <GroupAlertBehaviour-type>java.lang.String</GroupAlertBehaviour-type>
          <GroupKey>&lt;null&gt;</GroupKey>
          <GroupKey-type>java.lang.String</GroupKey-type>
          <IconExpanded>&lt;null&gt;</IconExpanded>
          <IconExpanded-type>java.lang.String</IconExpanded-type>
          <IsGroupSummary>false</IsGroupSummary>
          <IsGroupSummary-type>java.lang.Boolean</IsGroupSummary-type>
          <IsGroupVariable>&lt;null&gt;</IsGroupVariable>
          <IsGroupVariable-type>java.lang.String</IsGroupVariable-type>
          <MediaAlbum>&lt;null&gt;</MediaAlbum>
          <MediaAlbum-type>java.lang.String</MediaAlbum-type>
          <MediaArtist>&lt;null&gt;</MediaArtist>
          <MediaArtist-type>java.lang.String</MediaArtist-type>
          <MediaDuration>&lt;null&gt;</MediaDuration>
          <MediaDuration-type>java.lang.String</MediaDuration-type>
          <MediaIcon>&lt;null&gt;</MediaIcon>
          <MediaIcon-type>java.lang.String</MediaIcon-type>
          <MediaLayout>false</MediaLayout>
          <MediaLayout-type>java.lang.Boolean</MediaLayout-type>
          <MediaNextCommand>&lt;null&gt;</MediaNextCommand>
          <MediaNextCommand-type>java.lang.String</MediaNextCommand-type>
          <MediaPauseCommand>&lt;null&gt;</MediaPauseCommand>
          <MediaPauseCommand-type>java.lang.String</MediaPauseCommand-type>
          <MediaPlayCommand>&lt;null&gt;</MediaPlayCommand>
          <MediaPlayCommand-type>java.lang.String</MediaPlayCommand-type>
          <MediaPlaybackState>&lt;null&gt;</MediaPlaybackState>
          <MediaPlaybackState-type>java.lang.String</MediaPlaybackState-type>
          <MediaPosition>&lt;null&gt;</MediaPosition>
          <MediaPosition-type>java.lang.String</MediaPosition-type>
          <MediaPreviousCommand>&lt;null&gt;</MediaPreviousCommand>
          <MediaPreviousCommand-type>java.lang.String</MediaPreviousCommand-type>
          <MediaTrack>&lt;null&gt;</MediaTrack>
          <MediaTrack-type>java.lang.String</MediaTrack-type>
          <MessagingImages>&lt;null&gt;</MessagingImages>
          <MessagingImages-type>java.lang.String</MessagingImages-type>
          <MessagingOwnIcon>&lt;null&gt;</MessagingOwnIcon>
          <MessagingOwnIcon-type>java.lang.String</MessagingOwnIcon-type>
          <MessagingOwnName>&lt;null&gt;</MessagingOwnName>
          <MessagingOwnName-type>java.lang.String</MessagingOwnName-type>
          <MessagingPersonBot>&lt;null&gt;</MessagingPersonBot>
          <MessagingPersonBot-type>java.lang.String</MessagingPersonBot-type>
          <MessagingPersonIcons>&lt;null&gt;</MessagingPersonIcons>
          <MessagingPersonIcons-type>java.lang.String</MessagingPersonIcons-type>
          <MessagingPersonImportant>&lt;null&gt;</MessagingPersonImportant>
          <MessagingPersonImportant-type>java.lang.String</MessagingPersonImportant-type>
          <MessagingPersonNames>&lt;null&gt;</MessagingPersonNames>
          <MessagingPersonNames-type>java.lang.String</MessagingPersonNames-type>
          <MessagingPersonUri>&lt;null&gt;</MessagingPersonUri>
          <MessagingPersonUri-type>java.lang.String</MessagingPersonUri-type>
          <MessagingSeparator>&lt;null&gt;</MessagingSeparator>
          <MessagingSeparator-type>java.lang.String</MessagingSeparator-type>
          <MessagingTexts>&lt;null&gt;</MessagingTexts>
          <MessagingTexts-type>java.lang.String</MessagingTexts-type>
          <NotificationChannelBypassDnd>false</NotificationChannelBypassDnd>
          <NotificationChannelBypassDnd-type>java.lang.Boolean</NotificationChannelBypassDnd-type>
          <NotificationChannelDescription>&lt;null&gt;</NotificationChannelDescription>
          <NotificationChannelDescription-type>java.lang.String</NotificationChannelDescription-type>
          <NotificationChannelId>&lt;null&gt;</NotificationChannelId>
          <NotificationChannelId-type>java.lang.String</NotificationChannelId-type>
          <NotificationChannelImportance>&lt;null&gt;</NotificationChannelImportance>
          <NotificationChannelImportance-type>java.lang.String</NotificationChannelImportance-type>
          <NotificationChannelName>&lt;null&gt;</NotificationChannelName>
          <NotificationChannelName-type>java.lang.String</NotificationChannelName-type>
          <NotificationChannelShowBadge>false</NotificationChannelShowBadge>
          <NotificationChannelShowBadge-type>java.lang.Boolean</NotificationChannelShowBadge-type>
          <PersistentVariable>&lt;null&gt;</PersistentVariable>
          <PersistentVariable-type>java.lang.String</PersistentVariable-type>
          <PhoneOnly>false</PhoneOnly>
          <PhoneOnly-type>java.lang.Boolean</PhoneOnly-type>
          <PriorityVariable>&lt;null&gt;</PriorityVariable>
          <PriorityVariable-type>java.lang.String</PriorityVariable-type>
          <PublicVersion>&lt;null&gt;</PublicVersion>
          <PublicVersion-type>java.lang.String</PublicVersion-type>
          <ReplyAction>&lt;null&gt;</ReplyAction>
          <ReplyAction-type>java.lang.String</ReplyAction-type>
          <ReplyChoices>&lt;null&gt;</ReplyChoices>
          <ReplyChoices-type>java.lang.String</ReplyChoices-type>
          <ReplyLabel>&lt;null&gt;</ReplyLabel>
          <ReplyLabel-type>java.lang.String</ReplyLabel-type>
          <ShareButtonsVariable>&lt;null&gt;</ShareButtonsVariable>
          <ShareButtonsVariable-type>java.lang.String</ShareButtonsVariable-type>
          <SkipPictureCache>false</SkipPictureCache>
          <SkipPictureCache-type>java.lang.Boolean</SkipPictureCache-type>
          <SoundPath>&lt;null&gt;</SoundPath>
          <SoundPath-type>java.lang.String</SoundPath-type>
          <StatusBarIconString>&lt;null&gt;</StatusBarIconString>
          <StatusBarIconString-type>java.lang.String</StatusBarIconString-type>
          <StatusBarTextSize>16</StatusBarTextSize>
          <StatusBarTextSize-type>java.lang.String</StatusBarTextSize-type>
          <TextExpanded>&lt;null&gt;</TextExpanded>
          <TextExpanded-type>java.lang.String</TextExpanded-type>
          <Time>&lt;null&gt;</Time>
          <Time-type>java.lang.String</Time-type>
          <TimeFormat>&lt;null&gt;</TimeFormat>
          <TimeFormat-type>java.lang.String</TimeFormat-type>
          <Timeout>&lt;null&gt;</Timeout>
          <Timeout-type>java.lang.String</Timeout-type>
          <TitleExpanded>&lt;null&gt;</TitleExpanded>
          <TitleExpanded-type>java.lang.String</TitleExpanded-type>
          <UpdateNotification>false</UpdateNotification>
          <UpdateNotification-type>java.lang.Boolean</UpdateNotification-type>
          <UseChronometer>false</UseChronometer>
          <UseChronometer-type>java.lang.Boolean</UseChronometer-type>
          <UseHTML>false</UseHTML>
          <UseHTML-type>java.lang.Boolean</UseHTML-type>
          <Visibility>&lt;null&gt;</Visibility>
          <Visibility-type>java.lang.String</Visibility-type>
          <com.twofortyfouram.locale.intent.extra.BLURB>Title: my recording
Action on Touch: stop recording
Status Bar Text Size: 16
Id: my-recording
Dismiss on Touch: true
Priority: -1
Separator: ,</com.twofortyfouram.locale.intent.extra.BLURB>
          <com.twofortyfouram.locale.intent.extra.BLURB-type>java.lang.String</com.twofortyfouram.locale.intent.extra.BLURB-type>
          <config_action_1_icon>&lt;null&gt;</config_action_1_icon>
          <config_action_1_icon-type>java.lang.String</config_action_1_icon-type>
          <config_action_2_icon>&lt;null&gt;</config_action_2_icon>
          <config_action_2_icon-type>java.lang.String</config_action_2_icon-type>
          <config_action_3_icon>&lt;null&gt;</config_action_3_icon>
          <config_action_3_icon-type>java.lang.String</config_action_3_icon-type>
          <config_action_4_icon>&lt;null&gt;</config_action_4_icon>
          <config_action_4_icon-type>java.lang.String</config_action_4_icon-type>
          <config_action_5_icon>&lt;null&gt;</config_action_5_icon>
          <config_action_5_icon-type>java.lang.String</config_action_5_icon-type>
          <config_notification_action>stop recording</config_notification_action>
          <config_notification_action-type>java.lang.String</config_notification_action-type>
          <config_notification_action_button1>&lt;null&gt;</config_notification_action_button1>
          <config_notification_action_button1-type>java.lang.String</config_notification_action_button1-type>
          <config_notification_action_button2>&lt;null&gt;</config_notification_action_button2>
          <config_notification_action_button2-type>java.lang.String</config_notification_action_button2-type>
          <config_notification_action_button3>&lt;null&gt;</config_notification_action_button3>
          <config_notification_action_button3-type>java.lang.String</config_notification_action_button3-type>
          <config_notification_action_button4>&lt;null&gt;</config_notification_action_button4>
          <config_notification_action_button4-type>java.lang.String</config_notification_action_button4-type>
          <config_notification_action_button5>&lt;null&gt;</config_notification_action_button5>
          <config_notification_action_button5-type>java.lang.String</config_notification_action_button5-type>
          <config_notification_action_label1>&lt;null&gt;</config_notification_action_label1>
          <config_notification_action_label1-type>java.lang.String</config_notification_action_label1-type>
          <config_notification_action_label2>&lt;null&gt;</config_notification_action_label2>
          <config_notification_action_label2-type>java.lang.String</config_notification_action_label2-type>
          <config_notification_action_label3>&lt;null&gt;</config_notification_action_label3>
          <config_notification_action_label3-type>java.lang.String</config_notification_action_label3-type>
          <config_notification_action_on_dismiss>&lt;null&gt;</config_notification_action_on_dismiss>
          <config_notification_action_on_dismiss-type>java.lang.String</config_notification_action_on_dismiss-type>
          <config_notification_action_share>false</config_notification_action_share>
          <config_notification_action_share-type>java.lang.Boolean</config_notification_action_share-type>
          <config_notification_command>&lt;null&gt;</config_notification_command>
          <config_notification_command-type>java.lang.String</config_notification_command-type>
          <config_notification_content_info>&lt;null&gt;</config_notification_content_info>
          <config_notification_content_info-type>java.lang.String</config_notification_content_info-type>
          <config_notification_dismiss_on_touch>true</config_notification_dismiss_on_touch>
          <config_notification_dismiss_on_touch-type>java.lang.Boolean</config_notification_dismiss_on_touch-type>
          <config_notification_icon>&lt;null&gt;</config_notification_icon>
          <config_notification_icon-type>java.lang.String</config_notification_icon-type>
          <config_notification_indeterminate_progress>false</config_notification_indeterminate_progress>
          <config_notification_indeterminate_progress-type>java.lang.Boolean</config_notification_indeterminate_progress-type>
          <config_notification_led_color>&lt;null&gt;</config_notification_led_color>
          <config_notification_led_color-type>java.lang.String</config_notification_led_color-type>
          <config_notification_led_off>&lt;null&gt;</config_notification_led_off>
          <config_notification_led_off-type>java.lang.String</config_notification_led_off-type>
          <config_notification_led_on>&lt;null&gt;</config_notification_led_on>
          <config_notification_led_on-type>java.lang.String</config_notification_led_on-type>
          <config_notification_max_progress>&lt;null&gt;</config_notification_max_progress>
          <config_notification_max_progress-type>java.lang.String</config_notification_max_progress-type>
          <config_notification_number>&lt;null&gt;</config_notification_number>
          <config_notification_number-type>java.lang.String</config_notification_number-type>
          <config_notification_persistent>true</config_notification_persistent>
          <config_notification_persistent-type>java.lang.Boolean</config_notification_persistent-type>
          <config_notification_picture>&lt;null&gt;</config_notification_picture>
          <config_notification_picture-type>java.lang.String</config_notification_picture-type>
          <config_notification_priority>-1</config_notification_priority>
          <config_notification_priority-type>java.lang.String</config_notification_priority-type>
          <config_notification_progress>&lt;null&gt;</config_notification_progress>
          <config_notification_progress-type>java.lang.String</config_notification_progress-type>
          <config_notification_subtext>&lt;null&gt;</config_notification_subtext>
          <config_notification_subtext-type>java.lang.String</config_notification_subtext-type>
          <config_notification_text>&lt;null&gt;</config_notification_text>
          <config_notification_text-type>java.lang.String</config_notification_text-type>
          <config_notification_ticker>&lt;null&gt;</config_notification_ticker>
          <config_notification_ticker-type>java.lang.String</config_notification_ticker-type>
          <config_notification_title>my recording</config_notification_title>
          <config_notification_title-type>java.lang.String</config_notification_title-type>
          <config_notification_url>&lt;null&gt;</config_notification_url>
          <config_notification_url-type>java.lang.String</config_notification_url-type>
          <config_notification_vibration>&lt;null&gt;</config_notification_vibration>
          <config_notification_vibration-type>java.lang.String</config_notification_vibration-type>
          <config_status_bar_icon>&lt;null&gt;</config_status_bar_icon>
          <config_status_bar_icon-type>java.lang.String</config_status_bar_icon-type>
          <net.dinglisch.android.tasker.RELEVANT_VARIABLES>&lt;StringArray sr=""&gt;&lt;_array_net.dinglisch.android.tasker.RELEVANT_VARIABLES0&gt;%err
Error Code
Only available if you select &amp;lt;b&amp;gt;Continue Task After Error&amp;lt;/b&amp;gt; and the action ends in error&lt;/_array_net.dinglisch.android.tasker.RELEVANT_VARIABLES0&gt;&lt;_array_net.dinglisch.android.tasker.RELEVANT_VARIABLES1&gt;%errmsg
Error Message
Only available if you select &amp;lt;b&amp;gt;Continue Task After Error&amp;lt;/b&amp;gt; and the action ends in error&lt;/_array_net.dinglisch.android.tasker.RELEVANT_VARIABLES1&gt;&lt;/StringArray&gt;</net.dinglisch.android.tasker.RELEVANT_VARIABLES>
          <net.dinglisch.android.tasker.RELEVANT_VARIABLES-type>[Ljava.lang.String;</net.dinglisch.android.tasker.RELEVANT_VARIABLES-type>
          <net.dinglisch.android.tasker.extras.VARIABLE_REPLACE_KEYS>StatusBarTextSize config_notification_title config_notification_action notificaitionid config_notification_priority plugininstanceid plugintypeid </net.dinglisch.android.tasker.extras.VARIABLE_REPLACE_KEYS>
          <net.dinglisch.android.tasker.extras.VARIABLE_REPLACE_KEYS-type>java.lang.String</net.dinglisch.android.tasker.extras.VARIABLE_REPLACE_KEYS-type>
          <net.dinglisch.android.tasker.subbundled>true</net.dinglisch.android.tasker.subbundled>
          <net.dinglisch.android.tasker.subbundled-type>java.lang.Boolean</net.dinglisch.android.tasker.subbundled-type>
          <notificaitionid>my-recording</notificaitionid>
          <notificaitionid-type>java.lang.String</notificaitionid-type>
          <notificaitionsound>&lt;null&gt;</notificaitionsound>
          <notificaitionsound-type>java.lang.String</notificaitionsound-type>
          <plugininstanceid>9fca7d3a-cca6-4bfb-8ec4-a991054350c5</plugininstanceid>
          <plugininstanceid-type>java.lang.String</plugininstanceid-type>
          <plugintypeid>com.joaomgcd.autonotification.intent.IntentNotification</plugintypeid>
          <plugintypeid-type>java.lang.String</plugintypeid-type>
        </Vals>
      </Bundle>
      <Str sr="arg1" ve="3">com.joaomgcd.autonotification</Str>
      <Str sr="arg2" ve="3">com.joaomgcd.autonotification.activity.ActivityConfigNotify</Str>
      <Int sr="arg3" val="0"/>
      <Int sr="arg4" val="1"/>
    </Action>
    <Action sr="act11" ve="7">
      <code>559</code>
      <Str sr="arg0" ve="3">Go</Str>
      <Str sr="arg1" ve="3">default:default</Str>
      <Int sr="arg2" val="3"/>
      <Int sr="arg3" val="5"/>
      <Int sr="arg4" val="5"/>
      <Int sr="arg5" val="1"/>
      <Int sr="arg6" val="0"/>
      <Int sr="arg7" val="0"/>
    </Action>
    <Action sr="act12" ve="7">
      <code>455</code>
      <Str sr="arg0" ve="3">sync/recordings/%filename</Str>
      <Int sr="arg1" val="0"/>
      <Int sr="arg2" val="0"/>
      <Int sr="arg3" val="0"/>
      <Int sr="arg4" val="0"/>
    </Action>
    <Action sr="act13" ve="7">
      <code>38</code>
    </Action>
    <Action sr="act2" ve="7">
      <code>657</code>
    </Action>
    <Action sr="act3" ve="7">
      <code>559</code>
      <Str sr="arg0" ve="3">Done</Str>
      <Str sr="arg1" ve="3">default:default</Str>
      <Int sr="arg2" val="3"/>
      <Int sr="arg3" val="5"/>
      <Int sr="arg4" val="5"/>
      <Int sr="arg5" val="1"/>
      <Int sr="arg6" val="0"/>
      <Int sr="arg7" val="0"/>
    </Action>
    <Action sr="act4" ve="7">
      <code>2046367074</code>
      <Bundle sr="arg0">
        <Vals sr="val">
          <App>&lt;null&gt;</App>
          <App-type>java.lang.String</App-type>
          <CancelAll>false</CancelAll>
          <CancelAll-type>java.lang.Boolean</CancelAll-type>
          <CancelPersistent>false</CancelPersistent>
          <CancelPersistent-type>java.lang.Boolean</CancelPersistent-type>
          <CaseinsensitiveApp>false</CaseinsensitiveApp>
          <CaseinsensitiveApp-type>java.lang.Boolean</CaseinsensitiveApp-type>
          <CaseinsensitivePackage>false</CaseinsensitivePackage>
          <CaseinsensitivePackage-type>java.lang.Boolean</CaseinsensitivePackage-type>
          <CaseinsensitiveText>false</CaseinsensitiveText>
          <CaseinsensitiveText-type>java.lang.Boolean</CaseinsensitiveText-type>
          <CaseinsensitiveTitle>false</CaseinsensitiveTitle>
          <CaseinsensitiveTitle-type>java.lang.Boolean</CaseinsensitiveTitle-type>
          <ExactApp>false</ExactApp>
          <ExactApp-type>java.lang.Boolean</ExactApp-type>
          <ExactPackage>false</ExactPackage>
          <ExactPackage-type>java.lang.Boolean</ExactPackage-type>
          <ExactText>false</ExactText>
          <ExactText-type>java.lang.Boolean</ExactText-type>
          <ExactTitle>false</ExactTitle>
          <ExactTitle-type>java.lang.Boolean</ExactTitle-type>
          <InterceptApps>&lt;StringArray sr=""/&gt;</InterceptApps>
          <InterceptApps-type>[Ljava.lang.String;</InterceptApps-type>
          <InvertApp>false</InvertApp>
          <InvertApp-type>java.lang.Boolean</InvertApp-type>
          <InvertPackage>false</InvertPackage>
          <InvertPackage-type>java.lang.Boolean</InvertPackage-type>
          <InvertText>false</InvertText>
          <InvertText-type>java.lang.Boolean</InvertText-type>
          <InvertTitle>false</InvertTitle>
          <InvertTitle-type>java.lang.Boolean</InvertTitle-type>
          <OtherId>&lt;null&gt;</OtherId>
          <OtherId-type>java.lang.String</OtherId-type>
          <OtherPackage>&lt;null&gt;</OtherPackage>
          <OtherPackage-type>java.lang.String</OtherPackage-type>
          <OtherTag>&lt;null&gt;</OtherTag>
          <OtherTag-type>java.lang.String</OtherTag-type>
          <PackageName>&lt;null&gt;</PackageName>
          <PackageName-type>java.lang.String</PackageName-type>
          <RegexApp>false</RegexApp>
          <RegexApp-type>java.lang.Boolean</RegexApp-type>
          <RegexPackage>false</RegexPackage>
          <RegexPackage-type>java.lang.Boolean</RegexPackage-type>
          <RegexText>false</RegexText>
          <RegexText-type>java.lang.Boolean</RegexText-type>
          <RegexTitle>false</RegexTitle>
          <RegexTitle-type>java.lang.Boolean</RegexTitle-type>
          <Text>&lt;null&gt;</Text>
          <Text-type>java.lang.String</Text-type>
          <Title>&lt;null&gt;</Title>
          <Title-type>java.lang.String</Title-type>
          <com.twofortyfouram.locale.intent.extra.BLURB>Id: my-recording</com.twofortyfouram.locale.intent.extra.BLURB>
          <com.twofortyfouram.locale.intent.extra.BLURB-type>java.lang.String</com.twofortyfouram.locale.intent.extra.BLURB-type>
          <net.dinglisch.android.tasker.RELEVANT_VARIABLES>&lt;StringArray sr=""&gt;&lt;_array_net.dinglisch.android.tasker.RELEVANT_VARIABLES0&gt;%err
Error Code
Only available if you select &amp;lt;b&amp;gt;Continue Task After Error&amp;lt;/b&amp;gt; and the action ends in error&lt;/_array_net.dinglisch.android.tasker.RELEVANT_VARIABLES0&gt;&lt;_array_net.dinglisch.android.tasker.RELEVANT_VARIABLES1&gt;%errmsg
Error Message
Only available if you select &amp;lt;b&amp;gt;Continue Task After Error&amp;lt;/b&amp;gt; and the action ends in error&lt;/_array_net.dinglisch.android.tasker.RELEVANT_VARIABLES1&gt;&lt;/StringArray&gt;</net.dinglisch.android.tasker.RELEVANT_VARIABLES>
          <net.dinglisch.android.tasker.RELEVANT_VARIABLES-type>[Ljava.lang.String;</net.dinglisch.android.tasker.RELEVANT_VARIABLES-type>
          <net.dinglisch.android.tasker.extras.VARIABLE_REPLACE_KEYS>notificaitionid plugininstanceid plugintypeid </net.dinglisch.android.tasker.extras.VARIABLE_REPLACE_KEYS>
          <net.dinglisch.android.tasker.extras.VARIABLE_REPLACE_KEYS-type>java.lang.String</net.dinglisch.android.tasker.extras.VARIABLE_REPLACE_KEYS-type>
          <net.dinglisch.android.tasker.subbundled>true</net.dinglisch.android.tasker.subbundled>
          <net.dinglisch.android.tasker.subbundled-type>java.lang.Boolean</net.dinglisch.android.tasker.subbundled-type>
          <notificaitionid>my-recording</notificaitionid>
          <notificaitionid-type>java.lang.String</notificaitionid-type>
          <plugininstanceid>da51b00c-7f2a-483d-864c-7fee8ac384aa</plugininstanceid>
          <plugininstanceid-type>java.lang.String</plugininstanceid-type>
          <plugintypeid>com.joaomgcd.autonotification.intent.IntentCancelNotification</plugintypeid>
          <plugintypeid-type>java.lang.String</plugintypeid-type>
        </Vals>
      </Bundle>
      <Str sr="arg1" ve="3">com.joaomgcd.autonotification</Str>
      <Str sr="arg2" ve="3">com.joaomgcd.autonotification.activity.ActivityConfigCancelNotification</Str>
      <Int sr="arg3" val="0"/>
      <Int sr="arg4" val="1"/>
    </Action>
    <Action sr="act5" ve="7">
      <code>43</code>
    </Action>
    <Action sr="act6" ve="7">
      <code>394</code>
      <Bundle sr="arg0">
        <Vals sr="val">
          <net.dinglisch.android.tasker.RELEVANT_VARIABLES>&lt;StringArray sr=""&gt;&lt;_array_net.dinglisch.android.tasker.RELEVANT_VARIABLES0&gt;%current_time
00. Current time
&lt;/_array_net.dinglisch.android.tasker.RELEVANT_VARIABLES0&gt;&lt;_array_net.dinglisch.android.tasker.RELEVANT_VARIABLES1&gt;%dt_millis
1. MilliSeconds
Milliseconds Since Epoch&lt;/_array_net.dinglisch.android.tasker.RELEVANT_VARIABLES1&gt;&lt;_array_net.dinglisch.android.tasker.RELEVANT_VARIABLES2&gt;%dt_seconds
2. Seconds
Seconds Since Epoch&lt;/_array_net.dinglisch.android.tasker.RELEVANT_VARIABLES2&gt;&lt;_array_net.dinglisch.android.tasker.RELEVANT_VARIABLES3&gt;%dt_day_of_month
3. Day Of Month
&lt;/_array_net.dinglisch.android.tasker.RELEVANT_VARIABLES3&gt;&lt;_array_net.dinglisch.android.tasker.RELEVANT_VARIABLES4&gt;%dt_month_of_year
4. Month Of Year
&lt;/_array_net.dinglisch.android.tasker.RELEVANT_VARIABLES4&gt;&lt;_array_net.dinglisch.android.tasker.RELEVANT_VARIABLES5&gt;%dt_year
5. Year
&lt;/_array_net.dinglisch.android.tasker.RELEVANT_VARIABLES5&gt;&lt;/StringArray&gt;</net.dinglisch.android.tasker.RELEVANT_VARIABLES>
          <net.dinglisch.android.tasker.RELEVANT_VARIABLES-type>[Ljava.lang.String;</net.dinglisch.android.tasker.RELEVANT_VARIABLES-type>
        </Vals>
      </Bundle>
      <Int sr="arg1" val="1"/>
      <Int sr="arg10" val="0"/>
      <Str sr="arg11" ve="3"/>
      <Str sr="arg12" ve="3"/>
      <Str sr="arg2" ve="3"/>
      <Str sr="arg3" ve="3"/>
      <Str sr="arg4" ve="3"/>
      <Str sr="arg5" ve="3">yyyy_MM_dd_HH_mm_ss</Str>
      <Str sr="arg6" ve="3"/>
      <Str sr="arg7" ve="3">current_time</Str>
      <Int sr="arg8" val="0"/>
      <Int sr="arg9" val="0"/>
    </Action>
    <Action sr="act7" ve="7">
      <code>547</code>
      <Str sr="arg0" ve="3">%filename</Str>
      <Str sr="arg1" ve="3">%current_time.mp4</Str>
      <Int sr="arg2" val="0"/>
      <Int sr="arg3" val="0"/>
      <Int sr="arg4" val="0"/>
      <Int sr="arg5" val="3"/>
      <Int sr="arg6" val="1"/>
    </Action>
    <Action sr="act8" ve="7">
      <code>547</code>
      <Str sr="arg0" ve="3">%RECORDING</Str>
      <Str sr="arg1" ve="3">1</Str>
      <Int sr="arg2" val="0"/>
      <Int sr="arg3" val="0"/>
      <Int sr="arg4" val="0"/>
      <Int sr="arg5" val="3"/>
      <Int sr="arg6" val="1"/>
    </Action>
    <Action sr="act9" ve="7">
      <code>548</code>
      <Str sr="arg0" ve="3">%filename</Str>
      <Int sr="arg1" val="0"/>
      <Str sr="arg10" ve="3"/>
      <Int sr="arg11" val="1"/>
      <Int sr="arg12" val="0"/>
      <Str sr="arg13" ve="3"/>
      <Int sr="arg14" val="0"/>
      <Str sr="arg15" ve="3"/>
      <Int sr="arg2" val="0"/>
      <Str sr="arg3" ve="3"/>
      <Str sr="arg4" ve="3"/>
      <Str sr="arg5" ve="3"/>
      <Str sr="arg6" ve="3"/>
      <Str sr="arg7" ve="3"/>
      <Str sr="arg8" ve="3"/>
      <Int sr="arg9" val="1"/>
    </Action>
  </Task>
</TaskerData>

Overall, next steps

It looks like there are plenty of things I can do by voice. If I can talk, then I can record a braindump. If I can't talk but I can listen to things, then Emacs TV might be a good choice. If I want to read, I can read webpages or e-books. If my hands are busy, I can still add items to my grocery list or my Orgzly notebook. I just need to practice.

I can experiment with ARIA labels or Web Speech API interfaces on a simpler website, since emacs.tv is a bit complicated. If that doesn't let me do the speech interfaces I'm thinking of, then I might need to look into making a simple Android app.

I'd like to learn more about Orgzly Revived intents. At some point, I should probably learn more about Android programming too. There are a bunch of tweaks I might like to make to Orgzly Revived and the Emacs port of Android.

Also somewhat tempted by the idea of adding voice control or voice input to Emacs and/or Linux. If I'm on my computer already, I can usually just type, but it might be handy for using it hands-free while I'm in the kitchen. Besides, exploring accessibility early will also probably pay off when it comes to age-related changes. There's the ffmpeg+Whisper approach, there's a more sophisticated dictation mode with a voice cursor, there are some tools for Emacs tools for working with Talon or Dragonfly… There's been a lot of work in this area, so I might be able to find something that fits.

Promising!

View org source for this post

Hyperlinking SVGs

| drawing, supernote, emacs
Text and links from sketch

Hyperlinking SVGs - 2025-01-17-01

I like drawing my notes. I can jump around, draw connections, doodle for fun.

A sketch can only fit so much, though. (even if I write really small)

Idea: Links: They can be signposts for other trails.

Process:

I want to make maps for myself and other people.

This is easy to do because:

  • SVGs are XML, a text format
  • Emacs has code for XML and SVG manipulation, display
  • You can use Emacs to build a simple user interface.
  • Ideas
  • TO-DO: update sketch viewer
    • prioritize SVG
    • display Org

SuperNote also has its own hyperlinks, but:

  • typing long URLS on on-screen keyboards is not fun
  • I can't figure out how to convert those links to SVG
  • Rects are more compact

Preprocessing the image

This isn't the focus of this blog post, but I thought I'd include the code anyway in case someone might find it useful.

The fastest way to get a single file off the Supernote is to enable Browse & Access by swiping down from the top. It's the icon that looks like a two-way arrow between waves.

2025-01-21_10-28-16.png
Figure 1: Browse and Access

I have some Emacs Lisp code for downloading the latest exported file using the Supernote's web server.

my-supernote-get-exported-files
(defvar my-supernote-ip-address "192.168.1.221")
(defun my-supernote-get-exported-files ()
  (condition-case nil
      (let ((data (plz 'get (format "http://%s:8089/EXPORT" my-supernote-ip-address)))
            (list))
        (when (string-match "const json = '\\(.*\\)'" data)
          (sort
           (alist-get 'fileList (json-parse-string (match-string 1 data) :object-type 'alist :array-type 'list))
           :key (lambda (o) (alist-get 'date o))
           :lessp 'string<
           :reverse t)))
    (error nil)))

my-supernote-download-latest-exported-file: Save exported file in downloads dir.
(defun my-supernote-download-latest-exported-file ()
  "Save exported file in downloads dir."
  (interactive)
  (let* ((info (car (my-supernote-get-exported-files)))
         (dest-dir my-download-dir)
         (new-file (and info (expand-file-name (file-name-nondirectory (alist-get 'name info)) dest-dir)))
         renamed)
    (when info
      (copy-file
       (plz 'get (format "http://%s:8089%s" my-supernote-ip-address
                         (alist-get 'uri info))
         :as 'file)
       new-file
       t)
      new-file)))

Once I've downloaded the file, I process it:

  1. my-image-recognize: use Google Cloud Vision to recognize the text, rename it based on the ID
  2. my-sketch-rename: rename the file based on the ID if I've written one on the sketch
  3. my-sketch-convert-pdf: convert to SVG, copying over the links from the previous SVG if one exists
  4. my-sketch-clean: remove any images or templates
  5. my-sketch-color-to-hex: change the hex values for easier replacement and tinkering
  6. my-sketch-add-bg: add a plain white background rectangle
  7. my-sketch-change-fill-to-style: make the attributes more consistent
  8. my-sketch-recolor: change the highlight colour from gray to light yellow
  9. my-image-store: store it in either my private-sketches directory or my sketches directory, depending on the tags in the filename; leave untitled sketches in the same directory

my-supernote-process-sketch
(defun my-supernote-process-sketch (file)
  (interactive "FFile: ")
  (my-image-recognize file)
  (setq file (my-sketch-rename file))
  (pcase (file-name-extension file)
    ("pdf"
     (setq file
           (my-image-store
            (my-sketch-svg-prepare file))))
    ("png"
     (setq file
           (my-image-store
            (my-image-autorotate
             (my-image-autocrop
              (my-sketch-recolor-png
               file)))))))
  file)

my-sketch-svg-prepare: Clean up SVG for publishing.
(defvar my-debug-buffer (get-buffer-create "*temp*"))
(defun my-sketch-convert-pdf (pdf-file)
  "Returns the SVG filename."
  (interactive "FPDF: ")
  (if-let ((links (and (file-exists-p (concat (file-name-sans-extension pdf-file) ".svg"))
                       (dom-by-tag
                        (car (xml-parse-file (concat (file-name-sans-extension pdf-file) ".svg")))
                        'a))))
      ;; copy links over
      (let ((temp-file (concat (make-temp-name "svg-conversion") ".svg"))
            new-file)
        (unwind-protect
            (progn
              (call-process "pdftocairo" nil my-debug-buffer nil "-svg" (expand-file-name pdf-file)
                            temp-file)
              (setq new-file (car (xml-parse-file temp-file)))
              (dolist (link links)
                (dom-append-child new-file link))
              (with-temp-file (file-exists-p (concat (file-name-sans-extension pdf-file) ".svg"))
                (svg-print new-file)))
          (error
           (delete-file temp-file))))
    (delete-file (concat (file-name-sans-extension pdf-file) ".svg"))
    (call-process "pdftocairo" nil my-debug-buffer nil "-svg" (expand-file-name pdf-file)
                  (expand-file-name (concat (file-name-sans-extension pdf-file) ".svg"))))
  (concat (file-name-sans-extension pdf-file) ".svg"))

(defun my-sketch-change-fill-to-style (dom)
  "Inkscape handles these better when we split paths."
  (dolist (path (dom-by-tag dom 'path))
    (when (dom-attr path 'fill)
      (dom-set-attribute
       path 'style
       (if (dom-attr path 'style)
           (concat (dom-attr path 'style) ";fill:" (dom-attr path 'fill))
         (concat "fill:" (dom-attr path 'fill))))
      (dom-remove-attribute path 'fill)))
  dom)

(defun my-sketch-recolor (dom color-map &optional selector)
  "Colors are specified as ((\"#input\" . \"#output\") ...)."
  (if (symbolp color-map)
      (setq color-map
            (assoc-default color-map my-sketch-color-map)))
  (let ((map-re (regexp-opt (mapcar 'car color-map))))
    (dolist (path (if selector (dom-search dom selector)
                    (dom-by-tag dom 'path)))
      (dolist (attr '(style fill))
        (when (and (dom-attr path attr)
                   (string-match map-re (dom-attr path attr)))
          (dom-set-attribute
           path attr
           (replace-regexp-in-string
            map-re
            (lambda (match)
              (assoc-default match color-map))
            (or (dom-attr path attr) "")))))))
  dom)

(defun my-sketch-add-bg (dom)
  ;; add background rectangle
  (unless (dom-search dom (lambda (elem) (and (dom-attr elem 'class) (string-match "\\<background\\>" (dom-attr elem 'class)))))
    (let* ((view-box (mapcar 'string-to-number (split-string (dom-attr dom 'viewBox))))
           (bg-node (dom-node 'rect `((x . 0)
                                      (y . 0)
                                      (class . "background")
                                      (width . ,(elt view-box 2))
                                      (height . ,(elt view-box 3))
                                      (fill . "#ffffff")))))
      (if (dom-by-id dom "surface1")
          (push bg-node (cddr (car (dom-by-id dom "surface1"))))
        (push bg-node (cddr (car dom))))))
  dom)

(defun my-sketch-clean (dom)
  "Remove USE and IMAGE tags."
  (dolist (use (dom-by-tag dom 'use))
    (dom-remove-node dom use))
  (dolist (use (dom-by-tag dom 'image))
    (dom-remove-node dom use))
  dom)

(defun my-sketch-rotate (dom)
  (let* ((old-width (dom-attr dom 'width))
         (old-height (dom-attr dom 'height))
         (view-box (mapcar 'string-to-number (split-string (dom-attr dom 'viewBox))))
         (rotate (format "rotate(90) translate(0 %s)" (- (elt view-box 3)))))
    (dom-set-attribute dom 'width old-height)
    (dom-set-attribute dom 'height old-width)
    (dom-set-attribute dom 'viewBox (format "0 0 %d %d" (elt view-box 3) (elt view-box 2)))
    (dolist (g (dom-by-tag dom 'g))
      (dom-set-attribute g 'transform rotate)))
  dom)

(defun my-sketch-mix-blend-mode-darken (dom &optional selector)
  (dolist (p (if (functionp selector) (dom-search dom selector) (or selector (dom-by-tag dom 'path))))
    (when (and (dom-attr p 'style)
               (not (string-match "mix-blend-mode" (dom-attr p 'style))))
      (dom-set-attribute
       p 'style
       (replace-regexp-in-string ";;\\|^;" ""
                                 (concat
                                  (or (dom-attr p 'style) "")
                                  ";mix-blend-mode:darken")))))
  dom)

(defun my-sketch-color-to-hex (dom &optional selector)
  (dolist (p (if (functionp selector) (dom-search dom selector)
               (or selector (dom-search dom
                                        (lambda (p) (or (dom-attr p 'style)
                                                        (dom-attr p 'fill)))))))
    (dolist (attr '(style fill))
      (when (dom-attr p attr)
        (dom-set-attribute
         p attr
         (replace-regexp-in-string
          "rgb(\\([0-9\\.]+\\)%, *\\([0-9\\.%]+\\)%, *\\([0-9\\.]+\\)%)"
          (lambda (s)
            (color-rgb-to-hex
             (* 0.01 (string-to-number (match-string 1 s)))
             (* 0.01 (string-to-number (match-string 2 s)))
             (* 0.01 (string-to-number (match-string 3 s)))
             2))
          (dom-attr p attr))))))
  dom)

;; default for now, but will support more colour schemes someday
(defvar my-sketch-color-map
  '((blue
     ("#9d9d9d" . "#2b64a9")
     ("#9c9c9c" . "#2b64a9")
     ("#c9c9c9" . "#b3e3f1")
     ("#c8c8c8" . "#b3e3f1")
     ("#cacaca" . "#b3e3f1")
     ("#a6d2ff" . "#ffffff"))
    (t
     ("#9d9d9d" . "#888888")
     ("#9c9c9c" . "#888888")
     ("#cacaca" . "#f6f396")
     ("#c8c8c8" . "#f6f396")
     ("#a6d2ff" . "#ffffff")
     ("#c9c9c9" . "#f6f396"))))

(cl-defun my-sketch-svg-prepare (file &key color-map color-scheme new-file)
  "Clean up SVG for publishing."
  (when (string= (file-name-extension file) "pdf")
    (setq file (my-sketch-convert-pdf file)))
  (let ((dom (xml-parse-file file)))
    (setq dom (my-sketch-clean dom))
    (setq dom (my-sketch-color-to-hex dom))
    (setq dom (my-sketch-add-bg dom))
    (setq dom (my-sketch-change-fill-to-style dom))
    (setq dom (my-sketch-recolor dom
                                 (or color-map
                                     color-scheme
                                     t)))
    (with-temp-file (or new-file file) (svg-print (car dom)))
    (or new-file file)))

Editing and linking text

I've started keeping the text of the sketch in the same directory so that I can someday have full-text search for images. I have a keyboard shortcut for jumping to the text file. I like to open it in Org Mode.

my-org-sketch-open-text-file
(defun my-org-sketch-open-text-file (sketch)
  (interactive (list (my-complete-sketch-filename)))
  (find-file (concat (file-name-sans-extension sketch) ".txt"))
  (with-current-buffer (find-file-noselect sketch)
    (display-buffer-in-side-window
     (current-buffer)
     '((window-width . 0.5)
       (side . right)))))

The raw text from Google Cloud Vision is reasonably accurate but jumbled. I can move lines around with M-S-up and M-S-down in Org (org-shiftmetaup and org-shiftmetadown), which drag lines around. Once I add newlines, I can reorganize paragraphs with M-up and M-down (org-metaup and org-metadown). I can move list elements with M-S-right and M-S-left. (Idea: Avy probably has some awesome line-management functions I could get the hang of using.)

Once I've reorganized and cleaned up the text, I add links. Between my consult-omni shortcut and the new bookmarks I'm trying out (I should make a post about that), it's pretty easy.

Prompting for rectangles

Then it's a quick trip to Inkscape to draw rectangles over the things I want to link. It's easy to see where to draw the links because Org Mode highlights the links in the text. The style of the rectangles doesn't matter. After I save the SVG, I hop back into Emacs to turn them into links. This is the fun new part I just added.

Linkify rects

I like this because I got to reuse some code I'd written before to identify and reorder paths for easier animation of SVG topic maps. Using the links I defined in the previous step, all I needed to do was go through the rects (excluding the background rectangle) and offer completing-read on the titles and URLs. Then I createed the link elements and restyled the rectangles.

my-svg-linkify-rects
(defun my-svg-display (buffer-name svg &optional highlight-id full-window)
  "HIGHLIGHT-ID is a string ID or a node."
  (with-current-buffer (get-buffer-create buffer-name)
    (when highlight-id
      ;; make a copy
      (setq svg (with-temp-buffer (svg-print svg) (car (xml-parse-region (point-min) (point-max)))))
      (if-let* ((path (if (stringp highlight-id) (dom-by-id svg highlight-id) highlight-id))
                (view-box (split-string (dom-attr svg 'viewBox)))
                (box (my-svg-bounding-box path))
                (parent (car path)))
          (progn
            ;; find parents for possible rotation
            (while (and parent (not (dom-attr parent 'transform)))
              (setq parent (dom-parent svg parent)))
            (dom-set-attribute path 'style
                               (concat (dom-attr path 'style) "; stroke: 1px red; fill: #ff0000 !important"))
            ;; add a crosshair
            (dom-append-child
             (or parent svg)
             (dom-node 'path
                       `((d .
                            ,(format "M %f,0 V %s M %f,0 V %s M 0,%f H %s M 0,%f H %s"
                                     (elt box 0)
                                     (elt view-box 3)
                                     (elt box 2)
                                     (elt view-box 3)
                                     (elt box 1)
                                     (elt view-box 2)
                                     (elt box 3)
                                     (elt view-box 2)))
                         (stroke-dasharray . "5,5")
                         (style . "fill:none;stroke:gray;stroke-width:3px")))))
        (error "Could not find %s" highlight-id)))
    (let* ((inhibit-read-only t)
           (image (svg-image svg))
           (edges (window-inside-pixel-edges (get-buffer-window))))
      (erase-buffer)
      (if full-window
          (progn
            (delete-other-windows)
            (switch-to-buffer (current-buffer)))
        (display-buffer (current-buffer)))
      (insert-image (append image
                            (list :max-width
                                  (floor (* 0.8 (- (nth 2 edges) (nth 0 edges))))
                                  :max-height
                                  (floor (* 0.8 (- (nth 3 edges) (nth 1 edges)))) )))
      ;; (my-svg-resize-with-window (selected-window))
      ;; (add-hook 'window-state-change-functions #'my-svg-resize-with-window t)
      (current-buffer))))

(cl-defun my-svg-identify-paths (filename &key selector node-func dom)
  "Prompt for IDs for each path in FILENAME."
  (interactive (list (read-file-name "SVG: " nil nil
                                     (lambda (f)
                                       (or (string-match "\\.svg$" f)
                                           (file-directory-p f))))))
  (let* ((dom (or dom (car (xml-parse-file filename))))
         (paths (if (functionp selector)
                    (dom-search dom selector)
                  (or selector
                      (dom-by-tag dom 'path))))
         (vertico-count 3)
         (ids (seq-keep (lambda (path)
                          (and (dom-attr path 'id)
                               (unless (string-match "\\(path\\|rect\\)[0-9]+"
                                                     (or (dom-attr path 'id) "path0"))
                                 (dom-attr path 'id))))
                        paths))
         (edges (window-inside-pixel-edges (get-buffer-window)))
         id)
    (my-svg-display "*image*" dom nil t)
    (dolist (path paths)
      ;; display the image with an outline
      (unwind-protect
          (progn
            (my-svg-display "*image*" dom (dom-attr path 'id) t)
            (if (functionp node-func)
                (funcall node-func path dom)
              (setq id (completing-read
                        (format "ID (%s): " (dom-attr path 'id))
                        ids))
              ;; already exists, merge with existing element
              (if-let* ((old (dom-by-id dom id)))
                  (progn
                    (dom-set-attribute
                     old
                     'd
                     (concat (dom-attr (dom-by-id dom id) 'd)
                             " "
                             ;; change relative to absolute
                             (replace-regexp-in-string "^m" "M"
                                                       (dom-attr path 'd))))
                    (dom-remove-node dom path)
                    (setq id nil))
                (dom-set-attribute path 'id id)
                (add-to-list 'ids id)))))
      ;; save the image just in case we get interrupted halfway through
      (with-temp-file filename
        (svg-print dom)))))

(defun my-svg-identify-rects (filename)
  (interactive (list (read-file-name "SVG: " nil nil
                                     (lambda (f)
                                       (or (string-match "\\.svg$" f)
                                           (file-directory-p f))))))
  (my-svg-identify-paths
   filename
   :selector
   (lambda (elem)
     (and (eq (dom-tag elem) 'rect)
          (not (and (dom-attr elem 'class)
                    (string-match "\\<background\\>" (dom-attr elem 'class))))))))

(defun my-org-links-from-file (filename)
  "Return a list of (description . link) of the Org links in FILENAME."
  (when (file-exists-p filename)
    (let (results)
      (with-temp-buffer
        (insert-file-contents filename)
        (goto-char (point-min))
        (while (re-search-forward org-link-any-re nil t)
          (push (cons (match-string-no-properties 3)
                      (or (match-string-no-properties 2)
                          (match-string-no-properties 0)))
                results)))
      (reverse results))))

(defun my-svg-linkify-rects (filename)
  (interactive (list (read-file-name "SVG: " nil nil
                                     (lambda (f)
                                       (or (string-match "\\.svg$" f)
                                           (file-directory-p f))))))
  (let ((dom (car (xml-parse-file filename)))
        (links-from-text (my-org-links-from-file (concat (file-name-sans-extension filename) ".txt"))))
    (my-svg-identify-paths
     filename
     :dom
     dom
     :selector
     (append
      ;; not yet linked
      (dom-search dom
                  (lambda (elem)
                    (and (eq (dom-tag elem) 'rect)
                         (not (and (dom-attr elem 'class)
                                   (string-match "\\<background\\|link-rect\\>" (dom-attr elem 'class)))))))
      ;; linked
      (dom-search dom
                  (lambda (elem)
                    (and (eq (dom-tag elem) 'rect)
                         (string-match "\\<link-rect\\>" (or (dom-attr elem 'class) ""))))))

     :node-func
     (lambda (elem dom)
       (let* ((current-link-node (my-dom-closest dom elem 'a))
              (current-title-node (or (dom-by-tag elem 'title)
                                      (dom-by-tag current-link-node 'title)))
              (title (string-trim
                      (completing-read
                      "Title: "
                      (mapcar 'car links-from-text)
                      nil nil
                      (dom-text current-title-node))))
              (link (string-trim
                     (read-string
                       "URL: "
                       (or (dom-attr current-link-node 'href)
                           (assoc-default title links-from-text 'string=)))
                     )))
         (cond
          ((and current-link-node (not (string= link "")))
           (dom-set-attribute elem
                              'style
                              "stroke: blue; stroke-dasharray: 4; fill: #006fff; fill-opacity: 0.25")
           (dom-set-attribute current-link-node 'href link))
          ((and current-link-node (string= link ""))
           (dom-add-child-before
            (dom-parent dom current-link-node)
            elem)
           (dom-remove-node current-link-node))
          ((and (null current-link-node) (not (string= link "")))
           (setq current-link-node (dom-node
                                    'a
                                    `((href . ,link)
                                      (class . "link"))))
           (dom-add-child-before (dom-parent dom elem) current-link-node elem)
           (dom-remove-node dom elem)
           (dom-append-child current-link-node elem)
           (dom-remove-attribute elem 'fill)
           (dom-set-attribute elem
                              'style
                              "stroke: blue; stroke-dasharray: 4; fill: #006fff; fill-opacity: 0.25")
           (dom-set-attribute
            elem
            'class
            (if (dom-attr elem 'class)
                (concat (dom-attr elem 'class) " link-rect")
              "link-rect"))))
         (cond
          ((and (string= title "") current-title-node)
           (dom-remove-node current-title-node))
          ((and (not (string= title "")) (not current-title-node))
           (dom-append-child current-link-node (dom-node 'title nil title)))
          ((and (not (string= title "")) current-title-node)
           (setf (car (dom-children current-title-node))
                 title))))))))

(defun my-svg-update-links-from-text (filename)
  (interactive (list (read-file-name
                      "SVG: " nil
                      (if (file-exists-p (concat (file-name-sans-extension (buffer-file-name)) ".svg"))
                          (concat (file-name-sans-extension (buffer-file-name)) ".svg")
                        (cdr (my-embark-image)))
                      (lambda (f)
                        (or (string-match "\\.svg$" f)
                            (file-directory-p f))))))
  (let ((dom (car (xml-parse-file filename)))
        (links-from-text (my-org-links-from-file (concat (file-name-sans-extension filename) ".txt"))))
    (dolist (link (dom-by-tag dom 'a))
      (when (and
             (assoc-default (dom-text (dom-by-tag link 'title))
                            links-from-text)
             (not (string=
                   (dom-attr link 'href)
                   (assoc-default (dom-text (dom-by-tag link 'title))
                                  links-from-text))))
        (dom-set-attribute
         link
         'href
         (assoc-default (dom-text (dom-by-tag link 'title))
                        links-from-text))))
    (with-temp-file filename
      (svg-print dom))))



Writing about the sketch

I tweaked my function for drafting a blog post about a sketch. I added panning and zooming capabilities using Javascript, included the sketch text, and added any sections that I referred to using anchors. (TODO: Come to think of it, I should rewrite those to be absolute links using the permalink so that they'll still make sense even if people bookmark them from the main page of my blog.)

my-write-about-sketch
(defun my-insert-sketch-and-text (sketch)
  (interactive (list (my-complete-sketch-filename)))
  (insert
   (format "#+begin_panzoom\n%s\n#+end_panzoom\n\n"
           (if (string= (file-name-extension sketch) "svg")
               (org-link-make-string (concat "file:" sketch))
             (org-link-make-string (concat "sketchFull:" sketch)))))
  (let ((links (my-org-links-from-file (concat (file-name-sans-extension sketch) ".txt")))
        (subheading-level (1+ (org-current-level))))
    (insert (if links
                "#+begin_my_details Text and links from sketch\n"
              "#+begin_my_details Text from sketch\n"))
    (my-sketch-insert-text sketch)
    (unless (bolp) (insert "\n"))
    (insert "#+end_my_details")
    (dolist (section (seq-filter (lambda (entry) (string-match "^#" (cdr entry)))
                                 links))
      (org-end-of-subtree)
      (insert "\n\n")
      (org-insert-heading nil nil subheading-level)
      (insert (car section))
      (org-entry-put (point) "CUSTOM_ID" (substring (cdr section) 1)))))

(defun my-write-about-sketch (sketch)
  (interactive (list (my-complete-sketch-filename)))
  (shell-command "make-sketch-thumbnails")
  (find-file "~/sync/orgzly/posts.org")
  (goto-char (point-min))
  (unless (org-at-heading-p) (outline-next-heading))
  (org-insert-heading nil nil t)
  (insert (file-name-base sketch) "\n\n")
  (my-insert-sketch-and-text sketch)
  (delete-other-windows)
  (save-excursion
    (with-selected-window (split-window-horizontally)
      (find-file sketch))))

And then I can export the image as an inline SVGs in Org Mode HTML and Markdown exports, yay!

Other functions not included above are probably somewhere in my Emacs config.

Using an SVG as a sticky table of contents

… and now I can make the image a sticky table of contents as you scroll down, by wrapping it in something like this:

#+begin_sticky-toc-after-scrolling
#+begin_panzoom
file:/home/sacha/sync/sketches/2025-01-17-01 Hyperlinking SVGs -- drawing supernote inkscape svg.svg
#+end_panzoom
#+end_sticky-toc-after-scrolling

Mwahahaha! (Now I just need to make it highlight different sections as we scroll…)

Here's the snippet from my misc.js:

Sticky table of contents after scrolling
function stickyTocAfterScrolling() {
  const elements = document.querySelectorAll('.sticky-toc-after-scrolling');
  let lastScroll = window.scrollY;
  const cloneMap = new WeakMap();

  elements.forEach(element => {
    const clone = element.cloneNode(true);
    clone.setAttribute('class', 'sticky-toc');
    cloneMap.set(element, clone);
    element.parentNode.insertBefore(clone, element.nextSibling);
    const zoom = panZoom = svgPanZoom(clone.querySelector('svg'));
    zoom.resetZoom();
  });

  const observer = new IntersectionObserver(
    (entries) => {
      const currentScroll = window.scrollY;
      const scrollingDown = currentScroll > lastScroll;
      lastScroll = currentScroll;

      entries.forEach(entry => {
        const element = entry.target;
        const clone = cloneMap.get(element);

        if (!entry.isIntersecting && scrollingDown) {
          clone.setAttribute('class', 'sticky-toc');
          clone.style.display = 'block';
        } else if (entry.isIntersecting && !scrollingDown) {
          element.style.visibility = 'visible';
          clone.style.display = 'none';
        }
      });
    },
    {
      root: null,
      threshold: 0,
      rootMargin: '-10px 0px 0px 0px'
    }
  );

  elements.forEach(element => {
    observer.observe(element);
  });

  window.addEventListener('resize', () => {
    elements.forEach(element => {
      const clone = cloneMap.get(element);
      if (clone.style.display != 'none') {
        // reset didn't seem to work
        svgPanZoom(clone.querySelector('svg')).destroy();
        addPanZoomToElement(clone.querySelector('svg'));
      }
    });
  }, { passive: true });
}

stickyTocAfterScrolling();
View org source for this post