Categories: supernote

RSS - Atom - Subscribe via email

Using Emacs Lisp to export TXT/EPUB/PDF from Org Mode to the Supernote via Browse and Access

| supernote, org, emacs

I've been experimenting with the Supernote's Browse and Access feature because I want to be able to upload files quickly instead of waiting for Dropbox to synchronize. First, I want to store the IP address in a variable:

my-supernote-ip-address
(defvar my-supernote-ip-address "192.168.1.221")

Here's how to upload:

(defun my-supernote-upload (filename &optional supernote-path)
  (interactive "FFile: ")
  (setq supernote-path (or supernote-path "/INBOX"))
  (let* ((boundary (mml-compute-boundary '()))
         (url-request-method "POST")
         (url-request-extra-headers
          `(("Content-Type" . ,(format "multipart/form-data; boundary=%s" boundary))))
         (url-request-data
          (mm-url-encode-multipart-form-data
           `(("file" . (("name" . "file")
                        ("filename" . ,(file-name-nondirectory filename))
                        ("content-type" . "application/octet-stream")
                        ("filedata" . ,(with-temp-buffer
                                         (insert-file-contents-literally filename)
                                         (buffer-substring-no-properties (point-min) (point-max)))))))
           boundary)))
    (with-current-buffer
        (url-retrieve-synchronously
         (format "http://%s:8089%s" my-supernote-ip-address supernote-path))
      (re-search-backward "^$")
      (prog1 (json-read)
        (kill-buffer)))))

HTML isn't supported. Text works, but it doesn't support annotation. PDF or EPUB could work. It would make sense to register this as an export backend so that I can call it as part of the usual export process.

(defun my-supernote-org-upload-as-text (&optional async subtree visible-only body-only ext-plist)
  "Export Org format, but save it with a .txt extension."
  (interactive (list nil current-prefix-arg))
  (let ((filename (org-export-output-file-name ".txt" subtree))
        (text (org-export-as 'org subtree visible-only body-only ext-plist)))
    ;; consider copying instead of exporting so that #+begin_export html etc. is preserved
    (with-temp-file filename
      (insert text))
    (my-supernote-upload filename)))

(defun my-supernote-org-upload-as-pdf (&optional async subtree visible-only body-only ext-plist)
  (interactive (list nil current-prefix-arg))
  (my-supernote-upload (org-latex-export-to-pdf async subtree visible-only body-only ext-plist)))

(defun my-supernote-org-upload-as-epub (&optional async subtree visible-only body-only ext-plist)
  (interactive (list nil current-prefix-arg))
  (my-supernote-upload (org-epub-export-to-epub async subtree visible-only ext-plist)))

(org-export-define-backend
    'supernote nil
    :menu-entry '(?s "Supernote"
                     ((?s "as PDF" my-supernote-org-upload-as-pdf)
                      (?e "as EPUB" my-supernote-org-upload-as-epub)
                      (?o "as Org" my-supernote-org-upload-as-text))))

Adding this line to my Org file allows me to use \spacing{1.5} for 1.5 line spacing, so I can write in more annotations..

#+LATEX_HEADER+: \usepackage{setspace}

Sometimes I use custom blocks for HTML classes. When LaTeX complains about undefined environments, I can define them like this:

#+LATEX_HEADER+: \newenvironment{whatever_my_custom_environment_is_called}

Now I can export a subtree or file to my Supernote for easy review.

I wonder if multimodal AI models can handle annotated images with editing marks…

This is part of my Emacs configuration.
View org source for this post

org-attaching the latest image from my Supernote via Browse and Access

Posted: - Modified: | emacs, supernote, org

[2024-09-29 Sun]: Use sketch links when possible. Recolor before cropping so that the grid is removed.

2024-09-26-01 Supernote A5X Browse and Access %23supernote.png
Figure 1: Diagram of different ways to get drawings off my Supernote A5X
Text from sketch

Supernote A5X

  • Screen mirroring (pixelated) -> Puppeteer screenshot (or maybe .mjpeg?)
  • Browse & Access (HTTP) -> latest file: recognize text, recolor, crop, upload?
  • Dropbox/Google Drive (slow) -> batch process: recognize text, recolor, upload

Bonus: Autocropping encourages me to just get stuff out there even if I haven't filled a page

ideas: remove template automatically? I wonder if I can use another color…

2024-09-26-01

I want to quickly get drawings from my Supernote A5X into Emacs so that I can include them in blog posts. Dropbox/Google Drive sync is slow because it synchronizes all the files. The Supernote can mirror its screen as an .mjpeg stream. I couldn't figure out how to grab a frame from that, but I did find out how to use Puppeteer to take an screenshot of the Supernote's screen mirror. Still, the resulting image is a little pixelated. If I turn on Browse and Access, the Supernote can serve directories and files as webpages. This lets me grab the latest file and process it. I don't often have time to fill a full A5 page with thoughts, so autocropping the image encourages me to get stuff out there instead of holding on to things.

(defvar my-supernote-ip-address "192.168.1.221")
(defun my-supernote-get-exported-files ()
  (let ((data (plz 'get (format "http://%s:8089/EXPORT" my-supernote-ip-address)))
        (list))
    (when (string-match "const json = '\\(.*\\)'" data)
      (sort
       (alist-get 'fileList (json-parse-string (match-string 1 data) :object-type 'alist :array-type 'list))
       :key (lambda (o) (alist-get 'date o))
       :lessp 'string<
       :reverse t))))

(defun my-supernote-org-attach-latest-exported-file ()
  (interactive)
  ;; save the file to the screenshot directory
  (let ((info (car (my-supernote-get-exported-files)))
        new-file
        renamed)
    ;; delete matching files
    (setq new-file (expand-file-name
                    (replace-regexp-in-string " " "%20" (alist-get 'name info) (org-attach-dir))))
    (when (file-exists-p new-file)
      (delete-file new-file))
    (org-attach-attach
     (format "http://%s:8089%s" my-supernote-ip-address
             (alist-get 'uri info))
     nil
     'url)
    (setq new-file (my-latest-file (org-attach-dir)))
    ;; recolor
    (my-sketch-recolor-png new-file)
    ;; autocrop that image
    (my-image-autocrop new-file)
    ;; possibly rename
    (setq renamed (my-image-recognize-get-new-filename new-file))
    (when renamed
      (setq renamed (expand-file-name renamed (org-attach-dir)))
      (rename-file new-file renamed t)
      (my-image-store renamed) ; file it in my archive
      (setq new-file renamed))
    ;; use a sketch link if it has an ID
    (if (string-match "^[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]-[0-9][0-9] "
                      (file-name-base renamed))
        (org-insert-link nil (concat "sketchFull:" (file-name-base renamed)))
      ;; insert the link
      (org-insert-link nil (concat "attachment:" (replace-regexp-in-string "#" "%23" (file-name-nondirectory new-file)))))
    (org-redisplay-inline-images)))
This is part of my Emacs configuration.
View org source for this post

Using Puppeteer to grab an image from the SuperNote's screen mirror

| supernote

[2024-09-13 Fri] I added a mogrify call to automatically trim the image.

Partly inspired by John Kitchin's video showing how to copy screenshots from his iPad and do optical character recognition so he can use the images and text in Org Mode, I'd like to be able to draw quick notes while I'm thinking through a topic on my computer.

Krita might work, but it's awkward to draw on my tablet PC's screen when it's in laptop mode because of the angle. Flipping it to tablet mode is a bit disruptive.

I can draw on my Supernote, which feels a bit more natural. I have a good workflow for recoloring and renaming exported sketches, but exporting via Dropbox is a little slow since it synchronizes all the folders. The SuperNote has a built-in screen mirroring mode with an MJPEG that I can open in a web browser. Saving it to an image is a little complicated, though. ffmpeg doesn't work with the MJPEG that it streams, and I can't figure out how to get stuff out aside from using a browser. I can work around this by using Puppeteer and getting a screenshot. Here's a NodeJS snippet that saves that screenshot to a file.

/* This file is tangled to ~/bin/supernote-screenshot.js from my config at https://sachachua.com/dotemacs
Usage: supernote-screenshot.js [filename]
Set SUPERNOTE_URL to the URL.
*/

const process = require('process');
const puppeteer = require('puppeteer');
const url = process.env['SUPERNOTE_URL'] || 'http://192.168.1.221:8080/screencast.mjpeg';
const scale = 0.5;
const delay = 2000;

async function takeSupernoteScreenshot() {
  const browser = await puppeteer.launch({headless: 'new'});
  const page = await browser.newPage();
  await page.setViewport({width: 2808 * scale, height: 3744 * scale, deviceScaleFactor: 1});
  page.goto(url);
  await new Promise((resolve, reject) => setTimeout(resolve, delay));
  let filename = process.argv[2] || 'screenshot.png';
  await page.screenshot({type: 'png', path: filename, fullPage: true});
  await browser.close();
}

takeSupernoteScreenshot();

Then I can call that from Emacs Lisp and run it through my usual screenshot insertion process:

(defun my-org-insert-supernote-screenshot-from-mirror ()
  "Copy the current image from the SuperNote mirror."
  (interactive)
  (let ((filename (expand-file-name (format-time-string "%Y-%m-%d-%H-%M-%S.png") "~/recordings")))
    (shell-command-to-string (concat "NODE_PATH=/usr/lib/node_modules node ~/bin/supernote-screenshot.js " (shell-quote-argument filename)))
    ;; trim it
    (call-process "mogrify" nil nil nil "-trim" "+repage" filename)
    (shell-command-to-string (concat "~/bin/recolor.py --colors c0c0c0,f6f396 " (shell-quote-argument filename)))
    (call-interactively 'my-org-insert-screenshot)))

[2024-09-13 Fri] I already have some code elsewhere for using the Google Cloud Vision API to extract text from an image, so I should hook that up sometime. Also, OpenAI supports multimodal requests, so I've been thinking about using AI to recognize text and diagrams like in this Supernote to Markdown example. Fun fun fun!

This is part of my Emacs configuration.

Tweaking my writing workflow using SuperNote's new handwriting recognition

| blogging, supernote

Both Google Cloud Vision and SuperNote's new handwriting recognition handle my print fine. Neither handle columns the way I'd like, but to be fair, I'm not really sure how I want columns and wrapping handled anyway. I can always experiment with the standard use-case: one column of text, to export as text (with perhaps the occasional sketch, which I can crop and include).

If I can get the hang of writing my thoughts, then it turns some of those bedtime hours into writing hours. Writing by hand feels slow and linear, but it's better than nothing, and thinking takes most of the time anyway. While speech recognition feels like it might be faster in short bursts, I don't have a lot of "talking to myself" time (aside from sleepy brain dumps), and my workflow for processing audio is still slow and disjointed. I can't type on my phone because then A- will want to be on a screen too. I'm glad e-ink devices are different enough not to trigger her sense of unfairness, although sometimes she does ask if she can do mazes or connect-the-dots. Then I switch to knitting until it's really really time to go to bed.

I'm slowly figuring out my workflows for experimenting with and writing about code. Naturally, that's a little more challenging to write about by hand, but I could draft the context. I can think through life stuff too, and maybe look into saving more notes in my Org files.

I've experimented with handwritten blog posts before. Now that I have a little more time to tweak my workflow and think thoughts, maybe I'll get the hang of them!


It looks like the Supernote's real-time recognition is pretty accurate for my handwriting, getting the text out of multiple pages is pretty straightforward.

Here's the raw TXT output from the Supernote.

Here's what it took to edit it into the first part of this post - just adding line-breaks and fixing up some words:

"A screen recording showing editing"
Figure 1: My editing process - just added line breaks and fixed some words
Source images

[["The first page of my handwritten post"

"The second page of my handwritten post"
Figure 2: Second page

If I add more lines between paragraphs when writing, I might be able to skip adding them in the text export.

For comparison, here's the text output from Google Cloud Vision.

Tweaking my handwriting workflow
Both Google Cloud Vision and Super Note's new
handwriting recognition handle my print fine. Neither
handle columns the way I'd like, but to be fair,
I'm not really sure how I want columns and wrapping
handled anyway I can always experiment with the
standard use-case
use-case: One column of text, to export
as Text (with perhaps the occasional sketch, which
can crop and include).
If I can get the hang of writing my thoughts,
then it turns some of those bedtime hours into writi
writing
hours. Writing by hand feels slow and linear, but it's
better than nothing, and thinking takes most of the time
anyway while speech recognition feels like it might be
faster in short bursts, don't have a lot of "talking to
myself" time (aside from sleepy braindumps), and my workflow
for processing audio is still slow and disjointed. I can't
type on my phone because then A- will want to be on

I'm glad e-ink devices are different enough
not to trigger her sense of unfairness, although sometimes
she does ask if she can do mazes or connect-the-dots
a screen too
Then I switch to Knitting until it's really really time to
go to bed.
I'm slowly figuring out my workflows for experimenting
with and writing about code. Naturally, that's a little
more challenging to write about by hand, but I could
draft the context. I can think through life stuff too, and
maybe look into saving more notes in my org files
I've experimented with handwritten blog posts before
Now that I have a little more time to tweak my workflow
and think thoughts, maybe I'll get the hang of them!

I'm leaning towards SuperNote's recognition results for long text, although I don't get access to the confidence data so I'll probably just have to delete the misrecognized text if I include sketches.

Rename, recolor, and file my sketches automatically

| geek, supernote, python, drawing

I want to make it easier to process the sketchnotes I make on my Supernote. I write IDs of the form yyyy-mm-dd-nn to identify my sketches. To avoid duplicates, I get these IDs from the web-based journaling system I wrote. I've started putting the titles and tags into those journal entries as well so that I can reuse them in scripts. When I export a sketch to PNG and synchronize it, the file appears in my ~/Dropbox/Supernote/EXPORT directory on my laptop. Then it goes through this process:

  • I use Google Cloud Vision to detect handwriting so that I can find the ID.
    • I retrieve the matching entry from my journal system and rename the file based on the title and tags.
    • If there's no matching entry, I rename the file based on the ID.
  • If there are other tags or references in the sketch, I add those to the filename as well.
  • I recolor it based on the tags, so parenting-related posts are a little purple, tech/Emacs-related posts are blue, and things are generally highlighted in yellow otherwise.
  • I move it to a directory based on the tags.
    • If it's a private sketch, I move it to the directory for my private sketches.
    • If it's a public sketch, I move it to the directory that will eventually get synchronized to sketches.sachachua.com, and I reload the list of sketches after some delay.

The following code does that processing.

Download supernote-daemon

supernote-daemon source code
#!/usr/bin/python3
# -*- mode: python -*-

# (c) 2022-2023 Sacha Chua (sacha@sachachua.com) - MIT License

# Permission is hereby granted, free of charge, to any person
# obtaining a copy of this software and associated documentation files
# (the "Software"), to deal in the Software without restriction,
# including without limitation the rights to use, copy, modify, merge,
# publish, distribute, sublicense, and/or sell copies of the Software,
# and to permit persons to whom the Software is furnished to do so,
# subject to the following conditions:

# The above copyright notice and this permission notice shall be
# included in all copies or substantial portions of the Software.

# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
# BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
# ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.


import os
import json
import re
import requests
import time
from dotenv import load_dotenv
# Import the Google Cloud client libraries
from google.cloud import vision
from google.cloud.vision_v1 import AnnotateImageResponse
import sys
sys.path.append("/home/sacha/proj/supernote/")
import recolor   # noqa: E402  # muffles flake8 error about import
load_dotenv()


# Set the folder path where the png files are located
folder_path = '/home/sacha/Dropbox/Supernote/EXPORT/'
public_sketch_dir = '/home/sacha/sync/sketches/'
private_sketch_dir = '/home/sacha/sync/private-sketches/'

# Initialize the Google Cloud Vision client
client = vision.ImageAnnotatorClient()
refresh_counter = 0

def extract_text(client, file):
    json_file = file[:-3] + 'json'
    # TODO Preprocess to keep only black text
    with open(file, 'rb') as image_file:
        content = image_file.read()
    # Convert the png file to a Google Cloud Vision image object
    image = vision.Image(content=content)

    # Extract handwriting from the image using the Google Cloud Vision API
    response = client.document_text_detection(image=image)
    response_json = AnnotateImageResponse.to_json(response)
    json_response = json.loads(response_json)
    # Save the response to a json file with the same name as the png file
    with open(json_file, "w") as f:
        json.dump(json_response, f)


def maybe_rename(file):
    # TODO Match on ID
    json_file = file[:-3] + 'json'
    with open(json_file, 'r') as f:
        data = json.load(f)

    # Extract the text from the json file
    text = data['fullTextAnnotation']['text']

    # Check if the text contains a string matching the regex pattern
    pattern = r'(?<!ref:)[0-9]{4}-[0-9]{2}-[0-9]{2}-[0-9]{2}'
    match = re.search(pattern, text)
    if match:
        # Get the matched string
        matched_string = match.group(0)
        new_name = matched_string
        from_zid = get_journal_entry(matched_string).strip()
        if from_zid:
            new_name = matched_string + ' ' + from_zid
        tags = get_tags(new_name, text)
        if tags:
            new_name = new_name + ' ' + tags
        ref = get_references(text)
        if ref:
            new_name = new_name + ' ' + ref
        print('Renaming ' + file + ' to ' + new_name)
        # Rename the png and json files to the matched string
        new_filename = os.path.join(os.path.dirname(file), new_name + '.png')
        rename_set(file, new_filename)
        return new_filename


def get_tags(filename, text):
    tags = re.findall(r'(^|\W)#[ \n\t]+', text)
    return ' '.join(filter(lambda x: x not in filename, tags))


def get_references(text):
    refs = re.findall(r'!ref:[0-9]{4}-[0-9]{2}-[0-9]{2}-[0-9]{2}', text)
    return ' '.join(refs)


def get_journal_entry(zid):
    resp = requests.get('https://' + os.environ['JOURNAL_USER']
                        + ':' + os.environ['JOURNAL_PASS']
                        + '@journal.sachachua.com/api/entries/' + zid)
    j = resp.json()
    if j and not re.search('^I thought about', j['Note']):
        return j['Note']


def get_color_map(filename, text=None):
    if text:
        together = filename + ' ' + text
    else:
        together = filename
    if re.search('r#(parenting|purple|life)', together):
        return {'9d9d9d': '8754a1', 'c9c9c9': 'e4c1d9'}  # parenting is purplish
    elif re.search(r'#(emacs|geek|tech|blue)', together):
        return {'9d9d9d': '2b64a9', 'c9c9c9': 'b3e3f1'}  # geeky stuff in light/dark blue
    else:
        return {'9d9d9d': '884636', 'c9c9c9': 'f6f396'}  # yellow highlighter, dark brown


def rename_set(old_name, new_name):
    if old_name != new_name:
        old_json = old_name[:-3] + 'json'
        new_json = new_name[:-3] + 'json'
        os.rename(old_name, new_name)
        os.rename(old_json, new_json)


def recolor_based_on_filename(filename):
    color_map = get_color_map(filename)
    recolored = recolor.map_colors(filename, color_map)
    # possibly rename based on the filename
    new_filename = re.sub(' #(purple|blue)', '', filename)
    rename_set(filename, new_filename)
    recolored.save(new_filename)


def move_processed_sketch(file):
    global refresh_counter
    if '#private' in file:
        output_dir = private_sketch_dir
    elif '#' in file:
        output_dir = public_sketch_dir
        refresh_counter = 3
    else:
        return file
    new_filename = os.path.join(output_dir, os.path.basename(file))
    rename_set(file, new_filename)
    return new_filename


def process_file(file):
    json_file = file[:-3] + 'json'
    # Check if a corresponding json file already exists
    if not os.path.exists(json_file):
        extract_text(client, file)
    if not re.search('[0-9]{4}-[0-9]{2}-[0-9]{2}-[0-9]{2} ', file):
        file = maybe_rename(file)
    recolor_based_on_filename(file)
    move_processed_sketch(file)


def process_dir(folder_path):
    global processed_files
    # Iterate through all png files in the specified folder
    files = sorted(os.listdir(folder_path))
    for file in files:
        if file.endswith('.png') and '_' in file:
            print("Processing ", file)
            process_file(os.path.join(folder_path, file))


def daemon(folder_path, wait):
    global refresh_counter
    while True:
        process_dir(folder_path)
        time.sleep(wait)
        if refresh_counter > 0:
            refresh_counter = refresh_counter - 1
            if refresh_counter == 0:
                print("Reloading sketches")
                requests.get('https://' + os.environ['JOURNAL_USER'] + ':'
                             + os.environ['JOURNAL_PASS']
                             + '@sketches.sachachua.com/reload?python=1')


if __name__ == '__main__':
    # Create a set to store the names of processed files
    processed_files = set()
    if len(sys.argv) > 1:
        if os.path.isdir(sys.argv[1]):
            folder_path = sys.argv[1]
            daemon(folder_path, 300)
        else:
            for f in sys.argv[1:]:
                process_file(f)
    else:
        daemon(folder_path, 300)

It uses this script I wrote to recolor my sketches with Python.

I'm contemplating writing some annotation tools to make it easier to turn the detected text into useful text for searching or writing about because the sketches throw off the recognition (misrecognized text, low confidence) and the columns mess up the line wrapping. Low priority, though.

My handwriting (at least for numbers) is probably simple enough that I might be able to train Tesseract OCR to process that someday. And who knows, maybe some organization will release a pre-trained model for offline handwriting recognition that'll be as useful as OpenAI Whisper is for audio files. That would be neat!

Compiling selected blog posts into HTML and EPUB so I can annotate them

| blogging, 11ty, nodejs, supernote

[2023-01-04 Wed] Added a screenshot showing annotation.

I was thinking about how to prepare for my next 10-year review, since I'll turn 40 this year. I've been writing yearly reviews with some regularity and monthly reviews sporadically, and I figured it would be nice to have those posts in an EPUB so that I can read them on my e-reader and annotate them as I do my review.

I use the 11ty static site generator to publish my blog as HTML files, since I currently can't keep more than Emacs Lisp, Javascript, and Python in my brain. (No Hugo or Jekyll for me at the moment.) I briefly thought about getting 11ty to create that archive for me, but I realized it might be easier to just write it as an external script instead of trying to figure out how to get 11ty to export one thing conditionally.

One of the things I've configured 11ty to make is a JSON file that includes all of my posts with dates, titles, permalinks, and categories. It was easy to then parse this list and filter it to get the posts I wanted. I parsed the HTML out of the _site directory that 11ty produces instead of fetching the pages from my webserver. I got the images from my webserver, though, and I made a local cache and rewrote the URLs. That way, the EPUB conversion could include the images.

Download blog.js

blog.js
const blog = require('/home/sacha/proj/static-blog/_site/blog/all/index.json');
const cheerio = require('cheerio');
const base = '/home/sacha/proj/static-blog/_site';
const fs = require('fs');
const path = require('path');

function slugify(p) {
  return p.permalink.replace('/blog', 'post-').replace(/\//g, '-');
}

async function processPost(p) {
  console.log('Processing '+ p.permalink);
  let $ = cheerio.load(fs.readFileSync(base + p.permalink + 'index.html'));
  $('#comment').remove();
  let images = $('article img');
  await Promise.all(images.map((i, e) => {
    let url = $(e).attr('src');
    const outputFileName = 'images/' + path.basename(url).replace(/ |%20|%23/g, '-');
    $(e).attr('src', outputFileName);
    $(e).attr('style', 'max-height: 100%; max-width: 100%; ' + ($(e).attr('style') || ''));
    $(e).attr('srcset', null);
    $(e).attr('sizes', null);
    $(e).attr('width', null);
    $(e).attr('height', null);
    if (!fs.existsSync(outputFileName)) {
      console.log('fetch', outputFileName);
      return fetch(url).then(res => res.arrayBuffer()).then(data => {
        const buffer = Buffer.from(data);
        return fs.createWriteStream(outputFileName).write(buffer);
      });
    } else {
      console.log(outputFileName, 'exists');
      return null;
    }
  }));
  console.log('Done ' + p.permalink);
  let slug = slugify(p);
  $('article h2').attr('id', slug);
  let header = $('article header').html();
  let entry = $('article .entry').html();
  return `<article>${header}${entry}</article>`;
}

let last10 = blog.filter((p) => p.date >= '2013-08-01');
let posts = last10.filter((p) => p.categories.indexOf('yearly') >= 0)
    .concat(blog.filter((p) => p.title == 'Turning 30: A review of the last decade'))
    .concat(last10.filter((p) => p.categories.indexOf('monthly') >= 0));

let toc = '<h1>Table of Contents</h1><ul>' + posts.map((p) => {
  return `<li><a href="#${slugify(p)}">${p.title}</a></li>\n`;
}).join('') + '</ul>';

let content = posts.reduce(async (prev, val) => {
    return await prev + await processPost(val);
  }, '');
content.then((data) => {
  fs.writeFileSync('archive.html',
                   `<html><body>${toc}${data}</body></html>`);

});

This created an archive.html with my posts, using the images/ directory for the images. Then I used my shell script for converting and copying files to convert it to EPUB and copy it over.

On the SuperNote, I can highlight text by drawing square brackets around it. If I tap that text, I can write or draw underneath it. Here's what that looks like:

20230104_090739.png
Figure 1: Writing an annotation

These notes are collected into a "Digest" view, and I can export things from there. (Example: archive.pdf)

2023-01-04_09-23-57.png
Figure 2: Here's what that digest is like when exported.

(Hmm, maybe I should ask them about hiding the pencil icon…)

Anyway, I think that might be a good starting point for my review.

Building up my tech notes

| geek, supernote
  • [2023-01-04 Wed] Added extra CSS to force images to fit on the page
  • [2023-01-03 Tue] Updated shell script to use EPUB for more formats

A- wants me to sit with her at bedtime. She also wants to read a stack of books until she gets sleepy. This means I sometimes have an hour (or even two) of sitting quietly with her which I can use for writing, drawing, reading, or knitting, as long as I'm quiet. ("Mama, keepp your thoughts to yourself!")

My Supernote A5X supports EPUBs and PDFs, but doesn't support HTML files or my library's e-book platform (Libby), and I'm not too keen on the Kindle app. So I need to load it up with my own collection of books, manuals, API documentation, and notes.

Org Mode can export to EPUBs and PDFs well enough. If I make the output file a symbolic link to the same file in the Dropbox folder that's synchronized with my Supernote, I can re-export the EPUB and it will end up in the right place when I sync. I've started accumulating little snippets from the digest of my reading highlights, since putting them into Org Mode allows me to organize them and summarize them in different ways. It feels good to be collecting and organizing things I'm learning.

I plan to use this reading time to skim documentation for interesting things, since sometimes the challenges are more about knowing something exists and what it's called. Then I can copy the digests into my reference.org and export it as an EPUB or PDF, review that periodically, and maybe add some shortcuts to my Emacs configuration so that I can quickly jump to lines in my reference file.

HTML

The Supernote doesn't support HTML files, but I can convert HTML to PDFs with pandoc file.html -t latex -o file.pdf. This shell script copies files to my INBOX directory, converting HTML files along the way:

#!/bin/bash
INBOX=~/Dropbox/Supernote/INBOX
for FILE in "$@"; do
    if [[ "$FILE" == *.html ]]; then
        ebook-convert "$FILE" $INBOX/$(basename "$FILE" .html).epub --extra-css 'img { max-width: 100% !important; max-weight: 100% !important }'
        # or pdf: wkhtmltopdf --no-background "$FILE" $INBOX/$(basename "$FILE" .html).pdf
    elif [[ "$FILE" == *.xml ]]; then
        dbtoepub "$FILE" -o $INBOX/$(basename "$FILE" .xml).epub
    elif [[ "$FILE" == *.texi ]]; then
        texi2pdf "$FILE" -o $INBOX/$(basename "$FILE" .texi).pdf
    elif [[ "$FILE" == *.org ]]; then
        emacs -Q --batch "$FILE" --eval "(progn (package-initialize) (use-package 'ox-epub) (org-epub-export-to-epub))"
        cp "${FILE%.*}".epub $INBOX
    else
        cp "$FILE" $INBOX
    fi
done

Manpages

I'd like to be able to refer to manpages. I couldn't figure out how to get man -H to work with the Firefox inside a snap (it complained about having elevated permissions). I installed man2html and found the manpage for xdotool. zcat /usr/share/man/man1/xdotool.1.gz | man2html > /tmp/xdotool.html created the HTML file, and then I used ebook-convert /tmp/xdotool.html /tmp/xdotool.epub to create an EPUB file.

I tried getting the filename for the manpage by using the man command in Emacs, but I couldn't figure out how to get the filename from there. I remembered that Emacs has a woman command that displays manpages without using the external man command. That led me to woman-file-name, which gives me the path to the manpage given a command. Emacs handles uncompressing .gz files automatically, so everything's good to go from there.

(defvar my-supernote-inbox "~/Dropbox/Supernote/INBOX")
(defun my-save-manpage-to-supernote (path)
  (interactive (list (woman-file-name nil)))
  (let* ((base (file-name-base path))
         (temp-html (make-temp-file base nil ".html")))
    (with-temp-buffer
      (insert-file-contents path)
      (call-process-region (point-min) (point-max) "man2html" t t)
      (when (re-search-backward "Invalid Man Page" nil t)
        (delete-file temp-html)
        (error "Could not convert."))
      (write-file temp-html))
    (call-process "ebook-convert" nil (get-buffer-create "*temp*") nil temp-html
                  (expand-file-name (concat base ".epub") my-supernote-inbox))
    (delete-file temp-html)))

Info files

I turned the Elisp reference into a PDF by going to doc/lispref in my Emacs checkout and typing make elisp.pdf. It's 1470 pages long, so that should keep me busy for a while. Org Mode also has a make pdf target that uses texi2pdf to generate doc/org.pdf and doc/orgguide.pdf. Other .texi files could be converted with texi2pdf, or I can use makeinfo to create Docbook files and then use dbtoepub to convert them as in the shell script in the HTML section above.

Python documentation

I wanted to load the API documentation for autokey into one page for easy reference. The documentation at https://autokey.github.io/index.html was produced by epydoc, which doesn't support Python 3. I got to work using the sphinx-epytext extension. After I used sphinx-quickstart, I edited conf.py to include extensions = ['sphinx.ext.autodoc', 'sphinx_epytext', 'sphinx.ext.autosummary'], and I added the following to index.rst:

Welcome to autokey's documentation!
===================================

   .. autoclass:: autokey.scripting.Keyboard
      :members:
      :undoc-members:

   .. autoclass:: autokey.scripting.Mouse
      :members:
      :undoc-members:

   .. autoclass:: autokey.scripting.Store
      :members:
      :undoc-members:

   .. autoclass:: autokey.scripting.QtDialog
      :members:
      :undoc-members:

   .. autoclass:: autokey.scripting.System
      :members:
      :undoc-members:

   .. autoclass:: autokey.scripting.QtClipboard
      :members:
      :undoc-members:

   .. autoclass:: autokey.scripting.Window
      :members:
      :undoc-members:

   .. autoclass:: autokey.scripting.Engine
      :members:
      :undoc-members:

Then make pdf created a PDF. There's probably a way to get a proper table of contents, but it was a good start.