Using Puppeteer to grab an image from the SuperNote's screen mirror

| supernote

Partly inspired by John Kitchin's video showing how to copy screenshots from his iPad and do optical character recognition so he can use the images and text in Org Mode, I'd like to be able to draw quick notes while I'm thinking through a topic on my computer.

Krita might work, but it's awkward to draw on my tablet PC's screen when it's in laptop mode because of the angle. Flipping it to tablet mode is a bit disruptive.

I can draw on my Supernote, which feels a bit more natural. I have a good workflow for recoloring and renaming exported sketches, but exporting via Dropbox is a little slow since it synchronizes all the folders. The SuperNote has a built-in screen mirroring mode with an MJPEG that I can open in a web browser. Saving it to an image is a little complicated, though. ffmpeg doesn't work with the MJPEG that it streams, and I can't figure out how to get stuff out aside from using a browser. I can work around this by using Puppeteer and getting a screenshot. Here's a NodeJS snippet that saves that screenshot to a file. 

#!/usr/bin/env nodejs
# This file is tangled to ~/bin/supernote-screenshot.js from my config at https://sachachua.com/dotemacs
# Usage: supernote-screenshot.js [filename]
# Set SUPERNOTE_URL to the URL.

const process = require('process');
const puppeteer = require('puppeteer');
const url = process.env['SUPERNOTE_URL'] || 'http://192.168.1.221:8080/screencast.mjpeg';
const scale = 0.5;
const delay = 2000;

async function takeSupernoteScreenshot() {
  const browser = await puppeteer.launch({headless: 'new'});
  const page = await browser.newPage();
  await page.setViewport({width: 2808 * scale, height: 3744 * scale, deviceScaleFactor: 1});
  page.goto(url);
  await new Promise((resolve, reject) => setTimeout(resolve, delay));
  let filename = process.argv[2] || 'screenshot.png';
  await page.screenshot({type: 'png', path: filename, fullPage: true});
  await browser.close();
}

takeSupernoteScreenshot();

Then I can call that from Emacs Lisp and run it through my usual screenshot insertion process: 

(defun my-org-insert-supernote-from-mirror ()
  "Copy the current image from the SuperNote mirror."
  (interactive)
  (let ((filename (expand-file-name (format-time-string "%Y-%m-%d-%H-%M-%S.png") "~/recordings")))
    (shell-command-to-string (concat "NODE_PATH=/usr/lib/node_modules node ~/bin/supernote-screenshot.js " (shell-quote-argument filename)))
    (shell-command-to-string (concat "~/bin/recolor.py --colors c0c0c0,f6f396 " (shell-quote-argument filename)))
    (call-interactively 'my-org-insert-screenshot)))

Ideas for next steps:

  • Add image actions:
    • Annotate the image
    • Crop the image
    • Get the text for the image at point and insert it as a details block
    • Get the text for the image at point and copy it to the kill-ring
This is part of my Emacs configuration.
You can comment with Disqus or you can e-mail me at sacha@sachachua.com.