I'd like to write about code more often. It's easier for people to try
out ideas if they can copy the code without fiddling with selecting
the text, especially on mobile browsers, so "Copy code" buttons on
source code blocks would be nice. I used this tutorial for adding code buttons as a basis for the following CSS and JS code.
First, let's add the buttons with Javascript. I want the buttons to be
visible in the summary line if I'm using the <details /> element. If
not, they can go in the div with the org-src-container class.
Someday I'll figure out how to make it easier to tangle things to the
post's directory and make the file available for download. In the
meantime, this might be a good start.
I want to make it easier to process the sketchnotes I make on my
Supernote. I write IDs of the form yyyy-mm-dd-nn to identify my
sketches. To avoid duplicates, I get these IDs from the web-based
journaling system I wrote. I've started putting the titles and tags
into those journal entries as well so that I can reuse them in
scripts. When I export a sketch to PNG and synchronize it, the file
appears in my ~/Dropbox/Supernote/EXPORT directory on my laptop.
Then it goes through this process:
I retrieve the matching entry from my journal
system and rename the file based on the title and tags.
If there's no matching entry, I rename the file based on the ID.
If there are other tags or references in the sketch, I add those to the filename as well.
I recolor it based on the tags, so parenting-related posts are a little purple, tech/Emacs-related posts are blue, and things are generally highlighted in yellow otherwise.
I move it to a directory based on the tags.
If it's a private sketch, I move it to the directory for my private sketches.
If it's a public sketch, I move it to the directory that will eventually get synchronized to sketches.sachachua.com, and I reload the list of sketches after some delay.
#!/usr/bin/python3# -*- mode: python -*-# (c) 2022-2023 Sacha Chua (sacha@sachachua.com) - MIT License# Permission is hereby granted, free of charge, to any person# obtaining a copy of this software and associated documentation files# (the "Software"), to deal in the Software without restriction,# including without limitation the rights to use, copy, modify, merge,# publish, distribute, sublicense, and/or sell copies of the Software,# and to permit persons to whom the Software is furnished to do so,# subject to the following conditions:# The above copyright notice and this permission notice shall be# included in all copies or substantial portions of the Software.# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS# BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN# ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE# SOFTWARE.import os
import json
import re
import requests
import time
from dotenv import load_dotenv
# Import the Google Cloud client librariesfrom google.cloud import vision
from google.cloud.vision_v1 import AnnotateImageResponse
import sys
sys.path.append("/home/sacha/proj/supernote/")
import recolor # noqa: E402 # muffles flake8 error about import
load_dotenv()
# Set the folder path where the png files are locatedfolder_path = '/home/sacha/Dropbox/Supernote/EXPORT/'public_sketch_dir = '/home/sacha/sync/sketches/'private_sketch_dir = '/home/sacha/sync/private-sketches/'# Initialize the Google Cloud Vision clientclient = vision.ImageAnnotatorClient()
refresh_counter = 0
defextract_text(client, file):
json_file = file[:-3] + 'json'# TODO Preprocess to keep only black textwithopen(file, 'rb') as image_file:
content = image_file.read()
# Convert the png file to a Google Cloud Vision image objectimage = vision.Image(content=content)
# Extract handwriting from the image using the Google Cloud Vision APIresponse = client.document_text_detection(image=image)
response_json = AnnotateImageResponse.to_json(response)
json_response = json.loads(response_json)
# Save the response to a json file with the same name as the png filewithopen(json_file, "w") as f:
json.dump(json_response, f)
defmaybe_rename(file):
# TODO Match on IDjson_file = file[:-3] + 'json'withopen(json_file, 'r') as f:
data = json.load(f)
# Extract the text from the json filetext = data['fullTextAnnotation']['text']
# Check if the text contains a string matching the regex patternpattern = r'(?<!ref:)[0-9]{4}-[0-9]{2}-[0-9]{2}-[0-9]{2}'match = re.search(pattern, text)
ifmatch:
# Get the matched stringmatched_string = match.group(0)
new_name = matched_string
from_zid = get_journal_entry(matched_string).strip()
if from_zid:
new_name = matched_string + ' ' + from_zid
tags = get_tags(new_name, text)
if tags:
new_name = new_name + ' ' + tags
ref = get_references(text)
if ref:
new_name = new_name + ' ' + ref
print('Renaming ' + file + ' to ' + new_name)
# Rename the png and json files to the matched stringnew_filename = os.path.join(os.path.dirname(file), new_name + '.png')
rename_set(file, new_filename)
return new_filename
defget_tags(filename, text):
tags = re.findall(r'(^|\W)#[ \n\t]+', text)
return' '.join(filter(lambda x: x notin filename, tags))
defget_references(text):
refs = re.findall(r'!ref:[0-9]{4}-[0-9]{2}-[0-9]{2}-[0-9]{2}', text)
return' '.join(refs)
defget_journal_entry(zid):
resp = requests.get('https://' + os.environ['JOURNAL_USER']
+ ':' + os.environ['JOURNAL_PASS']
+ '@journal.sachachua.com/api/entries/' + zid)
j = resp.json()
if j andnot re.search('^I thought about', j['Note']):
return j['Note']
defget_color_map(filename, text=None):
if text:
together = filename + ' ' + text
else:
together = filename
if re.search('r#(parenting|purple|life)', together):
return {'9d9d9d': '8754a1', 'c9c9c9': 'e4c1d9'} # parenting is purplishelif re.search(r'#(emacs|geek|tech|blue)', together):
return {'9d9d9d': '2b64a9', 'c9c9c9': 'b3e3f1'} # geeky stuff in light/dark blueelse:
return {'9d9d9d': '884636', 'c9c9c9': 'f6f396'} # yellow highlighter, dark browndefrename_set(old_name, new_name):
if old_name != new_name:
old_json = old_name[:-3] + 'json'new_json = new_name[:-3] + 'json' os.rename(old_name, new_name)
os.rename(old_json, new_json)
defrecolor_based_on_filename(filename):
color_map = get_color_map(filename)
recolored = recolor.map_colors(filename, color_map)
# possibly rename based on the filenamenew_filename = re.sub(' #(purple|blue)', '', filename)
rename_set(filename, new_filename)
recolored.save(new_filename)
defmove_processed_sketch(file):
global refresh_counter
if'#private'infile:
output_dir = private_sketch_dir
elif'#'infile:
output_dir = public_sketch_dir
refresh_counter = 3
else:
returnfilenew_filename = os.path.join(output_dir, os.path.basename(file))
rename_set(file, new_filename)
return new_filename
defprocess_file(file):
json_file = file[:-3] + 'json'# Check if a corresponding json file already existsifnot os.path.exists(json_file):
extract_text(client, file)
ifnot re.search('[0-9]{4}-[0-9]{2}-[0-9]{2}-[0-9]{2} ', file):
file = maybe_rename(file)
recolor_based_on_filename(file)
move_processed_sketch(file)
defprocess_dir(folder_path):
global processed_files
# Iterate through all png files in the specified folderfiles = sorted(os.listdir(folder_path))
forfilein files:
iffile.endswith('.png') and'_'infile:
print("Processing ", file)
process_file(os.path.join(folder_path, file))
defdaemon(folder_path, wait):
global refresh_counter
whileTrue:
process_dir(folder_path)
time.sleep(wait)
if refresh_counter > 0:
refresh_counter = refresh_counter - 1
if refresh_counter == 0:
print("Reloading sketches")
requests.get('https://' + os.environ['JOURNAL_USER'] + ':'+ os.environ['JOURNAL_PASS']
+ '@sketches.sachachua.com/reload?python=1')
if__name__ == '__main__':
# Create a set to store the names of processed filesprocessed_files = set()
iflen(sys.argv) > 1:
if os.path.isdir(sys.argv[1]):
folder_path = sys.argv[1]
daemon(folder_path, 300)
else:
for f in sys.argv[1:]:
process_file(f)
else:
daemon(folder_path, 300)
I'm contemplating writing some annotation tools to make it easier to
turn the detected text into useful text for searching or writing about
because the sketches throw off the recognition (misrecognized text,
low confidence) and the columns mess up the line wrapping. Low priority, though.
My handwriting (at least for numbers) is probably simple enough that I
might be able to train Tesseract OCR to process that someday. And who
knows, maybe some organization will release a pre-trained model for
offline handwriting recognition that'll be as useful as OpenAI Whisper
is for audio files. That would be neat!
[2023-01-04 Wed] Added a screenshot showing annotation.
I was thinking about how to prepare for my next 10-year review, since
I'll turn 40 this year. I've been writing yearly reviews with some
regularity and monthly reviews sporadically, and I figured it would be
nice to have those posts in an EPUB so that I can read them on my
e-reader and annotate them as I do my review.
I use the 11ty static site generator to publish my blog as HTML files,
since I currently can't keep more than Emacs Lisp, Javascript, and
Python in my brain. (No Hugo or Jekyll for me at the moment.) I
briefly thought about getting 11ty to create that archive for me, but
I realized it might be easier to just write it as an external script
instead of trying to figure out how to get 11ty to export one thing
conditionally.
One of the things I've configured 11ty to make is a JSON file that
includes all of my posts with dates, titles, permalinks, and categories. It
was easy to then parse this list and filter it to get the posts I
wanted. I parsed the HTML out of the _site directory that 11ty
produces instead of fetching the pages from my webserver. I got the
images from my webserver, though, and I made a local cache and rewrote
the URLs. That way, the EPUB conversion could include the images.
This created an archive.html with my posts, using the images/
directory for the images. Then I used my shell script for converting
and copying files to convert it to EPUB and copy it over.
On the SuperNote, I can highlight text by drawing square brackets
around it. If I tap that text, I can write or draw underneath it.
Here's what that looks like:
Figure 1: Writing an annotation
These notes are collected into a "Digest" view, and I can export
things from there. (Example: archive.pdf)
Figure 2: Here's what that digest is like when exported.
(Hmm, maybe I should ask them about hiding the pencil icon…)
Anyway, I think that might be a good starting point for my review.
The SuperNote lets me draw with black, dark gray (0x9d), gray
(0xc9), or white. I
wanted to make it easy to recolor them, since a little splash of
colour makes sketches more fun and also makes them easier to pick out
from thumbnails. Here's the Python script I wrote:
#!/usr/bin/python3# Recolor PNGs## (c) 2022 Sacha Chua (sacha@sachachua.com) - MIT License## Permission is hereby granted, free of charge, to any person# obtaining a copy of this software and associated documentation files# (the "Software"), to deal in the Software without restriction,# including without limitation the rights to use, copy, modify, merge,# publish, distribute, sublicense, and/or sell copies of the Software,# and to permit persons to whom the Software is furnished to do so,# subject to the following conditions:# The above copyright notice and this permission notice shall be# included in all copies or substantial portions of the Software.# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS# BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN# ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE# SOFTWARE.import numpy as np
import os
import csv
import argparse
from PIL import Image
DARK_GRAY= 0x9d
GRAY= 0xc9
HEADER_GRAY= 0xca
WHITE= 0xfe
color_dict= {}
defcolor_to_tuple(color_dict, s):
if s in color_dict:
s= color_dict[s]
s= s.lstrip('#')
if (s =='.'):
return (None, None, None)
elif (len(s) == 2):
return (int(s, 16), int(s, 16), int(s, 16))
else:
returntuple(int(s[i:i + 2], 16) for i in (0, 2, 4))
defload_color_dict(filename):
dict= {}
withopen(os.path.expanduser(filename), newline='') as csvfile:
reader= csv.reader(csvfile, delimiter=',', quotechar='"')
for row in reader:
dict[row[0]] = row[1]
returndictdefremove_grid(input):
ifisinstance(input, str):
im= Image.open(input).convert('RGB')
else:
im=inputdata= np.array(im)
freq= get_colors_by_freq(input)
print(freq)
return Image.fromarray(data)
defmap_colors(input, color_map):
ifisinstance(input, str):
im= Image.open(input).convert('RGB')
else:
im=inputdata= np.array(im)
red, green, blue= data[:, :, 0], data[:, :, 1], data[:, :, 2]
for from_c, to_c in color_map.items():
from_r, from_g, from_b= color_to_tuple(color_dict, from_c)
to_r, to_g, to_b= color_to_tuple(color_dict, to_c)
mask= (red == from_r) & (green == from_g) & (blue == from_b)
data[:, :, :3][mask] = [to_r, to_g, to_b]
return Image.fromarray(data)
defset_colors_by_freq(input, color_list):
ifisinstance(input, str):
im= Image.open(input).convert('RGB')
else:
im=inputdata= np.array(im)
red, green, blue= data[:, :, 0], data[:, :, 1], data[:, :, 2]
sorted_colors= get_colors_by_freq(input)
freq=iter(color_list.split(','))
for i, f inenumerate(freq):
if f !='.':
to_r, to_g, to_b= color_to_tuple(color_dict, f)
by_freq= sorted_colors[i][1]
ifisinstance(by_freq, np.uint8):
mask= (red == by_freq) & (green == by_freq) & (blue == by_freq)
else:
mask= (red == by_freq[0]) & (green == by_freq[1]) & (blue == by_freq[2])
data[:, :, :3][mask] = [to_r, to_b, to_g]
return Image.fromarray(data)
defcolor_string_to_map(s):
color_map= {}
colors=iter(args.colors.split(','))
for from_c in colors:
to_c=next(colors)
color_map[from_c] = to_c
return color_map
defget_colors_by_freq(input):
ifisinstance(input, str):
im= Image.open(input).convert('RGB')
else:
im=inputcolors= im.getcolors(im.size[0] * im.size[1])
returnsorted(colors, key=lambda x: x[0], reverse=True)
defprint_colors(input):
sorted_colors= get_colors_by_freq(input)
for x in sorted_colors:
if x[0] > 10:
ifisinstance(x[1], np.uint8):
print('%02x %d'% (x[1], x[0]))
else:
print(''.join(['%02x'% c for c in x[1]]) +' %d'% x[0])
defprocess_file(input):
print(input)
if args.preview:
output=Noneelse:
output= args.output if args.output elseinputif os.path.isdir(output):
output= os.path.join(output, os.path.basename(input))
im= Image.open(input).convert('RGB')
if args.colors:
im= map_colors(im, color_string_to_map(args.colors))
elif args.freq:
im= set_colors_by_freq(im, args.freq)
else:
print_colors(im)
exit(0)
if args.preview:
im.thumbnail((700, 700))
im.show()
elif output:
im.save(output)
if__name__=='__main__':
parser= argparse.ArgumentParser(
description='Recolor a PNG.',
formatter_class=argparse.RawTextHelpFormatter,
epilog="If neither --colors nor --freq are specified, "+"display the most frequent colours in the image.")
parser.add_argument('--colors', help="""Comma-separated list of RGB hex values in the form of old,new,old,new Examples: 9d,ffaaaa,c9,ffd2d2 - reddish c9,ffea96 - yellow highlighter c9,d2d2ff - light blue """)
parser.add_argument('--freq', help="Color replacements in order of descending frequency (ex: .,ffea96). .: use original color")
parser.add_argument('--csv', help="CSV of color names to use in the form of colorname,hex")
parser.add_argument('--preview', help="Preview only", action='store_const', const=True)
parser.add_argument('input', nargs="+", help="Input file")
parser.add_argument('--output', help="Output file. If not specified, overwrite input file.")
args= parser.parse_args()
color_dict= load_color_dict(args.csv) if args.csv else {}
forinputin args.input:
process_file(os.path.join(os.getcwd(), input))
I don't think in hex colours, so I added a way to refer to colours by
names. I converted this list of Copic CSS colours to a CSV by copying
the text, pasting it into a file, and doing a little replacement. It's
not complete, but I can copy selected colours from this longer list. I
can also add my own. The CSV looks a little like this:
It doesn't do any fuzzing or clustering of similar colours, so it
won't work well on antialiased images. For the simple sketches I make
with the SuperNote, though, it seems to work well enough.
I can preview my changes with something like ./recolor.py ~/sketches/"2022-08-02-01 Playing with my drawing workflow #supernote #drawing #workflow #sketching #kaizen.png" --csv colors.csv --freq .,lightyellow --preview , and then I can
take the --preview flag off to overwrite the PNG.
I've had my SuperNote A5X for a month now, and I really like it.
Text from my sketch
I use it for:
untangling thoughts
sketchnoting books
planning
drafting blog posts
drawing
A- uses it for: (she's 6 years old)
practising cursive
doing mazes and dot-to-dots
drawing
reading lyrics
Things I'm learning:
Exporting PNGs at 200% works well for my workflow. I rename them in
Dropbox and upload them to sketches.sachachua.com.
Carefully copying & deleting pages lets me preserve page numbers. I use lassoed titles for active thoughts and maintain a manual index for other things.
Layouts:
Landscape: only easier to review on my laptop
Portrait columns: lots of scrolling up and down
Portrait rows: a little harder to plan, but easier to review
Many books fit into one page each.
Google Lens does a decent job of converting my handwriting to text (print or cursive, even with a background). Dropbox → Google Photos → Orgzly → Org
Draft blog posts go into new notebooks so that I can delete them once converted.
The Super Note helps me reclaim a lot of the time I spend waiting for A-. A digital notebook is really nice. Easy to erase, rearrange, export… It works well for me.
Part of my everyday carry kit
Ideas for growth:
Settle into monthly pages, bullet journaling techniques
Practise drawing; use larger graphic elements & organizers, different shades
I put my visual book notes and visual library notes into a Dropbox
shared folder so that you can check them out if you have a
Supernote. If you don't have a Supernote, you can find my visual book
notes at sketches.sachachua.com. Enjoy!
A- complains if I get screentime when she doesn't get screentime, so
it's hard to find time to write on my laptop or on my phone. I've
experimented with dictation before, since Google Recorder can make a
half-decent transcript. I'm not used to talking things out, though. I
keep correcting false starts, stutters, and mis-recognized words.
Fortunately, I can write on my A5X while waiting for A-. I get more
space than I do when writing on my phone, so it's easier for me to
think. I can export pages as PNGs, Dropbox, share each page, sync with
with Google Photos, and then use Lens to copy the text. I can then
paste it into Orgzly, which automatically syncs with Syncthing so that
I can edit it on my laptop with Emacs. It needs a little cleanup
(capitalization, stray punctuation, missed words, things in the wrong
order), but editing it feels easier than dealing with the output of
speech recognition, so it seems to be worth the extra time and effort.
Besides, it feels less embarrassing to write at the sandbox than it is
to talk to myself.
I can edit the text directly on my phone, but I still need my laptop
to publish my blog because I haven't set up my static site generator
on my server. Some day! In the meantime, this might be a good workflow
for getting thoughts out there.
What if I want to refer to sketches while I write? Flipping between
pages on the A5X can be challenging if they're not next to each other,
but I can keep my current writing page next to my sketch. I could also
view the sketch on my phone and balance it on the A5X, or use layers
to keep a small version of the sketch as a handy reference. Lots of
ideas to play around with…
W- was happy with his SuperNote A5X, so I ordered one for myself on
July 18. The company was still doing pre-orders because of the
lockdowns in China, but it shipped out on July 20 and arrived on July
25, which was pretty fast.
I noticed that the org-epub export makes verse blocks look
double-spaced on the SuperNote, probably because <br> tags are
getting extra spacing. I couldn't figure out how to fix it with CSS,
so I've been hacking around it by exporting it as a different class
without the <br> tags and just using { white-space: pre }. I also
ended up redoing the templates I made in Inkscape, since the gray I
used was too light to see on the SuperNote.
It was very tempting to dive into the rabbithole of interesting
layouts on /r/supernote and various journaling resources, but I still
don't have much time, so there's no point in getting all fancy about
to-do lists or trackers at the moment. I wanted to focus on just a
couple of things: untangling my thoughts and sketching. Sketchnoting
books would be a nice bonus (and I actually managed to do one on paper
during a recent playdate), but that can also wait until I have more
focused time.
I've had the A5X for five days and I really like it. Writing with the
Lamy pen feels like less work than writing with a pencil or regular
pen. It's smooth but not rubbery. I've still been drawing in landscape
form because that feels a little handier for reviewing on my tablet or
writing about on my blog, but I should probably experiment with
portrait form at some point.
So far, I've:
sketched out my thoughts
I used to use folded-over 8x14" to
sketch out two thoughts, but scanning them was a bit of a pain.
Sometimes I used the backs of our writing practice sheets in order
to reduce paper waste, but then scanning wasn't always as clean. I
really like using the SuperNote to sketch out thoughts like this
one. It's neat, and I can get the note into my archive pretty easily.
sketched stuff from life
This is easier if I take a quick
reference picture on my phone. I could probably even figure out some
kind of workflow for making that available as a template for tracing.
received many kiddo drawings
A- loves being able to use the
eraser and lasso to modify her drawings. Novelty's probably another
key attraction, too. She's made quite a few drawings for me, even
experimenting with drawing faces from the side like the way she's
been seeing me practice doing.
received many kiddo requests
A- likes to ask me to draw things.
She enjoys tracing over them in another layer. More drawing practice
for both of us!
used it to help A- practise coding, etc.
A- wanted to do some
coding puzzles with her favourite characters. I enjoyed being able
to quickly sketch it up, drawing large versions and then scaling
down as needed.
played a game of chess
I drew chess pieces just to see if I
could, and we ended up using those to play chess. I should share
these and maybe add other games as well.
referred to EPUBs and PDFs
I put our favourite songs and poems on
it. I've also started using org-chef to keep a cookbook.
doodled sketch elements
boxes, borders, little icons, people…
Probably should organize these and share them too.
I've figured out how to publish sketches by using my phone to rotate
them and sync them with my online sketches. Now I'm playing around
with my writing workflow to see if I can easily post them to my blog.
At some point, I think I'll experiment with using my phone to record
and automatically transcribe some commentary, which I can pull into
the blog post via some other Emacs Lisp code I've written. Whee!