Categories: geek » emacs » org

RSS - Atom - Subscribe via email

Having fun kerning using Org Mode and FontForge

| emacs, org

It turns out that working with font bearings and kerning tables using Org Mode makes lots of things so much easier.

Bearings in the top left, kerning matrix in the top right

While trying to figure out kerning, I came across this issue that described how you sometimes need a character-pair kern table instead of just class-based kerning. Since I had figured out character-based kerning before I figured out class-based kerning, it was easy to restore my Python code that takes the same kerning matrix and generates character pairs. Here’s what that code looks like.

def kern_by_char(font, kerning_matrix):
  # Add kerning by character as backup
  font.addLookupSubtable("kern", "kern-2")
  offsets = np.asarray(kerning_matrix)
  classes_right = [None if (x == "" or x == "None") else x.split(",") for x in offsets[0,1:]]
  classes_left = [None if (x == "" or x == "None") else x.split(',') for x in offsets[1:,0]]
  for r, row in enumerate(classes_left):
    if row is None: continue
    for first_letter in row:
      g = font.createMappedChar(first_letter)
      for c, column in enumerate(classes_right):
        if column is None: continue
        for second_letter in column:
          if kerning_matrix[r + 1][c + 1]:
            g.addPosSub("kern-2", second_letter, 0, 0, kerning_matrix[r + 1][c + 1], 0, 0, 0, 0, 0)
  return font

I wanted to be able to easily compare different versions of my font: my original glyphs versus my tweaked glyphs, simple spacing versus kerned. This was a hassle with FontForge, since I had to open different font files in different Metrics windows. If I execute a little bit of source code in my Org Mode, though, I can use my test web page to view all the different versions. By arranging my Emacs windows a certain way and adding :eval no to the Org Babel blocks I’m not currently using, I can easily change the relevant table entries and evaluate the whole buffer to regenerate the font versions, including exports to OTF and WOFF. Here’s the code for that:

font = fontforge.font()
font = import_glyphs(font, params)
font = set_bearings(font, bearings)
save_font(font, {**params, "new_otf": "sachacHandRaw.otf"})
font = kern_classes(font, kerning_matrix)
font = kern_by_char(font, kerning_matrix)
save_font(font, {**params, "new_otf": "sachacHandRawKerned.otf"})
font = load_font('SachaHandEdited.sfd')
font = set_bearings(font, bearings)
save_font(font, {**params, "new_otf": "sachacHandEdited.otf"})
font = kern_classes(font, kerning_matrix)
font = kern_by_char(font, kerning_matrix)
save_font(font, {**params, "new_otf": "sachacHand.otf"})

I also like the way it’s pretty easy to update multiple kerning values without clicking around. I sometimes use FontForge to get the number to set it to and then copy that into my table, but I also sometimes just tweak the number in Org Mode directly.

To see the results, I can generate a test HTML that shows me text with different versions of my font. I can also look at lots of kerning pairs at the same time. Here are the components of that test page:

def test_css(fonts):
  doc, tag, text, line = Doc().ttl()
  with tag('style'):
    for f in fonts:
      text("@font-face { font-family: '%s'; src: url('%s'); }" % (f[0], f[1]))
      text(".%s { font-family: '%s'; }" % (f[0], f[0]))
    text("table { font-size: inherit; font-weight: inherit }")
    text("td { text-align: left }")
    text(".blog-heading { font-weight: bold; font-size: 32px }")
    text(".default { color: gray }")
    text("body { font-family: woff, Arial, sans-serif; font-size: 32px; padding: 10px }")
  return doc.getvalue()
def test_html(strings):
  doc, tag, text, line = Doc().ttl()
  with doc.tag('table', style='border-bottom: 1px solid gray; width: 100%; border-collapse: collapse'):
    for s in strings:
      for i, f in enumerate(fonts):
        style = 'border-top: 1px solid gray' if (i == 0) else ""
        with tag('tr', klass=f[0], style=style):
          line('td', f[0])
          line('td', s)
  return doc.getvalue()
def test_kerning_matrix(kerning_matrix):
  doc, tag, text, line = Doc().ttl()
  with tag('table'):
    for r, row in enumerate(classes_left):
      if row is None: continue
      for first_letter in row:
        with tag('tr'):
          line('td', first_letter)
          for c, column in enumerate(classes_right):
            if column is None: continue
            for second_letter in column:
              klass = "kerned" if kerning_matrix[r + 1][c + 1] else "default"
              line('td', aglfn.to_glyph(first_letter) + aglfn.to_glyph(second_letter), klass=klass)
  return doc.getvalue()

This code actually generates the test file:

from yattag import Doc
import numpy as np
import aglfn
doc, tag, text, line = Doc().ttl()
fonts = [['raw', 'sachacHandRaw.otf'],
         ['raw-kerned', 'sachacHandRawKerned.otf'],
         ['edited', 'sachacHandEdited.otf'],
         ['woff', 'sachacHand.woff']]
strings = ["Python+FontForge+Org: I made a font based on my handwriting!",
           "Monthly review: May 2020",
           "Emacs News 2020-06-01"]
offsets = np.asarray(kerning_matrix)
classes_left = [None if (x == "" or x == "None") else x.split(',') for x in offsets[1:,0]]
classes_right = [None if (x == "" or x == "None") else x.split(",") for x in offsets[0,1:]]
with tag('html'):
  with tag('head'): 
  with tag('body'):
    line('h1', 'Test headings')
    with tag('div', klass="blog-heading"):
    line('h1', 'Kerning matrix')
return doc.getvalue()

And here’s what that test.html looks like:

Testing the font with a few blog headings
Kerning pairs: black for specified pairs, gray for defaults

Not bad… Now Emacs is my font editor! The code is at .

View or add comments

Python+FontForge+Org: I made a font based on my handwriting!

| emacs, geek, org

I wanted to make a font based on my handwriting using only free software. It turns out that FontForge can be scripted with Python. I know just a little about Python and even less about typography, but I managed to hack together something that worked for me. If you’re reading this on my blog at , you’ll probably see the new font being used on the blog post titles. Whee!

My rough notes are at . I wanted to write it as a literate program using Org Babel blocks. It’s not really fully reproducible yet, but it might be a handy starting point. The basic workflow was:

  1. Generate a template using other fonts as the base.
  2. Import the template into Medibang Paint on my phone and draw letters on a different layer. (I almost forgot the letter q, so I had to add it at the last minute.)
  3. Export just the layer with my writing.
  4. Cut the image into separate glyphs using Python and autotrace each one.
  5. Import each glyph into FontForge as an SVG and a PNG.
  6. Set the left side and right side bearing, overriding as needed based on a table.
  7. Figure out kerning classes.
  8. Hand-tweak the contours and kerning.
  9. Use sfnt2woff to export the web font file for use on my blog, and modify the stylesheet to include it.

I really liked being able to specify kerning classes through an Org Mode table like this:

  None o,a,c,e,d,g,q,w f,t,x,v,y,z h,b,l,i,k j m,n,p,r,u s T zero
None 0 0 0 0 0 0 0 0 0
f 0 -102 -61 -30 0 -60 0 -120 -70
t 0 -70 -41 -25 0 0 0 -120 -10
r 0 -82 -41 -25 0 -20 0 -120 29
k 0 -50 -81 -20 0 -20 -48 -120 -79
l 0 -41 -50 0 0 0 0 -120 -52
v 0 -40 -35 -30 0 0 0 -120 30
b,o,p 0 -20 -80 0 0 0 0 -120 43
a 0 -23 -60 0 0 0 0 -120 7
W 0 -40 -30 -20 0 0 0 -120 17
T 0 -190 -120 -60 0 -130 0 0 -188
F 0 -100 -90 -60 0 -70 -100 -40 -166
two 0 0 0 0 0 0 0 0 -53

I had a hard time defining classes using the FontForge interface because I occasionally ended up clearing my glyph selection, so it was great being able to just edit my columns and rows.

Clearly my kerning is still very rough–no actual values for j, for example–but it’s a start. Also, I can probably figure out how to combine this with character pair kerning and have two tables for easier tweaking.

A- insisted on tracing my handwriting template a few times, so I might actually be able to go through the same process to convert her handwriting into a font. Whee!

View or add comments

Python, Org Mode, and writing Org tables to CSVs so that I can read them back

| emacs, org, writing

I’ve been getting deeper into Python so that I can model our personal finances. I really like using the pandas library to manipulate data. All those years I spent trying to juggle increasing complex spreadsheets… Working with Python code in Org Babel blocks is just so much more fun. I like being able to keep my assumptions in tables without having to fuss around with naming cells for easy-to-read formulas, slice and summarize parts of my data frames, organize my notes in outlines and add commentary, and define more complicated functions that I don’t have to squeeze into a single line.

I haven’t quite been able to tempt W- into the world of Org Babel Python blocks. Still, I don’t want to give up the awesomeness of having pretty tables that I can easily edit and use. So I have a bunch of named tables (using #+NAME:), and some code that exports my tables to CSVs:

#+NAME: tables
| Table         | Key                 |
| assets_w      | Description         |
| assets_s      | Description         |
| tax_rates     |                     |
| disposition   | Asset               |
| probate_rates | Asset               |
| basic         | Client information  |
| base_expenses | Category            |
| general       | General assumptions |

#+begin_src emacs-lisp :results silent :var tables=tables :tangle no
  (defun my-tbl-export (row)
    "Search for table named `NAME` and export."
    (interactive "s")
      (goto-char (point-min))
      (let ((case-fold-search t))
        (when (search-forward-regexp (concat "#\\+NAME: +" (car row)) nil t)
          (org-table-export (format "%s.csv" (car row)) "orgtbl-to-csv")))))
  (mapc 'my-tbl-export tables)

and some code that imports them back in, and formats tables nicely if I’m displaying them in Org. The in_org block doesn’t get tangled into, so I don’t clutter command-line use with Org table markup.

#+begin_src python :results silent :tangle no

#+begin_src python :results silent :exports code
  import pandas as pd
  import numpy as np
  import orgbabelhelper as ob
  def out(df, **kwargs):
    if 'in_org' in globals():
      print(ob.dataframe_to_orgtable(df, **kwargs))
    return df

#+begin_src python :results silent :var tables=tables :colnames yes
  for row in tables:
    table = row[0]
    index = row[1] 
    if row[1] == '':
      index = None
    globals()[table] = pd.read_csv(table + '.csv', index_col=index).apply(pd.to_numeric, errors='ignore')
    # print(globals()[table])

Then I can use C-c C-v C-b (org-babel-execute-buffer) to update everything if I change the table in my Org file, and I can use C-c C-v C-t (org-babel-tangle) to create an that W- can read through or run without needing Org.

View or add comments

Turning an Org Mode outline into an HTML table with a column for more notes

| emacs, org

The last time A- had a babysitter, I quickly scribbled down some ideas for activities that A- might enjoy. The babysitter found the list very helpful, so I figured I’d make a list that I could easily update as A-‘s interests change. I wanted a two-column table with space for notes, but I didn’t want to fiddle with LibreOffice or manually writing HTML. Org Mode to the rescue! To turn something like this:

* Activity ideas  :noexport:
** Gross motor
*** Balance on one leg
*** Coast on the balance bike
** Fine motor
*** Cut along a curve or shape

into a table like this:

Two-column table

I wrote this bit of code inside a #+begin_src emacs-lisp :exports results :results value html#+end_src block:

(let* ((elements (org-element-parse-buffer))
          "Activity ideas"
          `(:parse-tree ,elements)))))
   "<table><tr><th width=\"50%%\">Activity ideas</th><th width=\"50%%\">Notes</th></tr>%s</table>"
    (lambda (section)
      (format "<tr><td><h2>%s</h2><ul>%s</ul></td><td></td></tr>"
              (org-element-property :raw-value section)
              (mapconcat (lambda (idea)
                           (format "<li>%s</li>"
                                   (org-element-property :raw-value idea)))
                         (org-element-contents section)
    (org-element-contents activity-ideas)

I can then export the buffer to HTML and get my nice neat table by itself, since the data comes from a subtree with the :noexport: tag. The code parses the elements in the buffer, looks at the children under the “Activity ideas” header, turns each child into a table row, and turns the children of those nodes into list items.

The code coincidentally takes advantage of the org-export-resolve-link I wrote to help someone with an Org feature request. Yay for multipurpose code!

For the next step, I’ll probably put TODO state filtering in so that I can keep a long list of activities and then select a few for each category. It’s nice to have an outline I can easily update!

View or add comments

Making a numpad-based hydra for categorizing Org list items

| emacs, org

I like to categorize links for Emacs News so that it’s not an overwhelmingly long wall of text. After I’ve deleted duplicate links, there are around a hundred links left to categorize. I used to use Helm and some custom code to simplify moving Org list items into different categories in the same list. Then I can type “org” to move something to the Org Mode category and “dev” to move something to the Emacs development category.

When I don’t have a lot of computer time, I usually do this categorization by SSHing into my server from my phone. It’s hard to type on my phone, though. I thought a numpad-based Hydra might be better for quick entry, like a phone system. I wanted to be able to use the numeric keypad to sort items into the most common categories, with a few shortcuts for making it easier to organize. Here’s a list of shortcuts:

0-9 Select options from the list, with 0 for other.
\/ Opens the URL in a web browser
, Selects a category by name, creating as needed
\* Shows the URL in the messages buffer, and toggles it
Deletes the item
. Quits

Here’s what the Helm version looked like on the left, and here’s the new numpad-powered one on the right. I liked how fewer buttons made it easier to hit the right one when I’m sorting on my phone. I can add new categories with completion. Because I assigned numbers to specific categories instead of having them automatically calculated based on the headings in the list, it was easy to get into the rhythm of tapping 6 for Org, 7 for coding, and so on.

Comparison of Helm and Hydra approaches

Here’s the code that makes it happen. I experimented with dynamically defining a hydra using eval and defmacro so that I could more easily define the menu in a variable. It seems to work fine so far.

(defvar my/org-categorize-emacs-news-menu 
  '(("0" . "Other")
    ("1" . "Emacs Lisp")
    ("2" . "Emacs development")
    ("3" . "Emacs configuration")
    ("4" . "Appearance")
    ("5" . "Navigation")
    ("6" . "Org Mode")
    ("7" . "Coding")
    ("8" . "Community")
    ("9" . "Spacemacs")))

(defun my/org-move-current-item-to-category (category)
  "Move current list item under CATEGORY earlier in the list.
CATEGORY can be a string or a list of the form (text indent regexp).
Point should be on the next line to process, even if a new category
has been inserted."
  (interactive "MCategory: ")
  (when category
    (let* ((beg (line-beginning-position))
           (end (line-end-position))
           (string (org-trim (buffer-substring-no-properties beg end)))
           (category-text (if (stringp category) category (elt category 0)))
           (category-indent (if (stringp category) 2 (+ 2 (elt category 1))))
           (category-regexp (if (stringp category) category (elt category 2)))
           (pos (point))
      (delete-region beg (min (1+ end) (point-max)))
      (unless (string= category-text "x")
        (if (re-search-backward category-regexp nil t)
            (forward-line 1)
          (setq s (concat "- " category-text "\n"))
          (insert s)
          (setq pos (+ (length s) pos)))
        (insert (make-string category-indent ?\ )
                string "\n")
        (goto-char (+ pos (length string) category-indent 1))

 `(defhydra my/org-categorize-emacs-news (global-map "C-c e")
       (lambda (x)
         `(,(car x)
           (lambda () (interactive) (my/org-move-current-item-to-category ,(concat (cdr x) ":")))
           ,(cdr x)))
     (lambda () (interactive)
        (completing-read (match-string 2) (my/org-get-list-categories))))
     "By string")
    ("/" (lambda () (interactive)
             (re-search-forward org-link-bracket-re)
     (lambda () (interactive)
       (if (string= (buffer-name) "*Messages*")
           (re-search-forward org-link-bracket-re)
           (message (match-string 1)))
         (switch-to-buffer "*Messages*")))
     "Show URL")
    ("-" kill-whole-line "Kill")
    ("." nil "Done")))

Let’s see how it holds up next week!

View or add comments

Adding :target option for the TOC keyword in Org Mode

Posted: - Modified: | emacs, org

Now that A- can be counted on to happily play with a babysitter for several hours once a week, I’ve decided to alternate consulting and personal projects. Two weeks ago, I used my personal time to make a script that renewed my library books automatically. This week, I set aside time to look at Org Mode. DC had asked me to update the patch I made to allow people to specify a target for the table of contents, and I was curious about whether I could hack something together.

Patch for adding :target to TOC keyword

Here’s a sample file that shows what I mean:

#+OPTIONS: toc:nil
* Not this section
** Heading X
** Heading Y
* Target
  :CUSTOM_ID: TargetSection
** Heading A
** Heading B
* Another section
#+TOC: headlines 1 :target "Target"

Here’s the core of how to make it work for HTML exports:

(defun org-html-keyword (keyword _contents info)
  "Transcode a KEYWORD element from Org to HTML.
CONTENTS is nil.  INFO is a plist holding contextual information."
  (let ((key (org-element-property :key keyword))
  (value (org-element-property :value keyword)))
     ((string= key "HTML") value)
     ((string= key "TOC")
      (let ((case-fold-search t))
   ((string-match "\\<headlines\\>" value)
    (let ((depth (and (string-match "\\<[0-9]+\\>" value)
          (string-to-number (match-string 0 value))))
      ;; link
      ((string-match ":target +\"\\([^\"]+\\)\"" value)
       (let ((link (with-temp-buffer
           (insert (org-make-link-string (match-string 1 value))))
         (pcase (org-element-property :type link)
           ((or "custom-id" "id") (org-export-resolve-id-link link info))
           ("fuzzy" (org-export-resolve-fuzzy-link link info))
           (_ nil))))
      ;; local
      ((string-match-p "\\<local\\>" value) keyword))))
      (org-html-toc depth info scope)))
   ((string= "listings" value) (org-html-list-of-listings info))
   ((string= "tables" value) (org-html-list-of-tables info))))))))

It was a lot of fun Doing the Right Thing(s): writing documentation, adding tests, and making it work for more than just HTML export. I found out where to make the changes by using grep to search for TOC in the Org Mode source code. All the heavy lifting was already done by org-export-collect-headlines, so it was just a matter of passing the right scope. It took me a while to figure out that I needed to pass an Org link element. An easy way of making that element work for both fuzzy and ID-specific links was to insert the target text into a temporary buffer (remembering to use org-make-link-string) and then calling org-element-link-parser.

I tried figuring out how to make it work with a link to another file, but I didn’t get very far, so I figured I’d just wrap things up nicely there.

I wasn’t sure if my original post made it through because I sent it through Gmane and Cc:d DC, who got it with an empty To:, so I ended up submitting it twice. I just realized I forgot to add test-ox-ascii.el. I don’t want to spam the list, so I’ll send that along with other changes if people have feedback.

But look! Open source contributions! I’m so excited. I wonder what I’ll get to do in two weeks from now. =)

View or add comments

Labeling toy storage bins with photos and text using ImageMagick and org-babel

| emacs, geek, org, organization, parenting, play

I wanted to make labels for A-‘s new toy storage: three low Trofast drawer frames all along the wall.

I liked how early childhood drop-in centres labeled their shelves with both pictures and text. That way, kids can find things before she can read, while still being exposed to print. I took pictures of the bin contents and renamed the files to the labels I wanted to put on them, such as 2x2 blocks.jpg. (We have a lot of Duplo.)

This time, I experimented with creating the labels entirely in Imagemagick instead of using LaTeX. First, I used a table in Org Mode to let me easily play with the dimensions and recalculate pixel sizes.

DPI   300
Columns 3  
Rows 5  
Paper width 14 4200
Paper height 8.5 2550
Minimum margins 0.5 150
Label width 4.3333333 1300
Label length 1.5 450

I passed the width and the height to the following code block by using header arguments. I liked using 400 pixels as the height instead of 450, so that’s what I used. My source image size was 4032×3024 pixels. If I resize them to a height of 400, that gives me a width of 533. Allowing 20 pixels for the text left and right borders gives me (- 1300 533 20 20) = 727 as the text width.

#+begin_src sh :dir ~/code/labels :var width=1300 :var textwidth=727 :var height=400 :var pointsize=72 :results silent
for x in source/*; do
  file=$(basename "$x")
  /usr/local/bin/convert \( \( "source/$file" -resize x${height} \) \
     \( -background white -fill black -font Alegreya-Regular -bordercolor White \
         -gravity West -border 20 -pointsize $pointsize -size ${textwidth}x caption:"${file%.*}" \) \
     +append \) \
     -extent ${width}x${height} \
     \( -fill none -stroke gray -draw "rectangle 0 0 $(echo $width - 1 | bc) $(echo $height - 1 | bc)" \) \

Sample resized label:

I moved the ones I wanted from the out directory to a ready directory and combined the ones I wanted to print into a PDF:

#+begin_src sh :dir ~/code/labels :results silent
montage ready/*.png -tile 3x5 -background none -geometry +0+0 print.png
convert print*.png -density 300 -quality 100 print.pdf

Then I printed the labels in colour on an 8.5×14″ sheet of paper (single-sided, landscape), cut them out, and taped them onto the bins with packing tape.

W- suggested taking macro shots that more clearly show the characteristics of things in the bins instead of just aiming down and taking pictures of the contents. Might be a good excuse to show A- basic product photography when we get back.

W- also recommended making the label text bigger. The first time I did it, I just picked a pointsize based on whatever fit the ones I wanted to print. I decided against letting Imagemagick maximize the font size because I didn’t want labels to have very different text sizes. After a little poking around, I figured out how to use caption: instead of label: to give me text that can neatly wrap within a given space, and that will probably let me use 90-point font instead of 72-point font. That will make the next iteration of labels even easier to read.

It’s nice having all these bins. A- is getting pretty good at heading straight for the bin she wants something from, and she even talks about them: “Horse is in animals bin.” I’m glad we labeled the most frequently used bins. I’ll tweak the labels when we get back from our trip. We’ll probably change some of the bin contents anyway.

Hooray for ImageMagick, and hooray for variables in org-babel blocks!

View or add comments