Here's a super quick demo of what I've been hacking together so far, doing vector search on some of my blog posts using the .org files I indexed with org-db-v3:

A few weeks ago, John Kitchin demonstrated a vector search server in his video Emacs RAG with LibSQL - Enabling semantic search of org-mode headings with Claude Code - YouTube. I checked out jkitchin/emacs-rag-libsql and got the server running. My system's a little slow (no GPU), so (setq emacs-rag-http-timeout nil) was helpful. It feels like a lighter-weight version of Khoj (which also supports Org Mode files) and maybe more focused on Org than jwiegley/rag-client. At the moment, I'm more interested in embeddings and vector/hybrid search than generating summaries or using a conversational interface, so something simple is fine. I just want a list of possibly-related items that I can re-read myself.

Of course, while these notes were languishing in my draft file, John Kitchin had already moved on to something else. He posted Fulltext, semantic text and image search in Emacs - YouTube, linking to a new vibe-coded project called org-db-v3 that promises to offer semantic, full-text, image, and headline search. The interface is ever so slightly different: POST instead of GET, a different data structure for results. Fortunately, it was easy enough to adapt my code. I just needed a small adapter function to make the output of org-db-v3 look like the output from emacs-rag-search.

(use-package org-db-v3
  :load-path "~/vendor/org-db-v3/elisp"
  :init
  (setq org-db-v3-auto-enable nil))

(defun my-org-db-v3-to-emacs-rag-search (query &optional limit filename-pattern)
  "Search org-db-v3 and transform the data to look like emacs-rag-search's output."
  (org-db-v3-ensure-server)
  (setq limit (or limit 100))
  (mapcar (lambda (o)
            `((source_path . ,(assoc-default 'filename o))
              (line_number . ,(assoc-default 'begin_line o))))
          (assoc-default 'results
                         (plz 'post (concat (org-db-v3-server-url) "/api/search/semantic")
                           :headers '(("Content-Type" . "application/json"))
                           :body (json-encode `((query . ,query)
                                                (limit . ,limit)
                                                (filename_pattern . ,filename-pattern)))
                           :as #'json-read))))

I'm assuming that org-db-v3 is what John's going to focus on instead of emacs-rag-search (for now, at least). I'll focus on that for the rest of this post, although I'll include some of the emacs-rag-search stuff just in case.

Both emacs-rag and org-db-v3 index Org files by submitting them to a local web server. Here are the key files I want to index:

• organizer.org: my personal projects and reference notes
• reading.org: snippets from books and webpages
• resources.org: bookmarks and frequently-linked sites
• posts.org: draft posts

(dolist (file '("~/sync/orgzly/organizer.org"
                "~/sync/orgzly/posts.org"
                "~/sync/orgzly/reading.org"
                "~/sync/orgzly/resources.org"))
  (org-db-v3-index-file-async file))

(emacs-rag uses emacs-rag-index-file instead.)

Then I figured I'd index my recent blog posts, except for the ones that are mostly lists of links, like Emacs News or my weekly/monthly/yearly reviews. I write my posts in Org Mode before exporting them with ox-11ty and converting them with the 11ty static site generator. I'd previously written some code to automatically export a copy of my Org draft in case people wanted to look at the source of a blog post, or in case I wanted to tweak the post in the future. (Handy for things like Org Babel.) This was generally exported as an index.org file in the post's directory. I can think of a few uses for a list of these files, so I'll make a function for it.

(defun my-blog-org-files-except-reviews (after-date)
  "Return a list of recent .org files except for Emacs News and weekly/monthly/yearly reviews.
AFTER-DATE is in the form yyyy, yyyy-mm, or yyyy-mm-dd."
  (setq after-date (or after-date "2020"))
  (let ((after-month (substring after-date 0 7))
        (posts (my-blog-posts)))
    (seq-keep
     (lambda (filename)
       (when (not (string-match "[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]-emacs-news" filename))
         (when (string-match "/blog/\\([0-9]+\\)/\\([0-9]+\\)/" filename)
           (let ((month (match-string 2 filename))
                 (year (match-string 1 filename)))
             (unless (string> after-month
                              (concat year "-" month))
               (let ((info (my-blog-post-info-for-url (replace-regexp-in-string "~/proj/static-blog\\|index\\.org$\\|\\.org$" "" filename) posts)))
                 (let-alist info

                   (when (and
                          info
                          (string> .date after-date)
                          (not (seq-intersection .categories
                                                 '("emacs-news" "weekly" "monthly" "yearly")
                                                 'string=)))
                     filename))))))))
     (sort
      (directory-files-recursively "~/proj/static-blog/blog" "\\.org$")
      :lessp #'string<
      :reverse t))))

This is in the Listing exported Org posts section of my config. I have a my-blog-post-info-for-url function that helps me look up the categories. I get the data out of the JSON that has all of my blog posts in it.

Then it's easy to index those files:

(mapc #'org-db-v3-index-file-async (my-blog-org-files-except-reviews))

Now that my files are indexed, I want to be able to turn up things that might be related to whatever I'm currently writing about. This might help me build up thoughts better, especially if a long time has passed in between posts.

org-db-v3-semantic-search-ivy didn't quite work for me out of the box, but I'd written an Consult-based interface for emacs-rag-search-vector that was easy to adapt. This is how I put it together.

First I started by looking at emacs-rag-search-vector. That shows the full chunks, which feels a little unwieldy.

Instead, I wanted to see the years and titles of the blog posts as a quick summary, with the ability to page through them for a quick preview. consult.el lets me define a custom completion command with that behavior. Here's the code:

(defun my-blog-similar-link (link)
  "Vector-search blog posts using `emacs-rag-search' and insert a link.
If called with \\[universal-argument\], use the current post's text.
If a region is selected, use that as the default QUERY.
HIDE-INITIAL means hide the initial query, which is handy if the query is very long."
  (interactive (list
                (if embark--command
                    (read-string "Link: ")
                  (my-blog-similar
                   (cond
                    (current-prefix-arg (my-11ty-post-text))
                    ((region-active-p)
                     (buffer-substring (region-beginning)
                                       (region-end))))
                   current-prefix-arg))))
  (my-embark-blog-insert-link link))

(defun my-embark-blog--inject-target-url (&rest args)
  "Replace the completion text with the URL."
  (delete-minibuffer-contents)
  (insert (my-blog-url (get-text-property 0 'consult--candidate (plist-get args :target)))))

(with-eval-after-load 'embark
  (add-to-list 'embark-target-injection-hooks '(my-blog-similar-link my-embark-blog--inject-target-url)))

(defun my-blog-similar (&optional query hide-initial)
  "Vector-search blog posts using `emacs-rag-search' and present results via Consult.
If called with \\[universal-argument\], use the current post's text.
If a region is selected, use that as the default QUERY.
HIDE-INITIAL means hide the initial query, which is handy if the query is very long."
  (interactive (list (cond
                      (current-prefix-arg (my-11ty-post-text))
                      ((region-active-p)
                       (buffer-substring (region-beginning)
                                         (region-end))))
                     current-prefix-arg))
  (consult--read
   (if hide-initial
       (my-org-db-v3-blog-post--collection query)
     (consult--dynamic-collection
         #'my-org-db-v3-blog-post--collection
       :min-input 3 :debounce 1))
   :lookup #'consult--lookup-cdr
   :prompt "Search blog posts (approx): "
   :category 'my-blog
   :sort nil
   :require-match t
   :state (my-blog-post--state)
   :initial (unless hide-initial query)))

(defvar my-blog-semantic-search-source 'org-db-v3)
(defun my-org-db-v3-blog-post--collection (input)
  "Perform the RAG search and format the results for Consult.
Returns a list of cons cells (DISPLAY-STRING . PLIST)."
  (let ((posts (my-blog-posts)))
    (mapcar (lambda (o)
              (my-blog-format-for-completion
               (append o
                       (my-blog-post-info-for-url (alist-get 'source_path o)
                                                  posts))))
            (seq-uniq
               (my-org-db-v3-to-emacs-rag-search input 100 "%static-blog%")
               (lambda (a b) (string= (alist-get 'source_path a)
                                      (alist-get 'source_path b)))))))

It uses some functions I defined in other parts of my config:

• Making it easier to add a category to a blog post
  • my-embark-blog-insert-link
  • my-blog-format-for-completion
• Tooting a link to the current post
  • my-11ty-post-text

When I explored emacs-rag-search, I also tried hybrid search (vector + full text). At first, I got "database disk image is malformed". I fixed this by dumping the SQLite3 database. Using hybrid search, I tended to get less-relevant results based on the repetition of common words, though, so that might be something for future exploration. Anyway, my-emacs-rag-search and my-emacs-rag-search-hybrid are in the emacs-rag-search part of my config just in case.

Along the way, I contributed some notes to consult.el's README.org so that it'll be easier to figure this stuff out in the future. In particular, it took me a while to figure out how to use :lookup #'consult--lookup-cdr to get richer information after selecting a completion candidate, and also how to use consult--dynamic-collection to work with slower dynamic sources.

It is kinda nice being able to look up posts without using the exact words.

Now I can display a list of blog posts that are somewhat similar to what I'm currently working on. It should be pretty straightforward to filter the list to show only posts I haven't linked to yet.

I can probably get this to index the text versions of my sketches, too.

It might also be interesting to have a multi-source Consult command that starts off with fast sources (exact title or headline match) and then adds the slower sources (Google web search, semantic blog post search via org-db-v3) as the results become available.

I'll save that for another post, though!

Building on Visual vocabulary practice - ABCs, I decided to make a regular grid that I could then automatically split up into individual images. I used Emacs's svg.el to generate the grid. I started with 4 rows of 7 boxes to match the alphabet example, but I realized that using 5 rows of 7 boxes each would let me reuse the grid for a monthly calendar. I numbered the boxes to make it easier to double-check if the lists line up, but I can write over the numbers for things like dates since the background won't be exported.

I used convert icon-grid.svg icon-grid.png to make it from the SVG produced by the following code.

(require 'svg)
(defvar my-dot-grid-boxes-params
  '(:num-rows 5
    :num-cols 7
    :dot-size 3
    :line-width 3
    :dot-spacing 60
    :grid-color "#a6d2ff"
    :row-size 6
    :col-size 6
    :text-size 50
    :margin-top 2))
(cl-defun my-dot-grid-boxes-template (&key (num-rows 5)
                                           (num-cols 7)
                                           (dot-size 3)
                                           (line-width 3)
                                           (dot-spacing 60)
                                           (grid-color "#a6d2ff")
                                           (row-size 6)
                                           (col-size 6)
                                           (text-size 50)
                                           (margin-top 2))
  "Prepare an SVG with a dot grid within a table with solid gridlines.
Each dot is a solid circle of DOT-SIZE filled with GRID-COLOR spaced DOT-SPACING apart.
The gridlines are also GRID-COLOR. They should divide the image into ROWS and COLUMNS, which are ROW-SIZE * DOT-SPACING and COL-SIZE * DOT-SPACING apart.
The table has a top margin with the dot grid, and this is MARGIN-TOP * DOT-SPACING tall.
All dots are centered on their x, y coordinates.
The rest of the image's background is white."
  (let* ((width (* num-cols col-size dot-spacing))
         (height (* dot-spacing (+ margin-top (* num-rows row-size))))
         (margin-top-height (* margin-top dot-spacing))
         (svg (svg-create width height)))
    (dotimes (row (+ (* num-rows row-size) margin-top))
      (dotimes (col (1+ (* num-cols col-size)))
        (let ((x (* col dot-spacing))
              (y (* row dot-spacing)))
          (svg-circle svg x y dot-size
                      :fill-color grid-color
                      :stroke-width 0))))
    (when (> text-size 0)
      (dotimes (i (* num-rows num-cols))
        (let ((x (* (% i num-cols) col-size dot-spacing))
              (y (+ margin-top-height (* (/ i num-cols) row-size dot-spacing))))
          (svg-text svg
                    (number-to-string (1+ i))
                    :x x :y (+ y text-size)
                    :fill-color grid-color
                    :font-size text-size
                    :stroke-width 0))))
    (dotimes (col (1+ num-cols))
      (let ((x (* col col-size dot-spacing)))
        (svg-line svg x margin-top-height x height
                  :stroke-color grid-color
                  :stroke-width line-width)))
    (dotimes (row (1+ num-rows))
      (let ((y (+ margin-top-height (* row row-size dot-spacing))))
        (svg-line svg 0 y width y
                  :stroke-color grid-color
                  :stroke-width line-width)))
    svg))

With that function defined, I can make a template with:

(with-temp-file "~/Dropbox/sketches/icon-grid.svg"
  (svg-print
   (my-dot-grid-boxes-template)))

I used that template to draw a bunch of little doodles. The Noteful app I use on my iPad makes it easy to import a template and then export my drawings without including the template.

(If this blog post is out of date, you can check the Dot-grid box templates section in my config for my current code.)

I've done this sort of thing before, when I made a font based on my handwriting. That time, I used Python to generate the template with sample characters, and I used Python again to cut the exported image into individual glyphs.

Once I imported the template into Noteful, it was easy to draw using fragments of time. 35 boxes are a lot, but each icon was just a few minutes of drawing, and I enjoyed seeing the progress.

Because I kept all my doodles within the template's boxes, it was easy to split up the images into individual files. First, I needed the text for all the labels. Sometimes I typed this in manually, and sometimes I used Google Cloud Vision to extract the text (editing it a little bit to put it in the right order and fix misrecognized text). Then I used Emacs Lisp to read the labels from the text file, calculate the coordinates, and use ImageMagick to extract that portion of the image into a file. I used filenames based on the label of the individual icon and the ID of the image it came from.

(cl-defun my-dot-grid-boxes-list (&key (num-rows 5)
                                       (num-cols 7)
                                       (dot-spacing 60)
                                       (row-size 6)
                                       (col-size 6)
                                       (text-bottom 1)
                                       (margin-top 2)
                                       filename
                                       &allow-other-keys)
  "Return a list of boxes."
  (let* ((margin-top-height (* margin-top dot-spacing))
         (max-image-size nil)
         (size (image-size (create-image filename nil nil :scale 1) t))
         (ratio (/ (car size) (* num-cols col-size dot-spacing 1.0)))
         results)
    (message "Expected adjusted height %f actual height %f"
             (* (+ margin-top (* num-rows row-size)) dot-spacing ratio)
             (cdr size))
    (dotimes (i (* num-rows num-cols))
      (let* ((r (/ i num-cols))
             (c (% i num-cols))
             (y (* (+ margin-top-height (* r col-size dot-spacing)) ratio))
             (x (* c row-size dot-spacing ratio))
             (width (* col-size dot-spacing ratio))
             (height (* (- row-size text-bottom) dot-spacing ratio)))
        (setq results (cons
                       `((r . ,r)
                         (c . ,c)
                         (i . ,i)
                         (x . ,(floor x))
                         (y . ,(floor y))
                         (w . ,(floor width))
                         (h . ,(floor height))
                         (x2 . ,(floor (+ x width)))
                         (y2 . ,(floor (+ y height))))
                       results))))
    (nreverse results)))

(defvar my-sketch-icon-directory "~/sync/sketches/icons")
(cl-defun my-dot-grid-boxes-extract (&rest args &key filename labels
                                           (output-dir my-sketch-icon-directory) force &allow-other-keys)
  (let* ((list (apply #'my-dot-grid-boxes-list args))
         (base (file-name-base filename))
         (ext (concat "." (file-name-extension filename)))
         (id
          (if (string-match "^[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]-[0-9][0-9]" base)
              (match-string 0 base)
            ""))
         results
         args)
    (dolist (icon list)
      (let-alist icon
        (let ((new-filename (expand-file-name
                             (concat (my-make-slug (elt labels .i)) "--"
                                     id
                                     (format "-%d-%d"
                                             .r .c)
                                     ext)
                             output-dir)))
          (push `((term . ,(elt labels .i))
                  (icon . ,(file-name-nondirectory new-filename))
                  (source . ,(file-name-nondirectory filename)))
                results)
          (when (or force (not (file-exists-p new-filename)))
            (setq args
                  (list (expand-file-name filename)
                        "-crop"
                        (format "%dx%d+%d+%d" .w .h .x .y)
                        "+repage"
                        new-filename))
            (message "%s" (concat "convert " (mapconcat #'shell-quote-argument args " ")))
            (apply #'call-process "convert" nil nil nil args)))))
    (nreverse results)))

(defun my-dot-grid-boxes-labels (id)
  (with-temp-buffer
    (insert-file-contents (concat (file-name-sans-extension (my-get-sketch-filename id)) ".txt"))
    (goto-char (point-min))
    (re-search-forward "^ *$")
    (split-string (string-trim (buffer-substring (point) (point-max))) "\n")))

I really liked being able to write code to extract and name images all in one go. If you don't want to dive into Emacs Lisp, though, you can slice up a large image into small ones using ImageMagick.

Thinking ahead: if I use a similar process for my daily drawings, I can extract an "on this day" slice like the one I have for blog posts and sketches (blog post about it).

I had worked on a similar visual vocabulary project in 2013, but I had made it as a shared notebook in Evernote. That's gone now, and I can't remember if I backed it up or where I would've saved a backup to. Ah well, no harm in starting again, with files under my control.

Now that I'd broken down the images into labelled components, I wanted to be able to quickly look up icons from a web browser; my own version of The Noun Project. First, I exported the label information into a JSON.

(defun my-sketch-icon-update-index (list)
  (let (data
        (index-file (expand-file-name "index.json" my-sketch-icon-directory)))
    (with-temp-file index-file
      (setq data
            (if (file-exists-p index-file)
                (json-read-file index-file)
              '()))
      (dolist (entry list)
        ;; Remove current entry
        (setq data (seq-remove (lambda (o)
                                 (and (string-match (regexp-quote (alist-get 'source o)) (alist-get 'source entry))
                                      (string= (alist-get 'term o) (alist-get 'term entry))))
                               data))
        ;; Add a new entry
        (push
         `((term . ,(alist-get 'term entry))
           (icon . ,(alist-get 'icon entry))
           (source . ,(alist-get 'source entry)))
         data))
      (insert (json-encode (sort data :key (lambda (o) (alist-get 'term o)) :lessp #'string<))))))

(defun my-dot-grid-boxes-process (id &optional force)
  (interactive
   (list
    (my-complete-sketch-filename "drawing")
    current-prefix-arg))
  (let* ((labels (my-dot-grid-boxes-labels id))
         list)
    (cl-assert (= (% (length labels) 7) 0))
    (cl-assert (> (length labels) 1))
    (setq list
          (my-dot-grid-boxes-extract :output-dir my-sketch-icon-directory
                                     :num-rows (/ (length labels) 7)
                                     :filename (my-get-sketch-filename id)
                                     :labels labels
                                     :force force))
    (my-sketch-icon-update-index list)))

(defun my-dot-grid-boxes-process-all-icons ()
  (interactive)
  (dolist (source (my-sketches "icons")) (my-dot-grid-boxes-process source)))

Then I made a simple interface for looking up icons.

I can filter it by terms, and I can exclude the icons I've copied from illustration books for practice.

I can even use it as a rudimentary visual menu for showing A+ some choices.

Oatmeal, cereal, pancakes

My curves are shaky. I'm mostly learning to ignore that and draw anyway. Good thing redoing them is a matter of a two-finger tap with my left hand, and then I can redraw until it feels mostly right. I try up to three times before I say, fine, let's just go with that.

I often draw with my iPad balanced on my lap, so there's an inherent wobbliness to it. I think this is a reasonable trade-off. Then I can keep drawing cross-legged in the shade at the playground instead of sitting at the table in the sun. The shakiness is still there when I draw on a solid table, though. I have a Paperlike screen protector, which I like more than the slippery feel of the bare iPad screen. That helps a little.

It's possible to cover it up and pretend to confidence that I can't draw with. I could smooth out the shakiness of my curves by switching to Procreate, which has more stylus sensitivity settings than Noteful does. A+ loves the way Procreate converts her curves to arcs. She moves the endpoints around to where she wanted to put them. I'm tempted to do the same, but I see her sometimes get frustrated when she tries to draw without that feature, and I want to show her the possibilities that come with embracing imperfection. It's okay for these sketches to be a little shaky. These are small and quick. They don't have to be polished.

The Internet says to draw faster and with a looser grip, and that lots of practice will build fine motor skills. I'm not sure I'll get that much smoother. I think of my mom and her Parkinson's tremors, and I know that time doesn't necessarily bring improvement. But it's better to keep trying than to shy away from it. Maybe as I relax more into having my own time, working on my own things and moving past getting things done, I'll give myself more time for drawing exercise, like filling pages with just lines and circles.

I had fun coming up with words and drawing them. I could start with whatever was in front of me and go from there. I used my phone to look up the occasional reference image, like the heart. Sometimes A+ suggested things to draw. Sometimes she even took over.

The books were handy when I didn't feel like thinking much. I could just reproduce the already-simplified drawings. I often felt like I still wanted to tweak things a bit more to make them feel like my own, though, which was a useful way to figure out more about what I like.

Instead of mimicking other people's sketches, I can mine my sketchnotes and pull out the concepts I tend to think about a lot. If I've drawn them in Noteful, I can even copy them from their original sketches, resize them, and make the lines a consistent thickness. If I've drawn them elsewhere, it's easy enough to redraw.

Ellane W's mustache post reminded me that Inktober has just started, so that's another source of ideas. Zsolt's 2021 post on sketchnoting for PKM led me to Quick, Draw!, which has 345 simple drawing prompts I can try. There's also a TU Berlin dataset with 250 drawing prompts. SketchDaily Reference Site could be good for randomness and inspiration, too.

Building a visual library is a great way to learn how to actually draw things. I'm curious about using this 30-minute drawing exercise to start paying attention to a few things, and maybe using the shrimp method if there's something I really want to nail down. Visual mnemonic links might be a way to explore the connections between things as I wander around ideas (even though this video is way more advanced than I am).

I think I'll keep drawing these visual vocabulary practice sketches, focusing more on my own ways of drawing. It's fun. I have 324 icons at the moment. I wonder what the collection will be like when I have a thousand terms in it.

On the Emacs side, it might be interesting to quickly add a related doodle to the margin of a blog post, or to look up or copy a personal reference image as I untangle my thoughts in a sketch. I'm tempted to write some Emacs Lisp that searches for these terms in my draft blog posts and adds a little hint whenever it finds a match. Another small piece of code might identify recurring nouns and verbs in recent posts and suggest those if I haven't drawn them yet. Could be fun.

Check out my icon library if you're curious!