Category Archives: org

Using Emacs to prepare files for external applications like Autodesk Sketchbook Pro

To make it easier to draw using Autodesk Sketchbook Pro on my laptop (a Lenovo X220 tablet PC), I’ve created several templates with consistent dot grids and sizes. Since I want to minimize typing when I’m drawing, I wrote a couple of functions to make it easier to copy these templates and set up appropriately-named files. That way, I can save them without the grid layer, flip between files using Sketchbook Pro’s next/previous file commands, and then process them all when I’m ready.

Index cards

I’ve been experimenting with a habit of drawing at least five index cards every day. Here’s a function that creates five index cards (or a specified number of them) and then opens the last one for me to edit.

(defvar sacha/autodesk-sketchbook-executable "C:/Program Files/Autodesk/SketchBook Pro 7/SketchBookPro.exe")
(defun sacha/prepare-index-cards (n)
  (interactive (list (or current-prefix-arg 5)))
  (let ((counter 1)
        (directory "~/Dropbox/Inbox")
        (template "c:/data/drawing-templates/custom/0 - index.tif")
        (date (org-read-date nil nil "."))
        temp-file)
    (while (> n 0)
      (setq temp-file
            (expand-file-name (format "%s-%d.tif" date counter)
                              directory))
      (unless (file-exists-p temp-file)
        (copy-file template temp-file)
        (setq n (1- n))
        (if (= n 0)
            (shell-command
             (concat (shell-quote-argument sacha/autodesk-sketchbook-executable)
                     " "
                     (shell-quote-argument temp-file) " &"))))
      (setq counter (1+ counter)))))

Afterwards, I call sacha/rename-scanned-cards function to convert the TIFFs to PNGs, display the files and ask me to rename them properly.

Rename scanned index cards

(defun sacha/rename-scanned-cards ()
  "Display and rename the scanned files."
  (interactive)
  (when (directory-files "~/Dropbox/Inbox" t "^[0-9]+-[0-9]+-[0-9]+-.*.tif")
    ;; Convert the TIFFs first
    (apply 'call-process "mogrify" nil nil nil "-format" "png" "-quality" "1"
           (directory-files "~/Dropbox/Inbox" t "^[0-9]+-[0-9]+-[0-9]+-.*.tif"))
    (mapc (lambda (x)
            (rename-file x "~/Dropbox/Inbox/backup"))
          (directory-files "~/Dropbox/Inbox" t "^[0-9]+-[0-9]+-[0-9]+-.*.tif")))
  (mapc (lambda (filename)
          (find-file filename)
          (delete-other-windows)
          (when (string-match "/\\([0-9]+-[0-9]+-[0-9]+\\)" filename)
            (let ((kill-buffer-query-functions nil)
                  (new-name (read-string "New name: "
                                         (concat (match-string 1 filename) " "))))
              (when (> (length new-name) 0)
                (revert-buffer t t)
                (rename-file filename (concat new-name ".png"))
                (kill-buffer)))))
        (directory-files "~/Dropbox/Inbox" t "^[0-9]+-[0-9]+-[0-9]+-.*.png")))

I might tweak the files a little more after I rename them, so I don’t automatically upload them. When I’m happy with the files, I use a Node script to upload the files to Flickr, move them to my To blog directory, and copy Org-formatted text that I can paste into my learning outline.

Automatically resize images

The image+ package is handy for displaying the images so that they’re scaled to the window size.

(use-package image+
 :load-path "~/elisp/Emacs-imagex"
 :init (progn (imagex-global-sticky-mode) (imagex-auto-adjust-mode)))

Get information for sketched books

For sketchnotes of books, I set up the filename based on properties in my Org Mode tree for that book.

(defun sacha/prepare-sketchnote-file ()
  (interactive)
  (let* ((base-name (org-entry-get-with-inheritance  "BASENAME"))
         (filename (expand-file-name (concat base-name ".tif") "~/dropbox/inbox/")))
    (unless base-name (error "Missing basename property"))
    (if (file-exists-p filename)
        (error "File already exists")
        (copy-file "g:/drawing-templates/custom/0 - base.tif" filename))
      (shell-command (concat (shell-quote-argument sacha/autodesk-sketchbook-executable)
                             (shell-quote-argument filename) " &"))))

By using Emacs Lisp functions to set up files that I’m going to use in an external application, I minimize fussing about with the keyboard while still being able to take advantage of structured information.

Do you work with external applications? Where does it make sense to use Emacs Lisp to make setup or processing easier?

Learn how to take notes more efficiently in Org Mode

How do you take notes in Org? Are you buried in a heap of uncategorized notes? Do you manually open the right file and navigate to the right heading? Are you mystified by org-capture and org-refile? Here’s a path that can help you learn how to more efficiently take (and organize!) notes in Org Mode.

  1. Set up a keyboard shortcut to go to your main Org file
  2. Use org-refile to file or jump to headings
  3. Use org-capture to write notes quickly
  4. Define your own org-capture templates for greater convenience
  5. Pull in additional information

Step 1. Set up a keyboard shortcut to go to your main Org file

Instead of using C-x C-f (find-file) all the time, set up shortcuts to jump to the Org files you use the most. This way, you can easily type that keyboard shortcut, go to the end of the file, and add your note. Here’s some sample code that sets the C-c o shortcut to open organizer.org in your home directory. You can add it to your ~/.emacs.d/init.el and then call M-x eval-buffer to load the changes.

(global-set-key (kbd "C-c o") 
                (lambda () (interactive) (find-file "~/organizer.org")))

Alternatively, you can use registers, which are Emacs data structures that can hold text, file references, and more. The following code sets the o register to organizer.org in your home directory:

(set-register ?o (cons 'file "~/organizer.org"))

You can then jump to it with C-x r j (jump-to-register), specifying o at the prompt.

Once you’re in your Org file, you can use M-> (end-of-buffer) to go to the end of the file, or you can use C-s (isearch-forward) to search for some text.

You’ll still need to switch back to your original buffer or window configuration when you’re done, but that’s something you can fix when you learn how to use org-capture.

Step 2. Use org-refile to file or jump to headings

The next improvement is to use org-refile to move the current subtree to a specified heading, or jump to one without moving any text. This will let you quickly go to a project or task from anywhere in Org Mode.

By default, org-refile will show you only the top-level headings of the current file. Let’s configure it to show you headings from all your agenda files. You can use M-x customize-variable to change org-refile-targets. Click on the INS button, then click on Value menu next to Identify target headline by. Change this to Max level number. In the Integer field, fill in a suitably high number, like 6. This is the maximum depth of headings that will be shown.

If you prefer to set your variables using Emacs Lisp, here’s the code that you can add it to your ~/.emacs.d/init.el. Call M-x eval-buffer to load the changes.

(setq org-refile-targets '((org-agenda-files . (:maxlevel . 6))))

Be sure to add your main Org Mode file to your agenda list. You can do so by going to the file and typing C-c [ (org-agenda-file-to-front), or by setting the org-agenda-files variable.

The standard Emacs completion interface isn’t as friendly as it could be. I use the Helm package to make it easier to select and complete input. Since Helm can be a little complex, you may want to start with ido-mode instead. Here’s how you can set Ido up to use with Org Mode:

(ido-mode)
(setq org-completion-use-ido t)

Once you’ve set up your org-refile-targets, your agenda files, and either Helm or Ido, you can get the hang of using org-refile. The standard keyboard shortcut for org-refile is C-c C-w when you’re in an Org Mode buffer. org-refile can do different things depending on how you call it:

  • By default, it moves the current subtree to the specified location.
  • If you call it with the prefix argument C-u (like so: C-u C-c C-w), it jumps to the specified location instead of moving the current subtree.
  • If you call it as C-u C-u C-c C-w, it jumps to the previous refiling location.

First, practise using it with the prefix argument (C-u C-c C-w) to jump to a location. Once you’ve gotten the hang of that, go to some of your uncategorized entries and use org-refile without the prefix argument (just C-c C-w) to move the entries to the right place.

org-refile gives you a quick way to jump to a heading, but you still have to find your way back to whatever you were working on before you wanted to take a note. After you’re comfortable with refiling notes to the right place, move on to learning how to use org-capture to quickly take notes from anywhere.

Step 3. Use org-capture to write notes quickly

org-capture can help you take notes quickly by popping up a window or leading you through prompts. When you’re done taking the note, it will return you to whatever you were looking at before you started. In order to take advantage of this, though, you’ll need to customize org-capture.

The Org Mode manual recommends giving org-capture a global keyboard shortcut such as C-c c.

(global-set-key (kbd "C-c c") 'org-capture)

You can use M-x customize-variable to set org-default-notes-file to the filename you would like notes to be saved to, or set it in Emacs Lisp code like this:

(setq org-default-notes-file "~/organizer.org")

Make sure that the file exists and is automatically opened in Org Mode.

If you type C-c c, org-capture will display a prompt. t is a simple task template, and C will show you the customization interface for org-capture-templates.

Let’s start with t. It will show you a buffer with a simple TODO entry. You can fill in the rest of the details, use C-c C-s (org-schedule) to schedule it for a particular day, set the deadline with C-c C-d (org-deadline), etc. You can change the TODO keyword or delete it.

When you’re done, type C-c C-c to automatically save it to your default notes file (as specified by org-default-notes-file). Changed your mind? Cancel with C-c C-k. After either C-c C-c or C-c C-k, you should be back to whatever it was that you were working on.

Practise using C-c c (org-capture) to quickly jot down several tasks or notes. Then go to your notes file and use C-c C-w (org-refile) to move the notes to the right place.

You can also refile the notes right from the capture buffer. Instead of typing C-c C-c to finish your note, use C-c C-w to refile it.

Get the hang of using org-capture to take notes, organizing them every so often (maybe at the end of your day, or once a week?) or refiling them as you go.

Step 4. Define your own org-capture-templates for greater convenience

If you find yourself capturing different kinds of notes often or you want to capture in another format (table entry? list item?), invest the time in customizing org-capture-templates. In the beginning, you might find the Customize interface you get from M-x customize-variable org-capture-templates to be easier to work with than setting the values in plain Emacs Lisp, since the Customize interface lists the options. Read the documentation and look at examples of how other people have configured their org-capture-templates for more ideas. I have quite a few templates defined in my config, and http://doc.norang.ca/org-mode.html has a number of templates too.

Step 5. Pull in additional information

org-capture and org-refile are great when you’re at your computer, but what if you’re away? Quite a few people use MobileOrg to take quick notes on the go. I haven’t gotten around to setting that up for my workflow properly; instead, I use Evernote to jot quick notes on my phone. As part of my weekly review process, I look at the notes in my Evernote inbox and copy them into Emacs as needed.

You can manually copy information from your preferred non-Emacs note-taking tools, or you can figure out an automatic way of doing so. For example, I have some code to copy Evernote notes titled “Journal” into an Org Mode file structured by year-month-day.

Tweak your workflow!

Here’s a quick sketch showing some of your workflow options when it comes to capturing and organizing information with Org Mode. Which combination do you prefer, and how could you make it even better?

2015-02-09 Capturing Org Mode notes more efficiently -- index card #emacs #org #capture #refile

2015-02-09 Capturing Org Mode notes more efficiently – index card #emacs #org #capture #refile

Org Mode: Reusing the date from file-datetree-prompt

Update 2015-02-17: Or you can just use %t in your org-capture-templates, as Seth Mason points out in the comments… =)

How can you get Org Mode to create and schedule entries within a year-month-day outline structure? You can define an org-capture-templates with the keyword file+datetree+prompt. This lets you specify a date for your entry, and Org will create the entry in a hierarchy organized by year, month, and day.

If you’d like to display the entry in your agenda, you’ll also need an active timestamp of the form <yyyy-mm-dd>. Fortunately, you can reuse the date you specified at the initial prompt to create the datetree entry. Looking at org-capture.el will show you that the org-capture function refers to the org-read-date-final-answer, which is set to whatever string you entered at the date prompt. For example, if you entered 18, then org-read-date-final-answer will be set to 18. You can use org-read-date to convert this back to a yyyy-mm-dd-style date.

How do you use this in org-capture-templates? You can use the %(...) syntax for calling an Emacs Lisp expression, like so:

(setq org-capture-templates '(
  ;; other entries go here
  ("s" "Journal entry with date, scheduled" entry
   (file+datetree+prompt "~/personal/journal.org")
    "* %^{Title}\n<%(org-read-date nil nil org-read-date-final-answer)>\n%i\n%?\n")))

Here’s sample output from that capture template:

* 2015
** 2015-12 December
*** 2015-12-31 Thursday
**** End of the year party!
<2015-12-31>

Thanks to Sean Miller for the nudge to think about this!

Digital index piles with Emacs: Rapid categorization of Org Mode items

Somewhat daunted by the prospect of categorizing more than a hundred sketches and blog posts for my monthly review, I spent some time figuring out how to create the digital equivalent of sorting index cards into various piles.

2015-02-01 Digital piles of index card -- index card #indexing #organization #pkm

2015-02-01 Digital piles of index cards – index card #indexing #organization #pkm

In fact, wouldn’t it be super-cool if the items could automatically guess which category they should probably go in, prompting me only if it wasn’t clear?

I wanted to write a function that could take a list structured like this:

  • Keyword A
    • Previous links
  • Keyword B
    • Previous links
  • Link 1 with Keyword A
  • Link 2 with Keyword B
  • Link 3 with Keyword A
  • Link 4

It should file Link 1 and 3 under Keyword A, Link 2 under Keyword B, and prompt me for the category for Link 4. At that prompt, I should be able to select Keyword A or Keyword B, or specify a new category.

Inspired by John Kitchin’s recent post on defining a Helm source, I wanted to get it to work with Helm.

First step: I needed to figure out the structure of the list, maybe including a sample from the category to make it clearer what’s included. org-list.el seemed to have useful functions for this. org-list-struct gave me the structure of the current list. Let’s say that a category is anything whose text does not match org-bracket-link-regexp.

(defun sacha/org-get-list-categories ()
  "Return a list of (category indent matching-regexp sample).
List categories are items that don't contain links."
  (let ((list (org-list-struct)) last-category results)
    (save-excursion
      (mapc
       (lambda (x)
         (goto-char (car x))
         (let ((current-item
                (buffer-substring-no-properties
                 (+ (point)
                    (elt x 1)
                    (length (elt x 2)))
                 (line-end-position))))
           (if (string-match
                org-bracket-link-regexp
                (buffer-substring-no-properties
                 (point)
                 (line-end-position)))
               ;; Link - update the last category
               (when last-category
                 (if (< (elt x 1) (elt last-category 1))
                     (setq results
                           (cons (append last-category
                                         (list
                                          (match-string-no-properties
                                           3
                                           (buffer-substring-no-properties
                                            (point)
                                            (line-end-position)))))
                                 (cdr results))))
                 (setq last-category nil))
             ;; Category
             (setq results
                     (cons
                      (setq last-category
                            (list
                             current-item
                             (elt x 1)
                             (concat "^"
                                     (make-string (elt x 1) ?\ )
                                     (regexp-quote
                                      (concat (elt x 2)
                                              current-item))
                                     "$")))
                      results)))))
       list))
    results))

The next step was to write a function that guessed the list category based on the item text, and moved the item there.

(defvar sacha/helm-org-list-candidates nil)
(defun sacha/helm-org-list-categories-init-candidates ()
  "Return a list of categories from this list in a form ready for Helm."
  (setq sacha/helm-org-list-candidates
        (mapcar (lambda (x)
                  (cons (if (elt x 3)
                            (format "%s - %s" (car x) (elt x 3))
                          (car x))
                        x))
                (sacha/org-get-list-categories))))

(defun sacha/org-move-current-item-to-category (category)
  (when category
    (let* ((beg (line-beginning-position))
           (end (line-end-position))
           (string (buffer-substring-no-properties beg end)))
      (save-excursion
        (when (re-search-backward (elt category 2) nil t)
          (delete-region beg (min (1+ end) (point-max)))
          (forward-line 1)
          (insert (make-string (+ 2 (elt category 1)) ?\ )
                  string "\n")))) t))

(defun sacha/org-guess-list-category (&optional categories)
  (interactive)
  (require 'cl-lib)
  (unless categories
    (setq categories
          (sacha/helm-org-list-categories-init-candidates)))
  (let* ((beg (line-beginning-position))
         (end (line-end-position))
         (string (buffer-substring-no-properties beg end))
         (found
          (cl-member string
                     categories
                     :test
                     (lambda (string cat-entry)
                       (string-match (regexp-quote (downcase (car cat-entry)))
                                     string)))))
    (when (car found)
      (sacha/org-move-current-item-to-category
       (cdr (car found)))
      t)))

After that, I wrote a function that used Helm to prompt me for a category in case it couldn’t guess the category. It took me a while to figure out that I needed to use :init instead of :candidates because I wanted to read information from the buffer before Helm kicked in.

(setq sacha/helm-org-list-category-source
      (helm-build-sync-source
          "Non-link categories in the current list"
        :init 'sacha/helm-org-list-categories-init-candidates
        :candidates 'sacha/helm-org-list-candidates
        :action 'sacha/org-move-current-item-to-category
        :fuzzy-match t))

(defun sacha/org-guess-uncategorized ()
  (interactive)
  (sacha/helm-org-list-categories-init-candidates)
  (let (done)
    (while (not done)
      (save-excursion
        (unless (sacha/org-guess-list-category sacha/helm-org-list-candidates)
          (unless
              (helm :sources
                    '(sacha/helm-org-list-category-source
                      sacha/helm-org-list-category-create-source))
            (setq done t))))
      (unless done
        (setq done (not (looking-at "^[-+] \\[")))))))

The :action above refers to this function, which creates a category if it doesn’t exist yet.

(setq sacha/helm-org-list-category-create-source
      (helm-build-dummy-source
          "Create category"
        :action (helm-make-actions
                 "Create category"
                 (lambda (candidate)
                   (save-excursion
                     (let* ((beg (line-beginning-position))
                            (end (line-end-position))
                            (string (buffer-substring beg end)))
                       (delete-region beg (min (1+ end) (point-max)))
                       (org-beginning-of-item-list)
                       (insert "- " candidate "\n  " string "\n")))
                   (sacha/helm-org-list-categories-init-candidates)))))

I’m new to fiddling with Helm, so this implementation is not the best it could be. But it’s nifty and it works the way I want it to, hooray! Now I can generate a list of blog posts and unblogged sketches, categorize them quickly, and then tweak the categorizations afterwards.

2015-02-01 Index card sketches and monthly reviews -- index card #organization #pkm #indexing

2015-02-01 Index card sketches and monthly reviews – index card #organization #pkm #indexing

You can see the results in my January 2015 review, or check my config to see if the code has changed.

My next step for learning more about Helm sources is probably to write a Helm command that creates a montage of selected images. John Kitchin has a post about handling multiple selection in Helm, so I just need to combine that with my code for using Imagemagick to create a montage of images. Whee!

Getting data from Org Mode tables

Org Mode is an amazingly powerful package for Emacs. I’ve been learning a lot about how to use its support for plain-text tables and spreadsheet calculations.

Using table data in Emacs Lisp with the :var argument

For example, I wanted to be able to define my abbreviations in an Org Mode table in my config. I remembered coming across this technique a few weeks ago, but I couldn’t find the webpage with the code. It turned out to be simple to write from scratch. Here’s the plain text I added to my config.

#+NAME: abbrev
| Base  | Expansion                             |
|-------+---------------------------------------|
| bc    | because                               |
| wo    | without                               |
| wi    | with                                  |
| ex    | For example,                          |
| qm    | [email protected]                   |
| qe    | http://sachachua.com/dotemacs         |
| qw    | http://sachachua.com/                 |
| qb    | http://sachachua.com/blog/            |
| qc    | http://sachachua.com/blog/emacs-chat/ |

#+begin_src emacs-lisp :exports code :var data=abbrev
(mapc (lambda (x) (define-global-abbrev (car x) (cadr x))) data)
#+end_src

The :var data=abbrev argument to the Emacs Lisp source block is where all the magic happens. Here, it takes the data from the table named “abbrev” (which I set using #+NAME: before the table) and makes it available to the code. Emacs evaluates that data when the code is tangled (or exported) to my configuration. The code that’s in my Sacha.el looks like this:

(let ((data (quote (("bc" "because")
                    ("wo" "without")
                    ("wi" "with")
                    ("ex" "For example,")
                    ("email" "[email protected]")
                    ("dote" "http://sachachua.com/dotemacs")
                    ("web" "http://sachachua.com/")
                    ("blog" "http://sachachua.com/blog/")
                    ("ec" "http://sachachua.com/blog/emacs-chat/")))))
  (mapc (lambda (x) (define-global-abbrev (car x) (cadr x))) data) )

Looking up data with org-lookup-first, org-lookup-last, and org-lookup-all

You can do more complex things with Org tables, too. Inspired by Eric Boyd’s talk on his Epic Quest of Awesome (which he based on Steve Kamb‘s), I started putting together my own. I made a list of little achievements, guessed at the years, and assigned arbitrary experience points.

The achievements table had rows like this:

Approximate date Category XP Description ID
2014 Life 50 Became a Canadian citizen – link L_CAN
2014 Programming 20 Used NodeJS and AngularJS for a client project – link P_NOD
2014 Programming 5 Pulled information out of Evernote

I wanted to summarize the points by year: points gained, total points, level (according to a lookup table based on D&D experience points), and description. The lookup table was structured like this:

#+TBLNAME: levels
| Total XP | Level | Adjective             |
|----------+-------+-----------------------|
|        0 |     1 | trained-initiate      |
|     1000 |     2 | experienced           |
|     2250 |     3 | savvy                 |
|     3750 |     4 | veteran               |
|     5500 |     5 | unusually experienced |

Now for the summary table. I created rows for different years, and then I used Org Mode to fill in the rest. (Org Mode! Wow.)

| Year | Points gained | Cumulative points | Level | Adjective        |
|------+---------------+-------------------+-------+------------------|
| 1997 |             0 |                 0 |     1 | trained-initiate |
| 1998 |            10 |                10 |     1 | trained-initiate |
| 1999 |            50 |                60 |     1 | trained-initiate |
| 2000 |            50 |               110 |     1 | trained-initiate |
| 2001 |           100 |               210 |     1 | trained-initiate |
| 2002 |            60 |               270 |     1 | trained-initiate |
| 2003 |           245 |               515 |     1 | trained-initiate |
| 2004 |           115 |               630 |     1 | trained-initiate |
| 2005 |           140 |               770 |     1 | trained-initiate |
| 2006 |            60 |               830 |     1 | trained-initiate |
| 2007 |           270 |              1100 |     2 | experienced      |
| 2008 |           290 |              1390 |     2 | experienced      |
| 2009 |           205 |              1595 |     2 | experienced      |
| 2010 |           215 |              1810 |     2 | experienced      |
| 2011 |           115 |              1925 |     2 | experienced      |
| 2012 |           355 |              2280 |     3 | savvy            |
| 2013 |           290 |              2570 |     3 | savvy            |
| 2014 |           350 |              2920 |     3 | savvy            |
| 2015 |            45 |              2965 |     3 | savvy            |
#+TBLFM: $2='(calc-eval (format "vsum(%s)" (vconcat (org-lookup-all $1 '(remote(accomplishments,@2$1..@>$1)) '(remote(accomplishments,@2$3..@>$3))))))::$3=vsum(@2$2..@+0$2)::$4='(org-lookup-last $3 '(remote(levels,@2$1..@>$1)) '(remote(levels,@2$2..@>$2)) '>=);N::$5='(org-lookup-last $3 '(remote(levels,@2$1..@>$1)) '(remote(levels,@2$3..@>$3)) '>=);L

The TBLFM (table formula) line is very long, so let me break it down.

Points gained:

(calc-eval
 (format "vsum(%s)"
         (vconcat
          (org-lookup-all
           $1
           '(remote(accomplishments,@2$1..@>$1))
           '(remote(accomplishments,@2$3..@>$3))))))

This uses org-lookup-all to look up the value of the first column ($1) in the accomplishments table, from the second row to the last row @2..@>, looking in the first column ($1). It returns the values from the third column of the matching rows ($3). This is then passed through calc’s vsum function to calculate the sum.

Cumulative points: vsum(@2$2..@+0$2) is the sum of the second column $2 from the second row @2 to the current row @+0.

Level: This uses org-lookup-last to find the last value where the operator function returns true. In this case, testing the level from column $3 against each of the values in the levels table’s column $1 while the given level is greater than or equal to the value from levels. When it finds the last matching row, it returns the $2 second column from it. ;N means treat everything as a number.

org-lookup-first is like org-lookup-last, but it returns the first matching row.

Adjective: This one works like Level does, but it returns the value from column $3 instead. I found that it converted the return values to 0 if I used ;N, so I used ;L instead.

Passing data to R or other external programs

Of course, you’re not limited to things that Emacs can do. I wanted to summarize the data in graphs, so here’s what I did.

#+RESULTS: category_analysis

#+name: category_analysis
#+begin_src R :var data=accomplishments :exports both :results graphics :file quest_category.png :height 300
library(plyr)
library(ggplot2)
categories <- ddply(data, c("Category"), summarize, Points=sum(XP))
cat_sorted <- transform(categories, Category=reorder(Category, Points))
plot <- ggplot(data = cat_sorted, aes(x = Category, y = Points))
plot <- plot + geom_bar(stat="identity")
plot <- plot + geom_text(aes(label = Points, x = Category, y = Points + 10, hjust=0))
plot <- plot + scale_y_continuous(expand=c(0,70))
plot <- plot + coord_flip()
print(plot)
#+end_src

I like including source code in published pages for fellow geeks, but having the results come first gives people more context for the source block. So I named the source block using the #+name: directive and defined a #+RESULTS: directive before it. The source block used the :var argument to bring the data in from the accomplishments table. With R blocks, the data becomes available as a data frame that you can then do interesting things with. I used the :file argument to save the output to quest_category.png.

Those are a few ways that you can get data out of Org Mode tables and into Emacs Lisp, other Org Mode tables, or external programs. As I learn more about Org Mode, I find myself using it for more of the things that I used to use Microsoft Excel for – tracking, analyzing, and even graphing. I found it a little difficult to piece together what I needed to do from the manuals and examples on the Web, so I hope this explanation will help you (and that it’ll help me when I forget the syntax, as I’m sure I will). If you come up with something really neat that uses Org Mode tables, tell me what you’ve figured out!

Thinking about how to make better use of Yasnippet in my Emacs workflow

One of the awesome things that Karl Voit demonstrated in this Emacs Chat was how he used YASnippet and Org Mode to quickly create projects with several related tasks, such as when organizing a group to attend an event. He selected the snippet and filled in different fields like the artist name and the event date, and Emacs generated all these sub-tasks and e-mail templates with the information already filled in.

I’ve used YASnippet once or twice, but mostly I’ve been using org-capture and org-capture-templates instead. YASnippet looks like it might be more flexible because you can fill in fields in a non-linear order and you can re-evaluate Emacs Lisp expressions as you type.

Lots of people do cool things with YASnippet. For example, it’s popular for programming because it lets people quickly expand short sequences into longer syntax. Check out this Emacs Rocks episode on YASnippet to get a sense of what it can do. (Note: YASnippet has changed its naming convention slightly, so things like yas/text have been replaced with yas-text.) People have used it for e-mail templates and to fill in metadata for blog posts.

I’d like to use YASnippet more. Where can I integrate it into my workflow? Probably wherever checklists and templates make sense. I’ve been thinking about checklists and templates as a way to improve how I do things.

Checklists are good for making sure that you complete tasks more consistently, not missing any important steps. You can work faster when there’s a guide, since you don’t have to keep thinking of the next step each time. The simple act of checking things off can encourage you to put in more effort, since the list shows you your progress. It also makes it easier to remember to follow up.

Templates help you improve the structure of your work. You can make sure you cover all the important parts. If you use similar structures for many things, then people get used to finding information in the same logical places. This doesn’t mean that you’re stuck with cookie-cutter formats. You can still adapt the format to your needs.

I’m particularly interested in using checklists and templates to improve in three areas:

  • Programming: I’d like to write with less friction and use best practices like testing
  • Helping the Emacs community: Checklists can help me make sure I do all the steps to prepare for and make the most of Emacs Hangouts and Emacs Chats. They might also lower the intimidation factor so that I end up scheduling these more often.
  • Writing: I think checklists and templates will help me invest more time into developing thoughts, relationships, and structure.

Programming

As mentioned, YASnippet’s popular for programming. You can take advantage of existing collections of snippets for different programming modes (ex: AndreaCrotti’s collection), or you can define your own.

I’d like to get better at developing single-page applications using AngularJS, Twitter Bootstrap / Zurb Foundation, and NodeJS. YASnippet might let me quickly put together short applications and test suites. If I get my workflow smooth enough, I might even be able to do an app-a-week (or app-a-day) sprint for deliberate practice. There are often lots of fiddly little syntax or keyword things that I look up while writing code. While practising typing those things in again and again will help me memorize them, there’s also some value in automating that part with snippets so that I can focus on the core skills of designing and implementing small web applications.

YASnippet might also be able to help me use Org Mode to keep track of ideas for features or small web applications throughout the implementation process. I wonder if I should implement this using lots of subtasks or using TODO states with logs. TODO states might be easier to filter or visualize with the kanban package for Org Mode. Maybe I’ll try both approaches. In any case, checklists will help me remember to think about designs and tests before implementing the code, and maybe I can keep track of deployment notes, lessons learned, and follow-up tasks.

Emacs Community

I find checklists to be really helpful when setting up live videocasts. I’m usually too frazzled to think of all the steps I need to do at the last minute. Paper checklists are good because I can refer to them while keeping my screen ready to be recorded. Still, an Org Mode-based checklist (possibly with dynamic date fields and e-mail templates provided by YASnippet) might go a long way towards standardizing the before- and after-event process, and that might in turn reduce the friction enough for me to do more of them. Both Emacs Chats and Emacs Hangouts seem to be popular, so it would be good to get more of these on the go.

The process would be something like:

  1. Reach out to the person who’s going to be featured on the Emacs Chat, or at least one other person who’s willing to be there for the Emacs Hangout (so that I don’t end up talking to myself for the first ten minutes, which is Awkward)
  2. Figure out what will be discussed (for Emacs Chats)
  3. Set up a time, considering timezones
  4. Set up the Google+ event page
  5. Update the Google Calendar
  6. Post a notice on Twitter and on my blog (I’ve been forgetting to do this step)
  7. On the day of the event
    1. Do the last-minute push (I’ve been forgetting to do this as well)
    2. Create the Google Hangout on Air
    3. Set it up for Q&A
    4. Invite the other person in for Emacs Chats, or post the URL for Emacs Hangouts
    5. Host the video chat
    6. Remind people where the recording can be found
    7. Update the Google+ page with the link to the next thing
  8. Extract the MP3 from the video, change the properties, and upload it to archive.org
  9. Post a blog post with the embedded video, podcast audio, and quick notes
  10. Transcribe the video or pay for transcription
  11. Edit the transcript
  12. Update the post with the transcript
  13. Update the Google+ event page with the link to the transcript, post to social networks (I’ve been forgetting this)
  14. Update EmacsLife.com, too (yet another thing I’ve been forgetting)

I think it would be totally awesome to get to the point where I can call an Emacs Lisp function that would Do The Right Thing at that point, like posting to Twitter or using something like org-trello to make a Trello card and assign it to the person who does my transcriptions.

Writing

I’m getting the hang of using outlines to write (and I should post a video about this soon), but it might be even cooler if I can get the hang of writing with more structure. For example, Michael Hyatt posted this blog post checklist that he had been using with Evernote. I like it because:

  • The template asks you to be explicit about the post’s objective and subject.
  • It encourages you to add more illustrations, links, and stories.
  • It reminds you to take steps that you might otherwise skip, and you might spend several days revising the post.

I might not use it for every post, but it’s good to flesh out some ideas further, especially the ones where I think I’m onto something particularly interesting.

It would be even cooler if I could take advantage of YASnippet’s dynamic Emacs Lisp evaluation to remind me of relevant links from my blog post outline given the category. I remember playing around with the Remembrance Agent, which monitored a few hundred words around your cursor and brought up files that had similar words. Matching on category isn’t going to be anywhere as sophisticated, but it still might be a good way to refresh my memory. Even if I had a quick Emacs Lisp interactive function that read whatever category property I’d set (chosen from the org-refile-able targets) and displayed the section from my blog post index in another window, I think that would be a pretty neat start.

I tend to draft posts within my sharing outline (which I sporadically publish at http://pages.sachachua.com/sharing/). When I’m done, I delete the subtree, sometimes replacing it with a link to the post to help me follow up on it in the future. This means losing that metadata, though. It might be interesting to keep the metadata so that I can review the goals and backstory of a blog post.

YASnippet can also help me keep track of the TODOs related to a post as well. For example, I might want to come up with sketches, tweet links, or follow up on ideas. If I use a YASnippet to plan my blog post in the first place, then I can create a TODO (possibly with a link back to the blog post) that I could leave in place or refile to the appropriate location in my regular Org Mode files.

I don’t think YASnippet dynamic fields persist after the file is saved and reloaded, though. How would that work if I need to change things? Maybe I can use multiple-cursors to mark all the matching text in the subtree, or do other clever things with it…

Next steps

Okay. It looks like setting up YASnippet for Emacs checklists would probably give me the quickest win. Programming is also pretty straightforward. Writing might be interesting too, if I can get the hang of working with that kind of structure. Let’s see how that goes. Once I figure out what those snippets will be like, I’ll post them on Github somewhere. =)

Is YASnippet part of your workflow? Have any thoughts, suggestions, or neat stories?