Category Archives: emacs

On this page:

Making my Emacs-related blog posts available for offline reading

Deepak Tripathi wanted to know how to download all of my Emacs-related posts for offline reading. It makes sense to put together something like that. Xah Lee even charges for an organized ZIP copy of his site (and he’s put together a lot of resources). I like putting together free/pay-what-you-want things, so I figured I’d add my blog posts to my git repository of Emacs notes.

My blog runs on WordPress. It has a whole bunch of other posts in it, so a straightforward Jekyll import wouldn’t do the trick. I had previously modified my WordPress theme to add a special ?dump=1 parameter for single post pages so that I could use it to archive pages. The first thing I needed to do was to come up with a list of Emacs-related blog posts.

Fortunately, http://sachachua.com/blog/emacs already lists all the pages in the Emacs category. I copied the HTML source for the list and started tinkering with the text. It was a good excuse to try the visual-regexp package – the vr/replace command made working with match groups much easier. Eventually I ended up with a list that looked like this:

wget -p http://sachachua.com/blog/2014/04/thinking-todo-keywords/?dump=1
wget -p http://sachachua.com/blog/2014/04/reflecting-10-episodes-emacs-chats/?dump=1
wget -p http://sachachua.com/blog/2014/04/org-mode-helps-deal-ever-growing-backlog/?dump=1
wget -p http://sachachua.com/blog/2014/04/reinvesting-time-and-money-into-emacs/?dump=1
...

This downloaded the posts and included images. Next, I wanted to process the downloaded HTML pages and turn them into Org files, since I’m more familiar with Org than with Markdown. Pandoc to the rescue! I took the output of find -name \*.html\* and processed it with lots more vr/replace

dos2unix ./sachachua.com/blog/2003/07/emacspeak-creator/index.html?dump=1; (echo "<html><body>"; cat ./sachachua.com/blog/2003/07/emacspeak-creator/index.html?dump=1; echo "</body></html>") > test.html; pandoc test.html -o 2003-07-emacspeak-creator.org
dos2unix ./sachachua.com/blog/2003/07/instructionalsoftwaredesign/index.html?dump=1; (echo "<html><body>"; cat ./sachachua.com/blog/2003/07/instructionalsoftwaredesign/index.html?dump=1; echo "</body></html>") > test.html; pandoc test.html -o 2003-07-instructionalsoftwaredesign.org
dos2unix ./sachachua.com/blog/2003/07/interesting-mail-stats/index.html?dump=1; (echo "<html><body>"; cat ./sachachua.com/blog/2003/07/interesting-mail-stats/index.html?dump=1; echo "</body></html>") > test.html; pandoc test.html -o 2003-07-interesting-mail-stats.org
dos2unix ./sachachua.com/blog/2003/08/emacs-macros/index.html?dump=1; (echo "<html><body>"; cat ./sachachua.com/blog/2003/08/emacs-macros/index.html?dump=1; echo "</body></html>") > test.html; pandoc test.html -o 2003-08-emacs-macros.org

This is a pretty long command line because I didn’t bother with writing a shell script to process files. When I tried running pandoc on the HTML snippets, it choked because the file didn’t have html or body tags, so I ended up adding them with echo before processing them with pandoc.

Anyway, now I had a lot of Org files. I wanted to rename them and clean up the titles. I used this totally hackish bit of code to process all the files in the directory, changing the header to a title and adding the day to the filename (just in case I want to move to Jekyll someday).

(defun sacha/process-blog-posts ()
  (interactive)
  (mapcar (lambda (x)
            (find-file x)
            ;; Set the title
            (goto-char (point-min))
            (when (looking-at "\\(\\*\\*\\)")
              (replace-match "#+TITLE:"))
            (save-buffer)
            ;; Rename the file
            (when (re-search-forward "\\([0-9]+\\)\\(th\\|nd\\|st\\|rd\\), \\([0-9]+\\)")
              (let ((day (match-string 1)))
                (rename-file
                 (buffer-file-name)
                 (replace-regexp-in-string
                  "/\\([0-9]+\\)-\\([0-9]+\\)-"
                  (concat "/processed\\&" day "-")
                  (buffer-file-name)))))
            (kill-buffer))
          (directory-files "." nil ".org$")))

I also wanted to change the image references to use the images that wget had downloaded into my uploads directory. This time I used wgrep, which is awesome. The wgrep package makes grep results editable if you use wgrep-change-to-wgrep-mode (bound to C-c C-p by default, but you can change wgrep-enable-key). If you combine that with vr/replace from visual-regexp or some keyboard macros, you can edit a whole lot of things quickly. Save the changes with C-x C-s (wgrep-finish-edit), and you’re done!

So now I had a bunch of Org files that were in reasonable shape. I wanted to regenerate the HTML pages so that people could browse them even if they weren’t familiar with Org. I set up the relevant org-publish-project-alist entries in a build-site.el that a Makefile could use, and I published the project.

Going from HTML to Org to HTML meant losing some information and lots of my blog posts need reviewing, but it’s a good start. Since I draft many of my Emacs-related blog posts in Org anyway, I can copy the Org source into my emacs-notes repository going forward.

Anyway, there you have it for your grepping pleasure! https://github.com/sachac/emacs-notes/

2048 in Emacs, and colours too

While browsing through M-x list-packages, I noticed that there was a new MELPA package that implemented the 2048 game in Emacs. I wrote the following code to colorize it. Haven’t tested the higher numbers yet, but they’re easy enough to tweak if the colours disagree with your theme. =)

2014-04-16 23_27_25-emacs@SACHA-X220

(defface 2048-2-face '((t (:foreground "red"))) "Face used for 2" :group '2048-game)
(defface 2048-4-face '((t (:foreground "orange"))) "Face used for 4" :group '2048-game)
(defface 2048-8-face '((t (:foreground "yellow"))) "Face used for 8" :group '2048-game)
(defface 2048-16-face '((t (:foreground "green"))) "Face used for 16" :group '2048-game)
(defface 2048-32-face '((t (:foreground "lightblue" :bold t))) "Face used for 32" :group '2048-game)
(defface 2048-64-face '((t (:foreground "lavender" :bold t))) "Face used for 64" :group '2048-game)
(defface 2048-128-face '((t (:foreground "SlateBlue" :bold t))) "Face used for 128" :group '2048-game)
(defface 2048-256-face '((t (:foreground "MediumVioletRed" :bold t))) "Face used for 256" :group '2048-game)
(defface 2048-512-face '((t (:foreground "tomato" :bold t))) "Face used for 512" :group '2048-game)
(defface 2048-1024-face '((t (:foreground "SkyBlue1" :bold t))) "Face used for 1024" :group '2048-game)
(defface 2048-2048-face '((t (:foreground "lightgreen" :bold t))) "Face used for 2048" :group '2048-game)

(defvar 2048-font-lock-keywords
  '(("\\<2\\>" 0 '2048-2-face)
    ("\\<4\\>" 0 '2048-4-face)
    ("\\<8\\>" 0 '2048-8-face)
    ("\\<16\\>" 0 '2048-16-face)
    ("\\<32\\>" 0 '2048-32-face)
    ("\\<64\\>" 0 '2048-64-face)
    ("\\<128\\>" 0 '2048-128-face)
    ("\\<256\\>" 0 '2048-256-face)
    ("\\<512\\>" 0 '2048-512-face)
    ("\\<1024\\>" 0 '2048-1024-face)
    ("\\<2048\\>" 0 '2048-2048-face)))

(defun sacha/2048-fontify ()
  (font-lock-add-keywords nil 2048-font-lock-keywords))

(defun sacha/2048-set-font-size ()
  (text-scale-set 5))

(use-package 2048-game
  :config
  (progn
   (add-hook '2048-mode-hook 'sacha/2048-fontify)
   (add-hook '2048-mode-hook 'sacha/2048-set-font-size)))

Getting R and ggplot2 to work in Emacs Org Mode Babel blocks; also, tracking the number of TODOs

I started tracking the number of tasks I had in Org Mode so that I could find out if my TODO list tended to shrink or grow. It was easy to write a function in Emacs Lisp to count the number of tasks in different states and summarize them in a table.

(defun sacha/org-count-tasks-by-status ()
  (interactive)
  (let ((counts (make-hash-table :test 'equal))
        (today (format-time-string "%Y-%m-%d" (current-time)))
        values output)
    (org-map-entries
     (lambda ()
       (let* ((status (elt (org-heading-components) 2)))
         (when status
           (puthash status (1+ (or (gethash status counts) 0)) counts))))
     nil
     'agenda)
    (setq values (mapcar (lambda (x)
                           (or (gethash x counts) 0))
                         '("DONE" "STARTED" "TODO" "WAITING" "DELEGATED" "CANCELLED" "SOMEDAY")))
    (setq output
          (concat "| " today " | "
                  (mapconcat 'number-to-string values " | ")
                  " | "
                  (number-to-string (apply '+ values))
                  " | "
                  (number-to-string
                   (round (/ (* 100.0 (car values)) (apply '+ values))))
                  "% |"))
    (if (called-interactively-p 'any)
        (insert output)
      output)))
(sacha/org-count-tasks-by-status)

I ran this code over several days. Here are my results as of 2014-05-01:

Date DONE START. TODO WAIT. DELEG. CANC. SOMEDAY Total % done + done +canc. + total + t – d – c Note
2014-04-16 1104 1 403 3 1 104 35 1651 67%
2014-04-17 1257 0 114 4 1 171 107 1654 76% 153 67 3 -217 Lots of trimming
2014-04-18 1292 0 74 4 5 183 100 1658 78% 35 12 4 -43 A little bit more trimming
2014-04-20 1305 0 80 4 5 183 100 1677 78% 13 0 19 6
2014-04-21 1311 1 78 4 4 184 99 1681 78% 6 1 4 -3
2014-04-22 1313 2 75 4 4 184 99 1681 78% 2 0 0 -2
2014-04-23 1369 4 66 4 5 186 101 1735 79% 56 2 54 -4 Added sharing/index.org
2014-04-24 1371 3 69 4 5 186 101 1739 79% 2 0 4 2
2014-04-25 1379 3 60 3 5 189 103 1742 79% 8 3 3 -8
2014-04-26 1384 3 65 3 5 192 103 1755 79% 5 3 13 5
2014-04-27 1389 2 66 3 5 192 103 1760 79% 5 0 5 0
2014-04-28 1396 3 67 3 5 192 103 1769 79% 7 0 9 2
2014-04-29 1396 3 67 3 5 192 103 1769 79% 0 0 0 0
2014-04-30 1404 4 70 4 5 192 103 1782 79% 8 0 13 5
2014-05-01 1413 4 80 3 4 193 103 1800 79% 9 1 18 8

Here’s the source for that table:

#+NAME: burndown
#+RESULTS:
|       Date | DONE | START. | TODO | WAIT. | DELEG. | CANC. | SOMEDAY | Total | % done | + done | +canc. | + total | + t - d - c | Note                       |
|------------+------+--------+------+-------+--------+-------+---------+-------+--------+--------+--------+---------+-------------+----------------------------|
| 2014-04-16 | 1104 |      1 |  403 |     3 |      1 |   104 |      35 |  1651 |    67% |        |        |         |             |                            |
| 2014-04-17 | 1257 |      0 |  114 |     4 |      1 |   171 |     107 |  1654 |    76% |    153 |     67 |       3 |        -217 | Lots of trimming           |
| 2014-04-18 | 1292 |      0 |   74 |     4 |      5 |   183 |     100 |  1658 |    78% |     35 |     12 |       4 |         -43 | A little bit more trimming |
| 2014-04-20 | 1305 |      0 |   80 |     4 |      5 |   183 |     100 |  1677 |    78% |     13 |      0 |      19 |           6 |                            |
| 2014-04-21 | 1311 |      1 |   78 |     4 |      4 |   184 |      99 |  1681 |    78% |      6 |      1 |       4 |          -3 |                            |
| 2014-04-22 | 1313 |      2 |   75 |     4 |      4 |   184 |      99 |  1681 |    78% |      2 |      0 |       0 |          -2 |                            |
| 2014-04-23 | 1369 |      4 |   66 |     4 |      5 |   186 |     101 |  1735 |    79% |     56 |      2 |      54 |          -4 | Added sharing/index.org    |
| 2014-04-24 | 1371 |      3 |   69 |     4 |      5 |   186 |     101 |  1739 |    79% |      2 |      0 |       4 |           2 |                            |
| 2014-04-25 | 1379 |      3 |   60 |     3 |      5 |   189 |     103 |  1742 |    79% |      8 |      3 |       3 |          -8 |                            |
| 2014-04-26 | 1384 |      3 |   65 |     3 |      5 |   192 |     103 |  1755 |    79% |      5 |      3 |      13 |           5 |                            |
| 2014-04-27 | 1389 |      2 |   66 |     3 |      5 |   192 |     103 |  1760 |    79% |      5 |      0 |       5 |           0 |                            |
| 2014-04-28 | 1396 |      3 |   67 |     3 |      5 |   192 |     103 |  1769 |    79% |      7 |      0 |       9 |           2 |                            |
| 2014-04-29 | 1396 |      3 |   67 |     3 |      5 |   192 |     103 |  1769 |    79% |      0 |      0 |       0 |           0 |                            |
| 2014-04-30 | 1404 |      4 |   70 |     4 |      5 |   192 |     103 |  1782 |    79% |      8 |      0 |      13 |           5 |                            |
| 2014-05-01 | 1413 |      4 |   80 |     3 |      4 |   193 |     103 |  1800 |    79% |      9 |      1 |      18 |           8 |                            |
#+TBLFM: @3$11..@>$11=$2-@-1$2::@3$13..@>$13=$9-@-1$9::@3$14..@>$14=$13-$11-($7-@-1$7)::@3$12..@>$12=$7-@-1$7

I wanted to graph this with Gnuplot, but it turns out that Gnuplot is difficult to integrate with Emacs on Microsoft Windows. I gave up after a half an hour of poking at it, since search results indicated there were long-standing problems with how Gnuplot got input from Emacs. Besides, I’d been meaning to learn more R anyway, and R is more powerful when it comes to statistics and data visualization.

Getting R to work with Org Mode babel blocks in Emacs on Windows was a challenge. Here are some of the things I ran into.

The first step was easy: Add R to the list of languages I could evaluate in a source block (I already had dot and ditaa from previous experiments).

(org-babel-do-load-languages
 'org-babel-load-languages
 '((dot . t)
   (ditaa . t) 
   (R . t)))

But my code didn’t execute at all, even when I was trying something that printed out results instead of drawing images. I got a little lost trying to dig into org-babel-execute:R with edebug, eventually ending up in comint.el. The real solution was even easier. I had incorrectly set inferior-R-program-name to the path of R in my configuration, which made M-x R work but which meant that Emacs was looking in the wrong place for the options to pass to R (which Org Babel relied on). The correct way to do this is to leave inferior-R-program-name with the default value (Rterm) and make sure that my system path included both the bin directory and the bin\x64 directory.

Then I had to pick up the basics of R again. It took me a little time to figure out that I needed to parse the columns I pulled in from Org, using strptime to convert the date column and as.numeric to convert the numbers. Eventually, I got it to plot some results with the regular plot command.

dates <- strptime(as.character(data$Date), "%Y-%m-%d")
tasks_done <- as.numeric(data$DONE)
tasks_uncancelled <- as.numeric(data$Total) - as.numeric(data$CANC.)
df <- data.frame(dates, tasks_done, tasks_uncancelled)
plot(x=dates, y=tasks_uncancelled, ylim=c(0,max(tasks_uncancelled)))
lines(x=dates, y=tasks_uncancelled, col="blue", type="o")
lines(x=dates, y=tasks_done, col="green", type="o")

r-plot

I wanted prettier graphs, though. I installed the ggplot2 package and started figuring it out. No matter what I did, though, I ended up with a blank white image instead of my graph. If I used M-x R instead of evaluating the src block, the code worked. Weird! Eventually I found out that adding print(...) around my ggplot made it display the image correctly. Yay! Now I had what I wanted.

library(ggplot2)
dates <- strptime(as.character(data$Date), "%Y-%m-%d")
tasks_done <- as.numeric(data$DONE)
tasks_uncancelled <- as.numeric(data$Total) - as.numeric(data$CANC.)
df <- data.frame(dates, tasks_done, tasks_uncancelled)
plot = ggplot(data=df, aes(x=dates, y=tasks_done, ymin=0)) + geom_line(color="#009900") + geom_point() + geom_line(aes(y=tasks_uncancelled), color="blue") + geom_point(aes(y=tasks_uncancelled))
print(plot)

 r-graph

The blue line represents the total number of tasks (except for the cancelled ones), and the green line represents tasks that are done.

Here’s something that looks a little more like a burn down chart, since it shows just the number of things to be done:

library(ggplot2)
dates <- strptime(as.character(data$Date), "%Y-%m-%d")
tasks_remaining <- as.numeric(data$Total) - as.numeric(data$CANC.) - as.numeric(data$DONE)
df <- data.frame(dates, tasks_remaining)
plot = ggplot(data=df, aes(x=dates, y=tasks_remaining, ymin=0)) + geom_line(color="#009900") + geom_point()
print(plot)

r-graph-2

The drastic decline there is me realizing that I had lots of tasks that were no longer relevant, not me being super-productive. =)

As it turns out, I tend to add new tasks at about the rate that I finish them (or slightly more). I think this is okay. It means I’m working on things that have next steps, and next steps, and steps beyond that. If I add more tasks, that gives me more variety to choose from. Besides, I have a lot of repetitive tasks, so those never get marked as DONE over here.

Anyway, cool! Now that I’ve gotten R to work on my system, you’ll probably see it in even more of these blog posts. =D Hooray for Org Babel and R!

Update 2014-05-09: Stephen suggested http://blogs.neuwirth.priv.at/software/2012/03/28/r-and-emacs-with-org-mode/ for more tips on setting up Org Mode with R and Emacs Speaks Statistics (ESS).

Thinking about my TODO keywords

It’s been twelve years since David Allen published Getting Things Done, with its geek-friendly flowcharts and processes for handling tasks in an interrupt-driven life. The way I manage my tasks is heavily influenced by GTD. I think in terms of next actions, waiting, and someday, and I have weekly reviews. I modified the TODO states a little to reflect what I need. It’s time to think about those states again to see what I can tweak and what reports I could use.

I use Org Mode in Emacs to manage my tasks and my notes. I can customize it to give me different kinds of reports, such as showing me all of my unscheduled tasks, or all tasks with a specific category, or even projects that are “stuck” (no next actions defined). Thinking about my processes will help me figure out what reports I want and how I want to use them.

Here are different types of tasks and how I track them:

  • Things I can work on right now (next actions): TODO
  • Things that I can work on after a different task is finished: currently WAITING, but probably better to implement with org-depend
  • Things I will revisit at a certain date, but I don’t need to think about them until then: TODO, scheduled (I used to use POSTPONED)
  • Things that would be nice to do someday, but maybe are incompletely specified or understood: SOMEDAY
  • Things I have decided not to work on: CANCELLED
  • Things I have asked someone else to do: DELEGATED
  • Things I can ask someone else to do: TODELEGATE
  • Things I am waiting for (usually not based on date) and that I need to follow up on: WAITING
  • Things I can write about: TOBLOG. These are pretty optional, so I don’t want them in my TODO list…
  • If something is a duplicate of something else – remove TODO keyword and add link?

I use the following code for an agenda view of unscheduled tasks:

(defun sacha/org-agenda-skip-scheduled ()
  (org-agenda-skip-entry-if 'scheduled 'deadline 'regexp "\n]+>"))

(add-to-list 'org-agenda-custom-commands
   '("u" "Unscheduled tasks" alltodo ""
     ((org-agenda-skip-function 'sacha/org-agenda-skip-scheduled)
     (org-agenda-overriding-header "Unscheduled TODO entries: "))))

So the to-do process looks like this:

  • Every week, review my evil plans and projects. Check my agenda without the routine tasks to see what new things I’m working on. Schedule a few tasks to encourage me to make regular progress.
  • Every day, go through my Org agenda (C-c a a) and do all the tasks that are scheduled.
  • When I’m done or if I feel like working on something else:
    • What do I feel like doing? If there’s a specific activity that I feel like:
      • Go to the relevant project/section of my TODO list, or check the TODOs by context (drawing, writing, etc.)
      • Clock in on that task.
    • If there’s a specific task I feel like working on:
      • Find the task, maybe with C-u C-c C-w (org-refile) and work on it.
    • If there’s a new idea I want to work on:
      • Use org-capture to create the task, file it in the appropriate project, and then clock in.
  • If I have an idea for a task, use org-capture to create the task and file it in the appropriate project.

How do I want to improve this?

  • Maybe get more used to working with contexts? I have all these Org Agenda commands and I hardly ever use them. I tend to work with projects instead. Actually, working with projects makes sense too, because that minimizes the real context shift.
  • Get better at reviewing existing tasks. I started tracking the number of tasks in each state (DONE, TODO, etc.), which nudged me to review the tasks and cross old tasks off. If I streamline my process for capturing tasks, filing them, and reviewing them by project/context/effort, then I can get better at choosing good tasks to work on from my existing TODO list.
  • Estimate effort for more tasks, and use that more often I have some reports that can filter or sort by estimated effort. I don’t use effort that much, though. Does it makes sense to get into the habit of choosing tasks by estimated time as an alternative approach? I usually have fairly large, flexible blocks of time…
  • Tag things by level of energy required? I want to take advantage of high-energy times. So, when I feel alert and creative, I want to focus on coding and writing. I can save things like paperwork for low-energy times. I can tag some tasks as :lowenergy: and then filter my reports.

Hmm…

Reflecting on 10 episodes of Emacs Chats

I’ve posted ten Emacs Chat episodes so far, and the transcripts for the most recent ones are coming soon. These are hour-long conversations with Emacs geeks about how they got started with Emacs, why they like it, and how they use it. We usually go through people’s config files, too, since that often leads to interesting tips.

janis_mancevics Jānis Mancēvičs

Literate programming, Unity game development, code folding

Emacs-Chat-Tom-Marble Tom Marble

Org Mode, time tracking, LaTeX, and invoice generation. Also, Clojure + Emacs and other good things.

Emacs-Chat-Iannis-Zannos Iannis Zannos

Music and SuperCollider

Emacs-Chat-Magnar-Sveen Magnar Sveen

Hanging out with other Emacs geeks, Emacs Rocks, and board games

Emacs-Chat-Bastien-Guerry Bastien Guerry

Org Mode maintenance, getting started with Emacs, hacking his life with Org

Emacs-Chat-Carsten-Dominik Carsten Dominik

Getting started with Emacs, the joys of Calc, and other cool things

Thomas-Kjeldahl-Nilsson-Emacs-chat Thomas Kjeldahl Nilsson
Thomas shares about Emacs and picking up configuration snippets from EmacsWiki.
Emacs-Chat-with-Avdi-Grimm Avdi Grimm
Org-mode literate programming, Ruby, and how he got started with Emacs.
John-Wiegley John Wiegley
Emacs Lisp development and other good things
and me! =) Sacha Chua
in which Bastien Guerry interviews me

I started this because it was so much fun meeting Emacs geeks in person at the Emacs Conference in London last year. (When are we having another one? I’m happy to sponsor a reasonable venue.) You pick up lots of tips when you watch how someone else uses Emacs, but not everyone has the luck of working near other Emacs geeks. (I don’t!) I also wanted to get to know other Emacs geeks so that I could “hear” their voices when reading mailing list messages and code snippets. I wanted other people to get that feeling of knowing people in the community – other real people who use Emacs.

I was pretty anxious about it in the beginning. Would I be able to ask interesting questions, or would there be dead silence? What if I hadn’t researched people well enough? Would asking people about their beginnings get repetitive after many episodes? I feel a little more relaxed now. It turns out that it’s easy to invite people to be on one of these conversations, and I always find the conversation interesting. People are so enthusiastic about Emacs. Yay!

It’s been great hearing stories from people who’ve been using Emacs for ages (like Iannis Zannos and Tom Marble) and people who’ve gotten into Emacs fairly recently (like the way Magnar Sveen only seriously started using it a few years ago). Org Mode frequently pops up in conversation. I’ve learned about lots of other interesting packages as well, like redshank and erefactor.

People tell me that they enjoy listening to the episodes. The episodes are still on the long side (an hour or so, versus short-and-punchy 15- or 30-minute chats), but they’re good for picking up odd tips.

Of the little podcast experiments I’ve been running, the Emacs Chats series is my favourite. Other experiments were easier to sketchnote (which people also really enjoyed), but I like the Emacs community the most. =)

From these experiments, I’ve learned that Google Hangout on Air is a convenient way to create an audio/video show with guests. With a little bit of work, you can turn these conversations into podcasts that people can download and subscribe to, transcripts that people can read, and so on.

I wanted to learn how to delegate a smoothly-running process. That worked out really well. Now, when I finish an episode, I simply add a card to my Trello board with the URL and my assistants will post the show notes and the transcript for me.

I could probably make this even better by following up. I can spend more time editing the transcripts, adding links, and summarizing key points. Maybe I’ll convert the transcripts to Org Mode and then structure things more from there.

In terms of scheduling, picking times that are a month or two away seems to be working well. I like proposing specific times with Boomerang Calendar. It feels more proactive than asking people to check http://sachachua.com/meet for meeting times, although both ways still involve a bit of work for the other person since they have to check their calendar. If I suggest the times and do the timezone conversions myself, that means we can set the time with fewer clicks required from the other person. It doesn’t feel as stand-offish as cc-ing an assistant who may or may not be able to quickly reply. (Although perhaps I should train my current assistants to do this, since they seem to be fairly responsive…)

I mostly find people through recommendations, so if you want to hear from someone, suggest them or introduce us by e-mail. I’d love to interview more women who use Emacs (maybe Amelia Andersdotter?), but I’m happy to chat with all sorts of folks about Emacs. You don’t have to be famous. =) If you’ve got an interesting demo to share, I’d love to hear from you too.

Onward! With Alex Poslavsky’s help, I’ve been adding more Emacs Chats resources to Github so that people can easily subscribe to it or contribute there. I noticed a few of them were missing transcripts, so we’ll work on that too. What else would make these Emacs Chats better or more useful for you?

How Org Mode helps me deal with an ever-growing backlog

If you’re like me, you probably have a to-do list several miles long. I like thinking of this as the backlog from agile programming. It’s a list of tasks that I could choose to work on, but I haven’t committed to doing everything on the list. This means I don’t have to waste energy feeling guilty about not getting everything done. Instead, I can treat it like a buffet of projects to choose from depending on what I feel like working on.

2014-04-28 Dealing with an ever-growing backlog

2014-04-28 Dealing with an ever-growing backlog

I think I add tasks faster than I cross tasks out. (Hmm, I should track this!) It never ends. Most tasks suggest next steps I could take after I finish the first ones. You might think that an ever-growing to-do list is a bad thing. This is okay. In fact, this is good. It means that I’ll always have a variety of tasks to choose from.

People manage tasks in different ways. For my personal tasks, I use several large text files in Org Mode for Emacs. Org Mode is an outline-based tool which makes it easy for me to organize my tasks into projects and projects into themes. It also supports tagging, links, agendas, dynamic views, and all sorts of other great ways to slice-and-dice my task list. Here’s how I deal with some of the common challenges people face with a large task backlog:

  • Making sure important, urgent tasks don’t fall through the cracks
  • Making sure you don’t neglect important but not urgent tasks
  • Keeping track of what you’re waiting for
  • Catching procrastination

Making sure important, urgent tasks don’t fall through the cracks

If something has a deadline, I add the deadline in Org using C-c C-d (org-deadline). This means that reminders will appear on my daily agenda for the 14 days before the deadline, counting down to the deadline itself. (The number of days is controlled by org-deadline-warning-days.) In addition, I usually schedule the task for a day that I want to work on it, so that I can get the task out of the way.

I’m careful about what I commit to, erring on the side of under-committing rather than over-committing. I’m selective about my client work and my volunteering. I keep my schedule as open as I can, and I’m not afraid to reschedule if I need to. Hardly anything I work on could be considered urgent. If an urgent request does come in, I ask questions to determine its true urgency, including potential alternatives and consequences of failure.

You might not have as much choice about what to work on, but you might also be surprised by how much you can push back. Be careful about what you allow to be urgent in your life.

Making sure you don’t neglect important but not urgent tasks

I have plenty of space to work on things that are important but not urgent because I manage my commitments carefully. This means that I can usually finish a few important-but-not-urgent tasks every day.

Which tasks do I consider important? I like thinking in terms of projects. Important tasks tend to be associated with projects instead of standing in isolation. Important tasks move me toward a specific goal. I have many goals and projects, but because they’re fewer than the number of tasks I have, I can prioritize them more easily. I can decide that some projects are in the background and some are in focus. Important tasks are the tasks that help me make more progress on the projects I consider important.

Because I like having two or three projects on the go, it helps to make sure that I make regular progress on those projects instead of getting carried away on just one. Tracking my time helps me stay aware of that balance. I also review my projects every week and schedule specific tasks for each of them, so I can make a little progress at least. Once I switch context and start thinking about a project, it’s easy to pick another couple of tasks in that area and get even more done.

If you’re struggling with creating enough space to work on important but not urgent tasks, you might be able to partner up with someone so that you can block off time to work on non-urgent things. Many teams have a rotating schedule for dealing with customer requests or urgent issues. One person covers the requests for a day, allowing the rest of the team to focus. Then the next person takes on that duty, and so on.

Keeping track of what you’re waiting for

One of the useful tips I picked up from David Allen’s Getting Things Done book was the idea of marking a task as WAITING. I usually add a description of what I’m waiting for, who’s responsible, and when I want to follow up. This makes it easier to follow up. When I’m waiting for a specific date (ex: the library makes a DVD hold-able after a certain date), I schedule the task for then.

I use the Boomerang for Gmail extension when I’m waiting for an e-mail reply. Boomerang lets me pop the message back into my inbox if I haven’t received a reply by a specific date, so I don’t have to keep track of that myself.

Handling less-important but still useful things

There are tasks on my to-do list that have been on that list for years. This is okay.
I’m getting better at noting names and contact information in my tasks so that I can follow up with people even after some time. This is particularly useful for book recommendations. I get a lot of book recommendations and I get most of my books from the library, so there’s usually a delay of a few weeks. Because Org Mode lets me add notes and links to the body of a task, I can look up information easily.

I work on less-important tasks when I don’t feel like working on my major tasks, or when I’m looking for small tasks so I can fill in the gaps of my day. Org Mode gives me plenty of ways to look up tasks. I usually look for tasks by projects, navigating through my outline. I can also look for tasks by effort estimate, so I can see everything that will probably take me less than 15 minutes. Context is useful too – I can search for various tags to find tasks I can do while I’m on the phone, or out on errands, or when I feel like writing or drawing.

I like thinking in terms of low-hanging fruit, so I often choose tasks that require little time or effort and have good impact. It can be overwhelming to look at a long list of tasks and decide which ones have good return. It’s easier to tag these tasks when you create the task, or to think in terms of projects instead.

Some tasks grow in importance or urgency over time. If I want to make sure that I revisit a task on a certain date, I schedule it for then.

Catching procrastination

I still end up rescheduling tasks multiple times. (I’ve been putting off redoing my business cards for a few months now!) I’ve noticed that there are different kinds of procrastination, including:

  • Procrastinating because you don’t have time today: It’s easy to reschedule things a few weeks or a month in advance. In fact, Org has a built-in command for bulk-scattering tasks. From the agenda view, you can type m to mark multiple tasks, then type B and then S to scatter tasks randomly over the next N days. (Call it with a prefix argument as C-u B S to limit it to weekdays.) If I catch myself procrastinating because I don’t have enough time, that’s usually a sign to be more cautious about my estimates and commitments, so I adjust those too.
  • Procrastinating because it’s less important than other tasks: This is related to the time reason. I have no qualms about pushing less-important tasks forward.
  • Procrastinating because you don’t feel like working on it: Is the task actually important? If it’s not, I usually get rid of it without feeling guilty. If it’s still useful, I might unschedule it so that I see it only if I’m looking for tasks in that project or in that context. Alternatively, I can just mark the task as CANCELLED or SOMEDAY. If the task is important, I think about whether I’m likely to feel like working on it at some point in the future. If I’m likely to not feel any different about it, I might delegate it, or I might just sit down and do it since procrastination doesn’t add value. On the other hand, if I’m likely to feel like working on it at some point, then I tag it with that context and push it out to some other date.
  • Procrastinating because you forgot about it: I usually check my agenda every day and Org shows forgotten things in a different colour, so I catch these quickly. If the tasks are more important than the tasks I’ve already scheduled, I might work on those first. Alternatively, I might schedule it for sometime later.

I procrastinate based on my to-do list, not based on my inbox. The inbox is a terribly unstructured way to manage your tasks. I use Boomerang for Gmail to defer some mail to a later date, but that’s usually so that I can pop it back into my inbox the day that I meet someone so that I have context and so that I don’t have to copy the link into the calendar entry or my TO-DO list.

Wrapping up

So that’s how I deal with having a large backlog. I focus first on the stuff that I need to do, and I make sure that shows up on my agenda. Then I make it easy to look for stuff that I want to do using Org’s support for projects, tags, time estimates, and so on. I don’t feel guilty about having lots of tasks to choose from. I view my backlog positively. It lets me do good stuff without worrying too much about how I spend my time.

How do you deal with your backlog? =)

Sneak peek! Writing this post prompted me to start tracking whether my backlog grew or shrank each day. Check out my preliminary results and the code I used to analyze my TODOs.
2014-04-27: Fixed typo in keybinding – thanks, Sujith Abraham!