Warning: include(): Unable to allocate memory for pool. in /var/www/sachachua.com/blog/wp-content/plugins/redirection/redirection.php on line 28 Warning: include(): Unable to allocate memory for pool. in /var/www/sachachua.com/blog/wp-content/plugins/redirection/redirection.php on line 29 Warning: include(): Unable to allocate memory for pool. in /var/www/sachachua.com/blog/wp-content/plugins/redirection/redirection.php on line 30 Warning: include_once(): Unable to allocate memory for pool. in /var/www/sachachua.com/blog/wp-settings.php on line 214 Warning: include_once(): Unable to allocate memory for pool. in /var/www/sachachua.com/blog/wp-settings.php on line 214 Warning: include_once(): Unable to allocate memory for pool. in /var/www/sachachua.com/blog/wp-settings.php on line 214 Warning: include(): Unable to allocate memory for pool. in /var/www/sachachua.com/blog/wp-content/plugins/w3-total-cache/w3-total-cache.php on line 50 Warning: require_once(): Unable to allocate memory for pool. in /var/www/sachachua.com/blog/wp-content/plugins/w3-total-cache/inc/define.php on line 1478 Warning: require_once(): Unable to allocate memory for pool. in /var/www/sachachua.com/blog/wp-content/plugins/w3-total-cache/inc/define.php on line 1478 Warning: require_once(): Unable to allocate memory for pool. in /var/www/sachachua.com/blog/wp-content/plugins/w3-total-cache/inc/define.php on line 1478 Warning: require_once(): Unable to allocate memory for pool. in /var/www/sachachua.com/blog/wp-content/plugins/w3-total-cache/inc/define.php on line 1478 Warning: require_once(): Unable to allocate memory for pool. in /var/www/sachachua.com/blog/wp-content/plugins/w3-total-cache/inc/define.php on line 1478 Warning: require(): Unable to allocate memory for pool. in /var/www/sachachua.com/blog/wp-settings.php on line 220 Warning: require_once(): Unable to allocate memory for pool. in /var/www/sachachua.com/blog/wp-includes/functions.php on line 2905 Warning: require_once(): Unable to allocate memory for pool. in /var/www/sachachua.com/blog/wp-content/plugins/wordpress-seo/wp-seo-main.php on line 149 Warning: require_once(): Unable to allocate memory for pool. in /var/www/sachachua.com/blog/wp-content/plugins/wordpress-seo/wp-seo-main.php on line 88 Warning: include(): Unable to allocate memory for pool. in /var/www/sachachua.com/blog/wp-settings.php on line 328 Warning: require_once(): Unable to allocate memory for pool. in /var/www/sachachua.com/blog/wp-content/plugins/powerpress/powerpress.php on line 1203 Warning: include(): Unable to allocate memory for pool. in /var/www/sachachua.com/blog/wp-includes/script-loader.php on line 51 Warning: include(): Unable to allocate memory for pool. in /var/www/sachachua.com/blog/wp-includes/script-loader.php on line 566 Warning: include(): Unable to allocate memory for pool. in /var/www/sachachua.com/blog/wp-includes/script-loader.php on line 51 Warning: include_once(): Unable to allocate memory for pool. in /var/www/sachachua.com/blog/wp-content/plugins/nextgen-gallery/pope/lib/class.component_registry.php on line 939 Warning: include_once(): Unable to allocate memory for pool. in /var/www/sachachua.com/blog/wp-content/plugins/nextgen-gallery/pope/lib/class.component_registry.php on line 939 Warning: include_once(): Unable to allocate memory for pool. in /var/www/sachachua.com/blog/wp-content/plugins/nextgen-gallery/pope/lib/class.component_registry.php on line 939 Warning: include_once(): Unable to allocate memory for pool. in /var/www/sachachua.com/blog/wp-content/plugins/nextgen-gallery/pope/lib/class.component_registry.php on line 939 Warning: include_once(): Unable to allocate memory for pool. in /var/www/sachachua.com/blog/wp-content/plugins/nextgen-gallery/pope/lib/class.component_registry.php on line 939 Warning: include_once(): Unable to allocate memory for pool. in /var/www/sachachua.com/blog/wp-content/plugins/nextgen-gallery/pope/lib/class.component_registry.php on line 939 Warning: include_once(): Unable to allocate memory for pool. in /var/www/sachachua.com/blog/wp-content/plugins/nextgen-gallery/pope/lib/class.component_registry.php on line 939 Warning: include_once(): Unable to allocate memory for pool. in /var/www/sachachua.com/blog/wp-content/plugins/nextgen-gallery/pope/lib/class.component_registry.php on line 939 Warning: include_once(): Unable to allocate memory for pool. in /var/www/sachachua.com/blog/wp-content/plugins/nextgen-gallery/pope/lib/class.component_registry.php on line 939 Warning: include_once(): Unable to allocate memory for pool. in /var/www/sachachua.com/blog/wp-content/plugins/nextgen-gallery/pope/lib/class.component_registry.php on line 939 Warning: include_once(): Unable to allocate memory for pool. in /var/www/sachachua.com/blog/wp-content/plugins/nextgen-gallery/pope/lib/class.component_registry.php on line 939 Warning: include_once(): Unable to allocate memory for pool. in /var/www/sachachua.com/blog/wp-content/plugins/nextgen-gallery/pope/lib/class.component_registry.php on line 939 Warning: include_once(): Unable to allocate memory for pool. in /var/www/sachachua.com/blog/wp-content/plugins/nextgen-gallery/pope/lib/class.component_registry.php on line 939 Warning: include_once(): Unable to allocate memory for pool. in /var/www/sachachua.com/blog/wp-content/plugins/nextgen-gallery/pope/lib/class.component_registry.php on line 939 org Archives - sacha chua :: living an awesome life

Category Archives: org

On this page:

Summarizing the last meeting dates in Org Contacts

Steffan Heilmann wanted to be able to quickly see the last time he interacted with someone if he tracked interactions in org-contacts. That is, given something like this:

* John Smith
** DONE Conversation
[2014-01-20]
** DONE E-mail
[2014-01-15]
* Jane Smith
** DONE Conversation
[2014-01-07]

… we want to see the latest timestamps for each contact entry.

Here’s the code that I came up with. It scans backward for timestamps or headings. Whenever it finds a timestamp, it compares the timestamp with the one that it has previously stored and keeps the later timestamp. Whenever it encounters a level-1 heading, it sets the property and clears the stored timestamp.

(defun sacha/org-update-with-last-meeting ()
  "Update each level 1 heading with the LASTMEETING property."
  (interactive)
  (goto-char (point-max))
  (let (last-meeting)
    (while (re-search-backward
            (concat "\\(" org-outline-regexp "\\)\\|\\("
                    org-maybe-keyword-time-regexp "\\)") nil t)
      (cond
       ((and (match-string 1)
             (= (nth 1 (save-match-data (org-heading-components))) 1)
             last-meeting)
        ;; heading
        (save-excursion (org-set-property "LASTMEETING" last-meeting))
        (setq last-meeting nil))
       ((and (match-string 2))
        (if (or (null last-meeting) (string< last-meeting (match-string 2)))
            (setq last-meeting (match-string 2))))))))

Scanning backwards works well here because that makes it easy to add information to the top-level heading we’re interested in. If we scanned it the other way around (say, with org-map-entries), we might need to backtrack in order to set the property on the top-level heading.

The result is something like this:

* John Smith
  :PROPERTIES:
  :LASTMEETING: [2014-01-20]
  :END:
** DONE E-mail
[2014-01-15]
** DONE Conversation
[2014-01-20]
* Someone without a meeting
* Jane Smith
  :PROPERTIES:
  :LASTMEETING: [2014-01-07]
  :END:
** DONE Conversation
[2014-01-07]

You can then use something like:

#+COLUMNS: %25ITEM %LASTMEETING %TAGS %PRIORITY %TODO
#+BEGIN: columnview :maxlevel 1
| ITEM                        | LASTMEETING  | TAGS | PRIORITY | TODO |
|-----------------------------+--------------+------+----------+------|
| * John Smith                | [2014-01-20] |      |          |      |
| * Someone without a meeting |              |      |          |      |
| * Jane Smith                | <2014-01-07> |      |          |      |
#+END:

… or even use M-x org-sort to sort the entries by the LASTMEETING property (R will reverse-sort by property).

Playing around with Clojure, Cider, and 4Clojure

4Clojure has a lovely series of exercises to help you practice Clojure. I don’t know much Clojure yet. I’ve basically been taking what I know of Emacs Lisp and trying to cram it into Clojure syntax. (compose is pretty cool!) I should probably read through a Clojure tutorial and some kind of syntax reference. (Hyperpolyglot is neat!) But hey, I’ve gotten through 21 problems so far.

Tom Marble and I were chatting about Clojure, Emacs, and Org Babel. As it turns out, there are lots of ways to interact with 4clojure problems from within Emacs. Tom told me about the 4clojure package by Joshua Hoff, which is probably slightly improved with the following code:

(require 'clojure-mode)
(defun my/4clojure-check-and-proceed ()
  "Check the answer and show the next question if it worked."
  (interactive)
  (let ((result (4clojure-check-answers)))
    (unless (string-match "failed." result)
       (4clojure-next-question))))
(define-key clojure-mode-map (kbd "C-c C-c") 'my/4clojure-check-and-proceed)

That one doesn’t track your progress on the website, though, so you’ll still want to copy and paste the solution yourself.

I like working within Org Mode so that I can easily take notes along the way. Here are the notes I took while figuring out how to get Clojure and Org to work together. http://www.braveclojure.com/basic-emacs/ is nice. http://bzg.fr/emacs-org-babel-overtone-intro.html has a good introduction. Here’s what I used from those:

Install Java (at least version 6), Clojure and Leiningen.

Install the clojure-mode and cider Emacs packages

Evaluate this by moving the point to the #+begin_src line and running C-c C-c

(add-to-list 'package-archives '("melpa" . "http://melpa.milkbox.net/packages/") t)
(package-refresh-contents)
(package-install 'clojure-mode)
(package-install 'cider)

And then evaluate this afterwards:

(add-to-list 'org-babel-load-languages '(emacs-lisp . t))
(add-to-list 'org-babel-load-languages '(clojure . t))
(org-babel-do-load-languages 'org-babel-load-languages org-babel-load-languages)
(setq nrepl-hide-special-buffers t
      cider-repl-pop-to-buffer-on-connect nil
      cider-popup-stacktraces nil
      cider-repl-popup-stacktraces t)
(cider-jack-in)

That should let you evaluate this:

(list? '(1 2 3 4))

—————–
And that let me do stuff like this for #27: Palindrome Detector:

(defn __ [x] (= (seq x) (reverse x)))
(list
  (false? (__ '(1 2 3 4 5)))
  (true? (__ "racecar"))
  (true? (__ [:foo :bar :foo]))
  (true? (__ '(1 1 3 3 1 1)))
  (false? (__ '(:a :b :c))))
true true true true true

If all the results are true, then I’ve passed. Yay! In the web interface, __ is where your answers go. Fortunately, it’s also a valid Lisp name, so I can defn a function to replace it when testing locally. The proper answer would probably be something like (fn [x] (= (seq x) (reverse x))) when submitted through the web interface, which is close enough.

it would be great to have something like 4clojure for Emacs Lisp – a site where you can practise solving small, well-defined problems. =) Has someone already written one?

Cobbling together a semi-auto-responder using Emacs, Gnus, and org-contacts

It turns out that lots of people are interested in an e-mail-based course for learning Emacs Lisp. Yay! =) Maybe it’s the idea of bite-size chunks. Maybe it’s the ease of asking questions. Maybe it’s the regular reminders to work on something. Who knows? Whatever the reason, it’s awesome to see so many people willing to join me on this experiment.

Since this is my first time to venture into the world of teaching people online, I wanted to see how far I could push actually doing all the mails myself, instead of just signing up for an Aweber account and handing everyone off to an impersonal autoresponder. I dusted off Gnus, offlineimap, and org-contacts, and started figuring out my workflow. I’ll share how that workflow’s evolving so that you can get a sense of how someone might write little bits of Emacs Lisp to make something repetitive easier.

For the first little while, I got by with using C-x r s (copy-to-register) and C-x r i (insert-register) to store the text that I needed.
Sometimes I needed to paste in the welcome message and checklist, and sometimes I needed to paste in the first lesson. By using registers, I could insert whatever I wanted instead of going through the kill ring. I also had another bit of templated code in yet another register so that I could easily create an org-contacts entry for the person whose mail I was replying to. In the beginning, I used tasks under each person’s heading to indicate that I had sent them the checklist or that I had sent them the first lesson. Eventually, I changed my org-contacts notes so that the TODO state of each person showed which lesson I was going to send them next, or CHECKLIST if I was waiting for their reply to the checklist. I also set up Org so that it would automatically log when the TODO state was changed.

#+TODO: TODO | DONE
#+TODO: CHECKLIST(c!) BEGINNER1(1!) BEGINNER2(2!) BEGINNER3(3!) BEGINNER4(4!) FULL(f!) | FINISHED(x!)
#+TODO: | CANCELLED

* Who
** CHECKLIST Jane Smith ...
** BEGINNER1 John Smith
   SCHEDULED: <2014-05-28 Wed>
   :PROPERTIES:
   :EMAIL: john@example.com
   :END:
(notes from the messages, etc.)

I wrote some code to make it easier to send someone a checklist and create a note for them in my org-contacts file. I bound it to C-c e c for convenience.
(The bind-key function is defined by a package.)

(setq sacha/elisp-course-checklist-body "... really long text here...")
(defun sacha/elisp-course-checklist ()
  "Copy this message and put it at the end as a checklist item. 
Start a message with the checklist."
  (interactive)
  (gnus-summary-scroll-up 1)
  (with-current-buffer gnus-article-buffer
    (let ((message (buffer-substring-no-properties (point-min) (point-max)))
          (email (cadr (org-contacts-gnus-get-name-email))))
      (with-current-buffer "elisp-course.org"
        (save-excursion
          (goto-char (point-max))
          (save-excursion
            (insert "\n** " message)
            (org-set-property "EMAIL" email)
            (org-todo "CHECKLIST"))))))
  (gnus-summary-followup-with-original nil)
  (goto-char (point-max))
  (insert sacha/elisp-course-checklist-body))
(bind-key "C-c e c" 'sacha/elisp-course-checklist)

This made it easier for me to read the starred messages from my inbox and use C-c e c to get a head start on processing people’s introductory messages.
Yay! I used the register trick to help me reply to people who were ready for the first lesson. After the first few replies, I noticed that the attachment code was fine even if I put that in the register too, so I added it as well.

Things got more complicated when I started processing lesson 2. I didn’t want to have to set up and remember lots of different registers, and I didn’t want to manually update the TODO states either. So I started defining functions that I could call with keyboard shortcuts:

(defun sacha/elisp-course-1 ()
  (interactive)
  (let ((marker (org-contacts-gnus-article-from-get-marker)))
    (if marker
        (org-with-point-at marker
          (org-todo "BEGINNER2"))))
  ;; Find the person's contact record
  (gnus-summary-scroll-up 1)
  (gnus-summary-followup-with-original nil)
  (message-goto-subject)
  (message-delete-line)
  (insert (concat "Subject: " sacha/elisp-course-1-subject "\n"))
  (goto-char (point-max))
  (insert sacha/elisp-course-1-body))
(bind-key "C-c e 1" 'sacha/elisp-course-1)
(defun sacha/elisp-course-2 ()
  (interactive)
  (let ((marker (org-contacts-gnus-article-from-get-marker)))
    (if marker
        (org-with-point-at marker
          (org-todo "BEGINNER3"))))
  ;; Find the person's contact record
  (gnus-summary-scroll-up)
  (gnus-summary-followup-with-original nil)
  (goto-char (point-max))
  (insert sacha/elisp-course-2-body))
(bind-key "C-c e 2" 'sacha/elisp-course-2)

Really, though, it doesn’t make sense to have a lot of duplicated code. So I wrote some code that would use the person’s TODO keyword to look up the message to send them, and then move them to the next keyword. Now I don’t need sacha/elisp-course-1 or sacha/elisp-course-2 any more.

(setq sacha/elisp-course-info
      `(("CHECKLIST" nil ,sacha/elisp-course-checklist-body)
        ("BEGINNER1" ,sacha/elisp-course-1-subject ,sacha/elisp-course-1-body)
        ("BEGINNER2" ,sacha/elisp-course-2-subject ,sacha/elisp-course-2-body)))

(defun sacha/elisp-course-process (subject body &optional state)
  "Process this course entry."
  (if (derived-mode-p 'org-mode)
      (progn
        ;; Move this node to the next state and compose a message
        (if state (org-todo state))
        (org-todo 'right)
        (message-mail (org-entry-get (point) "EMAIL") subject)
        (goto-char (point-max))
        (insert body))
    ;; Doing this from Gnus; find the person's info
    (let ((marker (org-contacts-gnus-article-from-get-marker)))
      (if marker (org-with-point-at marker
                   (if state (org-todo state))
                   (org-todo 'right)))
      ;; Compose a reply
      (gnus-summary-scroll-up 1)
      (gnus-summary-followup-with-original nil)
      (message-goto-subject)
      (message-delete-line)
      (insert (concat "Subject: " subject "\n"))
      (goto-char (point-max))
      (insert body))))

(defun sacha/elisp-course-guess-and-process (&optional state)
  (interactive (list (if current-prefix-arg (read-string "State: "))))
  (let ((current-state
         (or state (elt
                    (if (derived-mode-p 'org-mode)
                        (org-heading-components) 
                      (let ((marker (org-contacts-gnus-article-from-get-marker)))
                        (if marker (org-with-point-at marker (org-heading-components)))))
                    2))))
    (sacha/elisp-course-process
     (elt (assoc current-state sacha/elisp-course-info) 1)
     (elt (assoc current-state sacha/elisp-course-info) 2)
     state)))
(bind-key "C-c e e" 'sacha/elisp-course-guess-and-process)

Come to think of it, I should totally have it schedule the next update for the next Wednesday, too. ;) That’s just (org-schedule "+wed"). Neat, huh?
And I’m sure there are all sorts of ways the code can be simpler, but it works for me at the moment, so hooray!

I really like this approach. It lets me pull in standard information while also letting me customize the messages and how it fits into my task tracking. I can’t get that with Gmail (even with canned responses), and I’m not sure any CRM is going to be quite as awesome as this. I can’t wait to see how else we’ll tweak this as we go through more conversations. I’d like to get better at:

  • having a consistent place where I can process all the messages and make sure nothing falls through the cracks; I currently star messages to make sure I process them, since the Gmail label folder in IMAP seems to be missing some messages
  • seeing all Gnus conversations related to an org-contacts entry
  • reaching out to people proactively with the next lesson, even if they haven’t e-mailed me (or maybe I should wait for them?)

Anyway, that’s an example of writing a little bit of Emacs Lisp in order to connect different packages. Gnus handles mail, Org handles notes, org-contacts links the two together, and with a little bit of custom code, I can make the combination fit what I want to do. I read the source code of org-contacts to find out how I could look up the appropriate note, and I looked at org-shiftright to find out how to move things to the next TODO state. If you know something that works roughly like what you want it to work, you can find out how it does things and then copy that.

As for the course itself: I’ve been sending people links to the HTML output, attached .txt files (with -*- mode: org -*-) so they can open it in Emacs if they want, and inline text so that they can skim it briefly in their e-mail client if they want to. I’m not perfectly happy with the plain-text formats, but it seems to be a reasonable compromise, and so far people have been able to deal with it. I’ve been improving pieces of it based on feedback on clarity, suggestions for good examples, and so on. I didn’t take all the feedback; after thinking about some of the suggestions, I still preferred it my way. It’s shaping up quite nicely, though!

If you’re curious about the beginner’s course on reading Emacs Lisp, e-mail me at sacha@sachachua.com and we’ll see how this works out. I’m certainly learning a lot. =)

How to update the Org 7 that comes with Emacs to Org 8 (more configuration! better exports!)

Update 2014-05-12: Simplified thanks to Sebastian’s note that Org 8 is available in the built-in package repository, yay!

The Org Mode included in Emacs 24 is version 7. Version 8 has lots of new configuration variables and the exporting mechanism has been rewritten. However, it needs to be installed in an Emacs that has not yet loaded any Org code or files. Here’s how you can upgrade your Org:

  1. Start Emacs with emacs -q. This skips your personal configuration.
  2. You will need an Internet connection for this step. Type M-x package-install, and type in org. This will install the latest version of Org from the built-in package repository.
  3. Edit your ~/.emacs.d/init.el (or ~/.emacs, if you’re using that instead). Add the following code to the beginning of the file:
    (package-initialize)
    (setq package-enable-at-startup nil)
    

    This will load the installed packages when you start Emacs, overriding the buit-in Org 7 with the Org 8 version that you installed.

    Advanced note: If you’ve downloaded Emacs Lisp code that should override code already installed through packages, you need to change this to (package-initialize nil) instead, and add (package-initialize t) after your load-path settings.

  4. Check your configuration for references to the older version of Org. In particular, look for any configuration related to exporting (ex: (require 'org-html)). You can change those lines to their Org 8 equivalents (ex: (require 'ox-html)), but it’s probably easier to just comment them out for now. You can comment out lines by adding ; to the beginning.
  5. Save your init.el and restart Emacs (this time, without the -q option). M-x org-version should now start with Org-mode version 8.
  6. Review your Emacs configuration for any changes that you will need to make. You can ask the Org Mode mailing list for help if you get stuck.

Good luck!

Getting R and ggplot2 to work in Emacs Org Mode Babel blocks; also, tracking the number of TODOs

I started tracking the number of tasks I had in Org Mode so that I could find out if my TODO list tended to shrink or grow. It was easy to write a function in Emacs Lisp to count the number of tasks in different states and summarize them in a table.

(defun sacha/org-count-tasks-by-status ()
  (interactive)
  (let ((counts (make-hash-table :test 'equal))
        (today (format-time-string "%Y-%m-%d" (current-time)))
        values output)
    (org-map-entries
     (lambda ()
       (let* ((status (elt (org-heading-components) 2)))
         (when status
           (puthash status (1+ (or (gethash status counts) 0)) counts))))
     nil
     'agenda)
    (setq values (mapcar (lambda (x)
                           (or (gethash x counts) 0))
                         '("DONE" "STARTED" "TODO" "WAITING" "DELEGATED" "CANCELLED" "SOMEDAY")))
    (setq output
          (concat "| " today " | "
                  (mapconcat 'number-to-string values " | ")
                  " | "
                  (number-to-string (apply '+ values))
                  " | "
                  (number-to-string
                   (round (/ (* 100.0 (car values)) (apply '+ values))))
                  "% |"))
    (if (called-interactively-p 'any)
        (insert output)
      output)))
(sacha/org-count-tasks-by-status)

I ran this code over several days. Here are my results as of 2014-05-01:

Date DONE START. TODO WAIT. DELEG. CANC. SOMEDAY Total % done + done +canc. + total + t – d – c Note
2014-04-16 1104 1 403 3 1 104 35 1651 67%
2014-04-17 1257 0 114 4 1 171 107 1654 76% 153 67 3 -217 Lots of trimming
2014-04-18 1292 0 74 4 5 183 100 1658 78% 35 12 4 -43 A little bit more trimming
2014-04-20 1305 0 80 4 5 183 100 1677 78% 13 0 19 6
2014-04-21 1311 1 78 4 4 184 99 1681 78% 6 1 4 -3
2014-04-22 1313 2 75 4 4 184 99 1681 78% 2 0 0 -2
2014-04-23 1369 4 66 4 5 186 101 1735 79% 56 2 54 -4 Added sharing/index.org
2014-04-24 1371 3 69 4 5 186 101 1739 79% 2 0 4 2
2014-04-25 1379 3 60 3 5 189 103 1742 79% 8 3 3 -8
2014-04-26 1384 3 65 3 5 192 103 1755 79% 5 3 13 5
2014-04-27 1389 2 66 3 5 192 103 1760 79% 5 0 5 0
2014-04-28 1396 3 67 3 5 192 103 1769 79% 7 0 9 2
2014-04-29 1396 3 67 3 5 192 103 1769 79% 0 0 0 0
2014-04-30 1404 4 70 4 5 192 103 1782 79% 8 0 13 5
2014-05-01 1413 4 80 3 4 193 103 1800 79% 9 1 18 8

Here’s the source for that table:

#+NAME: burndown
#+RESULTS:
|       Date | DONE | START. | TODO | WAIT. | DELEG. | CANC. | SOMEDAY | Total | % done | + done | +canc. | + total | + t - d - c | Note                       |
|------------+------+--------+------+-------+--------+-------+---------+-------+--------+--------+--------+---------+-------------+----------------------------|
| 2014-04-16 | 1104 |      1 |  403 |     3 |      1 |   104 |      35 |  1651 |    67% |        |        |         |             |                            |
| 2014-04-17 | 1257 |      0 |  114 |     4 |      1 |   171 |     107 |  1654 |    76% |    153 |     67 |       3 |        -217 | Lots of trimming           |
| 2014-04-18 | 1292 |      0 |   74 |     4 |      5 |   183 |     100 |  1658 |    78% |     35 |     12 |       4 |         -43 | A little bit more trimming |
| 2014-04-20 | 1305 |      0 |   80 |     4 |      5 |   183 |     100 |  1677 |    78% |     13 |      0 |      19 |           6 |                            |
| 2014-04-21 | 1311 |      1 |   78 |     4 |      4 |   184 |      99 |  1681 |    78% |      6 |      1 |       4 |          -3 |                            |
| 2014-04-22 | 1313 |      2 |   75 |     4 |      4 |   184 |      99 |  1681 |    78% |      2 |      0 |       0 |          -2 |                            |
| 2014-04-23 | 1369 |      4 |   66 |     4 |      5 |   186 |     101 |  1735 |    79% |     56 |      2 |      54 |          -4 | Added sharing/index.org    |
| 2014-04-24 | 1371 |      3 |   69 |     4 |      5 |   186 |     101 |  1739 |    79% |      2 |      0 |       4 |           2 |                            |
| 2014-04-25 | 1379 |      3 |   60 |     3 |      5 |   189 |     103 |  1742 |    79% |      8 |      3 |       3 |          -8 |                            |
| 2014-04-26 | 1384 |      3 |   65 |     3 |      5 |   192 |     103 |  1755 |    79% |      5 |      3 |      13 |           5 |                            |
| 2014-04-27 | 1389 |      2 |   66 |     3 |      5 |   192 |     103 |  1760 |    79% |      5 |      0 |       5 |           0 |                            |
| 2014-04-28 | 1396 |      3 |   67 |     3 |      5 |   192 |     103 |  1769 |    79% |      7 |      0 |       9 |           2 |                            |
| 2014-04-29 | 1396 |      3 |   67 |     3 |      5 |   192 |     103 |  1769 |    79% |      0 |      0 |       0 |           0 |                            |
| 2014-04-30 | 1404 |      4 |   70 |     4 |      5 |   192 |     103 |  1782 |    79% |      8 |      0 |      13 |           5 |                            |
| 2014-05-01 | 1413 |      4 |   80 |     3 |      4 |   193 |     103 |  1800 |    79% |      9 |      1 |      18 |           8 |                            |
#+TBLFM: @3$11..@>$11=$2-@-1$2::@3$13..@>$13=$9-@-1$9::@3$14..@>$14=$13-$11-($7-@-1$7)::@3$12..@>$12=$7-@-1$7

I wanted to graph this with Gnuplot, but it turns out that Gnuplot is difficult to integrate with Emacs on Microsoft Windows. I gave up after a half an hour of poking at it, since search results indicated there were long-standing problems with how Gnuplot got input from Emacs. Besides, I’d been meaning to learn more R anyway, and R is more powerful when it comes to statistics and data visualization.

Getting R to work with Org Mode babel blocks in Emacs on Windows was a challenge. Here are some of the things I ran into.

The first step was easy: Add R to the list of languages I could evaluate in a source block (I already had dot and ditaa from previous experiments).

(org-babel-do-load-languages
 'org-babel-load-languages
 '((dot . t)
   (ditaa . t) 
   (R . t)))

But my code didn’t execute at all, even when I was trying something that printed out results instead of drawing images. I got a little lost trying to dig into org-babel-execute:R with edebug, eventually ending up in comint.el. The real solution was even easier. I had incorrectly set inferior-R-program-name to the path of R in my configuration, which made M-x R work but which meant that Emacs was looking in the wrong place for the options to pass to R (which Org Babel relied on). The correct way to do this is to leave inferior-R-program-name with the default value (Rterm) and make sure that my system path included both the bin directory and the bin\x64 directory.

Then I had to pick up the basics of R again. It took me a little time to figure out that I needed to parse the columns I pulled in from Org, using strptime to convert the date column and as.numeric to convert the numbers. Eventually, I got it to plot some results with the regular plot command.

dates <- strptime(as.character(data$Date), "%Y-%m-%d")
tasks_done <- as.numeric(data$DONE)
tasks_uncancelled <- as.numeric(data$Total) - as.numeric(data$CANC.)
df <- data.frame(dates, tasks_done, tasks_uncancelled)
plot(x=dates, y=tasks_uncancelled, ylim=c(0,max(tasks_uncancelled)))
lines(x=dates, y=tasks_uncancelled, col="blue", type="o")
lines(x=dates, y=tasks_done, col="green", type="o")

r-plot

I wanted prettier graphs, though. I installed the ggplot2 package and started figuring it out. No matter what I did, though, I ended up with a blank white image instead of my graph. If I used M-x R instead of evaluating the src block, the code worked. Weird! Eventually I found out that adding print(...) around my ggplot made it display the image correctly. Yay! Now I had what I wanted.

library(ggplot2)
dates <- strptime(as.character(data$Date), "%Y-%m-%d")
tasks_done <- as.numeric(data$DONE)
tasks_uncancelled <- as.numeric(data$Total) - as.numeric(data$CANC.)
df <- data.frame(dates, tasks_done, tasks_uncancelled)
plot = ggplot(data=df, aes(x=dates, y=tasks_done, ymin=0)) + geom_line(color="#009900") + geom_point() + geom_line(aes(y=tasks_uncancelled), color="blue") + geom_point(aes(y=tasks_uncancelled))
print(plot)

 r-graph

The blue line represents the total number of tasks (except for the cancelled ones), and the green line represents tasks that are done.

Here’s something that looks a little more like a burn down chart, since it shows just the number of things to be done:

library(ggplot2)
dates <- strptime(as.character(data$Date), "%Y-%m-%d")
tasks_remaining <- as.numeric(data$Total) - as.numeric(data$CANC.) - as.numeric(data$DONE)
df <- data.frame(dates, tasks_remaining)
plot = ggplot(data=df, aes(x=dates, y=tasks_remaining, ymin=0)) + geom_line(color="#009900") + geom_point()
print(plot)

r-graph-2

The drastic decline there is me realizing that I had lots of tasks that were no longer relevant, not me being super-productive. =)

As it turns out, I tend to add new tasks at about the rate that I finish them (or slightly more). I think this is okay. It means I’m working on things that have next steps, and next steps, and steps beyond that. If I add more tasks, that gives me more variety to choose from. Besides, I have a lot of repetitive tasks, so those never get marked as DONE over here.

Anyway, cool! Now that I’ve gotten R to work on my system, you’ll probably see it in even more of these blog posts. =D Hooray for Org Babel and R!

Update 2014-05-09: Stephen suggested http://blogs.neuwirth.priv.at/software/2012/03/28/r-and-emacs-with-org-mode/ for more tips on setting up Org Mode with R and Emacs Speaks Statistics (ESS).

Thinking about my TODO keywords

It’s been twelve years since David Allen published Getting Things Done, with its geek-friendly flowcharts and processes for handling tasks in an interrupt-driven life. The way I manage my tasks is heavily influenced by GTD. I think in terms of next actions, waiting, and someday, and I have weekly reviews. I modified the TODO states a little to reflect what I need. It’s time to think about those states again to see what I can tweak and what reports I could use.

I use Org Mode in Emacs to manage my tasks and my notes. I can customize it to give me different kinds of reports, such as showing me all of my unscheduled tasks, or all tasks with a specific category, or even projects that are “stuck” (no next actions defined). Thinking about my processes will help me figure out what reports I want and how I want to use them.

Here are different types of tasks and how I track them:

  • Things I can work on right now (next actions): TODO
  • Things that I can work on after a different task is finished: currently WAITING, but probably better to implement with org-depend
  • Things I will revisit at a certain date, but I don’t need to think about them until then: TODO, scheduled (I used to use POSTPONED)
  • Things that would be nice to do someday, but maybe are incompletely specified or understood: SOMEDAY
  • Things I have decided not to work on: CANCELLED
  • Things I have asked someone else to do: DELEGATED
  • Things I can ask someone else to do: TODELEGATE
  • Things I am waiting for (usually not based on date) and that I need to follow up on: WAITING
  • Things I can write about: TOBLOG. These are pretty optional, so I don’t want them in my TODO list…
  • If something is a duplicate of something else – remove TODO keyword and add link?

I use the following code for an agenda view of unscheduled tasks:

(defun sacha/org-agenda-skip-scheduled ()
  (org-agenda-skip-entry-if 'scheduled 'deadline 'regexp "\n]+>"))

(add-to-list 'org-agenda-custom-commands
   '("u" "Unscheduled tasks" alltodo ""
     ((org-agenda-skip-function 'sacha/org-agenda-skip-scheduled)
     (org-agenda-overriding-header "Unscheduled TODO entries: "))))

So the to-do process looks like this:

  • Every week, review my evil plans and projects. Check my agenda without the routine tasks to see what new things I’m working on. Schedule a few tasks to encourage me to make regular progress.
  • Every day, go through my Org agenda (C-c a a) and do all the tasks that are scheduled.
  • When I’m done or if I feel like working on something else:
    • What do I feel like doing? If there’s a specific activity that I feel like:
      • Go to the relevant project/section of my TODO list, or check the TODOs by context (drawing, writing, etc.)
      • Clock in on that task.
    • If there’s a specific task I feel like working on:
      • Find the task, maybe with C-u C-c C-w (org-refile) and work on it.
    • If there’s a new idea I want to work on:
      • Use org-capture to create the task, file it in the appropriate project, and then clock in.
  • If I have an idea for a task, use org-capture to create the task and file it in the appropriate project.

How do I want to improve this?

  • Maybe get more used to working with contexts? I have all these Org Agenda commands and I hardly ever use them. I tend to work with projects instead. Actually, working with projects makes sense too, because that minimizes the real context shift.
  • Get better at reviewing existing tasks. I started tracking the number of tasks in each state (DONE, TODO, etc.), which nudged me to review the tasks and cross old tasks off. If I streamline my process for capturing tasks, filing them, and reviewing them by project/context/effort, then I can get better at choosing good tasks to work on from my existing TODO list.
  • Estimate effort for more tasks, and use that more often I have some reports that can filter or sort by estimated effort. I don’t use effort that much, though. Does it makes sense to get into the habit of choosing tasks by estimated time as an alternative approach? I usually have fairly large, flexible blocks of time…
  • Tag things by level of energy required? I want to take advantage of high-energy times. So, when I feel alert and creative, I want to focus on coding and writing. I can save things like paperwork for low-energy times. I can tag some tasks as :lowenergy: and then filter my reports.

Hmm…