Category Archives: org

On this page:

New note-taking workflow with Emacs Org-mode

The new workflow looks like it works better for me. Or rather, it’s an old workflow with new tools. Now, instead of using Windows Live Writer or ScribeFire to post my notes directly to my blog, I’m back to using M-x remember and Emacs, keeping a superset of my notes in text files and publishing selected parts of it.

  • The new workflow
    • M-x remember saves quick notes into a large text file (~/personal/organizer.org), possibly with tags, with diagrams inserted later.
    • I regularly review and file items into the appropriate sections of ~/personal/outline.org.
    • I post selected items to my blog using C-u M-x org2blog-post-subtree, scheduling them by adding a timestamp or using the C-c C-s (org-schedule) command.

    I sometimes use Microsoft OneNote on my new tablet to take notes during meetings, but it’s easy enough to convert my handwriting to text and paste it into my Org-mode file. I still have to think of a better way to refer to images while keeping my file manageable, but a filename is probably okay.

  • A worked example

    This is being composed in a M-x remember window. (Well, remember is bound to C-c r on my system, so it’s easy to invoke).

    After I finish braindumping, I’ll use C-c C-c to save it somewhere.

    I may schedule the post immediately (C-c s (org-schedule) and then C-u M-x org2blog-post-subtree), or tag it for later review. (:toblog: – ready to go, but not scheduled? :rough: – needs more thinking?)

    When I review the items, I’ll copy this into the Geek – Emacs section of my outline.org.

    It feels nice having my notes in plain text, and being able to organize it in more than just chronological order…

  • The history

    From 2001 to about 2006, I kept an Emacs Planner wiki with all of my notes in it. Emacs Remember let me write notes that were automatically hyperlinked to whatever I was looking at, and I added code to Planner that made it easy for me to file the notes both chronologically and topically. Planner rocked. I loved being able to easily hyperlink between topics, and the wiki structure kept pages a mostly manageable size. (My public Planner files are still on the Net, but I need to regenerate the index or enable directory lists so that they’re usable.)

    When I moved to WordPress as a blogging platform in order to make it easier for people to leave comments, I hacked around with RSS to import my posts from Planner into WordPress (ex: http://sachachua.com/blog/2002/). Moving to WordPress meant a change in my workflow. I now had two places to store my notes: Planner and my blog.

    I tried Emacs Org because I liked the way it organized information. In Planner, we’d been struggling with elegant ways to manage tasks and notes that needed to be accessed in multiple contexts. The approach we had taken in Planner was to make copies of the information, but Org had a cleaner way to do it using different views. It was intriguing.

    When I started working at IBM, however, my information workflow diverged. I shifted to using a web-based to-do list and Lotus Notes, posting on an internal blog and an external one, and managing multiple sources and repositories of information.

    I wanted to go back to keeping my notes in plain text, encrypted if necessary, and to have a place where I could keep notes that might not be publishable. I still had to manage multiple computers, but synchronizing systems like Dropbox or SpiderOak got rid of some of the hassles I’d encountered with git. When I found out about org2blog thanks to a test link from punchagan, I modified the code to work with subtrees instead of new buffers, and that solved the blog publishing part of it.

Emacs Org mode and publishing a weekly review

2010-09-11 Sat 08:00

I like using Emacs Org-mode to organize my notes. One of the things it makes it easy to do is to keep a weekly review. I used to switch between using Windows Live Writer and using Emacs Org to draft the post, but with org2blog, I’ve been using Org more and more. Here’s how I use it.

At the beginning of my ~/personal/organizer.org, I have a headline for * Weekly review. Underneath it is a template that makes it easy for me to review my current projects and make sure that I’ve got next actions for each of them. Below that is a reverse-chronological list of weekly reviews, with the most recent weekly review first. This allows me to easily review my weekly priorities and copy that into a new entry. Here’s what the first part of my Org file looks like (minus the spaces at the beginning of the line)

* Weekly review
** Template
*** Plans for next week
**** Work
- [ ] *Support Classroom to Client:*
- [ ] *Build Connections Toolkit:*
- [ ] *Organize Idea Labs:*
- [ ] *Build career:*
**** Relationships
- [ ] *Plan Wedding:*
**** Life
- [ ] *Sew dress:*
- [ ] *Improve productivity:*
** Week ending September 12, 2010
*** From last week's plans
**** Work
- [X] *Classroom to Client:* Create community and structure online resoruces
- [X] *Connections Toolkit:* Build Activities reporter
- [X] *Classroom to Client:* Format Idea Lab reference presentation
- [X] *Idea Labs:* Assist with planning, process RSVPs
- [X] *Career:* Set up Ruby on Rails
- Helped Darrel Rader with blog feed
- Helped Sunaina with Notes e-mail conversion
- Finalized Idea Lab reference
- Had great conversation with Boz, Rooney, Kieran, etc. about culture and sharing
- Followed up on expertise location, sent draft report
- Collected interesting Lotus Connections practices into a presentation
- Put together match-up slide for IBM acquisitions
**** Relationships
- [X] *Wedding:* Plan NYC trip
**** Life
- [ ] *Sew dress:* Transfer dots and mark stitching lines
- [X] *Chair:* Paint and assemble chair
- [X] *Productivity:* Tweak GTD process - use Org for my weekly review/project template
- [X] *Productivity:* Organize files
- Added weekly lifestream archive
**** Plans for next week
***** Work
- [ ] *Support Classroom to Client:* Collect lessons learned and create new material
- [ ] *Build Connections Toolkit:* Make GUI
- [ ] *Organize Idea Labs:* Update invitation template
- [ ] *Build career:* Go through Ruby on Rails tutorials
- [ ] *Build career:* Prototype Drupal site and learn about new practices along the way
- [ ] *Build career:* Mentor people
***** Relationships
- [ ] *Plan wedding:* Plan BBQ reception
- [ ] *Plan wedding:* Make checklist and timeline for cleaning up, etc.
***** Life
- [ ] *Sew dress:* Machine-baste pieces together
- [ ] *Improve productivity:* File inbox items from my Org file

Most of the time, I leave the template section collapsed, and the “Plans from last week” expanded. Throughout the week, I cross items off and add quick notes about other accomplishments. When I reach the next week, I create a new entry, move the “Plans for next week” subtree and rename it “From last week’s plans”. When I do my weekly review (or throughout the week, as I notice new items), I create a “Plans for next week” section and fill it in. The editing can easily be automated, but I’ll tinker with it a bit first before writing code.

This approach means duplicate information in my task list. It would be interesting to use TODO items instead of list items for tracking my weekly priorities, with possible integration with my web-based task list through org-toodledo. However, I’d need to write code to make the TODO items publish as neatly as this list gets published using org2blog, and I don’t feel like going into that yet.

Anyway, that’s how I’m currently doing it. =)

Org-toodledo

I finally got around to asking my manager for permission to contribute org-toodledo as open source. Here it is. Enjoy!

;;; org-toodledo.el - Toodledo integration for Emacs Org mode
;; (c) 2010 Sacha Chua ([email protected])
;;
;; This file is not part of GNU Emacs.

;; This is free software; you can redistribute it and/or modify it under
;; the terms of the GNU General Public License as published by the Free
;; Software Foundation; either version 2, or (at your option) any later
;; version.
;;
;; This is distributed in the hope that it will be useful, but WITHOUT
;; ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
;; FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
;; for more details.
;;
;; You should have received a copy of the GNU General Public License
;; along with GNU Emacs; see the file COPYING.  If not, write to the
;; Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston,
;; MA 02111-1307, USA.

;; How to use:
;; 1. Customize org-toodledo-userid and org-toodledo-password
;; 2. Open a blank org file.
;; 3. Call org-toodledo-initialize-org
;; Call org-toodledo-update to bring in new/updated tasks (skips locally modified tasks newer than updated)
;; Call org-toodledo-sync-task to create or update the current task
;; Call org-toodledo-delete-current-task to delete the current task
;;
;; Doesn't do lots of error trapping. Might be a good idea to version-control your Org file.
;;
;; TOODLEDO ATTRIBUTES and how they are bi-directionally handled
;; Context: Handled by tags (ex:   :@work:  :@errands:)
;;   - will create new contexts if necessary
;; Task status: Mapped to TODO state.
;;   See org-toodledo-status-to-string and org-toodledo-parse-current-task for the mapping
;;   You will probably want something like this in your ~/.emacs:
;; (setq org-todo-keywords
;;      '((sequence
;;         "TODO(t)"  ; next action
;;         "PLAN(-)"
;;         "STARTED(s)"
;;         "WAITING(w@/!)"
;;         "POSTPONED(p)" "SOMEDAY(s@/!)" "|" "DONE(x!)" "CANCELLED(c@)")
;;      (type "DELEGATED(d@!)" "DONE(x)")))
;; Length: Mapped to effort
;; Priority: Mapped to [#A], [#B], or [#C]. (TODO: Change this to five levels of priority to match Toodledo)
;; Start date: Mapped to "SCHEDULED"
;; Due date: Mapped to "DEADLINE"
;; Tags: Mapped to tags
;; Note: Mapped to todo text. May get confused by asterisks, so don't use any starting asterisks in your body text.
;;   (or anything that looks like an Org headline).
;; Completed: Mapped to DONE todo state.
;;
;; TODO:
;; - [ ] Double-check new/changed/deleted task updating, still seems buggy
;; - [ ] Test, test, test - maybe make test harness?
;; - [ ] Move status<->string mapping to a variable - lookups are better than logic
;; - [ ] Make sure sync timestamps aren't getting updated more often than needed
;; - [ ] Suggest some kind of hook to make it easier to mark a task as locally modified

(require 'org)
(require 'w3m)
(require 'xml)
(defcustom org-toodledo-userid ""
  "UserID from Toodledo: http://www.toodledo.com/info/api_doc.php"
  :group 'org-toodledo
  :type 'string)

(defcustom org-toodledo-password ""
  "Password for Toodledo."
  :group 'org-toodledo
  :type 'string)

(defvar org-toodledo-token-expiry nil "Expiry time for authentication token.")
(defvar org-toodledo-token nil "Authentication token.")
(defvar org-toodledo-key nil "Authentication key.")

(require 'url)
(require 'url-http)

(defun org-toodledo-initialize-org ()
  "Replace buffer contents with Toodledo tasks."
  (interactive)
  (delete-region (point-min) (point-max))
  (let ((account-info (org-toodledo-get-account-info))
        (server-info (org-toodledo-get-server-info))
        (tasks (org-toodledo-get-tasks '(("notcomp" . "1")))))
    (insert "* Toodledo\n"
            ":PROPERTIES:\n"
            ":Last-modified: " (cdr (assoc "lastaddedit" account-info)) "\n"
            ":Last-deleted: " (cdr (assoc "lastdelete" account-info)) "\n"
            ":Last-sync: " (cdr (assoc "unixtime" server-info)) "\n"
            ":END:\n")
    (insert (mapconcat 'org-toodledo-task-to-string tasks "\n"))))

(defun org-toodledo-get-token ()
  "Retrieve authentication token valid for four hours."
  (if (and org-toodledo-token
           org-toodledo-token-expiry
           (time-less-p (current-time) org-toodledo-token-expiry))
      org-toodledo-token
    ;; Else retrieve a new token
    (let ((response
            (with-current-buffer
                (url-retrieve-synchronously
                 (concat "http://api.toodledo.com/api.php?method=getToken;userid="
                         org-toodledo-userid))
              (xml-parse-region (point-min) (point-max)))))
      (if (equal (car (car response)) 'error)
          (progn
            (setq org-toodledo-token nil
                  org-toodledo-key nil
                  org-toodledo-token-expiry nil)
            (error "Could not log in to Toodledo: %s" (elt (car response) 2)))
        (setq org-toodledo-token
              (elt (car response) 2))
        (setq org-toodledo-key (org-toodledo-key)
              ;; Set the expiry time
              org-toodledo-token-expiry
              (seconds-to-time
               (+ (time-to-seconds (current-time))
                  (* 60 60 4)))))   ;; four hours
      org-toodledo-token)))

(defun org-toodledo-key ()
  "Return authentication key used for each request."
  (if (and org-toodledo-token
           org-toodledo-token-expiry
           (time-less-p (current-time) org-toodledo-token-expiry)
           org-toodledo-key)
      org-toodledo-key
    (setq org-toodledo-key
          (md5 (concat (md5 org-toodledo-password)
                       org-toodledo-token
                       org-toodledo-userid)))))

(defun org-toodledo-get-url (method-name &optional params)
  "Return URL for METHOD-NAME and PARAMS."
  (org-toodledo-get-token)
  (concat "http://api.toodledo.com/api.php?method="
          (w3m-url-encode-string method-name)
          ";key=" (org-toodledo-key)
          (if params
              (concat
               ";"
               (mapconcat (lambda (x)
                            (concat
                             (w3m-url-encode-string (car x)) "="
                             (w3m-url-encode-string (cdr x))))
                          params
                          ";"))
            "")))

(defun org-toodledo-call-method (method-name &optional params)
  "Call METHOD-NAME with PARAMS and return the parsed XML."
  (setq params (cons (cons "unix" "1") params))
  (with-current-buffer
      (url-retrieve-synchronously
       (org-toodledo-get-url method-name params))
    (xml-parse-region (point-min) (point-max))))

(defmacro org-toodledo-defun (function-name api-name description)
  `(defun ,function-name (params)
     ,description
     (org-toodledo-call-method ,api-name params)))

(defun org-toodledo-get-server-info ()
  "Return server information."
  (org-toodledo-convert-xml-result-to-alist
    (car (org-toodledo-call-method "getServerInfo"))))

(defun org-toodledo-get-account-info ()
  "Return server information."
  (org-toodledo-convert-xml-result-to-alist
   (car (org-toodledo-call-method "getAccountInfo"))))

(org-toodledo-defun org-toodledo-add-task "addTask" "Add task with PARAMS.")
(org-toodledo-defun org-toodledo-edit-task "editTask" "Edit task with PARAMS.")
(org-toodledo-defun org-toodledo-delete-task "deleteTask" "Delete task with PARAMS.")

;; (setq temp (org-toodledo-get-tasks '(("notcomp" . "1"))))
;; (setq server-info (org-toodledo-get-server-info))
;; (setq account-info (org-toodledo-get-account-info))
(defun org-toodledo-convert-xml-result-to-alist (info)
  "Convert INFO to an alist."
  (delq nil
        (mapcar
         (lambda (item)
           (if (listp item)
               (cons (symbol-name (car item)) (elt item 2))))
         (xml-node-children (delete "\n\t" info)))))

(defun org-toodledo-get-tasks (&optional params)
  "Retrieve tasks using PARAMS.
Return a list of task alists."
  (mapcar
   'org-toodledo-convert-xml-result-to-alist
   (xml-get-children
    (car (org-toodledo-call-method "getTasks" params))
    'task)))

(defun org-toodledo-get-deleted (&optional params)
  "Retrieve deleted tasks using PARAMS.
Return a list of task alists."
  (mapcar
   'org-toodledo-convert-xml-result-to-alist
   (xml-get-children
    (car (org-toodledo-call-method "getDeleted" params))
    'task)))

(defun org-toodledo-entry-note ()
  "Extract the note for this entry."
  (save-excursion
    (org-back-to-heading)
    (when (looking-at org-complex-heading-regexp)
      (goto-char (match-end 0))
      (let ((text (buffer-substring-no-properties
                   (point)
                   (if (re-search-forward org-complex-heading-regexp nil t)
                       (match-beginning 0)
                     (org-end-of-subtree)))))
        (with-temp-buffer
          (insert text)
          (goto-char (point-min))
          (when (re-search-forward
                 (concat "\\<"
                         (regexp-quote org-deadline-string) " +<[^>\n]+>[ \t]*") nil t)
            (replace-match ""))
          (goto-char (point-min))
          (when (re-search-forward
                 (concat "\\<"
                         (regexp-quote org-scheduled-string) " +<[^>\n]+>[ \t]*") nil t)
            (replace-match ""))
          (goto-char (point-min))
          (while (re-search-forward "\n\n+" nil t)
            (replace-match "\n"))
          (org-export-remove-or-extract-drawers org-drawers nil nil)
          (buffer-substring-no-properties (point-min)
                                          (point-max)))))))

(defun org-toodledo-parse-current-task ()
  "Extract the status and Toodledo ID of the current task."
  (save-excursion
    (org-back-to-heading t)
    (when (and (looking-at org-complex-heading-regexp)
               (match-string 2)) ;; TODO
      (let* (info
             (status (match-string-no-properties 2))
             (priority (match-string-no-properties 3))
             (title (match-string-no-properties 4))
             (tags (match-string-no-properties 5))
             (id (org-entry-get (point) "Toodledo-ID"))
             (contexts (org-toodledo-get-contexts))
             context)
        ;; (add-to-list 'info (cons "title" (match-string-no-properties 1)))
        (if id (add-to-list 'info (cons "id" id)))
        (when tags
          (setq tags
              (delq nil
                    (mapcar
                     (lambda (tag)
                       (if (> (length tag) 0)
                           (if (string-match (org-re "@\\([[:alnum:]_]+\\)") tag)
                               (progn
                                 ;; Not recognized context
                                 (if (null (assoc (match-string 1 tag) contexts))
                                     ;; Create it if it does not yet exist
                                     (let ((result
                                            (org-toodledo-call-method
                                             "addContext"
                                             (list (cons "title" (match-string 1 tag))))))
                                       (if (eq (caar result) 'added)
                                           (setq org-toodledo-contexts
                                                 (cons (cons (match-string 1 tag)
                                                             (elt (car result) 2))
                                                       org-toodledo-contexts)
                                                 contexts org-toodledo-contexts))))
                                   ;; Get the ID of the context
                                 (setq context
                                       (cdr (assoc (match-string 1 tag) contexts)))
                                 nil)
                             tag)))
                     (split-string tags ":")))))
        (setq info
              (list
               (cons "id" id)
               (cons "title" title)
               (cons "length" (org-entry-get (point) "Effort"))
               (cons "context" context) 
               (cons "tag" (mapconcat 'identity tags " "))
               (cons "completed" (if (equal status "DONE") "1" "0"))
               (cons "status"
                     (cond 
                      ((equal status "STARTED") "2")
                      ((equal status "DELEGATED") "4")
                      ((equal status "SOMEDAY") "8")
                      ((equal status "CANCELLED") "9")
                      ((equal status "PLAN") "3")
                      ((equal status "WAITING") "5")                      
                      ((equal status "TODO") "1")))
               (cons "priority"
                     (cond
                      ((equal priority "[#A]") "2")
                      ((equal priority "[#B]") "1")
                      ((equal priority "[#C]") "0")))
               (cons "note"
                     (org-toodledo-entry-note))))
        (when (org-entry-get nil "DEADLINE")
          (setq info (cons (cons "duedate"
                                 (substring (org-entry-get nil "DEADLINE")
                                            0 10)) info)))
        (when (org-entry-get nil "SCHEDULED")
          (setq info (cons (cons "startdate"
                                 (substring (org-entry-get nil "SCHEDULED")
                                            0 10)) info)))
        info))))

(defun org-toodledo-sync ()
  "Synchronize all tasks."
  ;; Retrieve all tasks
  ;; For each task in the current buffer
  ;;   Synchronize an existing task that has changed
   (let ((regexp (concat "^\\*+[ \t]+\\(" org-todo-regexp "\\)")))
    (goto-char (point-min))
    (while (re-search-forward regexp nil t)
      (org-toodledo-sync-task))))

(defun org-toodledo-update ()
  "Insert new tasks and update previous tasks."
  (interactive)
  (let* ((server-info (org-toodledo-get-server-info))
         (account-info (org-toodledo-get-account-info))
         (changed (org-toodledo-account-changed account-info))
         (last-deleted (string-to-number (or (org-entry-get-with-inheritance "Last-deleted") "0")))
         (last-modified (string-to-number (or (org-entry-get-with-inheritance "Last-modified") "0")))
         (last-update (string-to-number (or (org-entry-get-with-inheritance "Last-sync") "0")))
         processed)
    ;; If tasks have been deleted or modified, then the Toodledo API
    ;; will give us the timestamps. We need to find out which tasks
    ;; have been deleted or modified since the last time we retrieved
    ;; the list of tasks that have been deleted or modified. We store
    ;; the last times in the properties of the root element.
    
    (if (and (assoc "deleted" changed) ;; Tasks have been deleted
             (>= (string-to-number (cdr (assoc "deleted" changed))) last-deleted))
        (setq processed
              (append (org-toodledo-process-deleted-tasks
                       last-deleted)
                       processed)))
    (if (and (assoc "modified" changed) ;; Tasks have been added or edited
             (>= (string-to-number (cdr (assoc "modified" changed)))
                last-modified))
        ;; Retrieve added/modified tasks
        (setq processed (append
                         (org-toodledo-process-modified-tasks last-modified) processed)))
    ;; TODO Look for tasks that were modified locally since the last synchronization
    (org-toodledo-process-locally-modified-tasks last-update processed)
    ;; TODO Update timestamps here
    (goto-char (point-min))
    (when (re-search-forward (concat "^\\(" outline-regexp "\\)") nil t)
      (org-entry-put (point)
                     "Last-sync"
                     (cdr (assoc "unixtime" server-info)))
      (when (assoc "lastaddedit" account-info)
        (org-entry-put (point)
                       "Last-modified"
                       (cdr 
                        (assoc "lastaddedit" account-info))))
      (when (assoc "lastdelete" account-info)
        (org-entry-put (point)
                         "Last-deleted"
                         (cdr
                          (assoc "lastdelete" account-info)))))))

(defun org-toodledo-process-locally-modified-tasks (last-update processed)
  "Synchronize tasks that were locally modified after LAST-UPDATE.
Skip tasks with IDs in PROCESSED."
  (goto-char (point-min))
  (let ((start (float-time (current-time))))
    (while (re-search-forward org-complex-heading-regexp nil t)
      ;; Look for all tasks in this buffer
      (if (match-string 2)
          ;; Is it a new task, or has it been modified since the last update?
          (let ((id (org-entry-get (point) "Toodledo-ID"))
                (modified (string-to-number (or (org-entry-get (point) "Modified") "")))
                (last-sync (if (org-entry-get (point) "Sync")
                               (string-to-number (org-entry-get (point) "Sync"))
                             0)))
            (if (or (null id)
                    (and (> modified last-sync)
                         (< modified start)
                         (not (member id processed))))
                (save-excursion (org-toodledo-sync-task))))))))

(defun org-toodledo-touch ()
  "Update the current task."
  (interactive)
  (org-entry-put (point) "Modified" (format "%d" (float-time (current-time)))))

(defvar org-toodledo-actually-delete t)
(defun org-toodledo-process-deleted-tasks (timestamp)
  "Remove tasks deleted after TIMESTAMP."
  (delq nil
        (mapcar
         (lambda (task)
           (when (org-toodledo-find-task task)
             (if org-toodledo-actually-delete
                 (delete-region (org-back-to-heading)
                                (if (re-search-forward org-complex-heading-regexp nil t)
                                    (match-beginning 0)
                                  (org-end-of-subtree)))
               (org-entry-delete (point) "Toodledo-ID")
               (org-entry-put (point) "Toodledo-Deleted" (timestamp)))
             (org-toodledo-task-id task)))
         (org-toodledo-get-deleted
          (list (cons "after" (number-to-string timestamp)))))))
  
(defun org-toodledo-process-modified-tasks (modified)
  "Handle all the tasks that have been modified since MODIFIED."
  (delq nil
        (mapcar
         (lambda (task)
           (if (org-toodledo-find-task task)
               (if (null (org-toodledo-update-task task modified))
                   (org-toodledo-task-id task))
             (org-toodledo-create-task task)))
         (org-toodledo-get-tasks (list (cons "modafter" (number-to-string modified)))))))



(defun org-toodledo-create-task (task)
  "Create a task for TASK."
  (goto-char (point-max))
  (if (point-at-eol) (insert "\n"))
  (insert (org-toodledo-task-to-string task))
  (org-toodledo-task-id task))

(defun org-toodledo-find-task (task)
  "Find the task specified by TASK."
  (goto-char (point-min))
  (re-search-forward
   (concat "^[ \t]*:Toodledo-ID:[ \t]+" (org-toodledo-task-id task) "$")
   nil t))
  
(defun org-toodledo-account-changed (account-info)
  "Return non-nil if the account has changed since the last check.
The result will be an alist of (\"modified\" . \"timestamp\") if tasks have
been added/edited and (\"deleted\" . \"timestamp\") if tasks have been deleted."
  (let ((last-modified (org-entry-get-with-inheritance "Last-modified"))
        (last-deleted (org-entry-get-with-inheritance "Last-deleted"))
        result)
    (if (> (string-to-number (or (cdr (assoc "lastaddedit" account-info)) "0"))
           (string-to-number (or last-modified "0")))
        (add-to-list 'result (cons "modified" last-modified)))
    (if (> (string-to-number (or (cdr (assoc "lastdelete" account-info)) ""))
           (string-to-number (or last-deleted "0")))
        (add-to-list 'result (cons "deleted" last-deleted)))
    result))
  
(defun org-toodledo-sync-task (&optional force)
  "Update my Toodledo for the current task."
  (interactive "P")
  (save-excursion
    (let ((task (org-toodledo-parse-current-task)))
      (if (null (org-toodledo-task-id task))
          ;; New task, create it
          (let ((result (org-toodledo-add-task task)))
            (when (eq (elt (car result) 0) 'added)
              (org-entry-put (point) "Toodledo-ID" (elt (car result) 2))
              (org-entry-put (point) "Sync"
                             (format "%d" (float-time (current-time)) 1000))))
        ;; Old task, update
        (when (org-toodledo-success-p (org-toodledo-edit-task task))
          (if (equal (org-toodledo-task-completed task) "1")
              (org-entry-put (point) "Completed" "1")
            (org-entry-put (point) "Status" (org-toodledo-task-status task)))
          (org-entry-put (point) "Sync"
                         (format "%d" (float-time (current-time)) 1000)))))))

;; (assert (equal (org-toodledo-format-date "2003-08-12") "<2003-08-12 Tue>"))
(defun org-toodledo-format-date (date &optional repeat)
  "Return yyyy-mm-dd day for DATE."
  (concat
   "<"
   (format-time-string
    "%Y-%m-%d %a"
    (cond
     ((listp date) date)
     ((numberp date) (seconds-to-time date))
     ((and (stringp date)
           (string-match "^[0-9]+$" date))
      (seconds-to-time (string-to-number date)))
     (t (apply 'encode-time (org-parse-time-string date)))))
   (if repeat (concat " " repeat) "")
   ">"))

;; (mapconcat 'org-toodledo-task-to-string temp "\n")
;; (setq task (elt temp 2))
;; (org-toodledo-task-to-string task)
(defun org-toodledo-task-to-string (task &optional level)
  "Return an Org-formatted version of TASK."
  (let* ((repeat (string-to-number (org-toodledo-task-repeat task)))
         (rep-advanced (org-toodledo-task-repeat-advanced task))
         (repeat-string (org-toodledo-repeat-to-string repeat rep-advanced))
         (priority (org-toodledo-task-priority task)))
    (concat
     (make-string (or level 2) ?*) " "
     (org-toodledo-status-to-string task) " "
     (cond
      ((equal priority "-1") "")
      ((equal priority "0") "[#C] ")
      ((equal priority "1") "[#B] ")
      ((equal priority "2") "[#A] ")
      ((equal priority "3") "[#A] "))
     (org-toodledo-task-title task)
     (if (org-toodledo-task-context task)
         (concat " :@" (org-toodledo-task-context task) ":") 
       "")
     "\n"
     (if (and (org-toodledo-task-duedate task)
              (not (equal (org-toodledo-task-duedate task) ""))
              (not (< (string-to-number (org-toodledo-task-duedate task)) 0)))
         (concat org-deadline-string " "
                 (org-toodledo-format-date
                  (org-toodledo-task-duedate task)
                  repeat-string)
                 "\n")
       "")
     (or (org-toodledo-task-note task) "") "\n"
     ":PROPERTIES:\n"
     ":Toodledo-ID: " (org-toodledo-task-id task) "\n"
     ":Modified: " (org-toodledo-task-modified task) "\n"
     ":Sync: " (format "%d" (float-time (current-time))) "\n"
     ":Effort: " (org-toodledo-task-length task) "\n"
     ":END:\n"
     )))

;; (assert (equal (org-toodledo-repeat-to-string 0) ""))
;; (assert (equal (org-toodledo-repeat-to-string 1) "+1w"))
;; (assert (equal (org-toodledo-repeat-to-string 2) "+1m"))
;; (assert (equal (org-toodledo-repeat-to-string 3) "+1y"))
;; (assert (equal (org-toodledo-repeat-to-string 4) "+1d"))
;; (assert (equal (org-toodledo-repeat-to-string 5) "+2w"))
;; (assert (equal (org-toodledo-repeat-to-string 6) "+2m"))
;; (assert (equal (org-toodledo-repeat-to-string 7) "+6m"))
;; (assert (equal (org-toodledo-repeat-to-string 8) "+3m"))
;; (assert (equal (org-toodledo-repeat-to-string 108) ".+3m"))
;; (assert (equal (org-toodledo-repeat-to-string 101) ".+1w"))
;; (assert (equal (org-toodledo-repeat-to-string 0) ""))

(defconst org-toodledo-repeat-intervals '("" "+1w" "+1m" "+1y" "+1d" "+2w" "+2m" "+6m" "+3m"))
(defun org-toodledo-status-to-string (task)
  (let ((comp (org-toodledo-task-completed task))
        (status (string-to-number (org-toodledo-task-status task))))
    (cond
     ((not (or (null comp) (equal comp "") (equal comp "0"))) "DONE")
     ((= status 0) "TODO")
     ((= status 1) "TODO")
     ((= status 2) "STARTED")
     ((= status 3) "PLAN")
     ((= status 4) "DELEGATED")
     ((= status 5) "WAITING")
     ((= status 6) "PLAN")  ; hold
     ((= status 7) "SOMEDAY")  ; postponed
     ((= status 8) "SOMEDAY")
     ((= status 9) "CANCELLED")
     )))

(defun org-toodledo-repeat-to-string (repeat &optional rep-advanced)
  "Turn TASK into a repeat sequence."
  (cond
   ((= repeat 0) nil)
   ((> repeat 100) (concat "+" (org-toodledo-repeat-to-string (mod repeat 100) rep-advanced)))
   ((and (= repeat 50) rep-advanced)
    (cond
     ((string-match "Every \\([0-9]+\\) week" rep-advanced)
      (concat "+" (match-string 1 rep-advanced) "w"))
     ((string-match "Every \\([0-9]+\\) month" rep-advanced)
      (concat "+" (match-string 1 rep-advanced) "m"))
     ((string-match "Every \\([0-9]+\\) year" rep-advanced)
      (concat "+" (match-string 1 rep-advanced) "y"))
     ((string-match "Every \\([0-9]+\\) day" rep-advanced)
      (concat "+" (match-string 1 rep-advanced) "d"))
     (t rep-advanced)))
   (t (elt org-toodledo-repeat-intervals repeat))))

(defun org-toodledo-delete-current-task ()
  "Delete the current task."
  (interactive)
  (org-back-to-heading t)
  (let ((task (org-toodledo-parse-current-task)))
    (and (> (length (org-toodledo-task-id task)) 0)
         (org-toodledo-success-p (org-toodledo-delete-task task)))
    (delete-region
     (point)
     (if (and (end-of-line)
              (re-search-forward org-complex-heading-regexp nil t))
         (match-beginning 0)
       (org-end-of-subtree t t)
       (point)))))

  
(defun org-toodledo-task-get-prop (task prop) (cdr (assoc prop task)))
(defmacro org-toodledo-task-prop-defun (field)
  `(defun ,(intern (concat "org-toodledo-task-" field)) (task)
     (cdr (assoc ,field task))))

(defun org-toodledo-success-p (result)
  "Return non-nil if RESULT indicates success."
  (eq (car (car result)) 'success))
        
(org-toodledo-task-prop-defun "id")
(org-toodledo-task-prop-defun "title")
(org-toodledo-task-prop-defun "status")
(org-toodledo-task-prop-defun "completed")
(org-toodledo-task-prop-defun "repeat")
(org-toodledo-task-prop-defun "context")
(org-toodledo-task-prop-defun "duedate")
(org-toodledo-task-prop-defun "modified")
(org-toodledo-task-prop-defun "priority")
(org-toodledo-task-prop-defun "note")
(org-toodledo-task-prop-defun "length")
;; defun'd separately because of the change in name
(defun org-toodledo-task-repeat-advanced (task)
  (cdr (assoc "rep_advanced" task)))

(defvar org-toodledo-contexts nil "An alist of (context . id).")
(defun org-toodledo-get-contexts (&optional force)
  "Store an alist of (context . id) in `org-toodledo-contexts'.
Reload if FORCE is non-nil."
  (if (or force (null org-toodledo-contexts))
      (setq org-toodledo-contexts
            (mapcar
             (lambda (node)
               (cons
              (car (xml-node-children node))
              (xml-get-attribute node 'id)))
             (xml-get-children (car
                                (org-toodledo-call-method "getContexts")) 'context)))
    org-toodledo-contexts))

(defun org-toodledo-agenda-touch ()
  "Update the Modified timestamp for the current entry in the agenda."
  (org-agenda-check-type t 'agenda 'timeline)
  (org-agenda-check-no-diary)
  (let* ((marker (or (org-get-at-bol 'org-marker)
                     (org-agenda-error)))
         (buffer (marker-buffer marker))
         (pos (marker-position marker)))
    (org-with-remote-undo buffer
     (with-current-buffer buffer
       (widen)
       (goto-char pos)
       (if (org-entry-get (point) "Modified")
           (org-entry-put (point) "Modified" (format "%d" (float-time (current-time)))))))))


(defun org-toodledo-update-task (task &optional last-update)
  (let* ((modified (string-to-number (or (org-entry-get (point) "Modified") "")))
         (last-sync (if (org-entry-get (point) "Sync")
                        (string-to-number (org-entry-get (point) "Sync"))
                      0))
         (level (car (org-heading-components)))
         (locally-modified (> modified last-sync)))
    ;; Locally modified? keep
    (if locally-modified
        nil
      ;; Not locally modified? replace
      ;; Figure out what our level is
      (delete-region (org-back-to-heading)
                     (progn (goto-char (match-end 0))
                            (if (re-search-forward org-complex-heading-regexp nil t)
                                (goto-char (match-beginning 0))
                              (org-end-of-subtree))))
      (insert (org-toodledo-task-to-string task level))
      t)))

(provide 'org-toodledo)

Emacs, Org, and BBDB: Hyperlinking names to blogs

Back when I used Planner for Emacs, I coded some shortcuts to make it easier to write about the people I met and the conversations I had. I used the hippie-expand module to complete names from my Big Brother Database addressbook, and I wrote a function that converted those names into links to people’s blogs or websites whenever I published my blog posts as HTML.

I switched to WordPress for my blog because I got tired of trying to figure out a way to enable comments without getting mired in spam-fighting. That meant I could explore other Emacs personal information managers like Org, which I turned into my main task manager. I often used the WordPress interface to write blog posts. I sometimes used Windows Live Writer to write posts about books (there’s a good book review plugin that makes this easy). I also sometimes used Emacs and Org to draft blog posts using Org’s friendlier markup, exporting snippets to HTML that I then pasted into my blog posts.

Reading the posts on Planet Emacsen reminded me that my customized configuration was pretty darn sweet. That and the conversation notes I’ve been blogging lately encouraged me to dust off my configuration files and get them to work under Org. So here’s the code:

 (defun sacha/org-bbdb-get (path)
   "Return BBDB record for PATH."
   (car (bbdb-search (bbdb-records) path path path)))
 
 (defun sacha/org-bbdb-export (path desc format)
   "Create the export version of a BBDB link specified by PATH or DESC.
 If exporting to HTML, it will be linked to the person's blog,
 www, or web address. If exporting to LaTeX FORMAT the link will be
 italicised. In all other cases, it is left unchanged."
     (cond
      ((eq format 'html)
       (let* ((record
             (sacha/org-bbdb-get path))
            url)
       (setq url (and record
                      (or (bbdb-record-getprop record 'blog)
                          (bbdb-record-getprop record 'www)
                          (bbdb-record-getprop record 'web))))
       (if url
           (format "<a href=\"%s\">%s</a>"
                   url (or desc path))
         (format "<em>%s</em>"
                 (or desc path)))))
      ((eq format 'latex) (format "\\textit{%s}" (or desc path)))
      (t (or desc path))))
 
 (defadvice org-bbdb-export (around sacha activate)
   "Override org-bbdb-export."
   (setq ad-return-value (sacha/org-bbdb-export path desc format)))
 
 ;;;_+ Hippie expansion for BBDB; map M-/ to hippie-expand for most fun
 (add-to-list 'hippie-expand-try-functions-list 'sacha/try-expand-bbdb-annotation)
 (defun sacha/try-expand-bbdb-annotation (old)
   "Expand from BBDB. If OLD is non-nil, cycle through other possibilities."
   (unless old
     ;; First time, so search through BBDB records for the name
     (he-init-string (he-dabbrev-beg) (point))
     (when (> (length he-search-string) 0)
       (setq he-expand-list nil)
       (mapcar
        (lambda (item)
        (let ((name (bbdb-record-name item)))
          (when name
            (setq he-expand-list
                  (cons (org-make-link-string
                       (org-make-link "bbdb:" name)
                       name)
                        he-expand-list)))))
        (bbdb-search (bbdb-records)
                     he-search-string
                     he-search-string
                     he-search-string
                     nil nil))))
   (while (and he-expand-list
               (or (not (car he-expand-list))
                   (he-string-member (car he-expand-list) he-tried-table t)))
     (setq he-expand-list (cdr he-expand-list)))
   (if (null he-expand-list)
       (progn
         (if old (he-reset-string))
         nil)
     (progn
       (he-substitute-string (car he-expand-list) t)
       (setq he-expand-list (cdr he-expand-list))
       t)))

If you’ve got Org and BBDB, drop this into your ~/.emacs and fiddle with it. =)

Emacs: Smarter interactive prompts with Org remember templates

Paul Lussier wanted to know how to interactively prompt for a value and use that value multiple times in org-remember-templates. His example is:

(setq org-link-abbrev-alist
      '(("RT" . "https://rt/Ticket/Display.html?id=")))

(setq org-remember-templates
      '(("Tasks" ?t "* TODO %?\n  %i\n  %a"            "~/organizer.org")
        ("Appts" ?a "* Appointment: %?\n%^T\n%i\n  %a" "~/organizer.org")
        ("RT"    ?R "* [[RT:%^{Number}][%^{Number}/%^{Description}]]" "~/org/rt.org")))

The version of Org on my system prompts for Number twice. We want to store the value in an associative list somewhere so that if Org encounters another prompt with the same text, it’ll use the stored value. Here’s a diff that’ll do the trick.

diff -u /home/sachac/elisp/org/org.el /tmp/buffer-content-4571Oz1
--- /home/sachac/elisp/org/org.el	2008-07-20 11:28:54.000000000 -0400
+++ /tmp/buffer-content-4571Oz1	2008-07-20 11:42:06.000000000 -0400
@@ -12822,38 +12822,52 @@
 	    (org-set-local 'org-remember-default-headline headline))
 	;; Interactive template entries
 	(goto-char (point-min))
-	(while (re-search-forward "%^\\({\\([^}]*\\)}\\)?\\([guUtT]\\)?" nil t)
-	  (setq char (if (match-end 3) (match-string 3))
-		prompt (if (match-end 2) (match-string 2)))
-	  (goto-char (match-beginning 0))
-	  (replace-match "")
-	  (cond
-	   ((member char '("G" "g"))
-	    (let* ((org-last-tags-completion-table
-		    (org-global-tags-completion-table
-		     (if (equal char "G") (org-agenda-files) (and file (list file)))))
-		   (org-add-colon-after-tag-completion t)
-		   (ins (completing-read
-			 (if prompt (concat prompt ": ") "Tags: ")
-			 'org-tags-completion-function nil nil nil
-			 'org-tags-history)))
-	      (setq ins (mapconcat 'identity
-				  (org-split-string ins (org-re "[^[:alnum:]]+"))
-				  ":"))
-	      (when (string-match "\\S-" ins)
+	(let (interactive-entries lookup)
+	  (while (re-search-forward "%^\\({\\([^}]*\\)}\\)?\\([guUtT]\\)?" nil t)
+	    (setq char (if (match-end 3) (match-string 3))
+		  prompt (if (match-end 2) (match-string 2))
+		  lookup (assoc prompt interactive-entries))
+	    (goto-char (match-beginning 0))
+	    (replace-match "")
+	    (cond
+	     ((member char '("G" "g"))
+	      (let* ((org-last-tags-completion-table
+		      (org-global-tags-completion-table
+		       (if (equal char "G") (org-agenda-files) (and file (list file)))))
+		     (org-add-colon-after-tag-completion t)
+		     (ins (if lookup
+			      (cdr lookup)
+			    (completing-read
+			     (if prompt (concat prompt ": ") "Tags: ")
+			     'org-tags-completion-function nil nil nil
+			     'org-tags-history))))
+		(if (null lookup)
+		    (setq interactive-entries (cons (cons prompt ins) interactive-entries)))
+		(setq ins (mapconcat 'identity
+				     (org-split-string ins (org-re "[^[:alnum:]]+"))
+				     ":"))
+		(when (string-match "\\S-" ins)
 		(or (equal (char-before) ?:) (insert ":"))
 		(insert ins)
 		(or (equal (char-after) ?:) (insert ":")))))
-	   (char
-	    (setq org-time-was-given (equal (upcase char) char))
-	    (setq time (org-read-date (equal (upcase char) "U") t nil
-				      prompt))
-	    (org-insert-time-stamp time org-time-was-given
-				   (member char '("u" "U"))
-				   nil nil (list org-end-time-was-given)))
-	   (t
-	    (insert (read-string
-		     (if prompt (concat prompt ": ") "Enter string"))))))
+	     (char
+	      (setq org-time-was-given (equal (upcase char) char))
+	      (setq time (if lookup (cdr lookup) (org-read-date (equal (upcase char) "U") t nil
+								prompt)))
+	      (if (null lookup)
+		  (setq interactive-entries (cons (cons prompt time) interactive-entries))
+		(org-insert-time-stamp time org-time-was-given
+				       (member char '("u" "U"))
+				       nil nil (list org-end-time-was-given))))
+	     (t
+	      (let ((text
+		     (if lookup
+			 (cdr lookup)
+		       (read-string
+			(if prompt (concat prompt ": ") "Enter string")))))
+		(insert (or text ""))
+		(if (null lookup)
+		    (setq interactive-entries (cons (cons prompt text) interactive-entries))))))))
 	(goto-char (point-min))
 	(if (re-search-forward "%\\?" nil t)
 	    (replace-match "")

Have fun!

Outlining Your Notes with Org

Large documents are almost impossible to write without outlines.  There’s just too much to fit in your head. Outlines help you work with a structure, so that you can see the big picture and how sections fit together. Outlines are also surprisingly useful when brainstorming. You can work with varying levels of detail, starting with a high-level overview and successively refining it, or starting with the details and then letting the structure emerge as you organize those details into groups.

Emacs has one of the most powerful outline editors I’ve come across. Although word processors like Microsoft Word and OpenOffice.org Writer support outlines too, Emacs has a gazillion keyboard shortcuts, and once you get the hang of them, you’ll want them in other applications as well. In addition, working in Emacs tends to force you to focus on the content instead of spending time fiddling with the formatting. Whether you’re writing personal notes or working on a document that’s due tomorrow, this is a good thing.

In this section, you’ll learn how to outline a document using Org. You’ll be able to create headings, sub-headings, and text notes. You’ll also learn how to manage outline items by promoting, demoting, and rearranging them.

Understanding Org

Org is primarily a personal information manager that keeps track of your tasks and schedule, and you’ll learn more about those features in chapters 8 and 9. However, Org also has powerful outline-management tools, which is why I recommend it to people who want to write and organize outlined notes.

The structure of an Org file is simple: a plain text file with headlines, text, and some additional information such as tags and timestamps. A headline is any line that stars with a series of asterisks. The more asterisks there are, the deeper the headline is. A second-level headline is the child of the first-level headline before it, and so on. For example:

* This is a first-level headline
Some text under that headline
** This is a second-level headline
Some more text here
*** This is a third-level headline
*** This is another third-level headline
** This is a second-level headline

Because Org uses plain text, it’s easy to back up or process using scripts. Org’s sophistication comes from keyboard shortcuts that allow you to quickly manipulate headlines, hide and show outline subtrees, and search for information.

GNU Emacs 22 includes Org. To automatically enable Org mode for all files with the .org extension, add the following to your ~/.emacs:

(require 'org)
(add-to-list 'auto-mode-alist '("\\.org$" . org-mode))

To change to org-mode manually, use M-x org-mode while viewing a file. To enable Org on a per-file basis, add

-*- mode: org -*- 

to the first line of the files that you would like to associate with Org.

Working with Outlines

You can keep your notes in more than one Org file, but let’s start by creating ~/notes.org. Open ~/notes.org, which will automatically be associated with Org Mode. Type in the general structure of your document. You can type in something like this:

 * Introduction
 * ...

You can also use M-RET (org-meta-return) to create a new headline at the same level as the one above it, or a first-level headline if the document doesn’t have headlines yet.

Create different sections for your work. For example, a thesis might be divided into introduction, review of related literature, description of study, methodology, results and analysis, and conclusions and recommendations. You can type in the starting asterisks yourself, or you can use M-RET (org-meta-return) to create headings one after the other.

Now that you have the basic structure, start refining it. Go to the first section and use a command like C-o (open-line) to create blank space. Add another headline, but this time, make it a second-level headline under the first. You can do that by either typing in two asterisks or by using M-RET (org-meta-return, which calls org-insert-heading) and then M-right (org-metaright, which calls org-do-demote). Then use M-RET (org-meta-return; org-insert-heading) for the other items, or type them in yourself. Repeat this for other sections, and go into as much detail as you want.

What if you want to reorganize your outline? For example, what if you realized you’d accidentally put your conclusions before the introduction? You could either cut and paste it using Emacs shortcuts, or you can use Org’s outline management functions. The following keyboard shortcuts are handy:

Action Command Shortcut Alternative
Move a subtree up org-metaup / org-move-subtree-up M-up C-c C-x u
Move a subtree down org-metadown / org-move-subtree-down M-down C-c C-x d
Demote a subtree org-shiftmetaright / org-demote-subtree S-M-right C-c C-x r
Promote a subtree org-shiftmetaleft / org-promote-subtree S-M-left C-c C-x l
Demote a headline org-metaright / org-do-demote M-right C-c C-x <right>
Promote a headline org-metaleft / org-do-promote M-left C-c C-x <left>
Collapse or expand a subtree org-cycle (while on headline) TAB  
Collapse or expand everything org-shifttab (org-cycle) S-TAB C-u TAB

Use these commands to help you reorganize your outline as you create a skeleton for your document. These commands make it easy to change your mind about the content or order of sections. You might find it easier to sketch a rough outline first, then gradually fill in more and more detail. On the other hand, you might find it easier to pick one section and keep drilling it down until you have headlines some seven levels deep. When you reach that point, all you need to do is add punctuation and words like "and" or "but" between every other outline item, and you’re done!

Well, no, not likely. You’ll probably get stuck somewhere, so here are some tips for keeping yourself going when you’re working on a large document in Org: brainstorming and getting a sense of your progress.

Brainstorming

Brainstorming is a great way to break your writer’s block or to generate lots of possibilities. The key idea is to come up with as many ideas as you can, and write them all down before you start evaluating them.

I usually switch to paper for mindmapping and brainstorming because paper helps me think in a more colorful and non-linear way. However, it can be hard to manage large mindmaps on paper, because you can’t reorganize nodes easily. Despite its text-heavy interface, Org is one of the best mindmapping tools I’ve come across. Because it’s built into Emacs, everything can be done through keyboard shortcuts.

When you’re brainstorming, you might like working from two different directions. Sometimes it’s easier to start with an outline and to add more and more detail. Other times, you may want to jot quick ideas and then organize them into groups that make sense. Org provides support for both ways of working.

Brainstorming bottom-up is similar to David Allen’s Getting Things Done method in that separating collection from organization is a good idea. That is, get the ideas out of your head first before you spend time trying to figure out the best way to organize them. To get things out of your head quickly, collect your ideas by using the M-RET (org-meta-return) to create a new heading, typing in your idea, and using M-RET (org-meta-return) to create the next heading. Do this as many times as needed.

One way to encourage yourself to brainstorm lots of ideas is to give yourself a quota. Charles Cave described this technique in an article on org-mode, and it’s a great way to use structure to prompt creativity. Simply start by adding a number of empty headings (say, 20), then work towards filling that quota. For example, you might start with ten blanks for ideas, then gradually fill them in like this:

* Things that make me happy
** Curling up with a good book
** Watching a brilliant sunset
** Giving or getting a big warm hug
** Writing a cool piece of Emacs code
** 
** 
** 
**
** 
**

When all of your ideas are in Org, start organizing them. This is where you move ideas around using M-S-up (org-shiftmetaup) and M-S-down (org-shift-metadown), which call org-move-subtree-up and org-move-subtree-down. This is also a good time to use headings to group things together.

Getting a Sense of Progress

You’ve brainstormed. You’ve started writing your notes. And if you’re working on a large document, you might lose steam at some point along the way. For example, while working on this book, I often find myself intimidated by just how much there is to write about Emacs. It helps to have a sense of progress such as the number of words written or the number of sections completed. To see a word count that updates every second, add this code to your ~/.emacs:

(defvar count-words-buffer
  nil
  "*Number of words in the buffer.")

(defun wicked/update-wc ()
  (interactive)
  (setq count-words-buffer (number-to-string (count-words-buffer)))
  (force-mode-line-update))
  
; only setup timer once
(unless count-words-buffer
  ;; seed count-words-paragraph
  ;; create timer to keep count-words-paragraph updated
  (run-with-idle-timer 1 t 'wicked/update-wc))

;; add count words paragraph the mode line
(unless (memq 'count-words-buffer global-mode-string)
  (add-to-list 'global-mode-string "words: " t)
  (add-to-list 'global-mode-string 'count-words-buffer t)) 

;; count number of words in current paragraph
(defun count-words-buffer ()
  "Count the number of words in the current paragraph."
  (interactive)
  (save-excursion
    (goto-char (point-min))
    (let ((count 0))
      (while (not (eobp))
	(forward-word 1)
        (setq count (1+ count)))
      count)))

The best way I’ve found to track my progress in terms of sections is to use a checklist. For example, the collapsed view for this section looks like this:

** Outline Notes with Org [7/9]    ;; (1)
*** [X] Outlining Your Notes...    ;; (2)
*** [X] Understanding Org...
*** [X] Working with Outlines...
*** [X] Brainstorming...
*** [ ] Getting a Sense of Progress... ;; (3)
*** [X] Searching Your Notes...
*** [X] Hyperlinks...
*** [X] Publishing Your Notes...
*** [ ] Integrating Remember with Org

  • (1): [7/9] shows that I’ve completed 7 of 9 parts.
  • (2): [X] indicates finished parts.
  • (3): [ ] indicates parts I still need to do.

This is based on the checklist feature in Org. However, the standard feature works only with lists like this:

*** Items [1/3]
- [X] Item 1
- [ ] Item 2
- [ ] Item 3

Add the following code to your ~/.emacs in order to make the function work with headlines:

(defun wicked/org-update-checkbox-count (&optional all)
  "Update the checkbox statistics in the current section.
This will find all statistic cookies like [57%] and [6/12] and update
them with the current numbers.  With optional prefix argument ALL,
do this for the whole buffer."
  (interactive "P")
  (save-excursion
    (let* ((buffer-invisibility-spec (org-inhibit-invisibility)) 
	   (beg (condition-case nil
		    (progn (outline-back-to-heading) (point))
		  (error (point-min))))
	   (end (move-marker
		 (make-marker)
		 (progn (or (outline-get-next-sibling) ;; (1)
			    (goto-char (point-max)))
			(point))))   
	   (re "\\(\\[[0-9]*%\\]\\)\\|\\(\\[[0-9]*/[0-9]*\\]\\)")
	   (re-box
	    "^[ \t]*\\(*+\\|[-+*]\\|[0-9]+[.)]\\) +\\(\\[[- X]\\]\\)")
	   b1 e1 f1 c-on c-off lim (cstat 0))
      (when all
	(goto-char (point-min))
	(or (outline-get-next-sibling) (goto-char (point-max))) ;; (2)
	(setq beg (point) end (point-max)))
      (goto-char beg)
      (while (re-search-forward re end t)
	(setq cstat (1+ cstat)
	      b1 (match-beginning 0)
	      e1 (match-end 0)
	      f1 (match-beginning 1)
	      lim (cond
		   ((org-on-heading-p)
		    (or (outline-get-next-sibling) ;; (3)
			(goto-char (point-max)))
		    (point))
		   ((org-at-item-p) (org-end-of-item) (point))
		   (t nil))
	      c-on 0 c-off 0)
	(goto-char e1)
	(when lim
	  (while (re-search-forward re-box lim t)
	    (if (member (match-string 2) '("[ ]" "[-]"))
		(setq c-off (1+ c-off))
	      (setq c-on (1+ c-on))))
	  (goto-char b1)
	  (insert (if f1
		      (format "[%d%%]" (/ (* 100 c-on)
					  (max 1 (+ c-on c-off))))
		    (format "[%d/%d]" c-on (+ c-on c-off))))
	  (and (looking-at "\\[.*?\\]")
	       (replace-match ""))))
      (when (interactive-p)
	(message "Checkbox statistics updated %s (%d places)"
		 (if all "in entire file" "in current outline entry")
		 cstat)))))
(defadvice org-update-checkbox-count (around wicked activate)
  "Fix the built-in checkbox count to understand headlines."
  (setq ad-return-value
	(wicked/org-update-checkbox-count (ad-get-arg 1))))

Now add [ ] or [X] to the lower-level headlines you want to track. Add [/] to the end of the higher-level headline containing those headlines. Type C-c # (org-update-checkbox-count) to update the count for the current headline, or C-u C-c C-# (org-update-checkbox-count) to update all checkbox counts in the whole buffer.

If you want to see the percentage of completed items, use [%] instead of [/]. I find 7/9 to be easier to understand than 71%, but fractions might work better for other cases.

Okay, you’ve written a lot. How do you find information again?

Searching Your Notes

When you’re writing your notes, you might need to refer to something you’ve written. You may find it helpful to split your screen in two with C-x 3 (split-window-horizontally) or C-x 2 (split-window-vertically). To switch to another window, type C-x o (other-window) or click on the window. To return to just one window, use C-x 1 (delete-other-windows) to focus on just the current window, or C-x 0 (delete-window) to get rid of the current window.

Now that you’ve split your screen, how do you quickly search through your notes? C-s (isearch-forward) and C-r (isearch-backward) are two of the most useful Emacs keyboard shortcuts you’ll ever learn. Use them to not only interactively search your Org file, but also to quickly jump to sections. For example, I often search for headlines by typing * and the first word. Org searches collapsed sections, so you don’t need to open everything before searching.

To search using Org’s outline structure, use C-c / r (org-sparse-tree, regexp), which will show only entries matching a regular expression. For more information about regular expressions, read the Emacs info manual entry on Regexps. Here are a few examples:

To find Search for
All entries containing "cat" cat
All entries that contain "cat" as a word by itself (example: "cat," but not "catch") \<cat\>
All entries that contain 2006, 2007, or 2008 200[678]

Hyperlinks

If you find yourself frequently searching for some sections, you might want to create hyperlinks to them. For example, if you’re using one file for all of your project notes instead of splitting it up into one file per project, then you probably want a list of projects at the beginning of the file so that you can jump to each project quickly.

You can also use hyperlinks to keep track of your current working position. For example, if you’re working on a long document and you want to keep your place, create a link anchor like <<TODO>> at the point where you’re editing, and add a link like [[TODO]] at the beginning of your file.

You can create a hyperlink to a text search by putting the keywords between two pairs of square brackets, like this:

  See [[things that make me happy]] 

You can open the link by moving the point to the link and typing C-c C-o (org-open-at-point). You can also click on the link to open it. Org will then search for text matching the query. If Org doesn’t find an exact match, it tries to match similar text, such as "Things that make me really really happy".

If you find that the link does not go where you want it to go, you can limit the text search. For example, if you want to link to a headline and you know the headline will be unique, you can add an asterisk at the beginning of the link text in order to limit the search to headlines. For example, given the following text:

In this file, I'll write notes on the things that make me happy. (1)
If I ever need a spot of cheering up, I'll know just what to do! 

** Things that make me happy (2) 
*** ...

Example 
Link 1: [[things that make me happy]] 

Link 2: [[*things that make me happy]] 

Link 1 would lead to (1), and Link 2 would lead to (2).

To define a link anchor, put the text in double angled brackets like this:

<<things that make me happy>> 

A link like [[things that make me happy]] would then link to that instead of other occurances of the text.

You can define keywords that will be automatically hyperlinked throughout the file by using radio targets. For example, if you’re writing a jargon-filled document and you frequently need to refer to the definitions, it may help to make a glossary of terms such as "regexp" and "radio target", and then define them in a glossary section at the end of the file, like this:

*** glossary
<<<radio target>>> A word or phrase that is automatically hyperlinked whenever it appears. 
<<<regexp>>> A regular expression. See the Emacs info manual. 

Radio targets are automatically enabled when an Org file is opened in Emacs. If you’ve just added a radio target, enable it by moving the point to the anchor and pressing C-c C-c (org-ctrl-c-ctrl-c). This turns all instances of the text into hyperlinks that point to the radio target.

Publishing Your Notes

You can keep your notes as a plain text file, or you can publish them as HTML or LaTeX. HTML seems to be the easiest way to let non-Emacs users read your notes, as a large text file without pretty colors can be hard to read.

To export a file to any of the formats that Org understands by default, type C-c C-e (org-export). You can then type ‘h’ (org-export-as-html) to export it as HTML for websites. You can also type ‘l’ (org-export-as-latex) to export it to LaTeX, a scientific typesetting language, which can then be published as PDF or PS. Another way to convert it to PDF is to export it as HTML, open it in OpenOffice.org Writer, and use the Export to PDF feature. You can also open HTML documents in other popular word processors to convert them to other supported formats.

By default, Org exports all the content in the current file. To limit it only to the visible content, use C-c C-e v (org-export-visible) followed by either ‘h’ for HTML or ‘l’ for LaTeX.

If you share your notes, you may want to export an HTML version every time you save an Org file. Here is a quick and dirty way to publish all Org files to HTML every time you save them:

(defun wicked/org-publish-files-maybe ()
  "Publish this file."
  (org-export-as-html-batch)
  nil)
(add-hook 'org-mode-hook  ;; (1)
 (lambda () 
  (add-hook (make-local-variable 'after-save-hook) ;; (2)
            'wicked/org-publish-files-maybe)))

(Update: Feb 22 2011: Thanks to Neilen for the fix!)

  • (1) Every time a buffer is set to Org mode…
  • (2) Add a function that publishes the file every time that file is saved.

What if most of your files are private, but you want to publish only a few of them? To control this, let’s add a new special keyword "#+PUBLISH" to the beginning of the files that you want to be automatically published. Replace the previous code in your ~/.emacs with this:

(defun wicked/org-publish-files-maybe ()
  "Publish this file if it contains the #+PUBLISH: keyword"
  (save-excursion
   (save-restriction
    (widen)
    (goto-char (point-min))
    (when (re-search-forward 
           "^#?[ \t]*\\+\\(PUBLISH\\)"
           nil t) 
     (org-export-as-html-batch)   
     nil))))

(add-hook 'org-mode-hook
 (lambda ()
  (add-hook (make-local-variable 'after-save-hook)
            'wicked/org-publish-files-maybe)))

and add #+PUBLISH on a line by itself to your ~/notes.org file, like this:

 #+PUBLISH
 * Things that make me happy

When you save any Org file that contains the keyword, the corresponding HTML page will also be created. You can then use a program like rsync or scp to copy the file to a webserver, or you can copy it to a shared directory.

Integrating Remember with Org

You can use Remember to collect notes that you will later integrate into your outline. Add the following code to your ~/.emacs to set it up:

(global-set-key (kbd "C-c r") 'remember)    ;; (1)
(add-hook 'remember-mode-hook 'org-remember-apply-template) ;; (2)
(setq org-remember-templates  
      '((?n "* %U %?\n\n  %i\n  %a" "~/notes.org")))  ;; (3)
(setq remember-annotation-functions '(org-remember-annotation)) ;; (4)
(setq remember-handler-functions '(org-remember-handler)) ;; (5)
  • 1: Handy keyboard shortcut for (r)emember
  • 2: Creates a template for (n)otes
  • 3: Create a note template which saves the note to ~/notes.org, or whichever Org file you want to use
  • 4: Creates org-compatible context links
  • 5: Saves remembered notes to an Org file

With this code, you can type C-c r n (remember, Notes template) to pop up a buffer (usually containing a link to whatever you were looking at), write your note, and type C-c C-c (remember-buffer) to save the note and close the buffer.

You can use this to store snippets in your notes file, or to quickly capture an idea that comes up when you’re doing something else.

Now you know how to sketch an outline, reorganize it, fill it in, brainstorm, stay motivated, and publish your notes. I look forward to reading what you have to share!