I sometimes feel like bringing out my stationery, my fountain pen, and a coil of stamps, and writing cards to random people in my address book. When I travel, I also enjoy writing quick postcards to people who live in the country I’m visiting. That’s why I wrote this code to filter my address book so that I could see only the contacts with snail-mail addresses, or only the contacts whose snail-mail addresses match a regular expression.

You can call M-x wicked/bbdb-find-people-with-addresses to filter the displayed BBDB records. Press RET at the “Regexp: ” prompt in order to show all records with addresses, or type in a regular expression that matches anything in the addresses field. By default, wicked/bbdb-find-people-with-addresses works on the BBDB records already shown in the *BBDB* window, or all records if none are shown. This allows you to successively filter BBDB records. (Combined with the other BBDB projects I’ll blog about, you’ll be able to get a list of all the people you haven’t talked to in three months but who you’ve talked to within the year, who are interested in Emacs and cooking but not social networking, and who have a phone number in your contact database! How’s that for targeted mail? ;) ) Anyway, if you want to start your search from scratch and you don’t want to call M-x bbdb with . as the regular expression, use the universal prefix argument (C-u M-x wicked/bbdb-find-people-with-addresses) and it will search your entire contact database.

Have fun! =)

(defun wicked/bbdb-find-people-with-addresses (&optional regexp records)
  "Filter the displayed BBDB records to those with addresses."
  (interactive "MRegexp: ")
  (let ((records (if current-prefix-arg (bbdb-records)
           (or records bbdb-records (bbdb-records))))
        filtered
        cons next)
    (while records
      (when (and (bbdb-record-get-field-internal (if (arrayp (car records))
                            (car records)
                                                 (caar records)) 'address)
         (or (null regexp)
             (string= regexp "")
             (delq nil
               (mapcar
                (lambda (address)
                  (string-match regexp (wicked/bbdb-address-string address)))
                (bbdb-record-get-field-internal
                 (if (arrayp (car records))
                 (car records)
                   (caar records)) 'address)))))
        (setq filtered (cons (if (arrayp (car records))
                                 (car records)
                               (caar records)) filtered)))
      (setq records (cdr records)))
    (bbdb-display-records (nreverse filtered))))

(defun wicked/bbdb-address-string (address)
  "Return ADDRESS as a string."
  (mapconcat
   'identity
   (delq nil
         (list
          (mapconcat 'identity (bbdb-address-streets address) ", ")
          (let ((s (bbdb-address-city address))) (and (not (string= s "")) s))
      (let ((s (bbdb-address-state address))) (and (not (string= s "")) s))
      (let ((s (bbdb-address-zip address))) (and (not (string= s "")) s))
      (let ((s (bbdb-address-country address))) (and (not (string= s "")) s))))
   ", "))

(defun wicked/bbdb-yank-addresses ()
  "Copy displayed addresses to the kill ring."
  (interactive)
  (kill-new
   (mapconcat
    (lambda (record)
      (concat
       (bbdb-record-name (car record)) "\n"
       (mapconcat
    (lambda (address)
      (concat (bbdb-address-location address) ": " (wicked/bbdb-address-string address)))
    (bbdb-record-get-field-internal (car record) 'address)
    "\n")))
    bbdb-records
    "\n\n")))

I haven’t been writing about Emacs lately. Here’s my outline so that you can help keep me honest. =) My next chapter is about the Big Brother Database (BBDB) and contact management in Emacs, which is one of the things that made people laugh when I showed my Emacs configuration at DemoCamp in Toronto. Anyway, here’s what I’m planning to write about:

Chapter 6: Being Big Brother (30 pages)

• Why use Emacs to manage your contacts?
  What is BBDB?
• Project xxx: Set up BBDB
  Project xxx: Import contacts: CSV, card
  Project xxx: Create a record
  Project xxx: Search records
• Mail
  Project xxx: Integrate BBDB with Mail
  Project xxx: Notice e-mail changes
  Project xxx: Filter mail according to record
  Project xxx: Categorize contacts with mail aliases
  Project xxx: Personalize greetings
  Project xxx: Personalize signatures
  Project xxx: Mail merge
  Project xxx: Track last contact
• Filtering records
  Project xxx: Show a phone list
  Project xxx: Show an address list
  Project xxx: Show no contact since
  Project xxx: Show tag queries
  Project xxx: Remember birthdays
• More data
  Project xxx: Snarf records
  Project xxx: Add pictures
  Project xxx: Export contacts
  Project xxx: Synchronize contacts
  Project xxx: Synchronize with LinkedIn

By golly, it’s starting to look like a book.

I just finished putting together my third chapter, which is really chapter 7 in the book: managing your notes and Emacs. This chapter is about taking notes in Emacs, focusing on Remember, Org, Planner, and blogs. At 38 pages, it’s a little over my planned 35 pages, and I haven’t even covered all the things that I wanted to like random information management with Howm, blogging to Blosxom, and customizing Planner templates. Maybe after some really fierce copy-editing, I’ll have some space.

I sent a copy off to my editor, and I just finished uploading a PDF and OpenOffice.org document that you can download and read. There’s also an HTML version, but the formatting is a little wonky. I hope you find this useful! I didn’t blog as much of this as I did last time, so I missed out on all the wonderful feedback people could’ve given me. I’ll do that next chapter.

I formatted most of the chapter this afternoon, hanging out with Leigh Honeywell, Seth Hardy, and a few other geeks at the Linux Caffe. Leigh’s working on a book proposal, and we’re thinking of organizing a writing group for technical authors. We’ll start by meeting this Thursday at Leigh’s apartment. iI enjoyed chatting with them as I worked on my book, drifting in and out of conversations. I think it would be a good idea to work somewhere quieter, with plenty of table room for assorted gadgets, but this was a good start.

Next chapter: contact management in Emacs. I’ve got a lot of fun hacks that I want to share here, so coming up with material shouldn’t be hard. I’ll keep you posted!

(UPDATE: Fixed links. Thanks to Leschinsky Oleg for pointing that out!)

I have about 41 pages for my current chapter on taking notes in Emacs, and I still have to write the introduction and wrap-up sections. This means that I’ll need to do some more editing. It’s clearly time to apply another piece of advice from the wonderful book How to Write Fast (While Writing Well), by David Fryxell: don’t over-write, because the time you waste writing more than necessary and then editing down to the required length could be spent writing about something else!

I also rock because I’m getting the hang of writing macros for OpenOffice.org. Basic is not my favorite language and OpenOffice.org macro programming is sufficiently obscure that it’s hard to find examples of what I want to do. OOoMacros.org helped me learn enough to write a few more functions to make my writing life better. I wrote two new functions today: one to mark up wingdings in my code, and another to mark up the source code examples. Both save me lots of tedious work. The wingding function replaced all instances of (1), ;; (1), (2), ;; (2), until (9) and ;; (9) with the corresponding wingding. The example function marked up everything enclosed in <example>..</example> tags, formatting the first and last lines in a different style and using a single-line code style for short examples. Between that and the two functions I’d previously written to format about 80% of my command and keyboard shortcuts, editing is just so much better.

It seems that most people don’t tinker with their tools. I like doing so. I like getting to know what I can do with a tool and how far I can push it. This can be frustrating sometimes (I spent all afternoon trying to get Emacs to play nicely with other weblogs!), but most of the time, it’s both fun and useful. Not only am I saving myself time, I’m also exercising my brain and opening opportunities for other hacks down the line. This is good.

Oh! And W- is giving Ledger a try. The tool he’d been using to keep track of finances dropped some more of his data recently. Because it’s all magic inside, he couldn’t figure out what happened. I’d been going on about how awesome Ledger is, this command-line double-entry accounting tool that provides no bells or whistles but with which I can do all sorts of crazy things. He described how he wanted to budget, and I showed him how to do it. No screencast, but I do plan to review the audio and create some supporting materials for my upcoming article on Ledger and Gnuplot.

Life is good. =)

Large documents are almost impossible to write without outlines.  There’s just too much to fit in your head. Outlines help you work with a structure, so that you can see the big picture and how sections fit together. Outlines are also surprisingly useful when brainstorming. You can work with varying levels of detail, starting with a high-level overview and successively refining it, or starting with the details and then letting the structure emerge as you organize those details into groups.

Emacs has one of the most powerful outline editors I’ve come across. Although word processors like Microsoft Word and OpenOffice.org Writer support outlines too, Emacs has a gazillion keyboard shortcuts, and once you get the hang of them, you’ll want them in other applications as well. In addition, working in Emacs tends to force you to focus on the content instead of spending time fiddling with the formatting. Whether you’re writing personal notes or working on a document that’s due tomorrow, this is a good thing.

In this section, you’ll learn how to outline a document using Org. You’ll be able to create headings, sub-headings, and text notes. You’ll also learn how to manage outline items by promoting, demoting, and rearranging them.

Understanding Org

Org is primarily a personal information manager that keeps track of your tasks and schedule, and you’ll learn more about those features in chapters 8 and 9. However, Org also has powerful outline-management tools, which is why I recommend it to people who want to write and organize outlined notes.

The structure of an Org file is simple: a plain text file with headlines, text, and some additional information such as tags and timestamps. A headline is any line that stars with a series of asterisks. The more asterisks there are, the deeper the headline is. A second-level headline is the child of the first-level headline before it, and so on. For example:

* This is a first-level headline
Some text under that headline
** This is a second-level headline
Some more text here
*** This is a third-level headline
*** This is another third-level headline
** This is a second-level headline

Because Org uses plain text, it’s easy to back up or process using scripts. Org’s sophistication comes from keyboard shortcuts that allow you to quickly manipulate headlines, hide and show outline subtrees, and search for information.

GNU Emacs 22 includes Org. To automatically enable Org mode for all files with the .org extension, add the following to your ~/.emacs:

(require 'org)
(add-to-list 'auto-mode-alist '("\\.org$" . org-mode))

To change to org-mode manually, use M-x org-mode while viewing a file. To enable Org on a per-file basis, add

-*- mode: org -*- 

to the first line of the files that you would like to associate with Org.

Working with Outlines

You can keep your notes in more than one Org file, but let’s start by creating ~/notes.org. Open ~/notes.org, which will automatically be associated with Org Mode. Type in the general structure of your document. You can type in something like this:

 * Introduction
 * ...

You can also use M-RET (org-meta-return) to create a new headline at the same level as the one above it, or a first-level headline if the document doesn’t have headlines yet.

Create different sections for your work. For example, a thesis might be divided into introduction, review of related literature, description of study, methodology, results and analysis, and conclusions and recommendations. You can type in the starting asterisks yourself, or you can use M-RET (org-meta-return) to create headings one after the other.

Now that you have the basic structure, start refining it. Go to the first section and use a command like C-o (open-line) to create blank space. Add another headline, but this time, make it a second-level headline under the first. You can do that by either typing in two asterisks or by using M-RET (org-meta-return, which calls org-insert-heading) and then M-right (org-metaright, which calls org-do-demote). Then use M-RET (org-meta-return; org-insert-heading) for the other items, or type them in yourself. Repeat this for other sections, and go into as much detail as you want.

What if you want to reorganize your outline? For example, what if you realized you’d accidentally put your conclusions before the introduction? You could either cut and paste it using Emacs shortcuts, or you can use Org’s outline management functions. The following keyboard shortcuts are handy:

Action	Command	Shortcut	Alternative
Move a subtree up	org-metaup / org-move-subtree-up	M-up	C-c C-x u
Move a subtree down	org-metadown / org-move-subtree-down	M-down	C-c C-x d
Demote a subtree	org-shiftmetaright / org-demote-subtree	S-M-right	C-c C-x r
Promote a subtree	org-shiftmetaleft / org-promote-subtree	S-M-left	C-c C-x l
Demote a headline	org-metaright / org-do-demote	M-right	C-c C-x <right>
Promote a headline	org-metaleft / org-do-promote	M-left	C-c C-x <left>
Collapse or expand a subtree	org-cycle (while on headline)	TAB
Collapse or expand everything	org-shifttab (org-cycle)	S-TAB	C-u TAB

Use these commands to help you reorganize your outline as you create a skeleton for your document. These commands make it easy to change your mind about the content or order of sections. You might find it easier to sketch a rough outline first, then gradually fill in more and more detail. On the other hand, you might find it easier to pick one section and keep drilling it down until you have headlines some seven levels deep. When you reach that point, all you need to do is add punctuation and words like "and" or "but" between every other outline item, and you’re done!

Well, no, not likely. You’ll probably get stuck somewhere, so here are some tips for keeping yourself going when you’re working on a large document in Org: brainstorming and getting a sense of your progress.

Brainstorming

Brainstorming is a great way to break your writer’s block or to generate lots of possibilities. The key idea is to come up with as many ideas as you can, and write them all down before you start evaluating them.

I usually switch to paper for mindmapping and brainstorming because paper helps me think in a more colorful and non-linear way. However, it can be hard to manage large mindmaps on paper, because you can’t reorganize nodes easily. Despite its text-heavy interface, Org is one of the best mindmapping tools I’ve come across. Because it’s built into Emacs, everything can be done through keyboard shortcuts.

When you’re brainstorming, you might like working from two different directions. Sometimes it’s easier to start with an outline and to add more and more detail. Other times, you may want to jot quick ideas and then organize them into groups that make sense. Org provides support for both ways of working.

Brainstorming bottom-up is similar to David Allen’s Getting Things Done method in that separating collection from organization is a good idea. That is, get the ideas out of your head first before you spend time trying to figure out the best way to organize them. To get things out of your head quickly, collect your ideas by using the M-RET (org-meta-return) to create a new heading, typing in your idea, and using M-RET (org-meta-return) to create the next heading. Do this as many times as needed.

One way to encourage yourself to brainstorm lots of ideas is to give yourself a quota. Charles Cave described this technique in an article on org-mode, and it’s a great way to use structure to prompt creativity. Simply start by adding a number of empty headings (say, 20), then work towards filling that quota. For example, you might start with ten blanks for ideas, then gradually fill them in like this:

* Things that make me happy
** Curling up with a good book
** Watching a brilliant sunset
** Giving or getting a big warm hug
** Writing a cool piece of Emacs code
** 
** 
** 
**
** 
**

When all of your ideas are in Org, start organizing them. This is where you move ideas around using M-S-up (org-shiftmetaup) and M-S-down (org-shift-metadown), which call org-move-subtree-up and org-move-subtree-down. This is also a good time to use headings to group things together.

Getting a Sense of Progress

You’ve brainstormed. You’ve started writing your notes. And if you’re working on a large document, you might lose steam at some point along the way. For example, while working on this book, I often find myself intimidated by just how much there is to write about Emacs. It helps to have a sense of progress such as the number of words written or the number of sections completed. To see a word count that updates every second, add this code to your ~/.emacs:

(defvar count-words-buffer
  nil
  "*Number of words in the buffer.")

(defun wicked/update-wc ()
  (interactive)
  (setq count-words-buffer (number-to-string (count-words-buffer)))
  (force-mode-line-update))
  
; only setup timer once
(unless count-words-buffer
  ;; seed count-words-paragraph
  ;; create timer to keep count-words-paragraph updated
  (run-with-idle-timer 1 t 'wicked/update-wc))

;; add count words paragraph the mode line
(unless (memq 'count-words-buffer global-mode-string)
  (add-to-list 'global-mode-string "words: " t)
  (add-to-list 'global-mode-string 'count-words-buffer t)) 

;; count number of words in current paragraph
(defun count-words-buffer ()
  "Count the number of words in the current paragraph."
  (interactive)
  (save-excursion
    (goto-char (point-min))
    (let ((count 0))
      (while (not (eobp))
	(forward-word 1)
        (setq count (1+ count)))
      count)))

The best way I’ve found to track my progress in terms of sections is to use a checklist. For example, the collapsed view for this section looks like this:

** Outline Notes with Org [7/9]    ;; (1)
*** [X] Outlining Your Notes...    ;; (2)
*** [X] Understanding Org...
*** [X] Working with Outlines...
*** [X] Brainstorming...
*** [ ] Getting a Sense of Progress... ;; (3)
*** [X] Searching Your Notes...
*** [X] Hyperlinks...
*** [X] Publishing Your Notes...
*** [ ] Integrating Remember with Org

• (1): [7/9] shows that I’ve completed 7 of 9 parts.
• (2): [X] indicates finished parts.
• (3): [ ] indicates parts I still need to do.

This is based on the checklist feature in Org. However, the standard feature works only with lists like this:

*** Items [1/3]
- [X] Item 1
- [ ] Item 2
- [ ] Item 3

Add the following code to your ~/.emacs in order to make the function work with headlines:

(defun wicked/org-update-checkbox-count (&optional all)
  "Update the checkbox statistics in the current section.
This will find all statistic cookies like [57%] and [6/12] and update
them with the current numbers.  With optional prefix argument ALL,
do this for the whole buffer."
  (interactive "P")
  (save-excursion
    (let* ((buffer-invisibility-spec (org-inhibit-invisibility)) 
	   (beg (condition-case nil
		    (progn (outline-back-to-heading) (point))
		  (error (point-min))))
	   (end (move-marker
		 (make-marker)
		 (progn (or (outline-get-next-sibling) ;; (1)
			    (goto-char (point-max)))
			(point))))   
	   (re "\\(\\[[0-9]*%\\]\\)\\|\\(\\[[0-9]*/[0-9]*\\]\\)")
	   (re-box
	    "^[ \t]*\\(*+\\|[-+*]\\|[0-9]+[.)]\\) +\\(\\[[- X]\\]\\)")
	   b1 e1 f1 c-on c-off lim (cstat 0))
      (when all
	(goto-char (point-min))
	(or (outline-get-next-sibling) (goto-char (point-max))) ;; (2)
	(setq beg (point) end (point-max)))
      (goto-char beg)
      (while (re-search-forward re end t)
	(setq cstat (1+ cstat)
	      b1 (match-beginning 0)
	      e1 (match-end 0)
	      f1 (match-beginning 1)
	      lim (cond
		   ((org-on-heading-p)
		    (or (outline-get-next-sibling) ;; (3)
			(goto-char (point-max)))
		    (point))
		   ((org-at-item-p) (org-end-of-item) (point))
		   (t nil))
	      c-on 0 c-off 0)
	(goto-char e1)
	(when lim
	  (while (re-search-forward re-box lim t)
	    (if (member (match-string 2) '("[ ]" "[-]"))
		(setq c-off (1+ c-off))
	      (setq c-on (1+ c-on))))
	  (goto-char b1)
	  (insert (if f1
		      (format "[%d%%]" (/ (* 100 c-on)
					  (max 1 (+ c-on c-off))))
		    (format "[%d/%d]" c-on (+ c-on c-off))))
	  (and (looking-at "\\[.*?\\]")
	       (replace-match ""))))
      (when (interactive-p)
	(message "Checkbox statistics updated %s (%d places)"
		 (if all "in entire file" "in current outline entry")
		 cstat)))))
(defadvice org-update-checkbox-count (around wicked activate)
  "Fix the built-in checkbox count to understand headlines."
  (setq ad-return-value
	(wicked/org-update-checkbox-count (ad-get-arg 1))))

Now add [ ] or [X] to the lower-level headlines you want to track. Add [/] to the end of the higher-level headline containing those headlines. Type C-c # (org-update-checkbox-count) to update the count for the current headline, or C-u C-c C-# (org-update-checkbox-count) to update all checkbox counts in the whole buffer.

If you want to see the percentage of completed items, use [%] instead of [/]. I find 7/9 to be easier to understand than 71%, but fractions might work better for other cases.

Okay, you’ve written a lot. How do you find information again?

Searching Your Notes

When you’re writing your notes, you might need to refer to something you’ve written. You may find it helpful to split your screen in two with C-x 3 (split-window-horizontally) or C-x 2 (split-window-vertically). To switch to another window, type C-x o (other-window) or click on the window. To return to just one window, use C-x 1 (delete-other-windows) to focus on just the current window, or C-x 0 (delete-window) to get rid of the current window.

Now that you’ve split your screen, how do you quickly search through your notes? C-s (isearch-forward) and C-r (isearch-backward) are two of the most useful Emacs keyboard shortcuts you’ll ever learn. Use them to not only interactively search your Org file, but also to quickly jump to sections. For example, I often search for headlines by typing * and the first word. Org searches collapsed sections, so you don’t need to open everything before searching.

To search using Org’s outline structure, use C-c / r (org-sparse-tree, regexp), which will show only entries matching a regular expression. For more information about regular expressions, read the Emacs info manual entry on Regexps. Here are a few examples:

To find	Search for
All entries containing "cat"	cat
All entries that contain "cat" as a word by itself (example: "cat," but not "catch")	\<cat\>
All entries that contain 2006, 2007, or 2008	200[678]

Hyperlinks

If you find yourself frequently searching for some sections, you might want to create hyperlinks to them. For example, if you’re using one file for all of your project notes instead of splitting it up into one file per project, then you probably want a list of projects at the beginning of the file so that you can jump to each project quickly.

You can also use hyperlinks to keep track of your current working position. For example, if you’re working on a long document and you want to keep your place, create a link anchor like <<TODO>> at the point where you’re editing, and add a link like [[TODO]] at the beginning of your file.

You can create a hyperlink to a text search by putting the keywords between two pairs of square brackets, like this:

  See [[things that make me happy]] 

You can open the link by moving the point to the link and typing C-c C-o (org-open-at-point). You can also click on the link to open it. Org will then search for text matching the query. If Org doesn’t find an exact match, it tries to match similar text, such as "Things that make me really really happy".

If you find that the link does not go where you want it to go, you can limit the text search. For example, if you want to link to a headline and you know the headline will be unique, you can add an asterisk at the beginning of the link text in order to limit the search to headlines. For example, given the following text:

In this file, I'll write notes on the things that make me happy. (1)
If I ever need a spot of cheering up, I'll know just what to do! 

** Things that make me happy (2) 
*** ...

Example 
Link 1: [[things that make me happy]] 

Link 2: [[*things that make me happy]] 

Link 1 would lead to (1), and Link 2 would lead to (2).

To define a link anchor, put the text in double angled brackets like this:

<<things that make me happy>> 

A link like [[things that make me happy]] would then link to that instead of other occurances of the text.

You can define keywords that will be automatically hyperlinked throughout the file by using radio targets. For example, if you’re writing a jargon-filled document and you frequently need to refer to the definitions, it may help to make a glossary of terms such as "regexp" and "radio target", and then define them in a glossary section at the end of the file, like this:

*** glossary
<<<radio target>>> A word or phrase that is automatically hyperlinked whenever it appears. 
<<<regexp>>> A regular expression. See the Emacs info manual. 

Radio targets are automatically enabled when an Org file is opened in Emacs. If you’ve just added a radio target, enable it by moving the point to the anchor and pressing C-c C-c (org-ctrl-c-ctrl-c). This turns all instances of the text into hyperlinks that point to the radio target.

Publishing Your Notes

You can keep your notes as a plain text file, or you can publish them as HTML or LaTeX. HTML seems to be the easiest way to let non-Emacs users read your notes, as a large text file without pretty colors can be hard to read.

To export a file to any of the formats that Org understands by default, type C-c C-e (org-export). You can then type ‘h’ (org-export-as-html) to export it as HTML for websites. You can also type ‘l’ (org-export-as-latex) to export it to LaTeX, a scientific typesetting language, which can then be published as PDF or PS. Another way to convert it to PDF is to export it as HTML, open it in OpenOffice.org Writer, and use the Export to PDF feature. You can also open HTML documents in other popular word processors to convert them to other supported formats.

By default, Org exports all the content in the current file. To limit it only to the visible content, use C-c C-e v (org-export-visible) followed by either ‘h’ for HTML or ‘l’ for LaTeX.

If you share your notes, you may want to export an HTML version every time you save an Org file. Here is a quick and dirty way to publish all Org files to HTML every time you save them:

(defun wicked/org-publish-files-maybe ()
  "Publish this file."
  (org-export-as-html-batch)
  nil)
(add-hook 'org-mode-hook  ;; (1)
 (lambda () 
  (add-hook (make-local-variable 'after-save-hook) ;; (2)
            'wicked/org-publish-files-maybe)))

(Update: Feb 22 2011: Thanks to Neilen for the fix!)

• (1) Every time a buffer is set to Org mode…
• (2) Add a function that publishes the file every time that file is saved.

What if most of your files are private, but you want to publish only a few of them? To control this, let’s add a new special keyword "#+PUBLISH" to the beginning of the files that you want to be automatically published. Replace the previous code in your ~/.emacs with this:

(defun wicked/org-publish-files-maybe ()
  "Publish this file if it contains the #+PUBLISH: keyword"
  (save-excursion
   (save-restriction
    (widen)
    (goto-char (point-min))
    (when (re-search-forward 
           "^#?[ \t]*\\+\\(PUBLISH\\)"
           nil t) 
     (org-export-as-html-batch)   
     nil))))

(add-hook 'org-mode-hook
 (lambda ()
  (add-hook (make-local-variable 'after-save-hook)
            'wicked/org-publish-files-maybe)))

and add #+PUBLISH on a line by itself to your ~/notes.org file, like this:

 #+PUBLISH
 * Things that make me happy

When you save any Org file that contains the keyword, the corresponding HTML page will also be created. You can then use a program like rsync or scp to copy the file to a webserver, or you can copy it to a shared directory.

Integrating Remember with Org

You can use Remember to collect notes that you will later integrate into your outline. Add the following code to your ~/.emacs to set it up:

(global-set-key (kbd "C-c r") 'remember)    ;; (1)
(add-hook 'remember-mode-hook 'org-remember-apply-template) ;; (2)
(setq org-remember-templates  
      '((?n "* %U %?\n\n  %i\n  %a" "~/notes.org")))  ;; (3)
(setq remember-annotation-functions '(org-remember-annotation)) ;; (4)
(setq remember-handler-functions '(org-remember-handler)) ;; (5)

• 1: Handy keyboard shortcut for (r)emember
• 2: Creates a template for (n)otes
• 3: Create a note template which saves the note to ~/notes.org, or whichever Org file you want to use
• 4: Creates org-compatible context links
• 5: Saves remembered notes to an Org file

With this code, you can type C-c r n (remember, Notes template) to pop up a buffer (usually containing a link to whatever you were looking at), write your note, and type C-c C-c (remember-buffer) to save the note and close the buffer.

You can use this to store snippets in your notes file, or to quickly capture an idea that comes up when you’re doing something else.

Now you know how to sketch an outline, reorganize it, fill it in, brainstorm, stay motivated, and publish your notes. I look forward to reading what you have to share!

Ideas come from everywhere. While reading this blog, you might come across interesting snippets that you’d like to save. While writing code, you might be hit by an idea for something you want to do with the program. While on a phone call, you might need to write down what you need to prepare for a meeting the next day.

How do you take notes now? Do you jot your notes on a scrap of paper or into a leather notebook? Do you copy and paste what you’re looking at into a plain text file or document? Do you scribble things into a personal digital assistant?

I’ve tried different note-taking strategies: colorful mindmaps, outlined text files, even voice recordings. I felt frustrated every time I had to write down whose e-mail or which webpage prompted the note (shouldn’t the computer do that automatically?), but I was even more frustrated when I’d come across a note and not remember why I wrote it.

Remember changed all that for me. The key ideas behind Remember are that you should be able to write notes with minimal distraction, and that as the context should be automatically picked up so that you don’t have to write it down. If you’re writing a note based on a mail message, Remember will pick up the details of the message and create a hyperlink so that you can view the original message when reviewing your notes. If you’re working on a file, Remember will link to it so that you can jump back to it. If you’re browsing a web page (in Emacs, of course), Remember will remember the title and URL, so that you can go back to the page when you want to clarify something in your notes. After you save the note, you’ll be back in the same environment you were: no need to switch applications and no need to remember different keyboard shortcuts.

You might think that Remember’s only worth it if you do _everything_ in Emacs. For me, it worked the other way around. I started by using Remember to take notes in Planner, a personal information manager available for Emacs. As I got accustomed to the way Remember and Planner just automatically hyperlinked to whatever I was looking at, I thought: Why doesn’t my mail client do this? Why doesn’t my web browser do this? Why doesn’t my chat client do this? So I ended up reading through the manuals, figuring out how to do all these things in Emacs—and I loved it, eventually doing most of my work (and play!) within an Emacs frame. Although I use other applications now, like Lotus Notes for work mail and Mozilla Firefox for browsing, I still switch back to Emacs for my notes.

In this section, you’ll learn how to set up Remember and take quick notes in Emacs. We’ll start by configuring Remember to save your notes to a file, and how to configure Remember to save to different places depending on the content. You’ll also learn how to quickly search your notes file for entries.

You can also integrate Remember into other note-taking systems in Emacs. The sections that cover those systems will also show you how to configure Remember to save your notes there.

Setting Up Remember

Remember is a separate package, which you can download from https://gna.org/projects/remember-el . As of this writing, the latest stable release is Remember 1.9. Download the latest version and unpack it into your ~/elisp directory. You should end up with a new directory, ~/elisp/remember-1.9 .

To configure Remember to save to plain text files, add this code to your ~/.emacs and evaluate it:

(add-to-list 'load-path "~/elisp/remember-1.9") ;; (1)
(require 'remember-autoloads)
(setq remember-data-file "~/notes.txt")  ;; (2)
(global-set-key (kbd "C-c r") 'remember) ;; (3)

(defun wicked/remember-review-file ()
 "Open `remember-data-file'."
 (interactive)
 (find-file-other-window remember-data-file))
(global-set-key (kbd "C-c R") 'wicked/remember-review-file) ;; (4)

;; Not (global-set-key (kbd "C-c r")... as originally written... Thanks for catching that, Seth!

• 1: Change this to the directory that contains remember.el
• 2: Notes will be saved to this file. You can change this if you want; just keep in mind that this section refers to ~/notes.txt.
• 3: C-c r (remember) is a handy shortcut key for remember. You can also bind it to other shortcut keys such as F9 r.
• 4: This shortcut makes it easy to check your remembered notes.

After you’ve configured Remember, try it out by typing C-c r (remember). Your Emacs frame will be split in two, and one of the windows will be a *Remember* buffer. Type your note. The first line will be the headline, and the rest of the buffer will be the body of the note. If you call C-c r (remember) from a file, the filename will automatically be included at the end of the buffer. Type C-c C-c (remember-buffer) to save the note.

Try it now by typing C-c r (remember) to bring up the buffer, typing in a short note, and using C-c c (remember-buffer) to save it. If you open ~/notes.txt to review your note, you’ll find something like this:

 ** Sat Jan 12 14:43:02 2008 (Your headline goes here)

 Your note body goes here

 /home/sacha/.emacs

You can even save yourself some copying and pasting. Mark a region of text and use C-u C-c r (remember). The selected text will be included in the buffer, so all you have to do is comment on it.

Make a habit of typing C-c C-r or C-u C-c C-r (remember) when you need to remember something. Type C-c C-c (remember-buffer) to get back to work, knowing that your notes have been safely saved in your ~/notes.txt file.

Reviewing Your Notes

Use C-c R (wicked/remember-review-file) to check your notes, or open ~/notes.txt yourself.
To search your notes, use C-c R (wicked/remember-review-file) to open the file, then use C-s (isearch-forward) to search for words interactively, or use M-x occur to find all lines containing a word.

You may notice that the default format that Remember uses is an outline format that is compatible with Org and Allout, both of which have powerful outline-editing tools. I prefer Org’s outline-editing commands, and you’ll learn about them in the “Outline Notes with Org” section. Here’s a quick summary.

First, you need to switch the buffer to Org mode by typing M-x org-mode. To automatically open your notes file in Org mode, add

-*- mode: org -*-

to the first line of your ~/notes.txt. Then, when you open your ~/notes.txt file, it will be in Org mode.

You can quickly collapse or expand all the outline entries by pressing S-TAB (org-shifttab). To collapse or expand a single entry, move the point to the headline (the line beginning with *, **, or any number of asterisks), then press TAB (org-cycle). To move an entry up or down, move the point to the headline and press S-UP (org-shiftup) or S-DOWN (org-shiftdown). To demote or promote a heading, press M-RIGHT (org-metaright) or M-LEFT (org-metaleft).

You can treat ~/notes.txt as your inbox, and keep your organized notes in another file or groups of files. Cut and paste the text between the files to clear your inbox, and use M-x grep to search multiple files. Alternatively, you can keep all of your notes in one large text file, and use C-s (isearch-forward) and M-x occur to search for information.

Now you know the basics of remembering information, saving it into a file, and reviewing the file. By default, Remember annotates your notes with a filename, if you were looking at a file when you called C-c r (remember). As you learn more about Emacs, you may want to configure Remember to add more intelligent annotations and other text to the Remember buffer. The more work Remember does for you, the less work you have to do!

Enabling annotation functions

The easiest way to get Remember to automatically understand mail messages, Web pages, info files, BBDB contact records, and other sources of information in Emacs is to use either Org or Planner. To learn how to integrate Remember with either Org or Planner, read the section on “Outline Your Notes with Org” and “Writing Your Journal with Planner”.

You can also define your own annotation functions. When you call C-c r (remember) from a buffer, Remember goes through each of the functions in remember-annotation-functions, and it uses the first non-nil value returned.

For example, you may work with many temporary buffers that don’t have filenames. To create an annotation function that adds buffer names, add the following code to your ~/.emacs after the basic Remember configuration code:

(eval-after-load 'remember
  '(progn
    (add-to-list 'remember-annotation-functions 'buffer-name t)))

This adds buffer-name to the end of the annotation list, making it a last resort.

What if you want line numbers included with the filename or buffer name? You could replace the previous code with this:

(defun wicked/remember-line-numbers-and-file-names ()
 "Return FILENAME line NUMBER."
 (save-restriction
  (widen)
   (format " %s line %d"
    (or (buffer-file-name) (buffer-name))
    (line-number-at-pos))))
(eval-after-load 'remember
  '(progn
     (add-to-list 'remember-annotation-functions
                  'wicked/remember-line-numbers-and-file-names)))

With that code, C-c r (remember) will automatically pick up the line number from your file or buffer.

By default, Remember saves your notes to a plain-text file, so you’ll have to open the files manually. The command M-x ffap or find-find-file-at-point may be convenient. If you want hyperlinks that you can visit easily, consider saving your notes in an Org or Planner file instead.

Now you’ve got context. What else can you do with the Remember buffer?

Adding Other Text to the Remember Buffer

Remember has plenty of hooks that let you modify the behavior. For example, you might want to insert a random tagline or fortune-cookie saying whenever you create a note. This is a fun way to encourage yourself to write more, because then there’s a little surprise every time you open a Remember buffer.

Here is a totally small-scale way to use random lines from a text file. Let’s say that you have a text file made up of movie quotes, taglines, knock-knock jokes, or short fortune-cookie sayings. When I wrote this code, I used Japanese/English sentence pairs about cats, because I was studying Japanese. You can use whatever tickles your fancy, as long as this text file (~/taglines.txt) has one line per saying.

(defun wicked/random-tagline (&optional file)
  "Return a random tagline."
  (with-current-buffer (find-file-noselect (or file "~/taglines.txt"))
    (goto-char (random (point-max)))
    (let ((string
           (buffer-substring (line-beginning-position)
                             (line-end-position))))
      string)))

(eval-after-load 'remember
  '(progn
     (defadvice remember (after wicked activate)
       "Add random tagline."
       (save-excursion
         (goto-char (point-max))
         (insert "\n\n" (wicked/random-tagline) "\n\n")))))

If you want multi-line sayings, look into the Emacs fortune cookie package, and replace wicked/random-tagline with a function that returns a random string.

This code modifies the behavior of C-c r (remember) by inserting a random tagline after the buffer has been prepared. You can use the same idea to insert a timestamp noting the time you started, use a template, or modify the text in other ways.

Saving to Different Places

You can also change how Remember saves its notes. For example, if you want all of the notes that contain the word “:EMACS:” or “:WORK:” to go into separate files, you can add this code to your ~/.emacs:

(defvar wicked/remember-keywords
  '((":EMACS:" . "~/emacs.txt")
    (":WORK:" . "~/work.txt"))
  "*List of (REGEXP . FILENAME).
If an entry matches REGEXP, it will be storied in FILENAME.
The first regular expression that matches is used.")
(eval-after-load 'remember
  '(progn
     (defadvice remember-region (around wicked activate)
       "Save notes matching `wicked/remember-keywords' elsewhere."
       (let* ((b (or beg (min (point) (or (mark) (point-min)))))
	      (e (or end (max (point) (or (mark) (point-max)))))
	      (string (buffer-substring-no-properties b e))
	      (done nil)
	      (keywords wicked/remember-keywords))
	 (while keywords
	   (when (string-match (caar keywords) string)
	     (let ((remember-data-file (cdar keywords)))
	       ad-do-it)
	     (setq keywords nil done t))
	   (setq keywords (cdr keywords)))
	 (unless done
	   ad-do-it)))))

You can even configure Remember to use different handler functions. This chapter covers several note-taking systems for Emacs, and you may want to use Remember to save to more than one note-taking system. For example, you can set up C-c r p to start a Remember buffer that saves to Planner, and C-c r o to start a Remember buffer that saves to Org. Here’s the code for your ~/.emacs:

(defun wicked/remember-to-org ()
  "Remember to Org."
  (let ((remember-annotation-functions   ;; (1)
	 (cons 'org-remember-annotation
	       remember-annotation-functions)))
    (remember)
    (set (make-variable-buffer-local
	  'remember-handler-functions)
	 '(org-remember-handler))))      ;; (2)

(defun wicked/remember-to-planner ()
  "Remember to Planner."
  (let ((remember-annotation-functions   ;; (3)
	 (append planner-annotation-functions
		 remember-annotation-functions)))
    (remember)
    (set (make-variable-buffer-local
	  'remember-handler-functions)
	 '(remember-planner-append))))      ;; (4)

(global-unset-key (kbd "C-c r"))  ;; (5)
(global-set-key (kbd "C-c r o") 'wicked/remember-to-org)
(global-set-key (kbd "C-c r p") 'wicked/remember-to-planner)

• 1: We need to make sure that Org-compatible links are created.
• 2: This makes C-c C-c save the note to the Org file.
• 3: Planner uses a list of annotation functions, so we add all of them to the beginning of the list.
• 4: This makes C-c C-c save the note to the Planner page.
• 5: We need to unset keyboard shortcuts before we can set longer keyboard shortcuts that start with the same sequence.

To learn more about configuring Remember with the different note-taking systems, read the sections for those systems.

Random Emacs symbol: compilation-find-file – Function: Find a buffer for file FILENAME.

I fell in love with Emacs because of Planner and Remember. I got hooked. By that, I mean that I discovered the joys of modifying the behavior of other people’s code through hooks. Well-written Emacs code has plenty of entry points so that you can change things without rewriting everything, and both Planner and Remember were well-written. The flexibility that John Wiegley had built into Planner and Remember (in his spare time!) convinced me that I absolutely needed to learn Emacs Lisp. I e-mailed John, singing high praises about Planner and volunteering to help track down any bug reports so that I could learn in the process of helping others.

What did he do? He made me the maintainer for Planner, the go-to person, the buck-stops-here person. Was I terrified? My first time being in charge of an open source project, a language that I had just begun to learn, and schoolwork to do on top of all of that… Of course I was scared. I found a way to make working on Planner part of my final-year project (might as well get academic credit for hacking on it!), made the Emacs Lisp Intro and the Emacs Lisp manuals my weekend, bedtime, and mealtime reading. Back then, Planner.el was one file, which made it a little less intimidating.

At first, I found it hard to think of a way to improve Planner, which was already a great way to track my tasks. Remember was also a good way to jot down short notes that were automatically added to my Planner day page. Both Planner and Remember had some support for picking up hyperlinks to whatever I was looking at when I created the task or note. What was missing? Timestamps so that I could tell when I wrote a note; more link support so that I could automatically link to a file or a mail message or a contact record; private, perhaps even encrypted segments; copying a note to another page; publishing a blog as RSS… For the next few years, I worked with incredible people who had thought
long and hard about how they managed their tasks and their notes, and
who taught me how they did things through their feature requests and
their code. I became more comfortable in Emacs Lisp than in any other computer language I knew (eventually maintaining Remember as well), I had formed a thriving community of about 200 people around the world who also contributed ideas and code, and I was hooked.

Because I could change so much about Emacs, I had so much fun
coming up with creative applications. When a friend and I
were studying Japanese, she wanted to know if I could set up Emacs with
flashcards and example sentences so that we could learn words in
context. I knew that Jim Breen had a huge database of example Japanese
sentences with their English translations and that this was freely
available on the Net. I downloaded the database and–one crazy idea
fueled by another in that natural sugar high that can happen when two
geeks start brainstorming–I made our learning fun by
extracting all the cat- and kitten-related sentences. This is how I
learned about such wonderfully onomatopeic words as “fuwafuwashita”,
which means “fluffy”. We both adored cute little kitties, and reading
example sentences about these furballs would make our studies
delightful. Now, where could we put these sentences so that we’d run
into them often? Aha! Remember! Every time we wrote a blog post or
needed a scratch pad to store some information, we would be rewarded by a
short sentence in Japanese about cute little kittens, and we could look
up the meaning and pronunciation with custom keyboard shortcuts. And
hey, it worked for me–I found myself writing because I wanted to see
what sentence would come up, and I found myself learning because I’d
run into all these cool words. For variety, I switched to sentences
about computers and other topics that I wanted to be able to discuss.
Try getting your blog editor or note manager to reward you each time you share a story or write down a tidbit of information. There is incredible power in customizing your note-taking system to just the way you think, and when you combine that with the delight of a surprise, you have something that is difficult to do with any other system–paper-based or electronic, text-based or graphical, commercial or free.

I turned the projects over to other maintainers when I started with graduate school, but as you can see, I still love working with Emacs. My mind does this little somersault of joy when I pull off a particularly nifty tweak or when I run across a beautiful piece of code. And yes, one of the reasons why I’m writing this book about Emacs is because I’d love to have an opportunity to explore all the nooks and crevices of this piece of software–well, not all, because by the time the book hits press, people will have added even more.

In this chapter on notetaking, I hope to be able to show you the mechanics of note-taking in Emacs–but more than that, I hope to show the sheer joy of having a system that you can play with, of not just filing notes in a database but M-x remember-ing in a system that will make you smile or laugh, of shaping and reshaping things until they fit you like clay fits your hand. Come and have fun.