Wicked Cool Emacs: BBDB: Set up BBDB

The main address book and contact management module for Emacs is the Insidious Big Brother Database (BBDB), which can be integrated into several mail clients and other modules within Emacs. If you use BBDB to keep track of contact information, you’ll be able to look up phone numbers or add notes to people’s records from your Emacs-based mail. Even if you don’t do e-mail within Emacs, you’ll find that BBDB’s customizability makes it surprisingly powerful.

In this project, you will learn how to set up BBDB as a basic address book. The BBDB homepage is at http://bbdb.sourceforge.net/. The development version fixes a number of bugs, so I recommend you try it instead of the stable version. However, if you are on Microsoft Windows or you do not have development tools handy, you might find the stable version easier to install. As of this writing, the stable version (2.35) can be downloaded from http://bbdb.sourceforge.net/bbdb-2.35.tar.gz . Download and unpack it to ~/elisp/bbdb-2.35, and save the pre-built bbdb-autoloads.el from http://bbdb.sourceforge.net/bbdb-autoloads.el into ~/elisp/bbdb-2.35/lisp .

To check out the development version, change to your ~/elisp directory and type in the following lines at the command prompt:

cvs -d :pserver:[email protected]:/cvsroot/bbdb login
cvs -d :pserver:[email protected]:/cvsroot/bbdb checkout bbdb

You should now have a directory called ~/elisp/bbdb. Change to that directory and run the following commands:

autoconf
./configure
make autoloads
make all

After installing either the stable or development version of BBDB, include it in your load-path by adding the appropriate line to your ~/.emacs:

(add-to-list 'load-path "~/elisp/bbdb-2.35/lisp")    ;; (1)
(add-to-list 'load-path "~/elisp/bbdb/lisp")         ;; (2)

(require 'bbdb) ;; (3)
(bbdb-initialize 'gnus 'message)   ;; (4)
(setq bbdb-north-american-phone-numbers-p nil)   ;; (5)

Use either ~/elisp/bbdb-2.35/lisp(1) or ~/elisp/bbdb/lisp(2) depending on the location of the installed BBDB Lisp files. Then load BBDB(3) and configure it for the Gnus mail client and the Message mode used to compose mail(4). It’s also a good idea to configure BBDB to accept any kind of phone number(5), not just North American numbers with a particular syntax.

After you evaluate this code or restart Emacs, BBDB should be part of your system. Next step: enter your address book!

Wicked Cool Emacs: BBDB: Work with Records

Creating Records

Creating a record in BBDB is not like creating a record in graphical address book programs. You will be prompted for each field through the minibuffer, one field at a time. Don’t worry about making mistakes while entering data, as you can always edit the records afterwards.

To create a record, use the command M-x bbdb-create. Here are the prompts you’ll encounter:

Prompt Notes Example
Name Full name John Doe
Company Company or organization ACME
Network Address E-mail address (comma-separated list) [email protected]
Address Description Short identifier for address (Home, Office, etc.) – tab completion available. Leave blank if you have no address information, or if you are done. Home
Street, line 1 Street address, line 1 (not including city, state, postal code or country) 1 Acme Road
Street, line … Street address, more lines – press RET to indicate the end of the street address
City Acme City
State Abbreviations are okay. Consistency helps. AC
Country Acme Country
Phone Location Short identifier for phone number (Home, Office, etc.) – tab completion available. Leave blank if you have no phone information, or if you are done. Home
Phone Phone number. I tend to specify the full number, using spaces to break it into readable chunks. +1 111 111 1111 x1111
Additional Comments Notes about the person, such as interests, how you met, and so on Likes rockets

Press RET to skip any fields for which you don’t have information. To
cancel the entry process, type C-g (keyboard-quit).

After you create the record, Emacs will display the record in another
window. You can then switch to the record and edit it. See Project XXX: Edit a BBDB record.

Searching Records

To search for a specific record, type M-x bbdb, or press b
(bbdb) while in the BBDB buffer. This prompts for a regular
expression and searches the name, company, network address, and notes
fields of all the records for a match against the regular expression
supplied. M-x bbdb-name, M-x bbdb-company, M-x bbdb-net, M-x
bbdb-notes, and M-x bbdb-phones search the corresponding fields only.

Updating Records

After creating or searching for a record, you can switch to the BBDB
window to edit it. Press C-o (bbdb-insert-field) to insert
custom fields. You can use tab completion on existing field names, and
you can also define your own fields by typing any field name. For
example, you may want to store people’s job titles in a field called
“job”.

To edit the value of a field, move your cursor to the field and press
e (bbdb-edit-current-field) to change the value. To delete
a field, move your cursor to the field and press C-k
(bbdb-delete-current-field-or-record).

Deleting Records

To delete an entire record, move the text cursor to the name and press C-k (bbdb-delete-current-field-or-record). You will be prompted for confirmation. Be careful! If you mistakenly delete a record, there’s no easy way to get it back. Fortunately, BBDB stores its data in a plain text file (~/.bbdb). Back up that file regularly and you’ll be able to recover from mistakes. You can also set up automatic file backups in Emacs (see Project XXX: Make Automatic Backups).

Now you know how to work with individual records. How can you import your address book information from other programs?

Wicked Cool Emacs: BBDB: Import CSV and vCard Files

If you have many contacts in another address book program, you can import them into BBDB. Two popular formats are comma-separated value files (CSV) and vCard files (VCF).

Project XXX: Import a CSV File into BBDB

To import a CSV file into BBDB, you will need csv.el from http://ulf.epplejasper.de/downloads/csv.el and lookout.el from http://ulf.epplejasper.de/downloads/lookout.el . Save both files to your ~/elisp directory. Make sure that your ~/elisp directory is in your load-path by adding the following line to your ~/.emacs:

(add-to-list 'load-path "~/elisp")

Export your contacts as an Outlook-style CSV file, then open the file in Emacs. After loading the following code, call M-x wicked/bbdb-import-csv-buffer to merge the CSV data into your address book. Emacs will try to update existing records based on the e-mail address or name provided, creating new records if necessary. After Emacs updates the records, the relevant records are displayed in the *BBDB* buffer. Here is the code to make that work:

ch6-bbdb-import-csv-buffer.el:

(require 'lookout)
(defconst wicked/lookout-bbdb-mapping-table-outlook
  '(("name" "Name")
    ("net" "E-mail Address")
    ("notes" "Notes")
    ("phones" "Mobile Phone"
     "Home Phone"
     "Home Phone 2"
     "Home Fax"
     "Business Phone"
     "Business Phone 2"
     "Business Fax"
     "Other Phone"
     "Other Fax")
    ("addr1" "Home Address")
    ("addr2" "Business Address")
    ("addr3" "Other Address")
    ("lastname" "Last Name")
    ("firstname" "First Name")
    ("job" "Job Title")
    ("company" "Company")
    ("otherfields" ""))
  "Field mappings for Outlook-type CSVs exported from Outlook, Gmail, LinkedIn, etc.")

(defun wicked/bbdb-import-csv-line (line)
  "Import LINE as a CSV, trying to merge it with existing records."
  (let* (record
	 (name  (lookout-bbdb-get-value "name" line))
	 (lastname (lookout-bbdb-get-value "lastname" line))
	 (firstname (lookout-bbdb-get-value "firstname" line))
	 (company   (lookout-bbdb-get-value "company" line))
         (job       (lookout-bbdb-get-value "job" line))
	 (net       (lookout-bbdb-get-value "net" line))
	 (addr1     (lookout-bbdb-get-value "addr1" line))
	 (addr2     (lookout-bbdb-get-value "addr2" line))
	 (addr3     (lookout-bbdb-get-value "addr3" line))
	 (phones    (lookout-bbdb-get-value "phones" line t)) ;; !
	 (notes     (lookout-bbdb-get-value "notes" line ))
         (j (concat job ", " company))
	 (otherfields (lookout-bbdb-get-value "otherfields" line t))
	 (addrs nil)
         name-search
	 (message ""))
    (if (string= company "") (setq company nil))
    (if (string= notes "") (setq notes nil))
    (if (string= name "") (setq name nil))
    (setq name-search (concat "^" (or name (concat firstname " " lastname))))
    (setq record (or (bbdb-search (bbdb-records) nil nil net)
		     (bbdb-search (bbdb-records) name-search)))
    (if record
	(progn
	  ;; Matching records found, update first matching record
	  (setq record (car record))
	  (let ((nets (bbdb-record-net record)))
	    (unless (member net nets)
	      ;; New e-mail address noticed, add to front of list
	      (add-to-list 'nets net)
	      (bbdb-record-set-net record nets)
	      (message "%s: New e-mail address noticed: %s"
		       (or name (concat firstname " " lastname)) net)))
	  ;; Check if job title and company have changed
	  (when (or job company)
	    (cond
	     ((string= (or (bbdb-record-company record) "") "")
	      (bbdb-record-set-company record j))
	     ((string= (bbdb-record-company record) j)
	      nil)
	     (t
	      (bbdb-record-set-notes
	       record
	       (concat "Noticed change from job title of "
		       (bbdb-record-company record)
		       "\n"
		       (bbdb-record-notes record)))
	      (message "%s: Noticed change from job title of %s to %s"
		       (or name (concat firstname " " lastname))
		       (bbdb-record-company record) j)
	      (bbdb-record-set-company record j)))))
      ;; No record found, create record
      (if (and addr1 (> (length addr1) 0))
	  (add-to-list 'addrs
		       (vector "Address 1" (list addr1) "" "" "" "")))
      (if (and addr2 (> (length addr2) 0))
	  (add-to-list 'addrs
		       (vector "Address 2" (list addr2) "" "" "" "")))
      (if (and addr3 (> (length addr3) 0))
	  (add-to-list 'addrs
		       (vector "Address 3" (list addr3) "" "" "" "")))
      (setq record (list
		    (wicked/lookout-bbdb-create-entry
		     (or name (concat firstname " " lastname))
		     (concat job ", " company)
		     net
		     addrs
		     phones
		     notes
		     otherfields))))
    record))
  
(defun wicked/lookout-bbdb-create-entry (name company net addrs phones notes
					      &optional otherfields)
  (when (or t (y-or-n-p (format "Add %s to bbdb? " name)))
    ;;(message "Adding record to bbdb: %s" name)
    (let ((record (bbdb-create-internal name company net addrs phones notes)))
      (unless record (error "Error creating bbdb record"))
      (mapcar (lambda (i)
		(let ((field (make-symbol (aref i 0)))
		      (value (aref i 1)))
		  (when (and value (not (string= "" value)))
		    (bbdb-insert-new-field record field value))))
	      otherfields)
      record)))

(defun wicked/bbdb-import-csv-buffer ()
  "Import this buffer."
  (interactive)
  (let ((lookout-bbdb-mapping-table
	 wicked/lookout-bbdb-mapping-table-outlook))
    (bbdb-display-records
     (mapcar
      'wicked/bbdb-import-csv-line
      (csv-parse-buffer t)))))

Project xxx: Import a vCard File into BBDB

To import a vCard file (VCF) into BBDB, you will need vcard.el from http://www.splode.com/~friedman/software/emacs-lisp/src/vcard.el and bbdb-vcard-import.el from http://www-pu.informatik.uni-tuebingen.de/users/crestani/downloads/bbdb-vcard-import.el . By default, these files allow you to import names and e-mail addresses from vCard files exported from various address book programs. Save vcard.el and bbdb-vcard-import.el to your ~/elisp directory and add the following lines to your ~/.emacs:

(add-to-list 'load-path "~/elisp")
(require 'bbdb-vcard-import)

Back up your ~/.bbdb file before calling M-x bbdb-vcard-import to import a file or M-x bbdb-vcard-import-buffer to import the current buffer. WARNING: If your vCard file includes fields with multiline values, you may get silent errors. Verify your import by browsing through the displayed entries. If some of them have been misread, revert to your backup ~/.bbdb by closing Emacs and copying your backup over the ~/.bbdb file. To fix the multi-line error, include the following lines in your ~/.emacs:

(defun wicked/vcard-parse-region (beg end &optional filter)
  "Parse the raw vcard data in region, and return an alist representing data.
This function is just like `vcard-parse-string' except that it operates on
a region of the current buffer rather than taking a string as an argument.

Note: this function modifies the buffer!"
  (or filter
      (setq filter 'vcard-standard-filter))
  (let ((case-fold-search t)
        (vcard-data nil)
        (pos (make-marker))
        (newpos (make-marker))
        properties value)
    (save-restriction
      (narrow-to-region beg end)
      (save-match-data
        ;; Unfold folded lines and delete naked carriage returns
        (goto-char (point-min))
        (while (re-search-forward "\r$\\|\n[ \t]" nil t)
          (goto-char (match-beginning 0))
          (delete-char 1))
        (goto-char (point-min))
        (re-search-forward "^begin:[ \t]*vcard[ \t]*\n")
        (set-marker pos (point))
        (while (and (not (looking-at "^end[ \t]*:[ \t]*vcard[ \t]*$"))
                    (re-search-forward ":[ \t]*" nil t))
          (set-marker newpos (match-end 0))
          (setq properties
                (vcard-parse-region-properties pos (match-beginning 0)))
          (set-marker pos (marker-position newpos))
          (re-search-forward "\n[-A-Z0-9;=]+:")   ;; change to deal with multiline
          (set-marker newpos (1+ (match-beginning 0))) ;; change to deal with multiline
          (setq value
                (vcard-parse-region-value properties pos (match-beginning 0)))
          (set-marker pos (marker-position newpos))
          (goto-char pos)
          (funcall filter properties value)
          (setq vcard-data (cons (cons properties value) vcard-data)))))
    (nreverse vcard-data)))
;; Replace vcard.el's definition
(fset 'vcard-parse-region 'wicked/vcard-parse-region)

Because address book programs don’t use standard labels for addresses and phone numbers, bbdb-vcard-import.el ignores those fields. For example, Gmail uses the generic field “Label” for address information and does not use separate fields for city, state, zip code, and country. While bbdb-snarf.el makes an attempt to extract addresses from plain text, it seems to be less trouble to export to the Outlook CSV format instead, or even to type the address in yourself. If you want to import addresses, see Project XXX: Import a CSV File into BBDB.

Here’s a partial workaround to enable you to import phone numbers. I tested this code with vCard files from Gmail and LinkedIn. To try it out, add the following modifications to your ~/.emacs:

(defun wicked/bbdb-vcard-merge (record)
  "Merge data from vcard interactively into bbdb."
  (let* ((name (bbdb-vcard-values record "fn"))
	 (company (bbdb-vcard-values record "org"))
	 (net (bbdb-vcard-get-emails record))
	 (addrs (bbdb-vcard-get-addresses record))
	 (phones (bbdb-vcard-get-phones record))
	 (categories (bbdb-vcard-values record "categories"))
	 (notes (and (not (string= "" categories))
		     (list (cons 'categories categories))))
	 ;; TODO: addrs are not yet imported.  To do this right,
	 ;; figure out a way to map the several labels to
	 ;; `bbdb-default-label-list'.  Note, some phone number
	 ;; conversion may break the format of numbers.
	 (bbdb-north-american-phone-numbers-p nil)
	 (new-record (bbdb-vcard-merge-interactively name
						     company
						     net
						     nil ;; Skip addresses
						     phones ;; Include phones
						     notes)))
    (setq bbdb-vcard-merged-records (append bbdb-vcard-merged-records 
					    (list new-record)))))
;; Replace bbdb-vcard-import.el's definition
(fset 'bbdb-vcard-merge 'wicked/bbdb-vcard-merge)

Evaluate this code or restart Emacs, then call M-x bbdb-import-vcard again, which should merge phone numbers into your BBDB records.

Faster mail with Emacs

I spent a few minutes getting offlineimap to synchronize my Gmail messages with dovecot, an IMAP server on my laptop. I also set up Gnus to work with the messages. Now I’m having fun speeding through my inbox. I don’t know why, but my text-based terminal seems a lot zippier and a lot easier to work with than Gmail… =)

I’m glad I’m back to doing my mail in Emacs!

That’s my dad!

Beauty is in eye of autistic youth – INQUIRER.net, Philippine News for Filipinos

He has come up with “the kind of shots that have eluded some of us, even with years of training,” said ace advertising photographer John Chua, who introduced Ian to his new hobby by chance.

That’s my dad – random acts of kindness turn into front-page news! =D

Emacs Gnus: Searching Mail

There are several ways to find messages in Emacs. From the summary
buffer, you can use / o (gnus-summary-insert-old-articles) to display
all or some old messages. You can then scan through the headers in the
summary buffer by using C-s (isearch-forward), or you can limit the
displayed messages with these commands:

Messages from a given author / a gnus-summary-limit-to-author
Messages whose subjects matching a given regular expression / / gnus-summary-limit-to-subject
Messages that match a given extra header / x gnus-summary-limit-to-extra-headers
Messages at least N days old / t gnus-summary-limit-to-age

Limits work on the messages that are currently displayed, so you can
apply multiple limits. If you make a mistake, use / w
(gnus-summary-pop-limit) to remove the previous limit. You can repeat
/ w (gnus-summary-pop-limit) until satisfied. To remove all the
limits, type C-u / w (gnus-summary-popl-limit).

If you specify a prefix, the limit’s meaning is reversed. For
example, C-u / a (gnus-summary-limit-to-author) will remove the
messages from the matching author or authors.

You can use Gnus to search the currently-displayed messages by using
M-s (gnus-summary-search-article-forward) and M-r
(gnus-summary-search-article-backward).

If you want to search a lot of mail, you’ll find NNIR handy. NNIR is a
front-end to mail search engines which can index your mail and return
search results quickly. If you want to use NNIR with a local or remote
IMAP server, you will need to use nnir.el and imap.el. If you download
your mail using fetchmail or connect to a POP3 server and use an nnml
backend, you can use NNIR with a search engine such as swish-e to
search your ~/Mail directory efficiently.

1.6.7.1 Setting up IMAP and NNIR

If you use IMAP, then your mail is stored on the mail server and
you’ll need to use the IMAP search interface to search through
it. Download nnir.el from
http://www.emacswiki.org/cgi-bin/wiki/download/nnir.el and save it to
your ~/elisp directory. You will also need an imap.el that is newer
than the one that comes with Emacs 22. Download imap.el from
http://www.emacswiki.org/cgi-bin/wiki/download/imap.el and save it to
your ~/elisp directory as well. Because Gnus comes with an older
version of imap.el, you will need to make sure that the new version of
imap.el is loaded. Add the following to your ~/.gnus:

(add-to-list 'load-path "~/elisp")
(load-file "~/elisp/imap.el")
(require 'nnir)

Restart your Emacs. You can check if the correct version of imap.el
has been loaded by typing M-x locate-library and specifying
imap.el. If Emacs reports “~/elisp/imap.el”, then Gnus is configured
to use the updated imap.el.

1.6.7.2 Setting up POP3 and NNIR

If you use the configuration for POP3 that is suggested in this
chapter, then your mail is stored in the nnml backend, which uses one
file per message. To search this using NNIR, to install nnir.el and an
external search mail engine. The Namazu search engine runs on Linux,
UNIX, and Microsoft Windows, so that’s what we’ll talk about here. To
find and configure other mail search engines supported by NNIR, check
out the comments in nnir.el.

First, you’ll need to download and install Namazu. If Namazu is
available as a package for your distribution, install it that way, as
it depends on a number of other programs. An installer for Microsoft
Windows can be found at http://www.namazu.org/windows/ . If you need
to build Namazu from source, you can get the source code and instructions
from http://www.namazu.org .

After you’ve installed Namazu, create a directory for Namazu’s index
files, such as ~/.namazu-mail. Then index your mail by typing this at
the command-line:

mknmz --mailnews -O ~/.namazu-mail ~/Mail

and add the following to your ~/.gnus:

(add-to-list 'load-path "~/elisp")
(require 'nnir)
(setq nnir-search-engine 'namazu)
(setq nnir-namazu-index-directory (expand-file-name "~/.namazu-mail"))
(setq nnir-namazu-remove-prefix (expand-file-name "~/Mail"))
(setq nnir-mail-backend gnus-select-method)
1.6.7.3 Searching your mail with NNIR

From the group buffer displayed by M-x gnus, you can type G G
(gnus-group-make-nnir-group) to search your mail for a keyword.

If you’re using the Namazu search engine, then you can use more
sophisticated search queries such as:

Linux Emacs messages that contain both “Linux” and “Emacs”
Linux or Emacs messages that contain either “Linux” or “Emacs”
Emacs not Linux messages that contain “Emacs” but not “Linux”
Emacs and (Linux or Windows) messages that contain “Emacs” and either “Linux” or “Windows”
“apple pie” messages that contain the phrase “apple pie”
{apple pie} messages that contain the phrase “apple pie”
+from:[email protected] messages with [email protected] in the From: header
+subject:”apple pie” messages with the phrase “apple pie” in the Subject: header
+subject:apple +subject:pie messages whose Subject: headers contain both “apple” and “pie”

If
matching messages are found, then you will see a temporary group with
the results. Although you can’t delete messages from this view,
reading and replying to these messages is the same as reading and
replying to regular messages.

To see a message in its original context, type G T
(gnus-summary-nnir-goto-thread) from the summary buffer. This opens
the message’s original group. If Gnus asks you how many articles to
load, press RET to accept the default of all the articles.


This is a draft for the Wicked Cool Emacs book I’m working on. =) Hope it helps!