On this page:

Chapter 6: Being Big Brother (plan)

I haven’t been writing about Emacs lately. Here’s my outline so that you can help keep me honest. =) My next chapter is about the Big Brother Database (BBDB) and contact management in Emacs, which is one of the things that made people laugh when I showed my Emacs configuration at DemoCamp in Toronto. Anyway, here’s what I’m planning to write about:

Chapter 6: Being Big Brother (30 pages)

  • Why use Emacs to manage your contacts?
    What is BBDB?
  • Project xxx: Set up BBDB
    Project xxx: Import contacts: CSV, card
    Project xxx: Create a record
    Project xxx: Search records
  • Mail
    Project xxx: Integrate BBDB with Mail
    Project xxx: Notice e-mail changes
    Project xxx: Filter mail according to record
    Project xxx: Categorize contacts with mail aliases
    Project xxx: Personalize greetings
    Project xxx: Personalize signatures
    Project xxx: Mail merge
    Project xxx: Track last contact
  • Filtering records
    Project xxx: Show a phone list
    Project xxx: Show an address list
    Project xxx: Show no contact since
    Project xxx: Show tag queries
    Project xxx: Remember birthdays
  • More data
    Project xxx: Snarf records
    Project xxx: Add pictures
    Project xxx: Export contacts
    Project xxx: Synchronize contacts
    Project xxx: Synchronize with LinkedIn

BBDB: Show a phone list

When I find myself in an airport, I sometimes take a little time to say hi to a bunch of people who are suddenly just a local call a way. Or sometimes I’m thinking of going somewhere, and instead of flipping through my phone’s address book, I’ll check my computer to see who might be interested.

You can use this function to filter phone numbers in your BBDB based on a regular expression. As usual, leaving the regular expression blank means that all records with phone numbers will be displayed. By default, the function works on the currently displayed records, allowing you to apply multiple filters. You can call it with a universal prefix argument (C-u M-x sacha/bbdb-find-people-with-phones) to match against all contacts in your database.

Here’s the code:

(defun sacha/bbdb-find-people-with-phones (&optional regexp records)
  "Search for phone numbers that match REGEXP in BBDB RECORDS.
Without a prefix argument, filter the list of displayed records.
Call with a prefix argument to search the entire database.  This
works best if you use a consistent format to store your phone
numbers.  The search will strip out non-numeric characters. For
example, +1-888-123-4567 will be treated as +18001234567.

To search for all numbers in Toronto, search for
\"+1\\(416\\|647\\)\". If you search for certain areas
frequently, it might be a good idea to define a function for
them."
  (interactive (list (read-string "Regexp: ")
		     (if current-prefix-arg
			 (bbdb-records)
		       (or bbdb-records (bbdb-records)))))
  (let (filtered next)
    (while records
      (when
          (and (bbdb-record-get-field-internal
		(if (arrayp (car records))
		    (car records)
		  (caar records)) 'phone)
               (or
                (null regexp)
		(string= regexp "")
                (delq nil
                      (mapcar
                       (lambda (phone)
			 (when (string-match regexp (sacha/bbdb-phone-string phone))
			   (concat (bbdb-phone-location phone) ": " (bbdb-phone-string phone))))
                       (bbdb-record-get-field-internal
                        (if (arrayp (car records))
                            (car records)
                          (caar records)) 'phone)))))
        (setq filtered (cons (if (arrayp (car records))
                                 (car records)
                               (caar records)) filtered)))
      (setq records (cdr records)))
    (bbdb-display-records (nreverse filtered))))

(defun sacha/bbdb-phone-string (&optional phone)
  "Strip non-numeric characters from PHONE, except for +."
  (replace-regexp-in-string "[^+1234567890]" "" (bbdb-phone-string phone)))
   
(defun sacha/bbdb-yank-phones ()
  "Copy a phone list into the kill ring."
  (interactive)
  (kill-new
   (mapconcat
    (lambda (record)
      (mapconcat
       (lambda (phone)
	 (concat (bbdb-record-name (car record)) "\t" 
                 (bbdb-phone-location phone) "\t"
		 (bbdb-phone-string phone)))
        (bbdb-record-get-field-internal (car record) 'phone)
        "\n"))
    bbdb-records
    "\n")))

Wicked Cool Emacs: BBDB: Keeping track of contact dates

I hadn’t realized just how much I missed my Big Brother Database until today. Three networking events packed into one week meant that I hadn’t set aside enough time for follow up, and I felt my memories of the conversations getting a little hazy. Fortunately I’d taken some notes on my Palm, but I knew I had to get it into some kind of contact management system quickly, and Gmail Contacts just wasn’t compelling enough for me. So it’s back to Emacs, plain text files, and a surprisingly sophisticated contact manager.

I also promised to do some work on the book today, so everything dovetailed nicely.

The following bit of code helps me filter displayed contacts to show only the people I haven’t contacted since a certain date. This is handy for remembering to keep in touch with old friends, for example. Or at least it would be handy if I used it more often and if I actually sent the letters that pile up in my e-mail drafts and my snail mail outbox… but at least it’s a step in the right direction.

If you want to know who you have or haven’t talked to in a while, you need to do two things. First, you need to keep track of when you talked to people. Second, you need to generate reports.

To be able to quickly add contact notes to BBDB records, add the following to your ~/.emacs:

ch6-bbdb-ping.el:

(define-key bbdb-mode-map "z" 'wicked/bbdb-ping-bbdb-record)
(defun wicked/bbdb-ping-bbdb-record (bbdb-record text &optional date regrind)
  "Adds a note for today to the current BBDB record.
Call with a prefix to specify date.
BBDB-RECORD is the record to modify (default: current).
TEXT is the note to add for DATE.
If REGRIND is non-nil, redisplay the BBDB record."
  (interactive (list (bbdb-current-record t)
                     (read-string "Notes: ")
                     ;; Reading date - more powerful with Planner, but we'll make do if necessary
                     (if (featurep 'planner)
                         (if current-prefix-arg (planner-read-date) (planner-today))
                       (if current-prefix-arg
                           (read-string "Date (YYYY.MM.DD): ")
                         (format-time-string "%Y.%m.%d")))
                     t))
  (bbdb-record-putprop bbdb-record
                       'contact
                       (concat date ": " text "\n"
                               (or (bbdb-record-getprop bbdb-record 'contact))))
  (if regrind
      (save-excursion
        (set-buffer bbdb-buffer-name)
        (bbdb-redisplay-one-record bbdb-record)))
  nil)

You can then use z in BBDB buffers to add a quick note to the “contact” field of the current record. The date is automatically noted. You can create a note for a specific date by calling {{C-u wicked/bbdb-ping-bbdb-record}} with a prefix argument. For convenience, the suggested configuration binds this to “z”, because it was one of the few unbound keys I could find. Use this after you meet, call, or e-mail people, and write down a short note about the conversation you had. You might find these notes useful later on.

If you met a number of people at an event in the past and you have Planner installed and loaded, you can use {{planner-timewarp}} to set the effective date to another date. To return to today, use {{M-x planner-timewarp nil}}.

To automatically add a datestamped copy of sent e-mail subjects to people’s BBDB records, add the following to your ~/.gnus:

ch6-bbdb-message-add-subject.el:

(defun wicked/message-add-subject-to-bbdb-record ()
  "Add datestamped subject note for each person this message has been sent to."
  (let* ((subject (concat (format-time-string "%Y.%m.%d")
                          ": E-mail: " (message-fetch-field "Subject") "\n"))
         (bbdb-get-addresses-headers
          (list (assoc 'recipients bbdb-get-addresses-headers)))
         records)
    (setq records
          (bbdb-update-records
           (bbdb-get-addresses nil gnus-ignored-from-addresses 'gnus-fetch-field)
           nil nil))
    (mapc (lambda (rec)
            (bbdb-record-putprop rec
                                 'contact
                                 (concat subject
                                         (or
                                          (bbdb-record-getprop rec 'contact)
                                          ""))))
          records)))
(add-hook 'message-send-hook 'wicked/message-add-subject-to-bbdb-record)

Now that you have the data, how can you use it to filter? Add the following to your ~/.emacs:

ch6-bbdb-show-only-no-contact-since.el:

(defun wicked/bbdb-show-only-no-contact-since (date &optional reverse records)
  "Show only people who haven't been pinged since DATE or at all.
If REVERSE is non-nil, show only the people you've contacted on or since DATE.
Call with a prefix argument to show only people you've contacted on or since DATE."
  (interactive (list
                (if (featurep 'planner)
                    (planner-read-date)
                  (read-string "Date (YYYY.MM.DD): "))
                current-prefix-arg (or bbdb-records (bbdb-records))))
  (let (new-records
        last-match
        timestamp
        omit
        notes)
    (while records
      ;; Find the latest date mentioned in the entry
      (let ((timestamp (wicked/bbdb-last-date
                        (if (vectorp (car records))
                            (car records)
                          (caar records)))))
        (if (if reverse
                ;; Keep if contact is >= date
                (null (string< timestamp date))
              ;; Keep if date > contact
              (string> date timestamp))
            (add-to-list 'new-records (if (vectorp (car records))
                            (car records)
                          (caar records)) t)))
      (setq records (cdr records)))
    (bbdb-display-records new-records)))

(defun wicked/bbdb-last-date (rec)
  "Return the most recent date for REC or nil if none.
Dates should be in the form YYYY.MM.DD.  The first date in the
notes field and the first date in the contact field are used, so
dates should be in reverse chronological order."
  (let* ((wicked/date-regexp
          "\\<\\([1-9][0-9][0-9][0-9]\\)\\.\\([0-9][0-9]?\\)\\.\\([0-9][0-9]?\\)\\>")
         ;; Get the first date mentioned in the notes field
         (notes-date
          (or (and (string-match wicked/date-regexp (or (bbdb-record-notes rec) ""))
                   (match-string 0 (or (bbdb-record-notes rec) "")))
              "0000.00.00"))
         ;; Get the first date mentioned in the contact field
         (contact-date
          (or (and (string-match wicked/date-regexp (or (bbdb-record-getprop rec 'contact) ""))
                   (match-string 0 (or (bbdb-record-getprop rec 'contact) "")))
              "0000.00.00")))
    ;; Compare the two dates
    (or (if (string< notes-date contact-date) contact-date notes-date)
        "0000.00.00")))

To generate a report, use {{M-x wicked/bbdb-show-only-no-contact-since}} and specify the date. These functions are much easier to use with Planner’s date-handling functions. Planner can read dates like “-1″ (yesterday), “-7fri” (seven Fridays ago), “2″ (the second of this month), “1.2″ (January 2 in this year), and “2007.01.02″ (January 2, 2007).

You can also flip the filter by using the universal prefix argument ({{C-u M-x wicked/bbdb-show-only-no-contact-since}}) to show only the people you’ve contacted since a certain date. This is good for knowing the size of your active network. Because the filter works on displayed records, you can combine it to find all the people you talked to last year but not this year. You can also combine it with other filters to find all the people you’ve marked as friends, but who you haven’t talked to in three months. Then you can send a personalized e-mail or make a phone list, and get back in touch. And that’s how you keep track of your contact dates!

Wicked Cool Emacs: BBDB: Use nicknames and custom salutations

Update 2014-05-13: The original code is for BBDB version 2. Thomas Morgan sent this update which makes it work with BBDB version 3 – see below.

I like starting my e-mail with a short salutation such as “Hello, Mike!”, “Hello, Michael”, or “Hello, Mikong!”, but it can be hard to remember which nicknames people prefer to use, and calling someone by the wrong name is a bit of a faux pas. Sometimes people sign e-mail with their preferred name, but what if you haven’t sent e-mail to or received e-mail from someone in a while? In this project, you’ll learn how to set up my BBDB to remember people’s nicknames for you using a custom “nick” field, and to use those nicknames when replying to messages in Gnus or composing messages from my BBDB.

The nickname code worked so well that I started thinking of what else I could customize. It was easy to go from nicknames to personalized salutations. This hack started because one of my friends is from Romania, so I thought I’d greet her in Romanian with “Salut, Letitia!” instead of just “Hello, Letitia!”. The code in this project uses a “hello” field to store these salutations in your BBDB.

To set up personalized nicknames and salutations, add the following code to your ~/.emacs:

For BBDB v2

(defvar wicked/gnus-nick-threshold 5 "*Number of people to stop greeting individually. Nil means always greet individually.")  ;; (1)
(defvar wicked/bbdb-hello-string "Hello, %s!" "Format string for hello. Example: \"Hello, %s!\"")
(defvar wicked/bbdb-hello-all-string "Hello, all!" "String for hello when there are many people. Example: \"Hello, all!\"")
(defvar wicked/bbdb-nick-field 'nick "Symbol name for nickname field in BBDB.")
(defvar wicked/bbdb-salutation-field 'hello "Symbol name for salutation field in BBDB.")

(defun wicked/gnus-add-nick-to-message ()
  "Inserts \"Hello, NICK!\" in messages based on the recipient's nick field."
  (interactive)
  (save-excursion
    (let* ((bbdb-get-addresses-headers ;; (2)
            (list (assoc 'recipients bbdb-get-addresses-headers)))
           (recipients (bbdb-get-addresses
                        nil
                        gnus-ignored-from-addresses
                        'gnus-fetch-field))
           recipient nicks rec net salutations)
      (goto-char (point-min))
      (when (re-search-forward "--text follows this line--" nil t)
        (forward-line 1)
        (if (and wicked/gnus-nick-threshold 
                 (>= (length recipients) wicked/gnus-nick-threshold))
            (insert wicked/bbdb-hello-all-string "\n\n") ;; (3)
          (while recipients
            (setq recipient (car (cddr (car recipients))))
            (setq net (nth 1 recipient))
            (setq rec (car (bbdb-search (bbdb-records) nil nil net)))
            (cond
             ((null rec) ;; (4)
              (add-to-list 'nicks (car recipient))) 
             ((bbdb-record-getprop rec wicked/bbdb-salutation-field) ;; (5)
              (add-to-list 'salutations 
                           (bbdb-record-getprop rec wicked/bbdb-salutation-field))) 
             ((bbdb-record-getprop rec wicked/bbdb-nick-field) ;; (6)
              (add-to-list 'nicks 
                           (bbdb-record-getprop rec wicked/bbdb-nick-field)))
             (t (bbdb-record-name rec))) ;; (7) 
            (setq recipients (cdr recipients))))
        (when nicks ;; (8)
          (insert (format wicked/bbdb-hello-string 
                          (mapconcat 'identity (nreverse nicks) ", "))
                  " "))
        (when salutations ;; (9)
          (insert (mapconcat 'identity salutations " ")))
        (when (or nicks salutations)
          (insert "\n\n")))))
  (goto-char (point-min)))

(defadvice gnus-post-news (after wicked/bbdb activate)
  "Insert nicknames or custom salutations."
  (wicked/gnus-add-nick-to-message))

(defadvice gnus-msg-mail (after wicked/bbdb activate)
  "Insert nicknames or custom salutations."
  (wicked/gnus-add-nick-to-message))

(defadvice gnus-summary-reply (after wicked/bbdb activate)
  "Insert nicknames or custom salutations."
  (wicked/gnus-add-nick-to-message))

For BBDB v3

;; This version is for BBDBv3 - thanks, Thomas!

(defvar wicked/gnus-nick-threshold 5 "*Number of people to stop greeting individually. Nil means always greet individually.")  ;; (1)
(defvar wicked/bbdb-hello-string "Hello, %s!" "Format string for hello. Example: \"Hello, %s!\"")
(defvar wicked/bbdb-hello-all-string "Hello, all!" "String for hello when there are many people. Example: \"Hello, all!\"")
(defvar wicked/bbdb-nick-field 'nick "Symbol name for nickname field in BBDB.")
(defvar wicked/bbdb-salutation-field 'hello "Symbol name for salutation field in BBDB.")

(defun wicked/gnus-add-nick-to-message ()
  "Inserts \"Hello, NICK!\" in messages based on the recipient's nick field."
  (interactive)
  (let ((recipients (bbdb-get-address-components))
        recipient nicks rec net salutations)
    (goto-char (point-min))
    (when (re-search-forward "--text follows this line--" nil t)
      (forward-line 1)
      (if (and wicked/gnus-nick-threshold 
               (>= (length recipients) wicked/gnus-nick-threshold))
          (insert wicked/bbdb-hello-all-string "\n\n") ;; (3)
        (while recipients
          (setq recipient (car recipients))
          (setq net (nth 1 recipient))
          (setq rec (car (bbdb-search (bbdb-records) nil nil net)))
          (cond
           ((null rec) ;; (4)
            (add-to-list 'nicks (car recipient))) 
           ((bbdb-record-xfield rec wicked/bbdb-salutation-field) ;; (5)
            (add-to-list 'salutations 
                         (bbdb-record-xfield rec wicked/bbdb-salutation-field))) 
           ((bbdb-record-xfield rec wicked/bbdb-nick-field) ;; (6)
            (add-to-list 'nicks 
                         (bbdb-record-xfield rec wicked/bbdb-nick-field)))
           (t
            (add-to-list 'nicks
                         (car (split-string (bbdb-record-name rec)))))) ;; (7) 
          (setq recipients (cdr recipients))))
      (when nicks ;; (8)
        (insert (format wicked/bbdb-hello-string 
                        (mapconcat 'identity (nreverse nicks) ", "))
                " "))
      (when salutations ;; (9)
        (insert (mapconcat 'identity salutations " ")))
      (when (or nicks salutations)
        (insert "\n\n")))))

(defadvice gnus-post-news (after wicked/bbdb activate)
  "Insert nicknames or custom salutations."
  (wicked/gnus-add-nick-to-message))

(defadvice gnus-msg-mail (after wicked/bbdb activate)
  "Insert nicknames or custom salutations."
  (wicked/gnus-add-nick-to-message))

(defadvice gnus-summary-reply (after wicked/bbdb activate)
  "Insert nicknames or custom salutations."
  (wicked/gnus-add-nick-to-message))

After you add this code, you can store personalized nicknames and salutations in your BBDB. Nicknames and salutations will be looked up using people’s e-mail addresses. While in the *BBDB* buffer, you can type C-o (bbdb-insert-new-field) to add a field to the current record. Add a nick field with the person’s nickname, or a hello field with a custom salutation. When you compose a message to or reply to a message from that person, the salutation or nickname will be included. If no nickname can be found, the recipient’s name will be used instead.

A number of variables can be used to modify the behavior of this code(1). For example, you may or may not want to greet 20 people individually. The default value of wicked/gnus-nick-threshold is to greet up to four people individually, and greet more people collectively. If you always want to greet people individually, add (setq wicked/gnus-nick-threshold nil) to your ~/.emacs. If you want to change the strings used to greet people individually or collectively, change wicked/bbdb-hello-string and wicked/bbdb-hello-all-string. If you want to store the data into different fields, change wicked/bbdb-nick-field and wicked/bbdb-salutation-field, but note that old data will not be automatically copied to the new fields.

Here’s how the code works. First, it retrieves the list of addresses from the header(2). If there are more addresses than wicked/gnus-nick-threshold, then wicked/bbdb-hello-all-string is used to greet everyone. If not, each recipient address is looked up. If the recipient cannot be found in your BBDB, then the recipient’s name or e-mail address is used(4). If there is a personalized salutation, it is used(5). If there is a nickname, it is used(6). If the person has a record but neither salutation or nickname, then the name of the record is used(7). After all recipients have been processed, the names are added to the message(8), followed by the salutations(9). This function is added to the different Gnus message-posting functions, so it should be called whenever you compose or reply to messages.

Wicked Cool Emacs: BBDB: Set up BBDB

The main address book and contact management module for Emacs is the Insidious Big Brother Database (BBDB), which can be integrated into several mail clients and other modules within Emacs. If you use BBDB to keep track of contact information, you’ll be able to look up phone numbers or add notes to people’s records from your Emacs-based mail. Even if you don’t do e-mail within Emacs, you’ll find that BBDB’s customizability makes it surprisingly powerful.

In this project, you will learn how to set up BBDB as a basic address book. The BBDB homepage is at http://bbdb.sourceforge.net/. The development version fixes a number of bugs, so I recommend you try it instead of the stable version. However, if you are on Microsoft Windows or you do not have development tools handy, you might find the stable version easier to install. As of this writing, the stable version (2.35) can be downloaded from http://bbdb.sourceforge.net/bbdb-2.35.tar.gz . Download and unpack it to ~/elisp/bbdb-2.35, and save the pre-built bbdb-autoloads.el from http://bbdb.sourceforge.net/bbdb-autoloads.el into ~/elisp/bbdb-2.35/lisp .

To check out the development version, change to your ~/elisp directory and type in the following lines at the command prompt:

cvs -d :pserver:[email protected]:/cvsroot/bbdb login
cvs -d :pserver:[email protected]:/cvsroot/bbdb checkout bbdb

You should now have a directory called ~/elisp/bbdb. Change to that directory and run the following commands:

autoconf
./configure
make autoloads
make all

After installing either the stable or development version of BBDB, include it in your load-path by adding the appropriate line to your ~/.emacs:

(add-to-list 'load-path "~/elisp/bbdb-2.35/lisp")    ;; (1)
(add-to-list 'load-path "~/elisp/bbdb/lisp")         ;; (2)

(require 'bbdb) ;; (3)
(bbdb-initialize 'gnus 'message)   ;; (4)
(setq bbdb-north-american-phone-numbers-p nil)   ;; (5)

Use either ~/elisp/bbdb-2.35/lisp(1) or ~/elisp/bbdb/lisp(2) depending on the location of the installed BBDB Lisp files. Then load BBDB(3) and configure it for the Gnus mail client and the Message mode used to compose mail(4). It’s also a good idea to configure BBDB to accept any kind of phone number(5), not just North American numbers with a particular syntax.

After you evaluate this code or restart Emacs, BBDB should be part of your system. Next step: enter your address book!

Wicked Cool Emacs: BBDB: Work with Records

Creating Records

Creating a record in BBDB is not like creating a record in graphical address book programs. You will be prompted for each field through the minibuffer, one field at a time. Don’t worry about making mistakes while entering data, as you can always edit the records afterwards.

To create a record, use the command M-x bbdb-create. Here are the prompts you’ll encounter:

Prompt Notes Example
Name Full name John Doe
Company Company or organization ACME
Network Address E-mail address (comma-separated list) [email protected]
Address Description Short identifier for address (Home, Office, etc.) – tab completion available. Leave blank if you have no address information, or if you are done. Home
Street, line 1 Street address, line 1 (not including city, state, postal code or country) 1 Acme Road
Street, line … Street address, more lines – press RET to indicate the end of the street address
City Acme City
State Abbreviations are okay. Consistency helps. AC
Country Acme Country
Phone Location Short identifier for phone number (Home, Office, etc.) – tab completion available. Leave blank if you have no phone information, or if you are done. Home
Phone Phone number. I tend to specify the full number, using spaces to break it into readable chunks. +1 111 111 1111 x1111
Additional Comments Notes about the person, such as interests, how you met, and so on Likes rockets

Press RET to skip any fields for which you don’t have information. To
cancel the entry process, type C-g (keyboard-quit).

After you create the record, Emacs will display the record in another
window. You can then switch to the record and edit it. See Project XXX: Edit a BBDB record.

Searching Records

To search for a specific record, type M-x bbdb, or press b
(bbdb) while in the BBDB buffer. This prompts for a regular
expression and searches the name, company, network address, and notes
fields of all the records for a match against the regular expression
supplied. M-x bbdb-name, M-x bbdb-company, M-x bbdb-net, M-x
bbdb-notes, and M-x bbdb-phones search the corresponding fields only.

Updating Records

After creating or searching for a record, you can switch to the BBDB
window to edit it. Press C-o (bbdb-insert-field) to insert
custom fields. You can use tab completion on existing field names, and
you can also define your own fields by typing any field name. For
example, you may want to store people’s job titles in a field called
“job”.

To edit the value of a field, move your cursor to the field and press
e (bbdb-edit-current-field) to change the value. To delete
a field, move your cursor to the field and press C-k
(bbdb-delete-current-field-or-record).

Deleting Records

To delete an entire record, move the text cursor to the name and press C-k (bbdb-delete-current-field-or-record). You will be prompted for confirmation. Be careful! If you mistakenly delete a record, there’s no easy way to get it back. Fortunately, BBDB stores its data in a plain text file (~/.bbdb). Back up that file regularly and you’ll be able to recover from mistakes. You can also set up automatic file backups in Emacs (see Project XXX: Make Automatic Backups).

Now you know how to work with individual records. How can you import your address book information from other programs?