Geek: How to use offlineimap and the dovecot mail server to read your Gmail in Emacs efficiently

2014-04-09: This post is from 2008. =) I think I used dovecot+offlineimap because Gnus and maildir weren’t getting along properly and directly connecting with IMAP to Gmail’s server was slow, but things have probably changed a fair bit since then. I eventually moved to using the Gmail web interface for most things, but I still miss my Gnus setup!
  1. Make sure you’ve set up Postfix or some other mail server that can send mail. I’m not going to cover that because my configuration for outgoing mail doesn’t use Gmail.
  2. Install dovecot (IMAP server) and offlineimap (IMAP synchronization). You can probably find binaries for your distribution.
  3. Edit /etc/dovecot.conf and set the following:
    default_mail_env = maildir:%h/Maildir
  4. Put the following in ~/.offlineimaprc, changing your_local_username, your_username, and your_gmail_password:
    [general]
    accounts = Gmail 
    maxsyncaccounts = 1
    
    [Account Gmail]
    localrepository = Local
    remoterepository = Remote
    
    [Repository Local]
    type = IMAP 
    remotehost = localhost
    port = 143
    remoteuser = your_local_username
    
    [Repository Remote]
    type = IMAP
    remotehost = imap.gmail.com
    remoteuser = [email protected]
    remotepass = your_gmail_password
    ssl = yes
    maxconnections = 1
    realdelete = no
    folderfilter = lambda foldername: foldername in ['INBOX']
    

    If you feel comfortable specifying your password for your local account in your ~/.offlineimaprc, you can do so by adding a remotepass line under the remoteuser line in the [Repository Local] section.

  5. chmod go-rwx ~/.offlineimaprc for a little bit of safety.
  6. Type offlineimap to start synchronizing.
  7. While that’s synchronizing, use something like this as your ~/.gnus:
    (setq gnus-select-method
          '(nnimap "Mail"
    	       (nnimap-address "localhost")
    	       (nnimap-stream network)
    	       (nnimap-authenticator login)))
    
    (setq user-mail-address "[email protected]")
    (setq gnus-ignored-from-addresses "youruser")
    
  8. Start Emacs. Start Gnus with M-x gnus. If you don’t see the INBOX group, press ^ (gnus-group-enter-server-mode), open nnimap:Mail, move your cursor to the INBOX, and either press RET to go into the group or press u (gnus-browse-unsubscribe-current-group) to toggle the subscription status until you’re subscribed to the group. Then it should show up on the group screen (M-x gnus).
  9. Hope that helps. Have fun!

Wicked Cool Emacs: BBDB: Keeping track of contact dates

I hadn’t realized just how much I missed my Big Brother Database until today. Three networking events packed into one week meant that I hadn’t set aside enough time for follow up, and I felt my memories of the conversations getting a little hazy. Fortunately I’d taken some notes on my Palm, but I knew I had to get it into some kind of contact management system quickly, and Gmail Contacts just wasn’t compelling enough for me. So it’s back to Emacs, plain text files, and a surprisingly sophisticated contact manager.

I also promised to do some work on the book today, so everything dovetailed nicely.

The following bit of code helps me filter displayed contacts to show only the people I haven’t contacted since a certain date. This is handy for remembering to keep in touch with old friends, for example. Or at least it would be handy if I used it more often and if I actually sent the letters that pile up in my e-mail drafts and my snail mail outbox… but at least it’s a step in the right direction.

If you want to know who you have or haven’t talked to in a while, you need to do two things. First, you need to keep track of when you talked to people. Second, you need to generate reports.

To be able to quickly add contact notes to BBDB records, add the following to your ~/.emacs:

ch6-bbdb-ping.el:

(define-key bbdb-mode-map "z" 'wicked/bbdb-ping-bbdb-record)
(defun wicked/bbdb-ping-bbdb-record (bbdb-record text &optional date regrind)
  "Adds a note for today to the current BBDB record.
Call with a prefix to specify date.
BBDB-RECORD is the record to modify (default: current).
TEXT is the note to add for DATE.
If REGRIND is non-nil, redisplay the BBDB record."
  (interactive (list (bbdb-current-record t)
                     (read-string "Notes: ")
                     ;; Reading date - more powerful with Planner, but we'll make do if necessary
                     (if (featurep 'planner)
                         (if current-prefix-arg (planner-read-date) (planner-today))
                       (if current-prefix-arg
                           (read-string "Date (YYYY.MM.DD): ")
                         (format-time-string "%Y.%m.%d")))
                     t))
  (bbdb-record-putprop bbdb-record
                       'contact
                       (concat date ": " text "\n"
                               (or (bbdb-record-getprop bbdb-record 'contact))))
  (if regrind
      (save-excursion
        (set-buffer bbdb-buffer-name)
        (bbdb-redisplay-one-record bbdb-record)))
  nil)

You can then use z in BBDB buffers to add a quick note to the “contact” field of the current record. The date is automatically noted. You can create a note for a specific date by calling {{C-u wicked/bbdb-ping-bbdb-record}} with a prefix argument. For convenience, the suggested configuration binds this to “z”, because it was one of the few unbound keys I could find. Use this after you meet, call, or e-mail people, and write down a short note about the conversation you had. You might find these notes useful later on.

If you met a number of people at an event in the past and you have Planner installed and loaded, you can use {{planner-timewarp}} to set the effective date to another date. To return to today, use {{M-x planner-timewarp nil}}.

To automatically add a datestamped copy of sent e-mail subjects to people’s BBDB records, add the following to your ~/.gnus:

ch6-bbdb-message-add-subject.el:

(defun wicked/message-add-subject-to-bbdb-record ()
  "Add datestamped subject note for each person this message has been sent to."
  (let* ((subject (concat (format-time-string "%Y.%m.%d")
                          ": E-mail: " (message-fetch-field "Subject") "\n"))
         (bbdb-get-addresses-headers
          (list (assoc 'recipients bbdb-get-addresses-headers)))
         records)
    (setq records
          (bbdb-update-records
           (bbdb-get-addresses nil gnus-ignored-from-addresses 'gnus-fetch-field)
           nil nil))
    (mapc (lambda (rec)
            (bbdb-record-putprop rec
                                 'contact
                                 (concat subject
                                         (or
                                          (bbdb-record-getprop rec 'contact)
                                          ""))))
          records)))
(add-hook 'message-send-hook 'wicked/message-add-subject-to-bbdb-record)

Now that you have the data, how can you use it to filter? Add the following to your ~/.emacs:

ch6-bbdb-show-only-no-contact-since.el:

(defun wicked/bbdb-show-only-no-contact-since (date &optional reverse records)
  "Show only people who haven't been pinged since DATE or at all.
If REVERSE is non-nil, show only the people you've contacted on or since DATE.
Call with a prefix argument to show only people you've contacted on or since DATE."
  (interactive (list
                (if (featurep 'planner)
                    (planner-read-date)
                  (read-string "Date (YYYY.MM.DD): "))
                current-prefix-arg (or bbdb-records (bbdb-records))))
  (let (new-records
        last-match
        timestamp
        omit
        notes)
    (while records
      ;; Find the latest date mentioned in the entry
      (let ((timestamp (wicked/bbdb-last-date
                        (if (vectorp (car records))
                            (car records)
                          (caar records)))))
        (if (if reverse
                ;; Keep if contact is >= date
                (null (string< timestamp date))
              ;; Keep if date > contact
              (string> date timestamp))
            (add-to-list 'new-records (if (vectorp (car records))
                            (car records)
                          (caar records)) t)))
      (setq records (cdr records)))
    (bbdb-display-records new-records)))

(defun wicked/bbdb-last-date (rec)
  "Return the most recent date for REC or nil if none.
Dates should be in the form YYYY.MM.DD.  The first date in the
notes field and the first date in the contact field are used, so
dates should be in reverse chronological order."
  (let* ((wicked/date-regexp
          "\\<\\([1-9][0-9][0-9][0-9]\\)\\.\\([0-9][0-9]?\\)\\.\\([0-9][0-9]?\\)\\>")
         ;; Get the first date mentioned in the notes field
         (notes-date
          (or (and (string-match wicked/date-regexp (or (bbdb-record-notes rec) ""))
                   (match-string 0 (or (bbdb-record-notes rec) "")))
              "0000.00.00"))
         ;; Get the first date mentioned in the contact field
         (contact-date
          (or (and (string-match wicked/date-regexp (or (bbdb-record-getprop rec 'contact) ""))
                   (match-string 0 (or (bbdb-record-getprop rec 'contact) "")))
              "0000.00.00")))
    ;; Compare the two dates
    (or (if (string< notes-date contact-date) contact-date notes-date)
        "0000.00.00")))

To generate a report, use {{M-x wicked/bbdb-show-only-no-contact-since}} and specify the date. These functions are much easier to use with Planner’s date-handling functions. Planner can read dates like “-1″ (yesterday), “-7fri” (seven Fridays ago), “2” (the second of this month), “1.2” (January 2 in this year), and “2007.01.02” (January 2, 2007).

You can also flip the filter by using the universal prefix argument ({{C-u M-x wicked/bbdb-show-only-no-contact-since}}) to show only the people you’ve contacted since a certain date. This is good for knowing the size of your active network. Because the filter works on displayed records, you can combine it to find all the people you talked to last year but not this year. You can also combine it with other filters to find all the people you’ve marked as friends, but who you haven’t talked to in three months. Then you can send a personalized e-mail or make a phone list, and get back in touch. And that’s how you keep track of your contact dates!

Managing my mail

I use Gnus, one of the many mail/news clients available for Emacs.
The following features help me manage the volume of mail I get each day.

Mail splitting

Yes, yes, the Gmail way is to keep everything in one folder and then
use searches to filter your messages. Still, I like being able to
glance at my screen and see 2 personal messages and 3 planner-related
messages.

Topics and group hiding

I use Gnus topics to divide my mail into folders and subfolders.
Mail groups are hidden unless they have mail. Some groups like
mail.misc and mail.planner are generally useful, so I keep them visible
even if they don’t have unread mail.

Scoring

Gnus allows you to automatically score threads and messages up and
down based on various criteria. You can set it to completely hide
boring messages, show them in a different color, show interesting
messages in a different color, etc.

On most mailing lists and newsgroups, I don’t bother reading message
bodies. I just scan through subjects, hitting k to kill entire threads
I don’t find interesting. Gnus remembers what threads I’ve killed,
marks them as read, and scores them down automatically. It also scores
up messages containing certain keywords, replies to my posts, and
threads I found interesting.

Integration with my contacts

I put interesting people in my BBDB contact database. Gnus indicates
messages from them with a little + beside their name in the message
summary. If someone I know is interested in a thread, I might find it
interesting as well.

Hiding and article washing

I’ve set Gnus up to hide quoted text. This makes browsing through
threads much easier because I can concentrate only on the the new
parts. I can hit a few keys to expose sections of the quoted text if
the replies aren’t immediately obvious from the context.

I can also set it up to remove ads at the bottom of messages,
particularly long signatures, To: lines with more than N recipients,
that sort of thing. I can tell it to strip out HTML, too.

Displaying parent article

Sometimes I’ll jump into the middle of a thread. I can use ^ to get to
the parent message.

Searching

I use swish++ to index and search through my personal and
planner-related mail.

Planner hyperlinks

Most of my tasks come in through e-mail. Planner lets me keep track of
my TODOs easily by automatically hyperlinking to the mail message I’m
looking at when I create a task. Dealing with a few items on my TODO
list is much easier than going through a large inbox! =)

On Technorati: , ,