Geek: How to use offlineimap and the dovecot mail server to read your Gmail in Emacs efficiently

2014-04-09: This post is from 2008. =) I think I used dovecot+offlineimap because Gnus and maildir weren’t getting along properly and directly connecting with IMAP to Gmail’s server was slow, but things have probably changed a fair bit since then. I eventually moved to using the Gmail web interface for most things, but I still miss my Gnus setup!
  1. Make sure you’ve set up Postfix or some other mail server that can send mail. I’m not going to cover that because my configuration for outgoing mail doesn’t use Gmail.
  2. Install dovecot (IMAP server) and offlineimap (IMAP synchronization). You can probably find binaries for your distribution.
  3. Edit /etc/dovecot.conf and set the following:
    default_mail_env = maildir:%h/Maildir
  4. Put the following in ~/.offlineimaprc, changing your_local_username, your_username, and your_gmail_password:
    [general]
    accounts = Gmail 
    maxsyncaccounts = 1
    
    [Account Gmail]
    localrepository = Local
    remoterepository = Remote
    
    [Repository Local]
    type = IMAP 
    remotehost = localhost
    port = 143
    remoteuser = your_local_username
    
    [Repository Remote]
    type = IMAP
    remotehost = imap.gmail.com
    remoteuser = [email protected]
    remotepass = your_gmail_password
    ssl = yes
    maxconnections = 1
    realdelete = no
    folderfilter = lambda foldername: foldername in ['INBOX']
    

    If you feel comfortable specifying your password for your local account in your ~/.offlineimaprc, you can do so by adding a remotepass line under the remoteuser line in the [Repository Local] section.

  5. chmod go-rwx ~/.offlineimaprc for a little bit of safety.
  6. Type offlineimap to start synchronizing.
  7. While that’s synchronizing, use something like this as your ~/.gnus:
    (setq gnus-select-method
          '(nnimap "Mail"
    	       (nnimap-address "localhost")
    	       (nnimap-stream network)
    	       (nnimap-authenticator login)))
    
    (setq user-mail-address "[email protected]")
    (setq gnus-ignored-from-addresses "youruser")
    
  8. Start Emacs. Start Gnus with M-x gnus. If you don’t see the INBOX group, press ^ (gnus-group-enter-server-mode), open nnimap:Mail, move your cursor to the INBOX, and either press RET to go into the group or press u (gnus-browse-unsubscribe-current-group) to toggle the subscription status until you’re subscribed to the group. Then it should show up on the group screen (M-x gnus).
  9. Hope that helps. Have fun!

  • http://rlazo.supersized.org rlazo

    This is what I’ve been looking for! Now everything works perfectly. Thank you very much Sacha

    • http://sachachua.com Sacha Chua

      You’re welcome! I’m glad it works for you.

  • Jan Van lent

    I recently created a very similar setup. I also use the following to start offlineimap from within emacs.


    (define-key gnus-group-mode-map (kbd "vo")
    '(lambda ()
    (interactive)
    (shell-command "offlineimap&" "*offlineimap*" nil)))

  • http://linuxworld.com/community/ Don Marti

    Why not just use a local Maildir folder instead of having to set up an IMAP server? I do something similar but just point my MUA at ~/Maildir

    • Kamal Al-Marhoobi

      I believe this has something to do with gnus and large Maildir directories. I recall reading that this can be bad.

      • i’m writing a maildir mode for gnus that can work off the straight mailfolders. i’ll post it here when its done.

        -hhh

  • Kamal Al-Marhoobi

    Thanks Sacha. I’d recently got offlineimap and dovecot out of darwinports for exactly this purpose, but had not got around to the setup. I’ll try sorting this out when I get home today.

  • brad

    This was very helpful. Since you are not using gmail’s smtp server I assume you are not relying on gmail to save copies of your sent mail. So, how are you keeping copies of your sent items. I’m using gnus with dovecot on a local machine and I’m having a devil of a time getting anythign I send to be save locally. All I can get working, by using gcc-self, is a copy in my INBOX rather then in my Sent Items.

  • tim

    Looks like the dovecot configuration has changed a bit in recent versions (I’m using version 1.0.10)

    The config file is now /etc/dovecot/dovecot.conf

    The default_mail_env option is now called mail_location.

  • http://nothingmuch.woobling.org Yuval Kogman

    offlineimap supports the ‘Gmail’ type, so you don’t need all that extra fluff (ssl, the host, maxconnections etc).

    Also, I like to add

    folderfilter = lambda foldername: foldername not in ['[Gmail]/All Mail’]

    Cheers

  • http://jamesnvc.blogspot.com James Cash

    This looks really neat…I’ve been trying to move away from Thunderbird to Emacs + gnus + bbdb for my email, so this is very useful.

    One problem I’m having though…After offlineimap synchronizes and dovecot is running, I can’t seem to actually see the group. I do M-x gnus, ^ to the groups buffer, but when I try to select the nnimap:Mail group, it just says “nnimap: Generating active list for Mail…” forever (that is, until Emacs freezes).

    Oddly enough, if I hit C-g at just the right time, I can see the list of folders/labels, but that only works occasionally.

    If I access the dovecot server with Thunderbird/Icedove, it works fine.

    Any advice?

  • http://zak.elep.name Zak B. Elep

    I’ve used this setup way back last year, using offlineimap to retrieve messages from my spunge.org mail and serving them up locally via dovecot. Its a bit Rube Goldberg-ish as it basically makes use of IMAP as POP++ then IMAP again, so when I got to the point where I had 500/1000+ messages coming to me per day, I had to quit it. Nowadays I’m coming back to Gnus and using nnimap to connect directly to my IMAP servers instead (I wish No Gnus imports Wanderlust’s IMAP library as an alternative nnimap backend soon, though…)

  • Pingback: Surely it can’t be so hard to be a better (Emacs and Docbook) geek? « Alec the Geek

  • Joe Bloggs

    Thanks for the great tip, I can now read my gmail in gnus. However.. I have several other online email accounts I also want to sync. I read that I need to specify seperate directories for each account to offlineimap, but how do I set up dovecot for this?

  • Henrik

    If you want to avoid being queried about your local password at all. the local repository section should look like this:

    [Repository Local]
    type = IMAP
    preauthtunnel = /usr/sbin/dovecot –exec-mail imap

    That way, offlineimap talks directly to the dovecot program instead of via a network connection.

    Similarily, in .gnus.el you can do the following:

    (setq imap-shell-program “/usr/sbin/dovecot –exec-mail imap”)
    gnus-secondary-select-methods
    ‘((nnimap “Mail”
    (nnimap-stream shell))))

    and you won’t be quieried for a password in Gnus either.

  • http://daveabrahams.com Dave Abrahams

    @Henrik — I think WordPress may have messed with your posting. Should be

    preauthtunnel = /usr/sbin/dovecot --exec-mail imap

    and

    (setq imap-shell-program "/usr/sbin/dovecot --exec-mail imap")

    Note the pairs of dashes before exec-mail, everyone.

    @Sacha — Also, aside from using paths under /opt/local for my Macports installation of dovecot, I had to open up read access to dovecot.conf, or I was only able to run offlineimap as root… or is that actually OK?

  • http://daveabrahams.com Dave Abrahams

    @Don Marti –

    That’s a good question; why not? I’m not sure what one gains by using a local dovecot. Maybe if you also want to use something like AppleMail or Thunderbird, which doesn’t read Maildir directly?

    Note to Mac users: if you want to use the curses UI for offlineimap, you may need to install py25-curses and make the fix described here: http://trac.macports.org/ticket/14017

  • http://sachachua.com Sacha Chua

    Don, Dave: I was having weird problems with using Maildir directly, for some reason. =) Maybe if I set it up again, things will be better.

  • http://daveabrahams.com Dave Abrahams

    Note to MacOS / macports users: these instructions won’t work unless you also set

    passdb pam {
    args = login
    }

    in /opt/local/etc/dovecot/dovecot.conf. See https://trac.macports.org/ticket/19545

  • http://daveabrahams.com Dave Abrahams

    Having an awful time getting this to work through the shell on a Mac: http://dovecot.org/pipermail/dovecot/2009-May/039765.html

    If anyone else as succeeded, please let me know!

  • http://daveabrahams.com Dave Abrahams

    Mac shell issue solved:

    port install dovecot configure.args=–with-ioloop=select

  • http://sachachua.com Sacha Chua

    Awesome, congrats! Thanks for sharing! =D

  • Jeremy

    The reason you want to use a local IMAP server with Gnus is because direct access to the maildir using built in Gnus functionality can block you out of emacs during reads of large amounts of mail. This is because emacs is single threaded and the read of the mail has to take over all of emacs – you can’t switch buffers or type or whatever during Gnus’ read of the maildir.

    So instead you delegate the task to an external process, and Gnus just checks periodically to find out when it’s done. That way you’re not blocked out of emacs.

    That’s my understanding of why a local IMAP server is preferable to just using the built-in Maildir functions in Gnus.

  • Bjoern

    I hope some people still read this comments and maybe someone can help me.

    I use exactly the same setup for my Debian Lenny system:
    local dovecot imap server; offlineimap; Gnus

    But I have one problem. After running offlineimap the new mails in my local imap server are shown as “unread” but not as “new”.

    Does someone has the same problem?
    Any idea how to solve it?

    Thanks!

  • Matt

    Has anybody used the offlineimap/dovecot method to server up mail for *multiple* accounts? I have a gmail account plus several others. I’d like to read them all in gnus via the offlineimap/dovecot pipe.

    But how can I get dovecot to server up multiple maildir folders (eg ~/Maildir-accnt1 ~/Maildir-acctn2)?

  • Joe Bloggs

    Yes it can be done, read up on virtual users on the Dovecot wiki:

    http://wiki.dovecot.org/VirtualUsers

    From what I remember it’s just a case of adding some users to /etc/dovecot/passwd and using “%u” at the end of your “mail_location” entry in dovecot.conf, e.g:

    mail_location = maildir:/home/email/MailDir/%u

    This will create seperate folders for each user under the /home/email/MailDir/ directory

  • Kevin

    I had a problem with the summary buffer showing the wrong subjects for certain messages (entering them would give the right message though). It seems gnus-agent was at fault (or gmail’s interactions with it). http://www.emacswiki.org/emacs/GnusGmail#toc11 gave the solution, just disable it for gmail.

  • http://avendael.blogspot.com Evan Dale

    Wow Sacha, I like this configuration very much! To tell the truth, I haven’t really got this the first time around, since I’m fairly new to the emacs universe. Later on, it came to me that I really haven’t started the dovecot process, which causes gnus to block emacs’ io events. I think you should also mention that the dovecot process should be up and running since other linux distros don’t automatically start dovecot upon installation. It will sure help a lot of newbies out there. Thanks a lot!

  • http://sachachua.com Sacha Chua

    Rumour has it that the new Gnus in Emacs 24 is much faster at doing IMAP, so you won’t need this in order to do IMAP mail with, say, Google mail. Might still be useful for offline use, though. =)

  • Pingback: 在gnus里阅读Gmail邮件(续) | Freeland | 逍遥郡

  • http://barrington-leigh.net Chris Barrington-Leigh

    Wow, this is super! I’ve been looking for years for someone to spell these out together.

    How do you deal with deleting (archiving) messages? With drafts? With labels?
    so that they sync with Gmail’s version nicely?

    Thanks,
    c

  • Pingback: Emacs & Email Revisited | Bills Blog

  • http://gravatar.com/oberger Olivier Berger
  • http://sachachua.com Sacha Chua

    Olivier: Great, thanks! =)