Cobbling together a semi-auto-responder using Emacs, Gnus, and org-contacts

It turns out that lots of people are interested in an e-mail-based course for learning Emacs Lisp. Yay! =) Maybe it’s the idea of bite-size chunks. Maybe it’s the ease of asking questions. Maybe it’s the regular reminders to work on something. Who knows? Whatever the reason, it’s awesome to see so many people willing to join me on this experiment.

Since this is my first time to venture into the world of teaching people online, I wanted to see how far I could push actually doing all the mails myself, instead of just signing up for an Aweber account and handing everyone off to an impersonal autoresponder. I dusted off Gnus, offlineimap, and org-contacts, and started figuring out my workflow. I’ll share how that workflow’s evolving so that you can get a sense of how someone might write little bits of Emacs Lisp to make something repetitive easier.

For the first little while, I got by with using C-x r s (copy-to-register) and C-x r i (insert-register) to store the text that I needed.
Sometimes I needed to paste in the welcome message and checklist, and sometimes I needed to paste in the first lesson. By using registers, I could insert whatever I wanted instead of going through the kill ring. I also had another bit of templated code in yet another register so that I could easily create an org-contacts entry for the person whose mail I was replying to. In the beginning, I used tasks under each person’s heading to indicate that I had sent them the checklist or that I had sent them the first lesson. Eventually, I changed my org-contacts notes so that the TODO state of each person showed which lesson I was going to send them next, or CHECKLIST if I was waiting for their reply to the checklist. I also set up Org so that it would automatically log when the TODO state was changed.

#+TODO: TODO | DONE
#+TODO: CHECKLIST(c!) BEGINNER1(1!) BEGINNER2(2!) BEGINNER3(3!) BEGINNER4(4!) FULL(f!) | FINISHED(x!)
#+TODO: | CANCELLED

* Who
** CHECKLIST Jane Smith ...
** BEGINNER1 John Smith
   SCHEDULED: <2014-05-28 Wed>
   :PROPERTIES:
   :EMAIL: [email protected]
   :END:
(notes from the messages, etc.)

I wrote some code to make it easier to send someone a checklist and create a note for them in my org-contacts file. I bound it to C-c e c for convenience.
(The bind-key function is defined by a package.)

(setq sacha/elisp-course-checklist-body "... really long text here...")
(defun sacha/elisp-course-checklist ()
  "Copy this message and put it at the end as a checklist item. 
Start a message with the checklist."
  (interactive)
  (gnus-summary-scroll-up 1)
  (with-current-buffer gnus-article-buffer
    (let ((message (buffer-substring-no-properties (point-min) (point-max)))
          (email (cadr (org-contacts-gnus-get-name-email))))
      (with-current-buffer "elisp-course.org"
        (save-excursion
          (goto-char (point-max))
          (save-excursion
            (insert "\n** " message)
            (org-set-property "EMAIL" email)
            (org-todo "CHECKLIST"))))))
  (gnus-summary-followup-with-original nil)
  (goto-char (point-max))
  (insert sacha/elisp-course-checklist-body))
(bind-key "C-c e c" 'sacha/elisp-course-checklist)

This made it easier for me to read the starred messages from my inbox and use C-c e c to get a head start on processing people’s introductory messages.
Yay! I used the register trick to help me reply to people who were ready for the first lesson. After the first few replies, I noticed that the attachment code was fine even if I put that in the register too, so I added it as well.

Things got more complicated when I started processing lesson 2. I didn’t want to have to set up and remember lots of different registers, and I didn’t want to manually update the TODO states either. So I started defining functions that I could call with keyboard shortcuts:

(defun sacha/elisp-course-1 ()
  (interactive)
  (let ((marker (org-contacts-gnus-article-from-get-marker)))
    (if marker
        (org-with-point-at marker
          (org-todo "BEGINNER2"))))
  ;; Find the person's contact record
  (gnus-summary-scroll-up 1)
  (gnus-summary-followup-with-original nil)
  (message-goto-subject)
  (message-delete-line)
  (insert (concat "Subject: " sacha/elisp-course-1-subject "\n"))
  (goto-char (point-max))
  (insert sacha/elisp-course-1-body))
(bind-key "C-c e 1" 'sacha/elisp-course-1)
(defun sacha/elisp-course-2 ()
  (interactive)
  (let ((marker (org-contacts-gnus-article-from-get-marker)))
    (if marker
        (org-with-point-at marker
          (org-todo "BEGINNER3"))))
  ;; Find the person's contact record
  (gnus-summary-scroll-up)
  (gnus-summary-followup-with-original nil)
  (goto-char (point-max))
  (insert sacha/elisp-course-2-body))
(bind-key "C-c e 2" 'sacha/elisp-course-2)

Really, though, it doesn’t make sense to have a lot of duplicated code. So I wrote some code that would use the person’s TODO keyword to look up the message to send them, and then move them to the next keyword. Now I don’t need sacha/elisp-course-1 or sacha/elisp-course-2 any more.

(setq sacha/elisp-course-info
      `(("CHECKLIST" nil ,sacha/elisp-course-checklist-body)
        ("BEGINNER1" ,sacha/elisp-course-1-subject ,sacha/elisp-course-1-body)
        ("BEGINNER2" ,sacha/elisp-course-2-subject ,sacha/elisp-course-2-body)))

(defun sacha/elisp-course-process (subject body &optional state)
  "Process this course entry."
  (if (derived-mode-p 'org-mode)
      (progn
        ;; Move this node to the next state and compose a message
        (if state (org-todo state))
        (org-todo 'right)
        (message-mail (org-entry-get (point) "EMAIL") subject)
        (goto-char (point-max))
        (insert body))
    ;; Doing this from Gnus; find the person's info
    (let ((marker (org-contacts-gnus-article-from-get-marker)))
      (if marker (org-with-point-at marker
                   (if state (org-todo state))
                   (org-todo 'right)))
      ;; Compose a reply
      (gnus-summary-scroll-up 1)
      (gnus-summary-followup-with-original nil)
      (message-goto-subject)
      (message-delete-line)
      (insert (concat "Subject: " subject "\n"))
      (goto-char (point-max))
      (insert body))))

(defun sacha/elisp-course-guess-and-process (&optional state)
  (interactive (list (if current-prefix-arg (read-string "State: "))))
  (let ((current-state
         (or state (elt
                    (if (derived-mode-p 'org-mode)
                        (org-heading-components) 
                      (let ((marker (org-contacts-gnus-article-from-get-marker)))
                        (if marker (org-with-point-at marker (org-heading-components)))))
                    2))))
    (sacha/elisp-course-process
     (elt (assoc current-state sacha/elisp-course-info) 1)
     (elt (assoc current-state sacha/elisp-course-info) 2)
     state)))
(bind-key "C-c e e" 'sacha/elisp-course-guess-and-process)

Come to think of it, I should totally have it schedule the next update for the next Wednesday, too. ;) That’s just (org-schedule "+wed"). Neat, huh?
And I’m sure there are all sorts of ways the code can be simpler, but it works for me at the moment, so hooray!

I really like this approach. It lets me pull in standard information while also letting me customize the messages and how it fits into my task tracking. I can’t get that with Gmail (even with canned responses), and I’m not sure any CRM is going to be quite as awesome as this. I can’t wait to see how else we’ll tweak this as we go through more conversations. I’d like to get better at:

  • having a consistent place where I can process all the messages and make sure nothing falls through the cracks; I currently star messages to make sure I process them, since the Gmail label folder in IMAP seems to be missing some messages
  • seeing all Gnus conversations related to an org-contacts entry
  • reaching out to people proactively with the next lesson, even if they haven’t e-mailed me (or maybe I should wait for them?)

Anyway, that’s an example of writing a little bit of Emacs Lisp in order to connect different packages. Gnus handles mail, Org handles notes, org-contacts links the two together, and with a little bit of custom code, I can make the combination fit what I want to do. I read the source code of org-contacts to find out how I could look up the appropriate note, and I looked at org-shiftright to find out how to move things to the next TODO state. If you know something that works roughly like what you want it to work, you can find out how it does things and then copy that.

As for the course itself: I’ve been sending people links to the HTML output, attached .txt files (with -*- mode: org -*-) so they can open it in Emacs if they want, and inline text so that they can skim it briefly in their e-mail client if they want to. I’m not perfectly happy with the plain-text formats, but it seems to be a reasonable compromise, and so far people have been able to deal with it. I’ve been improving pieces of it based on feedback on clarity, suggestions for good examples, and so on. I didn’t take all the feedback; after thinking about some of the suggestions, I still preferred it my way. It’s shaping up quite nicely, though!

If you’re curious about the beginner’s course on reading Emacs Lisp, e-mail me at [email protected] and we’ll see how this works out. I’m certainly learning a lot. =)

  • Raymond Zeitler

    First, I plan to sign up for the course. I’ll send an e-mail as instructed. I hope it’s not too late!
    Second, I’ve always thought C-x r was exclusively for Rectangle operations. I see now that C-x r r (extra “r”) is the one I’m confused with. So it’s neat (and not surprising) that Emacs has a psuedo-clipboard manager.
    Third, I’ve been wondering how you handle contacts in general. Have you copied them to “the cloud” so that you can use G+ (for example) with your android device(s)?

    • http://sachachua.com Sacha Chua

      There’s the kill ring, which is the real clipboard manager (try the browse-kill-ring package to make it easier to use). Registers are way awesomer than just copying and pasting text. =) You can store text, numbers, files, rectangles, window configurations, or positions, and they’re referred to by a single character or number. I still use kill and yank a lot, but registers are handy too.

      Contacts: I converted my Emacs-based Big Brother Database (BBDB) file to a CSV and imported into Google Contacts a long time ago, and I mostly use Google Contacts these days because of the sync. org-contacts has been really helpful for the workflow I described above, though. I don’t need to sync that with my phone, so I haven’t looked into that yet.