Category Archives: geek

On this page:

Cobbling together a semi-auto-responder using Emacs, Gnus, and org-contacts

It turns out that lots of people are interested in an e-mail-based course for learning Emacs Lisp. Yay! =) Maybe it’s the idea of bite-size chunks. Maybe it’s the ease of asking questions. Maybe it’s the regular reminders to work on something. Who knows? Whatever the reason, it’s awesome to see so many people willing to join me on this experiment.

Since this is my first time to venture into the world of teaching people online, I wanted to see how far I could push actually doing all the mails myself, instead of just signing up for an Aweber account and handing everyone off to an impersonal autoresponder. I dusted off Gnus, offlineimap, and org-contacts, and started figuring out my workflow. I’ll share how that workflow’s evolving so that you can get a sense of how someone might write little bits of Emacs Lisp to make something repetitive easier.

For the first little while, I got by with using C-x r s (copy-to-register) and C-x r i (insert-register) to store the text that I needed.
Sometimes I needed to paste in the welcome message and checklist, and sometimes I needed to paste in the first lesson. By using registers, I could insert whatever I wanted instead of going through the kill ring. I also had another bit of templated code in yet another register so that I could easily create an org-contacts entry for the person whose mail I was replying to. In the beginning, I used tasks under each person’s heading to indicate that I had sent them the checklist or that I had sent them the first lesson. Eventually, I changed my org-contacts notes so that the TODO state of each person showed which lesson I was going to send them next, or CHECKLIST if I was waiting for their reply to the checklist. I also set up Org so that it would automatically log when the TODO state was changed.

#+TODO: TODO | DONE
#+TODO: CHECKLIST(c!) BEGINNER1(1!) BEGINNER2(2!) BEGINNER3(3!) BEGINNER4(4!) FULL(f!) | FINISHED(x!)
#+TODO: | CANCELLED

* Who
** CHECKLIST Jane Smith ...
** BEGINNER1 John Smith
   SCHEDULED: <2014-05-28 Wed>
   :PROPERTIES:
   :EMAIL: [email protected]
   :END:
(notes from the messages, etc.)

I wrote some code to make it easier to send someone a checklist and create a note for them in my org-contacts file. I bound it to C-c e c for convenience.
(The bind-key function is defined by a package.)

(setq sacha/elisp-course-checklist-body "... really long text here...")
(defun sacha/elisp-course-checklist ()
  "Copy this message and put it at the end as a checklist item. 
Start a message with the checklist."
  (interactive)
  (gnus-summary-scroll-up 1)
  (with-current-buffer gnus-article-buffer
    (let ((message (buffer-substring-no-properties (point-min) (point-max)))
          (email (cadr (org-contacts-gnus-get-name-email))))
      (with-current-buffer "elisp-course.org"
        (save-excursion
          (goto-char (point-max))
          (save-excursion
            (insert "\n** " message)
            (org-set-property "EMAIL" email)
            (org-todo "CHECKLIST"))))))
  (gnus-summary-followup-with-original nil)
  (goto-char (point-max))
  (insert sacha/elisp-course-checklist-body))
(bind-key "C-c e c" 'sacha/elisp-course-checklist)

This made it easier for me to read the starred messages from my inbox and use C-c e c to get a head start on processing people’s introductory messages.
Yay! I used the register trick to help me reply to people who were ready for the first lesson. After the first few replies, I noticed that the attachment code was fine even if I put that in the register too, so I added it as well.

Things got more complicated when I started processing lesson 2. I didn’t want to have to set up and remember lots of different registers, and I didn’t want to manually update the TODO states either. So I started defining functions that I could call with keyboard shortcuts:

(defun sacha/elisp-course-1 ()
  (interactive)
  (let ((marker (org-contacts-gnus-article-from-get-marker)))
    (if marker
        (org-with-point-at marker
          (org-todo "BEGINNER2"))))
  ;; Find the person's contact record
  (gnus-summary-scroll-up 1)
  (gnus-summary-followup-with-original nil)
  (message-goto-subject)
  (message-delete-line)
  (insert (concat "Subject: " sacha/elisp-course-1-subject "\n"))
  (goto-char (point-max))
  (insert sacha/elisp-course-1-body))
(bind-key "C-c e 1" 'sacha/elisp-course-1)
(defun sacha/elisp-course-2 ()
  (interactive)
  (let ((marker (org-contacts-gnus-article-from-get-marker)))
    (if marker
        (org-with-point-at marker
          (org-todo "BEGINNER3"))))
  ;; Find the person's contact record
  (gnus-summary-scroll-up)
  (gnus-summary-followup-with-original nil)
  (goto-char (point-max))
  (insert sacha/elisp-course-2-body))
(bind-key "C-c e 2" 'sacha/elisp-course-2)

Really, though, it doesn’t make sense to have a lot of duplicated code. So I wrote some code that would use the person’s TODO keyword to look up the message to send them, and then move them to the next keyword. Now I don’t need sacha/elisp-course-1 or sacha/elisp-course-2 any more.

(setq sacha/elisp-course-info
      `(("CHECKLIST" nil ,sacha/elisp-course-checklist-body)
        ("BEGINNER1" ,sacha/elisp-course-1-subject ,sacha/elisp-course-1-body)
        ("BEGINNER2" ,sacha/elisp-course-2-subject ,sacha/elisp-course-2-body)))

(defun sacha/elisp-course-process (subject body &optional state)
  "Process this course entry."
  (if (derived-mode-p 'org-mode)
      (progn
        ;; Move this node to the next state and compose a message
        (if state (org-todo state))
        (org-todo 'right)
        (message-mail (org-entry-get (point) "EMAIL") subject)
        (goto-char (point-max))
        (insert body))
    ;; Doing this from Gnus; find the person's info
    (let ((marker (org-contacts-gnus-article-from-get-marker)))
      (if marker (org-with-point-at marker
                   (if state (org-todo state))
                   (org-todo 'right)))
      ;; Compose a reply
      (gnus-summary-scroll-up 1)
      (gnus-summary-followup-with-original nil)
      (message-goto-subject)
      (message-delete-line)
      (insert (concat "Subject: " subject "\n"))
      (goto-char (point-max))
      (insert body))))

(defun sacha/elisp-course-guess-and-process (&optional state)
  (interactive (list (if current-prefix-arg (read-string "State: "))))
  (let ((current-state
         (or state (elt
                    (if (derived-mode-p 'org-mode)
                        (org-heading-components) 
                      (let ((marker (org-contacts-gnus-article-from-get-marker)))
                        (if marker (org-with-point-at marker (org-heading-components)))))
                    2))))
    (sacha/elisp-course-process
     (elt (assoc current-state sacha/elisp-course-info) 1)
     (elt (assoc current-state sacha/elisp-course-info) 2)
     state)))
(bind-key "C-c e e" 'sacha/elisp-course-guess-and-process)

Come to think of it, I should totally have it schedule the next update for the next Wednesday, too. ;) That’s just (org-schedule "+wed"). Neat, huh?
And I’m sure there are all sorts of ways the code can be simpler, but it works for me at the moment, so hooray!

I really like this approach. It lets me pull in standard information while also letting me customize the messages and how it fits into my task tracking. I can’t get that with Gmail (even with canned responses), and I’m not sure any CRM is going to be quite as awesome as this. I can’t wait to see how else we’ll tweak this as we go through more conversations. I’d like to get better at:

  • having a consistent place where I can process all the messages and make sure nothing falls through the cracks; I currently star messages to make sure I process them, since the Gmail label folder in IMAP seems to be missing some messages
  • seeing all Gnus conversations related to an org-contacts entry
  • reaching out to people proactively with the next lesson, even if they haven’t e-mailed me (or maybe I should wait for them?)

Anyway, that’s an example of writing a little bit of Emacs Lisp in order to connect different packages. Gnus handles mail, Org handles notes, org-contacts links the two together, and with a little bit of custom code, I can make the combination fit what I want to do. I read the source code of org-contacts to find out how I could look up the appropriate note, and I looked at org-shiftright to find out how to move things to the next TODO state. If you know something that works roughly like what you want it to work, you can find out how it does things and then copy that.

As for the course itself: I’ve been sending people links to the HTML output, attached .txt files (with -*- mode: org -*-) so they can open it in Emacs if they want, and inline text so that they can skim it briefly in their e-mail client if they want to. I’m not perfectly happy with the plain-text formats, but it seems to be a reasonable compromise, and so far people have been able to deal with it. I’ve been improving pieces of it based on feedback on clarity, suggestions for good examples, and so on. I didn’t take all the feedback; after thinking about some of the suggestions, I still preferred it my way. It’s shaping up quite nicely, though!

If you’re curious about the beginner’s course on reading Emacs Lisp, e-mail me at [email protected] and we’ll see how this works out. I’m certainly learning a lot. =)

Emacs Chat: Bozhidar Batsov

UPDATE 2014-06-13: The transcript is now available.

Bozhidar Batsov (emacsredux.com) shares how he got into Emacs and Emacs Lisp. He also demonstrates cool features from Prelude and Projectile, which are great if you do a lot of programming. Check it out!

Quick Links: https://twitter.com/bbatsov , https://twitter.com/emacs_knight , http://emacsredux.com , https://github.com/bbatsov/prelude , https://github.com/bbatsov/projectile . If you like his work, there’s https://www.gittip.com/bbatsov/

Guest: Bozhidar Batsov

For the event page, you may click here.

Want just the audio? Get it from archive.org: MP3

Transcript here!

Check out Emacs Chat for more interviews like this. Got a story to tell about how you learned about or how you use Emacs? Get in touch!

Planning an e-mail-based course for Emacs Lisp

I’ve been working on an Emacs Lisp beginner’s course, something focused on helping people become more comfortable configuring Emacs. The web-based guide is taking shape quite nicely, but it’s still a lot of scrolling, and it can still feel overwhelming for newbies. I think it might make sense to offer it as an e-mail course. That way, I can spread the lessons out, help people with their questions, and improve things based on people’s feedback.

2014-05-12 How can I take Learn How to Read Emacs Lisp to the next level #emacs #packaging #writing #teaching

2014-05-12 How can I take Learn How to Read Emacs Lisp to the next level #emacs #packaging #writing #teaching

I can improve the guide by adding more structure, examples, exercises, and so on. I’ve requested several books on e-learning and course design, and I’m looking forward to learning more over the years. And I can also improve it by testing it with people… =)

2014-05-14 Planning an e-mail-based course for Emacs Lisp #emacs #teaching

2014-05-14 Planning an e-mail-based course for Emacs Lisp #emacs #teaching

I floated the idea on Twitter and lots of people e-mailed me to join. Instead of setting up an autoresponder, I decided that I would do things by hand as much as I could. That way, I can personalize the messages based on people’s interests and configuration, and I can enjoy more of the back-and-forth conversation.

After getting annoyed with the SSL hassles of setting up Gnus on Windows, I decided to just use my Linux-based virtual machine for handling mail. That was pretty straightforward, although for some reason, my IMAP view of Gmail doesn’t have all of the messages under a label. It just means that I have to manually re-check the messages to make sure nothing slips through the cracks.

I used an Org file to keep notes on each person, including TODOs under each of them. I sent everyone a checklist to see which section we should start with. A few people are starting at the beginning, and others will get the e-mails once I’ve updated those sections. Text registers (C-x r s) were really helpful since I was pasting different things into different e-mails. I’m still figuring out the workflow for this, and I’m sure I’ll automate pieces of it as more people move through the course.

I’ve sent the first section to some people already, including the Org version in the e-mail body and as an attachment, and linking to the web-based version. The Org version is a little more cluttered than the text export, but the text export uses box quotes, so I figured the Org version was the best to start with.

2014-05-16 A plan for delivering the Emacs Lisp course #emacs #teaching

2014-05-16 A plan for delivering the Emacs Lisp course #emacs #teaching

Want to be part of this? E-mail me at [email protected]

How to update the Org 7 that comes with Emacs to Org 8 (more configuration! better exports!)

Update 2014-05-12: Simplified thanks to Sebastian’s note that Org 8 is available in the built-in package repository, yay!

The Org Mode included in Emacs 24 is version 7. Version 8 has lots of new configuration variables and the exporting mechanism has been rewritten. However, it needs to be installed in an Emacs that has not yet loaded any Org code or files. Here’s how you can upgrade your Org:

  1. Start Emacs with emacs -q. This skips your personal configuration.
  2. You will need an Internet connection for this step. Type M-x package-install, and type in org. This will install the latest version of Org from the built-in package repository.
  3. Edit your ~/.emacs.d/init.el (or ~/.emacs, if you’re using that instead). Add the following code to the beginning of the file:
    (package-initialize)
    (setq package-enable-at-startup nil)
    

    This will load the installed packages when you start Emacs, overriding the buit-in Org 7 with the Org 8 version that you installed.

    Advanced note: If you’ve downloaded Emacs Lisp code that should override code already installed through packages, you need to change this to (package-initialize nil) instead, and add (package-initialize t) after your load-path settings.

  4. Check your configuration for references to the older version of Org. In particular, look for any configuration related to exporting (ex: (require 'org-html)). You can change those lines to their Org 8 equivalents (ex: (require 'ox-html)), but it’s probably easier to just comment them out for now. You can comment out lines by adding ; to the beginning.
  5. Save your init.el and restart Emacs (this time, without the -q option). M-x org-version should now start with Org-mode version 8.
  6. Review your Emacs configuration for any changes that you will need to make. You can ask the Org Mode mailing list for help if you get stuck.

Good luck!

Emacs Chat: Phil Hagelberg

Update 2014-06-13: Transcript now available

Phil Hagelberg talks about custom keyboards, pair-programming with syme.herokuapp.com , Clojure REPLs, starter kits and better defaults, packages, helping his kids learn to think systematically, and warming up his shed-turned-office through XMPP (from Emacs, no doubt).

Quick links: http://technomancy.us , https://syme.herokuapp.com/ , http://github.com/technomancy/better-defaults , https://github.com/technomancy/dotfiles/tree/master/.emacs.d , http://leiningen.org/ , http://technomancy.us/171 (heater), http://atreus.technomancy.us (keyboards)

Guest: Phil Hagelberg

For the event page, you may click here.

Want just the audio? Get it from archive.org: MP3

Transcript

Check out Emacs Chat for more interviews like this. Got a story to tell about how you learned about or how you use Emacs? Get in touch!

 

Update on time tracking with Quantified Awesome and with Emacs

With another Quantified Self Toronto meetup in a few weeks and a conversation with fellow self-trackers, it’s time for me to think about time again.

I’ve been fixing bugs and adding small pieces of functionality to Quantified Awesome, and I spent some time improving the integration with Emacs. Now I can type ! to clock in on a task and update Quantified Awesome. Completing the task clocks me out in Emacs and updates Beeminder if appropriate. (I don’t update Quantified Awesome when finishing a task, because I just clock into the next activity.) This allows me to take advantage of Org’s clock reports for project and task-level time, at least for discretionary projects that involve my computer. I’m not going to get full coverage, but that’s what Quantified Awesome’s web interface is for. It takes very little effort to track things now, if I’m working off my to-do list. Even if I’m not, it still takes just a few taps on my phone to switch activities.

Most of my data is still medium-level, since I’m still getting the hang of sorting out my time in Emacs. Looking at data from 2014 so far, dropping partial weeks, and doing the analysis on April 14 (which is when I’m drafting this), here’s what I’ve been finding.

  • I sleep a little more than I used to: an average of 8.9 hours a day, or 37% of the time. This is up from 8.3 hours last year.
  • It takes me about an hour to get ready in the mornings. If I have a quick breakfast instead of having rice and fried egg, I can get out the door in 30-45 minutes.
  • It takes me 50-60 minutes to get downtown, whether this is by transit or bicycle. Commuting takes 3% of my time.

Little surprises:

  • I’ve spent almost twice as much time on business building or discretionary productive activities (19%) as I have earning (11%) – good to see decisions in action!
  • I’ve spent more time drawing than writing this year (5% vs 3%). Next to writing, Emacs is the productive discretionary activity I spend most of my time on (2%).
  • I’ve spent 10% of my time this year on connecting with people, a surprisingly high number for me. E-mail takes 1% of my overall time.
  • It turns out that yes, coding and drawing are negatively correlated (-0.63 considering all coding-related activities). But writing and drawing are positively correlated (0.44), which makes sense – I draw, and then I write a blog post to glue sketches together and give context. Earning is slightly negatively correlated with building business/skills (-0.15), but connecting is even more negatively correlated with time spent building business/skills (-0.35). So it’s probably not that consulting takes me away from building skills. Sleep is slightly negatively correlated with all records related to socializing (-0.14), but strongly negatively correlated with productive discretionary activities (-0.55). Hmm. Something to tinker with.

Some things I’m learning from tracking time on specific tasks:

  • Outlining doubles the time I take to write (and drops me from about ~30wpm to about 9wpm), but I feel that it makes things more structured.
  • Drawing takes longer too, but it makes blog posts more interesting.
  • Trying to dictate posts takes me way more time than outlining or typing it, since I’m not as used to organizing my thoughts that way.
  • Encoding litter box data takes me about a minute per data point. So spending a lot of time trying to figure out computer vision and image processing in order to partially automate the process doesn’t strictly make sense, but I’m doing it out of curiosity.
  • I generally overestimate the time I need for programming-related tasks, which is surprising. That could just be me padding my estimates to account for distractions or to make myself feel great, though.
  • I generally underestimate the time I need to write, especially if I’m figuring things out along the way.

This post took me 1:20 to draft (including data analysis), although to be fair, part of that involved a detour checking electricity use for an unrelated question. =)