Category Archives: emacs

On this page:

Cobbling together a semi-auto-responder using Emacs, Gnus, and org-contacts

It turns out that lots of people are interested in an e-mail-based course for learning Emacs Lisp. Yay! =) Maybe it’s the idea of bite-size chunks. Maybe it’s the ease of asking questions. Maybe it’s the regular reminders to work on something. Who knows? Whatever the reason, it’s awesome to see so many people willing to join me on this experiment.

Since this is my first time to venture into the world of teaching people online, I wanted to see how far I could push actually doing all the mails myself, instead of just signing up for an Aweber account and handing everyone off to an impersonal autoresponder. I dusted off Gnus, offlineimap, and org-contacts, and started figuring out my workflow. I’ll share how that workflow’s evolving so that you can get a sense of how someone might write little bits of Emacs Lisp to make something repetitive easier.

For the first little while, I got by with using C-x r s (copy-to-register) and C-x r i (insert-register) to store the text that I needed.
Sometimes I needed to paste in the welcome message and checklist, and sometimes I needed to paste in the first lesson. By using registers, I could insert whatever I wanted instead of going through the kill ring. I also had another bit of templated code in yet another register so that I could easily create an org-contacts entry for the person whose mail I was replying to. In the beginning, I used tasks under each person’s heading to indicate that I had sent them the checklist or that I had sent them the first lesson. Eventually, I changed my org-contacts notes so that the TODO state of each person showed which lesson I was going to send them next, or CHECKLIST if I was waiting for their reply to the checklist. I also set up Org so that it would automatically log when the TODO state was changed.

#+TODO: TODO | DONE
#+TODO: CHECKLIST(c!) BEGINNER1(1!) BEGINNER2(2!) BEGINNER3(3!) BEGINNER4(4!) FULL(f!) | FINISHED(x!)
#+TODO: | CANCELLED

* Who
** CHECKLIST Jane Smith ...
** BEGINNER1 John Smith
   SCHEDULED: <2014-05-28 Wed>
   :PROPERTIES:
   :EMAIL: [email protected]
   :END:
(notes from the messages, etc.)

I wrote some code to make it easier to send someone a checklist and create a note for them in my org-contacts file. I bound it to C-c e c for convenience.
(The bind-key function is defined by a package.)

(setq sacha/elisp-course-checklist-body "... really long text here...")
(defun sacha/elisp-course-checklist ()
  "Copy this message and put it at the end as a checklist item. 
Start a message with the checklist."
  (interactive)
  (gnus-summary-scroll-up 1)
  (with-current-buffer gnus-article-buffer
    (let ((message (buffer-substring-no-properties (point-min) (point-max)))
          (email (cadr (org-contacts-gnus-get-name-email))))
      (with-current-buffer "elisp-course.org"
        (save-excursion
          (goto-char (point-max))
          (save-excursion
            (insert "\n** " message)
            (org-set-property "EMAIL" email)
            (org-todo "CHECKLIST"))))))
  (gnus-summary-followup-with-original nil)
  (goto-char (point-max))
  (insert sacha/elisp-course-checklist-body))
(bind-key "C-c e c" 'sacha/elisp-course-checklist)

This made it easier for me to read the starred messages from my inbox and use C-c e c to get a head start on processing people’s introductory messages.
Yay! I used the register trick to help me reply to people who were ready for the first lesson. After the first few replies, I noticed that the attachment code was fine even if I put that in the register too, so I added it as well.

Things got more complicated when I started processing lesson 2. I didn’t want to have to set up and remember lots of different registers, and I didn’t want to manually update the TODO states either. So I started defining functions that I could call with keyboard shortcuts:

(defun sacha/elisp-course-1 ()
  (interactive)
  (let ((marker (org-contacts-gnus-article-from-get-marker)))
    (if marker
        (org-with-point-at marker
          (org-todo "BEGINNER2"))))
  ;; Find the person's contact record
  (gnus-summary-scroll-up 1)
  (gnus-summary-followup-with-original nil)
  (message-goto-subject)
  (message-delete-line)
  (insert (concat "Subject: " sacha/elisp-course-1-subject "\n"))
  (goto-char (point-max))
  (insert sacha/elisp-course-1-body))
(bind-key "C-c e 1" 'sacha/elisp-course-1)
(defun sacha/elisp-course-2 ()
  (interactive)
  (let ((marker (org-contacts-gnus-article-from-get-marker)))
    (if marker
        (org-with-point-at marker
          (org-todo "BEGINNER3"))))
  ;; Find the person's contact record
  (gnus-summary-scroll-up)
  (gnus-summary-followup-with-original nil)
  (goto-char (point-max))
  (insert sacha/elisp-course-2-body))
(bind-key "C-c e 2" 'sacha/elisp-course-2)

Really, though, it doesn’t make sense to have a lot of duplicated code. So I wrote some code that would use the person’s TODO keyword to look up the message to send them, and then move them to the next keyword. Now I don’t need sacha/elisp-course-1 or sacha/elisp-course-2 any more.

(setq sacha/elisp-course-info
      `(("CHECKLIST" nil ,sacha/elisp-course-checklist-body)
        ("BEGINNER1" ,sacha/elisp-course-1-subject ,sacha/elisp-course-1-body)
        ("BEGINNER2" ,sacha/elisp-course-2-subject ,sacha/elisp-course-2-body)))

(defun sacha/elisp-course-process (subject body &optional state)
  "Process this course entry."
  (if (derived-mode-p 'org-mode)
      (progn
        ;; Move this node to the next state and compose a message
        (if state (org-todo state))
        (org-todo 'right)
        (message-mail (org-entry-get (point) "EMAIL") subject)
        (goto-char (point-max))
        (insert body))
    ;; Doing this from Gnus; find the person's info
    (let ((marker (org-contacts-gnus-article-from-get-marker)))
      (if marker (org-with-point-at marker
                   (if state (org-todo state))
                   (org-todo 'right)))
      ;; Compose a reply
      (gnus-summary-scroll-up 1)
      (gnus-summary-followup-with-original nil)
      (message-goto-subject)
      (message-delete-line)
      (insert (concat "Subject: " subject "\n"))
      (goto-char (point-max))
      (insert body))))

(defun sacha/elisp-course-guess-and-process (&optional state)
  (interactive (list (if current-prefix-arg (read-string "State: "))))
  (let ((current-state
         (or state (elt
                    (if (derived-mode-p 'org-mode)
                        (org-heading-components) 
                      (let ((marker (org-contacts-gnus-article-from-get-marker)))
                        (if marker (org-with-point-at marker (org-heading-components)))))
                    2))))
    (sacha/elisp-course-process
     (elt (assoc current-state sacha/elisp-course-info) 1)
     (elt (assoc current-state sacha/elisp-course-info) 2)
     state)))
(bind-key "C-c e e" 'sacha/elisp-course-guess-and-process)

Come to think of it, I should totally have it schedule the next update for the next Wednesday, too. ;) That’s just (org-schedule "+wed"). Neat, huh?
And I’m sure there are all sorts of ways the code can be simpler, but it works for me at the moment, so hooray!

I really like this approach. It lets me pull in standard information while also letting me customize the messages and how it fits into my task tracking. I can’t get that with Gmail (even with canned responses), and I’m not sure any CRM is going to be quite as awesome as this. I can’t wait to see how else we’ll tweak this as we go through more conversations. I’d like to get better at:

  • having a consistent place where I can process all the messages and make sure nothing falls through the cracks; I currently star messages to make sure I process them, since the Gmail label folder in IMAP seems to be missing some messages
  • seeing all Gnus conversations related to an org-contacts entry
  • reaching out to people proactively with the next lesson, even if they haven’t e-mailed me (or maybe I should wait for them?)

Anyway, that’s an example of writing a little bit of Emacs Lisp in order to connect different packages. Gnus handles mail, Org handles notes, org-contacts links the two together, and with a little bit of custom code, I can make the combination fit what I want to do. I read the source code of org-contacts to find out how I could look up the appropriate note, and I looked at org-shiftright to find out how to move things to the next TODO state. If you know something that works roughly like what you want it to work, you can find out how it does things and then copy that.

As for the course itself: I’ve been sending people links to the HTML output, attached .txt files (with -*- mode: org -*-) so they can open it in Emacs if they want, and inline text so that they can skim it briefly in their e-mail client if they want to. I’m not perfectly happy with the plain-text formats, but it seems to be a reasonable compromise, and so far people have been able to deal with it. I’ve been improving pieces of it based on feedback on clarity, suggestions for good examples, and so on. I didn’t take all the feedback; after thinking about some of the suggestions, I still preferred it my way. It’s shaping up quite nicely, though!

If you’re curious about the beginner’s course on reading Emacs Lisp, e-mail me at [email protected] and we’ll see how this works out. I’m certainly learning a lot. =)

Emacs Chat: Bozhidar Batsov

UPDATE 2014-06-13: The transcript is now available.

Bozhidar Batsov (emacsredux.com) shares how he got into Emacs and Emacs Lisp. He also demonstrates cool features from Prelude and Projectile, which are great if you do a lot of programming. Check it out!

Quick Links: https://twitter.com/bbatsov , https://twitter.com/emacs_knight , http://emacsredux.com , https://github.com/bbatsov/prelude , https://github.com/bbatsov/projectile . If you like his work, there’s https://www.gittip.com/bbatsov/

Guest: Bozhidar Batsov

For the event page, you may click here.

Want just the audio? Get it from archive.org: MP3

Transcript here!

Check out Emacs Chat for more interviews like this. Got a story to tell about how you learned about or how you use Emacs? Get in touch!

Planning an e-mail-based course for Emacs Lisp

I’ve been working on an Emacs Lisp beginner’s course, something focused on helping people become more comfortable configuring Emacs. The web-based guide is taking shape quite nicely, but it’s still a lot of scrolling, and it can still feel overwhelming for newbies. I think it might make sense to offer it as an e-mail course. That way, I can spread the lessons out, help people with their questions, and improve things based on people’s feedback.

2014-05-12 How can I take Learn How to Read Emacs Lisp to the next level #emacs #packaging #writing #teaching

2014-05-12 How can I take Learn How to Read Emacs Lisp to the next level #emacs #packaging #writing #teaching

I can improve the guide by adding more structure, examples, exercises, and so on. I’ve requested several books on e-learning and course design, and I’m looking forward to learning more over the years. And I can also improve it by testing it with people… =)

2014-05-14 Planning an e-mail-based course for Emacs Lisp #emacs #teaching

2014-05-14 Planning an e-mail-based course for Emacs Lisp #emacs #teaching

I floated the idea on Twitter and lots of people e-mailed me to join. Instead of setting up an autoresponder, I decided that I would do things by hand as much as I could. That way, I can personalize the messages based on people’s interests and configuration, and I can enjoy more of the back-and-forth conversation.

After getting annoyed with the SSL hassles of setting up Gnus on Windows, I decided to just use my Linux-based virtual machine for handling mail. That was pretty straightforward, although for some reason, my IMAP view of Gmail doesn’t have all of the messages under a label. It just means that I have to manually re-check the messages to make sure nothing slips through the cracks.

I used an Org file to keep notes on each person, including TODOs under each of them. I sent everyone a checklist to see which section we should start with. A few people are starting at the beginning, and others will get the e-mails once I’ve updated those sections. Text registers (C-x r s) were really helpful since I was pasting different things into different e-mails. I’m still figuring out the workflow for this, and I’m sure I’ll automate pieces of it as more people move through the course.

I’ve sent the first section to some people already, including the Org version in the e-mail body and as an attachment, and linking to the web-based version. The Org version is a little more cluttered than the text export, but the text export uses box quotes, so I figured the Org version was the best to start with.

2014-05-16 A plan for delivering the Emacs Lisp course #emacs #teaching

2014-05-16 A plan for delivering the Emacs Lisp course #emacs #teaching

Want to be part of this? E-mail me at [email protected]

How to update the Org 7 that comes with Emacs to Org 8 (more configuration! better exports!)

Update 2014-05-12: Simplified thanks to Sebastian’s note that Org 8 is available in the built-in package repository, yay!

The Org Mode included in Emacs 24 is version 7. Version 8 has lots of new configuration variables and the exporting mechanism has been rewritten. However, it needs to be installed in an Emacs that has not yet loaded any Org code or files. Here’s how you can upgrade your Org:

  1. Start Emacs with emacs -q. This skips your personal configuration.
  2. You will need an Internet connection for this step. Type M-x package-install, and type in org. This will install the latest version of Org from the built-in package repository.
  3. Edit your ~/.emacs.d/init.el (or ~/.emacs, if you’re using that instead). Add the following code to the beginning of the file:
    (package-initialize)
    (setq package-enable-at-startup nil)
    

    This will load the installed packages when you start Emacs, overriding the buit-in Org 7 with the Org 8 version that you installed.

    Advanced note: If you’ve downloaded Emacs Lisp code that should override code already installed through packages, you need to change this to (package-initialize nil) instead, and add (package-initialize t) after your load-path settings.

  4. Check your configuration for references to the older version of Org. In particular, look for any configuration related to exporting (ex: (require 'org-html)). You can change those lines to their Org 8 equivalents (ex: (require 'ox-html)), but it’s probably easier to just comment them out for now. You can comment out lines by adding ; to the beginning.
  5. Save your init.el and restart Emacs (this time, without the -q option). M-x org-version should now start with Org-mode version 8.
  6. Review your Emacs configuration for any changes that you will need to make. You can ask the Org Mode mailing list for help if you get stuck.

Good luck!

Emacs Chat: Phil Hagelberg

Update 2014-06-13: Transcript now available

Phil Hagelberg talks about custom keyboards, pair-programming with syme.herokuapp.com , Clojure REPLs, starter kits and better defaults, packages, helping his kids learn to think systematically, and warming up his shed-turned-office through XMPP (from Emacs, no doubt).

Quick links: http://technomancy.us , https://syme.herokuapp.com/ , http://github.com/technomancy/better-defaults , https://github.com/technomancy/dotfiles/tree/master/.emacs.d , http://leiningen.org/ , http://technomancy.us/171 (heater), http://atreus.technomancy.us (keyboards)

Guest: Phil Hagelberg

For the event page, you may click here.

Want just the audio? Get it from archive.org: MP3

Transcript

Check out Emacs Chat for more interviews like this. Got a story to tell about how you learned about or how you use Emacs? Get in touch!

 

Making my Emacs-related blog posts available for offline reading

Deepak Tripathi wanted to know how to download all of my Emacs-related posts for offline reading. It makes sense to put together something like that. Xah Lee even charges for an organized ZIP copy of his site (and he’s put together a lot of resources). I like putting together free/pay-what-you-want things, so I figured I’d add my blog posts to my git repository of Emacs notes.

My blog runs on WordPress. It has a whole bunch of other posts in it, so a straightforward Jekyll import wouldn’t do the trick. I had previously modified my WordPress theme to add a special ?dump=1 parameter for single post pages so that I could use it to archive pages. The first thing I needed to do was to come up with a list of Emacs-related blog posts.

Fortunately, http://sachachua.com/blog/emacs already lists all the pages in the Emacs category. I copied the HTML source for the list and started tinkering with the text. It was a good excuse to try the visual-regexp package – the vr/replace command made working with match groups much easier. Eventually I ended up with a list that looked like this:

wget -p http://sachachua.com/blog/2014/04/thinking-todo-keywords/?dump=1
wget -p http://sachachua.com/blog/2014/04/reflecting-10-episodes-emacs-chats/?dump=1
wget -p http://sachachua.com/blog/2014/04/org-mode-helps-deal-ever-growing-backlog/?dump=1
wget -p http://sachachua.com/blog/2014/04/reinvesting-time-and-money-into-emacs/?dump=1
...

This downloaded the posts and included images. Next, I wanted to process the downloaded HTML pages and turn them into Org files, since I’m more familiar with Org than with Markdown. Pandoc to the rescue! I took the output of find -name \*.html\* and processed it with lots more vr/replace

dos2unix ./sachachua.com/blog/2003/07/emacspeak-creator/index.html?dump=1; (echo "<html><body>"; cat ./sachachua.com/blog/2003/07/emacspeak-creator/index.html?dump=1; echo "</body></html>") > test.html; pandoc test.html -o 2003-07-emacspeak-creator.org
dos2unix ./sachachua.com/blog/2003/07/instructionalsoftwaredesign/index.html?dump=1; (echo "<html><body>"; cat ./sachachua.com/blog/2003/07/instructionalsoftwaredesign/index.html?dump=1; echo "</body></html>") > test.html; pandoc test.html -o 2003-07-instructionalsoftwaredesign.org
dos2unix ./sachachua.com/blog/2003/07/interesting-mail-stats/index.html?dump=1; (echo "<html><body>"; cat ./sachachua.com/blog/2003/07/interesting-mail-stats/index.html?dump=1; echo "</body></html>") > test.html; pandoc test.html -o 2003-07-interesting-mail-stats.org
dos2unix ./sachachua.com/blog/2003/08/emacs-macros/index.html?dump=1; (echo "<html><body>"; cat ./sachachua.com/blog/2003/08/emacs-macros/index.html?dump=1; echo "</body></html>") > test.html; pandoc test.html -o 2003-08-emacs-macros.org

This is a pretty long command line because I didn’t bother with writing a shell script to process files. When I tried running pandoc on the HTML snippets, it choked because the file didn’t have html or body tags, so I ended up adding them with echo before processing them with pandoc.

Anyway, now I had a lot of Org files. I wanted to rename them and clean up the titles. I used this totally hackish bit of code to process all the files in the directory, changing the header to a title and adding the day to the filename (just in case I want to move to Jekyll someday).

(defun sacha/process-blog-posts ()
  (interactive)
  (mapcar (lambda (x)
            (find-file x)
            ;; Set the title
            (goto-char (point-min))
            (when (looking-at "\\(\\*\\*\\)")
              (replace-match "#+TITLE:"))
            (save-buffer)
            ;; Rename the file
            (when (re-search-forward "\\([0-9]+\\)\\(th\\|nd\\|st\\|rd\\), \\([0-9]+\\)")
              (let ((day (match-string 1)))
                (rename-file
                 (buffer-file-name)
                 (replace-regexp-in-string
                  "/\\([0-9]+\\)-\\([0-9]+\\)-"
                  (concat "/processed\\&" day "-")
                  (buffer-file-name)))))
            (kill-buffer))
          (directory-files "." nil ".org$")))

I also wanted to change the image references to use the images that wget had downloaded into my uploads directory. This time I used wgrep, which is awesome. The wgrep package makes grep results editable if you use wgrep-change-to-wgrep-mode (bound to C-c C-p by default, but you can change wgrep-enable-key). If you combine that with vr/replace from visual-regexp or some keyboard macros, you can edit a whole lot of things quickly. Save the changes with C-x C-s (wgrep-finish-edit), and you’re done!

So now I had a bunch of Org files that were in reasonable shape. I wanted to regenerate the HTML pages so that people could browse them even if they weren’t familiar with Org. I set up the relevant org-publish-project-alist entries in a build-site.el that a Makefile could use, and I published the project.

Going from HTML to Org to HTML meant losing some information and lots of my blog posts need reviewing, but it’s a good start. Since I draft many of my Emacs-related blog posts in Org anyway, I can copy the Org source into my emacs-notes repository going forward.

Anyway, there you have it for your grepping pleasure! https://github.com/sachac/emacs-notes/