Category Archives: geek

On this page:

How to update the Org 7 that comes with Emacs to Org 8 (more configuration! better exports!)

Update 2014-05-12: Simplified thanks to Sebastian’s note that Org 8 is available in the built-in package repository, yay!

The Org Mode included in Emacs 24 is version 7. Version 8 has lots of new configuration variables and the exporting mechanism has been rewritten. However, it needs to be installed in an Emacs that has not yet loaded any Org code or files. Here’s how you can upgrade your Org:

  1. Start Emacs with emacs -q. This skips your personal configuration.
  2. You will need an Internet connection for this step. Type M-x package-install, and type in org. This will install the latest version of Org from the built-in package repository.
  3. Edit your ~/.emacs.d/init.el (or ~/.emacs, if you’re using that instead). Add the following code to the beginning of the file:
    (package-initialize)
    (setq package-enable-at-startup nil)
    

    This will load the installed packages when you start Emacs, overriding the buit-in Org 7 with the Org 8 version that you installed.

    Advanced note: If you’ve downloaded Emacs Lisp code that should override code already installed through packages, you need to change this to (package-initialize nil) instead, and add (package-initialize t) after your load-path settings.

  4. Check your configuration for references to the older version of Org. In particular, look for any configuration related to exporting (ex: (require 'org-html)). You can change those lines to their Org 8 equivalents (ex: (require 'ox-html)), but it’s probably easier to just comment them out for now. You can comment out lines by adding ; to the beginning.
  5. Save your init.el and restart Emacs (this time, without the -q option). M-x org-version should now start with Org-mode version 8.
  6. Review your Emacs configuration for any changes that you will need to make. You can ask the Org Mode mailing list for help if you get stuck.

Good luck!

Emacs Chat: Phil Hagelberg

Update 2014-06-13: Transcript now available

Phil Hagelberg talks about custom keyboards, pair-programming with syme.herokuapp.com , Clojure REPLs, starter kits and better defaults, packages, helping his kids learn to think systematically, and warming up his shed-turned-office through XMPP (from Emacs, no doubt).

Quick links: http://technomancy.us , https://syme.herokuapp.com/ , http://github.com/technomancy/better-defaults , https://github.com/technomancy/dotfiles/tree/master/.emacs.d , http://leiningen.org/ , http://technomancy.us/171 (heater), http://atreus.technomancy.us (keyboards)

Guest: Phil Hagelberg

For the event page, you may click here.

Want just the audio? Get it from archive.org: MP3

Transcript

Check out Emacs Chat for more interviews like this. Got a story to tell about how you learned about or how you use Emacs? Get in touch!

 

Update on time tracking with Quantified Awesome and with Emacs

With another Quantified Self Toronto meetup in a few weeks and a conversation with fellow self-trackers, it’s time for me to think about time again.

I’ve been fixing bugs and adding small pieces of functionality to Quantified Awesome, and I spent some time improving the integration with Emacs. Now I can type ! to clock in on a task and update Quantified Awesome. Completing the task clocks me out in Emacs and updates Beeminder if appropriate. (I don’t update Quantified Awesome when finishing a task, because I just clock into the next activity.) This allows me to take advantage of Org’s clock reports for project and task-level time, at least for discretionary projects that involve my computer. I’m not going to get full coverage, but that’s what Quantified Awesome’s web interface is for. It takes very little effort to track things now, if I’m working off my to-do list. Even if I’m not, it still takes just a few taps on my phone to switch activities.

Most of my data is still medium-level, since I’m still getting the hang of sorting out my time in Emacs. Looking at data from 2014 so far, dropping partial weeks, and doing the analysis on April 14 (which is when I’m drafting this), here’s what I’ve been finding.

  • I sleep a little more than I used to: an average of 8.9 hours a day, or 37% of the time. This is up from 8.3 hours last year.
  • It takes me about an hour to get ready in the mornings. If I have a quick breakfast instead of having rice and fried egg, I can get out the door in 30-45 minutes.
  • It takes me 50-60 minutes to get downtown, whether this is by transit or bicycle. Commuting takes 3% of my time.

Little surprises:

  • I’ve spent almost twice as much time on business building or discretionary productive activities (19%) as I have earning (11%) – good to see decisions in action!
  • I’ve spent more time drawing than writing this year (5% vs 3%). Next to writing, Emacs is the productive discretionary activity I spend most of my time on (2%).
  • I’ve spent 10% of my time this year on connecting with people, a surprisingly high number for me. E-mail takes 1% of my overall time.
  • It turns out that yes, coding and drawing are negatively correlated (-0.63 considering all coding-related activities). But writing and drawing are positively correlated (0.44), which makes sense – I draw, and then I write a blog post to glue sketches together and give context. Earning is slightly negatively correlated with building business/skills (-0.15), but connecting is even more negatively correlated with time spent building business/skills (-0.35). So it’s probably not that consulting takes me away from building skills. Sleep is slightly negatively correlated with all records related to socializing (-0.14), but strongly negatively correlated with productive discretionary activities (-0.55). Hmm. Something to tinker with.

Some things I’m learning from tracking time on specific tasks:

  • Outlining doubles the time I take to write (and drops me from about ~30wpm to about 9wpm), but I feel that it makes things more structured.
  • Drawing takes longer too, but it makes blog posts more interesting.
  • Trying to dictate posts takes me way more time than outlining or typing it, since I’m not as used to organizing my thoughts that way.
  • Encoding litter box data takes me about a minute per data point. So spending a lot of time trying to figure out computer vision and image processing in order to partially automate the process doesn’t strictly make sense, but I’m doing it out of curiosity.
  • I generally overestimate the time I need for programming-related tasks, which is surprising. That could just be me padding my estimates to account for distractions or to make myself feel great, though.
  • I generally underestimate the time I need to write, especially if I’m figuring things out along the way.

This post took me 1:20 to draft (including data analysis), although to be fair, part of that involved a detour checking electricity use for an unrelated question. =)

Making my Emacs-related blog posts available for offline reading

Deepak Tripathi wanted to know how to download all of my Emacs-related posts for offline reading. It makes sense to put together something like that. Xah Lee even charges for an organized ZIP copy of his site (and he’s put together a lot of resources). I like putting together free/pay-what-you-want things, so I figured I’d add my blog posts to my git repository of Emacs notes.

My blog runs on WordPress. It has a whole bunch of other posts in it, so a straightforward Jekyll import wouldn’t do the trick. I had previously modified my WordPress theme to add a special ?dump=1 parameter for single post pages so that I could use it to archive pages. The first thing I needed to do was to come up with a list of Emacs-related blog posts.

Fortunately, http://sachachua.com/blog/emacs already lists all the pages in the Emacs category. I copied the HTML source for the list and started tinkering with the text. It was a good excuse to try the visual-regexp package – the vr/replace command made working with match groups much easier. Eventually I ended up with a list that looked like this:

wget -p http://sachachua.com/blog/2014/04/thinking-todo-keywords/?dump=1
wget -p http://sachachua.com/blog/2014/04/reflecting-10-episodes-emacs-chats/?dump=1
wget -p http://sachachua.com/blog/2014/04/org-mode-helps-deal-ever-growing-backlog/?dump=1
wget -p http://sachachua.com/blog/2014/04/reinvesting-time-and-money-into-emacs/?dump=1
...

This downloaded the posts and included images. Next, I wanted to process the downloaded HTML pages and turn them into Org files, since I’m more familiar with Org than with Markdown. Pandoc to the rescue! I took the output of find -name \*.html\* and processed it with lots more vr/replace

dos2unix ./sachachua.com/blog/2003/07/emacspeak-creator/index.html?dump=1; (echo "<html><body>"; cat ./sachachua.com/blog/2003/07/emacspeak-creator/index.html?dump=1; echo "</body></html>") > test.html; pandoc test.html -o 2003-07-emacspeak-creator.org
dos2unix ./sachachua.com/blog/2003/07/instructionalsoftwaredesign/index.html?dump=1; (echo "<html><body>"; cat ./sachachua.com/blog/2003/07/instructionalsoftwaredesign/index.html?dump=1; echo "</body></html>") > test.html; pandoc test.html -o 2003-07-instructionalsoftwaredesign.org
dos2unix ./sachachua.com/blog/2003/07/interesting-mail-stats/index.html?dump=1; (echo "<html><body>"; cat ./sachachua.com/blog/2003/07/interesting-mail-stats/index.html?dump=1; echo "</body></html>") > test.html; pandoc test.html -o 2003-07-interesting-mail-stats.org
dos2unix ./sachachua.com/blog/2003/08/emacs-macros/index.html?dump=1; (echo "<html><body>"; cat ./sachachua.com/blog/2003/08/emacs-macros/index.html?dump=1; echo "</body></html>") > test.html; pandoc test.html -o 2003-08-emacs-macros.org

This is a pretty long command line because I didn’t bother with writing a shell script to process files. When I tried running pandoc on the HTML snippets, it choked because the file didn’t have html or body tags, so I ended up adding them with echo before processing them with pandoc.

Anyway, now I had a lot of Org files. I wanted to rename them and clean up the titles. I used this totally hackish bit of code to process all the files in the directory, changing the header to a title and adding the day to the filename (just in case I want to move to Jekyll someday).

(defun sacha/process-blog-posts ()
  (interactive)
  (mapcar (lambda (x)
            (find-file x)
            ;; Set the title
            (goto-char (point-min))
            (when (looking-at "\\(\\*\\*\\)")
              (replace-match "#+TITLE:"))
            (save-buffer)
            ;; Rename the file
            (when (re-search-forward "\\([0-9]+\\)\\(th\\|nd\\|st\\|rd\\), \\([0-9]+\\)")
              (let ((day (match-string 1)))
                (rename-file
                 (buffer-file-name)
                 (replace-regexp-in-string
                  "/\\([0-9]+\\)-\\([0-9]+\\)-"
                  (concat "/processed\\&" day "-")
                  (buffer-file-name)))))
            (kill-buffer))
          (directory-files "." nil ".org$")))

I also wanted to change the image references to use the images that wget had downloaded into my uploads directory. This time I used wgrep, which is awesome. The wgrep package makes grep results editable if you use wgrep-change-to-wgrep-mode (bound to C-c C-p by default, but you can change wgrep-enable-key). If you combine that with vr/replace from visual-regexp or some keyboard macros, you can edit a whole lot of things quickly. Save the changes with C-x C-s (wgrep-finish-edit), and you’re done!

So now I had a bunch of Org files that were in reasonable shape. I wanted to regenerate the HTML pages so that people could browse them even if they weren’t familiar with Org. I set up the relevant org-publish-project-alist entries in a build-site.el that a Makefile could use, and I published the project.

Going from HTML to Org to HTML meant losing some information and lots of my blog posts need reviewing, but it’s a good start. Since I draft many of my Emacs-related blog posts in Org anyway, I can copy the Org source into my emacs-notes repository going forward.

Anyway, there you have it for your grepping pleasure! https://github.com/sachac/emacs-notes/

2048 in Emacs, and colours too

While browsing through M-x list-packages, I noticed that there was a new MELPA package that implemented the 2048 game in Emacs. I wrote the following code to colorize it. Haven’t tested the higher numbers yet, but they’re easy enough to tweak if the colours disagree with your theme. =)

2014-04-16 23_27_25-emacs@SACHA-X220

(defface 2048-2-face '((t (:foreground "red"))) "Face used for 2" :group '2048-game)
(defface 2048-4-face '((t (:foreground "orange"))) "Face used for 4" :group '2048-game)
(defface 2048-8-face '((t (:foreground "yellow"))) "Face used for 8" :group '2048-game)
(defface 2048-16-face '((t (:foreground "green"))) "Face used for 16" :group '2048-game)
(defface 2048-32-face '((t (:foreground "lightblue" :bold t))) "Face used for 32" :group '2048-game)
(defface 2048-64-face '((t (:foreground "lavender" :bold t))) "Face used for 64" :group '2048-game)
(defface 2048-128-face '((t (:foreground "SlateBlue" :bold t))) "Face used for 128" :group '2048-game)
(defface 2048-256-face '((t (:foreground "MediumVioletRed" :bold t))) "Face used for 256" :group '2048-game)
(defface 2048-512-face '((t (:foreground "tomato" :bold t))) "Face used for 512" :group '2048-game)
(defface 2048-1024-face '((t (:foreground "SkyBlue1" :bold t))) "Face used for 1024" :group '2048-game)
(defface 2048-2048-face '((t (:foreground "lightgreen" :bold t))) "Face used for 2048" :group '2048-game)

(defvar 2048-font-lock-keywords
  '(("\\<2\\>" 0 '2048-2-face)
    ("\\<4\\>" 0 '2048-4-face)
    ("\\<8\\>" 0 '2048-8-face)
    ("\\<16\\>" 0 '2048-16-face)
    ("\\<32\\>" 0 '2048-32-face)
    ("\\<64\\>" 0 '2048-64-face)
    ("\\<128\\>" 0 '2048-128-face)
    ("\\<256\\>" 0 '2048-256-face)
    ("\\<512\\>" 0 '2048-512-face)
    ("\\<1024\\>" 0 '2048-1024-face)
    ("\\<2048\\>" 0 '2048-2048-face)))

(defun sacha/2048-fontify ()
  (font-lock-add-keywords nil 2048-font-lock-keywords))

(defun sacha/2048-set-font-size ()
  (text-scale-set 5))

(use-package 2048-game
  :config
  (progn
   (add-hook '2048-mode-hook 'sacha/2048-fontify)
   (add-hook '2048-mode-hook 'sacha/2048-set-font-size)))

Getting R and ggplot2 to work in Emacs Org Mode Babel blocks; also, tracking the number of TODOs

I started tracking the number of tasks I had in Org Mode so that I could find out if my TODO list tended to shrink or grow. It was easy to write a function in Emacs Lisp to count the number of tasks in different states and summarize them in a table.

(defun sacha/org-count-tasks-by-status ()
  (interactive)
  (let ((counts (make-hash-table :test 'equal))
        (today (format-time-string "%Y-%m-%d" (current-time)))
        values output)
    (org-map-entries
     (lambda ()
       (let* ((status (elt (org-heading-components) 2)))
         (when status
           (puthash status (1+ (or (gethash status counts) 0)) counts))))
     nil
     'agenda)
    (setq values (mapcar (lambda (x)
                           (or (gethash x counts) 0))
                         '("DONE" "STARTED" "TODO" "WAITING" "DELEGATED" "CANCELLED" "SOMEDAY")))
    (setq output
          (concat "| " today " | "
                  (mapconcat 'number-to-string values " | ")
                  " | "
                  (number-to-string (apply '+ values))
                  " | "
                  (number-to-string
                   (round (/ (* 100.0 (car values)) (apply '+ values))))
                  "% |"))
    (if (called-interactively-p 'any)
        (insert output)
      output)))
(sacha/org-count-tasks-by-status)

I ran this code over several days. Here are my results as of 2014-05-01:

Date DONE START. TODO WAIT. DELEG. CANC. SOMEDAY Total % done + done +canc. + total + t – d – c Note
2014-04-16 1104 1 403 3 1 104 35 1651 67%
2014-04-17 1257 0 114 4 1 171 107 1654 76% 153 67 3 -217 Lots of trimming
2014-04-18 1292 0 74 4 5 183 100 1658 78% 35 12 4 -43 A little bit more trimming
2014-04-20 1305 0 80 4 5 183 100 1677 78% 13 0 19 6
2014-04-21 1311 1 78 4 4 184 99 1681 78% 6 1 4 -3
2014-04-22 1313 2 75 4 4 184 99 1681 78% 2 0 0 -2
2014-04-23 1369 4 66 4 5 186 101 1735 79% 56 2 54 -4 Added sharing/index.org
2014-04-24 1371 3 69 4 5 186 101 1739 79% 2 0 4 2
2014-04-25 1379 3 60 3 5 189 103 1742 79% 8 3 3 -8
2014-04-26 1384 3 65 3 5 192 103 1755 79% 5 3 13 5
2014-04-27 1389 2 66 3 5 192 103 1760 79% 5 0 5 0
2014-04-28 1396 3 67 3 5 192 103 1769 79% 7 0 9 2
2014-04-29 1396 3 67 3 5 192 103 1769 79% 0 0 0 0
2014-04-30 1404 4 70 4 5 192 103 1782 79% 8 0 13 5
2014-05-01 1413 4 80 3 4 193 103 1800 79% 9 1 18 8

Here’s the source for that table:

#+NAME: burndown
#+RESULTS:
|       Date | DONE | START. | TODO | WAIT. | DELEG. | CANC. | SOMEDAY | Total | % done | + done | +canc. | + total | + t - d - c | Note                       |
|------------+------+--------+------+-------+--------+-------+---------+-------+--------+--------+--------+---------+-------------+----------------------------|
| 2014-04-16 | 1104 |      1 |  403 |     3 |      1 |   104 |      35 |  1651 |    67% |        |        |         |             |                            |
| 2014-04-17 | 1257 |      0 |  114 |     4 |      1 |   171 |     107 |  1654 |    76% |    153 |     67 |       3 |        -217 | Lots of trimming           |
| 2014-04-18 | 1292 |      0 |   74 |     4 |      5 |   183 |     100 |  1658 |    78% |     35 |     12 |       4 |         -43 | A little bit more trimming |
| 2014-04-20 | 1305 |      0 |   80 |     4 |      5 |   183 |     100 |  1677 |    78% |     13 |      0 |      19 |           6 |                            |
| 2014-04-21 | 1311 |      1 |   78 |     4 |      4 |   184 |      99 |  1681 |    78% |      6 |      1 |       4 |          -3 |                            |
| 2014-04-22 | 1313 |      2 |   75 |     4 |      4 |   184 |      99 |  1681 |    78% |      2 |      0 |       0 |          -2 |                            |
| 2014-04-23 | 1369 |      4 |   66 |     4 |      5 |   186 |     101 |  1735 |    79% |     56 |      2 |      54 |          -4 | Added sharing/index.org    |
| 2014-04-24 | 1371 |      3 |   69 |     4 |      5 |   186 |     101 |  1739 |    79% |      2 |      0 |       4 |           2 |                            |
| 2014-04-25 | 1379 |      3 |   60 |     3 |      5 |   189 |     103 |  1742 |    79% |      8 |      3 |       3 |          -8 |                            |
| 2014-04-26 | 1384 |      3 |   65 |     3 |      5 |   192 |     103 |  1755 |    79% |      5 |      3 |      13 |           5 |                            |
| 2014-04-27 | 1389 |      2 |   66 |     3 |      5 |   192 |     103 |  1760 |    79% |      5 |      0 |       5 |           0 |                            |
| 2014-04-28 | 1396 |      3 |   67 |     3 |      5 |   192 |     103 |  1769 |    79% |      7 |      0 |       9 |           2 |                            |
| 2014-04-29 | 1396 |      3 |   67 |     3 |      5 |   192 |     103 |  1769 |    79% |      0 |      0 |       0 |           0 |                            |
| 2014-04-30 | 1404 |      4 |   70 |     4 |      5 |   192 |     103 |  1782 |    79% |      8 |      0 |      13 |           5 |                            |
| 2014-05-01 | 1413 |      4 |   80 |     3 |      4 |   193 |     103 |  1800 |    79% |      9 |      1 |      18 |           8 |                            |
#+TBLFM: @3$11..@>$11=$2-@-1$2::@3$13..@>$13=$9-@-1$9::@3$14..@>$14=$13-$11-($7-@-1$7)::@3$12..@>$12=$7-@-1$7

I wanted to graph this with Gnuplot, but it turns out that Gnuplot is difficult to integrate with Emacs on Microsoft Windows. I gave up after a half an hour of poking at it, since search results indicated there were long-standing problems with how Gnuplot got input from Emacs. Besides, I’d been meaning to learn more R anyway, and R is more powerful when it comes to statistics and data visualization.

Getting R to work with Org Mode babel blocks in Emacs on Windows was a challenge. Here are some of the things I ran into.

The first step was easy: Add R to the list of languages I could evaluate in a source block (I already had dot and ditaa from previous experiments).

(org-babel-do-load-languages
 'org-babel-load-languages
 '((dot . t)
   (ditaa . t) 
   (R . t)))

But my code didn’t execute at all, even when I was trying something that printed out results instead of drawing images. I got a little lost trying to dig into org-babel-execute:R with edebug, eventually ending up in comint.el. The real solution was even easier. I had incorrectly set inferior-R-program-name to the path of R in my configuration, which made M-x R work but which meant that Emacs was looking in the wrong place for the options to pass to R (which Org Babel relied on). The correct way to do this is to leave inferior-R-program-name with the default value (Rterm) and make sure that my system path included both the bin directory and the bin\x64 directory.

Then I had to pick up the basics of R again. It took me a little time to figure out that I needed to parse the columns I pulled in from Org, using strptime to convert the date column and as.numeric to convert the numbers. Eventually, I got it to plot some results with the regular plot command.

dates <- strptime(as.character(data$Date), "%Y-%m-%d")
tasks_done <- as.numeric(data$DONE)
tasks_uncancelled <- as.numeric(data$Total) - as.numeric(data$CANC.)
df <- data.frame(dates, tasks_done, tasks_uncancelled)
plot(x=dates, y=tasks_uncancelled, ylim=c(0,max(tasks_uncancelled)))
lines(x=dates, y=tasks_uncancelled, col="blue", type="o")
lines(x=dates, y=tasks_done, col="green", type="o")

r-plot

I wanted prettier graphs, though. I installed the ggplot2 package and started figuring it out. No matter what I did, though, I ended up with a blank white image instead of my graph. If I used M-x R instead of evaluating the src block, the code worked. Weird! Eventually I found out that adding print(...) around my ggplot made it display the image correctly. Yay! Now I had what I wanted.

library(ggplot2)
dates <- strptime(as.character(data$Date), "%Y-%m-%d")
tasks_done <- as.numeric(data$DONE)
tasks_uncancelled <- as.numeric(data$Total) - as.numeric(data$CANC.)
df <- data.frame(dates, tasks_done, tasks_uncancelled)
plot = ggplot(data=df, aes(x=dates, y=tasks_done, ymin=0)) + geom_line(color="#009900") + geom_point() + geom_line(aes(y=tasks_uncancelled), color="blue") + geom_point(aes(y=tasks_uncancelled))
print(plot)

 r-graph

The blue line represents the total number of tasks (except for the cancelled ones), and the green line represents tasks that are done.

Here’s something that looks a little more like a burn down chart, since it shows just the number of things to be done:

library(ggplot2)
dates <- strptime(as.character(data$Date), "%Y-%m-%d")
tasks_remaining <- as.numeric(data$Total) - as.numeric(data$CANC.) - as.numeric(data$DONE)
df <- data.frame(dates, tasks_remaining)
plot = ggplot(data=df, aes(x=dates, y=tasks_remaining, ymin=0)) + geom_line(color="#009900") + geom_point()
print(plot)

r-graph-2

The drastic decline there is me realizing that I had lots of tasks that were no longer relevant, not me being super-productive. =)

As it turns out, I tend to add new tasks at about the rate that I finish them (or slightly more). I think this is okay. It means I’m working on things that have next steps, and next steps, and steps beyond that. If I add more tasks, that gives me more variety to choose from. Besides, I have a lot of repetitive tasks, so those never get marked as DONE over here.

Anyway, cool! Now that I’ve gotten R to work on my system, you’ll probably see it in even more of these blog posts. =D Hooray for Org Babel and R!

Update 2014-05-09: Stephen suggested http://blogs.neuwirth.priv.at/software/2012/03/28/r-and-emacs-with-org-mode/ for more tips on setting up Org Mode with R and Emacs Speaks Statistics (ESS).