Category Archives: org

Org Mode publishing workflow for Sketched Books collection

I want to publish things in chunks that are bigger and more logical than blog posts, so I’ve been experimenting with my ZIP/PDF/EPUB/MOBI workflow.

Org Mode, Calibre, and Vagrant are terrific tools. Org Mode lets me write easy-to-modify source that I can export to different formats, like HTML and LaTeX (with the Beamer package), which lets me use PdfLatex to convert to PDF. Calibre converts HTML to EPUB and MOBI. Since tools can be difficult to set up on Windows, I use Vagrant to set up a virtual machine running Linux and I share my working directory with it.

multiple-cursors was so useful when I was wrangling the directory listing into the right format for Org. I’m glad I learned how to use it!

Here’s a Makefile I put together that simplifies the process for me:

all: index.html sketched-books.epub sketched-books.mobi ebook.pdf sketched-books.zip

clean:
	rm -f *.dvi *.log *.nav *.out *.tex *.snm *.toc

distclean: clean
	rm -f Sketched\ Books.zip index.html *.epub *.pdf *.mobi

Sketched\ Books.zip: *.png index.html
	(cd ..; zip sketched-books/sketched-books.zip sketched-books/* -i *.css -i *.png -i *.html)

index.html: index.org
	emacs --batch -l build.el index.org -f org-html-export-to-html --kill
	cp index.html index.tmp
	sed -e "s/org-ul/org-ul small-block-grid-3/" -e 's/div id="content"/div id="content" class="columns"/' -e 's/class="status"/class="status columns"/' index.tmp > index.html
	rm -f index.html~ index.tmp

ebook.html: ebook.org
	emacs --batch -l build.el ebook.org -f org-html-export-to-html --kill

cover-base.png:
	montage *Sketched*.png -geometry -30-30 -thumbnail x400 -tile 6x5 cover.png

sketched-books.epub: ebook.html
	ebook-convert ebook.html sketched-books.epub --cover cover.png --authors "Sacha Chua" --language "English"

sketched-books.mobi: ebook.html
	ebook-convert ebook.html sketched-books.mobi --cover cover.png --authors "Sacha Chua" --language "English"

ebook.tex: ebook.org
	emacs --batch -l build.el ebook.org -f org-beamer-export-to-latex --kill

ebook.pdf: ebook.tex
	pdflatex ebook.tex
	cp ebook.pdf sketched-books.pdf
	rm ebook.pdf

And here’s a very simple build.el:

(require 'package)
(package-initialize)
(require 'ox-beamer)
(setq org-html-validation-link nil)
(setq org-export-with-section-numbers nil)
(setq backup-directory-alist '(("." . nil)))

This assumes I’ve already set up the environment by installing the latest Org from MELPA.

You can check out the index.org and ebook.org I use, too.

I’m not quite sure about the MOBI output yet. I have to test it on a Kindle, or in the app on my tablet. Most of the things display fine on my computer, though. Hooray!

Neat, huh? I want to get into the habit of making and also making it easy for me to update these things. You can check out the results at http://sketchedbooks.com/collection .

Someday I might even figure out how to use the Gumroad API to publish updated resources automatically. Wouldn’t that be neat? In the meantime, I’ll just have to replace them myself.

I like giving people the ability to choose which files to download. If I get annoyed with replacing multiple files, though, I might change this to one large ZIP that has the images, PDF, EPUB, and MOBI.

View the source on Github

Emacs Chat: Karl Voit

Org Mode, Memacs, lazyblorg, .emacs, Yasnippet, tags . http://karl-voit.at , http://twitter.com/n0v0id , http://github.com/novoid.

Check out Karl’s notes for more details. (Or at least, you can check them out when his server is up again!)

Thanks, Karl!

Got an interesting Emacs workflow? Please share. =) Happy to bring on more people for Emacs Chats. Also, check out the upcoming Emacs Hangout on Dec 17 (8 PM Toronto)!

Check out TRANSCRIPT here!

Improving how I organize notes with Org Mode

Let me think about how I organize my Org Mode files, and how I might improve that. =)

Separate files

You can put different things in different files, of course. I use a few large Org files instead of lots of small ones because I prefer searching within files rather than searching within directories. Separate files make sense when I want to define org-custom-agenda-commands that summarize a subset of my tasks. No sense in going through all my files if I only want the cooking-related ones.

What would help me make better use of lots of files? I can practise on my book notes, which I’ve split up into one file per book. It’s easy enough to open files based on their titles (which I put in my filenames). But I don’t have that overall sense of it yet. Maybe #+INDEX: entries, if I can get them to generate multiple hyperlinks and I have a shortcut to quickly grep across multiple files (maybe with a few lines of context)? Maybe a manual outline, an index like the one I’ve been building for my blog posts? I can work with that as a starter, I think.

Okay. So, coming at it from several directions here:

  • A manual map based on an outline with lots of links, with some links between topics as well – similar to my blog outline or to my evil plans document
  • Quick way to grep? helm-do-grep works, but my long filenames are hard to read.
  • Links between notes and to blog posts
  • TODOs, agenda views

Outlines

Within each file, outlines work really well. You can create any number of headings by using *, and you can use TAB to collapse or expand headings. You can promote or demote subtrees, move them around, or even sort them.

I generally have a few high-level headings, like this:

* Projects
** One heading per current project
*** TODO Project task
* Reference
Information I need to keep track of
* Other notes
* Tasks
** TODO Lots of miscellaneous tasks go here
** TODO Lots of miscellaneous tasks go here
** TODO Lots of miscellaneous tasks go here

Every so often, I do some clean-up on my Org files, refiling or archiving headings as needed. This makes it easier to review my current list of projects. I keep this list separate from the grab-bag of miscellaneous tasks and notes that might not yet be related to particular projects.

I use org-refile with the C-u argument (so, C-u C-c C-w) to quickly jump to headings by typing in part of them. To make it easy to jump to the main headings in any of my agenda files, I set my org-refile-targets like this:

(setq org-refile-targets '((org-agenda-files . (:maxlevel . 6))))

How can I get better at organizing things with outlines? My writing workflow is a natural place to practise. I’ve accumulated lots of small ideas in my writing file, so if I work on fleshing those out even when I don’t have a lot of energy–breaking things down into points, and organizing several notes into larger chunks–that should help me become more used to outlines.

Tags

In addition to organizing notes in outlines, you can also use tags. Tags go on the ends of headings, like this:

** Heading title     :tag:another-tag:

You can filter headings by tags using M-x org-match-sparse-tree (C-c \) or M-x org-tags-view (C-c a m).

Tags are interesting as a way to search for or filter out combinations. I used tags a lot more before, when I was using them for GTD contexts. I don’t use them as much now, although I’ve started tagging recipes by main ingredient and cooking method. (Hmm, maybe I should try visualizing things as a table…) I also use tags to post entries under WordPress blog categories.

How can I get better at using tags? I can look for things that don’t lend themselves well to outlines, but have several dimensions that I may want to browse or search by. That’s probably going to be recipe management for now. If I figure out a neat way to add tags to my datetree journal notes and then visualize them, that might be cool too.

Links

Org Mode links allow me to refer not only to web pages, files, headings, and text searches, but to things like documentation or even executable code. When I find myself jumping between places a lot, I tend to build links so that I don’t have to remember what to jump to. My evil plans Org Mode file uses links to create and visualize structure, so that’s pretty cool, too. But there’s still a lot more that I could probably do with this.

How can I use links more effectively? I can link to more types of things, such as Lisp code. I can go back over my book notes and fill in the citation graph out of curiosity. Come to think of it, I could do that with my writing as well. My writing ideas rarely fit in neat outlines. I often feel like I’m combining multiple threads, and links could help me see those connections.

In addition to explicit links, I can also define “radio targets” that turn any instance of that text into a hyperlink back to that location. Only seems to work within a single file, though, and I’ve never actually used this feature for something yet.

Properties

You can set various properties for your Org Mode subtrees and then display those properties in columns or filter your subtrees by those properties. I’ve used Effort to keep track of effort estimates and I have some agenda commands that use that. I also use a custom Quantified property to make it easier to clock into tasks using my Quantified Awesome system.

I could track energy level as either tags or properties. Properties allow for easier sorting, I think. Can I define a custom sort order, or do I have to stick with numeric codes? Yeah, I can sort by a custom function, so I can come up with my own thing. Okay. That suggests a way I can learn to use properties more effectively.

There are even more ways to organize Org Mode notes in Emacs (agenda views, exports, etc.), but the ones above look like good things to focus on. So much to try and learn!

Using Org Mode to keep a process journal

I (re)started keeping a journal in Org Mode – chronologically-ordered snippets on what I’m doing, how I’m doing things, and how I’m thinking of improving. I’d mentioned the idea previously on my blog. In this post, I want to share the workflow and configuration that makes it easier for me to log entries.

When I’m on my personal computer, I use Org Mode’s org-capture command to quickly capture notes. I’ve bound org-capture to C-c r, a remnant from the way I still think of it as related to the even older org-remember and remember functions. Anyway, org-capture allows you to define several org-capture-templates, and it will display these templates in a menu so that you can choose from them when creating your note.

Here’s the template I’ve been using for my captures:

(setq org-capture-templates
      '(;; other entries
        ("j" "Journal entry" plain
         (file+datetree+prompt "~/personal/journal.org")
         "%K - %a\n%i\n%?\n")
        ;; other entries
        ))

This stores a link to the currently-clocked task and to whatever context I was looking at when I started the journal entry. It also copies the active region (if any), then positions my cursor after that text. Unlike the default template, this template does not include an Org heading. That way, I don’t have to think of a headline, and I can also just clear the buffer and close the window without adding lots of half-filled-in entries in my journal.

The file+datetree+prompt keyword means that the entries will be stored in ~/personal/journal.org in an outline corresponding to the year, month, and day that I specify. This makes it easy to write an entry for today or for any particular date. For example, I often find myself adding more notes for the previous day (-1) because of something I remembered.

I’m thinking of making Fridays my day for reviewing what I’ve learned and writing up more notes. With the date-tree structure, it’s easy to review my notes by day and look for little things to harvest.

If I know I want to revisit something, I can also add a TODO right in the journal entries using the regular Org syntax or as a keyword that I can search for. If it’s a separate heading (ex: *** TODO Take over the world), I can use org-refile to move it to its proper project.

When I want to flesh out those rough notes into a blog post, I can copy the entry to my blog post outline, fill in the details, and then use org2blog/wp-post-subtree to post it to WordPress. Alternatively, I might edit my rough notes in-place to make them ready to post, and then post them directly from that buffer (possibly changing the heading).

Since I’m not always on my personal computer, I need to be able to pull in notes from elsewhere. I can add quick notes to Evernote on my phone. So far, I’ve been okay with copying snippets manually. If I find that I’m creating lots of notes, though, I might look into reusing the code that I have for building my weekly review list from Evernote notes.

Time-wise, I find that spending 15-30 minutes at the end of the day helps me braindump the key points. If I take little writing breaks throughout the day, that helps me capture more details (especially in terms of development or technical troubleshooting). Re-reading my notes is part of my regular weekly review process, so it’s pretty quick (plus a little more time if I’m writing longer blog posts).

That’s how Org Mode helps me keep a process journal. It’s great to be able to quickly write notes in the same thing you use to do everything else, and to tweak your workflow. Whee!

Publishing WordPress thumbnail images using Emacs and Org2Blog

I often include large images in my blog posts since I use sketches as another way to think out loud. I’d gotten used to using the WordPress web interface to drag and drop them into the relevant section of the page. I write most text in Emacs/Org Mode/Org2Blog because of the better outlining and writing tools, and then I used sacha/org-copy-region-as-html (which you can grab from my Emacs configuration) to copy the HTML markup and paste it into WordPress. Of course, I use Emacs for source-code heavy posts that make the most of its syntax formatting support.

Someone asked me recently about how to post and update blog posts with images through Org2blog, and if I had any recommendations for workflow. I’d dropped Windows Live Writer since it was flaking out on me and the WordPress web interface had improved a lot, but before recommending just using WordPress to add images, I was curious about whether I could improve my blogging workflow by digging into Org Mode and Org2Blog further.

It turns out (like it usually does in the Emacs world) that someone had already solved the problem, and I just didn’t have the updated version. Although the upstream version of Org2Blog didn’t yet have the thumbnail code, searching for “org2blog wordpress thumbnail” led me to cpbotha’s Github issue and pull request. Punchagan’s version had some changes that were a little bit ahead of cpbotha’s, so I dusted off my ancient org2blog repository, cloned it onto my computer, and issued the following commands:

git remote add upstream https://github.com/punchagan/org2blog
git pull upstream master
git remote add cpbotha https://github.com/cpbotha/org2blog.git
git pull cpbotha image-thumbnail

and tested it out on a blog post I’d already drafted in Org. It took me a little while to remember that the file URLs didn’t like ~, so I specified a relative path to the image instead. But then it all worked, yay! A quick git push later, and my Github repository was up to date again.

So now I’m back to running a Git version of org2blog instead of the one that I had installed using the built-in packaging system. The way I make it work is that I have this near the beginning of my Emacs configuration:

;; This sets up the load path so that we can override it
(package-initialize nil)
;; Override the packages with the git version of Org and other packages
(add-to-list 'load-path "~/elisp/org-mode/lisp")
(add-to-list 'load-path "~/elisp/org-mode/contrib/lisp")
(add-to-list 'load-path "~/code/org2blog")
(add-to-list 'load-path "~/Dropbox/2014/presentations/org-reveal")
;; Load the rest of the packages
(package-initialize t)
(setq package-enable-at-startup nil)

This allows me to mostly use the packages and to satisfy dependencies, but override some of the load paths as needed.

Hope that helps someone else!

Summarizing the last meeting dates in Org Contacts

Steffan Heilmann wanted to be able to quickly see the last time he interacted with someone if he tracked interactions in org-contacts. That is, given something like this:

* John Smith
** DONE Conversation
[2014-01-20]
** DONE E-mail
[2014-01-15]
* Jane Smith
** DONE Conversation
[2014-01-07]

… we want to see the latest timestamps for each contact entry.

Here’s the code that I came up with. It scans backward for timestamps or headings. Whenever it finds a timestamp, it compares the timestamp with the one that it has previously stored and keeps the later timestamp. Whenever it encounters a level-1 heading, it sets the property and clears the stored timestamp.

(defun sacha/org-update-with-last-meeting ()
  "Update each level 1 heading with the LASTMEETING property."
  (interactive)
  (goto-char (point-max))
  (let (last-meeting)
    (while (re-search-backward
            (concat "\\(" org-outline-regexp "\\)\\|\\("
                    org-maybe-keyword-time-regexp "\\)") nil t)
      (cond
       ((and (match-string 1)
             (= (nth 1 (save-match-data (org-heading-components))) 1)
             last-meeting)
        ;; heading
        (save-excursion (org-set-property "LASTMEETING" last-meeting))
        (setq last-meeting nil))
       ((and (match-string 2))
        (if (or (null last-meeting) (string< last-meeting (match-string 2)))
            (setq last-meeting (match-string 2))))))))

Scanning backwards works well here because that makes it easy to add information to the top-level heading we’re interested in. If we scanned it the other way around (say, with org-map-entries), we might need to backtrack in order to set the property on the top-level heading.

The result is something like this:

* John Smith
  :PROPERTIES:
  :LASTMEETING: [2014-01-20]
  :END:
** DONE E-mail
[2014-01-15]
** DONE Conversation
[2014-01-20]
* Someone without a meeting
* Jane Smith
  :PROPERTIES:
  :LASTMEETING: [2014-01-07]
  :END:
** DONE Conversation
[2014-01-07]

You can then use something like:

#+COLUMNS: %25ITEM %LASTMEETING %TAGS %PRIORITY %TODO
#+BEGIN: columnview :maxlevel 1
| ITEM                        | LASTMEETING  | TAGS | PRIORITY | TODO |
|-----------------------------+--------------+------+----------+------|
| * John Smith                | [2014-01-20] |      |          |      |
| * Someone without a meeting |              |      |          |      |
| * Jane Smith                | <2014-01-07> |      |          |      |
#+END:

… or even use M-x org-sort to sort the entries by the LASTMEETING property (R will reverse-sort by property).