Category Archives: emacs

Emacs Hangout 2015-03-18 show notes

We chatted about Org coolness and workflows; learning Emacs and deliberate practice; speech recognition; alternate layouts.

Event page: https://plus.google.com/events/cbj3rg26d8j9ifaiff5uq00ncr4

What’s this Emacs Hangout thing about? This is an informal way for Emacs geeks to get together and swap tips/notes/questions. You can find the previous Hangouts or sign up for the mailing list at http://sachachua.com/blog/tag/emacs-hangout/ . The next Emacs Hangout will be on April 15, 2015, at 8 PM Toronto time (12 midnight GMT): https://plus.google.com/events/c3igul0rj6cfc1qbcar91lrcj0s Want to get notified about upcoming hangouts? You can sign up for notifications at http://eepurl.com/bbi-Ir .

Experimenting with org-timer. Times are approximate, despite the seeming precision. =)

  • 0:00:12 Intros
  • 0:04:22 Howard – Prettify Your Quotation Marks
  • 0:06:37 Org Mode Spreadsheets
  • 0:12:44 org-aggregate
  • 0:12:50 summing up tables
  • 0:15:38 Hiding things from export with :PRIVATE: .. :END:
  • 0:16:30 Navigation
  • 0:17:10 Relative line numbers
  • 0:19:32 Org prettiness: org-bullets
  • 0:20:51 EMagicians Starter Kit
  • 0:26:23 More table stuff
  • 0:27:32 Org Mode workflows, reasons to use Org
  • 0:28:16 Org to replace Markdown, executable code samples, etc. Literate programming. Attachments.
  • 0:29:34 Org tables
  • 0:30:11 Literate programming, technical notes
  • 0:30:49 Links
  • 0:31:54 Keeping all your notes, reference information, and so forth
  • 0:32:25 As a simple database
  • 0:34:59 Leo’s demo: Org task management, scheduling, linking, refiling, org-pomodoro (with sound and tweaks) and time reporting
  • 0:44:47 clock reports, clocktable
  • 0:46:09 C-u org-refile
  • 0:47:53 tagging strategies (prefixes, @ for contexts, etc.), search
  • 0:49:45 using org-drill
  • 0:51:19 scheduling vs using agendas
  • 0:51:34 org-habits for visualizing consistency
  • 0:54:34 erc and other IRC clients
  • 0:55:20 devices
  • 0:56:47 Other apps like Wunderlist – tasks to synchronize; work/home separation
  • 0:59:44 org-trello – close, but no cigar
  • 1:00:26 orgzly
  • 1:01:44 MobileOrg =( hasn’t been updated in a long time, etc.
  • 1:03:15 org-timer, meeting notes, colouring neato
  • 1:05:37 spacemacs as a way for Vi converts to try Emacs out
  • 1:06:47 keybindings – rebind lots, or not? a few frequently used ones? ErgoEmacs?
  • 1:10:23 hydra, key-chord, prefixes, see Sacha’s config for making key-chord more specific
  • 1:12:52 pair-programming? tmux and different modes?
  • 1:13:09 httpd, impatient-mode, multi-tty?
  • 1:18:04 deliberate practice and learning new things
  • 1:18:41 putting new things in a temporary section (1 month)
  • 1:19:02 reading blog posts (ex: planet.emacsen.org), reading configs, doc.norang.ca/org-mode.html
  • 1:21:29 cheat sheet
  • 1:22:14 another vote for a staging area (experimental.el)
  • 1:24:35 blogging is awesome, org2blog, etc.
  • 1:28:59 edebug is a great way to explore how things work
  • 1:31:39 watching over people’s shoulders when it comes to Emacs; Hangouts, virtual conference? Moderators are helpful. Handing a virtual microphone out – moderator names someone who can then check in, ask the question, etc. monitoring comments. (Could use help reaching out to speakers, organizing schedule, etc.)
  • 1:34:55 Speech interface: less navigation, just commands. Better speech integration – requires C, or can be done with Emacs Lisp? What you’re operating on is different. Intent (what has to happen) vs action (what to do), audio wizards with contextual grammar. Pretty far out still. Translating dictation-friendly names to code-friendly names and vice versa.
  • 1:42:02 Quick wins: accelerators, focusing on intent
  • 1:44:19 Context
  • 1:46:03 LaTeX for math notes? Export to LaTeX or HTML with images, inline images. LyX if you need WYSIWYG? Snippets can help you work at a higher level (instead of memorizing a lot)
  • 1:48:52 Alternative keyboard layouts and keybindings; also, typing less (abbreviations, expansions, thinking, etc.), Kinesis
  • 1:53:57 Speech recognition. Saying things again instead of editing.
  • 1:58:06 Suggestions – more Org workflows, chatting leading up to the next Hangout, asking for help

Text chat:

Philip Stark 8:04 PM This one?
Cameron Desautels 8:04 PM yup
Howard Abrams 8:04 PM Yes!
Cameron Desautels 8:04 PM If you guys haven’t seen this, we should all try to contribute and help out: https://github.com/sachac/emacsconf2015/blob/gh-pages/index.org
me 8:08 PM (Dan: I muted you because of background noise, but you can always unmute yourself when you’re ready to ask a question )
Jonathan Hill 8:10 PM does anybody know how to full screen the hang out? I guess I could just full-screen chrome on the mac..
me 8:10 PM If you reduce your font size a lot, the main screen gets bigger, but let me just ask Howard to increase his text scale
Leo Ufimtsev 8:11 PM Btw, I came across an aggregator module: https://github.com/tbanel/orgaggregate allows sql-like aggregation for tables
Jonathan Hill 8:14 PM Howard, I’ve got a sidebar question about how you’re jumping around with the cursor seemingly arbitrarily when you can find a convenient spot to answer.
me 8:15 PM He might not see the chat while screensharing, so maybe you can ask out loud when there’s a gap
Cameron Desautels 8:15 PM Could be the mouse Though it took me a minute to realize that.
me 8:16 PM By the way, the nifty bullets on the headings are probably because of org-bullets.el, if you like that effect
Jonathan Hill 8:16 PM his modeline is gorgeous
Cameron Desautels 8:16 PM That’s likely powerline. https://github.com/milkypostman/powerline (Busted!)
me 8:17 PM <laugh> Ooh, relative line numbers
Cameron Desautels 8:19 PM I think I have some bad lag. Sorry about that. My ISP is terrible.
jason Peak 8:22 PM hi everyone, sorry I’m late
Leo Ufimtsev 8:23 PM I don’t think I’ll have time to prepare org-aggregator in this hangout. I can show a org-dot-emacs setup thou. Or some stuff on time-reporting
me 8:23 PM No worries!
jason Peak 8:24 PM yep
Philip Stark 8:26 PM how do you increase the font quickly?
Cameron Desautels 8:26 PM C-x C-=
Philip Stark 8:26 PM and decrease would be C-x C–?
Cameron Desautels 8:27 PM (text-scale-adjust) yep!
Philip Stark 8:27 PM great, thanks.
Will Monroe 8:28 PM I spend most of my time in emacs in org-mode FWIW
Leo Ufimtsev 8:30 PM I’m on twice. My hangouts crashed on my phone, (black screen). @Sacha would you be able to kick out my 2nd instance? (the one with black screen, not the one with picture)
Leo Ufimtsev 8:31 PM thanks
jason Peak 8:31 PM Complete beginner – I like the idea of using org as an organically developing a database of My stuff
Cameron Desautels 8:32 PM Will: that’s interesting. Do you want to speak about your uses? You might have a unique perspective.
Jonathan Hill 8:32 PM WOW That’s absurdly powerful, S
Will Monroe 8:33 PM Cameron: I’d be glad to. Very much a beginner though! ; )
Cameron Desautels 8:33 PM There are other beginners here—it’s probably exactly what they’d need!
Philip Stark 8:34 PM yeah
Leo Ufimtsev 8:35 PM I could show some things if folks are interested.
Howard Abrams 8:40 PM Can’t believe that Chrome crashed and I lost the chat history… I’ll try to catch up.
Philip Stark 8:41 PM http://0paste.com/7356-3b5998af chat history so far.
Will Monroe 8:45 PM Is that ticking?
Howard Abrams 8:46 PM The problem with demonstrating org-mode is not revealing too much about ourselves. Good thing we’re among friends, eh?
Cameron Desautels 8:46 PM haha, indeed
Philip Stark 8:46 PM indeed. and on youtube :S
Cameron Desautels 8:46 PM Just don’t read the comments. :p
Will Monroe 8:46 PM Ha!
jason Peak 8:47 PM how many tags is too much ?
me 8:47 PM Heh. Should I redact this text chat segment from the show notes?
jason Peak 8:47 PM how many tags are too many ?
Will Monroe 8:48 PM That’s a great question.
jason Peak 8:48 PM loaded question; just curious about others no mic, Sacha thanks, Will!
Jonathan Hill 8:49 PM someone has a lot of static on their mic
me 8:50 PM That’s Leo’s background noise, oh well.
Philip Stark 8:50 PM That’s Leo, I think. he’s probably on a phone.
me 8:50 PM <laugh!>
jason Peak 8:51 PM I’ve wondered about namespacing tags; org does not support namespaces as part of its tag data structure, though? Maybe I need to make it ?!!
me 8:51 PM I recognize that technique
jason Peak 8:51 PM thank you! yes
me 8:51 PM jason: You can probably combine a tag with other search filters, like the category (file)
Cameron Desautels 8:52 PM org-habit
jason Peak 8:53 PM sacha: thank you, I will exploreit
me 8:53 PM
Cameron Desautels 8:54 PM His setup minds me of mine from a couple years back…these days I use org-mode less because I want my tasks and calendar notifications on all of my devices. *reminds me
Will Monroe 8:54 PM I would love to hear about simple setups for erc
Philip Stark 8:55 PM What is erc?
Will Monroe 8:55 PM or any other emacs irc client
Howard Abrams 8:56 PM I like circe.
Jonathan Hill 8:58 PM Cameron, what about a cloud-based sort or read-only solution like syncing your stuff on dropbox or google drive or something? s/sort or/sort of/
Cameron Desautels 8:59 PM Yeah, I’ve actually tried that sort of thing, exporting an HTML version of my list into Dropbox so I could read it on any of my devices.
Howard Abrams 8:59 PM http://goo.gl/uWjFye
Cameron Desautels 8:59 PM But I’d really like r/w access on multiple platforms.
Cameron Desautels 9:00 PM And more ability to filter on the go.
Leo Ufimtsev 9:01 PM Sorry about vacum cleaning noise >_<
jason Peak 9:01 PM trello is great! not sure about the integration
Cameron Desautels 9:02 PM http://www.orgzly.com/
Jonathan Hill 9:02 PM There’s MobileOrg in the App Store for IOS
Leo Ufimtsev 9:04 PM I was wondering about that. Nifty.
jason Peak 9:04 PM very nice!
Jonathan Hill 9:04 PM does org-calendar interface with Exchange calendar with a tolerable level of setup pain?
Will Monroe 9:05 PM Jonathan: someone recently released an interface for Exchange.
Cameron Desautels 9:05 PM https://github.com/syl20bnr/spacemacs
Will Monroe 9:06 PM It was announced on the listserv…can’t recall the name. But it was within the last few weeks.
Leo Ufimtsev 9:09 PM I use a different keyboard layout “Colemak” (like modern Dvorak). I found emacs was more friendly for non-standard keyboard layouts like colemak.
jason Peak 9:09 PM I’m back, but only to say goodbye for now; Sacha, everyone, thanks for all you’ve shared; I look forward to being back here soon…
Leo Ufimtsev 9:09 PM take care
Will Monroe 9:09 PM bye jason!
Will Monroe 9:14 PM Leo: thanks for mentioning that. I’ve wondered about other keyboard layouts being more /less appropriate for emacs too.
Howard Abrams 9:17 PM I need to run, but I wanted to point out this gist that I wrote today … https://gist.github.com/howardabrams/67d60458858f407a13bd
me 9:19 PM (I use Dvorak, and haven’t remapped extensively – I mostly keep things the same, although my key-chords are Dvorak-friendly)
Leo Ufimtsev 9:20 PM http://doc.norang.ca/org-mode.html http://emacs.stackexchange.com/ <<< very good place for asking for help http://www.emacswiki.org/emacs/OrgDotemacs << organise your .emacs in orgmode
Cameron Desautels 9:26 PM https://gist.github.com/camdez/c0f4c53ee847d5d1e2f0 ^ That’s how I add my experiments. Nothing fancy, but it gets the job done. If you use org-mode you could use capture targets for something similar.
Will Monroe 9:29 PM Thanks, Cameron! That’s a great idea. I like the idea of keeping new and potentially unstable functions in a separate file…and the idea of commenting on them could lead to a nice diary of sorts.
Cameron Desautels 9:30 PM Exactly. Plus I don’t commit this so I don’t have to be embarrassed about pushing gross code to my public config.
me 9:30 PM Cameron: Hah! That’s a great idea. Although maybe I’ll keep the weird area in my config, just because
Cameron Desautels 9:31 PM Oh, and I load it like this so someone can pull down my config and have it still work whether or not the file exists: (load camdez/experiments-file ‘no-error)
Will Monroe 9:34 PM Thanks!
Jonathan Hill 9:35 PM @cameron I didn’t know about ‘no-error. neat trick
Leo Ufimtsev 9:35 PM Use trello lol
Cameron Desautels 9:39 PM Jonathan: really that argument can be anything non-nil, I just use that symbol name so it’s self-explanatory.
Jonathan Hill 9:39 PM Thinking Miserable? was that the phrase?
Cameron Desautels 9:40 PM ¯\_(ツ)_/¯ I’m not entirely sure how we got here.
me 9:41 PM It’s a thread from near the beginning of the Hangout in a Q&A question (Eric said he might have something to demo), so it’s not quite connected to the conference conversation
Leo Ufimtsev 9:41 PM Is the voice recognition business cross-platform or specific to windows/mac/linux?
me 9:41 PM (Although we can continue that in the backchannel if people have ideas/suggestions)
Cameron Desautels 9:42 PM Ah! Ok. I must have missed the thread.
Eric S. Johansson 9:42 PM https://docs.google.com/drawings/d/1M-TzfRaSaAhFXQk1OmcmHNOaW31_7W_7q0bf8CAJqSw/edit
Leo Ufimtsev 9:46 PM Btw, does anyone use Emacs with Latex.>? for math notes..?
Will Monroe 9:50 PM Sacha (or anyone): this is a much earlier comment you made, but I’d be interested in hearing about your experience using the “standard” Emacs keybindings with Dvorak some time. Maybe during the next Emacs Meetup!
Jonathan Hill 9:54 PM /me E I gather that’s likely not what you actually said. I seem to have gotten stuck on it, though
Will Monroe 9:56 PM Thanks for the advice, everyone. This was the first time I’ve been able to interact with so many Emacs users. Hope to return next month and perhaps again at the possible conference. All the best!
Cameron Desautels 9:57 PM Take care, Will. Nice to meet you.
me 9:57 PM See you!
Leo Ufimtsev 9:57 PM See you next time :-=
Will Monroe 9:57 PM You too! Mabye next time I’ll have something to share.
Jonathan Hill 9:58 PM /me C let me get your info before we go if you don’t mind oops

Emacs Org Mode and the power of plain text

When I build a tool for other people to use and I want to store data, I usually have to think in terms of relational databases: tables, fields, and queries. There are other kinds of databases out there, like ones with flexible documents or ones that are optimized for graphs, but I haven’t gotten the hang of them yet.

When I build a tool for myself and I want to store data, I usually use plain text. (Or maybe a spreadsheet, but now that I’m getting the hang of Org Mode tables, I’m leaning more and more towards text.)

2015-02-02 Plain text -- index card #data #organization #pkm #org

2015-02-02 Plain text – index card #data #organization #pkm #org

I like the flexibility of plain text. Sometimes I want to organize my thoughts in an outline or an index. Sometimes I want to make a graph, like the way I wanted to visualize how my goals are related to each other. Sometimes I change my mind about what I want. (All the time, actually. =) ) Plain text lets me add structure the way I want to. It’s all in my text editor, so I can move things around or reorganize things using the tools in Emacs.

Sure, sometimes I mess up because of formatting mistakes or the lack of validation. For example, typos in my personal ledger show up when the numbers don’t match my bank balances or there’s a new category with a misspelled name. But these are easy enough to catch and fix, and I can’t completely guard against them with a database anyway. And it’s nice to know that version control can let me visually step through the changes or recover from mistakes.

What about speed? Databases can be much faster than plain text for large quantities of data, for sure. I tend to work with pretty small quantities of data. For example, my blog index has 3257 lines, and the file that I’m drafting this in is under a megabyte. Even with whatever Emacs Lisp I’ve written to extract or cross-reference data, I’m still mostly bottlenecked by my brain instead of my computer. Sure, it took me a little longer to figure out how to do table calculations using Org Mode, but now that I have some notes on that, I should be able to come up with future calculations more easily. Besides, if I need to analyze things quickly, I can export and then crunch the numbers using a different tool.

Speaking of tools, staying with lightly-structured plain text lets me build a toolkit of text manipulation techniques. When I’m editing things in Emacs, scripting with Emacs Lisp, searching with grep, or writing Javascript/Ruby/Perl code to work with text, I’m developing skills that I can use in a wide range of situations.

If you’re interested in keeping your data in plain text with Org Mode, here are some tips that can help you learn how to work with your information.

Start with tables

  1. Learn how to use keyboard shortcuts to create, move, or delete rows and columns.
  2. Learn how to sort tables.
  3. Learn how to use the column with specifier (ex: <10>) to limit the displayed size of your column while still being able to add more information.
  4. Use Org Mode’s support for calculations to do math or perform other operations on your table.

Consider using properties

Org tables don’t do well with paragraphs or more complex information, so you might want to use Org subtrees with properties.

You can use Org columns to display property values, or use Org dynamic blocks to put a summary of the values into your Org Mode file. See org-collector.el for a propview report.

If you want more control, you can work with the information using Emacs Lisp. You can use org-entry-get, org-entry-get-multivalued-property, or org-entry-get-with-inheritance to get the value of the property. If you want to go through all the subtrees (or a subset of them), use org-map-entries to call your own function at each of the matching headlines in the scope. org-heading-components will give you the information from the current heading, and you can use org-end-of-subtree to give you the boundary of the subtree if you want to process it further.

You can parse Org Mode lists with org-list-struct. I haven’t dug into this deeply yet, but it looks interesting.

Parse free-form text

In addition to working with tables and properties, you can write functions that use regular expressions or other techniques to extract data from text. re-builder can be useful for visual feedback while you’re figuring out the right regular expression to build. Remember, you’re in Emacs, so you don’t have to come up with the perfect regular expression that extracts all the data in one go. You can search for a regular expression, use a command like forward-line, save something to a variable, and so forth. Try thinking about how you would do something by hand, and then using repeat-complex-command to see what functions Emacs called when you did that.

save-restriction, narrow-to-region, and save-excursion are very useful when it comes to limiting the scope of your processing or saving your position, so check them out in the Emacs Lisp manual.

I find plain text to be really useful when I’m figuring things out (so, all the time), since I don’t have to build a complex interface for working with it. As I learn more about Org Mode’s features, I find myself using it for more and more of my data. Org’s slogan is “Organize your life in plain text!” – and I think it just might be serious about that!

Getting Helm and org-refile to clock in or create tasks

I’ve been thinking about how to improve the way that I navigate to, clock in, and create tasks in Org Mode. If the task is one of the ones I’ve planned for today, I use my Org agenda. If I know that the task exists, I use C-u C-c C-w (org-refile) to jump to it, and then ! (one of my org-speed-commands-user options) to clock in and track it on Quantified Awesome. If I want to resume an interrupted task, I use C-u C-c j (my shortcut for org-clock-goto). For new tasks, I go to the appropriate project entry and create it, although I really should be using org-capture instead.

2015-01-30 Org Mode jumping to tasks -- index card #emacs #org

2015-01-30 Org Mode jumping to tasks – index card #emacs #org

I thought about how I can reduce some of these distinctions. For example, what if it didn’t matter whether or not a task already exists? I can modify the org-refile interface to make it easier for me to create tasks if my description doesn’t match anything. To make things simpler, I’ll just reuse one of my org-capture-templates, and I’ll pre-fill it with the candidate from Helm.

(ert-deftest sacha/org-capture-prefill-template ()
  (should
   ;; It should fill things in one field at a time
   (string=
    (sacha/org-capture-prefill-template
     "* TODO %^{Task}\nSCHEDULED: %^t\n:PROPERTIES:\n:Effort: %^{effort|1:00|0:05|0:15|0:30|2:00|4:00}\n:END:\n%?\n"
     "Hello World")
    "* TODO Hello World\nSCHEDULED: %^t\n:PROPERTIES:\n:Effort: %^{effort|1:00|0:05|0:15|0:30|2:00|4:00}\n:END:\n%?\n"
    ))
  (should
   (string=
    (sacha/org-capture-prefill-template
     "* TODO %^{Task}\nSCHEDULED: %^t\n:PROPERTIES:\n:Effort: %^{effort|1:00|0:05|0:15|0:30|2:00|4:00}\n:END:\n%?\n"
     "Hello World" "<2015-01-01>")
    "* TODO Hello World\nSCHEDULED: <2015-01-01>\n:PROPERTIES:\n:Effort: %^{effort|1:00|0:05|0:15|0:30|2:00|4:00}\n:END:\n%?\n"))
  (should
   (string=
    (sacha/org-capture-prefill-template
     "* TODO %^{Task}\nSCHEDULED: %^t\n:PROPERTIES:\n:Effort: %^{effort|1:00|0:05|0:15|0:30|2:00|4:00}\n:END:\n%?\n"
     "Hello World" "<2015-01-01>" "0:05")
    "* TODO Hello World\nSCHEDULED: <2015-01-01>\n:PROPERTIES:\n:Effort: 0:05\n:END:\n%?\n")))

(defun sacha/org-capture-prefill-template (template &rest values)
  "Pre-fill TEMPLATE with VALUES."
  (setq template (or template (org-capture-get :template)))
  (with-temp-buffer
    (insert template)
    (goto-char (point-min))
    (while (re-search-forward
            (concat "%\\("
                    "\\[\\(.+\\)\\]\\|"
                    "<\\([^>\n]+\\)>\\|"
                    "\\([tTuUaliAcxkKInfF]\\)\\|"
                    "\\(:[-a-zA-Z]+\\)\\|"
                    "\\^\\({\\([^}]*\\)}\\)"
                    "?\\([gGtTuUCLp]\\)?\\|"
                    "%\\\\\\([1-9][0-9]*\\)"
                    "\\)") nil t)
      (if (car values)
          (replace-match (car values) nil t))
      (setq values (cdr values)))
    (buffer-string)))

(defun sacha/helm-org-create-task (candidate)
  (let ((entry (org-capture-select-template "T")))
    (org-capture-set-plist entry)
    (org-capture-get-template)
    (org-capture-set-target-location)
    (condition-case error
        (progn
          (org-capture-put
           :template
           (org-capture-fill-template
            (sacha/org-capture-prefill-template (org-capture-get :template)
                                                candidate)))
          (org-capture-place-template
           (equal (car (org-capture-get :target)) 'function)))
      ((error quit)
       (if (get-buffer "*Capture*") (kill-buffer "*Capture*"))
       (error "Capture abort: %s" error)))) t)

Next, I want to add this to the way that Helm prompts me to refile. That means that my creation task should return something ready for org-refile. Actually, maybe I don’t have to do that if I know I’m always going to call it when I want to jump to something. I might as well add that bit of code that sets up clocking in, too.

(defvar sacha/helm-org-refile-locations nil)

(defun sacha/helm-org-clock-in-and-track-from-refile (candidate)
  (let ((location (org-refile--get-location candidate sacha/helm-org-refile-locations)))
    (save-window-excursion
      (org-refile 4 nil location)
      (sacha/org-clock-in-and-track)
      t)))

(defun sacha/helm-org-refile-read-location (tbl)
  (setq sacha/helm-org-refile-locations tbl)
  (helm
   (list
    (helm-build-sync-source "Refile targets"
      :candidates (mapcar 'car tbl)
      :action '(("Select" . identity)
                ("Clock in and track" . sacha/helm-org-clock-in-and-track-from-refile))
      :history 'org-refile-history)
    (helm-build-dummy-source "Create task"
      :action (helm-make-actions
               "Create task"
               'sacha/helm-org-create-task)))))

(defun sacha/org-refile-get-location (&optional prompt default-buffer new-nodes no-exclude)
  "Prompt the user for a refile location, using PROMPT.
PROMPT should not be suffixed with a colon and a space, because
this function appends the default value from
`org-refile-history' automatically, if that is not empty.
When NO-EXCLUDE is set, do not exclude headlines in the current subtree,
this is used for the GOTO interface."
  (let ((org-refile-targets org-refile-targets)
        (org-refile-use-outline-path org-refile-use-outline-path)
        excluded-entries)
    (when (and (derived-mode-p 'org-mode)
               (not org-refile-use-cache)
               (not no-exclude))
      (org-map-tree
       (lambda()
         (setq excluded-entries
               (append excluded-entries (list (org-get-heading t t)))))))
    (setq org-refile-target-table
          (org-refile-get-targets default-buffer excluded-entries)))
  (unless org-refile-target-table
    (user-error "No refile targets"))
  (let* ((cbuf (current-buffer))
         (partial-completion-mode nil)
         (cfn (buffer-file-name (buffer-base-buffer cbuf)))
         (cfunc (if (and org-refile-use-outline-path
                         org-outline-path-complete-in-steps)
                    'org-olpath-completing-read
                  'org-icompleting-read))
         (extra (if org-refile-use-outline-path "/" ""))
         (cbnex (concat (buffer-name) extra))
         (filename (and cfn (expand-file-name cfn)))
         (tbl (mapcar
               (lambda (x)
                 (if (and (not (member org-refile-use-outline-path
                                       '(file full-file-path)))
                          (not (equal filename (nth 1 x))))
                     (cons (concat (car x) extra " ("
                                   (file-name-nondirectory (nth 1 x)) ")")
                           (cdr x))
                   (cons (concat (car x) extra) (cdr x))))
               org-refile-target-table))
         (completion-ignore-case t)
         cdef
         (prompt (concat prompt
                         (or (and (car org-refile-history)
                                  (concat " (default " (car org-refile-history) ")"))
                             (and (assoc cbnex tbl) (setq cdef cbnex)
                                  (concat " (default " cbnex ")"))) ": "))
         pa answ parent-target child parent old-hist)
    (setq old-hist org-refile-history)
    ;; Use Helm's sources instead
    (setq answ (sacha/helm-org-refile-read-location tbl))
    (if (and (stringp answ)
             (setq pa (org-refile--get-location answ tbl)))
        (progn
          (org-refile-check-position pa)
          (when (or (not org-refile-history)
                    (not (eq old-hist org-refile-history))
                    (not (equal (car pa) (car org-refile-history))))
            (setq org-refile-history
                  (cons (car pa) (if (assoc (car org-refile-history) tbl)
                                     org-refile-history
                                   (cdr org-refile-history))))
            (if (equal (car org-refile-history) (nth 1 org-refile-history))
                (pop org-refile-history)))
          pa)
      (if (and (stringp answ) (string-match "\\`\\(.*\\)/\\([^/]+\\)\\'" answ))
          (progn
            (setq parent (match-string 1 answ)
                  child (match-string 2 answ))
            (setq parent-target (org-refile--get-location parent tbl))
            (when (and parent-target
                       (or (eq new-nodes t)
                           (and (eq new-nodes 'confirm)
                                (y-or-n-p (format "Create new node \"%s\"? "
                                                  child)))))
              (org-refile-new-child parent-target child)))
        (if (not (equal answ t)) (user-error "Invalid target location"))))))

(fset 'org-refile-get-location 'sacha/org-refile-get-location)

Hooray! Now C-u C-c C-w (org-refile) also lets me use TAB or F2 to select the alternative action of quickly clocking in on a task. Mwahaha.

You can check out this code in my config to see if anything has been updated. Want to learn more about modifying Helm? Check out these posts by John Kitchin and Rubikitch.

I think I’m getting the hang of tweaking Helm.  Yay!

Using Emacs to prepare files for external applications like Autodesk Sketchbook Pro

To make it easier to draw using Autodesk Sketchbook Pro on my laptop (a Lenovo X220 tablet PC), I’ve created several templates with consistent dot grids and sizes. Since I want to minimize typing when I’m drawing, I wrote a couple of functions to make it easier to copy these templates and set up appropriately-named files. That way, I can save them without the grid layer, flip between files using Sketchbook Pro’s next/previous file commands, and then process them all when I’m ready.

Index cards

I’ve been experimenting with a habit of drawing at least five index cards every day. Here’s a function that creates five index cards (or a specified number of them) and then opens the last one for me to edit.

(defvar sacha/autodesk-sketchbook-executable "C:/Program Files/Autodesk/SketchBook Pro 7/SketchBookPro.exe")
(defun sacha/prepare-index-cards (n)
  (interactive (list (or current-prefix-arg 5)))
  (let ((counter 1)
        (directory "~/Dropbox/Inbox")
        (template "c:/data/drawing-templates/custom/0 - index.tif")
        (date (org-read-date nil nil "."))
        temp-file)
    (while (> n 0)
      (setq temp-file
            (expand-file-name (format "%s-%d.tif" date counter)
                              directory))
      (unless (file-exists-p temp-file)
        (copy-file template temp-file)
        (setq n (1- n))
        (if (= n 0)
            (shell-command
             (concat (shell-quote-argument sacha/autodesk-sketchbook-executable)
                     " "
                     (shell-quote-argument temp-file) " &"))))
      (setq counter (1+ counter)))))

Afterwards, I call sacha/rename-scanned-cards function to convert the TIFFs to PNGs, display the files and ask me to rename them properly.

Rename scanned index cards

(defun sacha/rename-scanned-cards ()
  "Display and rename the scanned files."
  (interactive)
  (when (directory-files "~/Dropbox/Inbox" t "^[0-9]+-[0-9]+-[0-9]+-.*.tif")
    ;; Convert the TIFFs first
    (apply 'call-process "mogrify" nil nil nil "-format" "png" "-quality" "1"
           (directory-files "~/Dropbox/Inbox" t "^[0-9]+-[0-9]+-[0-9]+-.*.tif"))
    (mapc (lambda (x)
            (rename-file x "~/Dropbox/Inbox/backup"))
          (directory-files "~/Dropbox/Inbox" t "^[0-9]+-[0-9]+-[0-9]+-.*.tif")))
  (mapc (lambda (filename)
          (find-file filename)
          (delete-other-windows)
          (when (string-match "/\\([0-9]+-[0-9]+-[0-9]+\\)" filename)
            (let ((kill-buffer-query-functions nil)
                  (new-name (read-string "New name: "
                                         (concat (match-string 1 filename) " "))))
              (when (> (length new-name) 0)
                (revert-buffer t t)
                (rename-file filename (concat new-name ".png"))
                (kill-buffer)))))
        (directory-files "~/Dropbox/Inbox" t "^[0-9]+-[0-9]+-[0-9]+-.*.png")))

I might tweak the files a little more after I rename them, so I don’t automatically upload them. When I’m happy with the files, I use a Node script to upload the files to Flickr, move them to my To blog directory, and copy Org-formatted text that I can paste into my learning outline.

Automatically resize images

The image+ package is handy for displaying the images so that they’re scaled to the window size.

(use-package image+
 :load-path "~/elisp/Emacs-imagex"
 :init (progn (imagex-global-sticky-mode) (imagex-auto-adjust-mode)))

Get information for sketched books

For sketchnotes of books, I set up the filename based on properties in my Org Mode tree for that book.

(defun sacha/prepare-sketchnote-file ()
  (interactive)
  (let* ((base-name (org-entry-get-with-inheritance  "BASENAME"))
         (filename (expand-file-name (concat base-name ".tif") "~/dropbox/inbox/")))
    (unless base-name (error "Missing basename property"))
    (if (file-exists-p filename)
        (error "File already exists")
        (copy-file "g:/drawing-templates/custom/0 - base.tif" filename))
      (shell-command (concat (shell-quote-argument sacha/autodesk-sketchbook-executable)
                             (shell-quote-argument filename) " &"))))

By using Emacs Lisp functions to set up files that I’m going to use in an external application, I minimize fussing about with the keyboard while still being able to take advantage of structured information.

Do you work with external applications? Where does it make sense to use Emacs Lisp to make setup or processing easier?

Learn how to take notes more efficiently in Org Mode

How do you take notes in Org? Are you buried in a heap of uncategorized notes? Do you manually open the right file and navigate to the right heading? Are you mystified by org-capture and org-refile? Here’s a path that can help you learn how to more efficiently take (and organize!) notes in Org Mode.

  1. Set up a keyboard shortcut to go to your main Org file
  2. Use org-refile to file or jump to headings
  3. Use org-capture to write notes quickly
  4. Define your own org-capture templates for greater convenience
  5. Pull in additional information

Step 1. Set up a keyboard shortcut to go to your main Org file

Instead of using C-x C-f (find-file) all the time, set up shortcuts to jump to the Org files you use the most. This way, you can easily type that keyboard shortcut, go to the end of the file, and add your note. Here’s some sample code that sets the C-c o shortcut to open organizer.org in your home directory. You can add it to your ~/.emacs.d/init.el and then call M-x eval-buffer to load the changes.

(global-set-key (kbd "C-c o") 
                (lambda () (interactive) (find-file "~/organizer.org")))

Alternatively, you can use registers, which are Emacs data structures that can hold text, file references, and more. The following code sets the o register to organizer.org in your home directory:

(set-register ?o (cons 'file "~/organizer.org"))

You can then jump to it with C-x r j (jump-to-register), specifying o at the prompt.

Once you’re in your Org file, you can use M-> (end-of-buffer) to go to the end of the file, or you can use C-s (isearch-forward) to search for some text.

You’ll still need to switch back to your original buffer or window configuration when you’re done, but that’s something you can fix when you learn how to use org-capture.

Step 2. Use org-refile to file or jump to headings

The next improvement is to use org-refile to move the current subtree to a specified heading, or jump to one without moving any text. This will let you quickly go to a project or task from anywhere in Org Mode.

By default, org-refile will show you only the top-level headings of the current file. Let’s configure it to show you headings from all your agenda files. You can use M-x customize-variable to change org-refile-targets. Click on the INS button, then click on Value menu next to Identify target headline by. Change this to Max level number. In the Integer field, fill in a suitably high number, like 6. This is the maximum depth of headings that will be shown.

If you prefer to set your variables using Emacs Lisp, here’s the code that you can add it to your ~/.emacs.d/init.el. Call M-x eval-buffer to load the changes.

(setq org-refile-targets '((org-agenda-files . (:maxlevel . 6))))

Be sure to add your main Org Mode file to your agenda list. You can do so by going to the file and typing C-c [ (org-agenda-file-to-front), or by setting the org-agenda-files variable.

The standard Emacs completion interface isn’t as friendly as it could be. I use the Helm package to make it easier to select and complete input. Since Helm can be a little complex, you may want to start with ido-mode instead. Here’s how you can set Ido up to use with Org Mode:

(ido-mode)
(setq org-completion-use-ido t)

Once you’ve set up your org-refile-targets, your agenda files, and either Helm or Ido, you can get the hang of using org-refile. The standard keyboard shortcut for org-refile is C-c C-w when you’re in an Org Mode buffer. org-refile can do different things depending on how you call it:

  • By default, it moves the current subtree to the specified location.
  • If you call it with the prefix argument C-u (like so: C-u C-c C-w), it jumps to the specified location instead of moving the current subtree.
  • If you call it as C-u C-u C-c C-w, it jumps to the previous refiling location.

First, practise using it with the prefix argument (C-u C-c C-w) to jump to a location. Once you’ve gotten the hang of that, go to some of your uncategorized entries and use org-refile without the prefix argument (just C-c C-w) to move the entries to the right place.

org-refile gives you a quick way to jump to a heading, but you still have to find your way back to whatever you were working on before you wanted to take a note. After you’re comfortable with refiling notes to the right place, move on to learning how to use org-capture to quickly take notes from anywhere.

Step 3. Use org-capture to write notes quickly

org-capture can help you take notes quickly by popping up a window or leading you through prompts. When you’re done taking the note, it will return you to whatever you were looking at before you started. In order to take advantage of this, though, you’ll need to customize org-capture.

The Org Mode manual recommends giving org-capture a global keyboard shortcut such as C-c c.

(global-set-key (kbd "C-c c") 'org-capture)

You can use M-x customize-variable to set org-default-notes-file to the filename you would like notes to be saved to, or set it in Emacs Lisp code like this:

(setq org-default-notes-file "~/organizer.org")

Make sure that the file exists and is automatically opened in Org Mode.

If you type C-c c, org-capture will display a prompt. t is a simple task template, and C will show you the customization interface for org-capture-templates.

Let’s start with t. It will show you a buffer with a simple TODO entry. You can fill in the rest of the details, use C-c C-s (org-schedule) to schedule it for a particular day, set the deadline with C-c C-d (org-deadline), etc. You can change the TODO keyword or delete it.

When you’re done, type C-c C-c to automatically save it to your default notes file (as specified by org-default-notes-file). Changed your mind? Cancel with C-c C-k. After either C-c C-c or C-c C-k, you should be back to whatever it was that you were working on.

Practise using C-c c (org-capture) to quickly jot down several tasks or notes. Then go to your notes file and use C-c C-w (org-refile) to move the notes to the right place.

You can also refile the notes right from the capture buffer. Instead of typing C-c C-c to finish your note, use C-c C-w to refile it.

Get the hang of using org-capture to take notes, organizing them every so often (maybe at the end of your day, or once a week?) or refiling them as you go.

Step 4. Define your own org-capture-templates for greater convenience

If you find yourself capturing different kinds of notes often or you want to capture in another format (table entry? list item?), invest the time in customizing org-capture-templates. In the beginning, you might find the Customize interface you get from M-x customize-variable org-capture-templates to be easier to work with than setting the values in plain Emacs Lisp, since the Customize interface lists the options. Read the documentation and look at examples of how other people have configured their org-capture-templates for more ideas. I have quite a few templates defined in my config, and http://doc.norang.ca/org-mode.html has a number of templates too.

Step 5. Pull in additional information

org-capture and org-refile are great when you’re at your computer, but what if you’re away? Quite a few people use MobileOrg to take quick notes on the go. I haven’t gotten around to setting that up for my workflow properly; instead, I use Evernote to jot quick notes on my phone. As part of my weekly review process, I look at the notes in my Evernote inbox and copy them into Emacs as needed.

You can manually copy information from your preferred non-Emacs note-taking tools, or you can figure out an automatic way of doing so. For example, I have some code to copy Evernote notes titled “Journal” into an Org Mode file structured by year-month-day.

Tweak your workflow!

Here’s a quick sketch showing some of your workflow options when it comes to capturing and organizing information with Org Mode. Which combination do you prefer, and how could you make it even better?

2015-02-09 Capturing Org Mode notes more efficiently -- index card #emacs #org #capture #refile

2015-02-09 Capturing Org Mode notes more efficiently – index card #emacs #org #capture #refile

Emacs: Peer-to-peer coaching is easier when you use impatient-mode to share your buffer

It turns out that coaching someone on Emacs goes really quickly when you can sit side by side. I’ve been helping Sean Miller with Org Mode and Emacs Lisp, with the condition that he has to turn those notes into blog posts. Since he happens to live in the same neighbourhood, this is pretty convenient.

E-mailing snippets back and forth would’ve been tedious, and working on only one of the computers would’ve slowed us down. I remembered Christopher Wellons‘ demonstration of impatient-mode, which lets you display your buffer contents on a web page that automatically updates when you make changes. Here’s what I did:

  1. Install the impatient-mode package.
  2. Call M-x httpd-start.
  3. Configure the firewall to allow incoming connections.
  4. Put the selected buffer into impatient-mode.
  5. Share the link with my IP address (form: http://my.ip.ad.dress:8080/imp/)

Then Sean was able to easily copy text from my screen and try things out on his computer. For example, we dug into org-capture-templates to find out how to create datetree items that were also scheduled on the specified date in the agenda, and we converted his manually-dated headings to include active timestamps so that those would be on the agenda too. It would’ve been a hassle to dictate or re-type the regular expressions, and since we were changing the code rapidly based on our conversation, e-mail would’ve been a hassle. Using impatient-mode worked out well.

What did I like about this two-computer setup? I had access to all my configuration files so that I could easily pick up snippets, and I could demonstrate interesting features as I figured out the code or explained my approach. At the same time, he had access to all of his files and he could take notes in his own buffer. We didn’t have to fuss about with keybindings.

This kind of setup could probably work with a virtual coaching session, too, especially if both people have multiple monitors or if there was an easy way to switch between views.

This is what the webpage looked like:

2015-02-06 11_31_01-sean

Neat, huh?