Category Archives: learning

Improving how I organize notes with Org Mode

Let me think about how I organize my Org Mode files, and how I might improve that. =)

Separate files

You can put different things in different files, of course. I use a few large Org files instead of lots of small ones because I prefer searching within files rather than searching within directories. Separate files make sense when I want to define org-custom-agenda-commands that summarize a subset of my tasks. No sense in going through all my files if I only want the cooking-related ones.

What would help me make better use of lots of files? I can practise on my book notes, which I’ve split up into one file per book. It’s easy enough to open files based on their titles (which I put in my filenames). But I don’t have that overall sense of it yet. Maybe #+INDEX: entries, if I can get them to generate multiple hyperlinks and I have a shortcut to quickly grep across multiple files (maybe with a few lines of context)? Maybe a manual outline, an index like the one I’ve been building for my blog posts? I can work with that as a starter, I think.

Okay. So, coming at it from several directions here:

  • A manual map based on an outline with lots of links, with some links between topics as well – similar to my blog outline or to my evil plans document
  • Quick way to grep? helm-do-grep works, but my long filenames are hard to read.
  • Links between notes and to blog posts
  • TODOs, agenda views

Outlines

Within each file, outlines work really well. You can create any number of headings by using *, and you can use TAB to collapse or expand headings. You can promote or demote subtrees, move them around, or even sort them.

I generally have a few high-level headings, like this:

* Projects
** One heading per current project
*** TODO Project task
* Reference
Information I need to keep track of
* Other notes
* Tasks
** TODO Lots of miscellaneous tasks go here
** TODO Lots of miscellaneous tasks go here
** TODO Lots of miscellaneous tasks go here

Every so often, I do some clean-up on my Org files, refiling or archiving headings as needed. This makes it easier to review my current list of projects. I keep this list separate from the grab-bag of miscellaneous tasks and notes that might not yet be related to particular projects.

I use org-refile with the C-u argument (so, C-u C-c C-w) to quickly jump to headings by typing in part of them. To make it easy to jump to the main headings in any of my agenda files, I set my org-refile-targets like this:

(setq org-refile-targets '((org-agenda-files . (:maxlevel . 6))))

How can I get better at organizing things with outlines? My writing workflow is a natural place to practise. I’ve accumulated lots of small ideas in my writing file, so if I work on fleshing those out even when I don’t have a lot of energy–breaking things down into points, and organizing several notes into larger chunks–that should help me become more used to outlines.

Tags

In addition to organizing notes in outlines, you can also use tags. Tags go on the ends of headings, like this:

** Heading title     :tag:another-tag:

You can filter headings by tags using M-x org-match-sparse-tree (C-c \) or M-x org-tags-view (C-c a m).

Tags are interesting as a way to search for or filter out combinations. I used tags a lot more before, when I was using them for GTD contexts. I don’t use them as much now, although I’ve started tagging recipes by main ingredient and cooking method. (Hmm, maybe I should try visualizing things as a table…) I also use tags to post entries under WordPress blog categories.

How can I get better at using tags? I can look for things that don’t lend themselves well to outlines, but have several dimensions that I may want to browse or search by. That’s probably going to be recipe management for now. If I figure out a neat way to add tags to my datetree journal notes and then visualize them, that might be cool too.

Links

Org Mode links allow me to refer not only to web pages, files, headings, and text searches, but to things like documentation or even executable code. When I find myself jumping between places a lot, I tend to build links so that I don’t have to remember what to jump to. My evil plans Org Mode file uses links to create and visualize structure, so that’s pretty cool, too. But there’s still a lot more that I could probably do with this.

How can I use links more effectively? I can link to more types of things, such as Lisp code. I can go back over my book notes and fill in the citation graph out of curiosity. Come to think of it, I could do that with my writing as well. My writing ideas rarely fit in neat outlines. I often feel like I’m combining multiple threads, and links could help me see those connections.

In addition to explicit links, I can also define “radio targets” that turn any instance of that text into a hyperlink back to that location. Only seems to work within a single file, though, and I’ve never actually used this feature for something yet.

Properties

You can set various properties for your Org Mode subtrees and then display those properties in columns or filter your subtrees by those properties. I’ve used Effort to keep track of effort estimates and I have some agenda commands that use that. I also use a custom Quantified property to make it easier to clock into tasks using my Quantified Awesome system.

I could track energy level as either tags or properties. Properties allow for easier sorting, I think. Can I define a custom sort order, or do I have to stick with numeric codes? Yeah, I can sort by a custom function, so I can come up with my own thing. Okay. That suggests a way I can learn to use properties more effectively.

There are even more ways to organize Org Mode notes in Emacs (agenda views, exports, etc.), but the ones above look like good things to focus on. So much to try and learn!

Figuring out how my temporary sleep schedule interacts with programming, writing, and drawing

I was thinking about how I can use these snippets of time to improve in programming, writing, and drawing. I realized that although I can easily imagine how other people can write or draw using fragmented time (writers scribbling in notebooks on top of washing machines, artists doodling on the subway), programming seems a lot less tractable. It doesn’t feel like you can break it up and squeeze it into different parts of your day as much.

It is generally accepted that context switching is evil when it comes to programming. So I’ve been carrying around this idea that Real Programmers are people who can pull all-nighters hacking on tough problems, holding elaborate structures in their heads. Your standard hero programmer stereotype, with the pinnacle being someone either building complex, cool stuff, possibly maintaining large and useful open source software.

Hence this little mental disconnect. I’m pretty certain I can get there someday if I really want to, but probably not if I extrapolate from current circumstances. Even maintaining a tiny piece of software sounds like more commitment than I want at the moment. (Heck, I might go a few weeks without responding to e-mail.)

Fortunately, I spent my first few working years in a corporate environment, where mentors showed me that it’s totally possible to be an Awesome Geek while still working a roughly 9-to-5 job, having families and hobbies, and getting plenty of sleep. Thank goodness. So I have this alternate model in my head, not of a Hero Programmer, but rather of solid contributors who keep making gradual progress, help teams of people become more productive, and who enjoy solving interesting challenges and expanding their skills.

So let’s say that I want to play with my assumption that programming is the sort of thing that’s hard to squeeze into the nooks and crannies of one’s day, at least not the way writing and drawing can. I know that I can go through technical documentation and design resources even if my mind isn’t completely awake, and I can still pick up useful things.

What is it about writing and drawing that make them suitable even in small doses, and how can I tweak programming? Writers can think about stuff during other activities. I can reflect on ideas while walking or cooking, for example. When I program, I still need more of that back-and-forth with a computer and an Internet connection, but maybe I’ll need less of that as I develop more experience. I can set pen to paper during any spare moment, sketching a quick line and seeing where it takes me from there. I might not be able to do that with implementation, but I can use that same playfulness to explore design. Behavior-driven development makes it easier to break projects down into tiny, clear steps, and have a way of verifying progress (without too much backsliding!). Getting deeper into frameworks and tools will help me do more with less effort when I do sit down at a computer.

Okay. I can do this. Worst-case scenario, I just move slowly until I get past this particular phase. I’ve seen role models who’ve pulled that off well, so that’s totally cool. Best-case scenario, I figure out how to hack around some of my current cognitive limitations, and maybe that might help other people who find themselves in the same situation too.

This could work.

Keeping a process journal

I post a lot of notes on my blog, and I keep more snippets in my personal files so that I can learn from them and turn them into blog posts later. There’s something still missing here, though, something I can tweak. Reading Louise DeSalvo’s The Art of Slow Writing (2014), I recognized part of what was missing in her description of process journals. Here’s a relevant excerpt:

p82. In Steinbeck’s April 9, 1951, entry, written as he composed East of Eden, he evaluates his desk’s new surface, determines how to keep his pencil drafts from smudging, figures when it’s best to do his laundry, plans his week’s work, determines to try to write somewhat more, assesses his energy level, discusses his fear of interruptions derailing his work, pledges maintaining his focus to complete the work by managing his work in his journal.

… Here we see Steinbeck deliberately managing his work before he begins the labor of writing. He evaluates his tools–his desk and pencils–shapes his day, sketches the new scene, deals with his emotions, summarizes and evaluates his progress, and figures how to move his work forward. And Steinbeck engaged in this process each day.

(Oh! I love writers Have Thought About Stuff. It’s like the way programmers also tend to apply tools and systems to more than just programming… Come to think of it, I wonder how geeks of other persuasions end up applying their geekdoms to the rest of life!)

Anyway: a place to clear your thoughts, a deliberate reflection on processes and practices, and perhaps a way to browse through those entries in chronological order or based on context… My blog is a little like that, but there’s so much more stuff than I publish on it and it will continue to be like that if I insist on keeping to my mostly-one-post-a-day limit and scheduling things in advance.

I’ve been keeping a small journal–just a few keywords per day, scribbled into a paper notebook shortly before going to bed–for the past three months. It’s amazing how that’s enough to help me get back to those days, remembering more details than I could without them.

Org Mode for Emacs has built-in support for quickly capturing notes and organizing them in an outline by date. I think I’ll use that for at least quick memories, since those make sense in a timeline, and then I’ll keep the larger notes in a topic-focused outline. Technically, I’m using a computer, so I should be able to organize things both ways: using tags and links to connect items by topic, and using Org’s log view to view things by date.

It would be good to start with this kind of deliberate, constant improvement in a few areas of my life:

  • Web development: I’d like to learn more about design, and also developing better code
  • Writing: I can pay more attention to the questions I formulate and how I explore them
  • Cooking: Hmm, more notes on how we make the recipes and what the cooking process is like?

If I make Fridays the days I focus on harvesting my notes from the previous week and plan some ideas for the next one, that would fit in nicely with reviewing this process journal and seeing what I can build on the next week. (I’m still going to post random snippets during the week, probably… =) )

My current book workflow

I pick up a lot of information from books. We have an amazing public library system, and I’m at the library at least once a week. Here’s my current workflow for reading and taking notes on books.

Finding books to read: I check the library’s new releases for interesting titles. I have a Ruby script that extracts the information and puts it into a text file. I delete anything that doesn’t interest me, and then I copy the IDs into another Ruby on Rails system that requests all the books for me. Sometimes I search for books by topic or get recommendations from other people.

Reading books: I’ve successfully weaned myself off the bad habit of folding over the corners of book pages (dogearing). Instead, I use book darts to point to the passages that I want to copy into my notes. I have only one tin of them, so that encourages me to harvest the notes from books before moving on to other books. If I don’t have my book darts handy, I use strips of paper, but they’re not as awesome.

I also keep paper and pen handy (index cards, or the small notebook I always have in my vest) so that I can take notes on ideas, questions, and other things that aren’t directly in the text of the book.

I mostly read non-fiction, so that’s easy to skim for interesting bits. I usually check the table of contents to get an overview of the book. I have no qualms about jumping straight to specific chapters and then wandering around a bit, or even picking just a few pages out of a book. I rarely use the index (and most books don’t have a good one anyway), although maybe I should check that more often.

Taking notes: I keep individual text files named after the title of the book (and sometimes authors as well). I usually include the ISBN so that I can easily look up a book later. The text files contain quotes, ideas, TODOs, and other notes.

If no one else is around and I feel like patiently dealing with speech recognition, I open Dragon Naturally Speaking and dictate the passages from the books. This helps me train Dragon’s speech model as well, which might be handy someday. If other people are around, though, I’ll just type in the segments from the book.

I usually type in my paper notes so that they’re more coherent (since I tend to write keywords instead of full thoughts). If I want to scan my index cards or notebook pages, I pop those into my Fujitsu Scansnap ix500 and scan them as JPG. I convert the JPGs to PNG and rename them with the date and title, and then I move them to a folder that gets automatically imported into Evernote. Evernote lets me search for text, and it also tries to find text even in scans of handwriting. It’s not perfect, but it’s decent, and I’ve learned to write more clearly because of that.

Looking things up: Since most of my notes are text, I can use grep to search through them.

Sharing what I know: I sometimes include excerpts or ideas in my blog posts. When I do, I link to the blog post from my text notes as well, so I can see what I’ve digested further and shared. If I’m reading a book that I know I’ll want to share with other people later (or if the authors have asked me nicely =) ), I sometimes visually summarize the book.

Following up on ideas: I add TODOs to my Org Mode agenda. I can also schedule reminders for things. I’m a little hesitant to add my books directory to the org-agenda-files list that Org Mode checks for TODO items (I have hundreds of book notes now!), so I’ve defined a custom agenda command that looks at just the book directory instead. Alternatively, it’s easy enough to grep the TODO keyword.

Planned improvements: I’m curious about the idea of a syntopicon, which I picked up from Adler and van Doren’s How to Read a Book. A syntopicon is a map of ideas across multiple books. With Org Mode’s support for indexing and links, I should be able to make something like it. I’m also looking forward to writing more about what I do with what I’m learning from books. This helps me challenge myself to learn actively instead of just letting a book flow through my brain.

Mmm, books!

If you’re curious, you can read about my past workflows:

Planning little improvements

I like re-planning when things are a little bit clearer and when things change. It’s nice to take a look at where I am, where I might get to, and maybe what I can do with more reinvestment.

wpid-2014-11-01-Baselines-and-possible-improvements-part-1.png wpid-2014-11-01-Baselines-and-possible-improvements-part-2.png

A year still feels a little abstract. A 12-week span might be interesting for concrete goal-setting and momentum; maybe something to experiment. In any case, here’s a small achievement list I can work towards…

  1. Development
    • Propose a calendar of prototypes with business-value descriptions
    • Design prototype and help team members write it instead of coding it myself
    • Think syntactically
  2. Reporting
    • Make Tableau reports snappy
    • Identify business questions for a valuable regular report
    • Analyze my own data in R
  3. Writing: Put together the intermediate Emacs config guide
  4. Drawing: Sketch people quickly
  5. Cooking: Map the families of recipes I want to try, and try them
  6. Learning: Map the things I know and what I want to learn, and maybe find a coach
  7. Tracking: Do grocery tracking in Quantified Awesome
  8. Making: Sew those box cushion covers
  9. Organizing house stuff
    • Simplify wardrobe
    • Tile floor
  10. Biking: Maybe bike in winter
  11. Pet care: Get Luke used to the toothbrush
  12. Exercise: Do the exercise ladder for twelve weeks
  13. Relationship: Work on more projects together
  14. Community:
    • Set up Emacs hangout experiment
    • Hang out at Hacklab during winter

Coming to terms with online courses

There are lots of free online courses available on sites like Coursera, but I’ve always had a hard time sticking with them. I don’t like listening to lectures; they feel too slow. Slides and subtitles are inferior to properly-formatted tutorials or books. I sometimes sign up for courses, but then I wander off when I lose interest.

I decided to try online courses again, since one of the other Hacklab members spoke highly of the R course she was taking on Coursera. This time, I tried skipping the lectures, focusing instead on answering the quizzes and doing the programming assignments – essentially, treating it as an open-book exam. That worked out pretty well, actually. I quickly completed all the quizzes, and it took me a few more hours to get the programming assignments sorted out. Many of the programming assignments had self-checking mechanisms, so I didn’t have to wait for peer evaluation.

I like that a lot more than the old way I used to try to get through these online courses. By focusing on the assessments, I can get through the course quickly, identify anything I want to dig deeper into, and try something new with the ability to check my work. Sure, I miss out on testing my ability to retain more information and I might miss out on important points not covered by quiz questions, but at least I’m getting some value out of online courses. I’m sure I’ll get the hang of other ways to study later on!