Category Archives: organization

How to manage a large blog archive


I’m celebrating my 30th birthday this August. Milestone birthdays are great excuses to look behind and look ahead. I don’t know how other people do it. I can barely remember what happened last week, much less ten years ago. Me, I cheat. I have blog archive, which 18-year-old me had the foresight to experiment with (although back then, I was just looking for a way to remember all those class notes and Emacs tidbits I was picking up). I’ve written more than six thousand blog posts in the last eleven years. (See Quantifying my blog posting history for a nifty visualization of my blog posting history.) My published posts probably include well over two million words. This is awesome.

Since not a lot of people have the same experience of blogging consistently over more than a decade, I thought I’d share what I’ve been learning along the way.

Have your own domain name. One of my first websites was on Geocities. Another was on (hosted by I-Manila, which was our ISP then). Both services are long gone. I registered in 2006 and moved everything over to that. Since my name can be hard to spell, I registered in 2008. I‘ve started experimenting with my own URL shortening domains, and . While domain names are a recurring expense, they’ve been well worth it.

Move your data instead of starting from scratch. I changed blogging platforms (Emacs Planner Mode to WordPress) and moved web hosts, but I’d taken pains to move my data instead of starting fresh. Now I’m enjoying the benefits of having that archive handy.

Back up, back up, back up. I want this to be around in another sixty years. I like backing up the data in many different ways: database, files, HTML dumps, PDFs, even paper. I lost a bunch of photos and drawings when my Gallery2 setup got hacked, but I restored a number of them from files I found elsewhere. I look forward to being able to review decades and decades of notes.

Weekly, monthly, and yearly reviews go a long way towards making it easier to remember what happened. Day-to-day living makes it hard to see what’s important. A week seems to be the most natural chunk of time for my reviews. I wrote a little bit of code that auto-summarizes my task list and accomplishments. Every month, I

Search is your friend. If it takes a lot of digging to find something, make it more findable. I often use Google Search or my blog’s built-in search to find posts based on keywords that I remember. If it takes me a while to find something, I edit the post and add categories or tags to make it easier to find in the future. I sometimes write a new post that shares what I’ve learned since then, linking to the previous post for history.

Comments on older posts are awesome. Search engines are a wonderful, wonderful thing. I love it when people comment on old posts – it’s nice to know those posts are still helpful. Sometimes people comment on things I’ve completely forgotten writing, so it’s a great way to refresh my memory as well.

Check your analytics once in a while. I don’t really care about the number of visitors or the bounce rate, but I’m curious about what people are reading and where they’re coming from.

Indexes are good, too. Every month, I update this categorical index of my blog posts. I probably should go back and make sure that the WordPress categories match this as well, although in WordPress, I tend to use categories more like tags (I file a post in multiple categories).

Cultivate synchronicity and randomness. WordPress plugins help recommend similar posts, other posts that were written on the same day, and random posts. It might mean that my pages are overloaded with links… but it might also spark an aha! or an interesting conversation with someone browsing around, so I think it’s worth it. Besides, at this point, a computer will often be better than I could be at recommending other things that people should check out, so I use those features myself when I’m browsing my blog.

Write about the small stuff. I used to wonder whether the weekly reviews were worth posting on my blog, seeing as they’re mostly my task lists. Reviewing my blog years later, I was surprised to find that the weekly reviews were excellent at helping me remember what was going on. They were also great for filling in the blanks in my records – When did I fly out? What did I do? Whatever happened to that thing? Hooray for the small stuff.

Revise and summarize. It’s okay to write about something you’ve written before. In fact, it can be a great excuse to learn more and get closer to understanding the big picture.

If you’re starting out today, don’t worry. Stick with it, and in ten years, you’ll have something pretty darn awesome too.

Out of curiosity, do I know anyone else who’s got a big archive? How do you manage yours?

Emacs: How I organize my Org files

Michael Jones wanted to know how I organized my Org Mode files. Here’s how I do things!

Org Mode for Emacs is an outliner that lets you add a little structure to plain text files. Not only can you use it to move around, hide, and show sections of your outline, but you can also:

  • schedule tasks and mark them as complete,
  • add hyperlinks and formatting,
  • estimate effort and track time,
  • export to HTML and other formats,
  • and even include code that you can run in-line.

I started with a single Org Mode file (appropriately called, but I’ve gradually fleshed this out into a number of files. My goals for organizing my files this way are to be able to:


  • Publish some files while keeping other files private,
  • Add or remove groups of tasks from my agenda, or focus my agenda/search on the current file,
  • Simplify processing my weekly review (categorizing accomplishments/tasks),
  • Get a quick overview of important things, and
  • Have file-specific options, like columns.

I often use Org agenda custom commands to jump around. For example, one agenda command lists projects, and pressing RET on an agenda line will take me to that project. I also use org-capture to take a note from anywhere, and I use org-goto to navigate my files. For jumping to a specific file, I use ido-find-file.

I use several Org Mode files. The six files below have a little more than 1.3MB of text in total – tiny! – but they help me tremendously. I also have lots of other Org files like my Emacs configuration and my blog index (I often use Org for publishing), but these are my main files.

Personal tasks and notes:

This is the catch-all for any tasks or notes that don’t belong to the files below. Here’s the rough structure:

  • Quick notes: Tidbits that might not make it into their own blog posts, but which can be included in weekly reviews
  • Reference: Hours, license keys, etc.
  • Open loops: Anything I need to check on every so often
  • Projects: High-level things I’m focusing on
  • Financial goals: Things to save up for
  • Someday/maybe: Projects to do someday
  • Weekly review: Divided by year
  • Monthly review: More summaries
  • Plans: Personal plans
  • 2011, 2012, 2013…: I use org-capture to quickly jot down notes. The datetree option automatically files them by day, which makes older ones easier to archive.
  • Tasks: A bucket for miscellaneous tasks

Anything to do with business:

I organize these by the types of tasks I focus on and the notes I want to keep.

  • Earn
    • Clients
    • Leads
  • Build
    • Projects
    • Research
    • Business ideas
    • Blog
    • Delegation
    • Planning
    • Business hygiene (accounting, etc.)
    • Learning
  • Connect
    • Meetups
    • Hangouts
    • Other
  • Reference
  • Tasks


I organize these by relationships so that I can remember who’s out there.

  • Family
  • Extended family
  • Canada friends
  • Hacklab
  • Barkada
  • Letters
  • Meetups
  • Bloggers
  • Family friends
  • Other tasks

Regular tasks:

I organize these by frequency and omit the tasks from my weekly review. This also contains my “In case of…” scenarios and my backup documentation.

  • Every day
  • Once a week
  • Once a month
  • Once a quarter
  • Once a year
  • When…

Outline for future blog posts: sharing/

I organize this by topic. See for the published version

Decision review:

I organize these by status. I also use org-choose markers (ex: CHOSEN, MAYBE) inside the categories, but the headings make it easier to review.

  • Pending
  • Current
  • For review
  • Someday / maybe
  • Archive

Personal finance:

I use John Wiegley’s command-line Ledger program to manage my finances. My financial data is in separate ledger-mode files, and I use an Org file with org-babel to make it easier for me to answer some questions about my finances. For example:

  • Given my average monthly expenses and the amount of money I’ve set aside, how long can I sustain my early-retirement experiment?
  • Am I ahead or behind in terms of household contributions?
  • What did I spend on last month?
  • Are my virtual envelopes balanced?

How do you organize your Org files or outlines?

Everyone’s got different ways of organizing outlines, and people also also change over time. How do you organize yours?

How I organize and publish my sketches

In a recent blog post, Mel Chua wrote: “I’m still trying to figure out how to best store/catalogue my (growing) collection of sketches so it’s easy for people to access it.” So, here’s how I handle mine!

How I organize and publish my sketches

I have three types of sketches:

  • A1. Public: Sketches I can publish (and usually that I want to write about someday)
  • A2. Public, blogged: Sketches I have written about
  • B. Private sketches to help me think

My goals are to:

  • Support my writing: Blogging, naturally.
  • Search my sketches: Evernote’s fantastic for this, since I can have my public and private sketches in one place
  • Make my sketches publicly searchable: Evernote shared folders are great for that; Flickr and my blog are useful too. I often use Evernote to give someone a keyword search or tag search through my sketchnotes, sketchbook, or visual vocabulary. (Evernote users can join these notebooks and get updates automatically!)
  • Publish most of my sketches: Blogging can take me a while, so I try to get public sketches out there as soon as possible so that they don’t get lost. Flickr and Evernote help.
  • Flip through my sketches: Great for doing reviews and triggering memories. =) Can’t beat paging through local files manually or in a slideshow.
  • Organize my sketches by topic: Evernote, Flickr, and my blog let me tag things with keywords, while a mindmap lets me give my sketches more structure.
  • Share sketches widely: My blog and Twitter seem to be the best ways to do this, although Flickr is useful too.
  • Open sketches for discussion: My blog is the best place for that, although Flickr and Twitter are handy as well.
  • Update my sketches: Whether I’m colouring things in or checking off boxes, I want an easy way to get to a sketch and update it in Evernote and Flickr. If I’ve blogged about it, I’m okay with the blog post having the archived version of the image.
  • Archive my sketches: I want to back up digital copies in several places so that I can recreate my collection if needed. Blogging, Flickr, Dropbox/file backups, PDF collections…

Stuff I’ve tried that didn’t work out so well:

  • Referring to external services in my blog posts: Flickr? The Gallery2 instance I installed? Problematic if code changes, services go down, accounts are discontinued, or (in one annoying case) my self-hosted Gallery2 gets compromised. Disk space is cheap, so I just re-publish images using WordPress’ upload mechanisms (most blogging tools handle this automatically).
  • Picking just one way to publish stuff: Flickr is better for volume and some discussion, Evernote is better for search, my blog is better for sharing and long-term search. Since no tool has everything I need, I’ll just have to put up with the hassle of replicating information.
  • Just using automatic organization: For the last few years, I relied on Dropbox folders and Evernote items. Dropbox folders are fine for organizing by date and Evernote’s great for tags, but I want manual organization as well – organizing things by topics and subtopics, tracking things in progress, and so on. That’s why I’m experimenting with mindmaps now.

Stuff I’m working on next:

  • Monthly and quarterly PDF packages of my sketches, organized by date or topic: for ease of printing and review
  • Letting people know about available resources (my Flickr stream or Evernote notebooks) so that they can search/discuss/subscribe

See my drawing workflow for other notes about my process. Hope this helps!

Making the most of paper notes

I’ve been drawing a lot more on paper lately, so I should update my Sketching Tools page. It is nice as a way to quickly get my thoughts down without the tiring brightness of the computer screen or the distractions of the Internet. Here’s what I’ve been learning about the differences between drawing on paper and drawing on my computer:

2013-10-28 Drawing on paper versus drawing on my computer

There’s always room to make things better, of course. How can we think on paper more effectively? The mindmap that I’ve been working on gives me a useful overview, letting me see when I’ve accumulated several sketches in a particular area so that I can put them into a blog post. I’ve also figured out how to include the sketches in my review process, thanks to this Flickr metadata downloader (Python). Speech recognition still hasn’t made its way into my toolkit, though…

2013-10-21 How can I think on paper more effectively

Now that I’ve got a decent archive of paper notes, the next challenge is making these easy to search and organize. I’ve put together some tips for making your paper notes more searchable here:

2013-11-11 Make your paper notes more searchable (low-tech and hi-tech tips)

… and getting them into your computer so that you can organize them along with the rest of your notes.

2013-11-14 Integrating paper and digital notes  

Lately I’ve been using Flickr for sharing and tagging images and Evernote for the occasional hand-written search. Let’s see how this works out…

I’m really curious about how other people manage their paper notes. I’ve been trying to find more details on how Isaac Asimov organized his notes – filing cabinets, apparently, but it would be great to get more detail! Do you have a large paper archive? How do you manage it? Do you know anyone who does this really well?

Building a simple sketch navigator for myself

In 2015, I built the habit of drawing daily index-card-sized thoughts. Some days, I reached (or blew past!) my target of five index cards a day. Other days, I backfilled my daily journal entries based on fuzzy memories and my time logs. In February 2015, I started using a yyyy-mm-dd<letter> naming convention so that I could easily refer to individual cards. For a while, I tried to be fairly disciplined about organizing sketches into outlines and building up chunks into blog posts. That fell by the wayside when I found it difficult to concentrate, but I kept drawing anyway. I saved the images to my hard drive, started tagging them with keywords in the filenames, and uploaded my sketches to Flickr as a way to back up and share my files.

I’ve recently been exploring ways to take advantage of the Samsung tablet that W- bought some time ago. It works wonderfully as a second screen that I can use to review an old sketch while I draw another one. The FlickFolio app lets me do a random slideshow of the past 2000 images in my photostream or in an album. This turns out to be an excellent way to jog my memory and prompt me to revisit questions or decisions.

2016-01-22d What do I want from my idea pipeline -- index card #zettelkasten #ideas #thinking #questions #index-cards

2016-01-22d What do I want from my idea pipeline – index card #zettelkasten #ideas #thinking #questions #index-cards.png

Now those fragmented thoughts are starting to pay off. Many of my old index cards are on topics I haven’t thought about in a while. It’s fun to see what I’ve learned in the meantime, or to follow up on things I’ve forgotten. It’s like I’d been filling an idea pipeline or seeding an idea garden. I get this steady stream of questions, decisions, observations, memories, ideas, and notes, and I can build on those prompts instead of having to start from scratch.

2016-01-18g How do I want my sketches to help me think -- index card #drawing #sketches #zettelkasten #notes ref 2015-11-18a

2016-01-18g How do I want my sketches to help me think – index card #drawing #sketches #zettelkasten #notes ref 2015-11-18a.png

I want these index cards to help me shuffle ideas and possibly see serendipitous combinations. They’re small, so capturing thoughts on them doesn’t require as much effort as, say, writing a blog post. They’re a good way to smooth out and organize thoughts, and I can chunk up those thoughts into longer posts. The sketches are easy to review, too, so they work well as digital footprints. I re-read the year’s blog posts as part of my annual review and sometimes I reread my posts on a monthly basis as well, but it’s not as immediate or as wide-ranging as flipping through a random selection of images.

A random slideshow is a good start, but I’ve been meaning to make a sketch browser that’s a little more tuned to what I want.

2016-01-18d What do I want in a sketch browser -- index card #drawing #coding #plan

2016-01-18d What do I want in a sketch browser – index card #drawing #coding #plan.png

I wrote a simple NodeJS server that I can run on my laptop and access from the tablet (or other devices) while I’m on my home network. At first, I was trying to figure out an interface that would let me navigate by month/week/day, but then I realized that something simpler might be a more useful way to begin.

I started off by making it display random sketches:


Then I extended it to let me browse by tag:


And then to see a list of tags by frequency:


Or by alphabetic order, since that might be handier if I’m trying to look for something specific on a mobile device


I like distinguishing between daily/weekly/monthly/yearly reviews and non-journal sketches, too:




It’s nice to be able to build custom little tools like that. =)

Here’s the script on Github, in case you’re curious or you want to build on the idea.

What do I want from my Org Mode files?

What do I want from the notes I keep in Org Mode, how’s that working out, and how can I improve?

Remind me of important tasks, especially recurring ones or ones in the far future
This works pretty well, especially with my weekly review. I mostly trust it, although it might be nice to use the timeline view to review tasks over the next few years just to make sure the important ones are there. And backups!
Keep detailed checklists, instructions, and notes, so that I don’t miss any steps or have to figure things out again when I’m sleep-deprived
I’ve found this useful when dealing with my business paperwork, and I look forward to documenting more routines.
Capture quick thoughts and tasks so that they don’t clutter up my mind
org-capture is good when I’m at my computer, and Google Tasks is a decent inbox when I’m away. Not very good at reviewing and refiling the items, though, but I can do that when I have more discretionary time.
Break bigger projects down into manageable tasks
I don’t have the brainspace right now to work on projects, so most of these have been shelved. I need to tweak my Org refile targets to make organizing things easier. I might be running into a limit with too many targets. Sometimes I can’t use org-refile to select a task that I already know exists.
Help me untangle my thoughts or keep a trail of breadcrumbs as I solve problems
Pretty good at this. Limited by computer time at the moment.
Pull together information and help me summarize
The code I wrote for my weekly and monthly reviews is working well. The code for Emacs News is decent, too, although I can save a little more time if I fix my function for opening all links in a region.
Draft blog posts
This is working well. It could be a little better if I sorted out image uploading and resizing, but my current workflow is fine.
Help me make the most of my limited computer time by prioritizing small tasks that move me forward
This is probably the weakest area. Right now, I tend to prioritize drawing, then checking my agenda for urgent/quick tasks, and maybe writing if I can squeeze it in. I mostly save writing for my phone, though, because I can write on my phone and I can’t do the other tasks then. Coding might help me improve efficiency, but that might have to wait until I have more focused time. It’s okay, I’ll get back to that someday. I think getting better at writing and posting will pay off well enough in the short term. If I give myself permission to post short, possibly incomplete thoughts (like I tell people to! :) ), I’ll get more stuff out there, and then I can build up from there.
Keep notes on people
Little memories, triggers for thoughtfulness, etc. I’m definitely the bottleneck here, not Org.
Help me review my decisions
It’s good to write down goals, options considered, criteria, trade-offs, reasons, predicted results, and so on. My bottleneck is probably taking the time to do so. People are good at rationalization, so I’m not trying to judge whether something was a good decision or a bad decision, but it’s interesting to see what decisions and evaluations reveal about my preferences and values.
Remind me about tools, how to use them, why, and so on
This is partly why I have a literate configuration – so the outline can remind me about stuff I’ve already coded. It’s also handy to keep track of commands and scripts that help me with various tasks. I just need to remember to copy and paste stuff as I do things.

Overall, I’m okay with input and output. Processing is my bottleneck at the moment. If I either fix that org-refile issue I’ve been running into, or come up with an alternative flexible search that will help me find outline entries when I don’t quite remember the headline, that should make processing a bit easier. A bit of outline gardening would help, too – archiving things that are no longer relevant, refiling notes and improving their headlines/text for searchability, maybe prioritizing tasks based on costs and benefits… I’m not entirely sure I’d be comfortable doing that on my phone, so it will have to wait for computer time.

In the meantime, I’m glad I have a place to accumulate (and eventually organize) all those notes!