Weekly review: Week ending February 6, 2015

This week, I experimented with treating consulting as a dessert to encourage me to make decent progress on my personal projects. Mixed results. I learned a lot about continuous integration and code coverage reporting in Emacs (yay!), created my own Helm sources, and improved my index card workflow, but the setup made it easy to give in to the temptation of working on other people’s stuff instead of sitting with my own resistance. Still, I did a good job with those consulting hours, building a couple of interesting prototypes and reports.

My digital workflow for drawing and filing index cards is getting better and better, though. =) I used the Emacs widget library for the first time, and I can’t wait to build more of these little interfaces to make my life easier. Whee!

Didn’t cook last weekend because the fridge had lots of leftovers, so pickings were a little slim this week (what with the snowstorm, too). Looking forward to getting back to a regular routine of cooking this week, maybe even doing bulgogi with a bunch of side dishes. =)

2015-02-07 Week ending 2015-02-06 -- index card #weekly

2015-02-07 Week ending 2015-02-06 – index card #weekly

Blog posts

Sketches

  1. 2015.01.31 Clearing my fabric stash – index card #tidying #decluttering
  2. 2015.01.31 Digital index card trade-offs – index card #drawing
  3. 2015.01.31 Most of my questions are ‘How can I…’ – index card #learning #question
  4. 2015.01.31 What’s worth remembering – index card #memory
  5. 2015.01.31 When I don’t really like talking, and what I might be able to do about it – index card #connecting #introvert
  6. 2015.02.01 Accelerating my learning – index card #learning #accelerating
  7. 2015.02.01 Accelerating the Emacs community – index card #accelerating #emacs
  8. 2015.02.01 Digital piles of index cards – index card #indexing #organization #pkm
  9. 2015.02.01 Index card sketches and monthly reviews – index card #organization #pkm #indexing
  10. 2015.02.01 January 2015 – index card #monthly #review
  11. 2015.02.02 Core areas – index card #life #priorities #values #goals
  12. 2015.02.02 Imagining an Emacs conference – index card #emacs #conference #plans #organizing-people
  13. 2015.02.02 Making a virtual Emacs conference happen – index card #emacs #organizing-people #conference #planning #questions
  14. 2015.02.02 Plain text – index card #data #organization #pkm #org
  15. 2015.02.02 What if I use the lure of work to help me grow – index card #consulting #experiment
  16. 2015.02.03 Benefits of sharing my index cards – index card #sharing #drawing
  17. 2015.02.03 Better Emacs Testing – index card #testing #emacs
  18. 2015.02.03 Delegation and dreaming small dreams – index card #delegation
  19. 2015.02.03 Getting better at imagining – index card #planning
  20. 2015.02.03 Scooped – index card #writing #sharing
  21. 2015.02.03 Tree view, forest view – index card #org #pkm #tasks
  22. 2015.02.03 What kinds of personal knowledge do I want to manage – index card #pkm
  23. 2015.02.03 Why map what you’re learning – index card #mapping #learning
  24. 2015.02.04 Digital index cards are working well – index card #drawing #digital
  25. 2015.02.04 Improving my Org clocking workflow – index card #org
  26. 2015.02.04 Shrinking my Learn-Do-Share cycle – index card #sharing #learning
  27. 2015.02.04 Yay, passport – index card #canada #travel #paperwork
  28. 2015.02.04 Yay, testing in Emacs – index card #testing #emacs
  29. 2015.02.05 Biscotti – index card #cooking #baking
  30. 2015.02.05 How can I make Thursdays more like other days – index card #experiment
  31. 2015.02.05 How can I organize my questions – and why – index card #questions #organization #pkm
  32. 2015.02.05 How can non-profits regularly reach out to people over social media – index card #social #social-media #connecting
  33. 2015.02.05 If consulting is dessert, what are my vegetables – index card #consulting #experiment #balance
  34. 2015.02.06 How can I brainstorm and organize questions in order to speed up my writing and drawing – index card #thinking #questions #pkm #acceleration
  35. 2015.02.06 How can I more regularly add entries to my private journal – index card #drawing #habit #journal
  36. 2015.02.06 What activities would I like to share with friends – index card #connecting #friends
  37. 2015.02.06 What if I made myself take someone out to lunch or coffee every month – index card #connecting #friends
  38. 2015.02.06 What kind of coaching do I like to give – index card #coaching #teaching
  39. 2015.02.06 What would my ideal question-thinking workflow be like – index card #learning #questions #thinking

Link round-up

Focus areas and time review

  • Business (42.1h – 25%)
    • Earn (20.3h – 48% of Business)
      • Prepare invoice
      • Work on reports
      • Earn: E1: 1-2 days of consulting
    • Build (20.6h – 48% of Business)
      • Drawing (11.1h)
      • Delegation (0.0h)
      • Packaging (2.4h)
      • Paperwork (0.3h)
        • File payroll return
      • Flesh out plan
      • Change my set-descriptions.js to be able to use any blog URLs (archive page, future post, etc.)
      • Pass theme check required changes
      • Set up coveralls for Quantified Awesome
    • Connect (1.1h – 2% of Business)
  • Relationships (5.9h – 3%)
  • Discretionary – Productive (23.8h – 14%)
    • Emacs (14.6h – 8% of all)
      • Get travis going
      • Learn about Emacs testing
      • Fix my config after some weird error
      • Get Cask working
      • Set up Coveralls
      • Add a test to f.el
      • Get Travis emacs23 back on
      • Write tests for one of Steve Purcell’s packages
      • Write tests for one of abo-abo’s packages
      • Set up shortcuts for jumping with refile
      • Think about unconference infrastructure for Emacs
      • Help Sean
      • Measure performance of init
      • Finish Avdi’s transcript
      • Check on @mattl’s upcoming visit to Toronto
    • Organize questions
    • Identify index-card-sized questions
    • Consider skill tree
    • Read chapter 3
    • Writing (8.3h)
      • Start git directory for big drafts
  • Discretionary – Play (1.9h – 1%)
  • Personal routines (23.8h – 14%)
  • Unpaid work (9.2h – 5%)
  • Sleep (61.4h – 37% – average of 8.8 per day)

Japanese curry at Hacklab, curry udon at home

We were planning to make roasted cauliflower for the Hacklab open house dinner, but the cauliflowers were CAD 4.50+ each. Instead, we made vegan Japanese curry (roux recipe), which has become one of my favourite recipes. It’s a great way to cook carrots, potatoes, green beans, daikon, bell peppers, broccoli, mushrooms, peas, and other vegetables you might have.

At home, I made another huge pot of Japanese curry, but with chicken instead of tofu, and with a non-vegan roux recipe. We tried it with udon instead of rice. Udon gives it a chewier texture and it works well too. I also made quickly-pickled cucumber-daikon-carrot salad, which balanced the taste nicely. Not counting the rice, I think the cost per portion worked out to be around $1. (Wow!)

2015-01-24 Curry udon -- index card #cooking

2015-01-24 Curry udon – index card #cooking

J- loves Japanese food, so it’s great to be able to make things at home. Next time, I think I’ll try making udon from flour, water, and salt. It doesn’t require a pasta roller, just patience. On the other hand, frozen udon is around CAD 4 for a pack of five and it takes one minute to cook, so there’s something to be said for that.

2015-01-25 Planning how to level up in cooking -- index card #cooking #learning #plans

2015-01-25 Planning how to level up in cooking – index card #cooking #learning #plans

While I occasionally play with the idea of going on a cooking vacation so that I can try local markets and learn from cooking classes, there’s so much that I haven’t yet explored here. There are all these ingredients I haven’t yet looked up and tried. (Thank goodness for such a diverse city!) Maybe someday I might even try out cooking lessons or private instruction.

For now, I’m focusing on cooking with more vegetables. I liked the balance of meat to vegetables in that curry we made – about three chicken thighs of meat in one of our biggest pots. It would be great to increase vegetable volume and variety. Looking forward to exploring the rest of the produce section.

I also want to remember to cook different things. I have an Org Mode file with recipes, but I haven’t set up scheduled reminders or Org habits for them yet. In the meantime, I’ve put these index cards on our fridge door to remind me when I’m planning groceries:

2015-01-10 Favourite meals -- index card #cooking

2015.01.10 Favourite meals – index card #cooking

2015-01-11 Snacks we can make -- index card #cooking

2015.01.11 Snacks we can make – index card #cooking

Digital index piles with Emacs: Rapid categorization of Org Mode items

Somewhat daunted by the prospect of categorizing more than a hundred sketches and blog posts for my monthly review, I spent some time figuring out how to create the digital equivalent of sorting index cards into various piles.

2015-02-01 Digital piles of index card -- index card #indexing #organization #pkm

2015-02-01 Digital piles of index cards – index card #indexing #organization #pkm

In fact, wouldn’t it be super-cool if the items could automatically guess which category they should probably go in, prompting me only if it wasn’t clear?

I wanted to write a function that could take a list structured like this:

  • Keyword A
    • Previous links
  • Keyword B
    • Previous links
  • Link 1 with Keyword A
  • Link 2 with Keyword B
  • Link 3 with Keyword A
  • Link 4

It should file Link 1 and 3 under Keyword A, Link 2 under Keyword B, and prompt me for the category for Link 4. At that prompt, I should be able to select Keyword A or Keyword B, or specify a new category.

Inspired by John Kitchin’s recent post on defining a Helm source, I wanted to get it to work with Helm.

First step: I needed to figure out the structure of the list, maybe including a sample from the category to make it clearer what’s included. org-list.el seemed to have useful functions for this. org-list-struct gave me the structure of the current list. Let’s say that a category is anything whose text does not match org-bracket-link-regexp.

(defun sacha/org-get-list-categories ()
  "Return a list of (category indent matching-regexp sample).
List categories are items that don't contain links."
  (let ((list (org-list-struct)) last-category results)
    (save-excursion
      (mapc
       (lambda (x)
         (goto-char (car x))
         (let ((current-item
                (buffer-substring-no-properties
                 (+ (point)
                    (elt x 1)
                    (length (elt x 2)))
                 (line-end-position))))
           (if (string-match
                org-bracket-link-regexp
                (buffer-substring-no-properties
                 (point)
                 (line-end-position)))
               ;; Link - update the last category
               (when last-category
                 (if (< (elt x 1) (elt last-category 1))
                     (setq results
                           (cons (append last-category
                                         (list
                                          (match-string-no-properties
                                           3
                                           (buffer-substring-no-properties
                                            (point)
                                            (line-end-position)))))
                                 (cdr results))))
                 (setq last-category nil))
             ;; Category
             (setq results
                     (cons
                      (setq last-category
                            (list
                             current-item
                             (elt x 1)
                             (concat "^"
                                     (make-string (elt x 1) ?\ )
                                     (regexp-quote
                                      (concat (elt x 2)
                                              current-item))
                                     "$")))
                      results)))))
       list))
    results))

The next step was to write a function that guessed the list category based on the item text, and moved the item there.

(defvar sacha/helm-org-list-candidates nil)
(defun sacha/helm-org-list-categories-init-candidates ()
  "Return a list of categories from this list in a form ready for Helm."
  (setq sacha/helm-org-list-candidates
        (mapcar (lambda (x)
                  (cons (if (elt x 3)
                            (format "%s - %s" (car x) (elt x 3))
                          (car x))
                        x))
                (sacha/org-get-list-categories))))

(defun sacha/org-move-current-item-to-category (category)
  (when category
    (let* ((beg (line-beginning-position))
           (end (line-end-position))
           (string (buffer-substring-no-properties beg end)))
      (save-excursion
        (when (re-search-backward (elt category 2) nil t)
          (delete-region beg (min (1+ end) (point-max)))
          (forward-line 1)
          (insert (make-string (+ 2 (elt category 1)) ?\ )
                  string "\n")))) t))

(defun sacha/org-guess-list-category (&optional categories)
  (interactive)
  (require 'cl-lib)
  (unless categories
    (setq categories
          (sacha/helm-org-list-categories-init-candidates)))
  (let* ((beg (line-beginning-position))
         (end (line-end-position))
         (string (buffer-substring-no-properties beg end))
         (found
          (cl-member string
                     categories
                     :test
                     (lambda (string cat-entry)
                       (string-match (regexp-quote (downcase (car cat-entry)))
                                     string)))))
    (when (car found)
      (sacha/org-move-current-item-to-category
       (cdr (car found)))
      t)))

After that, I wrote a function that used Helm to prompt me for a category in case it couldn’t guess the category. It took me a while to figure out that I needed to use :init instead of :candidates because I wanted to read information from the buffer before Helm kicked in.

(setq sacha/helm-org-list-category-source
      (helm-build-sync-source
          "Non-link categories in the current list"
        :init 'sacha/helm-org-list-categories-init-candidates
        :candidates 'sacha/helm-org-list-candidates
        :action 'sacha/org-move-current-item-to-category
        :fuzzy-match t))

(defun sacha/org-guess-uncategorized ()
  (interactive)
  (sacha/helm-org-list-categories-init-candidates)
  (let (done)
    (while (not done)
      (save-excursion
        (unless (sacha/org-guess-list-category sacha/helm-org-list-candidates)
          (unless
              (helm :sources
                    '(sacha/helm-org-list-category-source
                      sacha/helm-org-list-category-create-source))
            (setq done t))))
      (unless done
        (setq done (not (looking-at "^[-+] \\[")))))))

The :action above refers to this function, which creates a category if it doesn’t exist yet.

(setq sacha/helm-org-list-category-create-source
      (helm-build-dummy-source
          "Create category"
        :action (helm-make-actions
                 "Create category"
                 (lambda (candidate)
                   (save-excursion
                     (let* ((beg (line-beginning-position))
                            (end (line-end-position))
                            (string (buffer-substring beg end)))
                       (delete-region beg (min (1+ end) (point-max)))
                       (org-beginning-of-item-list)
                       (insert "- " candidate "\n  " string "\n")))
                   (sacha/helm-org-list-categories-init-candidates)))))

I’m new to fiddling with Helm, so this implementation is not the best it could be. But it’s nifty and it works the way I want it to, hooray! Now I can generate a list of blog posts and unblogged sketches, categorize them quickly, and then tweak the categorizations afterwards.

2015-02-01 Index card sketches and monthly reviews -- index card #organization #pkm #indexing

2015-02-01 Index card sketches and monthly reviews – index card #organization #pkm #indexing

You can see the results in my January 2015 review, or check my config to see if the code has changed.

My next step for learning more about Helm sources is probably to write a Helm command that creates a montage of selected images. John Kitchin has a post about handling multiple selection in Helm, so I just need to combine that with my code for using Imagemagick to create a montage of images. Whee!

The 5-year experiment: A conversation with my anxious side, and how sharing time might be better than giving money

(If you want, you can skip past the reflection on anxiety and safety and jump straight to the part on how you can help. =) )

Having resolved to learn how to work on my own things, I’m experimenting with reducing my consulting to one day a week (from last year’s routine of two days a week). I spend most of the week reading, drawing, writing, experimenting, and coding.

2015-01-09 What do I do on my non-consulting days -- index card

2015.01.09 What do I do on my non-consulting days – index card

It’s not a big change in terms of hours. I already have plenty of time for personal projects. But I feel the shift in the balance. I can hear that inner self-doubt saying, “Is this real work? Is it worthwhile? Is it sustainable? Are you undermining your safety by goofing off?”

2015-01-07 Real Work -- index card

2015.01.07 Real Work – index card

It’s okay. I expected this resistance, this anxiety. It’s just one of those mental barriers I have to break. Fortunately, all those Stoic philosophers are there to remind me that it’s just a negative impression, not reality, and the truth is that I have nothing to fear.

I’m getting better at telling that anxious part of my mind: “Look. Even though I offer all those resources for free, people willingly pay for it. And other people write wonderful comments and send me e-mail telling me that I’ve inspired them to learn more and that they want to help, so that counts too. Yeah, there’s a chance I might need to go back to Regular Work if the stock market crashes or a catastrophe happens, but in the meantime, just give this a chance. And really, that scenario isn’t the end of the world. Other people do okay. I can too. Besides, that’s why we have safety nets, right?”

2015-01-06 Planning my safety nets -- index card

2015.01.06 Planning my safety nets – index card

2015-01-06 Safe, a little better, comfortable -- index card

2015.01.06 Safe, a little better, comfortable – index card

And then my anxious side goes, “Okay, you’ve probably got the basics covered. But what if your expenses grow, or W- gets tired of living frugally and wants to upgrade lifestyles a little bit? Is this really enough?”

2015-01-06 Is this enough for me -- index card

2015.01.06 Is this enough for me – index card

And then I say, “We’ll probably have some time to adjust our plans for that, and I can always go back to doing Real Work that satisfies you. Besides, if we want to upgrade our life experiences, learning the skills to make stuff for ourselves often works out better than buying things. Like cooking!”

(It’s true! It’s even called the IKEA effect.)

Then my anxious side goes, “Fine. Maybe you have enough space to experiment right now. You want to learn things and help people. But look at your blog! It’s so self-centred. You talk about your questions and reflections, and you rarely give people tips they can directly apply to their lives.”

Then I say, “I’ll get better at writing for other people. In the meantime, this seems to be working okay so far. People translate my reflections into stuff that they can use.”

Here’s how I think my blog helps other people at the moment. Maybe you come across my blog because of a search. You find something that saves you a little time. You browse around a little and learn about things you didn’t even think about searching for. Maybe you come back once in a while for more of those ideas. You bump into other topics you’re curious about, and you explore. You might subscribe, even though you know I post practically every day. You skim the headlines for things that interest you, and you dive into stuff you like. Sometimes you might even feel moved to comment, e-mail, invest time, or even send some money.

2015-01-04 What kind of difference do I want to make, and for whom - index card

2015.01.04 What kind of difference do I want to make, and for whom – index card

How people can help

My anxious side grumbles, “Okay. I’m not sure your blog counts as Real Work, but I’ll grant that people seem to find some value in it. I’d feel better if you were more serious about building a business around it – if you could cover more of your expenses with this instead of consulting income or dividends.”

To which I say, “You know, I’m not sure any amount of money would get you to the point of not worrying. Besides, it’s good that you worry, because that helps keep us safe. This stream will grow as I figure out how to make things that are truly valuable to people. I bet you I can pull it off while still keeping the free/pay-what-you-want aspect, because that’s important to me. Given that you tend to squirrel away additional money to build up safety instead of getting better at investing it to build up capabilities, what we really should be thinking about is if we can make better exchanges of time instead of money. That will probably make a bigger difference anyway.”

My anxious side is sufficiently boggled by that idea and can’t come up with a good rejoinder. This is promising. Let me dig into it further, then.

One of the concepts I picked up from Your Money or Your Life (Dominguez and Robin, 1999) is that you can think of money in terms of the time it took you to earn it, a sobering thought when you apply it to your expenses.

I can apply that idea to other people, too; if other people pay money for something I made, it represents the chunk of their life that they spent earning it (and the opportunity cost of anything else they could’ve bought or invested in, including saving up for their own freedom).

I’m frugal (bordering on being a cheapskate), having gotten very good at making the most of inexpensive resources. Because of the typical mind fallacy, I tend to think that other people should be frugal as well so that they can save up for their own freedom. I suspect that people might get marginally more value from saving that money than I would get from them giving it to me, since their stress reduction or freedom expansion will likely outweigh my slightly increased feeling of safety. On the other hand, people do get value from feeling generous and from patronizing something that they would like to see flourish, so I can agree with that.

If we translate it back to time, though, I’m more comfortable with the exchange.

I already have enough time for the priorities in my life, while many people feel that they don’t have enough time for the priorities in theirs. Adding more money to my life doesn’t easily translate into additional or more effective time (aside from transcripts and tools, which I already budget for), while translating that money back into time might make more of a difference in other people’s lives. So a direct swap doesn’t make sense.

However, if we can exchange time in an apples-and-oranges sort of way, that might make sense. That is, if someone gives me 15 minutes of their time that translates to much more than 15 minutes of my time or might even be something I could not do on my own, that would be fantastic. This could be something that takes advantage of someone’s:

  • experience or particular mix of interests
  • ideas, knowledge
  • perspective (writing, coding, and all sorts of things can be improved with the perspective of someone who is not me)
  • questions
  • connections

Technically, delegation is supposed to help me translate money into time that is qualitatively different from my time, but my anxious side has not been very good at evaluating, trusting, or making the most of learning from people who know different things than I do.

Figuring out a way to effectively receive other people’s gifts of time might be what I need to break through this barrier.

2015-01-04 Thinking in terms of an exchange of time - index card

2015.01.04 Thinking in terms of an exchange of time – index card

In fact, receiving time might be more effective than receiving money. Not only could that get around my difficulty with finding and paying other people for the qualitatively different time that I want, but if we structure it right, people will gain from the time that they give. If someone asks me a good question that prompts me to learn, reflect on, or share something, we both gain. If they invest more time into experimenting with the ideas, we gain even more. I can’t actually buy that on any of the freelancing or outsourcing marketplaces. There’s no way for me to convert money into that kind of experience.

So, how can people can give me 15 minutes of time in a way that helps them and helps me? Let me think about different things I’m learning about:

2015-01-09 Time is greater than money -- index card

2015.01.09 Time is greater than money – index card

2015-01-09 What am I learning more about, and how can people help -- index card

2015.01.09 What am I learning more about, and how can people help – index card

It makes sense to organize this by interest instead of by action.

  • Emacs: Ask a question, pass along a tip, share a workflow. Also, I really appreciate people showing up at Emacs Hangouts or being on Emacs Chats, because my anxious side is always firmly convinced that this will be the day when no one else shows up to a party or that conversation will be super-awkward.
  • Coding in general: There are so many ways I want to improve in order to become a better programmer. I should set up continuous integration, write more tests, refactor my code, learn more frameworks and learn them more deeply, write more idiomatic code, improve performance and security, get better at designing… I find it difficult to pay someone to give me feedback and coach me through setting things up well (hard to evaluate people, anxious side balks at the price and argues we can figure things out on our own, good programmers have high rates), but this might be something we can swap. Or I could work on overriding my anxious side and just Go For It, because good habits and infrastructure pay off.
  • Writing: Comments, questions, and links help a lot. A few of my posts have really benefited from people’s feedback on the content and the structure of ideas, and I’d love to learn from more conversations like that. I don’t worry a lot about typos or minor tweaks, so the kind of editing feedback I can easily get from freelancers doesn’t satisfy me. I want to get better at writing for other people and organizing more complex thoughts into resources, so I could benefit a lot from feedback, questions, as well as advice on what to learn and in what order.
  • Drawing: I’m not focused on drawing better (I can probably get away with stick figures for what I want to do!), but rather on being able to think more interesting thoughts. What would help with this? Hearing from people about which thoughts spark ideas in them, which ones I should flesh out further. Book recommendations and shared experiences would help too.

So: Paying for free/pay-what-you-want-resources is great at helping me tell my anxious side, “Look, people find this valuable,” and that’s much appreciated. But giving me time works too. If we can figure out how to do this well, that might be able to help me grow more (at least until I sort out a way to talk my anxious side into letting me invest more in capabilities). Shifting the balance towards time is probably going to make my anxious side more anxious, but I might be able to tell it to give me a year or two to experiment, which is coincidentally the rest of this 5-year span.

Wild success might look like:

  • Thanks to people’s gifts of time and attention, I’m learning and doing stuff that I couldn’t do on my own or with the resources I could get in marketplaces
  • Thanks to people’s gifts of money (and maybe teaching), I’ve addressed more of my anxious side’s concerns and am getting better at experimenting with the resources I can get in marketplaces
  • I can incorporate people’s feedback and revealed preferences in my prioritization so that I work on things that other people find valuable

I could use your help with this. =) Shall we figure it out together?

Clear out your drafts by scheduling Minimum Viable Posts

Do you have dozens of drafts languishing on your blog or on your computer?

I sometimes hear from other bloggers who say they just don’t think their posts are good enough. Maybe they’ve written a few paragraphs before fizzling out. Maybe they’ve already written a full post, but it’s missing… something.

Are you holding off because you’re a perfectionist? Although research shows that perfectionists actually procrastinate less than other people do, since blogs don’t have deadlines, it’s easy to dilly-dally. You can always make something better.

Me, I am definitely not a perfectionist. I’m generally happy if I get 80% of the way to where I want to go. But I know what it’s like to hold a post back because I’m not sure if I’m expressing myself clearly enough. As I write this, there are nineteen drafts in my WordPress interface and countless more on my computer. The oldest draft I have in WordPress is from February 2014. Come to think of it, the time for that topic has passed. Eighteen drafts now.

Do you want to know something that works better than drafting posts?

Scheduling them.

In Lean Startup, there’s this idea of a “Minimum Viable Product” – the smallest thing you can build so that you can test your business assumptions and get feedback from real customers. You can use this in writing, too.

Instead of finely crafting and endlessly polishing each blog post, I write a blog post that I’m reasonably–not completely, just reasonably–happy with. I schedule it a few weeks out. You can do that from the WordPress edit screen – click on Edit near Publish immediately and change it to the date you want. It’s even easier with the Editorial Calendar plugin, which lets you spread posts over weeks.

So then I have this imperfect post that will be published even if I forget about it. My mind keeps working on it in the meantime. (Now that I’ve learned about the Zeigarnik effect, I see it everywhere.) Sometimes I come up with a thought I’d like to add. I might share a scheduled post using the Share a Draft plugin. Maybe I’ll re-read the post and find a typo to fix or a gap to fill.

And hey, even if it isn’t the height of perfection when it finally gets published, at least it’s out there. Then people can tell me what they found interesting or ask questions about what they didn’t understand.

2015-01-10 Writing into the future -- index card #writing #blogging

2015-01-10 The idea of the Minimum Viable Post -- index card #writing #blogging

You don’t get that feedback if your thoughts are stuck in your drafts.

But what if you make a mistake? Edit your post. Even if you’ve already published it, you can still edit it. (Many people add the date and a description of what they changed.)

What if you turned out to be completely wrong about something? At least you learned you were wrong.

What if you skipped over some things that you could have explained? Let someone ask questions and pull that information from you.

Write (and schedule!) minimum viable posts: the simplest, roughest cut of your ideas that will move you towards learning. You can treat the schedule as your new deadline for improving the post.

Resist the temptation to reschedule posts again and again. If the deadline is here and you still can’t quite settle on your post, publish it as is. Then listen to your dissatisfaction for clues to how you can improve the next post.

Get stuff out there. Good luck!

p.s. Right after I scheduled this post with the title “Clear out your drafts by writing Minimum Viable Posts,” I realized it made more sense to title it “Clear out your drafts by scheduling Minimum Viable Posts.” So I changed it. See? We can harness the power of that inner “Wait just a minute here!” =)

De-dupe and link: Using the Flickr API to neaten up my archive and link sketches to blog posts

I’ve been thinking about how to manage the relationships between my blog posts and my Flickr sketches. Here’s the flow of information:

2015-01-06 Figuring out information flow -- index card

2015.01.06 Figuring out information flow – index card

I scan my sketches or draw them on the computer, and then I upload these sketches to Flickr using photoSync, which synchronizes folders with albums. I include these sketches in my outlines and blog posts, and I update my index of blog posts every month. I recently added a tweak to make it possible for people to go from a blog post to its index entry, so it should be easier to see a post in context. I’ve been thinking about keeping an additional info index to manage blog posts and sketches, including unpublished ones. We’ll see how well that works. Lastly, I want to link my Flickr posts to my blog posts so that people can see the context of the sketch.

My higher goal is to be able to easily see the open ideas that I haven’t summarized or linked to yet. There’s no shortage of new ideas, but it might be interesting to revisit old ones that had a chance to simmer a bit. I wrote a little about this in Learning from artists: Making studies of ideas. Let me flesh out what I want this archive to be like.

2015-01-05 Thinking about my archive -- index card
2015.01.05 Thinking about my archive

When I pull on an idea, I’d like to be able to see other open topics attached to it. I also want to be able to see open topics that might jog my memory.

How about the technical details? How can I organize my data so that I can get what I want from it?

2015-01-05 Figuring out the technical details of this idea or visual archive I want -- index card
2015.01.05 Figuring out the technical details of this idea or visual archive I want – index card

Because blog posts link to sketches and other blog posts, I can model this as a directed graph. When I initially drew this, I thought I might be able to get away with an acyclic graph (no loops). However, since I habitually link to future posts (the time traveller’s problem!), I can’t make that simplifying assumption. In addition, a single item might be linked from multiple things, so it’s not a simple tree (and therefore I can’t use an outline). I’ll probably start by extracting all the link information from my blog posts and then figuring out some kind of Org Mode-based way to update the graph.

2015-01-07 Mapping the connections in my blog -- index card
2015.01.07 Mapping the connections in my blog – index card

To get one step closer to being able to see open thoughts and relationships, I decided that my sketches on Flickr:

  • should not have duplicates despite my past mess-ups, so that:
    • I can have an accurate count
    • it’s easier for me to categorize
    • people get less confused
  • should have hi-res versions if possible, despite the IFTTT recipe I tried that imported blog posts but unfortunately picked up the low-res thumbnails instead of the hi-res links
  • should link to the blog posts they’re mentioned in, so that:
    • people can read more details if they come across a sketch in a search
    • I can keep track of which sketches haven’t been blogged yet

I couldn’t escape doing a bit of manual cleaning up, but I knew I could automate most of the fiddly bits. I installed node-flickrapi and cheerio (for HTML parsing), and started playing.

Removing duplicates

Most of the duplicates had resulted from the Great Renaming, when I added tags in the form of #tag1 #tag2 etc. to selected filenames. It turns out that adding these tags en-masse using Emacs’ writable Dired mode broke photoSync’s ability to recognize the renamed files. As a result, I had files like this:

  • 2013-05-17 How I set up Autodesk Sketchbook Pro for sketchnoting.png
  • 2013-05-17 How I set up Autodesk Sketchbook Pro for sketchnoting #tech #autodesk-sketchbook-pro #drawing.png

This is neatly resolved by the following Javascript:

exports.trimTitle = function(str) {
    return str.replace(/ --.*$/g, '').replace(/#[^ ]+/g, '').replace(/[- _]/g, '');
};

and a comparison function that compared the titles and IDs of two photos:

exports.keepNewPhoto = function(oldPhoto, newPhoto) {
    if (newPhoto.title.length > oldPhoto.title.length)
        return true;
    if (newPhoto.title.length < oldPhoto.title.length)
        return false;
    if (newPhoto.id < oldPhoto.id) 
        return true;
    return false;
};

So then this code can process the photos:

exports.processPhoto = function(p, flickr) {
    var trimmed = exports.trimTitle(p.title);
    if (trimmed && hash[trimmed] && p.id != hash[trimmed].id) {
        // We keep the one with the longer title or the newer date
        if (exports.keepNewPhoto(hash[trimmed], p)) {
            exports.possiblyDeletePhoto(hash[trimmed], flickr);
            hash[trimmed] = p;
        }
        else if (p.id != hash[trimmed].id) {
            exports.possiblyDeletePhoto(p, flickr);
        }
    } else {
        hash[trimmed] = p;
    }
};

You can see the code on Gist: duplicate_checker.js.

High-resolution versions

I couldn’t easily automate this, but fortunately, the IFTTT script had only imported twenty images or so, clearly marked by a description that said: “via sacha chua :: living an awesome life…”. I searched for each image, deleting the low-res entry if a high-resolution image was already in the system and replacing the low-res entry if that was the only one there.

Linking to blog posts

This was the trickiest part, but also the most fun. I took advantage of the fact that WordPress transforms uploaded filenames in a mostly consistent way. I’d previously added a bulk view that displayed any number of blog posts with very little additional markup, and I modified the relevant code in my theme to make parsing easier.

See this on Gist:

/**
 * Adds "Blogged" links to Flickr for images that don't yet have "Blogged" in their description.
 * Command-line argument: URL to retrieve and parse
 */

var secret = require('./secret');
var flickrOptions = secret.flickrOptions;
var Flickr = require("flickrapi");
var fs = require('fs');
var request = require('request');
var cheerio = require('cheerio');
var imageData = {};
var $;

function setDescriptionsFromURL(url) {
  request(url, function(error, response, body) {
    // Parse the images
    $ = cheerio.load(body);
    $('article').each(function() {
      var prettyLink = $(this).find("h2 a").attr("href");
      if (!prettyLink.match(/weekly/i) && !prettyLink.match(/monthly/i)) {
        collectLinks($(this), prettyLink, imageData);
      }
    });
    updateFlickrPhotos();
  });
}

function updateFlickrPhotos() {
    Flickr.authenticate(flickrOptions, function(error, flickr) {
      flickr.photos.search(
        {user_id: flickrOptions.user_id,
         per_page: 500,
         extras: 'description',
         text: ' -blogged'}, function(err, result) {
           processPage(result, flickr);
           for (var i = 2 ; i < result.photos.pages; i++) {
             flickr.photos.search(
               {user_id: flickrOptions.user_id, per_page: 500, page: i,
                extras: 'description', text: ' -blogged'},
               function(err, result) {
                 processPage(err, result, flickr);
               });
           }
         });
    });
}

function collectLinks(article, prettyLink, imageData) {
  var results = [];
  article.find(".body a").each(function() {
    var link = $(this);
    if (link.attr('href')) {
      if (link.attr('href').match(/sachachua/)
          || !link.attr('href').match(/^http/)) {
        imageData[exports.trimTitle(link.attr('href'))] = prettyLink;
      } else if (link.attr('href').match(/flickr.com/)) {
        imageData[exports.trimTitle(link.text())] = prettyLink;
      }
    }
  });
  return results;
}

exports.trimTitle = function(str) {
  return str.replace(/^.*\//, '').replace(/^wpid-/g, '').replace(/[^A-Za-z0-9]/g, '').replace(/png$/, '').replace(/[0-9]$/, '');
};

function processPage(result, flickr) {
  if (!result) return;
  for (var i = 0; i < result.photos.photo.length; i++) {
    var p = result.photos.photo[i];
    var trimmed = exports.trimTitle(p.title);
    var noTags = trimmed.replace(/#.*/g, '');
    var withTags = trimmed.replace(/#/g, '');
    var found = imageData[noTags] || imageData[withTags];
    if (found) {
      var description = p.description._content;
      if (description.match(found)) continue;
      if (description) {
        description += " - ";
      }
      description += '<a href="' + found + '">Blogged</a>';
      console.log("Updating " + p.title + " with " + description);
      flickr.photos.setMeta(
        {photo_id: p.id,
         description: description},
        function(err, res) {
          if (err) { console.log(err, res); }
        } );
    }
  }
}

setDescriptionsFromURL(process.argv[2]);

And now sketches like 2013-11-11 How to think about a book while reading it are now properly linked to their blog posts. Yay! Again, this script won’t get everything, but it gets a decent number automatically sorted out.

Next steps:

  • Run the image extraction and set description scripts monthly as part of my indexing process
  • Check my list of blogged images to see if they’re matched up with Flickr sketches, so that I can identify images mysteriously missing from my sketchbook archive or not correctly linked

Yay code!