Working with fragmented thoughts

Some days it’s hard to hold a single thought and dive deeper into it. Sometimes it’s because I get distracted by other shiny thoughts. Sometimes my interest peters out. Sometimes I bump into the limit of what I can think about on my own, without experiments or research.

I’ve come to really like the way index cards let me capture ideas that aren’t quite blog-post-sized. Technically, I haven’t drawn a physical index card since early February, but the digital index cards I draw are calibrated to that scale.

Still, some days it takes me a really long time to draw five index cards. I catch myself wondering if I’ve picked a good question. Sometimes it takes a while to find the next step in the thought. Sometimes it’s easier to let my attention drift to other things.

On the other hand, there are some days when my mind is overflowing with little thoughts. It’s pretty easy for me to switch to another index card, scribble down part of a thought, and then come back to it later.

2015-06-01e Fragmented writing and drawing -- index card #fuzzy #fatigue #writing #drawing #fragmentation

2015-06-01e Fragmented writing and drawing – index card #fuzzy #fatigue #writing #drawing #fragmentation

I’ve been figuring out a better way to work with fragmented thoughts. I tried flipping my habit by writing before drawing. Sometimes that’s a good way to clear my backlog, but sometimes it means I don’t get around to drawing.

Lately I’ve been experimenting with quickly capturing text fragments – a chunk even smaller than index cards. A few taps on my phone bring up a single-line prompt. Whatever I type into that dialog gets saved to a timestamped file named something like yyyy-mm-dd hh.mm timestamp - keyword.txt, and that’s synchronized over Dropbox to my computer. I have some code in Emacs to read those files and add them to a date-based outline, and I’ve included the code at the end of this blog post just in case it’s handy.

I’ve found myself capturing more and more of these snippets these days. When a possibly interesting thought occurs to me while I’m walking around, it’s easy enough to take a moment to unlock my phone and add a note. My Emacs-based workflow fits me a bit better than the Evernote-based one I used to use, but that’s the benefit of customization.

2015-05-24e Working with surface thoughts -- index card #fuzzy #drawing #thinking

2015-05-24e Working with surface thoughts – index card #fuzzy #drawing #thinking

There’s still the challenge of bringing those thoughts together, of course. The text titles and fragment keywords are often enough to remind me of what I was thinking and how the different thoughts might be connected to each other, and I can always open the sketches in a new window if I want to refer to them. I have an ever-growing outline of sketches that haven’t yet been chunked into blog posts, and now I have a chronological tree of these little fragments. I have another bit of Emacs Lisp that lets me quickly get a montage of the sketches listed in part of my outline. Maybe I could use that more often – perhaps even randomly picking an outline node, coming up with a montage, and prompting me to either glue the chunks together into a blog post or draw whatever’s missing.

So this is what the index card workflow looks like as a whole:

2015-05-08b My index card management system -- index card #zettelkasten #workflow #index-cards #drawing

2015-05-08b My index card management system – index card #zettelkasten #workflow #index-cards #drawing

and then the text fragments feed into the beginning of that thinking process.

It’s been almost six months of thinking with index cards. I sometimes feel pretty fragmented, but there are confounding factors so I don’t know whether that’s a side-effect of this way of thinking. But I think it’s unlikely that my past self was that much more coherent and better at concentrating. Remembering what it was like to write my notes before and what it’s like to write my notes now, I think I like this way a lot. I feel like I’m getting better at writing about the small things, not just the big things, and I’m gradually getting better at tying things together.

What might be some interesting next steps for this system?

2015-06-12h 6-month reflection on index cards -- index card #index-cards #drawing #zettelkasten #chunking

2015-06-12h 6-month reflection on index cards – index card #index-cards #drawing #zettelkasten #chunking

It might be cool to visualize how much has been chunked and what’s still isolated, in a way that’s more engaging than my outline. I’m also curious about the time separation of thoughts. For example, this post brings together four cards spread over a little more than a month, a set of connections I probably wouldn’t have been able to follow without these notes.

The fragment code I mentioned:

(defun my/read-phone-entries ()
  "Copy phone data to a summary Org file."
  (interactive)
  (mapc
   (lambda (filename)
     (let ((base (file-name-base filename)) contents timestamp category encoded-time date)
       (when (string-match "^[^ ]+ [^ ]+ \\([^ ]+\\) - \\(.*\\)" base)
         (setq time (seconds-to-time (/ (string-to-number (match-string 1 base)) 1000))
               encoded-time (decode-time time)
               date (list (elt encoded-time 4) (elt encoded-time 3) (elt encoded-time 5))
               category (match-string 2 base))
         (with-temp-buffer
           (insert-file-contents filename)
           (setq contents (s-trim (buffer-string))))
         (with-current-buffer
             (find-file "~/dropbox/tasker/summary.txt")
           (org-datetree-find-date-create date)
           (unless (save-excursion (re-search-forward (regexp-quote base) nil t))
             (goto-char (line-end-position))
             (insert "\n")
             (insert "**** " contents "  :" category ":\n" base "\n")
             (insert (format-time-string "[%Y-%m-%d %a %H:%M]\n" time))

             (if (member category '("Think" "Do"))
                 (save-excursion
                   (org-back-to-heading t)
                   (if (looking-at org-outline-regexp) (goto-char (1- (match-end 0))))
                   (unless (looking-at org-todo-regexp)
                     (org-todo "TODO"))))
             (if (string-match "^Energy \\([0-9]\\)" contents)
                 (org-set-property "ENERGY" (match-string 1 contents)))))
         (delete-file filename))))
   (directory-files "~/dropbox/tasker/data" t "\\.txt$")))

Using your own Emacs Lisp functions in Org Mode table calculations: easier dosage totals

UPDATE 2015-06-17: In the comments below, Will points out that if you use proper dates ([yyyy-mm-dd] instead of yyyy-mm-dd), Org will do the date arithmetic for you. Neato! Here’s what Will said:

Hi Sacha. Did you know you can do date arithmetic directly on org’s inactive or active timestamps? It can even give you an answer in fractional days if the time of day is different in the two timestamps:

| Start                  | End                    | Interval |
|------------------------+------------------------+----------|
| [2015-06-16 Tue]       | [2015-06-23 Tue]       |        7 |
| <2015-06-13 Sat>       | <2015-06-15 Mon>       |        2 |
| [2015-06-10 Wed 20:00] | [2015-06-17 Wed 08:00] |      6.5 |
#+TBLFM: $3=$2 - $1 

Here’s my previous convoluted way of doing things… =)
—-

I recently wrote about calculating how many doses you need to buy using an Org Mode table. On reflection, it’s easier and more flexible to do that calculation using an Emacs Lisp function instead of writing a function that processes and outputs entire tables.

First, we define a function that calculates the number of days between two dates, including the dates given. I put this in my Emacs config.

(defun my/org-days-between (start end)
  "Number of days between START and END.
This includes START and END."
  (1+ (- (calendar-absolute-from-gregorian (org-date-to-gregorian end))
         (calendar-absolute-from-gregorian (org-date-to-gregorian start)))))

Here’s the revised table. I moved the “Needed” column to the left of the medication type because this makes it much easier to read and confirm.

| Needed | Type         | Per day |      Start |        End | Stock |
|--------+--------------+---------+------------+------------+-------|
|     30 | Medication A |       2 | 2015-06-16 | 2015-06-30 |     0 |
|      2 | Medication B |     0.1 | 2015-06-16 | 2015-06-30 |   0.2 |
#+TBLFM: @2$1..@>$1='(ceiling (- (* (my/org-days-between $4 $5) (string-to-number $3)) (string-to-number $6)))

C-c C-c on the #+TBLFM: line updates the values in column 1.

@2$1..@>$1 means the cells from the second row (@2) to the last row (@>) in the first column ($1).  '  tells Org to evaluate the following expression as Emacs Lisp, substituting the values as specified ($4 is the fourth column’s value, etc.).

The table formula calculates the value of the first column (Needed) based on how many you need per day, the dates given (inclusive), and how much you already have in stock. It rounds numbers up by using the ceiling function.

Because this equation uses the values from each row, the start and end date must be filled in for all rows. To quickly duplicate values downwards, set org-table-copy-increment to nil, then use S-return (shift-return) in the table cell you want to copy. Keep typing S-return to copy more.

This treats the calculation inputs as strings, so I used string-to-number to convert some of them to numbers for multiplication and subtraction. If you were only dealing with numbers, you can convert them automatically by using the ;N flag, like this:

| Needed | Type         | Per day | Days | Stock |
|--------+--------------+---------+------+-------|
|      6 | Medication A |       2 |    3 |     0 |
|      1 | Medication B |     0.1 |    3 |   0.2 |
#+TBLFM: @2$1..@>$1='(ceiling (- (* $3 $4) $5)));N

Providing values to functions in org-capture-templates

Over at the Emacs StackExchange, Raam Dev asked how to define functions for org-capture-templates that could take arguments. For example, it would be useful to have a function that creates a Ledger entry for the specified account. Functions used in org-capture-templates can’t take any arguments, but you can use property lists instead. Here’s the answer I posted.

You can specify your own properties in the property list for the template, and then you can access those properties with plist-get and org-capture-plist. Here’s a brief example:

Here’s a brief example:

(defun my/expense-template ()
  (format "Hello world %s" (plist-get org-capture-plist :account)))
(setq org-capture-templates '(("x" "Test entry 1" plain
                               (file "~/tmp/test.txt")
                               (function my/expense-template)
                               :account "Account:Bank")
                              ("y" "Test entry 2" plain
                               (file "~/tmp/test.txt")
                               (function my/expense-template)
                               :account "Account:AnotherBank")))

I hope that helps!

Weekly review: Week ending June 12, 2015

It was a good week for small steps. I’ve been settling into a new daily routine: a little bit of code, a book, some thinking… In terms of code, I’ve been focusing on adding D3 visualizations and other smarts to my Angular+Node tracker. I’ve been reading about thinking and memory, and thinking about what I can build or practice in order to get better at those things. Overall, though, I continue to be taking it easy – letting go of more of my previous commitments, clearing the way for other experiments.

2015-06-14c Week ending 2015-06-12 -- index card #journal #weekly

output

Blog posts

Sketches

Link round-up

  • 5 reasons not to use your virtual assistant: the Amy scheduling AI looks interesting. It would be neat to check that out when it’s more open. Interesting approach (probably hybrid person+AI) and conversational interface. I wonder what I can build and use along those lines…

Focus areas and time review

  • Business (23.0h – 13%)
    • Earn (6.0h – 25% of Business)
      • Earn: E1: 1-2 days of consulting
      • Earn: E1: 1-2 days of consulting
    • Build (14.2h – 61% of Business)
      • Drawing (9.4h)
      • Paperwork (0.0h)
    • Connect (2.8h – 12% of Business)
  • Relationships (10.6h – 6%)
  • Discretionary – Productive (11.6h – 6%)
    • Emacs (0.7h – 0% of all)
      • Reschedule Emacs Lisp Development Tips episode with jwiegley
      • Announce Emacs Hangout 2015-06-17
      • Host Emacs Hangout
    • Writing (4.5h)
    • Get travis working again for quantified awesome
    • Call with new totals
  • Discretionary – Play (20.9h – 12%)
  • Personal routines (28.3h – 16%)
  • Unpaid work (14.5h – 8%)
  • Sleep (59.2h – 35% – average of 8.5 per day)

Growth, experiments, and shifting my preferences

I’ve been thinking about how to respond to e-mails from former virtual assistants who are looking for additional work. I remember what it was like to feel that the world was a candy store of talent. My experiments with delegation led to interesting experiences. But at the moment, I can’t really think of tasks that I want to specify or how much I would value someone else doing them.

Besides, I tend to get rid of tasks or write programs before I consider paying people to do things, so that tends to get in the way of delegation experiments. I find it more difficult to give instructions to people than to computers.

2015-02-03 Delegation and dreaming small dreams -- index card #delegation

2015-02-03 Delegation and dreaming small dreams – index card #delegation

I tried thinking about ways I want to improve my life at the moment, and how I might want to accelerate those improvements. Compared to my answers from 2013, my current ideas feel closer to where I am, less of a stretch.

2015-06-12a Questions to revisit -- index card #kaizen #experiment #delegation

2015-06-12a Questions to revisit – index card #kaizen #experiment #delegation

Considering various ideas, I catch myself thinking, “Well, that would be nice to experience/be/have and I can imagine being happy with that, but I can also imagine being happy without that.” I wondered whether that detachment came came out of avoidance or peace, and whether I wanted to tweak my balance of detachment and desire – ambition can also be quite a handy thing.

I know that a fuzzy brain dampens my ability to plan and anticipate. This is normal. I also know the fuzziness is temporary, so I’m not too worried about it. Still, I find it interesting to explore.

2015-06-12b Finding my own balance between desire and detachment -- index card #stoicism #philosophy #detachment #desire

2015-06-12b Finding my own balance between desire and detachment – index card #stoicism #philosophy #detachment #desire

2015-06-12c Exploring this distance -- index card #stoicism #philosophy #detachment #desire

2015-06-12c Exploring this distance – index card #stoicism #philosophy #detachment #desire

On reflection, I think it’s less about avoidance / running away from, and maybe more about not preferring something as much as I think I should. Consciously developing your preferences is an idea from Stoicism that I’d like to explore a little more.

2015-06-12d When I don't prefer something as much as I think I should -- index card #stoicism #philosophy #preference

2015-06-12d When I don’t prefer something as much as I think I should – index card #stoicism #philosophy #preference

“Should” is a funny word, anyway. I avoid using that word with other people, but I sometimes still slip up and use that word for myself. So maybe it’s more like I think it might be interesting if I had stronger, clearer preferences for things that were generally acknowledged to be good, but if I don’t, that’s more of an opportunity for learning than a personal failure.

In addition to general fuzziness, I think that gap happens when I have conflicting factors or motivations, and when I underestimate benefits or overestimate costs. I can untangle conflicting factors with reflection and honesty, even if sometimes that leads to uncomfortable realizations about my current self. I tend to overestimate costs more than I underestimate benefits, especially in terms of energy. In any case, my perception of one affects the other. I can work around this by giving things a shot, like the way skills often become more enjoyable the better you get at them.

Even when I have a good idea of the benefits and costs of different choices, sometimes it would be better for me to prefer things that have a lower short-term value than other things I could do.

A tangent: This might be pretty similar to how startups disrupt incumbent companies, actually. An incumbent company initially has lower marginal costs because of its investments. It would be more expensive for that company to shift to a new technology. On the other hand, a start-up doesn’t have those sunk costs, so it’s easier to invest in new technologies. Some startups succeed, getting to the point where they can beat the old technology in terms of return on investment. Other startups fail. But it’s hard to tell which is which until you try, so it makes sense to have a bunch of start-up-like experiments even in larger companies.

2015-06-12e Thinking about disrupting myself -- index card #experiment #disruption

2015-06-12e Thinking about disrupting myself – index card #experiment #disruption

So, what would it be like to use these tools to develop my preferences? There’s the slow evolution of my preferences through reflection and incremental improvement. At the same time, it might be interesting to mentally budget X% of my time for exploring things even if I feel a little meh about them: not just “Hell, yeah” or No, but also things I still feel mediocre at. (‘Cause you don’t get to awesome without being mediocre first!) Doing those small experiments to play with my understanding and preferences might even be easier during fuzzy times than during sharp times, since my opportunity costs are lower.

I might keep my goals and experiments a little close to myself at the moment, focusing on elimination and automation rather than delegation. Maybe I’ll branch out again when I have a little more brainspace to manage and train people, since I don’t want to get to the point where I resent other people because of the consequences of my own mediocrity in delegation. In the meantime, little by little, I’d like to get better at understanding my preferences, and maybe shifting them ever so slightly.

Thinking about problem-solving and sequencing

We’ve been helping J- with her culminating project in Grade 11 programming class, a text-based blackjack game. She was getting stuck because she wasn’t comfortable enough with Java or programming. She didn’t know where to start or what to do. I didn’t write the program for her (and that was never on the table, anyway), but I offered to guide her through the process. More experienced learners can do that kind of planning on their own, but when you’re new to a subject (and especially if you’re under time pressure), it helps to have that kind of scaffolding.

The first thing I did was to properly set up Eclipse on her computer. She had already tried setting it up, but it kept quitting with a cryptic error message. I realized that she didn’t have the Java software development kit installed. Once we added that, Eclipse started working.

Then I set up a sequence of tiny, tiny steps that she could implement with a little guidance. This was not the same sequence of events that would be in the final game: betting, shuffling, dealing, scoring, and so on. Instead, I focused on the steps she could build on bit by bit. It went something like this:

  1. Display a random number.
  2. Initialize the deck array based on the face array and suit array J- had already set up. This would contain strings like “ace of hearts”.
  3. Print the deck array.
  4. Set up a parallel array of card values, and print that as well. This would contain integers such as 1 for ace.
  5. Shuffle the deck by swapping each card with a random card, swapping the same card’s value as well.
  6. Write a function that takes an array of card values and returns the value of the hand.
  7. Modify the function to take aces into account, since aces could be either 1 or 11.
  8. Handle the initial dealing of two cards each by keeping track of the top card and updating a status array.
  9. Display the cards for a specified hand based on the status array.
  10. Update the hand value function to take the status array into account.
  11. Deal cards to the player as requested.
  12. Check if the player has lost.
  13. Follow the rules for dealing additional cards to the dealer.
  14. Check if the dealer has lost.
  15. Figure out who won that round.
  16. Read the bet.
  17. Update the bet after the player has won or lost the round.
  18. Check if the player has doubled their money or lost all their money.
  19. Add more error-checking.
  20. Add more flexibility.

2015-06-11a Blackjack implementation sequence -- index card #sequencing #problem-solving

2015-06-11a Blackjack implementation sequence – index card #sequencing #problem-solving

This sequence meant that she could write and test the less-interactive parts first (preparing the deck, shuffling the cards, calculating the score) before slowing down her compile-run-test cycle with input. It also let her work on the simplest parts first without writing a lot of redundant code and with just a little prompting from me.

Instead of this zigzag path, we could have followed the chronological flow of the program, especially if I introduced her to the practice of stubbing functions until you’re ready to work on them. This shuffled order felt a bit better in terms of demonstrable progress, and she seems to have been able to follow along with the construction process. In retrospect, the chronological flow might have been easier for her to learn and apply to other projects, though, since breaking down and shuffling the order of tasks is a skill that programming newbies probably need a while to develop.

Anyway, helping J- with her project got me thinking about how I work my way through programming challenges on my personal projects and in my consulting. I tend to try to figure things out by myself instead of asking mentors or the Internet. I also tend to write things in a bottom-up instead of top-down order: starting with small things I can write and test quickly, and then gradually building more elaborate processes around them.

2015-06-09e What do I mean by sequencing -- index card #learning #problem-solving #sequencing

2015-06-09e What do I mean by sequencing – index card #learning #problem-solving #sequencing

I think of the process of breaking down tasks and organizing them into a useful order as “sequencing”, which is part of the general domain of problem-solving. There’s probably an official term for it, but until I find it, that’s the term makes sense to me. When I was teaching undergraduate computer science, I noticed that students often struggled with the following aspects:

  • Imagining a good goal, if they were doing a self-directed project (not too big, not too small, etc.)
  • Fleshing out their goal into specifications
  • Identifying possible chunks along the path
  • Breaking those chunks down into smaller chunks as needed
  • Figuring out how to bridge those chunks together
  • Prioritizing chunks
  • Re-evaluating paths, progress, and goals as they learned more
  • Managing motivation
  • Finding resources
  • Translating or adapting those resources to what they needed to do
  • Figuring out what they could start with (what they already knew, or that first hand-hold on a chunk)

2015-06-09h Sequencing challenges -- index card #learning #problem-solving #sequencing #challenges

2015-06-09h Sequencing challenges – index card #learning #problem-solving #sequencing #challenges

I sometimes have a hard time with these aspects too, especially when I’m learning a new toolkit or language. I think I’m getting better at picking ridiculously tiny steps and celebrating that progress instead of getting frustrated by blocks. When I can’t figure tiny steps out, I know that going through tutorials and reading other people’s code will help me build my vocabulary. That way, I can get better at finding resources and planning chunks. I have a better idea of what’s out there, what things are called, and maybe even what’s probably harder and what’s probably easier.

2015-06-09f How do I develop my own sequencing skills -- index card #learning #problem-solving #sequencing

2015-06-09f How do I develop my own sequencing skills – index card #learning #problem-solving #sequencing

Programming gives me plenty of opportunities to develop my sequencing skills. By writing notes with embedded code snippets and TODOs, I can break large chunks down into smaller chunks that fit in my working memory. Sometimes I read programming documentation or source code without any particular project in mind, just collecting ideas and expanding my imagination. If I focus on writing tiny chunks that start off as “just good enough” rather than making them as elaborate as I can, that lets me move on to the rest of the program and get feedback faster. Saving snippets of unused or evolving code (either manually in my notes or automatically via a version contro system) lets me reuse them elsewhere.

I imagine that getting even better at sequencing might involve:

  • Sharing intermediate or neighbouring products to get feedback faster and create more value
  • Figuring out (and remembering!) sequences for getting into various areas such as Angular or D3
  • Becoming fluent with many styles of sequencing, like working top-down instead of just bottom-up
  • Building reliable automated tests along the way, even as my understanding of the project evolves

What about helping other people get better at sequencing?

2015-06-09g How can I help other people develop sequencing skills -- index card #learning #problem-solving #sequencing

2015-06-09g How can I help other people develop sequencing skills – index card #learning #problem-solving #sequencing

  1. If I share a pre-defined sequence, that helps people with less experience follow a possibly more efficient path.
  2. If I customize the sequence for someone I’m helping, that’s even more useful.
  3. Talking about how we’re customizing the sequence exposes sequencing as a skill.
  4. Guiding someone through the breakdown of a small chunk (from a task to pseudocode, for example) can help them get the hang of turning things into specifics.
  5. Guiding someone through the breakdown of a larger chunk into smaller chunks helps them think of larger chunks as doable.
  6. Once someone has identified a few chunks, I can help with prioritizing them based on effort, relationship between chunks, etc.
  7. In preparation for independent learning, it’s good to practice the skill of figuring out those prioritization factors: small experiments, research, and so on.
  8. And at some point (woohoo!), I can help someone reflect on and tweak a plan they developed.

I’m not yet at the point of being able to do even step 1 well, but maybe someday (with lots of practice)!

Since this is a super-useful skill and not everyone’s into programming, it might be interesting to look around for non-programming situations that can help develop sequencing skills.

2015-06-09i Non-programming examples of sequencing -- index card #learning #problem-solving #sequencing

2015-06-09i Non-programming examples of sequencing – index card #learning #problem-solving #sequencing

On the passive side, it’s interesting to learn about how things work or how they’re made. Better yet, you can actively practise sequencing with math problems, video games with quests… Writing is like this too, if you think of key points and use tools such as outlines or index cards. Making things involves imagining what you want and figuring out lots of small steps along the way. Life is filled with little opportunities for everyday problem-solving.

I’d love to learn more about this. Chunk decomposition? Problem-solving order? Problem decomposition? Step-wise refinement? I’m more comfortable with bottom-up problem-solving, but top-down solving has its merits (focus, for one), so maybe I can practice that too. Using both approaches can be helpful. (Ooh, there’s a Wikipedia article on top-down and bottom-up design…) Time to check out research into the psychology of problem-solving!