Weekly review: Week ending June 19, 2015

I haven’t edited the Emacs Hangout video yet, but I should get around to that sometime this week. Aside from that, the week has been pretty low-key. W- and I picked up LEGO Jurassic World (PS3 and PS Vita) and have been playing it. Galumphing around as a triceratops or smashing things up as a T-rex is surprisingly satisfying.

2015-06-22a Week ending 2015-06-19 -- index card #journal #weekly output

Blog posts

Sketches

Focus areas and time review

  • Business (25.2h – 14%)
    • Earn (9.1h – 36% of Business)
      • Earn: E1: 1-2 days of consulting
    • Build (16.0h – 63% of Business)
      • Drawing (7.7h)
      • Paperwork (1.3h)
      • Amazon search and replace
    • Connect (0.0h – 0% of Business)
      • Chat with Philip C about Quantified Awesome
  • Relationships (8.9h – 5%)
    • Call with new totals
    • Meet M
  • Discretionary – Productive (7.1h – 4%)
    • Emacs (2.3h – 1% of all)
      • Announce Emacs Hangout 2015-06-17
      • Host Emacs Hangout
      • Edit Emacs Hangout video to blur avy-jump demo
    • Writing (2.7h)
    • Add a summary bar chart of category totals
    • Get grunt working again
    • Add history to refiling
    • Add Jurassic Park to my tracker
    • fix sleep track activity
    • fix categories so that it returns empty ones too
    • Convert icicle chart into a sunburst
    • Switch to hierarchical bar chart
    • Add time filter to date
  • Discretionary – Play (22.3h – 13%)
  • Personal routines (27.2h – 16%)
  • Unpaid work (14.6h – 8%)
  • Sleep (62.7h – 37% – average of 9.0 per day)

Moving past getting things done

2015-06-19a Moving past getting things done -- index card #present #mindset #being

2015-06-19a Moving past getting things done – index card #present #mindset #being

When I have a lot of energy, it’s easy to do good things for my consulting clients or on my personal projects. This energizes me further, and so on. This is a good cycle.

When I’m feeling blah, or when I don’t feel like I’m accomplishing interesting things, I tend to feel even more blah – even if I know that the difficulties are temporary, local, and impersonal.

I realized that my feelings about my days tend to be influenced by whether I made progress. This makes sense; there’s even a book about it.

2015-01-07 Sketched Book - The Progress Principle - Using Small Wins to Ignite Joy, Engagement, and Creativity at Work - Teresa Amabile, Steven Kramer

2015-01-07 Sketched Book – The Progress Principle – Using Small Wins to Ignite Joy, Engagement, and Creativity at Work – Teresa Amabile, Steven Kramer

While it’s useful to be motivated by progress, I wonder if I can tweak my mind to get better at enjoying life even when it feels cyclic and mundane. Instead of noting just the new, non-routine tasks of the day, I could reflect on whether I’m getting better at routine stuff like self-care – to enjoy being, not just doing.

2015-06-19c The gap between who I am and who I wish I was -- index card #gap #mindset

2015-06-19c The gap between who I am and who I wish I was – index card #gap #mindset

I find it a little difficult to relax into this mindset, though. Part of me is pulled towards the satisfaction of making progress, and I find myself wishing I could be better at that. I could improve my skills. I could make things better at home. I could be more energetic. I could cover more ground.

2015-06-19d Maybe the delta is okay -- index card #gap #mindset

2015-06-19d Maybe the delta is okay – index card #gap #mindset

But then again, maybe the outcomes of this hypothetical self and my current self are not that different. Sure, it would be nice to make all the progress a hypothetical me could make. But whatever’s important can be handled by other people, and whatever’s not important isn’t worth stressing out over. Besides, this path can also be interesting.

So, back to this curious thought. What’s beyond getting things done? I’m learning things that are hard to check off a list: how to forget annoyances and frustrations, how to enjoy ripe fruits and sunshine, how to listen to the moment and the silence. How to embrace squirrel-brain, fuzzy-brain, and foggy-brain, and how to gently fan a spark of interest.

It will be worth it, I think, learning how to sit still. “Don’t just do something, sit there!”, as the flipped phrase go.

When the check-things-off part of myself gets antsy, I code or read for an hour or so. Once it’s satisfied, I explore things with payoffs that are less straightforward.

Another thing I used to be antsier about: The thought “Will I ask good-enough questions? Will I think good-enough thoughts?” intrudes less and less these days. I trust that when I sit down to draw, I’ll notice something I want to explore; and if not, it might be a good time for a walk.

2015-05-10d The best thing I can do with my time -- index card #experiment

2015-05-10d The best thing I can do with my time – index card #experiment

It might be interesting to decide, even if it’s temporary and on faith, that this is the best thing I can do with my time.

2015-06-15g Re-evaluating my experiment failure mode -- index card #experiment #failure #equanimity #premortem #narrative

2015-06-15g Re-evaluating my experiment failure mode – index card #experiment #failure #equanimity #premortem #narrative

On a larger scale, I might even become comfortable with this as the general flavour of my experiment. In the beginning, I identified “5 years and nothing to show for it, not even a good story” as one of my potential issues in my experiment pre-mortem. I feel myself starting to let go of the need for a neat story.

Getting things done is good. There are also other things that are good. I wonder what it’s like to live an awesome life, or better yet: live a life awesomely.

Finding missing dates in PostgreSQL

My analytics numbers were way off from what I expected them to be. When I did a day-by-day comparison of my numbers and the reference set of numbers, I realized that a few weeks of data were missing from the year of data I was analyzing – a couple of days here, two weeks there, and so on. I manually identified the missing dates so that I could backfill the data. Since this was the second time I ran into that problem, though, I realized I needed a better way to catch this error and identify gaps.

Initially, I verified the number of days in my PostgreSQL database table with a SQL statement along the lines of:

SELECT year, month, COUNT(*) AS num_days FROM
(SELECT date_part('year', day_ts) AS year,
 date_part('month', day_ts) AS month,
 day_ts FROM (SELECT DISTINCT day_ts FROM table_with_data) AS temp) AS temp2
ORDER BY year, month

I checked each row to see if it matched the number of days in the month.

It turns out there’s an even better way to look for missing dates. PostgreSQL has a generate_sequence command, so you can do something like this:

SELECT missing_date
FROM generate_series('2015-01-01'::date, CURRENT_DATE - INTERVAL '1 day') missing_date
WHERE missing_date NOT IN (SELECT DISTINCT day_ts FROM table_with_data)
ORDER BY missing_date

Neat, huh?

Working with fragmented thoughts

Some days it’s hard to hold a single thought and dive deeper into it. Sometimes it’s because I get distracted by other shiny thoughts. Sometimes my interest peters out. Sometimes I bump into the limit of what I can think about on my own, without experiments or research.

I’ve come to really like the way index cards let me capture ideas that aren’t quite blog-post-sized. Technically, I haven’t drawn a physical index card since early February, but the digital index cards I draw are calibrated to that scale.

Still, some days it takes me a really long time to draw five index cards. I catch myself wondering if I’ve picked a good question. Sometimes it takes a while to find the next step in the thought. Sometimes it’s easier to let my attention drift to other things.

On the other hand, there are some days when my mind is overflowing with little thoughts. It’s pretty easy for me to switch to another index card, scribble down part of a thought, and then come back to it later.

2015-06-01e Fragmented writing and drawing -- index card #fuzzy #fatigue #writing #drawing #fragmentation

2015-06-01e Fragmented writing and drawing – index card #fuzzy #fatigue #writing #drawing #fragmentation

I’ve been figuring out a better way to work with fragmented thoughts. I tried flipping my habit by writing before drawing. Sometimes that’s a good way to clear my backlog, but sometimes it means I don’t get around to drawing.

Lately I’ve been experimenting with quickly capturing text fragments – a chunk even smaller than index cards. A few taps on my phone bring up a single-line prompt. Whatever I type into that dialog gets saved to a timestamped file named something like yyyy-mm-dd hh.mm timestamp - keyword.txt, and that’s synchronized over Dropbox to my computer. I have some code in Emacs to read those files and add them to a date-based outline, and I’ve included the code at the end of this blog post just in case it’s handy.

I’ve found myself capturing more and more of these snippets these days. When a possibly interesting thought occurs to me while I’m walking around, it’s easy enough to take a moment to unlock my phone and add a note. My Emacs-based workflow fits me a bit better than the Evernote-based one I used to use, but that’s the benefit of customization.

2015-05-24e Working with surface thoughts -- index card #fuzzy #drawing #thinking

2015-05-24e Working with surface thoughts – index card #fuzzy #drawing #thinking

There’s still the challenge of bringing those thoughts together, of course. The text titles and fragment keywords are often enough to remind me of what I was thinking and how the different thoughts might be connected to each other, and I can always open the sketches in a new window if I want to refer to them. I have an ever-growing outline of sketches that haven’t yet been chunked into blog posts, and now I have a chronological tree of these little fragments. I have another bit of Emacs Lisp that lets me quickly get a montage of the sketches listed in part of my outline. Maybe I could use that more often – perhaps even randomly picking an outline node, coming up with a montage, and prompting me to either glue the chunks together into a blog post or draw whatever’s missing.

So this is what the index card workflow looks like as a whole:

2015-05-08b My index card management system -- index card #zettelkasten #workflow #index-cards #drawing

2015-05-08b My index card management system – index card #zettelkasten #workflow #index-cards #drawing

and then the text fragments feed into the beginning of that thinking process.

It’s been almost six months of thinking with index cards. I sometimes feel pretty fragmented, but there are confounding factors so I don’t know whether that’s a side-effect of this way of thinking. But I think it’s unlikely that my past self was that much more coherent and better at concentrating. Remembering what it was like to write my notes before and what it’s like to write my notes now, I think I like this way a lot. I feel like I’m getting better at writing about the small things, not just the big things, and I’m gradually getting better at tying things together.

What might be some interesting next steps for this system?

2015-06-12h 6-month reflection on index cards -- index card #index-cards #drawing #zettelkasten #chunking

2015-06-12h 6-month reflection on index cards – index card #index-cards #drawing #zettelkasten #chunking

It might be cool to visualize how much has been chunked and what’s still isolated, in a way that’s more engaging than my outline. I’m also curious about the time separation of thoughts. For example, this post brings together four cards spread over a little more than a month, a set of connections I probably wouldn’t have been able to follow without these notes.

The fragment code I mentioned:

(defun my/read-phone-entries ()
  "Copy phone data to a summary Org file."
  (interactive)
  (mapc
   (lambda (filename)
     (let ((base (file-name-base filename)) contents timestamp category encoded-time date)
       (when (string-match "^[^ ]+ [^ ]+ \\([^ ]+\\) - \\(.*\\)" base)
         (setq time (seconds-to-time (/ (string-to-number (match-string 1 base)) 1000))
               encoded-time (decode-time time)
               date (list (elt encoded-time 4) (elt encoded-time 3) (elt encoded-time 5))
               category (match-string 2 base))
         (with-temp-buffer
           (insert-file-contents filename)
           (setq contents (s-trim (buffer-string))))
         (with-current-buffer
             (find-file "~/dropbox/tasker/summary.txt")
           (org-datetree-find-date-create date)
           (unless (save-excursion (re-search-forward (regexp-quote base) nil t))
             (goto-char (line-end-position))
             (insert "\n")
             (insert "**** " contents "  :" category ":\n" base "\n")
             (insert (format-time-string "[%Y-%m-%d %a %H:%M]\n" time))

             (if (member category '("Think" "Do"))
                 (save-excursion
                   (org-back-to-heading t)
                   (if (looking-at org-outline-regexp) (goto-char (1- (match-end 0))))
                   (unless (looking-at org-todo-regexp)
                     (org-todo "TODO"))))
             (if (string-match "^Energy \\([0-9]\\)" contents)
                 (org-set-property "ENERGY" (match-string 1 contents)))))
         (delete-file filename))))
   (directory-files "~/dropbox/tasker/data" t "\\.txt$")))

Using your own Emacs Lisp functions in Org Mode table calculations: easier dosage totals

UPDATE 2015-06-17: In the comments below, Will points out that if you use proper dates ([yyyy-mm-dd] instead of yyyy-mm-dd), Org will do the date arithmetic for you. Neato! Here’s what Will said:

Hi Sacha. Did you know you can do date arithmetic directly on org’s inactive or active timestamps? It can even give you an answer in fractional days if the time of day is different in the two timestamps:

| Start                  | End                    | Interval |
|------------------------+------------------------+----------|
| [2015-06-16 Tue]       | [2015-06-23 Tue]       |        7 |
| <2015-06-13 Sat>       | <2015-06-15 Mon>       |        2 |
| [2015-06-10 Wed 20:00] | [2015-06-17 Wed 08:00] |      6.5 |
#+TBLFM: $3=$2 - $1 

Here’s my previous convoluted way of doing things… =)
—-

I recently wrote about calculating how many doses you need to buy using an Org Mode table. On reflection, it’s easier and more flexible to do that calculation using an Emacs Lisp function instead of writing a function that processes and outputs entire tables.

First, we define a function that calculates the number of days between two dates, including the dates given. I put this in my Emacs config.

(defun my/org-days-between (start end)
  "Number of days between START and END.
This includes START and END."
  (1+ (- (calendar-absolute-from-gregorian (org-date-to-gregorian end))
         (calendar-absolute-from-gregorian (org-date-to-gregorian start)))))

Here’s the revised table. I moved the “Needed” column to the left of the medication type because this makes it much easier to read and confirm.

| Needed | Type         | Per day |      Start |        End | Stock |
|--------+--------------+---------+------------+------------+-------|
|     30 | Medication A |       2 | 2015-06-16 | 2015-06-30 |     0 |
|      2 | Medication B |     0.1 | 2015-06-16 | 2015-06-30 |   0.2 |
#+TBLFM: @2$1..@>$1='(ceiling (- (* (my/org-days-between $4 $5) (string-to-number $3)) (string-to-number $6)))

C-c C-c on the #+TBLFM: line updates the values in column 1.

@2$1..@>$1 means the cells from the second row (@2) to the last row (@>) in the first column ($1).  '  tells Org to evaluate the following expression as Emacs Lisp, substituting the values as specified ($4 is the fourth column’s value, etc.).

The table formula calculates the value of the first column (Needed) based on how many you need per day, the dates given (inclusive), and how much you already have in stock. It rounds numbers up by using the ceiling function.

Because this equation uses the values from each row, the start and end date must be filled in for all rows. To quickly duplicate values downwards, set org-table-copy-increment to nil, then use S-return (shift-return) in the table cell you want to copy. Keep typing S-return to copy more.

This treats the calculation inputs as strings, so I used string-to-number to convert some of them to numbers for multiplication and subtraction. If you were only dealing with numbers, you can convert them automatically by using the ;N flag, like this:

| Needed | Type         | Per day | Days | Stock |
|--------+--------------+---------+------+-------|
|      6 | Medication A |       2 |    3 |     0 |
|      1 | Medication B |     0.1 |    3 |   0.2 |
#+TBLFM: @2$1..@>$1='(ceiling (- (* $3 $4) $5)));N

Providing values to functions in org-capture-templates

Over at the Emacs StackExchange, Raam Dev asked how to define functions for org-capture-templates that could take arguments. For example, it would be useful to have a function that creates a Ledger entry for the specified account. Functions used in org-capture-templates can’t take any arguments, but you can use property lists instead. Here’s the answer I posted.

You can specify your own properties in the property list for the template, and then you can access those properties with plist-get and org-capture-plist. Here’s a brief example:

Here’s a brief example:

(defun my/expense-template ()
  (format "Hello world %s" (plist-get org-capture-plist :account)))
(setq org-capture-templates '(("x" "Test entry 1" plain
                               (file "~/tmp/test.txt")
                               (function my/expense-template)
                               :account "Account:Bank")
                              ("y" "Test entry 2" plain
                               (file "~/tmp/test.txt")
                               (function my/expense-template)
                               :account "Account:AnotherBank")))

I hope that helps!