Category Archives: quantified

On this page:

Quantified Awesome: Added sparklines and percentages

As I was answering the standard question of “Who are you and what do you do?”, I thought it might be interesting to come up with the percentages for what I actually do based on my time records. After all, I have the data. In the past, I used to export my records to a spreadsheet and do some easy number-crunching. Why bother, though, if I can program the system to do this for me?`

I ended up spending 3.5 hours adding percentages and sparklines to Quantified Awesome, updating my RSpec tests along the way. (100% coverage, yay!) Here’s what the result looks like:

2014-06-11 13_34_46-quantified awesome

The sparklines let me easily see trends and exceptions, while percentages can be easily multiplied by 168 hours to get weekly estimates or 24 hours to get daily ones. For example, sleep took up 35.8% of my time from 2012-02-17 to 2014-06-11, or an average of 8.6 hours a day. Activities directly related to earning money took up 10.9% of my time, or roughly 18.3 hours a week. The sprints and spikes are easier to see with sparklines than with tabular data, and they were easy to implement with JQuery Sparklines.

So now I can be more accurate when answering the question: “What do you do?” It doesn’t make sense to include all the minutiae. People don’t really answer that question with “Sleep,” even though that takes up much of people’s time. However, I can pick a threshold (1%?) and look at the activities above that. That is, for the ~28 months of this 5-year experiment so far, I:

  • consult (10.3%)
  • connect with people (5.0%)
  • write (3.0%)
  • draw (2.5%)
  • cook (2.3%)
  • bike (2.0%)
  • work on Emacs (1.2%) and
  • work on Quantified Awesome and other tracking tools (1%)

Data! Mwahahaha…

Hmm, maybe I’m not slacking off after all

Even though I’ve got the steady accumulation of DONE tasks showing my slow-but-constant progress, I still sometimes feel like I’m leaving something on the table when it comes to how I use my time. I feel like I’m living with a more relaxed pace, especially compared with the world of work around me or my fuzzed-by-time recollections of pre-experiment and early-experiment days.

Top line = All tasks excluding cancelled ones, bottom line = DONE

Top line = All tasks excluding cancelled ones, bottom line = DONE

I was thinking about how my time use has shifted over the past few years. I compared my percentages in different categories for 2012, 2013, and for 2014 to date. But the numbers say I’m actually spending more time on work and personal projects, and I do seem to manage to check off lots of things on my TODO list. =) So maybe I’m doing okay with this after all, even though sometimes I think I’m slacking off.

Top-level categories:

  • Sleep: Pretty consistent (34.5-36.6%) – this works out to 8.3-8.8 hours a day.
  • Business: Down, then up lately – 24%, 21%, 26%; but I expect this to be a little lower this year, since I’m taking three months off. =) I’ll probably focus on even more writing, drawing, and Emacs geekery then. (And maybe a crash course in a useful skill…)
    Avg hours per week 2012 2013 2014 to date
    Earn 20 15 17
    Build 12 13 18
    Connect 8 7 9
    Total 40 35 44
  • Discretionary: Up, then down – 18%, 22%, 16%
  • Personal care: Pretty consistent (13-14%)
  • Chores/unpaid work: Pretty consistent (7-8%)

As before, the business/discretionary trade-off is really the main thing that moves. The rest of my life stays pretty much the same. The second level of categories is worth looking at too:

  • Writing is pretty consistent at 3%, or roughly 5 hours a week. Still, I think I’d like to write more. What should get reduced? Ah, video games have been soaking up a little time – although they’re exercise too. Hmm, I could intensify that exercise so that I get more out of it. Oh! I’ve been spending more time gardening lately; that could be another reason. I like both of those alternative activities too, and I think they’ll taper off after a while. That’s okay, there’ll be time enough to write more. Besides, some of my writing is filed under Emacs-related time instead. =)
  • Trending up:
    • Drawing (2.0-4.1%): This is good.
    • Planning (0.2-1.5%): Hmm, this is interesting. Am I running into diminishing returns here? Maybe less time planning, more time experimenting.
    • Emacs (0.4-2.8%), and I’m looking forward to spending even more time on this.
    • Relaxing (0.6-2.0%)
  • Trending down:
    • Tidying up, cleaning the kitchen (2.3-1.6%) – about 3 hours a week? I should do more around the house (or maybe I am, and I’m not tracking it properly)
    • Working on Quantified Awesome (1.4-0.8%) – steady-state since I’m happy with the code so far?
    • Reading fiction (1.2-0.4%) – subsumed into other activities
    • Socializing (8.0-1.4%) – big drop here; winter, becoming more selective?
    • Networking (4.7-1.7%) – big drop here too; not networking as actively
    • Biking (2.4-0.7%) – but then it’s still early in the biking season, and I work fewer days too

I’ll continue to focus on gardening for a bit until the garden is more established. I want to exercise and bike more as well. And there’s all sorts of Emacs coolness to learn about and share! =) Writing will have to be content with these little snippets–thinking out loud, sharing what I learn, and other things like that–until I can spend more time focusing on developing ideas. Mostly, the increase in time on other activities seems to be coming from the time I used to spend socializing. I actually like this new balance. The stuff I make and share online seems to lead to more ongoing conversations than those hi-hellos at tech events, and I’m still happy to spend a few hours getting to know people or going somewhere.

I got the time numbers from http://quantifiedawesome.com and a bit of spreadsheet number-crunching, and the task numbers from Emacs + Org Mode + R. =) Yay data!

Update on time tracking with Quantified Awesome and with Emacs

With another Quantified Self Toronto meetup in a few weeks and a conversation with fellow self-trackers, it’s time for me to think about time again.

I’ve been fixing bugs and adding small pieces of functionality to Quantified Awesome, and I spent some time improving the integration with Emacs. Now I can type ! to clock in on a task and update Quantified Awesome. Completing the task clocks me out in Emacs and updates Beeminder if appropriate. (I don’t update Quantified Awesome when finishing a task, because I just clock into the next activity.) This allows me to take advantage of Org’s clock reports for project and task-level time, at least for discretionary projects that involve my computer. I’m not going to get full coverage, but that’s what Quantified Awesome’s web interface is for. It takes very little effort to track things now, if I’m working off my to-do list. Even if I’m not, it still takes just a few taps on my phone to switch activities.

Most of my data is still medium-level, since I’m still getting the hang of sorting out my time in Emacs. Looking at data from 2014 so far, dropping partial weeks, and doing the analysis on April 14 (which is when I’m drafting this), here’s what I’ve been finding.

  • I sleep a little more than I used to: an average of 8.9 hours a day, or 37% of the time. This is up from 8.3 hours last year.
  • It takes me about an hour to get ready in the mornings. If I have a quick breakfast instead of having rice and fried egg, I can get out the door in 30-45 minutes.
  • It takes me 50-60 minutes to get downtown, whether this is by transit or bicycle. Commuting takes 3% of my time.

Little surprises:

  • I’ve spent almost twice as much time on business building or discretionary productive activities (19%) as I have earning (11%) – good to see decisions in action!
  • I’ve spent more time drawing than writing this year (5% vs 3%). Next to writing, Emacs is the productive discretionary activity I spend most of my time on (2%).
  • I’ve spent 10% of my time this year on connecting with people, a surprisingly high number for me. E-mail takes 1% of my overall time.
  • It turns out that yes, coding and drawing are negatively correlated (-0.63 considering all coding-related activities). But writing and drawing are positively correlated (0.44), which makes sense – I draw, and then I write a blog post to glue sketches together and give context. Earning is slightly negatively correlated with building business/skills (-0.15), but connecting is even more negatively correlated with time spent building business/skills (-0.35). So it’s probably not that consulting takes me away from building skills. Sleep is slightly negatively correlated with all records related to socializing (-0.14), but strongly negatively correlated with productive discretionary activities (-0.55). Hmm. Something to tinker with.

Some things I’m learning from tracking time on specific tasks:

  • Outlining doubles the time I take to write (and drops me from about ~30wpm to about 9wpm), but I feel that it makes things more structured.
  • Drawing takes longer too, but it makes blog posts more interesting.
  • Trying to dictate posts takes me way more time than outlining or typing it, since I’m not as used to organizing my thoughts that way.
  • Encoding litter box data takes me about a minute per data point. So spending a lot of time trying to figure out computer vision and image processing in order to partially automate the process doesn’t strictly make sense, but I’m doing it out of curiosity.
  • I generally overestimate the time I need for programming-related tasks, which is surprising. That could just be me padding my estimates to account for distractions or to make myself feel great, though.
  • I generally underestimate the time I need to write, especially if I’m figuring things out along the way.

This post took me 1:20 to draft (including data analysis), although to be fair, part of that involved a detour checking electricity use for an unrelated question. =)

Getting R and ggplot2 to work in Emacs Org Mode Babel blocks; also, tracking the number of TODOs

I started tracking the number of tasks I had in Org Mode so that I could find out if my TODO list tended to shrink or grow. It was easy to write a function in Emacs Lisp to count the number of tasks in different states and summarize them in a table.

(defun sacha/org-count-tasks-by-status ()
  (interactive)
  (let ((counts (make-hash-table :test 'equal))
        (today (format-time-string "%Y-%m-%d" (current-time)))
        values output)
    (org-map-entries
     (lambda ()
       (let* ((status (elt (org-heading-components) 2)))
         (when status
           (puthash status (1+ (or (gethash status counts) 0)) counts))))
     nil
     'agenda)
    (setq values (mapcar (lambda (x)
                           (or (gethash x counts) 0))
                         '("DONE" "STARTED" "TODO" "WAITING" "DELEGATED" "CANCELLED" "SOMEDAY")))
    (setq output
          (concat "| " today " | "
                  (mapconcat 'number-to-string values " | ")
                  " | "
                  (number-to-string (apply '+ values))
                  " | "
                  (number-to-string
                   (round (/ (* 100.0 (car values)) (apply '+ values))))
                  "% |"))
    (if (called-interactively-p 'any)
        (insert output)
      output)))
(sacha/org-count-tasks-by-status)

I ran this code over several days. Here are my results as of 2014-05-01:

Date DONE START. TODO WAIT. DELEG. CANC. SOMEDAY Total % done + done +canc. + total + t – d – c Note
2014-04-16 1104 1 403 3 1 104 35 1651 67%
2014-04-17 1257 0 114 4 1 171 107 1654 76% 153 67 3 -217 Lots of trimming
2014-04-18 1292 0 74 4 5 183 100 1658 78% 35 12 4 -43 A little bit more trimming
2014-04-20 1305 0 80 4 5 183 100 1677 78% 13 0 19 6
2014-04-21 1311 1 78 4 4 184 99 1681 78% 6 1 4 -3
2014-04-22 1313 2 75 4 4 184 99 1681 78% 2 0 0 -2
2014-04-23 1369 4 66 4 5 186 101 1735 79% 56 2 54 -4 Added sharing/index.org
2014-04-24 1371 3 69 4 5 186 101 1739 79% 2 0 4 2
2014-04-25 1379 3 60 3 5 189 103 1742 79% 8 3 3 -8
2014-04-26 1384 3 65 3 5 192 103 1755 79% 5 3 13 5
2014-04-27 1389 2 66 3 5 192 103 1760 79% 5 0 5 0
2014-04-28 1396 3 67 3 5 192 103 1769 79% 7 0 9 2
2014-04-29 1396 3 67 3 5 192 103 1769 79% 0 0 0 0
2014-04-30 1404 4 70 4 5 192 103 1782 79% 8 0 13 5
2014-05-01 1413 4 80 3 4 193 103 1800 79% 9 1 18 8

Here’s the source for that table:

#+NAME: burndown
#+RESULTS:
|       Date | DONE | START. | TODO | WAIT. | DELEG. | CANC. | SOMEDAY | Total | % done | + done | +canc. | + total | + t - d - c | Note                       |
|------------+------+--------+------+-------+--------+-------+---------+-------+--------+--------+--------+---------+-------------+----------------------------|
| 2014-04-16 | 1104 |      1 |  403 |     3 |      1 |   104 |      35 |  1651 |    67% |        |        |         |             |                            |
| 2014-04-17 | 1257 |      0 |  114 |     4 |      1 |   171 |     107 |  1654 |    76% |    153 |     67 |       3 |        -217 | Lots of trimming           |
| 2014-04-18 | 1292 |      0 |   74 |     4 |      5 |   183 |     100 |  1658 |    78% |     35 |     12 |       4 |         -43 | A little bit more trimming |
| 2014-04-20 | 1305 |      0 |   80 |     4 |      5 |   183 |     100 |  1677 |    78% |     13 |      0 |      19 |           6 |                            |
| 2014-04-21 | 1311 |      1 |   78 |     4 |      4 |   184 |      99 |  1681 |    78% |      6 |      1 |       4 |          -3 |                            |
| 2014-04-22 | 1313 |      2 |   75 |     4 |      4 |   184 |      99 |  1681 |    78% |      2 |      0 |       0 |          -2 |                            |
| 2014-04-23 | 1369 |      4 |   66 |     4 |      5 |   186 |     101 |  1735 |    79% |     56 |      2 |      54 |          -4 | Added sharing/index.org    |
| 2014-04-24 | 1371 |      3 |   69 |     4 |      5 |   186 |     101 |  1739 |    79% |      2 |      0 |       4 |           2 |                            |
| 2014-04-25 | 1379 |      3 |   60 |     3 |      5 |   189 |     103 |  1742 |    79% |      8 |      3 |       3 |          -8 |                            |
| 2014-04-26 | 1384 |      3 |   65 |     3 |      5 |   192 |     103 |  1755 |    79% |      5 |      3 |      13 |           5 |                            |
| 2014-04-27 | 1389 |      2 |   66 |     3 |      5 |   192 |     103 |  1760 |    79% |      5 |      0 |       5 |           0 |                            |
| 2014-04-28 | 1396 |      3 |   67 |     3 |      5 |   192 |     103 |  1769 |    79% |      7 |      0 |       9 |           2 |                            |
| 2014-04-29 | 1396 |      3 |   67 |     3 |      5 |   192 |     103 |  1769 |    79% |      0 |      0 |       0 |           0 |                            |
| 2014-04-30 | 1404 |      4 |   70 |     4 |      5 |   192 |     103 |  1782 |    79% |      8 |      0 |      13 |           5 |                            |
| 2014-05-01 | 1413 |      4 |   80 |     3 |      4 |   193 |     103 |  1800 |    79% |      9 |      1 |      18 |           8 |                            |
#+TBLFM: @3$11..@>$11=$2-@-1$2::@3$13..@>$13=$9-@-1$9::@3$14..@>$14=$13-$11-($7-@-1$7)::@3$12..@>$12=$7-@-1$7

I wanted to graph this with Gnuplot, but it turns out that Gnuplot is difficult to integrate with Emacs on Microsoft Windows. I gave up after a half an hour of poking at it, since search results indicated there were long-standing problems with how Gnuplot got input from Emacs. Besides, I’d been meaning to learn more R anyway, and R is more powerful when it comes to statistics and data visualization.

Getting R to work with Org Mode babel blocks in Emacs on Windows was a challenge. Here are some of the things I ran into.

The first step was easy: Add R to the list of languages I could evaluate in a source block (I already had dot and ditaa from previous experiments).

(org-babel-do-load-languages
 'org-babel-load-languages
 '((dot . t)
   (ditaa . t) 
   (R . t)))

But my code didn’t execute at all, even when I was trying something that printed out results instead of drawing images. I got a little lost trying to dig into org-babel-execute:R with edebug, eventually ending up in comint.el. The real solution was even easier. I had incorrectly set inferior-R-program-name to the path of R in my configuration, which made M-x R work but which meant that Emacs was looking in the wrong place for the options to pass to R (which Org Babel relied on). The correct way to do this is to leave inferior-R-program-name with the default value (Rterm) and make sure that my system path included both the bin directory and the bin\x64 directory.

Then I had to pick up the basics of R again. It took me a little time to figure out that I needed to parse the columns I pulled in from Org, using strptime to convert the date column and as.numeric to convert the numbers. Eventually, I got it to plot some results with the regular plot command.

dates <- strptime(as.character(data$Date), "%Y-%m-%d")
tasks_done <- as.numeric(data$DONE)
tasks_uncancelled <- as.numeric(data$Total) - as.numeric(data$CANC.)
df <- data.frame(dates, tasks_done, tasks_uncancelled)
plot(x=dates, y=tasks_uncancelled, ylim=c(0,max(tasks_uncancelled)))
lines(x=dates, y=tasks_uncancelled, col="blue", type="o")
lines(x=dates, y=tasks_done, col="green", type="o")

r-plot

I wanted prettier graphs, though. I installed the ggplot2 package and started figuring it out. No matter what I did, though, I ended up with a blank white image instead of my graph. If I used M-x R instead of evaluating the src block, the code worked. Weird! Eventually I found out that adding print(...) around my ggplot made it display the image correctly. Yay! Now I had what I wanted.

library(ggplot2)
dates <- strptime(as.character(data$Date), "%Y-%m-%d")
tasks_done <- as.numeric(data$DONE)
tasks_uncancelled <- as.numeric(data$Total) - as.numeric(data$CANC.)
df <- data.frame(dates, tasks_done, tasks_uncancelled)
plot = ggplot(data=df, aes(x=dates, y=tasks_done, ymin=0)) + geom_line(color="#009900") + geom_point() + geom_line(aes(y=tasks_uncancelled), color="blue") + geom_point(aes(y=tasks_uncancelled))
print(plot)

 r-graph

The blue line represents the total number of tasks (except for the cancelled ones), and the green line represents tasks that are done.

Here’s something that looks a little more like a burn down chart, since it shows just the number of things to be done:

library(ggplot2)
dates <- strptime(as.character(data$Date), "%Y-%m-%d")
tasks_remaining <- as.numeric(data$Total) - as.numeric(data$CANC.) - as.numeric(data$DONE)
df <- data.frame(dates, tasks_remaining)
plot = ggplot(data=df, aes(x=dates, y=tasks_remaining, ymin=0)) + geom_line(color="#009900") + geom_point()
print(plot)

r-graph-2

The drastic decline there is me realizing that I had lots of tasks that were no longer relevant, not me being super-productive. =)

As it turns out, I tend to add new tasks at about the rate that I finish them (or slightly more). I think this is okay. It means I’m working on things that have next steps, and next steps, and steps beyond that. If I add more tasks, that gives me more variety to choose from. Besides, I have a lot of repetitive tasks, so those never get marked as DONE over here.

Anyway, cool! Now that I’ve gotten R to work on my system, you’ll probably see it in even more of these blog posts. =D Hooray for Org Babel and R!

Update 2014-05-09: Stephen suggested http://blogs.neuwirth.priv.at/software/2012/03/28/r-and-emacs-with-org-mode/ for more tips on setting up Org Mode with R and Emacs Speaks Statistics (ESS).

Quantified Self: Analyzing 32 months of clothing data

I added the ability to analyze clothing use by week, month, or year to Quantified Awesome, where I’ve been tracking my clothes since August 2011. With a little bit of Microsoft Excel formatting, here’s part of the analysis I get:

2014-04-21 12_01_02-Microsoft Excel - Book1

 

(more rows omitted)

You can see how I’ve practically been living in snow pants this past winter, which has been very cold. Unsurprisingly, I don’t wear it at all during warmer months. Sweaters and long-sleeved tops are bursty too, naturally. I bought my two pairs of jeans from a thrift store in April 2012 and have been wearing them fairly regularly since. I also wear my office slacks a lot, since I have fewer of them compared to tops.

Nothing particularly unintuitive about this data, but it’s interesting to see. One of the later rows reminds me, hey, where did my black WPengine shirt go? That was comfy, but I haven’t worn it in 4 months – it might have disappeared into J-‘s closet. Now that we’re almost the same size, sometimes things get misfiled. Also, might be time to donate more clothes…

Rethinking my time categories: the blurring of business and discretionary activities

I track my time with medium-level categories (not detailed enough that I’m tracking individual tasks, but not so high level that it’s hard to make sense of the data). From time to time, I notice categories drift, or they stop fitting. Consulting is definitely business, but does working on Emacs really belong there? Why is coding classified under business but writing is classified as discretionary time? Most of my categories still make sense a year or two later, but some of them could use more thinking about.

What is business, anyway? I suppose it can include anything related to the earning of money, including support such as paperwork or delegation. Packaging (by which I mean creating e-books and other resources) is part of business, since I earn a small income from that (and pay taxes on it, too!). So is responding to e-mail. Technically, Emacs is related to money, because people have actually booked and paid me for help sessions online (http://sachachua.com/help). I consider programming-related activities to be part of maintaining my technical skills and network. In that sense, coding, web development, system administration, and other geek things are business-related. I distinguish between sketchnoting for client engagements and drawing on my own. Many of my drawings are more along the lines of personal or business planning. Perhaps I should track more under those categories now that I’ve established drawing as a way of thinking, and shift to using “Business – Drawing” when I’m specifically working on illustrations or improving skills.

Discretionary time includes the stuff I do just for fun and the things I learn about just because (Latin and Morse, for example). Probably the only weird thing in here is that I classify writing as discretionary time. It’s fun. Coding is fun too. Coding is more obviously valued, though, so I guess that’s why I consider it business time. And also, if I classify writing as coding time, I’ll tip over way too often into the “working too many hours a week” zone, when I’m not really doing so.

Maybe a better approach is to classify coding, drawing, and other fun things as discretionary time instead, even if they occasionally result in money. Benefit: I get to celebrate having more discretionary time and a lighter workload. (Yeah, it’s all mental anyway…)

Or maybe I need to take a step back and ask myself what kinds of questions I want to be able to answer with my categorical data.

In general, I want to make sure I don’t spend too much time working, because I want to force myself to work on my own projects. That’s why I track the time spent consulting, doing paperwork, and connecting with people (including responding to e-mail). I usually keep a close eye on my Business – Earn subcategory, since that’s the one that can creep up on me unawares. That’s fine with my current categories.

I also want to look for patterns in time use. How does spending more time on one activity (and less time on other activities) influence what I do and how I feel? How bursty am I when it comes to different discretionary projects? As long as I’m tracking at the subcategory level, it doesn’t really matter what the root category is.

Hmm. Since I’m not actually using the distinction between discretionary and business for reports or visualizations that nudge my behaviour, I can probably leave my categories alone if I remind myself that those ones have fuzzy boundaries. It would matter more if I wanted to set goals for investing X hours a week on business things (or, conversely, spending Y hours on discretionary non-business related things, which is oddly harder). Since I don’t care about that at the moment, I’m fine. Also, it’s easy enough to reassign the parent categories, so I still leave the door open for analyses at a later date.

As long as I can keep things clear enough in my head so that I feel confident that I can explain to any auditors that yes, my  business expenses make sense, I should be fine. I feel a little weird about not having a proper business plan for lots of things I’m working on. I mean, I can write them (or draw business model canvasses, more likely), but I prefer this pay-what-you-want model. Oh, hey, there’s an assumption there that I can dig into. People can (and do!) build metrics around freemium or pay-what-you-want models. Maybe I can figure out how to approach this in a business-like-but-still-generous way.

What would a more business-y way look like? I would float an idea to see if it’s useful. Then I would make stuff (and sometimes I’d make it anyway, just because). I might actually track conversions, and try things out, and reach out to people and communities. I’d publish little guides and videos, and maybe add a tip jar for smaller pieces of content so that people can “vote” for things they like more.

All things to do in due course. In the meantime, knowing that the path is there means I can leave all of this stuff still filed under the Business category, because it is. Even if it’s fun. Writing still feels more discretionary than business-y (even posts like this, for example), so I’ll leave that where it is. So no change, but I understand things better.

Do you track your time and have fuzzy boundaries between categories? How do you deal with it?