Category Archives: quantified

Waking up: looking at my data

Whenever I manage to wake up early a few days in a row, I feel great about it. But I don’t do it consistently. I spend a couple of days waking up before 6 AM and enjoying a good spurt of writing, and then I find myself slipping back into later bedtimes and later wake-up times (~ 7 AM) or hitting the snooze. Clearly there are some things I still need to tweak about my system.

Time-tracking means I’ve got a way to see what my current sleep patterns are like:


  • Average sleep length when waking up before 6 AM: 7:09
  • Average sleep length when waking up after 6 AM: 8:47
  • Average sleep time for wake-up times before 6 AM: 9:45 PM, which is a bit of a stretch but is doable.

Here are the questions I’m thinking about:

  • Is it a matter of getting to bed earlier?
  • Would it help to disable snooze entirely?
  • Is it a matter of setting my alarm clock even earlier? (Ex: Set it for 4 AM so that I eventually get out of bed at 5 AM.)
  • Would it help to set our programmable thermostat warmer in the morning, or promise myself a hot cup of tea when I get up?
  • Would it help to set my snooze interval to 5 minutes instead of 10?
  • How about if I find a way to turn my Android into a light clock? (Using Tasker to bring up a bright app, maybe…)
  • What if I give up on waking up early and instead shift to more of a night owl schedule? Advantage: can sync up with W-. I’ll need to figure out how to give my personal pursuits the creative energy they need, though.

Hmm. More things to hack…

More thoughts on time analysis: correlations and revealed preferences

People often ask about the time analyses I do as part of my weekly review. My weekly time tracking reports go back to about December 11, 2010, when I started tracking my time using the free Time Recording app on the Android. I do it because of the following reasons:

  • I need to track my project-level time for work anyway,
  • I want to see where I spend my time and if that’s in line with my priorities,
  • I want to know how much time it takes me to do certain things, in order to improve my estimates and get better at planning,
  • I want to avoid burning myself out
  • I want to make sure I allocate enough time to important activities instead of, say, getting carried away with lots of fun work and flow experiences, and
  • I want to cultivate other deep interests and relationships.

Fatigue and burnout are particularly big concerns for developers. There’s always the temptation to be unrealistic about one’s schedule, either through over-optimistic estimates or through business pressures. However, sustained crunch mode decreases productivity and may even result in negative productivity. Sleep deprivation severely cuts into cognitive ability and increases the chance of catastrophic error. I like what I do too much to waste time burning out.

Development is so engaging for me. I could keep writing code and building systems late into the night, at the expense of other things I could do. Tracking time helps me keep a careful eye on how much time I spend programming. Like the way a good budgeting system helps me make the most of my expenses and gives me the freedom to take advantage of opportunities, a good time budgeting system helps me make the most of my focused work time and allows me to also focus on other things that matter (the care and feeding of relationships, the development of new skills, and so on).

So here are some new things I’ve learned from time tracking:

  • I sleep a median of 59 hours a week, which is about eight and a half hours a day. This is more than I expected, but I manage to get a lot done anyway, so it’s okay.
  • I work a little over 40 hours each week, except for the occasional week of crunch time or travel. I don’t make a habit of 50-hour weeks, and I get a little twitchy when I work too intensely several weeks in a row (46 hours or so). This means that when I estimate timelines or project my utilization, I should assume 38 or 40-hour weeks instead of 44 hours.
  • I spend most of my time sleeping (44%), working (31%), or connecting with people (11%). Regular routines take up 9% of my time, while my favourite hobby (writing) takes only 5%. I enjoy my work and I sleep well at night, so this time allocation is fine.

In economics, there’s the idea of a revealed preference, which is basically what your actions show compared to what you might say or think you prefer. I may think I’d like to sew or learn languages or do the piano, but if I spend time playing LEGO Star Wars III instead, then that tells me that sewing, Latin, and Schumann are lower on my priority list. (Rationalization: LEGO Star Wars is awesome and it counts as bonding time with W- and J-, so it’s not all that bad.)

So, how do I really trade my time? Which activities are positively or negatively correlated with other activities? I made a correlation matrix to see how I spent my time. I used conditional formatting to make high correlations jump out at me. I found some interesting patterns in how I shift time from one category to another.

Activity 1 Activity 2 Linear correlation coefficient (r) Notes
Prep Personal 0.87 Getting things in order means I can give myself permission to learn something new
Cooking Prep 0.86 Makes perfect sense. Big chore days.
Break Drawing 0.75 More relaxing time = more drawing time
Travel Work 0.69 When I commute to work, I probably tend to work longer. Also, I needed to go to the office for some of the crunchy projects.
Sleep Break 0.67 Relaxed days
Sleep Writing 0.60 Nice to know writing isn’t conflicting with sleep
Social Drawing -0.50 The Saturday afternoons or weekday evenings I spend with people instead of sketching
Routines Drawing -0.65 Lots of chores = less drawing time
Personal Drawing -0.55 Learning other things = less time spent on drawing
Travel Cooking -0.60 Lots of travel = live off home-made frozen lunches
Sleep Cooking -0.62 Late weekend mornings = less cooking?
Sleep Prep -0.58 Likewise
Sleep Personal -0.57 More sleep = less time spent learning other things

I can guess at the causality of some of these relationships, but the others are up in the air. =) Still, I’m learning quite a lot from this exercise. For example, I thought I was giving up sleep in order to write more or draw more. It turns out that sleep cuts into cooking, prep, and other personal interests (sewing, piano, etc.), and doesn’t have much effect on work, writing, or drawing. I do sleep quite well, though, so it may be interesting to experiment with that.

I’m also happy to see I don’t give up too much because of travel – a median of 3.4 hours / week, much of which is spent reading, brainstorming, or listening to audiobooks with W-. Travel time reduces cooking time, but that’s okay because we batch-cook in order to minimize weekday cooking. It’s good to see that it doesn’t affect my other activities a lot.

The same dataset lets me analyze my sleeping patterns, report project-level breakdowns at work, and review quick notes on my day. I’m in consulting, so I need to track and bill my time per project. Time Recording makes it easy to do that, and I’m thinking of tweaking my workflow further so that I can use task-level times to improve my estimates.

So that’s where I am, tracking-wise. It takes me a few seconds to clock into a new category, and the habit is handy for making sure I know where my phone is. Tracking my time also helps me stay more focused on what I’m doing. If you’re curious about the idea and you have a smartphone or other mobile device, find a time-tracking application and give it a try. Have fun!

2011-03-29 Tue 21:54

Learning from my mood data

One of the unexpected benefits of switching my phone plan to something that includes unlimited international texting is that I can participate in nifty things like Experimonth, which is a month-long study about moods. I get regular text messages prompting me to rate my happiness on a scale of 1-10, and it graphs it for me. I can probably come up with similar graphs using KeepTrack and a bit of spreadsheet magic, but the convenience and the social data make this fun and interesting.

Here’s how my mood data stacks up so far:


I stay on a fairly even keel, with awesome happy experiences possibly any day of the week. Hmm, maybe I should track text notes too, so I can get a better handle on what causes the 10s or the 6s. It might also be interesting to combine the happiness ratings with my time analyses to see if there any correlations.

Here are the results they’ve collected so far:

Sketchnotes from Quantified Self Toronto meetup: Conferences, pollution sensing, and growing old at home

14 people at today for the Quantified Self Toronto meetup:




Stuff I’m going to do before the next meeting (~6 weeks?)

  • Get back into tracking time so that I can check on hobbies and share what I’m learning with other people
  • Track community-supported agriculture box in more detail, cross-reference with groceries

To find out about upcoming meetings, join the Quantified Self Toronto meetup group!

Quantified: How I spent seven weeks

At the other Quantified Self Toronto meeting, I promised to get back into time tracking and to share my results. I’ve got seven full weeks of data from August 6 to September 23, and I can start exploring a few interesting angles.

Influenced by the OECD time study, I’ve categorized my time into sleep, work, unpaid work, personal care, and discretionary time. Sleep and work are self-explanatory. Unpaid work cover the routine things I could theoretically pay someone else to do: chores, cooking, and so on. I also include travel and commute time. Personal care involves daily routines. Discretionary time includes connecting with other people, responding to mail, exploring personal interests, and other things I choose to do.

I slept an average of 8.2 hours a day. I’ve been trying a different pattern: stay up until I feel sleepy, and wake up at around the same time. This gets me mostly in sync with my night-owl husband W-, who gets by on less sleep than I do. (Maybe it’s because he drinks coffee and I don’t.) Lately, I’ve been working on being in bed by 11, and sometimes even earlier.

Staying up means getting more discretionary time, as my wake-up times generally don’t shift unless my phone’s powered off or I sleep through my alarm. (Happened twice, fortunately with no consequences.) I think it has to do with lots of sunlight in the morning – it makes it much easier to get up. Sunrise will get later and later, though, so I’ll need to adapt.

More usefully, staying up later means creating the possibility of chunks of focused time, which is great for things like playing around with the Arduino or working on personal code. For some interests, a four-hour chunk may be better than two two-hour chunks. Setting up for woodworking or sewing can take time, for example, so it might be better to batch things.

Did I take advantage of those chunks of time? Here’s what the numbers say:

Time in 49 days Typical activities
4-5-hour chunks 3 working on personal projects (2), electronics (1)
3-hour chunk 5 volunteering (4), blogging (1)
2-hour chunk 21 writing (6), personal projects (5), electronics (3), drawing (2), piano (1), relaxing (1), volunteering (1), learning (1), reading (1)
1-hour chunk 41 writing (10), personal projects (7), drawing (7), relaxing (6), other (3), reading (3), volunteering (2), piano (1), learning (1), sewing (1)
Less than 1 hour 153 writing (42), drawing (26), personal projects (21), relaxing (21), reading (14), other (9), piano (8), learning (6), delegating (2), Latin (2), volunteering (1), gardening (1)

This tells me that freeing up a 4-hour chunk isn’t super-important, and that I can squeeze a lot of activities into the nooks and crannies of a regular sort of day.

Sleep: When I stayed up late, I felt like the discretionary time was occasionally of lower quality. It’s not quite about being tired, more like not being as excited. Maybe being up early gives you a certain smugness and feeling of control. Maybe it’s about momentum. I can see if I can move my chunks of time earlier in the morning (downside: less ambient socialization), or if I can tweak my afternoon my momentum (start work a little earlier, use a nap or household routines to transition from work, then rock on).

Tracking time affects how I spend my day. It’s like the way tracking expenses can influence what you choose to spend on. (I track practically all my expenses – tracking’s great for making better decisions.) Mostly, tracking time encourages me to keep work within limits, because I know I’ve only got so many discretionary hours to spend on my own interests.

I tend to work about 40 hours a week, sometimes a little more. This doesn’t mean that I watch the clock, waiting for the seconds to tick by. If I’m in the zone, I’ll code until I come to a good place to stop. I’ve been tweaking my non-billable work to focus on the things I can make the most difference in. For example, I maintain a Lotus Connections toolkit to help people make community newsletters and get metrics. I tend to focus on small, quick fixes that help many people. Anything bigger than that gets added to my list, and I encourage people to find someone who can work with the source code if they need it sooner. I also nudge people to send happy-notes to my manager, as he needs to provide air cover for these sorts of things whenever there’s a heavy focus on utilization.

Limiting my work hours also means that I focus more on work when I’m at work. I’ve planned the projects based on how much time I think I’ll need to finish the work, and I don’t want to get into a last-minute scramble at the end. Although my estimates factor in a reasonable buffer for meetings and other interruptions, I still don’t want to waste that margin. Result so far: pretty happy clients. My manager is happy too, as my estimates aren’t over-optimistic. (In fact, I tend to turn things around quickly, but that’s more of a bonus.) It also helps that I know I’ll have discretionary time for exploring other interests.

Our routines fit our life well. There aren’t any big gaps where I could significantly improve things for a small investment of time or money. I’m working on misplacing things less often. We’re going to experiment with scaling up. I’ve considered outsourcing or getting assistance with food preparation, but I still have to crunch the numbers on whether the increase in discretionary time makes up for the increase in our food budget. There’s no point in doing it if I’m going to waste the time, but maybe it compares well with delegating or postponing other things I want to do.

12-Aug 19-Aug 26-Aug 2-Sep 9-Sep 16-Sep 23-Sep Total Percentage of total time
UW – Cooking 6.4 4.7 1.5 7.4 3.1 4.1 1.2 28.4 2%
UW – Tidying 2.5 5.0 3.8 3.7 5.7 6.3 3.6 30.5 3%
UW – Travel 0.8 0.6 1.4 5.2 2.8 10.8 1%
P – Eating 5.0 6.1 2.0 5.0 2.8 1.7 2.1 24.5 2%
Unpaid work total 8.9 10.5 5.3 11.7 10.2 15.5 7.6 69.7 6%
P – Exercise 5.9 2.5 12.2 6.2 5.6 2.7 5.5 40.5 3%
P – Prep 0.0 0.0 0%
P – Routines 7.7 7.9 8.2 6.1 6.3 11.0 8.7 55.9 5%
Personal care 18.6 16.4 22.4 17.2 14.8 15.3 16.3 120.9 10%

My “discretionary time” allowance stays pretty consistent. It turns out that I have roughly 4.6 hours of discretionary time during weekdays and 9.3 hours of discretionary time during weekends. What I choose to spend that time on tells me about my changing interests. For example, I’ve been shifting time from Latin and piano to electronics and drawing. I’m pretty happy with that decision, although I’m thinking I might shift some time back to Latin so that I don’t lose too much to forgetting. We’ve been volunteering a lot, so we’ll see how that works out.

Discretionary time:

12-Aug 19-Aug 26-Aug 2-Sep 9-Sep 16-Sep 23-Sep Total Percentage of discretionary time
D – Break 0.7 2.4 1.9 2.4 2.0 3.4 6.0 18.8 6%
D – Delegating 0.6 0.1 0.7 0%
D – Drawing 4.1 4.3 10.0 2.0 3.2 1.9 0.7 26.2 8%
D – Electronics 2.0 2.0 1%
D – Gardening 0.2 0.2 0%
D – Latin 1.4 0.5 1.9 1%
D – Learning 0.2 1.2 9.5 10.8 3%
D – Other 4.9 4.9 2.5 12.2 4%
D – Personal 3.9 13.5 12.3 0.8 12.8 43.2 14%
D – Piano 6.6 2.6 9.2 3%
D – Reading 0.7 3.4 0.1 5.5 2.9 0.3 13.1 4%
D – Sewing 1.6 1.6 1%
D – Shopping 1.1 2.0 2.5 3.4 11.9 20.9 7%
D – Social 11.5 11.2 7.8 9.0 19.2 12.1 4.7 75.5 24%
D – Volunteering 6.3 8.0 3.8 3.5 3.7 25.4 8%
D – Writing 8.1 6.6 5.0 11.4 7.2 11.4 0.8 50.4 16%
Discretionary time total 39.5 46.4 47.0 43.3 52.0 43.0 40.9 312.2

How can I make this even better?

  • Plan the projects I want to focus on, list the next actions, and see how much of my discretionary time is used for making tangible progress towards long-term goals. It’s like the way I analyze my expenses based on short-term goals and long-term goals.
  • Shift wake-up a little earlier so that I can experiment with two smaller chunks of time instead of just one evening chunk.
  • Experiment with greater delegation.
  • Experiment with finer-grained tracking using notes.
  • Continue adding to my life dashboard (currently tracking time and clothes).

2011-09-02 Fri 19:45

Tracking and organizing my clothes: substituting mathematics for fashion sense

Thumbnails of clothes

Inspired by my sister’s photo-assisted organization of her shoes, I decided to tackle my wardrobe. Taking an inventory would make it easier to simplify, replace, or supplement my clothes. Analyzing colour would help me substitute mathematics for a sense of style. Combining the images with the clothes log I’ve been keeping would make it easier to see patterns and maybe do some interesting visualizations. Geek time!

I took pictures of all my clothes against a convenient white wall. I corrected the images using Bibble 5 Pro and renamed the files to match my clothes-tracking database, creating new records as needed. AutoHotkey and Colorette made the task of choosing representative colours much less tedious than it would’ve been otherwise. After I created a spreadsheet of IDs, representative colours, and tags, I imported the data into my Rails-based personal dashboard, programming in new functionality along the way. (Emacs keyboard macros + Rails console = quick and easy data munging.) I used Acts as Taggable On for additional structure.

It turns out that the math for complementary and triadic colour schemes is easy when you convert RGB to HSL (hue, saturation, lightness). I used the Color gem for my RGB-HSL conversions, then calculated the complementary and triadic colours by adding or subtracting degrees as needed (180 for complementary, +/- 120 for triadic).

Here’s what the detailed view looks like now:


And the clothing log:


Clothing summary, sorted by frequency (30 days of data as of writing)



  • White balance and exposure are a little off in some shots. I tweaked some representative colours to account for that. It would be neat to get that all sorted out, and maybe drop out the background too. It’s fine the way it is. =)
  • Matches are suggested based on tags, and are not yet sorted by colour. Sorting by colour or some kind of relevance factor would be extra cool.
  • Sorting by hue can be tricky. Maybe there’s a better way to do this…
  • My colour combinations don’t quite agree with other color scheme calculators I’ve tried. They’re in the right neighbourhood, at least. Rounding errors?
  • I’ll keep an eye out for accessories that match triadic colours for the clothes I most frequently wear.
  • Quick stats: 28 casual tops, 15 skirts, 12 office-type tops, 8 pairs of pants, 5 pairs of slacks – yes, there’s definitely room to trim. It would be interesting to visualize this further. Graph theory can help me figure out if there are clothing combinations that will help me simplify my wardrobe, and it might be fun to plot colours and perhaps usage. Hmm…

Other resources: