Category Archives: quantified

On this page:

Sketchnotes from Quantified Self Toronto Oct 21 2011

David Phillips talked about surveillance and his impressions of the Quantified Self conference, I showed my personal tracking system, and Brent talked about tracking lots of biomarkers. Here are some notes:

image

image

Geek travel update: Mostly as planned

I planned my packing for the Oct 5 – Oct 15 trip to the Philippines using several matrices. Life worked out mostly as planned. This was how I thought it would work out:

  Cream tee White tee Pink tee Dress
Dress       2011-10-9
Brown skirt 2011-10-07   2011-10-10  
Stretch pants 2011-10-15 2011-10-05 2011-10-08  
Cargo pants 2011-10-13 2011-10-11 2011-10-06  
Board shorts   2011-10-14 2011-10-12  

I liked the planning method, so I built the analysis into my home dashboard. This is what my clothing logs tell me:

image

Here’s that listed by date (also from the same page):

Clothing item 04 05 06 07 08 09 10 11 12 13 14 15 Total
pink board shorts                   A     1
dark brown capri pants                     A   1
white Wanko lace       B     A           2
cream v-neck shirt                 A   A   2
rose shirt     A             A     2
brown cable sweater A                     A 2
paint-spattered stretch pants A A     A             A 4
cream shift dress           A             1
light blue Jockey T-shirt       A                 1
white Jockey shirt A A     A     A       A 5
brown skirt with pattern       B     A           2
beige vest A                     A 2
blue cargo     A A       A A       4
Total 4 2 2 4 2 1 2 2 2 2 2 4  

A and B distinguish different outfits on the same day.

I might need to fix the clothing logs as the pictures come in. I forgot to track some of the earlier days, so I filled them in from memory.

I wore the white T-shirt / stretch pants combination on the plane rides, hence the double-counted days.

Next time, I should bring more polyester and less cotton. I brought the cotton T-shirts because they were easy to mix and match. They were fine in the city, where we spent most of our time in airconditioned comfort. The polyester top I borrowed from W- was much more comfortable in hot weather. It dried faster after washing, too. Many of my polyester tops are more sport-like and can’t easily be matched with skirts, but I’ll keep an eye out for other tops that would be a good fit. The travel pants my dad got me should also make it easier to use my travel tops.

I should plan for clothes to dry in two days, not one. Maybe even three days, for safety.

We packed just the right number of clothes, I think. I didn’t feel like I packed too few clothes, and I didn’t end up with lots of clothes unworn.

W- totally won in terms of clothing. It turns out that scrubs are excellent for air travel: a loose fit for comfort; pockets for pens, passports, and boarding passes; and room to avoid aggravating sunburns.

Yay geek travel!

Quantified Awesome: Data from waking up 3 hours earlier than normal

ccattrib_toronto_dawn_2009_mac_mcgillivray

I’ve been taking advantage of jetlag to wake up earlier than usual. I go to bed when I’m tired and wake up when I’m ready. Most of the time, I wake up on my own, although I set my alarm for 6 AM as a safeguard. From 2011-10-15 to 2011-10-28, I woke at 5:41 on average, with a median of 5:12 and a standard deviation of 1:09. (Weekends…)

I liked being up early in the morning. No guilt about hitting snooze, no rush to the bathroom, some time for personal projects or work momentum before the workday starts.

I was curious about a few things:

  • Around what time should I plan to be in bed if I want to continue waking up early?
  • Does waking up early actually give me more discretionary or work time, or do I give up time because I get sleepier in the evening?
  • Waking up early usually means I’m tired at the end of the day. What kind of an effect does that have on the things I feel like doing?

Fortunately, I’ve been collecting time data for a while, so I can compare it with a similar two-week period where I’d wake up around 7. Let’s see how the data stacks up against the data from 2011-08-20 to 2011-09-02, a two-week period also without any long weekends.

Going to bed

n = 14 for each condition

Late Early
Wake-up average 8:33 AM 5:41 AM
Wake-up median 8:30 AM 5:12 AM
Wake-up stdev 0:29 1:09
Bedtime average 1:03 AM 10:16 PM
Bedtime median 0:59 AM 10:09 PM
Bedtime stdev 0:38 1:15
Sleep average including naps 7.8 hours 8.2 hours

So if I want to get up at around 5:30, I should be in bed by around 10.

I tend to sleep less when I sleep later. Work gives me a reason to pay attention to my alarm clock, so even if I hit snooze and have lower-quality sleep, I’m out of bed. I used to stay up late so that I could spend more time hanging out with W- or working on projects. Sometimes it took me a while to go to bed because I’d get carried away hacking. My standard deviations for the late condition are pretty low – mornings because of the alarm clock, and evenings because I eventually look at the clock and go "I really should be in bed."

Waking up early and going to bed when I’m reasonably tired means wasting less sleep time fidgeting, enjoying better sleep and morning quality, and less stress in the morning. Based on this two-week sample, the difference is around 24 minutes a day. It’s hard to tell whether the increased wakefulness of being up early and on my own schedule compares well with the stress and bleariness of mornings jumpstarted by an alarm clock and stress. I think it’s worth it. Besides, if I zoom out and look at more than just the two-week period – say, two-month period 2011-08-01 to 2011-09-30 – I find that I actually sleep around 8.0 hours on average, so the difference isn’t that great.

I wonder if jet lag affected my numbers, increasing the amount of time I needed to sleep. If I look at just last week’s data (2011-10-22 to 2011-10-28), though, it turns out that I ended up sleeping a little more after I had theoretically recovered from jet lag: 8.7 hours on average, or 8.2 hours on weekdays and 10.1 hours on weekends. Part of that might be due to the Halloween event we photographed on Saturday – I got a case of introvert overwhelm and napped for 4.5 hours afterwards to recharge. No significant differences, though: unpaired t-test between number of hours slept during first week (M=7.6, SD=1.5) and second week (M=8.6, SD=1.4); t(12)=1.49 p=0.16. We’ll see how the numbers work out as my routine stabilizes.

I do like the sleep quality. People can spend a lot of time and money in the quest to improve their sleep. I had been playing around with using eyemasks or eye pillows I’ve made myself, and had even considered getting a light-based alarm clock and/or blackout curtains. Going to bed when I’m tired means not needing any of those things, so I can save that money for other things.

In a previous experiment with early-morning wakeups, my husband and I noticed that our schedules were diverging a bit. He’d stay up late, I’d wake up early, and we had less conversation time. He was similarly jetlagged this time around, so we’ve settled into a good routine with plenty of time in the evening and some high-quality morning time too. It’s been working well.

Waking up early and discretionary time

Late Early
Discretionary time average 6.4 hours 5.7 hours
Weekday average 5.2 hours 3.8 hours
Weekend average 9.4 hours 10.6 hours
Work 5.7 hours 6.1 hours
Weekday average 8.0 hours 8.4 hours
Weekend average   0.4 hours
Unpaid work average 1.9 hours 1.2 hours
Personal care average 2.4 hours 2.8 hours
Sleep average 7.8 hours 8.2 hours
Discretionary + work 12.1 hours 11.8 hours

For productive time, I looked at the sum of time I spent on work and the time I spent on discretionary projects. This took into account the extra time I shifted towards working last week. It turns out that there’s a little difference between the discretionary + work time I had (late: M=12.2, SD=2.0; early: M=11.8, SD=2.1), but it’s not significant either (unpaired t-test t(26) = 0.45, p = 0.65). So it looks like waking up earlier doesn’t mean giving up too much – or gaining a lot – in terms of focused time.

Activity substitution

If I have about the same amount of discretionary time anyway, does waking up earlier affect the kind of things I spend my time on? This one is harder to figure out, because other variables affect how I spend my time. I spend more time drawing when I’m attending events or preparing for presentations. I spend more time working on my personal dashboard when I’m buzzing with ideas. I spend more time writing when I don’t have lots of posts queued up. It’s hard to say.

Looking at my time graphs, though, I see that when I woke up early, I didn’t really have the chunks of discretionary time that I’d hope to have in the mornings, and my evenings were more fragmented and other-focused. When I woke up late, I tended to have more me-time at the end of the day, and I still had enough energy to make the most of it.

A previous analysis showed that even when I stayed up late, I didn’t really have many discretionary activities that used a four-hour chunk of time, so waking up early doesn’t mean I’m missing out on activities that need a long chunk of time. However, after-school hours tend to involve discretionary social activities, and I usually carve out time for personal projects either late at night or for a short time in the morning.

Overall, I’m happy with how I spend my discretionary time. I feel like I’ve made reasonable progress on my personal projects, and I’m glad I’ve been able to help with things like homework. I might shift things around so that I can write and program more, probably when I get work back under control. As we improve routine processes like cooking, we’ll free up more time for other pursuits too.

Summary

It looks like waking up early doesn’t have a significant impact on how much time I sleep or how much focused time I have. I like the sleep quality and the lack of stress in the mornings. It might come at the cost of not having a longer window of discretionary time focused on personal projects, but social time is good too. Overall, I’m happy with waking up early and the resulting shifts in my schedule, and will continue waking up early and going to bed late.

Testing your assumptions and trying new things is much easier when you collect data. I’m thinking of sharing observations every Monday. Check back next week for more!

Photo of Toronto at dawn © 2009 Mac McGillivray, Creative Commons Attribution License

Quantified Awesome: Development-driven behaviour and integrated tests for life

2011-12-14: Updated step code

In terms of testing code, behaviour-driven development is fantastic. You can write your tests in pretty much plain English using a testing tool like Cucumber for Rails, which makes it easier to communicate with other people (including clients!). There’s a certain satisfaction in getting your tests to pass, and when they break, you know something needs fixing.

I’ve been thinking about what automated tests might look like in life. It turned out to be easy to prototype, thanks to the data I’m already collecting. It’s almost like development-driven behavior: can I apply the tools I use in software development to help me change my behaviours in life?

Here are some results from my very first integration test of real life:

Feature: Development-driven behaviour

  Scenario: Check for overdue books                # features/life.feature:2
    When I check our library items                 # features/step_definitions/life.rb:3
    Then there should be no items that are overdue # features/step_definitions/life.rb:7

  Scenario: Check my work load                       # features/life.feature:5
    When I look at my time use for the past 7 days   # features/step_definitions/life.rb:11
    Then I should have time data                     # features/step_definitions/life.rb:19
    And I should have worked between 40 and 44 hours # features/step_definitions/life.rb:24
      <46.5166666666667> expected to be
      <=
      <44.0>. (Test::Unit::AssertionFailedError)
      ./features/step_definitions/life.rb:26:in `/^I should have worked between (\d+) and (\d+) hours$/'
      features/life.feature:8:in `And I should have worked between 40 and 44 hours'

  Scenario: Check if I'm sleeping                        # features/life.feature:9
    When I look at my time use for the past 7 days       # features/step_definitions/life.rb:11
    Then I should have slept between 8 and 9 hours a day # features/step_definitions/life.rb:29

Failing Scenarios:
cucumber features/life.feature:5 # Scenario: Check my work load

3 scenarios (1 failed, 2 passed)
7 steps (1 failed, 6 passed)
0m0.833s

Cucumber highlights failing tests in red and it lists the failures as well.

Here’s the steps.rb that I’ve started fleshing out:


When /^I look at my time use for the past (\d+) days?$/ do |arg1|
  @start_time = (Date.today - arg1.to_i.days).midnight.in_time_zone
  @end_time = Date.today.midnight.in_time_zone
  @log = TimeTrackerLog.new(User.first)
  @entries = @log.entries(@start_time, @end_time)
  @summary = @log.summarize(@start_time, @end_time)
end

Then /^I should have time data$/ do
  assert @entries != nil
  assert @entries.size > 0
end

Then /^I should have worked between (\d+) and (\d+) hours$/ do |min, max|
  assert_operator @summary['A - Work'] / 1.hour, :>=, min.to_f
  assert_operator @summary['A - Work'] / 1.hour, :< =, max.to_f
end

Then /^I should have slept between (\d+) and (\d+) hours a day$/ do |min, max|
  average = @summary['A - Sleep'] * 1.0 / (1.hour * ((@end_time - @start_time) / 1.day))
  assert_operator average, :>=, min.to_f
  assert_operator average, :< =, max.to_f
end

# LIBRARY

When /^I check our library items$/ do
  nil # Actually stored in database, so we don't need anything here. This is more for semantics
end

Then /^there should be no items that are overdue$/ do
  assert_equal 0, LibraryItem.where('status = ? AND due < ?', 'due', Date.today).size
end

I am pleasantly boggled that this is possible, and will probably write all sorts of odd tests now. Because Cucumber can fill in web forms, click on stuff, and so on, I might even be able to use it to check information on other sites. (When I check my mail, then all the messages in my inbox should be less than a week old?)

Oh, the possibilities…

Quantified Awesome: How much music do you have?

I don’t listen to music a lot. Words interfere with my programming or writing (hmm, I should test to see how big the effect is), and I got used to working in silence or with white noise. Some people have a lot of music, though. R. Galacho wrote this Python script that uses the ID3 information in MP3s to sum up listening time in each genre, and wanted me to share it in case anyone else might find it useful:

# -*- coding: utf-8 -*-

__doc__="""
muasure v.0.1

How long could you listen...  Many times I've talked with friends about
my digital record collection's size (mmm... we are talking in the
order of GB) and how long could I've been listen if I make a playlist
with the hole collection and play it completely.  Well, I made some
mind calculations setting up average time and making a proportion to
the number of files taken in my HDD.

So, spare time and the speed and versatility inherent to Python give me
the rest.

Music Measure (Muasure for short) calculates the total time of your
music collection. Finally show data in screen and writes a text file
with that information into collection base directory (so if you clean
your screen you don't have to relaunch the process).

The only parameter expected is the base location of your record
collection (by default is the current directory when invoked).

Written, tested, runned and commited on GNU Emacs 23.2.1 (i686-pc-linux-gnu, GTK+ Version 2.24.4) ;)

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or (at
your option) any later version.

This program is distributed in the hope that it will be useful, but
WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program.  If not, see <http://www.gnu.org/licenses/>.
"""
__author__="R. Galacho"
__version__="0.0.1"
__date__="20111010"

import os, sys, re
from datetime import timedelta
from mutagen.mp3 import MP3, HeaderNotFoundError
from collections import OrderedDict
from cStringIO import StringIO

def get_total_time(directory, collection):
    content_types = dict()
    lengths = 0
    for mp3_file in collection:
        try:
            id3v = MP3(mp3_file)
            lengths += id3v.info.length
            content = str(id3v.get('TCON'))
            if content_types.get(content) == None:
                content_types[content] = id3v.info.length
            else:
                content_types[content] += id3v.info.length
        except HeaderNotFoundError:
            sys.stderr.write("Error reading file %s\n" % mp3_file)

    total_time = timedelta(seconds = lengths)
    avg_length = timedelta(seconds = float(lengths / len(collection)))
    file_str = StringIO()

    file_str.writelines(["Total time  : ", str(total_time), "\nAverage time: ", str(avg_length), "\n\n"])
    ord_content_types = OrderedDict(sorted(content_types.items(), key=lambda t: t[1], reverse=True))
    for (k, v) in ord_content_types.items():
        total_time = timedelta(seconds = content_types.get(k))
        file_str.writelines([k.ljust(15), ": ", str(total_time),"\n"])

    print file_str.getvalue()
    if os.access(directory, os.W_OK):
        result_file = open(('%s%smuasure-data.txt' % (directory, os.sep)), 'w')
        result_file.write(file_str.getvalue())
        result_file.flush()
        result_file.close()

    file_str.close()


def main(collection_dir):
    directory = os.path.expanduser(collection_dir)

    if not os.access(directory, os.R_OK):
        raise Exception("Not enough permission on %s" % directory)

    collection = []
    pattern = re.compile(r'\.mp3')

    for dir, subdirs, files in os.walk(directory):
        collection.extend("%s%s%s" % (dir, os.sep, f) for f in filter(lambda x: pattern.search(x), files))

    collection = map(os.path.abspath, collection)
    get_total_time(directory, collection)


if __name__ == "__main__":
    if len(sys.argv) > 1:
        main(sys.argv[1])
    else:
        main("./")

This requires Python 2.6 or later and Python-mutagen 1.19 or later.

What else can you automatically extract from the files or data you already have? People have done interesting analyses based on geocoded photos, times of tweets, and so on. Have fun exploring!

Quantified Awesome: A place for everything, and everything in its place

A month is a good size for experimenting with life. This month, I’m focusing on having a place for everything and everything in its place.

Brains are bad at recall. Infrequently-used items fade into the fog of memory, like the loyalty cards and IDs I stash in a sometimes too-secret place. Frequently-moved items end up with confused associations: did I leave my belt bag on top of the dresser, or is it on the kitchen table? Under stress, recall is an even more difficult task, and it’s easy to glaze over and miss something when you’re trying to think of where else it might be.

Dean Vaughn’s How to Remember Anything has great tips on making object locations more memorable by visually exaggerating the association between an object and its location. For example, if you put your keys on the table, imagine locking your door with the entire table. The unusual association will probably make it easier to recall one-off locations, but

There’s the classic advice to have one place for everything, and put everything in its place. This works if you have foresight, discipline, and an organized space. Our house looks more like like a Martha Stewart centerfold… the “before” picture.

Because I’m a geek and I’m tired of rummaging through the house for things I’ve misplaced, I added a simple system for tracking things to my home dashboard. I’ll start by tracking the things that frustrate me the most. Infrequently-used things like loyalty cards and passports, frequently-moved items like my lunch bag or my mouth guard…

On my dashboard, I can see what’s out of place and where it should be returned. Here’s a screenshot from November 4:

image

I can view a summary and do some quick updates:

image

The detail page makes it easy to see where something is, change an item’s location or view other things that are associated with it. Here, my keys are in my belt bag, which is on the kitchen table.

image

I’ve just added this idea of a context, which groups together things. This way, I can check whether everything’s in the right place, and I can mark everything moved in one go.

image

Slowing down and tracking things might help me improve my peace of mind.  Even if I don’t always update my system, I think seeing a list of possible locations will help a lot. A table of stuff, current locations, and home locations will also make it easier to ensure that everything has a rightful place. I can identify things that are out of place and where they should be returned, which would be great for daily and weekly sweeps.

I think this would be even better if I built an Android app, as sometimes web access from my Android is slow. (Or maybe it’s my web hosting: I’m using too much memory, so I’m swapping out.)

I hope to collect interesting data over time. Maybe usage stats will tell me what’s worth improving or eliminating. Maybe this is something I’ll discontinue after a month, or maybe it’s something I’ll open up for wider use. Who knows?  I’ll give it a try to work out the usage patterns, then maybe I’ll look into tweaking my personal dashboard so that people can try using it to track their own lives too.

Whee!

Like this? Check out my other self-tracking posts.