LifeCampTO social graph

After LifeCampTO, I asked people to give me the list of people they wanted to talk to (or, well, those people’s primary keys ;) ). I’m still figuring out how to do a great little mail merge that reminds people of the keywords, but along the way, I thought I might I’d learn more about network visualization.

Here’s the resulting graph: (click on it for a larger version)

LifeCampTO social graph

So, what does this graph say?

You can see that most people have quite a lot of follow-up conversations ahead. It wasn’t the kind of event where most people walked away with only two or three conversations, although they might have smaller follow-up conversations with different groups of people. It might be interesting to do some cluster analysis around topics, and maybe someday I’ll figure out how to encode the data in order to make that analysis easier. ;) Based on this, our on-the-fly decision to have three big conversations turned out to have made sense, although it would also be interesting to try having small conversations about both popular and niche topics, and then having people come together at the end (or on a wiki).

Getting to this graph (and to the individualized graphs I’ve just figured out how to produce – it highlights each person’s connections) involved a lot of bubblegum and string.

  1. I typed in the data people had written down, using OpenOffice.org to form the upper triangle of an adjacency matrix. Two people’s sheets were missing, and one person didn’t have any connections incoming or outgoing. =( Thank you, programming competitions, for all those lovely data structures.
  2. I copied the adjacency matrix and pasted it onto itself using OOo’s Paste Special – Transpose, Skip Empty Cells. This gave me a full adjacency matrix.
  3. I used a really long and hairy OOo formula to concatenate the cells into Emacs Lisp code as an associative list, with extra information and an edge list.
  4. I copied that into Emacs and processed the associative list’s edges. I needed to do that anyway in order to be able to e-mail people personalized e-mail with all of their introductions, instead of sending one e-mail per edge. Along the way, I got the idea of visualizing the network diagram, so I spun off some code to output a full edge list in DOT format for visualization with circo.
  5. I used a command like
    circo -Gsplines=true < lifecampto.dot -Tpng > lifecampto.png

    to generate the graph shown.

  6. Then I thought it would be cool to personalize the graphs, too, so I wrote some more Emacs Lisp to generate personalized DOT files that highlighted the recipient in green and the recipient’s requested links/nodes in green, too. I used a Bash for loop to turn all those personalized DOT files into PNG files.

Example of a personalized image:

Tomorrow, I’ll work on the mail merge. =)

A little computer science is a dangerous, dangerous thing.

Quantified Awesome: Analyzing time data–the questions I ask and how I answer them

I track my time using QuantifiedAwesome.com because I’ve built an interface that fits the way I work (mobile/web, lets me backdate entries, lets me disambiguate categories with text), but you can use whatever works for you – even a paper notebook where you write down the time and the category.

Here are some of the questions I ask about time and how I slice the data to answer them.

First level of analysis

There are a lot of things you can quickly analyze based on time, particularly if you have durations already pre-calculated. Here’s what I often look at:

How much sleep am I getting? To answer this question, I split my time records by midnight so that I can easily get the sum of sleep durations per day. This accounts for naps and late nights much better than just looking at the starting timestamp does. I can quickly check my sleep length by looking at my dashboard, which shows me how much sleep I got the previous day. I also have some Emacs Lisp code that gets the data from QuantifiedAwesome using an API and calculates my average sleep for a week, which I include in my weekly reviews.

Am I working too much? I want to keep my “Business – Earn” total to less than or equal to 20 hours per week (50%), and my total business-related hours to be less than 44 hours per week. I do this because otherwise work can be tempting to focus on, and I want to remind myself to do other things as well. This is reported on my dashboard, and I also see it in my weekly reviews.

How much discretionary time do I have? How much time do I have for hobbies, socializing, and other activities outside work, chores, personal routines, and sleep? This helps me appreciate the freedom I have in each day and to focus on making the most of it. I can quickly see this by looking at the “Discretionary” row in my time review, or by adding categories and summing up the times in my spreadsheets. This is particularly mind-boggling to look at over a year. I had 1563 hours of discretionary time in 2012 – that’s a decent-sized time for developing skills or building relationships.

What do I spend my discretionary time on? How much am I using for socializing, productive interests, and relaxation? I answer this question by looking at my time review or by creating pivot tables in my spreadsheet. It turns out that I spend more time on social things than I expected, so I don’t feel as guilty about blocking off time to work on my own things. I also have a lot of productive hobbies, so I can give myself more permission to play with less productive things or give myself downtime.

Second level of analysis

This might involve throwing your data into a spreadsheet and playing around with it. Here’s where I start digging into patterns and correlations.

If I spend more time on some activities, where do I spend less time? I looked at correlations for time spent on various activities per day.

How consistently do I do things? I’m curious about whether my sleep times, bedtimes, workload, etc. vary wildly from week to week or if they’re fairly stable and predictable. It’s easy to get a sense of this by looking at graphs and calculating standard deviations.

How does my bedtime affect my wake-up time? I compared starting timestamps with ending timestamps, discarding naps and differentiating between weekdays and weekends. (I should rerun this analysis now that I have more control over my wake-up times…)

How frequently do I write? I extracted the date from each timestamp and visualized it using a heatmap.

How consistently do I bike, and when’s the earliest I started biking this year? How much have I saved by biking? I review my bike time or visualize it as a pivot table, a bar graph, or a heatmap in order to see patterns. (I biked in January! =D) Since most bike trips replace public transit trips (currently $2.60 per trip), I can also use my time data to estimate how much I’m saving. (259 trips in 2012 = ~$670+, 122 trips so far in 2013.)

Do I need longer chunks of time to concentrate? I looked at discretionary activities and counted how many took 4+ hours, 3 hours, 2 hours, 1 hour, or less than an hour. I look at the character of the activities, too, so that I can figure out what I might not be working on if I only have short periods of time. It turns out that I get a lot of things done even in 1-2 hour chunks, and I tend to not take advantage of longer chunks even if I have them. (Hmm, I should do an analysis to see the longest chunks of straight discretionary time I have…)

Does waking up early give me more discretionary time or longer chunks of time to work with? It turns out that I actually have a lot of “me” time even if I wake up at ~8 AM. Since this meshes well with my sleep needs if I stay up to 11 PM or 12 midnight, I can guiltlessly sleep in knowing that I’m not missing out on a lot of productivity.

How does bulk cooking affect our time? How often do we do it? I look at sparklines and pivot tables to get a sense of when we’re doing lots of bulk cooking. (I should analyze this to find out how much time we actually save and where it goes to…)

How do my patterns shift over time? What hobbies did I pick up or let go? How does my life adapt to external events and other commitments? Pivot tables, sparklines and line graphs are great for looking at the patterns in my data.

How much time does it take me to get somewhere compared to, say, the estimates from Google Maps? I can figure this out by looking at the estimate before the trip and then comparing it with the time I’ve logged (door to door ). For example, a recent bike trip that was 32 minutes in Google was actually more like 43 minutes (including waiting for downtown traffic lights and finding someplace to lock up my bike). My morning commute is 36 minutes according to Google and around 48 minutes by my clock.

Third level of analysis: What do I want to do with my life?

This is where I take a step back and check: Am I happy? Is W- happy? Do I need to shift the way that I spend my time? What would I like to move my time towards? Am I getting the kind of value that I want to get out of my time? Then I can experiment. For example, I’m currently experimenting with increasing the time I spend exercising, and I’m curious about how that affects other parts of my life.

Wrapping up…

You can get a ton of information out of simple time tracking, even without anything else to correlate it with. Durations, start and end times, frequencies… There’s a lot you can do with a spreadsheet, some charts, or your own tools.

It gets even more interesting when you start matching it up with other data. One day I should try comparing my bike data with temperature and wind, or time spent cooking with how many portions we produce and at what cost per portion. =)

I get a lot of value out of my time-tracking. It helps me stay focused and be aware of the moment. I like reviewing and analyzing my data, too. I’m experimenting with ways to capture short sprints of more detail so that I can ask even more questions, and I love comparing notes with other people who track their time.

Check out more posts about my Quantified Self tracking!