Category Archives: quantified

On this page:

Quantified Self: a year of grocery data

I started tracking our grocery expenses when we decided not to sign up for a community-supported agriculture program. I’d tracked several seasons of the CSA, and I wanted to see if we would still eat lots of vegetables without the bi-weekly commitment of a farm share. I also wanted to get a sense of what we bought.

I started scanning my receipts, and I found an assistant who could type them in. I set up a spreadsheet where he could type in the dates, stores, and line items (including quantity, unit price, and total). There were occasional typos, but I could find and fix them. I used a look-up table to match the line items with friendlier names (ex: RDPATH SUGAR is white sugar) and file them under categories.

The data below isn’t complete because there were a number of receipts that slipped through the cracks. If I let too much time pass between data updates, I couldn’t remember what some things were. Still, it should give a general idea of how the year went. The data covers April 2012 to March 2013 and includes 1223 line items.

Here are some questions I wanted to explore:

  • A. How much did we spend in various categories, and how does that vary month by month? For example, how much do we spend on vegetables? Is this in line with what we want our diet to be?
  • B. What items do we spend the most on? This could point to better ways to economize (buying in bulk, finding cheaper choices) or show us where it’s worth spending on better quality because we use so much of it.
  • C. How frequently do we buy certain items? Can we predict consumption patterns or sale patterns, and stock up when things are on sale?
  • D. What are the normal prices and the sale prices for various items? When and where does it make sense to buy different things?

So, let’s see! Click on the images to view larger versions.

A. How much did we spend in various categories, and how does that vary month by month?

image

Grocery expenses worked out to be $422/month for this family of three in Toronto, Canada. We ate pretty well, enjoying our favourite foods, the occasional snack, and fresh fruits and vegetables, and buying organic milk (which turned out to be a large part of the budget, but probably worth it). Because we cook in bulk, some months have larger grocery bills and some months involve more freezer-raiding. The standard deviation was $160.

We spent the most on meat ($59/month) and dairy ($53/mo), but fruit ($47/mo) and vegetables ($46/mo) also made a respectable showing. Vegetables worked out to $22 every two weeks, which is less than what we were paying for the CSA box. That could be accounted for as a pricing difference between conventional and organic produce, and we still bought extra vegetables when we were in the CSA. Paying attention to our increasing vegetable spending helped us learn lots of ways to prepare food. Yakiudon turned out to be a house favourite, and other stir-fries are great too. We haven’t been able to get our vegetable spending to overtake meat, but that’s probably because of the occasional indulgence in lamb korma.

The month-by-month pattern made me think there were bigger differences, but because stocking up and bulk cooking means our monthly patterns probably aren’t a good source of information. Our vegetable spending is positively correlated with our overall grocery spending (0.7), which means that ~50% of the ups and downs are explained by the ups and downs in our grocery bill (maybe we just bought less).

Anyway, I feel pretty good about how the proportions worked out. There’s hope for us yet!

B. What items do we spend the most on?

image

We buy milk because J- likes it. It turns out that Canada prohibits the use of artificial growth hormones for dairy cows and antibiotics are also controlled, but we still get the organic version for extra safety. Lamb shanks from the butcher are a splurge when it comes to making lamb korma. We found that Metro often has the lamb cuts we want, though, so we check there first now. Shrimp sees a lot of use in pad thai, stirfries, and other wok-based dishes, plus our occasional wonton marathons. My standard breakfast is rice and fried egg, and we use lots of eggs in baking and stirfrying too. I was surprised that much butter (and we do, even though we try to stock up during sales!) because of baking, and that grapes made it into our top ten despite being something we don’t eat that often. We buy grapes only when they’re super-crisp, and sometimes we forget to eat all of them before they soften. Also, we usually buy chicken legs or drumsticks, but it was interesting to see that whole chickens turned up on this list even though we don’t buy them frequently.

C. How frequently do we buy certain items?

A block of butter, a carton of 18 eggs, and a bag of 4L milk every 1.5 weeks (eggs and milk feel more frequent than that, though…)
A 2kg bag of white sugar every ~2 months, a 1kg(?) bag of demerara sugar every ~6 months

It’s a little harder to tell how often things go on sale and how much we want to stock up, because we skip sales if we still have stuff in stock (ex: butter) and we shift our buying patterns depending on what’s on sale (ex: 30% on a particular meat package that’s nearing its best-by date). It looks like butter is always good to get on sale, though, and that seems to be every other month.

D. What are the normal prices and the sale prices for various items?

Hmm, I think it might be useful to remember which ones sometimes go on big sales, so then it makes sense to postpone until things are in season.

Butter is usually $4.97, sometimes $2.88. Salami is sometimes $4.20 off ($5.29), clementines are sometimes $3 off ($3.99), and bacon is sometimes $2.58 off ($2.97). Sometimes we can get Japanese udon noodles for $1 instead of $2.19. And then there was that time that Campbell’s condensed chicken soup was on sale for $0.50 instead of its usual $0.97, and we bought a lot. =)

In general, our neighbourhood No Frills supermarket has pretty prices for stuff, although some things necessitate a special trip to the Sweet Potato organic food store or the Welcome or Oriental Harvest ethnic supermarkets. Metro also stocks some sauces and lamb cuts that are hard to find elsewhere. I sometimes look up prices from my records, but the difference is usually pretty small.

 

So that’s roughly a year of data. Hmm… Should I continue? Maybe I’ll scan and stash the receipts, but I might not have someone type in the information until I have more questions I want to ask. It was interesting to collect that data over a decade, though!

Quantified Self time-tracking: Choosing your buckets

This post missed its publishing schedule. I’m posting it today so that it doesn’t get lost.

Kate asked me how I chose the categories I use for tracking my time, and if I had any tips for someone who’s starting out.

I track my time at a medium level of detail – not so high-level that I can’t ask interesting questions, but not so low-level that it’s hard to summarize. To select an activity category (the non-bolded text in the table below), I type in parts of it. For example, “un subway” becomes “Unpaid work – Subway” and “quantified” becomes “Business – Quantified Awesome.” If something is ambiguous, the system shows me all the matches and lets me pick one. I can make some activity categories inactive so that they don’t get matched by the search. If the text doesn’t match anything, I’m shown the category creation screen, and the timestamped record is automatically created once I create the category for it. For “Other”-type activities and other activities that I’ve added a note field to, I can add a pipe character followed by a note (ex: “disc other | Yada yada yada”) for more details.

Here’s the general structure. I based the top-level categories on the OECD time studies so that I can compare my numbers with averages from other developed countries. The top-level categories I use are:

  • Business: Anything related to entrepreneurship or professional development
  • Discretionary: Hobbies, socializing, and other ways I choose to spend time
  • Personal: Personal care, daily routines, exercise, and things you can’t outsource because the point of the activity is personal benefit
  • Unpaid work: Chores, commuting, and other things you could theoretically outsource or eliminate
  • Work: Working as an employee; also, the occasional work lunch
  • Sleep: Sleep and naps!

The second-level group (Business – Build, Business – Connect, etc.) are the ones I recently created for reporting purposes. They look useful, so I might figure out how to build them into my database for more reporting goodness.

Within those major groups, I have one or two levels of record categories that I really use to track time. The higher groups are just for reporting. I create more activity types as needed.

Business
Business – Build
Business – Android
Business – Book review
Business – Business development
Business – Coding
Business – Delegation
Business – Drawing
Business – Learn
Business – Marketing
Business – Other
Business – Paperwork
Business – Plan
Business – Quantified Awesome
Business – Research
Business – Sales
Business – Connect
Business – Connect
Business – Correspondence
Business – Presentation
Business – Pro bono
Business – Earn
Business – Consulting – E1 – Conf
Business – Consulting – E1 – General
Business – Consulting – R1
Business – E-book
Business – Illustration – I1
Business – Illustration – I2 – UPV
Business – Illustration – I3 – M
Business – Illustration – I4 – SR
Business – Illustration – I5 – MT / G
Business – Sketchnoting
Discretionary
Discretionary – Other
Discretionary – Other
Discretionary – Play
Discretionary – Harry Potter (… because I forgot I already had “Discretionary – Play – LEGO Harry Potter”)
Discretionary – Play – Final Fantasy
Discretionary – Play – Katamari Forever
Discretionary – Play – LEGO Batman
Discretionary – Play – LEGO Harry Potter
Discretionary – Play – LEGO Heroica
Discretionary – Play – LEGO Indiana Jones
Discretionary – Play – LEGO Lord of the Rings
Discretionary – Play – LEGO Pirates
Discretionary – Play – LEGO Star Wars
Discretionary – Play – Nethack
Discretionary – Play – Other
Discretionary – Read – Nonfiction
Discretionary – Relax
Discretionary – Productive
Discretionary – Emacs
Discretionary – Gardening
Discretionary – Latin
Discretionary – Read – Blogs
Discretionary – Read – Fiction
Discretionary – Sewing
Discretionary – Tracking
Discretionary – Travel
Discretionary – Writing
Discretionary – Social
Discretionary – Family
Discretionary – Social
Personal
Personal – Exercise
Personal – Bike
Personal – Exercise
Personal – Scoot
Personal – Walk – Home
Personal – Walk – Other
Personal – Walk – Subway
Personal – Walk – Work
Personal – Life
Personal – Eat – Breakfast (… sometimes I track meals separately, but usually they’re just part of Personal – Routines)
Personal – Eat – Dinner
Personal – Eat – Lunch
Personal – Plan
Personal – Planning (… because I forgot I already have Personal – Plan)
Personal – Routines
Sleep
Sleep
Sleep
Unpaid work
Unpaid work – Commute
Unpaid work – Subway
Unpaid work – Wait
Unpaid work – Errands
Unpaid work – Errands
Unpaid work – Groceries
Unpaid work – Home
Unpaid work – Clean the kitchen
Unpaid work – Cook
Unpaid work – Laundry
Unpaid work – Tidy up
Unpaid work – Other
Unpaid work – Other
Unpaid work – Other travel
Work
Work
Work – C
Work – Lunch
Work – O
Work – Other
Work – T

I track business projects as their own categories so that I can bill for my time or figure out if something was worth doing. I track games separately so that I can figure out what I spend more time on.

I usually create a tracking record at the beginning of the activity so that quantifiedawesome.com can timestamp it. If I forget, I can say things like “-15m relax” to note that I started relaxing 15 minutes ago, or say things like “13:30 writ” to note that I started writing at 1:30 PM. If I’m seriously late, I can specify the date like this: “3/24 19:05 social”, or use the batch entry form. When I record an entry, the system shows me the edit form, so if I was wrong (I thought I was going to start Personal – Routines, but really, I went back to sleep), I can change the category using a dropdown and save it. I can also adjust start and end times, and the previous or next record is automatically adjusted too.

I track my time based on the primary activity so that I don’t double-count the time. For example, if I’m taking the subway, I file it as “Unpaid work – Subway” instead of “Discretionary – Read – Nonfiction” even if I read a book during the trip.

I have to build some kind of split/merge/refactor activity category tool someday, but so far, this is fine. And more reports! Reports are fun.

Tips and lessons learned:

If you’re starting out, a simple thing that lets you capture some text with a timestamp will work just fine. Jot down a few keywords that explain what you’re doing – enough to remember. Do this for a few days to a week in order to get a sense of what categories you may want to file things under.

Once you’ve figured out what general categories you want, use a button-based tracker like Tap Log or a list-based tracker like Time Recording (both Android). They’re great for selecting something from a defined list or structure. The downside is that it takes a liiittle more time to add a new category.

When you have lots of categories, going back to text input makes a lot more sense. No scrolling, no clicking around, and you can add new things fairly quickly. The substring search I put into quantifiedawesome.com works really well for me because I know which shortcuts map to which categories, and the structure is better than freeform text because reporting is easier.

Reporting is a lot more fun if you’re comfortable with spreadsheet pivot tables and other nifty features. I should do a screencast of how I use Excel to slice and dice my data. =)

Next step for me: Time estimates

I’ve started recording time estimates for more detailed tasks/activities so that I can a) figure out if I routinely overestimate or underestimate certain things, and b) get finer-grained time data. I know that it takes me roughly an hour from the time I get up to the time I get out of the house with my usual morning routine and maybe half an hour for the rush version, but it would be great to break that down into components and perhaps experiment with it. I’m also curious about how much time it takes me to get to places so that I can adjust Google Maps estimates for walking, biking, or public transit.

I write predictions down in Evernote (“Predict home by 7:05”). Evernote automatically timestamps the creation date, and I update the note with the actual time and any other notes I want to include (“home at 7:03; bike”). When I have several estimates and measurements, I’ll make a spreadsheet. When the spreadsheet structure settles down, I might build the functionality into Quantified Awesome. Successive prototyping helps me figure out how the data feels before I spend time building a structure for it. =)

So that’s how I track my time! See Where the Time Went for a recent presentation sharing my results.

Quantified Self Toronto: Where the Time Went

Update 2013-6-6: Added a link to the video!

Carlos Rizo convinced me to quickly throw together a presentation for today’s Quantified Self Toronto meetup. Here are my slides!

Check out quantifiedawesome.com for my data, dashboard, and source code, and read through my Quantified Self blog posts for more geekery.

Quantified Awesome feedback

I just realized that my feedback form on Quantified Awesome didn’t let people fill in their e-mail addresses, and people didn’t think of adding it to their message… d’oh! So I’ll just respond to questions here and hope that people who checked out Quantified Awesome happen to also have subscribed to my blog.

Hi Sacha,  I looked at the statistics that you are collecting here (at quantifiedawesome.com).  One one hand it seems very beneficial for personal development/goal tracking and other uses.  On the other hand the feeling that someone else might read my stats and know in so much detail how I live gives my shudders.  Is there a way to keep one’s privacy while collecting useful personal stats?

My data is the only one that’s public. Theoretically, I have access to the database, but I won’t post anything. Also, there’s no SSL, so someone could potentially sniff your password and look around. But it’s probably okay. If you really want to keep things private, I suppose you could track using pen and paper or a spreadsheet on your computer.

Your blog is crazy interesting to me.  I’m curious about your quantification approach.  Any inspiration from Nicholas Felton at feltron.com?
Oh and crop your clothes photos more closely.  Don’t need to see the hanger and wall but more detailed shot of the clothes would be nice.
Best
Pat

I checked out his annual report some time ago. It’s pretty, and people often point me to it. The data visualizations are neat, but I’m even more curious about his ongoing data collection and analysis processes. =) Clothes photos: That’s possible, but I can’t be bothered at the moment. Winking smile Minimal effort wins out.

Quantified Awesome: Time and building mastery

image

There’s an often-repeated number in studies on expertise: it takes 10,000 hours of deliberate practice in order to build a skill to mastery.

Last year, I logged 265 hours writing, or around 45 minutes a day. Don’t be scared off by that. If you’re repurposing something you’ve already done, blogging takes maybe 5-10 minutes. It takes an astonishingly long time to think through new things. I can type at 90-110 words per minute, but my brain chugs along at 16 words per minute when reflecting, and I haven’t quite gotten the hang of using speech recognition or dictation to get past that barrier. I suspect I won’t be significantly faster. Thinking takes time. That 265.5 hours is butt-in-seat time. Yes, that’s the professional term for it. I have surprisingly little of it, considering how much I perceive writing to be a part of my life. (Really? Just 45 minutes a day? What would happen if I doubled that?)

Generously including quick blog posts as part of this practice, assuming that I’ve maintained a similar pace since around 2003, neglecting the fact that real writing involves a whole lot more rewriting (which I tend to do out of forgetfulness rather than deliberate improvement), and ignoring the assigned writing I slogged through in school (or the countless e-mails I dash off), the calculations show that I’ll probably be inching closer to awesomeness… oh… when I’m 57 or so. I know life doesn’t quite work out like that, but let’s pretend for the sake of calculation.

It doesn’t actually look half-bad, you know. If I can get a decade or two of great writing out right around the time I should have tons of experiences to write about, that should be fine. Of course, with the unreliability of memory (both mine and the computer’s), I’ll just have to hope my blog will survive the years. And if I turn out to be a passably good writer who can package up what I’ve learned and turn it over to the next generation of young whippersnappers, then that’s great. Don’t need to win any awards.

On a similar note, I logged 198 hours drawing last year. This does not include the times that I filed drawing under “Business – Plan” or “Personal – Plan” instead. Even less revision going on over here, but I’m working on ways to improve that, and other ways to increase the proportion of writing and drawing in my life. Malcolm Gladwell’s Outliers: The Story of Success broke down that 10,000 hours into around 10 years of daily 3-hour practices.

I have spent a ton of time coding. I’ve been coding since I was six or so, and I worked on quite a few web development engagements at IBM. I’m still nowhere near “master” level. I can point to lots and lots and lots of people who are way better than I am. I have fun with it, though. I can make stuff happen.

It’s good to remember that invisibly sunk time, the accumulation of experience over years. That way, I don’t get frustrated about drawing if it feels less natural than coding, and I can see all this writing as building a skill step by step.

Thank goodness for visible progress. Hooray for a blog that lets me go back in time! Compared to a year ago, I draw so much more than I used to. I feel a little more organized and coherent as a writer – headings for my paragraphs, an index for my posts. I still don’t have the deft interweaving of personal story and insight I envy in Penelope Trunk’s blog or the lyrical geekiness of Mel Chua (no relation), but I’m growing into my own style.

I think it would be fascinating to have 10+ years of time data. It’ll be interesting to see how I change things along the way. Not that quantity of time controls everything, but it’s fun to ask questions and realize that the composition of your time doesn’t always match up to what you think it is. Then you can tweak it.

Other 2012 numbers to put this into perspective, because I have the data anyway:

image

  • 3024 hours of sleep – 8.3 hours per night and I’m still fidgety when I go to bed; I wonder how one deliberately practises sleeping
  • 729 hours socializing (in person, answering e-mail, etc.) – not much deliberate practice going on there, but good to spend that time
  • 411 hours connecting with people for business (in person, answering e-mail, etc.) – a little bit of systematization and experimentation
  • 102 hours reading fiction, 88 hours reading nonfiction – funny, I thought it would be the other way around, although some of it did get classified as “Business – Drawing” instead
  • ~75 hours playing LEGO video games

More time analysis looking at percentages

So that’s where all the time went!

Quantified Awesome: Taking inventory of stuff in closets and drawers

“Did I buy that aviator hat, or was I just thinking about it? Did I give it away?” “Have you seen my Twiddler one-handed keyboard?” “Now where did I put those markers…”

Sometimes I find it hard to remember where I’ve put stuff, or even if I bought the stuff in the first place. I think so carefully and so vividly about whether I want to buy some things that I can find it difficult to distinguish between memories of buying and using that thing versus my imagined tests of whether I would use it if I bought it.

It’s useful to know what stuff you have and where. It means not needing to buy things again. It means not wasting time turning the house upside down. It means being able to cut down on clutter instead of letting it invisibly pile up.

I’m no stranger to using external tools to get around the limitations of my brain. Since my Quantified Awesome dashboard includes some support for tracking stuff, I spent 45 minutes adding a bulk-entry interface and making sure it updated my main list.

image 

With my newly-improved stuff tracking system, I spent two hours taking inventory of various things stashed in drawers, tucked away in cabinets, and otherwise placed in forgettable locations. Along the way, I tossed out old business cards, miscellaneous electronics, and other clutter. I consolidated things so that there was one place for all the index card containers I had. I tallied a total of 207 items – not everything in the house, but a good start.

It turns out that I didn’t buy that aviator hat after all, or if I had, I donated it. I found the Twiddler keyboard in basement drawer #4. The markers were in my backpack.

Having answered those questions, I can rest – or at least, until another half-memory sends me searching.