August 2011

Looking back at life as a 27-year-old

August 1, 2011 - Categories: review, yearly

I’ll be celebrating my 28th birthday this August. Hooray!

I spent part of the Civic Holiday reviewing my blog posts and memories. It’s been a wonderful year. In August 2010, I wrote:

What will life as a 27-year-old look like? I’m excited about long-term growth: marriage, work, friendships, interests. I’m looking forward to small, constant improvements in the way we live. I want to get even better at learning and sharing. When I turn 28, I hope to be able to look back and say that I:

  • helped build an excellent foundation for a loving partnership
  • made a difference at work and grew in my career
  • experimented with ways of living better and shared my results
  • shared lots of ideas, questions, and insights with people
  • lived another year of an awesome life.

… and looking back, I can say that and more.

Life as a 27 year old was mainly about preparation. W- and I got married in October in a ceremony as small as we could get away with – just us, immediate family, and practically-family friends. I’m glad we did. The paperwork helps us make our long-term plans more solid, and it feels great to namedrop “my husband.” We celebrated my middle sister’s wedding in May, too. I’m curious how married life will influence the way my sisters and I interact.

We learned a lot about communication while planning our wedding, mostly inspired by Jay Heinrich’s excellent book Thank You for Arguing and our decision to keep both our names. I’m sure those skills will come in handy in the future as well. We’ve also been developing our teaching skills while helping J- and her friends learn more about math and other subjects. It’s worth investing the time into making things understandable and enjoyable.

At work, I’ve been developing some very useful skills: gathering requirements; scoping and estimating projects; working with Drupal, Ruby on Rails, and Websphere (not all together, fortunately!); working with other companies and teams. It’s been a bit more stressful because I’ve taken on more responsibilities, but it’s good preparation for many possible next steps: consulting and development in IBM, or perhaps a startup if a business opportunity seizes my imagination. The richness of my extracurricular life means I’ve cut back a little on non-project work – the presentations I used to do, the communities I used to help out with – but I’ve still been able to help make many things happen.

Lots of preparations outside work as well. We’ve been tweaking our household routines – simplifying the kitchen, improving the entrance flow, reorganizing the living room. We’ve added more vegetables to our life thanks to a community-supported agriculture program with the stereotypical abundance of zucchini. I’m getting better at gardening. We got more vegetables and fruits out of the garden this year, although bitter melons were a non-starter.  Oh yes, this was the year we disassembled a washing machine and a dryer in order to get the 27” machines down a 26” staircase. I hadn’t seen that coming at all, but it was a wonderful experience.

Lots of reflection and analysis this year, too. With a few exceptions, I’ve been able to share at least one blog post a day for the past year, and that’s been really helpful for reconstructing and remembering. The Quantified Self meetups in Toronto have been inspiring me to measure, analyze my decisions, and review them afterwards. Printing out my blog and flipping through the stories has helped, too. It’s interesting reading things I’ve somewhat forgotten writing. There are my reflections on routinely waking up at 5 AM – did I, before? But it sounds like me, so I must have. Perhaps I’ll try that again.

I’m getting better at drawing. I’m starting to feel more comfortable playing with colors and sketching ideas, relying less on the ability to nudge drawings into the right shape and more on the ability to repeat sketches until they feel right. I take informal sketchnotes at the events I go to, and I’m starting to develop note-taking and presentation workflows that fit me well.

I levelled up in terms of personal finance by opening a discount brokerage account and investing in index funds. The market has been up, down, and sideways, but I’m going to keep investing anyway. It’s reassuring to see that nest egg grow, even though it grows slowly. While the returns are nowhere near the breathless rates I read about in personal finance books written before the financial crisis, they’re okay. Updating my books makes me feel a little more grown-up, even though I don’t go for anything more complicated than passive growth.

My interests shifted, unsurprisingly. I haven’t sewed as much as I thought I would; other hobbies keep me busy. Working at home means I’ve been biking less than I used to. I haven’t fired up our woodworking tools even once this summer. But there are new and renewed interests that fill my hours: writing, drawing, piano, learning Latin with W- and delighting in unexpected discoveries. For example, I learned cras is Latin for tomorrow, which made me think about the word “procrastinate” – ah, “for tomorrow”,  doesn’t that make perfect sense now…

I think the year ahead of me – life as a 28-year-old – will be a year of slowing down, polishing, and finishing. It will be interesting. Drawings, links, and plans to follow.

Rhetoric and advocacy: the value of a different approach

August 2, 2011 - Categories: Uncategorized

UPDATE: Changed the title from “the value of the right approach” to “the value of a different approach” – thanks to Aaron for the nudge!

I was thinking about how to respond to this. I found myself wanting to share rhetoric tips, so I’m posting this as a blog entry instead of a comment. =)

On my post about the Manila Zoo, Anna commented: “Don’t you love animals? Then why are you eating them? What’s the difference between the animal that you ‘love’ and the animal that is on your plate? If you really love them, you’ll stop having them for dinner.”

One of the benefits of learning about rhetoric and argument is being able to recognize what’s going on. Here, Anna tries to set up a dichotomy: either you love animals and are vegan, or you eat animals and don’t love them. Relying on such a premise weakens Anna’s case. I don’t have to accept this premise, and I can see other choices.

This looks like an inarguable situation: she’s not going to convince me to adopt a vegan diet through these words, and I’m probably not going to convince her to be more precise and more compassionate in her rhetoric. But I’d like to explore this anyway, because there’s something interesting here about the difference between what she’s trying and how I’d do it. (When life gives you lemons, write a reflective blog post about them!) If I were in Anna’s shoes and I wanted to nudge someone to move towards a more plant-based diet, here’s what I would try.

You can very rarely make someone do something. If you want to influence someone’s behaviour, you have a much better chance if you can inspire them rather than if you criticize them or force them. Part of that is building a bridge between the two of you so that the other person can understand and listen to you, and part of that is helping the other person imagine how much better their life would be with your proposal.

I know that can sound frustrating and slow. There have been many times I wished I could just wave a magic wand (or write a program!) to get people to change their behaviour, understand a new concept, or stop e-mailing huge files around. But in all the books I’ve read and through all the coaching I’ve done, I keep coming back to these lessons again and again: you can’t change people’s minds for them, and influencing cooperation can be much easier than sparking conflict.

So I would start by building common ground, instead of approaching it antagonistically. This is a common mistake for radicals, influencers, and people carried away by their passions. Goodness knows I’ve got enough examples of doing this myself in the early years of my blog. When you get stuck in an “us versus them” mindset, it becomes difficult to connect with people in a compassionate, respectful manner. Instead of trying to imply that the person I’m talking to hates animals or is hypocritical, I’d probably start off by highlighting things we have in common. Something like this: “I’m happy to see you love animals a lot.” This validates what the other person has said, affirms them, and starts off on a positive note.

Then I would use personal experiences as a bridge, showing people I’ve been where they are and they can relate to me. If you want to make it easier for people to see what you see, you need to show them that you’ve stood where they stand, acknowledging challenges along the way. That way, you can connect with people and help them be inspired. In this hypothetical argument, it might be something like “I love animals too, which is why I’ve been shifting to an all-plant diet. It’s sometimes hard to stick with it, particularly when I’m hanging out with friends, but it’s easier when I remember the troubles animals go through and the kind of world I’d rather build for them.”

I’d soften the call to action. People don’t like being manipulated by false dichotomies or preachy advice. I would probably explore the waters with a question like, “Have you thought about shifting to a vegetarian or vegan diet, too?” By backing off a little, I acknowledge the other person’s choices and reasons instead of trying to make decisions for them.

Depending on whether I thought it was necessary, I might include some social proof or alternative reasons. For example, plant-based diets can be healthier and less expensive than diets with a lot of meat. They can have a smaller environmental footprint, too. It’s good to anticipate and acknowledge the difficulties. Growing plants isn’t automatically guilt-free: see the clearing of land to support commercial agriculture; the dangers of monoculture, fertilizers, and pesticides; the consequences of transportation.

I’d end by showing my respect for people’s choices and finishing on a positive note. This would be a good place to thank the person again and highlight common ground, remembering that the goal isn’t to score points, but to open up a possible conversation enriched by personal experience. 

—-

So here’s what that might look like, if I wanted to influence someone to eat more vegetables and fewer animals.

Before: “Don’t you love animals? Then why are you eating them? What’s the difference between the animal that you ‘love’ and the animal that is on your plate? If you really love them, you’ll stop having them for dinner.”

After: “I’m happy to see you love animals a lot. I love animals too, which is why I’ve been shifting to an all-plant diet. It’s sometimes hard to stick with it, particularly when I’m hanging out with friends, but it’s easier when I remember the troubles animals go through and the kind of world I’d rather build for them. Have you thought about shifting to a mostly-vegetable, vegetarian, or vegan diet, too? I’ve found that it usually comes out cheaper than my old meals, and I feel healthier and more energetic too. Hope to hear from you soon!”

Your mileage may vary, of course. You might feel that this more compassionate I’m-on-your-side approach is too mild for you. I present it as an alternative, so it’s easier to see that not all advocacy has to be confrontational.

Having reframed the comment in a more positive tone, what would be my personal response to it? I’m aware of the arguments for and against vegetarianism and vegan diets. I do eat mostly vegetables, thanks in part to a community-supported agriculture program that keeps me busy figuring out what to do with zucchini, in part to concern over what goes into the food that goes into us, and in part to a stubborn frugality that dislikes paying the premium for steak. I don’t think I’ll ever follow a strict vegetarian or vegan diet, though, because I don’t like inconveniencing friends and family, or proselytizing at the kitchen table. I’ll follow my own decisions when it comes to food I can control, but I’ll try to go with the flow when it comes to what people share with me. (I still opt out of balut and other things that make my mind boggle, although many people consider such things delicacies.) So even this tweaked message isn’t going to make my decisions for me, but it will leave me with more respect than aversion to how people try to get their messages across.

Parting thoughts: If you come to a conversation prepared for a fight, that’s what you’ll get. If you come to a conversation with love and compassion, you’ll have more opportunities to learn and grow. It’s amazing how much of a difference your starting point can make. It takes practice to be able to consider different approaches and choose one that fits, and, if necessary, to translate what other people say into what they might have meant. Hope to help more people think about and consciously choose how to approach conversations!

Learning by imitation: getting inspired by Exploding Dog

August 3, 2011 - Categories: sketches

I like the way Exploding Dog has these colourful, creative drawings. It is awesome. I’ve never seen anyone less afraid of saturated colours. Except maybe Hyperbole and a Half. And five-year-old kids.

One of the things I tell people who want to write is that it’s okay to start off being unoriginal. Imitation is a great way to learn. So I’m going to take that advice myself. I haven’t drawn enough to discover my original voice. I won’t get there until I try lots of styles.

This is me drawing like the way Exploding Dog makes me feel.

image

image

image

My goal is to grow down until I can draw like a five-year-old again, but with a twenty-seven-almost-eight-year-old’s ideas.

Drupal and JQuery 1.5: Fixing the JSON encoding of ampersands

August 4, 2011 - Categories: drupal, geek, work

Drupal 6′s drupal_json method encodes ampersands incorrectly for JQuery 1.5, causing the rather cryptic error:

Uncaught Syntax error, unrecognized expression: ...

(If you’re lucky.)

The way to fix this is to borrow the JSON-handling code from Drupal 7. Here’s something you might be able to use:

function yourmodule_json_encode($var) {
  return str_replace(array('<', '>', '&'), array('\u003c', '\u003e', '\u0026'), $var);
}

// Fix Drupal JSON problems from http://witti.ws/blog/2011/03/14/jquery-15-json-parse-error
function yourmodule_json($var) {
  drupal_set_header('Content-Type: text/javascript; charset=utf-8');
  if (isset($var)) {
    echo yourmodule_json_encode(json_encode($var));
  }
}

Use yourmodule_json instead of drupal_json wherever applicable.

Hat tip to Greg Payne (Witti) for pointing me in the right direction!

2011-08-04 Thu 14:01

Drupal, HTML Purifier, and embedding IFRAMES from YouTube

August 5, 2011 - Categories: drupal, geek, work

I know, I know. I shouldn’t allow IFRAMEs at all. But the client’s prospective users were really excited about images and video, and Drupal’s Media module wasn’t going to be quite enough. So I’ve been fighting with CKEditor, IMCE, and HTML Purifier to figure out how to make it easier. I’m hoping that this will be like practically all my other Drupal posts and someone will comment with a much better way to do things right after I describe what I’ve done. =)

First: images. There doesn’t seem to be a cleaner way than the “Browse server” – “Upload” combination using CKEditor and IMCE. I tried using WYSIWYG, TinyMCE and IMCE. I tried ImageBrowser, but I couldn’t get it to work. I tried FCKEditor, which looked promising, but I got tangled in figuring out how to control other parts of it. I’m just going to leave it as CKEditor and IMCE at the moment, and we can come back to that if it turns out to be higher priority than all the other things I’m working on. This is almost certainly my limitation rather than the packages’ limitations, but I don’t have the time to exhaustively tweak this until it’s right. Someday I may finally learn how to make a CKEditor plugin, but it will not be in the final week of this Drupal project.

Next: HTMLPurifier and Youtube. You see, Youtube switched to using IFRAMEs instead of Flash embeds. Allowing IFRAMEs is like allowing people to put arbitrary content on your webpage, because it is. The HTML Purifier folks seem firmly against it because it’s a bad idea, which it also is. But you’ve got to work around what you’ve got to workaround. Based on the Allow iframes thread in the HTMLPurifier forum, this is what I came up with:

Step 1. Create a custom filter in htmlpurifier/library/myiframe.php.

<?php
// Iframe filter that does some primitive whitelisting in a
// somewhat recognizable and tweakable way
class HTMLPurifier_Filter_MyIframe extends HTMLPurifier_Filter
{
  public $name = 'MyIframe';
  public function preFilter($html, $config, $context) {
    $html = preg_replace('/<iframe/i', '<img class="MyIframe"', $html);
    $html = preg_replace('#</iframe>#i', '', $html);
    return $html;
  }
  public function postFilter($html, $config, $context) {
    $post_regex = '#<img class="MyIframe"([^>]+?)>#';
    return preg_replace_callback($post_regex, array($this, 'postFilterCallback'), $html);
  }
  protected function postFilterCallback($matches) {
    // Whitelist the domains we like
    $ok = (preg_match('#src="http://www.youtube.com/#i', $matches[1]));
    if ($ok) {
      return '<iframe ' . $matches[1] . '></iframe>';
    } else {
      return '';
    }
  }
}

Step 2. Include the filter in HTMLPurifier_DefinitionCache_Drupal.php. I don’t know if this is the right place, but I saw it briefly mentioned somewhere.

// ... rest of file
require_once 'myiframe.php';

Step 3. Create the HTML Purifier config file. In this case, I was changing the config for “Filtered HTML”, which had the input format ID of 1. I copied config/sample.php to config/1.php and set the following:

function htmlpurifier_config_1($config) {
  $config->set('HTML.SafeObject', true);
  $config->set('Output.FlashCompat', true);
  $config->set('URI.DisableExternalResources', false);
  $config->set('Filter.Custom', array(new HTMLPurifier_Filter_MyIframe()));
}

Now I can switch to the source view in CKEditor, paste in my IFRAME code from Youtube, and view the results. Mostly. I still need to track down why I sometimes need to refresh the page in order to see it, but this is promising.

2011-08-05 Fri 16:34

Weekly review: Week ending August 5, 2011; thoughts on discretionary time

August 6, 2011 - Categories: review, weekly

We had a four-day weekend: Civic Holiday on Monday, then an IBM floater holiday on Tuesday (to balance the holidays other provinces get). It was much appreciated.

Plans from last week

  • Work
    • [X] Send mid-year update to manager
    • [X] Work on Project M issues/requests
    • Reviewed the estimates and scoping for project T
    • Had mid-year chat with manager, learned about Maqetta
    • Sketched another possibility for an IBM comic
  • Relationships
    • [/] Work on Latin – 30 minutes daily; three times this week!
    • [X] Help J- with piano
    • Posted more info on http://friendsofmanilazoo.org and Facebook page
  • Life
    • [X] Practise piano – 30 minutes daily
    • [-] Draw – also 30 minutes daily – not quite daily, but did pretty well
    • [X] Track a complete week of time
    • Drew colourful things!
    • Posted yearly update

Plans for next week

  • Work
    • [ ] Wrap up project M
    • [ ] Set up meeting for project T
    • [ ] Follow up with project O
    • [ ] Figure out what next week will look like
  • Relationships
    • [X] Volunteer to lead a build class at Free Geek
    • [ ] Hang out with W-’s family
  • Life
    • [ ] Sketch yearly update
    • [ ] Practise piano some more
    • [ ] Decide whether to continue piano lessons

Time tracking

Activity Hours Notes
Break 6.5
Cooking 8.8 Big cooking batch this week
Drawing 4.8 Yay colours!
Eating 7.0
Exercise 2.6 Biked to work
Gardening 1.2 Tidied up dried peas
Latin 4.3 Plugging away at “Latin Made Simple”
Piano 4.5 At least half an hour each day
Reading 1.6 Plus the books I’d read at breakfast, etc.
Routines 8.8
Shopping 11.0 Lots of shopping: replaced our microwave and our toaster oven
Sleep 60.8 Average of 8.7 hours a day
Social 6.5
Tidying 8.6
Work 24.2 4-day weekend, yay!
Writing 6.5

Nudged by Christopher Olah, I’ve resumed my time tracking. This time around, I’m particularly interested in how much discretionary time I can work with. It’s a little like tracking your expenses so that you can see how much you can save and if your expenses are in line with your priorities.

This week was unusual, thanks to a four-day weekend. Comparing it with the last time I summarized my time use, though, I can see some similarities. I still sleep for about eight and a half hours each day. I still work around 40 hours in a typical week (24 here, for a 3-day workweek) whenever I can. If I look at the daily breakdowns, I can get a decent idea of what discretionary time looks like on a weekday, a weekend, or a holiday. I’ll need more data points, for sure, but I can also check my numbers against my old time records.

Based on my numbers, I have around 4 hours of discretionary time each workday. This doesn’t include the daily routines of cooking, eating, tidying up, and so on. This is just the time I can choose to spend on writing, drawing, learning Latin, practising the piano, reading, connecting with people, or following other interests.

This block of four hours sounds familiar. I’ve come across it in other places, like in my post about how long it takes me to prepare a presentation (and thus the interest in increasing ROI). I think I’ve written about it in the context of time-tracking and unstructured time activities, too. Four hours is enough of a chunk for me to get into the zone on something.

Right now, I split that 4-hour discretionary time chunk into daily activities instead of devoting each chunk to one activity. For example, I might use half an hour to an hour for piano practice, half hour for drawing, an hour for Latin, and the rest for writing and socializing. This doesn’t give me the benefits of a deep dive, but it does mean I can space learning out a bit and give my brain time to integrate new information into long-term memory. Practising piano for half an hour each day is much more useful than spending four hours at the keyboard one day and neglecting it the rest of the week.

I’ve tried it the other way, using chunks of time for focused activities. Sometimes it’s the only way to focus, like when I’m working on open source. But skipping writing for too long makes me antsy, and trying to write seven blog posts in one go makes me miss out on that practise of daily reflection and improvement. I think I’ll take the gang-of-snails approach rather than the hare’s relay: slow and steady progress on my key priorities instead of focused sprints.

Splitting my discretionary time up into different activities also means I tend to spend less time on things with higher startup costs, like sewing. That’s okay. There’ll be time for that too, maybe when these other projects get wrapped up or when the need for it arises. Weekends are a good time for focused chunks. We need to see how the build classes we’ve started doing at Free Geek Toronto will shift our weekend routines, though. We’ll see how the numbers go.

Related:

Jalapeño jelly

August 7, 2011 - Categories: cooking, life

I stirred the jalapeño jelly mixture as it boiled, mock-grumbling about the heat and humidity. “Why do we do this again?” I asked. There wasn’t a heat wave, thank goodness, but the canning process had already raised the temperature inside the kitchen by three degrees Celsius. To save electricity, we had turned airconditioning off while I worked in the kitchen. So it was just me, a fan, and three stove burners going at once: the jelly mix, the canning water bath, and a small pot to sterilize the lids.

I lifted the Mason jars out of the large canning pot, where they had been simmering for the last fifteen minutes. The routine was starting to come back to me: place the canning funnel; ladle the jelly into the jar; leave some headroom, scooping jelly out if needed; pop the bubbles; wipe the rim and the jar threads; use the magnet to pick up and place a lid; tighten the ring, then loosen it just a little bit.

It was still hard to imagine that I’d once prepared 95 bottles of jam/jelly/syrup during last year’s epic Canada Day weekend jam-making session – tokens for our wedding guests and gifts for other occasions. We’d somehow managed to give away or finish all of our blueberry jam. I’ve been waiting for blueberries to go on sale, but we haven’t seen prices like last year’s – maybe the yield hasn’t been as good this year. But jalapeño peppers were back to their sale price, so we scooped up enough for a double recipe.

As I bottled more jelly, the motions began to feel more familiar. I was happy to see the recipe was just the right volume for the seven bottles I’d sterilized. I lowered the jalapeño jelly into the large canning pot filled with hot water, set the timer, and turned up the heat.

After fifteen minutes, I used the canning tongs to pull out a jar. As the air inside the first jar cooled and contracted, the lid was sucked in by the vacuum. It popped reassuringly, resulting in a slight depression in the center of the lid. Each jar popped within a few seconds of removal from the canner. With each pop, the heat seemed to recede a bit further, replaced with memories of jalapeño spread on cream cheese and crackers. I moved the jars to the kitchen table and started working on the second batch of seven jars. It went more smoothly. If we had more produce – and a long weekend ahead of us – I could see how I might’ve kept on going.

For us, canning isn’t about the $1/lb difference between on-sale jalapeños and regular jalapeños (or other produce), the ability to enjoy the taste of summer while snow buries the earth, or even traditional recipes handed down through generations. The first time I ever canned something was during our August 2009 staycation, using recipes from the Internet, and I don’t actually tend to reach for jam or jelly when I have toast.

I sweat through the canning process for other reasons: enjoying tastes that are difficult to find at the local supermarket (such as apricot syrup and jalapeño jelly), and making gifts instead of giving things that people might be able to buy elsewhere. At the end of the day, I think it’s worth it.

Jalapeño jelly adapted from What’s Cooking America; we had an extra pepper, so I added a little more liquid pectin. Processed in hot water for 15 minutes, based on our old recipe.

Fills 7 – 8 500ml jars, depending on the sizes of your peppers

Remove the stems from the peppers. Puree the whole jalapeño peppers in a blender. Add the peppers without the seeds, and chop them in the blender. Add white vinegar and the bell peppers.

Transfer the pepper mixture into a large non-reactive pot over medium-high heat. Get it to a hard boil (lots of small bubbles), then continue boiling it for 10 minutes, stirring occasionally. Now is also a good time to start sterilizing your jars and lids in hot water, if you haven’t done so already.

Remove the pepper mixture from heat. Add the sugar and the lemon juice. Stir thoroughly. Put the mixture back on medium-high heat, and return to a hard boil. When it’s boiling again, add liquid pectin and green food colouring (I used 10 drops), return to a boil, and boil for a full minute. (If you omit food colouring, the jelly may turn brown.)

The original recipe calls for straining the jelly, but we skipped that step because we don’t mind texture.

Pour into sterilized jars, then process in hot water for 15 minutes.

In retrospect, I think I added too much pectin, because I didn’t do the ounce-to-milliliter check with the original recipe. We’ll just have to see if it’s too solid or still somewhat spreadable. Meep! Worst-case scenario, we’ll treat this as our stash and make another batch for gifts. Oh my, whatever shall we do… Winking smile

Drupal: Overriding Drupal autocompletion to pass more parameters

August 8, 2011 - Categories: drupal, geek
Update 2014-02-19: See LittleDynamo’s comment with a link to this StackOverflow answer.
Update 2014-02-11: See Claus’ comment below for a better way to do this.

Drupal autocompletion is easy – just add #autocomplete_path to a Form API element, set it to something that returns a JSON hash, and off you go.

What if you want to pass form values into your autocompletion function so that you can filter results?

Searching, I found some pages that suggested changing the value in the hidden autocomplete field so that it would go to a different URL. However, that probably doesn’t handle the autocomplete cache. Here’s another way to do it:

Drupal.ACDB.prototype.customSearch = function (searchString) {
    searchString = searchString + "/" + $("#otherfield").val();
    return this.search(searchString);
};

Drupal.jsAC.prototype.populatePopup = function () {
  // Show popup
  if (this.popup) {
    $(this.popup).remove();
  }
  this.selected = false;
  this.popup = document.createElement('div');
  this.popup.id = 'autocomplete';
  this.popup.owner = this;
  $(this.popup).css({
    marginTop: this.input.offsetHeight +'px',
    width: (this.input.offsetWidth - 4) +'px',
    display: 'none'
  });
  $(this.input).before(this.popup);

  // Do search
  this.db.owner = this;
  if (this.input.id == 'edit-your-search-field') {
    this.db.customSearch(this.input.value);
  } else {
    this.db.search(this.input.value);
  }
}

Drupal.behaviors.rebindAutocomplete = function(context) {
    // Unbind the behaviors to prevent multiple search handlers
    $("#edit-your-search-field").unbind('keydown').unbind('keyup').unbind('blur').removeClass('autocomplete-processed');
    // Rebind autocompletion with the new code
    Drupal.behaviors.autocomplete(context);
}

You’ll need to use drupal_add_js to add misc/autocomplete.js before you add the Javascript file for your form.

Hope this helps!

2011-08-08 Mon 19:16

Decision: Piano lessons?

August 9, 2011 - Categories: analysis, decision

My last piano lesson is on Thursday, so I’m thinking about whether or not to sign up for more lessons, take lessons elsewhere, or do something else.

Benefits from the past three lessons:

Costs:

Considerations:

Options:

A. Finish this set of lessons, then switch to learning on my own. I can find Youtube videos of the pieces in my piano book, which will help me with timing. I may also try different pieces. I might continue my experiment with regular practice times and see how far that gets me.

B. Give the lessons another month. Possibly talk to the teacher to see if I can refocus the lesson on the parts that motivate me more.

C. Shop around – try different teachers. Can do this, but I’m less inclined to do so.

D. Dial down piano and dial up a different interest. Writing. Drawing. Latin. Volunteering. There’s a lot I can do with time and a little money set aside for learning.

I’m leaning towards D with a touch of A, maybe practising every other day and working on other interests the rest of the time. Blogging it here to remember my reasons for decisions. =)

2011-08-09 Tue 20:25

Lessons learned from project M

August 10, 2011 - Categories: drupal, ibm, kaizen, review, work

I’m wrapping up a Drupal 6 project which was funded by one of IBM’s corporate citizenship grants. The Snake Hill folks we’ve been working with will continue working with the client until they’re ready to launch. For my part, I’ve been in user acceptance testing and development mode for almost a month, rolling out new features, fixing bugs, and getting feedback.

The project manager has shuffled some hours around and made sure that I’ve got some “support” hours for follow-up questions after we turn the project over.

What worked well

Hey, I can do this stuff after all! I gathered requirements, estimated the effort, negotiated the scope, communicated with the clients and other team members, and generally did other technical-lead-ish stuff. I’ve done that on other projects, but usually that was just me working by myself and talking to clients. This one was more complex. It was fun figuring out what would fit, how things were prioritized, whether or not we were on track, and how to make things happen. I’d love to do it again. (And with the way the world works, I will probably get an opportunity to do so shortly!)

Understanding a project deeply: I was on the first phase of this project as well, and the experience really helped. We didn’t have any disruptions in technical leadership on our part, unlike in the first phase. See, last year, the IBM technical lead who had been talking to the client ended up leaving the country, so we had to repeat a few discussions about requirements. This time, I could draw on my experience from the first phase and our ongoing discussions about the client’s goals for the project. That was fun.

I’ll be turning the project over to the other development company, and the client’s concerned about whether they’ll be able to pick things up and run with it. I’ve tried to write down as many notes as I can, and I also invested time in briefing the other developers on the overall goals as well as the specific work items. Hope that works out!

Externally-accessible issue tracking: In the previous phase of this project, issue tracking consisted of e-mailing spreadsheets around. It was painful. One of the first things I did when we started this phase of development was to set up a Redmine issue tracker on the client’s server. After we gathered and prioritized requirements, I logged them as features in Redmine and split them up over the different phases. I reviewed our list of outstanding work and filed them as bugs, too. As feedback came in, I tracked bugs. I took advantage of Redmine-Git integration and referred to issue numbers in my commit messages. When people e-mailed me their feedback or posted messages on Basecamp, I created issues and added hyperlinks.

Having an externally-accessible issue tracker helped me worry less about missing critical bugs. I also shared some reporting links with the clients and the project manager so that they could track progress and review the priorities.

On future projects, I would love to get to the point of having clients and testers create issues themselves. Wouldn’t that be nifty?

Git for version control: I’m so glad I used Git to manage and share source code between multiple developers. The other developers were fairly new to Git, but they did okay, and I figured out how to clean up after one of the developers wiped out a bit of code after some commit confusion. Git stash and git branch were really helpful when I was trying lots of experimental code.

Developing with a non-default theme: We had a lot of development items to work on while the No.Inc creative team got their Drupal theme together. Once No.Inc sent the theme, I layered it on top of the site, fixed the usual problems, and had even more fun working on a site that looked halfway done. Definitely recommend getting a reliable theme in place sooner rather than later.

Mentoring people: I helped a new developer start to get the hang of Drupal. It was a slow process (must raise estimates even more when dealing with newbies), but I hope the investment pays off. I wrote (and updated!) documentation. I identified small tasks that he could work on first. I checked on him every so often. I successfully resisted the urge to just do things myself. Slowly getting there…

Decision log: I used a wiki to keep track of the design decisions I needed to make, the alternatives I considered, and what I eventually chose. That was helpful for me. I hope it will help future developers, too.

Linux VM on a Microsoft Windows host, XMing, and Plink: I’ve tried lots of different configurations in the course of this project. Doing my development inside a virtual machine has saved me so much time in terms of restoring from backup or being able to tweak my operating environment. I started with a Linux VM on a Windows host, using Samba to access my files and either Eclipse or Emacs to edit them. That was a bit slow. Then I shifted to a Linux VM on a Linux host, SSHing to the VM and using Emacs from the VM itself. That was great for being able to do Linux-y stuff transparently. But then I found myself wanting to be back on Microsoft Windows so that I could use Autodesk Sketchbook Pro (Inkscape and MyPaint aren’t quite as awesome). I ran XMing to create an X server in my Windows environment, used plink to connect, and then started a graphical Emacs running on my virtual machine. Tada! I could probably make this even better by upgrading to 64-bit Microsoft Windows, adding more RAM, and upgrading to a bigger hard disk. (Alternatively, I could host the VM somewhere else instead of on my laptop…)

What I’m going to work on improving next time

Better browser testing, including cross-browser: I’m getting slightly better at testing the actual site, motivating myself with (a) interest in seeing my new code actually work, (b) the remembered embarrassment of gaping bugs, and (c) the idea of slowing down and getting things right. Juggling multiple browsers still doesn’t make me happy, but maybe I can turn it into a game with myself. Selenium might be useful here as well.

Continuous integration: I set up Jenkins for continuous integration testing, but it fell by the wayside as I wasn’t keeping my tests up to date and I wanted more CPU/memory for development. I ran into a number of embarrassing bugs along the way, though, so it might be worth developing stricter discipline around this. I’m still envious of one of the Drupal projects I heard about in IBM, which got through UAT without identified defects thanks to lots of manual and automated testing. If I add more power to my development machine or offload testing to another machine, that might be a good way to stick to this process.

Closer communication with clients and external developers: We set up short daily meetings for the developers, but sometimes people still felt a little lost or out of touch. On future projects, I’ll make sure the clients have it on their calendar as an optional meeting, and maybe see about getting e-mail from people who can’t join on the phone. If I’m the tech lead on a future project, I’ll sit in on all client status update meetings, too. We found out about some miscommunications only when I handled one of the status calls. Fortunately, it was early enough that we could squeeze in the critical functionality while reprioritizing the others. Tense moment, though!

Better vacation planning: I realized we had a 4-day weekend the week before we had it, and we forgot about some people’s vacations too. Heh. I should get better at looking at the entire project span and listing the gaps up front.

Earlier pipeline-building: I nudged some project opportunities about a month before our projected end date, but that wasn’t long enough to deal with the paperwork lag. Oh well! Next time, I’ll set aside some time each week to do that kind of future pipeline-building, and I’ll set myself reminders for two months and a month before the project ends. Not a big problem.

My manager’s been lining up other Drupal and Rails projects for me to work on. Looking forward to learning all sorts of lessons on those as well!

Other Drupal lessons learned:

2011-08-10 Wed 17:08

Thinking about blogging and planning ahead

August 11, 2011 - Categories: blogging, planning, writing

Most of my blog posts look back – lessons learned, moments experienced. Some of them are written about the present: making decisions, figuring out knotty problems. A few of my posts look towards the future: goals, things I’d like to learn.

Here’s a rough categorization of the blog posts from the past two months:

June 2011 July 2011
Past 33 20
Present 1 7
Future 5 5

Most posts were written in response to things that happened that week, often even that day.

A blog doesn’t always have to look back. I’d like to get better at writing ahead: picking topics I want to learn about and writing about them. This is an odd sort of thing. I’m looking ahead to the things I want to reflect back on. They’ll still count as “past” blog posts, except I’m planning them for the future.

It’ll be like an editorial calendar – a list of ideas to work off, and thoughts about what kinds of information to seek out or save, and a way to plan upcoming posts. Something that gets me to write more post series like the one I did on the value of blogging.

Thinking about what chunks of knowledge I’d like to learn and share will help me write more deliberately instead of writing about whatever crosses my mind, although casual writing like that can also be helpful in patching together memories. I like the idea of a lifeline of books, gradually adding to an outline of knowledge I’d like to pass on. An Org-mode outline, even, so I can use it to organize my snippets.

This kind of list will help me separate brainstorming from writing. If I can keep a list of ideas that inspire me, then I can write even during the blah moments. It’s like the way that a brainstormed list of ideas for single-panel comics has helped me put together three for IBM’s intranet.

It will also help me plan research or note-taking. If I want to write about discretionary time, it’ll be good to track how I spend my time and research how other people spend theirs. If I want to learn more about happiness, I can supplement personal experience with research.

Planning ahead will help me recognize things I should stash in my archives for later use. If I want to write about my experiences doing buy-and-hold index investing over 10 years, I can think about what I’d like to include in that 2017 post. I’d probably want to mention my first investment here (1.116 shares of the TD Canadian Index e-fund, each share priced at $22.4100 on December 4, 2007) for extra flavour. I may also want to write about any hiccups along the way, like how I’m dealing with this current financial slump. (Portfolio underwater? Buying more.) Many writers have a snippets file, also known as a morgue. This blog and the Org text files I keep for unfinished or private notes will probably be quite a useful resource over the years, if I can make sure the data doesn’t get trashed.

I’m curious about time, decisions, writing, learning, and life. We’ll see how this works. =)

2011-08-11 Thu 07:29

Living an awesome life as a 27-year-old: a year in blog posts

August 12, 2011 - Categories: blogging, review, yearly

Happy birthday to me! =) I’m celebrating my 28th birthday today. Here’s the year in blog posts (August 2010 – July 2011): http://sachachua.com/notebook/files/sacha-chua-27.pdf .

Blogging as a 27-year-old: 391 published posts, 382 pages long, more than 190,000 words. If you’re curious, you can see what I was blogging as a 26-year-old and as a 25-year-old.

The short version: my birthday wrap-up. It’s been a great year, and I’m looking forward to learning and sharing even more!

Monthly chunks: August 2010, September 2010, October 2010, November 2010, December 2010, January 2011, February 2011, March 2011, April 2011, May 2011, June 2011, July 2011

My birthday wish: tell me what your life was like when you were 28 what you wish you knew when you were 28, or what I can improve on to make life as a 28-year-old totally awesome. =) Younger than I am? Tell me who you’re planning to grow into when you’re 28!

(Happy birthday to my parents, too. After all, they did all the hard work.)

Weekly review: Week ending August 12, 2011

August 13, 2011 - Categories: review, weekly

It was a wonderful week. I talked to clients about the scope of a new project and wrapped up a project with another set of clients. And it was my 28th birthday on Friday, too! Whee. =)

From last week’s plans

  • Work
    • [X] Wrap up project M
    • [X] Set up meeting for project T
    • [X] Follow up with project O
    • [X] Figure out what next week will look like
  • Relationships
    • [X] Volunteer to lead a build class at Free Geek
    • [X] Hang out with W-’s family
    • Got build process notes sorted out, followed up with Free Geek Toronto
  • Life
    • [-] Sketch yearly update
    • [X] Practise piano some more
    • [X] Decide whether to continue piano lessons: I’ll study on my own for a bit
    • Ordered movie ticket coupons

Plans for next week

  • Work
    • [ ] Tidy up project M some more
    • [ ] Shepherd paperwork for project T
    • [ ] Work on my project assessments
    • [ ] Learn about Maqetta
  • Life
    • [ ] Sketch year in review
    • [ ] Write, write, write

Time analysis

Activity This week Last week Delta Notes
D – Break 0.7 6.5 -5.8
D – Drawing 4.2 4.8 -0.6
D – Gardening 1.2 -1.2
D – Latin 1.4 4.3 -2.9 Less travel time, so less Latin review time
D – Piano 6.6 4.5 2.1 At least half an hour each day
D – Reading 0.7 1.6 -0.9
D – Shopping 11.0 -11.0 Moved time to socializing and volunteering instead
D – Social 11.5 6.5 5.0
D – Volunteering 6.3 6.3
D – Writing 8.1 6.5 1.6
R – Cooking 6.4 8.8 -2.4
R – Eating 4.9 7.0 -2.1
R – Exercise 5.9 2.6 3.3 Biked to work
R – Routines 7.7 8.8 -1.1
R – Sleep 60.9 60.8 0.1
R – Tidying 2.5 8.6 -6.1
Work 40.0 24.2 15.8 Back to regular work

I met some clients for the first time, and they mentioned that they read my blog. One client told the other that it’s a wonder I got any sleep. Thanks to my time-tracking, I could tell her that I actually do manage to get eight-and-a-half to nine hours of sleep everyday, but that I don’t watch television. =)

Back to regular programming. Last week’s average of about 4 hours of discretionary time per workday held steady. I had about 9 hours of discretionary time each weekend day, which was up from last week. We spent more time cleaning last week, because we were reorganizing. This week felt like more of a typical week.

Life is great.

Reflections on creativity

August 14, 2011 - Categories: life

Creativity is about making something new. So why is it that when we talk about creativity, we usually think about artists and kindergarteners instead of our everyday lives? Why is it that when we’re asked for examples of how we’re creative, we reach for doodling and photography instead of spreadsheets or code?

The truth is that we create. Constantly. Every moment we make something new that has never existed before, and that would not exist without us. We can’t help but create. Even when we’re vegetating on the couch, we’re creating our lives. The “creativity” we recognize, I think, is a combination of the consciousness with which we create and the ability to create something unexpected.

I’m tired of this stereotyping of accountants, lawyers, and other “left-brain” types as uncreative. There are limits to how creative you want your accountant to be, of course – but creativity loves constraints, and even Picasso worked within the physical limitations of paint. Within the constraints of what’s legal, an accountant can help you find unexpected savings and opportunities to optimize your business. I think of my web development as intensely creative. On one level, I write code; on another, I build new ways for people to connect and work together. (This is also why I’m careful about work, life, and happiness: burnout is a common danger.)

Stop drawing lines between “left brain stuff” and “right brain stuff.” If you let yourself separate those two, you can make up all sorts of excuses. “My job isn’t really that creative.” “I can’t draw.” “Oh, this is just a hobby.” Creativity isn’t just about arts and crafts. It isn’t something that’s reserved for the gifted. It’s for everything and everyone.

Profoundly “left-brain” stuff can be profoundly “right-brain” as well. Take mathematics. Whether you’re looking at grade school kids learning how to do multiplication for the first time, the teachers teaching them, or academics chasing down elusive proofs, there’s a lot of room for creativity there.

Profoundly “right-brain” stuff can be profoundly “left-brain” as well. Leonardo da Vinci’s art was informed by his knowledge of anatomy and his attention to detail. Fugues and minuets were composed with mathematics in mind.

You don’t even have to have a white-collar job to find opportunities for creativity in your work. You just have to make things happen. For example, no training manual could’ve anticipated what this guy turned his job into:

A janitor sweeping the floor at a nursing home might see her work as creating a neater and more comfortable environment for people. A greeter might take the opportunity to experiment with conversation. How about you?

Don’t leave art to the artists. Create your life and create a better world.

We constantly create. The trick is directing our creativity towards the kind of things we want to build. The playground bully is creative, too.

If you don’t think of yourself as creative, try thinking of a time you thought about doing something, making something, becoming someone, and you went ahead and did it. Imagination becomes life.

Creating can be frustrating. We judge ourselves harshly. We feel our limitations. I can draw, but I can’t draw well enough to get the ideas out of my head. I can write, but there’s still something missing. I can code, but I occasionally write embarrassing bugs. Then there are external challenges as well: the limitations of the world we work with, other people’s conflicting ideas and desires.

Be kind to yourself. Remember that you’ve got to start from somewhere, and here is as good a place as any to start.

Obstacles and constraints help you become stronger at creating. If everything came easy to you – if you could wave a wand to make your imagination reality – how would you grow? Challenges provoke you to figure things out.

We tend to stereotype creativity as arts and crafts, and ignore the beauty in things like mathematical proofs, code, and cabinets.

Yes, cabinets. The IKEA cabinet in our kitchen, for example. Your ordinary, run-of-the-mill, mass-produced sort of furniture. There are probably hundreds of thousands like it in the world. But there is so much creativity wrapped up in this simple thing. Design, of course – how to create something simple, utilitiarian, and affordable. Something that can be flat-packed, transported, and reassembled – and beyond that, the idea of flat-packing, the business innovations that led to this being here. Take another step back. Think about the logistics involved in obtaining wood and metal and shaping them into this cabinet. The invention of screws and brackets and drywall fasteners. A step closer: our own creativity in installing the heavy cabinet, balancing it on top of volumes of Childcraft on top of a set of shelves. A step back: the automated production of this in some remote factory’s assembly line; the robots that do the work, the people who invented them and the people who manage them today, the orchestration of all these processes on the factory floor. Further back: the invention of mass-production. Forward: Our cats’ discovery that the top of the cabinet makes an excellent perch. I could play all day among the unfolding levels of complexity of a cabinet.

Not that cabinets are the most special things in the world – I chose a cabinet simply because the word sprung to mind. If you want to, you can reflect on a single grain of salt (the crystalline structure, the discovery, the richness of human history, the influence on words like “salary”, the logistics that brought that salt to you, the way it makes recipes sparkle, the science around it).

If you start looking, you’ll find a world of wonders around you.

—-

Thanks to #kaizenblog for the nudge!

Volunteer notes: Free Geek Toronto

August 15, 2011 - Categories: geek

From Saturday: We’ve just come back from volunteering at Free Geek Toronto, where we’ve been helping people refurbish donated PCs. It’s a good idea: help people develop computer skills while reducing waste. Unusable components are separated and bundled for recycling.

Working with computer components as old as the ones they get there can be challenging. People need to identify and isolate problems, replace parts that don’t work, and set up Ubuntu Linux.

W- and I started volunteering there after J- did a short stint with them during her school’s community volunteering week. Free Geek Toronto needed a lot of help, so we wanted to see what we could do to get them to the next level. They hadn’t had build classes in a while, so we volunteered to lead Saturday sessions. We’ve also been working with other volunteers to document the process and make it easier for other people to pick up. My goals for this part are to write things down, go through a couple of iterations of build classes, and encourage at least one other person to lead build classes, so that we can then free up the time to work on other things.

The build sessions have been going well. It helps that we have a mix of experience levels. Today, one of the volunteers worked on a computer with chassis intrusion detection. A small switch detected when the cover was off and set a flag in CMOS, which caused the computer to halt while booting. He and another volunteer figured out how to reset the alarm and how to disable it. I’m learning a lot by osmosis, too, and will probably be writing more about hardware on this blog.

Future build classes might have more people who have little experience with computers. We need to make it easier for them to get started. Here are some things we and other people can work on:

The organization also needs to sort out waitlists and communication, but that involves interpersonal stuff and we’re not deep enough in the organization yet to be able to understand or nudge the dynamics. Sometime!

What could wild success look like?

Build classes run regularly. They have clear goals: each student will successfully assemble a computer, practising troubleshooting skills along the way. They’ve got a checklist, a short summary guide, and detailed step-by-step instructions, as well as a troubleshooting guide. There are a lot of parts, but some have been pre-tested. The class also helps students get to know other experienced volunteers, so that people feel comfortable coming and volunteering on their own. After the classes, the students feel confident about building computers with the help of other experienced people, and they come regularly. The build class facilitators sign up in three-week chunks; they do it for fun and for good. Several people volunteer to run these ongoing build sessions, and other experienced volunteers hang out to work on their own interests.

Maybe we somehow track the progress of a box, so we can contact volunteers when a box they built has been sold or donated, and we tell them the story of the difference they helped make. Maybe this encourages them to come back and build another one. Maybe there’s some kind of build volunteer tracking, so we can tell when a build box hasn’t been worked on for a while, and invite people back or release it for the next build class.

Might be interesting. =)

You know what I’m curious about? Well, on one level, there’s getting better at working with computers. And then there’s the really fascinating level of tweaking an organization and its processes. We’ll see. =) I’m sure I’ll learn tons along the way!

Discovering Yourself through Blogging (free teleconference Aug 16, 6 PM PDT, 9 PM EDT, 9 AM Philippine time)

August 16, 2011 - Categories: blogging, life, sketches, writing

image

I used to hate writing. I thought it was just about term papers and book reports. It took blogging for me to discover that I could write for myself, not just for teachers, and that writing – my technical notes, my decisions, my “here’s how I’m trying to figure out life” confusions – could save me a lot of time. So now I want to help other people get over that hump, get out of that rut, get going. Get writing.

Holly Tse will be interviewing me at 9 PM EDT today (Aug 16) on how blogging can help you with your process of self-discovery. I don’t have sound-bites or snazzy “here, buy this e-book” self-promotions, I just want to help people figure out how to use blogging to learn more about life. It’s not just about personal branding, search engine optimization, or being cool. It’s useful for getting the hang of things, taking notes, and sharing them with others.

Want to pick up ideas or ask questions? You can listen for free on the web or on the phone – sign up at http://lotusblossoming.com/sacha and the access details will be e-mailed to you. If you use the web interface, you can ask your questions using the Q&A interface right there. Please post questions as soon as you think of them. I’ll work on following up with a more detailed as we figure things out together. Can’t make it? I’ll post notes on my blog (of course!). You can subscribe to make sure you get it, or if you leave a comment here, I can send it to you too. =)

You might also be interested in other things I’ve written about blogging, including this seven-part series packed with thoughts and tips on the value of blogging. Hope that helps!

Session follow-up #1: Discovering Yourself through Blogging

August 17, 2011 - Categories: blogging, presentation, speaking
This entry is part 16 of 16 in the series Discovering Yourself Through Blogging

I enjoyed chatting with Holly Tse about blogging and how it can help you learn more about life, connect with people, save time, and do awesome. For the next day or so (Aug 17), you can listen to a free recording of my interview with Holly at http://instantteleseminar.com/?eventid=21913131 . I’m working on putting together a transcript and some follow-up notes, but here are some quick thoughts.

Blogging doesn’t have to be about building a personal brand or improving your search engine ranking. You can write as a way to learn, understand, remember, share, and save time.

Trying to figure out how to write about something possibly sensitive or offensive? Take a step back and try to take a really, really positive approach. Don’t focus on past hurts, focus on how to move forward. Don’t focus on what other people are doing wrong, focus on what you can do and what you can change about yourself. Write through things in your private notes if you need to, then see what insights and ideas you can share with others.

Where can you find the time to write? Holly Tse mentioned spending most of her time focused on her husband and their toddler, organizing this telesummit, and taking care of other essentials. I mentioned that mommy blogging (and parent blogging in general – let’s not forget the blogs!) was popular for lots of reasons: grown-up connections, memories, ideas, sanity checks, and so on. I also shared some time-saving tips, like cooking in larger batches. =)

You might be boring. In fact, you almost certainly will bore yourself from time to time. Writing will feel awkward if you haven’t been doing it a lot, and even if you have, it can still be frustrating. Keep writing. Don’t worry about being interesting. Don’t worry if no one reads your notes. Write in order to think clearly, write in order to remember, and write in order to share. You can grow into a good writer, but only if you write. You don’t need to win the Pulitzer Prize to write notes that can help you and other people.

How frequently should you write? As frequently as you can or would like to. =) Don’t beat yourself up if you can’t post every day or you blog sporadically. That said, try using writing as a tool for thinking. Try asking yourself questions like: What do I want to remember? What did I learn today? What do I want to do better tomorrow? What do I want to work on learning? If you do that, you’ll probably find that there’s a ton of stuff worth writing about.

More thoughts to follow. Feel free to ask more questions! Leave a comment so that other people can also share their thoughts with you, or use the contact form to get in touch with me. Have fun!

Series Navigation« Transcript: Blogging (Part 15): Tools to help you get started

Thinking my way through a tough MS SQL Server 2000 replication problem

August 18, 2011 - Categories: geek, ibm, work

Thinking through a particularly tough replication problem at work. You see, I’ve been working on a dashboard for a small call center. Since they sometimes have really short calls, they’d like the display to be as close to real-time as possible. Cisco handles the calls and stores the information in either of two Microsoft SQL Server 2000 servers depending on the failover situation, which use merge replication to stay in sync. We have a custom script that takes the call information and adds it to our own database. Then the PHP dashboard I wrote uses JQuery to poll the server every 5 seconds and display updates.

Since our database wasn’t part of the standard configuration, I also set up SQL Server replication for it. I have little experience with Microsoft SQL Server and I found it even more stressful trying to look up steps while people were watching (ah, web conferences), so I heaved a huge sigh of relief when I saw a row replicate properly.

I originally programmed the dashboards to check only for any updates since it last checked. It turns out that I need to account for late synchronization, too. It seems to take about ten seconds, maybe more, for the information to replicate from one server. This meant that if the dashboard on the second computer had already checked for updates, it would miss the older updates that had just newly replicated, and dashboards polling the two servers would eventually get out of sync. Both servers needed to support read/write operations. The systems don’t have a static master-slave relationship. Either one of the two servers could become the primary server – the first server to receive updates – transparently, depending on the failovers.

I’m modifying the script to check for older updates. It’s a trade-off. A larger window means more transfer time, higher server load, possibly more delays. A smaller window could miss updates. If I set the dashboard to check the last 45 seconds, I get a delay of about 25 seconds from the time that a call shows up on a dashboard looking at the primary server and the time the call shows up on a dashboard looking at the secondary server. I’m reasonably happy with the code.

I also need to change the SQL Server configurations, so I’m waiting for a maintenance window and the go-ahead to do so. Anything I can learn or clarify before then will help me make the most of the limited time (and reduce stress, which is also important). Here are some things I need to properly figure out:

Working with Microsoft SQL Server 2000, I wish I had lot of these things I’m reading about, like MS SQL Server 2005′s Database Mirroring and MS SQL Server 2008′s Configure Peer-to-Peer Topology Wizard. There are these instructions for setting up nonpartitioned, bidirectional, transactional replication, but I don’t know if they’ll really get me to where I want to go, and they look complicated. The stored procedures look familiar, though, so maybe that’s how I got replication working even though the config doesn’t reflect it.

Theoretically, I should be able to just reach out into this really big company of ours and find a Microsoft SQL Server expert who can set this up properly in all of five minutes, but it’s complicated. Something’s changed, or maybe I have. I’m letting the focus on utilization get to me. I hate having to say no to other people because I’m supposed to focus on billable work, and I know it’s a hassle and a half to set up project codes for spot consulting, and I don’t feel comfortable putting other people in that position. Anyway, I’ve posted a question in a community with 500 members, and I’ll try reaching out to one of the community SMEs tomorrow.

You know how it’s a good learning experience to work outside your comfort zone? This is really far out of my comfort zone. I tried asking for a proper DBA. I didn’t get one. No amount of searching and speed-reading is going to turn me into an experienced one, at least for this project. I really hate this feeling – wanting to do right by the client, but feeling the limitations of my experience, and worrying about messing things up even more.

I’m going to see if another attempt to set up proper continuous push replication will do the trick. I’m also going to try e-mailing people to ask for their help, and see if our resource deployment managers know of anyone in between engagements who might be able to help. It’s a long shot, but if I take enough long shots, it might work.

You know, this makes dealing with cross-browser issues look like fun. But it needs to be done, and there will be rare moments like this anyway, so I might as well learn how to even more effectively ask for help in the company.

2011-08-17 Wed 20:13

Git bisect and reversing a mistaken patch

August 19, 2011 - Categories: development, geek

2011/12/09: Updated links

Using version control software such as git is like slicing the bread of programming. It lets you deal with changes in small chunks instead of having to troubleshoot everything at the same time. This comes in really handy when you’re trying to isolate a problem. If you can tell which change broke your system, then you can review just those changes and look for ways to fix them.

For example, some views I’d created in Drupal 6 had mysteriously vanished from the web interface. Fortunately, I’d exported them to code using Features, so I knew I could get them back. I needed to find out which change removed them so that I could make sure I didn’t accidentally undo the other relevant changes.

git bisect to the rescue! The idea behind git bisect is the same one behind the marvelous efficiencies of binary search: test something in the middle of what you’re looking at. If it’s good, take the later half and test the middle. If it’s bad, take the earlier half and test the middle. It’s like what people do when guessing a number between 1 and 100. It makes sense to start at 50 and ask: is the number greater than 50? If it is, ask: is the number greater than 75? And so on. Handy trick, except sometimes it can be difficult to add or subtract in your head and figure out the next number you should ask.

git bisect does that adding-up for you. You start with git bisect start in the root of your source tree. You tell it if the current version is considered broken, using git bisect bad. You tell it the last known working version, with git bisect good changeset-identifier. Then it takes the middle of that range. Test it to see whether it works, and type in git bisect good or git bisect bad depending on what you get. It’ll present you with another changeset, and another, until it can identify the first changeset that fails. If you can automate the test, you can even use the git bisect run command to quickly identify the problem.

Now that you’ve identified the relevant changeset, you can use git show changeset-identifier to look at the changes. If you save it to a file, you can edit the diff and then use the patch command to reverse and apply the diff. Alternatively, you can undo or tweak your changes by hand.

The git bisect section in the free Git SCM book has more information, as does the manual page. Hope this helps!

Kids’ cartoons and learning piano

August 20, 2011 - Categories: learning, life

One of the reasons I signed up for piano lessons was because I’d flipped through the Alfred Adult All-in-One Piano Course Level 2 book and found myself skipping pasts songs I didn’t recognize. Light and Blue? Theme from Solace? La Raspa? Mexican Hat Dance?

I looked up Youtube videos of people playing. So, this mysterious La Raspa?

And the Mexican Hat Dance?

Is it just me, or were those in Looney Tunes or something similar?

I may have wasted hours of my life watching cartoons as a kid, but if those memories now encourage me to get through all these piano pieces, maybe they weren’t such a waste after all. =)

Then there’s the link between Looney Tunes and opera… I wonder what surprising connections these childhood pastimes will make?

Weekly review: Week ending August 19, 2011

August 21, 2011 - Categories: review, weekly

From last week’s plans

  • Work
    • [X] Tidy up project M some more
    • [X] Shepherd paperwork for project T
    • [-] Work on my project assessments
    • [-] Learn about Maqetta
    • Drew more IBM comics and sent them in
  • Life
    • [-] Sketch year in review
    • [X] Write, write, write

Plans for next week

  • Work
    • [ ] Get started on project T
    • [ ] Learn about project O
    • [ ] Follow up on project I
    • [ ] Draw more IBM comics
    • [ ] Learn about iRise
  • Relationships
    • [ ] Electronics hacking, yay!
    • [ ] Wrap up build class for Free Geek Toronto
    • [ ] Update build process and checklist
    • [ ] Cook a new recipe (Vietnamese bun with nuoc cham?)
    • [ ] Catch up on mail
    • [ ] Transcribe interview
  • Life
    • [ ] Sleep at some point

Time analysis

Activity This week Last week Delta
D – Break 2.4 0.7 1.7
D – Drawing 4.2 4.2 0.0
D – Gardening 0.2 0.2
D – Latin 0.5 1.4 -0.9
D – Learning 1.1 1.1
D – Other 4.9 4.9
D – Piano 2.6 6.6 -4.0
D – Reading 3.4 0.7 2.7
D – Shopping 1.1 1.1
D – Social 11.2 11.5 -0.3
D – Volunteering 8.0 6.3 1.7
D – Writing 6.6 8.1 -1.5
R – Cooking 4.7 6.4 -1.7
R – Eating 6.1 4.9 1.2
R – Exercise 2.5 5.9 -3.4
R – Routines 7.9 7.7 0.2
R – Sleep 53.9 60.9 -7.0
R – Tidying 5.0 2.5 2.5
R – Travel 0.8 0.8
Work 40.8 40.0 0.8

I used 46.4 hours of discretionary time (up from 39.5 last week, probably freed up by less sleep) this week, an average of 5.1 hours each weekday and 10.4 hours on weekends. Sleep dropped to an average of 7.7 hours a day, but I still felt happy and energetic. I think it’s partly the shift to the new curtains, which block out more light, and partly the acceptance of a later schedule. Like work, sleep expands to fit available time. Sleeping later (but not too late) might be okay, although I’ll want to check for delayed effects over the next few weeks.

I decided to shift piano and Latin time to other pursuits, carving out time to learn more about electronics and work on projects at home. =) Whee! Looking forward to blogging about those.

Thinking about outsourcing transcription or doing it myself

August 22, 2011 - Categories: analysis, decision, kaizen, speaking

I like reading much more than I like listening to someone talk, and much, much more than listening to myself talk. Text can be quickly read and shared. Audio isn’t very searchable. Besides, I still need to work on breathing between sentences and avoiding the temptation to let a sentence run on and on because another cool idea has occurred to me. Perhaps that’s what I’d focus on next, if I ever resume Toastmasters; my prepared speeches can be nice and tight, but my ad-libbed ones wander. More pausing needed.

So. Transcription. I could do it myself. I type quickly. Unfortunately, I speak quite a bit faster than I type, so I usually need to slow it down to 50% and rewind occasionally. ExpressScribe keyboard shortcuts are handy. I’ve remapped rewind to Ctrl-H so that I don’t need to take my fingers off the home row. But there’s still the there’s the argh factor of listening to myself. This is useful for reminding me to breathe, yes, but it only takes five minutes for me to get that point. ;) The other night, it took me an hour to get through fifteen minutes, which is slower than I expected. An hour-long podcast interview should take about four hours of work, then.

I could use transcription as an excuse to train Dragon NaturallySpeaking 11, the dictation software I’d bought but for this very purpose but haven’t used as much as I thought I would. It recognizes many words, but I have a lot of training to do before I get it up to speed, and I still need to edit. This would be a time investment for uncertain rewards. I still need to time how long it takes me to dictate and edit a segment.

Foot pedals would be neat, particularly if I could reprogram them for other convenient shortcuts. Three-button pedals cost from $50-$130, not including shipping. In addition to using it to stop, play, and rewind recordings, I’d love to use it for scrolling webpages or pressing modifier keys. I often work with two laptops, so it’s tempting. (And then there’s the idea of learning how to build my own human interface device using the Arduino… ) – UPDATE: I’ve built one using the Arduino! I can’t wait to try it out.

In terms of trading money for time, I’ve been thinking about trying Casting Words, which is an Amazon Mechanical Turk-based business that slices up submitted files into short chunks. Freelancers work on transcribing these chunks, which are then reassembled and edited. The budget option costs USD 0.75 per audio minute, which means an hour-long interview will cost about USD 45 to transcribe. That option doesn’t have a guaranteed turnaround, though, so I could be waiting for weeks. In addition, I tend to talk quickly, so that might trigger a “Difficult Audio” surcharge of another USD 0.75 per minute, or about USD 90 per audio hour.

For better quality at a higher price, I could work with other transcription companies. For example, Transcript Divas will transcribe audio for CAD 1.39/minute, and they guarantee a 3-day turnaround (total for 1 hour: CAD 83.40). Production Transcripts charges USD 2.05/minute for phone interviews.

I could hire a contractor through oDesk or similar services. One of the benefits of hiring someone is that he or she can become familiar with my voice and way of speaking. Pricing is based on effort instead of a flat rate per audio minute, and it can vary quite a bit. One of my virtual assistants took 14 hours to transcribe three recordings that came to 162 minutes total. At $5.56 per work hour, that came to $0.48 per audio minute, or $28 per audio hour. oDesk contractors are usually okay with an as-needed basis, which is good because I’ve scaled down my talks a lot. (I enjoy writing more!)

So here are the options:

I’m going to go with dictating into Dragon NaturallySpeaking because I need to train it before I can get a sense of how good it is. It takes advantage of something I already own and am underusing. Who knows, if I can get the hang of this, I might use it to control more functionality. We’ll see!

“Hello, Monday!” comic series launched on IBM intranet home page; now I’m a comic artist!

August 22, 2011 - Categories: ibm, sketches

If you’re an IBMer, you might have noticed this rather odd-looking Top Story on our intranet homepage:

image

It takes you to this:

future-ibmer-at-the-beach

More than a hundred comments so far, and it’s only noon! Looks like lots of people appreciate a bit of humour on Mondays. =) The w3 editorial team planned this as a 3rd-Monday-of-the-month feature, so the next one will come out on September 19.

I like IBM, so these comics will be more about gently poking fun at our culture than about sharp cynicism or Dilbert-like humour. Someone’s got to be able to look at the bright side of corporate life. =)

This comic’s visual style was inspired by Exploding Dog, which I love for its use of colours and simple shapes. I drew it on a Lenovo X61 tablet using Autodesk Sketchbook Pro.

I’ve requested a whole bunch of books from the library and am making time for regular practice. Looking forward to learning more about comics while squeezing this into the gaps of my day job (consulting and Drupal/Rails web development). Glad to make people’s Mondays a little bit brighter!

Built a USB foot pedal using the Arduino Uno

August 23, 2011 - Categories: geek

(W- and I are planning to go to the hacklab.to open house today. Hope to see folks there!)

This weekend was definitely an electronics hacking weekend. Whee!

It started when I found a three-way foot pedal at Active Surplus for the grand total of CAD 7.50. It’s the kind of foot pedal musicians use to control their equipment: metal, sturdy, and with an odd-shaped connector at the end. I’d been meaning to try foot switches as a way to control my computer (handy for transcription and for using multiple computers). I wanted to get into electronics, as W- enjoyed it so much. I was curious about the Arduino. I figured that making my own foot pedal would mean lots of learning, fun, and relationship-building time. Who knows, I might even get a productivity boost out of it.

USB foot pedal, then! I used my Kindle to look up which model of Arduino could be convinced to act as a USB keyboard. The Arduino Uno was the best bet, so I picked one up at Creatron, and W- got one too. With foot pedal and Arduino in hand, it was off to the house to see what I could make from it. First: figure out the switches. I’d forgotten to write down the labels from Active Surplus and the box had no information. No problem. I snipped the ends off the connector, stripped the wires, opened up the foot pedal, and started figuring out the circuit with the help of the 7-segment LED from the lab kit.

Programming the Arduino was straightforward. I’d already played around with debouncing buttons based on the sample code. I extended my code to debounce three buttons. I looked up Arduino USB keyboard resources (http://hunt.net.nz/users/darran/), reflashed the firmware on the USB chip (atmega8u2), looked up the USB keycodes to send, and wrote the code. I set it up so that left sends PageUp, right sends PageDown, and forward sends F13.

To clean up the circuit further, W- and I went to Active Surplus for the second time that weekend. I bought some headers so that I could solder the wires to them and slot them neatly into the Arduino Uno. I ran into people I know, and I found myself directing them towards the components they were looking for. I think we might be spending too much time at Active Surplus. ;) (But it’s fun!)

During the bike ride back, I was thinking about how to make the foot pedal even more awesome. W- had suggested differentiating between short and long presses, so I worked on that. I started getting confused with the different flags and variables I was using. I redid it as a finite state machine, and that was so much easier to write. (By golly, I did get to use that after all.)

So now I have a foot pedal that cost me around CAD 36.50, plus the time I spent learning how to make it. It doesn’t require a driver. It pretends to be a normal keyboard. Because I’m using function keys that don’t conflict with anything on my keyboard, I can use AutoHotkey to translate them to whatever I want: other keys, sequences of keys, commands, even context-sensitive shortcuts. I don’t have to reflash my firmware to change my keyboard settings – I just reload my AutoHotkey script.

It’s awesome. I’ve used it to flip through an e-book while eating lunch. I’m looking forward to using it while transcribing my presentations. I might remap it to other functions while drawing or programming.

One of the limitations is that the long presses can trigger key repeats. This is handy if you’re mapping it to something like down-arrow, but not so handy if you’re trying to use it as a keyboard shortcut prefix. I’d like to figure out how to control key repeat and key delay on a per-keyboard basis. If I can’t, I might either figure out how to selectively debounce the keys in AutoHotkey, or have a toggle that controls whether keys repeat.

Not at all bad for a weekend hack and my first electronic creation! I’ll ask the Powers that Be at IBM for permission to release schematics and code. (Ah, paperwork.) The circuit is basic (umm, switches), and I’m sure people will point out lots of ways it can be improved. In the meantime, here’s a picture:

C360_2011-08-21 22-16-23

Left: Foot pedal (it can rock forward or to either side), middle: really, really long USB printer cable (will replace with shorter one soon), right: Arduino Uno with the cable wired in and the program loaded.

I’m looking forward to catproofing it in my new project box. Not that anything can really be catproof in this house, but at least I can remove the temptation of wire strands and LEDs.

How to set up more frequent merge replications in SQL Server 2000

August 24, 2011 - Categories: geek, work

The quick answer: Set up continuous merge replication, then set its polling interval to the number of seconds you’d like.

The slightly-more-detailed answer:

  1. After you set up merge replication, find your agent in the replication monitor under “Merge Agents”.
  2. Right-click on your agent and choose Agent properties.
  3. Click on the Steps tab.
  4. Find the step named Run agent. Double-click on it to edit its properties.
  5. Edit the command and add -PollingInterval number-of-seconds (ex: -PollingInterval 1). You might also want to minimize logging by adding -HistoryVerboseLevel 0 and -OutputVerboseLevel 0 to improve performance. (More performance tips)

Step by step:

Set up your tables for replication:

Use Enterprise Manager to create the tables you need on both servers, if they don’t already exist.

Configure publishers and subscribers:

  1. Right-click on Replication and choose Configure Publishing, Subscribers, and Distribution.
  2. Click on Publishers. Select your server.
  3. Click on Databases. Check the Merge checkbox for your database.
  4. Click on Subscribers. Select the other server.
  5. Click on OK.

Create the publication:

  1. Right-click on ReplicationPublications. Choose New Publication.
  2. Follow the wizard to create a publication including the tables you want to replicate.
  3. Right-click on the publication and choose Push New Subscription.
  4. Follow the wizard to create a subscription for the second server.

Change the polling interval:

  1. Click on Replication Monitor – Agents – Merge Agents to view the list of replication agents.
  2. Right-click on your agent and choose Agent properties.
  3. Click on the Steps tab.
  4. Find the step named Run agent. Double-click on it to edit its properties.
  5. Edit the command and add -PollingInterval number-of-seconds (ex: -PollingInterval 1). You might also want to minimize logging by adding -HistoryVerboseLevel 0 and -OutputVerboseLevel 0 to improve performance. (More performance tips)

The backstory:

What’s an open source web development geek doing with Microsoft SQL Server 2000? It’s a long story involving PHP, AJAX, a critical situation at work (previous developer left IBM without a good transition plan), and a contact center. I had fun building the dashboard that the client needed, but I sweated my way through setting up the SQL Server 2000 replication because I was trying to piece it together from disparate web pages, and I forgot to account for delays in replication. Result: the dashboards on the different servers got out of sync.

I found myself getting stressed out. I felt inexperienced and frustrated, and I hated leaving clients hanging. I felt overwhelmed by all the information available on the Net, but I couldn’t find exactly the kind of information I was looking for. I felt confused by the options and the version differences. So I did what I always do in difficult situations: I wrote my way through it.

Writing helped me organize my questions and thoughts. I needed to figure out what kind of replication would fit our needs. Merge replication made the most sense, because two-phase commits could be problematic in failover situations.

I knew I needed to figure out how much time it took for updates to propagate from one dashboard to another. I measured it with a stopwatch, listening for the audio alerts from my dashboard: updates took about 45 seconds, which was too long.

When I read that continuous updates were like scheduling the updates to run every minute (an expected average of 30s for updates, then), I almost gave up. It was still too long, and I didn’t know whether what I wanted was even possible with the version I had.

Fortunately, I came across a brief mention of PollingInterval, which sounded promising. After lots of sifting through search pages, I found enough to work with. I combined thoughts from a forum thread about merge agents, a tip on changing your polling interval, and transactional replication performance tuning tips.

I tested it with a different database until I was confident about the steps to take and the possible results, and I reported the progress at our checkpoint call. Once the clients gave me the go ahead, it took me ten minutes to make the changes and maybe forty minutes of checking, double-checking, timing, and cleaning up. Results: the new configuration reduced average wait times from 45 seconds down to 4.6s, with observed maximums of 6.7s and a lucky minimum of 2.0s. Decently fast, considering the Javascript dashboard checked for updates every 5 seconds. I checked the network usage, too. Latency looked stable, so the server wasn’t getting overloaded.

It felt great to solve that, and on my own, too. We’ll see if the problem stays solved, but it looks good.

Re-setting up my computer

August 25, 2011 - Categories: geek

(From Wednesday night): With dozens of tabs open in a browser, dozens of buffers open in Emacs, and a virtual machine or two running in the background, I envied W’s setup downstairs: 12 GB of memory for much awesomeness. My installation of 32-bit Windows 7 could only use 3 GB of the 4 GB I had available, and the memory limits were starting to get in my way. I could use 4 GB if I booted into Linux, but drawing meant I preferred to keep Autodesk Sketchbook Pro handy.

I backed up my files and installed 64-bit Windows 7. I had to restore my old setup in order to retrieve some information from it. Fortunately, the restore process was straightforward. Getting all the Thinkpad drivers working again wasn’t, though. I’ve got the pen working again, but there are still fiddly little bits I have to track down. Total time so far: about four hours spread over two evenings.

I want to fix up my X61 so that the important parts work again, but I don’t want to spend too much time restoring my configuration. After much deliberation, I’ve put in an order for a Lenovo X220 tablet. I’m looking forward to working with even more memory, as the X220 has a maximum capacity of 8GB. I’m also looking forward to a cooler-running laptop with longer battery life. This X61T can get quite hot!

Now that I’ve reinstalled, I’ve finally followed that practice of backing up a clean system. That will make it easier to restore the system to a clean state when I turn it over to J-. When I get the new laptop, I’ll create a system image for that one too.

So far, this fresh install of Windows 7 is doing fine. My memory usage hovers around 3GB, although I’m still seeing some paging. I’m looking forward to editing more photos, which had been frustrating because of all the application crashes. We’ll see how this works out. In a few weeks, things will get even better.

Oops. Just noticed that I hadn’t correctly copied my backup, so my clean-system backup overwrote it. Darn. Fortunately, I had a network backup from 8/22. (I did? Neat. Yay backups.) It’s a large backup, so I’ll need to copy it from my network server to my external hard drive, and then mount the virtual hard drive from there. We’ll see how it works. If I don’t manage to recover, well, I’ll just have to rewrite my foot pedal program. (See, I was just about to post it, but then I decided to reinstall first. D’oh.)

All this tweaking means I haven’t spent time on other interests like building a web app for our daily activities or planning my next Arduino project. That’s okay. It’s good to spend time preparing my tools. =)

Lessons learned (I hope):

Onward!

Mailing non-Emacs users your Org notes

August 26, 2011 - Categories: emacs, org, tips

Andras uses Emacs Org-mode to take notes during meetings, and wanted to know how to share those notes (including tables) with colleagues afterwards. Here are some tips for sharing Org notes with non-Org people.

You can copy the information as plain text. If you don’t have too much Org markup, you can copy and paste the text into your mail message. To get tables and other segments to line up nicely, make sure you format the text with a monospace font such as Courier New or Lucida Console.

You can export the information to HTML and then copy it into your message. Export the entire file with M-x org-export or export a region with M-x org-export-region-as-html. Save it to a file, open that file in your browser, then copy and paste the information. If you find yourself working with the same files often, consider using Org’s publishing support to simplify the creation of related HTML files.

You can also publish your notes on an internal or external blog. I post many of my notes on my blog (including this one!) using org2blog.el. If you publish your posts on a blog, you can send people a link, update your post with new information, and share your post with others.

Hope that helps!

Practising drawing

August 26, 2011 - Categories: sketches

I didn’t grow up drawing. Now’s as good a time as any to catch up. Here’s me working through chapters 1 – 7 of “Drawing Cartoons & Comics for Dummies” last night:

imageimage

image

All the characters are male, but that’s not me, that’s the book. I’m still working on making stuff up in my head.

Getting there. I don’t have a “style” yet, but learning how to do different kinds of styles can’t hurt.

I do know I’m not a satirist. I’m not going to take corporate culture apart, like Scott Adams does in Dilbert does. I figure there are enough people in the world

I want to learn how to draw so that I can explain things, like the way Larry Gonick did in the Cartoon Guide to Statistics (there’s a whole series!), and how Scott McCloud explains things in Understanding Comics and his other books. I want to tell stories. I want to point out little things about life that make me go “Aww…”

Let’s see where it takes us!

Code and circuit for a six-function Arduino-based USB footswitch

August 27, 2011 - Categories: geek, sketches

image

I’d been thinking about footswitches for a while, but I held off on buying one because they were expensive. Turns out that building your own is easy, even for someone with limited electronics experience. I do have the unfair advantage of having a spouse who’s an electrical engineer, but I figured out this circuit and code by myself!

The hardware for this circuit is really simple. If you’re lucky, you might find a three-way foot switch at your local audio-equipment-carrying surplus shop. If not, you could make your own, but I haven’t tried doing that yet. =)

The fun part is in the code that makes this a six-function USB keyboard. The code below maps left, center, and right short presses to F13, F14, and F15, while left, center, and right long presses send F16, F17, and F18. Here’s the code:

const int redPin = 9;
const int tanPin = 10;
const int bluePin = 11;
const int orangePin = 12;
const int debounceDelay = 150;
const int longPressThreshold = 650;

int currentState;
int lastSwitch;
long lastDebounce;
long lastPressed;
int lastSwitchDebounced;

uint8_t buf[8] = { 0 };	/* Keyboard report buffer */

#define SWITCH_NONE 0
#define SWITCH_LEFT 1
#define SWITCH_CENTER 2
#define SWITCH_RIGHT 3

#define STATE_WAITING 0
#define STATE_SHORT_PRESSED 1
#define STATE_LONG_PRESSED 2

#define KEY_F13	0x68
#define KEY_F14	0x69
#define KEY_F15	0x6A
#define KEY_F16	0x6B
#define KEY_F17	0x6C
#define KEY_F18	0x6D
#define KEY_F1D	0x6E
#define KEY_PAGEUP 0x4b
#define KEY_PAGEDOWN 0x4e

void setup() {
  pinMode(redPin, INPUT); digitalWrite(redPin, HIGH);
  pinMode(tanPin, INPUT); digitalWrite(tanPin, HIGH);
  pinMode(orangePin, INPUT); digitalWrite(orangePin, HIGH);
  pinMode(bluePin, INPUT); digitalWrite(bluePin, HIGH);
  Serial.begin(9600);
  delay(200);
  lastSwitch = 0;
  lastDebounce = millis();
  currentState = 0;
}

int getCurrentSwitch() {
  if (!digitalRead(orangePin)) { return SWITCH_LEFT; }
  if (!digitalRead(tanPin)) { return SWITCH_CENTER; }
  if (!digitalRead(redPin)) { return SWITCH_RIGHT; }
  return SWITCH_NONE;
}

void sendKey(int currentSwitch, boolean isShort, boolean keyDown) {
  buf[0] = 0;
  buf[1] = 0;
  int debug = 0;
  if (keyDown) {
    switch (currentSwitch) {
        case SWITCH_LEFT: buf[2] = isShort ? KEY_F13 : KEY_F16; break;
        case SWITCH_CENTER: buf[2] = isShort ? KEY_F14 : KEY_F17; break;
        case SWITCH_RIGHT: buf[2] = isShort ? KEY_F15 : KEY_F18; break;
    }
    if (debug) { 
      Serial.println(currentSwitch); 
      Serial.println(((int) buf[2]) - KEY_F13); 
      Serial.println("Down"); 
      Serial.println(isShort ? "Short" : "Long"); 
    } 
  } else {
    buf[2] = 0;
    if (debug) { Serial.println("Up"); Serial.println(isShort ? "Short" : "Long"); }
  }
  if (!debug) { Serial.write(buf, 8); }
}

void loop() {
  int currentSwitch = getCurrentSwitch();
  if (currentSwitch != lastSwitch) {
    lastDebounce = millis();
  }
//  Serial.println(currentSwitch);
  // Debounce it
  if (millis() - lastDebounce > debounceDelay) {
    switch (currentState) {
      case STATE_WAITING:
        // No keys pressed yet
        if (currentSwitch != SWITCH_NONE) {
          lastPressed = millis();
          currentState = STATE_SHORT_PRESSED;
        }
        break;
      case STATE_SHORT_PRESSED:
        // Wait to see if this counts as a long press
        if (currentSwitch == SWITCH_NONE) {
          // Send the keystroke
          sendKey(lastSwitchDebounced, true, true);          
          sendKey(lastSwitchDebounced, true, false);          
          currentState = STATE_WAITING;
        } else if (currentSwitch != lastSwitch) {
          // Shouldn't happen, but just in case you're using a different footpedal...
          sendKey(lastSwitchDebounced, true, true);          
          sendKey(lastSwitchDebounced, true, false);          
          lastPressed = millis();
        } else if (millis() - lastPressed > longPressThreshold) {
          currentState = STATE_LONG_PRESSED;
          sendKey(lastSwitch, false, true);
        }
        break;
      case STATE_LONG_PRESSED:
        // Wait for the transition
        if (currentSwitch == SWITCH_NONE) {
          currentState = STATE_WAITING;
          sendKey(lastSwitch, false, false);
        } else if (currentSwitch != lastSwitch) {
          // Likewise, switching between inputs shouldn't happen with this footpedal, 
          // but just in case...
          sendKey(lastSwitch, false, false);          
          currentState = STATE_SHORT_PRESSED;
          lastPressed = millis();          
        }
    }
    lastSwitchDebounced = currentSwitch;
  }
  lastSwitch = currentSwitch;
}

After you upload the code to the Arduino, you’ll also need to reflash the ATMega8U2 chip so that it can act like a USB keyboard. This sounds scary, but the instructions on the Arduino site can help you. When you’ve gotten the hang of reflashing the ATMega8U2 with the standard firmware, reflash it with the Arduino-keyboard-0.3.hex (Uno) or Arduino-keyboard-0.3-mega2560.hex (Mega) firmware from Arduino Hacking. After you reflash, unplug, and re-plug your Arduino, it should now appear as a keyboard. If you made a mistake, don’t panic. Just reflash the standard firmware onto it, and you can upload new programs again.

The last step is to map the F13..F18 function keys to something useful on the computer. I do this in software instead of hardcoding it into sendKey so that I can easily change the keycodes without reflashing the device. I’m on Windows 7 for work and other reasons, so I use AutoHotkey to map the keys. For example, the following AutoHotkey code maps left and right to Page Up and Page Down, and the center to Alt-Tab.

F13::Send, {PgUp}
F14::Send, !{Tab}
F15::Send, {PgDn}

This is a great combination for, say, reading an e-book while eating noodles.

On Linux, you can use Xmodmap or XBindKeys to remap your keys. For the Mac, KeyRemap4MacBook might work – haven’t tried it, though.

Picture!

Making a USB footswitch turned out to be an easy and fun weekend Arduino project. Hope you can build on this idea for more awesomeness! =) I’m looking forward to finding my next project idea. Hmm…

Weekly review: Week ending August 27, 2011

August 28, 2011 - Categories: review, weekly

The week looked intimidating at the outset, but it turned out to be quite reasonable when I got through it. Lots of hacking! I blogged about how I made a six-function USB footswitch, and I figured out how to control a 16×2 character display using the Arduino’s LiquidCrystal library.

On the web development front at work, I’ve been fixing bugs and working on projects. I’ve also started refreshing my Ruby on Rails skills by building little things at home to make my life easier. For example, I moved my clothes-tracking notes from paper to a simple Web dashboard with autocomplete. Tracking the clothes I wear can help me figure out what kinds of clothes I wear more often, if there are infrequently-worn clothes I might phase out, how long it takes for clothes to cycle through, and so on. I’m working on another small Ruby thing that analyzes my time. Eventually, this will grow into some kind of home dashboard that W- and I can use to track shared information such as library books, grocery lists, upcoming calendar events as well as personal stuff like my time- and clothes-tracking. =)

Lots of drawing, too. “Hello, Monday!” launched at IBM, and my next comic will come out mid-September. I’ve checked out a whole stack of books on cartooning, and am slowly working my way through them. Learning lots!

From last week’s plans

  • Work
    • [X] Get started on project T – lots of progress!
    • [X] Learn about project O – set up virtual machine for Rails on OpenSUSE
    • [X] Follow up on project I – fixed bugs
    • [X] Draw more IBM comics
    • [X] Learn about iRise – nifty for prototyping, but expensive
    • Helped debug errors in project M
  • Relationships
    • [X] Electronics hacking, yay!
    • [X] Wrap up build class for Free Geek Toronto
    • [X] Update build process and checklist
    • [-] Cook a new recipe (Vietnamese bun with nuoc cham?)
    • [X] Catch up on mail
    • [-] Transcribe interview – made myself a foot pedal, though!
  • Life
    • [X] Sleep at some point

Plans for next week

  • Work
    • [ ] Make more improvements for project T
    • [ ] Follow up on looking for an information architect for project T
    • [ ] Follow up on code for project I
    • [ ] Prepare technology selection notes for project O
  • Relationships
    • [X] Figure out LCD character display
    • [X] Help J- learn how to program the Arduino
    • [X] Reupholster chairs
    • [ ] Transcribe blogging interview, really
  • Life
    • [ ] Sketch year in review
    • [ ] Continue working on dashboard
    • [ ] Try microphone as sound sensor
    • [ ] Prepare month in review with stats

Time analysis

Activity This week Last week Delta Notes
D – Break 1.9 2.4 -1.1
D – Drawing 10.0 4.2 5.8 Lots of practice
D – Gardening 0.2 -0.2
D – Latin 0.5 -0.5
D – Learning 9.5 1.1 8.4 Electronics!
D – Other 4.8 4.9 -0.1 Also electronics!
D – Personal 3.9 3.9 Rails hacking for home dashboard
D – Piano 2.6 -2.6
D – Reading 0.1 3.4 -3.3
D – Shopping 1.1 -1.1
D – Social 7.8 11.2 -3.4
D – Volunteering 3.8 8.0 -4.2 Short gap between build classes
D – Writing 5.0 6.6 -1.6
R – Exercise 12.2 2.5 9.7 Lots of biking
R – Routines 8.2 7.9 -0.8
R – Cooking 1.5 4.7 -3.2
R – Eating 2.0 6.0 -4.0
R – Tidying 3.7 5.0 -1.5
R – Travel 0.8 -0.8 Yay working from home
Sleep 53.5 53.9 1.4 7.6 hours a night, still okay
Work 40.1 40.8 -0.7

(Update: Noticed some post-midnight timestamps weren’t being correctly exported, so I’ve updated the output.)

Slightly more discretionary time compared to last week. Further shifted piano and Latin time towards drawing and learning about electronics. Shifted volunteering time towards those, too. Learned lots! Bedtime shifted quite a bit later, though. Should probably look into tapering off electronics / web development late at night, because it’s so tempting to stay up for just one more tweak. For example, I’m writing this at 12:59 AM. Bedtime!

Drupal debugging story: Rules defined in multiple Features

August 29, 2011 - Categories: drupal, geek, story

Fatal error: Unsupported operand types in …../patched/rules/rules/rules.module on line 347

That was the error message Daniel sent me when he asked for my help in debugging. After some digging, I found out that the rules had been defined in two features, so Drupal got thoroughly confused. After I commented out one of the implementations of hook_rules_defaults and deleted the relevant rows from rules_rules in the database, the site worked again.

Daniel wanted to know how I figured out that problem, so here’s the story.

The line number told me that rules_sort_children was having problems. I added a var_dump to it so that I could find out what kind of unexpected data was causing the errors.

if (!is_numeric($element[$key]['#weight'])) { 
  var_dump($key, $element[$key]['#weight']); 
}

The output showed that the regular rules were fine, but our custom rules weren’t – weight was an array instead of an integer.

I looked at the difference between the features in code and the features in the database. The rules were being duplicated. I tried updating the features to match the information in the database, but the code looked wrong, so I used git to stash the changes. I also tried reverting the features, but that didn’t solve the problem either. Time to dig deeper.

I backed up the database, then deleted our custom rules from rules_rules. Still problematic. I commented out the rule definitions in our site_structure.features.inc. The rules administration page now successfully displayed – but mysteriously, the rule definitions were still available.

I looked at the rule tags to see where else they might be defined, and found that another feature had included the rules. Aha! I commented those out. As expected, the rules disappeared from the administration page. I’d identified the problem: the rules had been defined in more than one module, which had thoroughly confused Drupal.

Because it made more sense to define the rules in our site_structure feature than in the other feature, I uncommented the site_sitestructure_rules_defaults definitions and left the other feature’s rules commented. That worked.

I tried restoring the rule customizations from the database, but that gave the same error. The database copy had multiple definitions, and I didn’t feel up to picking my way through the data or writing a Drush script to manipulate the rows. I re-deleted the customizations to get a clean copy. Just in case the other feature had more recent definitions of the rules, I compared the two. Aside from the extra tag, they were identical, so I didn’t need to copy any information over. It meant that Daniel would have to make his changes again, though.

Features: When it’s good, it’s very very good. When it’s bad, it results in quirky bugs. Make sure you don’t define your rulesets in multiple features. Drupal Features still has some rough spots when it comes to Rules. I remember when I created this feature with the rules in it – it created duplicate rules, so I needed to delete the old ones. Still, it’s a great way to export rules and other configuration changes to code, even though it takes some getting used to (and the occasional bit of database-diving).

Anyway, that’s the story of how I identified that issue and worked around it.

When you’re faced with a fatal error involving unsupported operand types, figure out what kind of operands it expects and print out anything that doesn’t match. Then you can start figuring out the real problem, which is how that data got in there in the first place. I’ve used this to find form elements that were mistakenly defined, array elements that were unexpectedly null, and so on. Don’t be afraid to add debugging code to core or contributed modules, particularly if you can use source code control to restore a clean copy. If you use a runtime debugger like XDebug, you can easily explore the data and the call stack. If not, there’s always var_dump.

Hope that helps!

Batch cooking

August 30, 2011 - Categories: cooking, kaizen

From Sunday night: Cooking large batches of food can be tiring, but for us, it’s worth it. Today, W- and I baked two pans of lasagna, stuffed and roasted two eggplants (eggplant and sausage stuffing), four peppers (rice, tomato, and sausage stuffing), and two roast chickens (couscous and dried fruit stuffing). No special recipes – just notes from the Internet and from cookbooks.

Cooking roughly 48 servings of food and cleaning up along the way took a little less than five hours, with both of us working in the kitchen. Ingredients came to about $2 per serving. Eating out costs roughly $6.50-11/meal. $210-$430 after-tax savings is pretty good for 10 hours of enjoyable work.

Now we’ve got a pot of baked beans in the oven. Because the beans will take a few more hours to bake, W- and J- made brownies. The difference in sizes between the small and extra-large eggs prompted a quick economics lesson on pricing strategies. In the meantime, I’ve heated up some strawberry-rhubarb tarts for a late-night snack. Life is good.

We could scale up even more. If we converted the raw ingredients in our freezer to cooked portions, bought more food containers, and planned the additional recipes, we could double our capacity easily, and maybe even reach quadruple the capacity. The chest freezer can hold around 60 square Rubbermaid TakeAlongs with some additional room for loose ingredients, and the fridge freezer can hold some more. The Internet has many sites dedicated to once-a-month cooking and other batch cooking ideas, so it’ll be easy to find recipe ideas to add to our repertoire.

What would doubling our output look like? Let’s say we use today’s cooking as a template. For variety, we’d add two Shake-n-Bake chickens and a large pot of curry. We’d probably need 18 additional cups of rice, and more food containers. It would probably take 2-3 more hours; maybe just one additional hour if we use pre-sectioned chicken thighs and drumsticks. If our freezer restocking time coincides with a sale on chicken, I think we’ll give it a try.

Hmm…

Getting rid of the “I can’t draw”s

August 31, 2011 - Categories: drawing, sketches

In a comment, The Average Jane said that she can’t draw to save her life. I want to say something about that, because I keep hearing self-put-downs like that from people. I realized that I’ve got very few “I can’ts” in my life. Maybe I can nudge people towards that kind of feeling, too.

So. Drawing. Forget all the pretty pictures that other people can make. Can you draw as well as a 3-year-old kid draws? For reference, here are some typical developmental stages: http://www.learningdesign.com/Portfolio/DrawDev/kiddrawing.html

You can probably draw at least as well as a typical 3-year-old child. It’s not hard. A circle, a few lines, and your imagination can fill in the rest. That’s drawing. Drawing doesn’t have to be amazing. You can start wherever you are. You can even get better if you want.

imageWhen I feel stuck, I draw the most stereotypically kid-like drawings I can think of in order to get me past the “I can’t draw”s. I celebrate the fact that I can draw something recognizable. This makes me realize my challenge is more about “I can’t draw as well as I’d like to.” That’s manageable. That’s just about time and attention and practice. It doesn’t matter if I have years of catching up to do. I can draw, and I can get better. Differentiating between my “I can’t”s helps me stay motivated.

Do you tell yourself that you can’t draw? What happens if you tweak your expectations?