I joined IBM four years ago today, sliding right from my master’s degree into a position that was tailored to my passions. I wanted to focus on Web 2.0 consulting and open source web development, and I did. I’ve facilitated workshops around the world, coached clients and co-workers, and helped flesh out and implement social media strategies. I’ve gotten pretty good at Drupal, and I’ve done sites in Ruby on Rails, too. I’ve learned a lot about automated testing, system administration, automation, and other useful skills. I’ve been promoted, and I’ve taken on additional responsibilities like estimating effort, preparing statements of work, and leading other developers.

Energy

Well past the honeymoon period, I’ve somehow escaped the cynicism that saps the energy of many recent hires. I see IBM like I saw it in the beginning: an organization with its own challenges, but still fundamentally inspiring and wonderful. When asked how I am, I find myself answering “Fantastic!” – and meaning it.

I’ve kept myself engaged by taking responsibility for my motivation. My manager helps by showing me how to work with the system and helping me find projects that fit me well. In the end, though, I choose how to respond to the small triumphs and frustrations of everyday work. I’m generally good at celebrating successes and fixing annoyances, which helps a lot.

I’ve worked on making my experience of IBM pretty good, and I’ve had a remarkably wonderful time as a new hire. I’ve been lucky that both of my managers have been great allies, and that I have plenty of co-workers and mentors who share their insights and help me figure out IBM. Investing in tools pays off: automation minimizes frustrating work, and an extra laptop makes development go faster. I often find myself saving time by referring to the notes in my blog, and the blog has helped me connect with clients, co-workers, and other developers.

Time

I continue to work around 40 hours a week, which forces me to be good at estimating how much I can do within that time and focusing on doing it. It means that I have time for other priorities, such as life and relationships. It also means that I can bring a lot of energy to work because I don’t feel like it’s taking over my life. I minimize travel, as trips require a lot of paperwork and disrupt a lot of things.

In the beginning, I took on lots of volunteer things: coaching other IBMers on Web 2.0 through the BlueIQ initiative, writing a lot on my personal blog, skimming through everything published in IBM’s internal blogosphere. (Back then, it was possible – there was just one place to find people’s blogs, volume was manageable, and you felt like you really got to know people.) Now, I’m more selective about the things I volunteer to do, and I try to help other people build their capabilities as much as possible. That means the occasional bit of work on:

• a Lotus Connections community toolkit that makes it easy for lots and lots of community owners to create newsletters, get metrics, and perform other tools;
• answering questions and sharing resources on using Lotus Connections for facilitating virtual brainstorming
• drawing comics about life at IBM

I find myself thinking about these side projects like a semi-passive income stream of good karma. I look for places where a little effort can translate into a lot of benefit.

Things I didn’t expect when I signed up, but which worked out really well:

• I’ve worked on a number of websites for non-profits. More than half of my months at IBM involved one non-profit project or another, sometimes balanced with another project and sometimes as my main focus. It turns out to be incredibly fulfilling and one of the reasons that might convince me to stay around. Wins all around: clients have better web capabilities, IBM gets to help make a difference, my department earns internal dollars, and I learn and use cool skills while working on fascinating challenges.
• I’ve been able to try all sorts of things. Presentations, blog posts, comics, videos, virtual reality discussions, group videoconferencing, telepresence, research… I guess when people know you’re a positive geek who might come up with related ideas, links or tools, they invite you to check out what they’re working on. =)
• The system is not that scary. Sometimes things don’t work out or they’re more difficult than they could be. Most of the time, people are great at being flexible.
• I think I’m figuring out a growth path that doesn’t involve aspiring to be an executive. I’d like to become a good, solid developer like my role models are. I’d also like to train/mentor more people so that we can increase our organizational capacity for these kinds of projects.

Looking ahead

I could happily continue doing this sort of work for years, I think. I like the mix of development and consulting. I might gradually move to leading more projects and training people along the way. It would be a good way to scale up. The kind of projects I love working on – small rapid web development projects – don’t typically involve large teams, though. Growth will probably involve going deeper (say, customizing Drupal and Rails even more), building assets so that we can save time, and mentoring people working on other projects.

I like working with IBM, even though sometimes I grumble about the paperwork. I really like these non-profit projects I get to work on, and it’s hard to imagine having quite the same kind of set-up anywhere else. I learn a lot from our commercial projects, too.

Good financial planning makes riskier choices easier to consider. A different position? A career change? We’ll see. The status quo is pretty darn awesome, though, and there’s plenty of room to grow.

Would I make the same decision again, if a time machine took me back to October 2007? Yes, without hesitation.

Four years. Thanks to writing, I know where the time went, and I can see how I’ve grown. There’s still a lot to learn, and I’m looking forward to sharing that with you.

I wasn’t sure which variant would work out best, so I’ll let the intranet editorial team pick their favourite, and I’ll share others I considered here:

Watson is the question-asking trivia-spouting Jeopardy-winning artificial intelligence that IBM has been working on. Aside from handily beating human contestants and prompting rounds of “I for one welcome our new robot overlords”, Watson might have interesting applications in medicine, law, and other challenging fields. What else might we use this kind of power for?

I don’t think Watson can recognize speech yet, but maybe someday!

The part in which I think out loud and make things less funny in the process of understanding humour

I’m not entirely happy with the helpdesk setup here. It’s a classic joke, but it twinges the same part of me that replaces "user" with "person" whenever I can. Also, totally awesome people use the systems I build. I run into far more programmer bugs and d’oh moments myself than classic "user errors". “Is it plugged in?” is further along the spectrum than “Have you rebooted?”, as rebooting is (unfortunately) still very much a valid approach to problems. Anyway, it was a joke that needed to be made. Now that we’ve made it, we can move on.

“Toronto” refers to Watson’s mystifying response to a question about US cities – understandable when you learn about how Watson works, but still interesting and strangely reassuring.

The "server on fire" bit refers to this idea of printer on fire, which I think I came across in a compilation of funny Linux source code comments. There’s a story about that, of course.

I wonder what related situations might be interesting. Relationships are fair game. Time travel: Watson in scenes of childhood or old age? Watson in the distant past, or in the distant future? There’s inverting it: imagining a cluster of people madly looking things up behind the scenes like a distributed Mechanical Turk (the old-school kind). There’s swapping other things in: what would a cat be like with response probabilities? Hmm…

If you’re an IBMer, you might have noticed this rather odd-looking Top Story on our intranet homepage:

It takes you to this:

More than a hundred comments so far, and it’s only noon! Looks like lots of people appreciate a bit of humour on Mondays. =) The w3 editorial team planned this as a 3rd-Monday-of-the-month feature, so the next one will come out on September 19.

I like IBM, so these comics will be more about gently poking fun at our culture than about sharp cynicism or Dilbert-like humour. Someone’s got to be able to look at the bright side of corporate life. =)

This comic’s visual style was inspired by Exploding Dog, which I love for its use of colours and simple shapes. I drew it on a Lenovo X61 tablet using Autodesk Sketchbook Pro.

I’ve requested a whole bunch of books from the library and am making time for regular practice. Looking forward to learning more about comics while squeezing this into the gaps of my day job (consulting and Drupal/Rails web development). Glad to make people’s Mondays a little bit brighter!

Thinking through a particularly tough replication problem at work. You see, I’ve been working on a dashboard for a small call center. Since they sometimes have really short calls, they’d like the display to be as close to real-time as possible. Cisco handles the calls and stores the information in either of two Microsoft SQL Server 2000 servers depending on the failover situation, which use merge replication to stay in sync. We have a custom script that takes the call information and adds it to our own database. Then the PHP dashboard I wrote uses JQuery to poll the server every 5 seconds and display updates.

Since our database wasn’t part of the standard configuration, I also set up SQL Server replication for it. I have little experience with Microsoft SQL Server and I found it even more stressful trying to look up steps while people were watching (ah, web conferences), so I heaved a huge sigh of relief when I saw a row replicate properly.

I originally programmed the dashboards to check only for any updates since it last checked. It turns out that I need to account for late synchronization, too. It seems to take about ten seconds, maybe more, for the information to replicate from one server. This meant that if the dashboard on the second computer had already checked for updates, it would miss the older updates that had just newly replicated, and dashboards polling the two servers would eventually get out of sync. Both servers needed to support read/write operations. The systems don’t have a static master-slave relationship. Either one of the two servers could become the primary server – the first server to receive updates – transparently, depending on the failovers.

I’m modifying the script to check for older updates. It’s a trade-off. A larger window means more transfer time, higher server load, possibly more delays. A smaller window could miss updates. If I set the dashboard to check the last 45 seconds, I get a delay of about 25 seconds from the time that a call shows up on a dashboard looking at the primary server and the time the call shows up on a dashboard looking at the secondary server. I’m reasonably happy with the code.

I also need to change the SQL Server configurations, so I’m waiting for a maintenance window and the go-ahead to do so. Anything I can learn or clarify before then will help me make the most of the limited time (and reduce stress, which is also important). Here are some things I need to properly figure out:

• What kind of replication should I set up – transactional or merge?
• How can I set up push-push replication without looping?
• Is it realistic to try to push this faster than database replications every 10-25 seconds? There aren’t a ton of updates – maybe 1 a second at the peak – but they do want the databases to replicate as frequently as possible. Continuous replication should be immediate – you can even do two-phase commits to make sure things are always in step – but I don’t know if I’m doing this right.

Working with Microsoft SQL Server 2000, I wish I had lot of these things I’m reading about, like MS SQL Server 2005’s Database Mirroring and MS SQL Server 2008’s Configure Peer-to-Peer Topology Wizard. There are these instructions for setting up nonpartitioned, bidirectional, transactional replication, but I don’t know if they’ll really get me to where I want to go, and they look complicated. The stored procedures look familiar, though, so maybe that’s how I got replication working even though the config doesn’t reflect it.

Theoretically, I should be able to just reach out into this really big company of ours and find a Microsoft SQL Server expert who can set this up properly in all of five minutes, but it’s complicated. Something’s changed, or maybe I have. I’m letting the focus on utilization get to me. I hate having to say no to other people because I’m supposed to focus on billable work, and I know it’s a hassle and a half to set up project codes for spot consulting, and I don’t feel comfortable putting other people in that position. Anyway, I’ve posted a question in a community with 500 members, and I’ll try reaching out to one of the community SMEs tomorrow.

You know how it’s a good learning experience to work outside your comfort zone? This is really far out of my comfort zone. I tried asking for a proper DBA. I didn’t get one. No amount of searching and speed-reading is going to turn me into an experienced one, at least for this project. I really hate this feeling – wanting to do right by the client, but feeling the limitations of my experience, and worrying about messing things up even more.

I’m going to see if another attempt to set up proper continuous push replication will do the trick. I’m also going to try e-mailing people to ask for their help, and see if our resource deployment managers know of anyone in between engagements who might be able to help. It’s a long shot, but if I take enough long shots, it might work.

You know, this makes dealing with cross-browser issues look like fun. But it needs to be done, and there will be rare moments like this anyway, so I might as well learn how to even more effectively ask for help in the company.

2011-08-17 Wed 20:13

I’m wrapping up a Drupal 6 project which was funded by one of IBM’s corporate citizenship grants. The Snake Hill folks we’ve been working with will continue working with the client until they’re ready to launch. For my part, I’ve been in user acceptance testing and development mode for almost a month, rolling out new features, fixing bugs, and getting feedback.

The project manager has shuffled some hours around and made sure that I’ve got some “support” hours for follow-up questions after we turn the project over.

What worked well

Hey, I can do this stuff after all! I gathered requirements, estimated the effort, negotiated the scope, communicated with the clients and other team members, and generally did other technical-lead-ish stuff. I’ve done that on other projects, but usually that was just me working by myself and talking to clients. This one was more complex. It was fun figuring out what would fit, how things were prioritized, whether or not we were on track, and how to make things happen. I’d love to do it again. (And with the way the world works, I will probably get an opportunity to do so shortly!)

Understanding a project deeply: I was on the first phase of this project as well, and the experience really helped. We didn’t have any disruptions in technical leadership on our part, unlike in the first phase. See, last year, the IBM technical lead who had been talking to the client ended up leaving the country, so we had to repeat a few discussions about requirements. This time, I could draw on my experience from the first phase and our ongoing discussions about the client’s goals for the project. That was fun.

I’ll be turning the project over to the other development company, and the client’s concerned about whether they’ll be able to pick things up and run with it. I’ve tried to write down as many notes as I can, and I also invested time in briefing the other developers on the overall goals as well as the specific work items. Hope that works out!

Externally-accessible issue tracking: In the previous phase of this project, issue tracking consisted of e-mailing spreadsheets around. It was painful. One of the first things I did when we started this phase of development was to set up a Redmine issue tracker on the client’s server. After we gathered and prioritized requirements, I logged them as features in Redmine and split them up over the different phases. I reviewed our list of outstanding work and filed them as bugs, too. As feedback came in, I tracked bugs. I took advantage of Redmine-Git integration and referred to issue numbers in my commit messages. When people e-mailed me their feedback or posted messages on Basecamp, I created issues and added hyperlinks.

Having an externally-accessible issue tracker helped me worry less about missing critical bugs. I also shared some reporting links with the clients and the project manager so that they could track progress and review the priorities.

On future projects, I would love to get to the point of having clients and testers create issues themselves. Wouldn’t that be nifty?

Git for version control: I’m so glad I used Git to manage and share source code between multiple developers. The other developers were fairly new to Git, but they did okay, and I figured out how to clean up after one of the developers wiped out a bit of code after some commit confusion. Git stash and git branch were really helpful when I was trying lots of experimental code.

Developing with a non-default theme: We had a lot of development items to work on while the No.Inc creative team got their Drupal theme together. Once No.Inc sent the theme, I layered it on top of the site, fixed the usual problems, and had even more fun working on a site that looked halfway done. Definitely recommend getting a reliable theme in place sooner rather than later.

Mentoring people: I helped a new developer start to get the hang of Drupal. It was a slow process (must raise estimates even more when dealing with newbies), but I hope the investment pays off. I wrote (and updated!) documentation. I identified small tasks that he could work on first. I checked on him every so often. I successfully resisted the urge to just do things myself. Slowly getting there…

Decision log: I used a wiki to keep track of the design decisions I needed to make, the alternatives I considered, and what I eventually chose. That was helpful for me. I hope it will help future developers, too.

Linux VM on a Microsoft Windows host, XMing, and Plink: I’ve tried lots of different configurations in the course of this project. Doing my development inside a virtual machine has saved me so much time in terms of restoring from backup or being able to tweak my operating environment. I started with a Linux VM on a Windows host, using Samba to access my files and either Eclipse or Emacs to edit them. That was a bit slow. Then I shifted to a Linux VM on a Linux host, SSHing to the VM and using Emacs from the VM itself. That was great for being able to do Linux-y stuff transparently. But then I found myself wanting to be back on Microsoft Windows so that I could use Autodesk Sketchbook Pro (Inkscape and MyPaint aren’t quite as awesome). I ran XMing to create an X server in my Windows environment, used plink to connect, and then started a graphical Emacs running on my virtual machine. Tada! I could probably make this even better by upgrading to 64-bit Microsoft Windows, adding more RAM, and upgrading to a bigger hard disk. (Alternatively, I could host the VM somewhere else instead of on my laptop…)

What I’m going to work on improving next time

Better browser testing, including cross-browser: I’m getting slightly better at testing the actual site, motivating myself with (a) interest in seeing my new code actually work, (b) the remembered embarrassment of gaping bugs, and (c) the idea of slowing down and getting things right. Juggling multiple browsers still doesn’t make me happy, but maybe I can turn it into a game with myself. Selenium might be useful here as well.

Continuous integration: I set up Jenkins for continuous integration testing, but it fell by the wayside as I wasn’t keeping my tests up to date and I wanted more CPU/memory for development. I ran into a number of embarrassing bugs along the way, though, so it might be worth developing stricter discipline around this. I’m still envious of one of the Drupal projects I heard about in IBM, which got through UAT without identified defects thanks to lots of manual and automated testing. If I add more power to my development machine or offload testing to another machine, that might be a good way to stick to this process.

Closer communication with clients and external developers: We set up short daily meetings for the developers, but sometimes people still felt a little lost or out of touch. On future projects, I’ll make sure the clients have it on their calendar as an optional meeting, and maybe see about getting e-mail from people who can’t join on the phone. If I’m the tech lead on a future project, I’ll sit in on all client status update meetings, too. We found out about some miscommunications only when I handled one of the status calls. Fortunately, it was early enough that we could squeeze in the critical functionality while reprioritizing the others. Tense moment, though!

Better vacation planning: I realized we had a 4-day weekend the week before we had it, and we forgot about some people’s vacations too. Heh. I should get better at looking at the entire project span and listing the gaps up front.

Earlier pipeline-building: I nudged some project opportunities about a month before our projected end date, but that wasn’t long enough to deal with the paperwork lag. Oh well! Next time, I’ll set aside some time each week to do that kind of future pipeline-building, and I’ll set myself reminders for two months and a month before the project ends. Not a big problem.

My manager’s been lining up other Drupal and Rails projects for me to work on. Looking forward to learning all sorts of lessons on those as well!

Other Drupal lessons learned:

• June 2009
• April 2009
• Lots of Drupal-related posts

2011-08-10 Wed 17:08

Henry Will asked me how I got to work on such interesting projects. What worked particularly well: blogs? presentations? networking?

Working in a big company is a bit different from marketing yourself outside. In a big company, it’s easier to establish and maintain large networks of people, and the organizational structure also helps pass messages up and down. When you hook in through a number of connectors (for example, my manager), figure out the tools for finding opportunities on your own, or build a reputation, opportunities can flow easily. Outside a large company, word of mouth is still powerful, but it can be difficult to build those relationships over distance and with an large number of competitors.

I do a lot of work related to Web 2.0, social media, Gen Y, PHP, Rails, and AJAX. For consulting and strategy work related to Web 2.0, social media, or Gen Y, I find that most of the leads come in through the presentations I’ve given, or from people I’ve worked with in the past. Short presentations with catchy titles or designs can go a long way. I haven’t been proactively investing in presentations. I tend to create them on request. Presentations take a lot of time for me to prepare, so I try to maximize their ROI. In fact, I get a lot more value from the blog posts that I write before a presentation (full speaker notes, ideas, etc.) and after a presentation (questions, lessons learned).

Many of my PHP/Drupal, Ruby/Rails, and AJAX work comes in through my manager, who knows about my different skills and interests. Sometimes I search our professional marketplace for upcoming opportunities requiring those skills so that I’l always have projects in the pipeline. I actually like this work more than consulting (which can be fuzzy and hard to define), so my manager and I try to pick development projects that will keep me busy and happy while still being flexible enough to accept consulting work.

If you work in a company, it really helps if your manager knows what you’re good at and what you’re interested in. He or she may be plugged into streams of opportunities, and help the right ones flow to you. It also helps to invest time into sharing what you know and helping other people out. That way, people know what you’re good at, and they can keep an eye out for things that fit too. You might get to the point of having too many opportunities, which is a great problem to have. If so, build relationships and help others by sharing those opportunities. Good luck and have fun!

(From April 23:)

I’m helping out with a proposal at work. The team asked me to condense a 250+-page requirements document into a spreadsheet of use cases. I’m new to architecture, but I gave it my best shot, sending the architect quick drafts so that we could zero in on something useful.

My first draft was too low-level, too detailed. My second draft was a bit better, but still too granular. My third draft was at about the right level, but some use cases were still too big. My fourth draft was workable. Hooray!

When you’re learning something that’s hard to pick up on your own, figure out how you can iteratively improve with feedback. Even if an expert doesn’t have the time to walk you through the process, he or she might be able to quickly tell you if you’re on the right track. See if you can break your work down into small portions you can work on until you get them right, and apply what you learn there to the rest of the work. Good luck!