The Drupal site I’m working on is slowly coming together. I still feel an itch to redo the big, hardcoded structures that a previous developer left behind, but I’m focusing on all the functional pieces first. Then I’ll do the styling and spacing tweaks, and then I’ll think of making that structure more flexible.

I like working with other people’s code, even if the code occasionally makes me go “Huh? What were they thinking?” It’s good to be able to work with other people’s structures and gradually immerse yourself in a project. Otherwise, you’d be limited to just the things you can build from scratch.

We’re still quite a bit away from having something that would be ready for launch, but we’re making progress. I don’t know if I’ll be on this project through launch, but it would be nice.

I like launches. =) I want to have more of them.

Yesterday’s coding session with CSS was fantastic. I used WinWarden to make my browser translucent, and I overlaid it on my reference documents. This made it a breeze to check alignment, because I didn’t have to use any measuring tools. I used Chrome’s developer tools to manually adjust the stylesheets until things looked right, adding display: block to the parts I was working with. Then I copied the numbers into my SASS file so that it could generate the CSS.

I also found a GIMP script for exporting all layers as separate images. I had to rename a few layers, but the results made it much easier to flip through images instead of toggling visibility trying to find the logos I needed. (It turned out that the logos were not included, so I’ve asked the design firm to send them to me.)

I converted the complex front page into a Drupal panel layout, getting rid of thirteen regions that were cluttering up the main block management screen. This also makes it much easier to update the content, yay! I’m looking forward to converting other pages. The previous developer used multiple regions instead of controlling visibility through configuration, so there are a lot of templates and regions.

Dual-screen worked out great, too, although I still need to fiddle a little with my ergonomics to make sure everything works out.

I’m looking forward to making this even better. I’ve only got a few more weeks on this project, but I might take on more styling in the future if it turns out I can deal with the headaches associated with cross-browser styling.

After I get the rest of the basic requirements in place, I want to automate testing and screenshots, particularly for regression-checking and for cross-browser compatibility. Selenium and WebDriver look like the way to go if I want to simulate hover events. If I can’t figure out how to use WebDriver within the time I’ll set aside for learning this, I can use JQuery to fake toggling the classes. Automated screenshots + PDF Split and Merge + ImageMagick for compositing (maybe 50% opacity?) will make it easy to spot glaring errors.

That will have to wait for next week. In the meantime, there’s a three-day weekend ahead, so I’m going to make lots of progress on Quantified Awesome. Yay!

This project transition is smoother than the others. I’m joining another Drupal project right away. I’ve drafted estimates and documents of understanding for another Drupal project, and my manager is lining up some consulting for us to work on together. I’m not taking a vacation other than the statutory holidays. Two trips for my sister’s wedding took all of my vacation days! (Worth it, though.)

What’s working well about this transition:

• Getting the agreements set up for my next project took much less time than I expected, even though it’s cross-border. It could be because I’m joining an existing project.
• We’re winding down this project gracefully. It helps that we’ve focused on lower-risk items towards the end, and that we’ve scheduled in time for documentation. It also helps that we launched two weeks before project end. We’ve had time to deal with bugs and add features, and we can be more confident that the client’s set up for success.

I want to keep working on Rails at work, but I don’t think I’ll get any Rails projects in the pipeline any time soon. I’m sure the upcoming projects will be lots of fun too, though, and there are so many non-Rails techniques I want to improve on anyway. I’d like to learn how to use Selenium to test websites using multiple browsers, and I might get a chance to do that on this project. It would be good for me to learn more about consulting skills on other projects, and I’m definitely looking forward to taking advantage of my manager’s mentoring. I’ve got Quantified Awesome as a personal Rails project, and I might tinker around with Android development as well.

Now that we’re not scrambling to squeeze in as much as we can before the project end date, I’m looking forward to normal hours. More space for extracurriculars!

Rails is awesome. We built a workflow/reporting system for ~120 users using Rails 3. My part of the project came to about 468 hours, or roughly 60 workdays (~ 3 months), and I worked with another developer who also put in around the same number of hours. We worked with a graphic designer, a CSS/HTML developer, a tester, a project manager, and the client, and we put together a surprisingly large set of functionality.

It’s amazing how quickly the site came together. I built a simple prototype to help the other developer get started with Rails, and we fleshed it out with the client’s input while waiting for the creative design. I started with web_app_theme so that we could have a decent-looking interface for starters. When the client approved the graphic design, another developer sliced it up into HTML, CSS, and images for us. I took those, converted them into HAML and ERB, and we were off and running. Every weekly sprint meant a chance to show off useful functionality and get feedback. It was awesome.

We were initially worried that building all the UIs from scratch in Ruby on Rails would mean taking up more time because we couldn’t use CCK or Views to quickly throw everything together. It turned out that HAML, partials, and semantic_form_for made the forms and reports easy to do. Filtering reports was straightforward with ActiveRecord and scopes. Because we built the screens ourselves, we didn’t have to fight with Views or CCK for the last 20% of tweaking, and I didn’t have to kludge any SQL queries (yay, no views_pre_execute!).

I was working on a Drupal project shortly before this, and I spent some time supporting the Drupal project during this one. Rails made my brain much happier. I felt that I could organize my code more cleanly, and I could test it more thoroughly, too. I didn’t have to fight so much with other people’s modules or themes. I like Drupal, and I’m still looking forward to doing more projects with it. But I wouldn’t mind working on more Rails projects, and I’m glad I’ve got Quantified Awesome as a personal project.

Drupal does many things better than Rails. Drupal modules tend to be more mature and better-documented, and it seems like there’s been more work on scaling Drupal. Internationalization is also more straightforward in Drupal, although Rails I18n is easy to use once you’ve gotten the hang of it. Drupal module dependencies seem a little easier to sort out, too. But Rails is fun!

Tests will keep you sane. This was the first project where we invested in developing a large suite of automated tests. We used Cucumber for high-level tests and RSpec for everything else. The tests caught many regression errors we might have otherwise missed. Test-driven development was fun, too, because the tests gave us tangible progress and simplified checking.

There were times when I gave in to the temptation to commit without running the tests, and I almost always regretted them. (Particularly after friendly finger-wagging from the other developer!)

Issue-tracking rocks. We’ve liked using Rational Team Concert in the past. Getting an externally-accessible instance was complicated, so I set up a Redmine issue tracker as soon as we started the project. We used Redmine to plan work, track bugs, and even collect feedback from the client. As of the time of writing, we had created 766 items and closed 683 of them (89%).

We started with story points, but didn’t end up continuing with them for the rest of the tasks. When we needed to prioritize, we estimated the hours required for each task in order to help the client decide. That worked out quite well. I haven’t tracked item-level time spent, but that seemed to be roughly around my estimates.

I now estimate more time than I used to, because I’ve started factoring in both writing and running tests. It’s a little strange being the pessimistic estimator instead of the optimistic one, but it’s good for the project.

Selenium is great for screenshots, too. Not only is Selenium good for automated browser-based testing of web pages, but it’s also a handy way to capture screenshots for documentation or demos.

Lotus Symphony and Microsoft Word don’t get along. We wasted a few hours trying to update the user guide with the new screenshots, only to find out that the PDF still got screwed up because I didn’t have the fonts the client used. Those fonts were part of Microsoft Office and weren’t on my system. The client took care of updating the user guide so that she could format it the way she wanted, and we focused on code.

Plan with the end in mind. Short projects mean that milestones can sneak up on you before you notice. Half-way through the project, we realized that the project end date was coming Really Soon Now, and we scrambled to put together a launch plan. We wanted to launch a few weeks before the end of the project, to give people time for feedback and updates. That meant that we needed to send pre-launch e-mails one week and two weeks before the launch, which meant… that we needed to start sending those e-mails within a week. Fortunately, the client, IBM PR, and everyone involved managed to get it all sorted out, and we launched.

I like launching. I would like to do more of it. We don’t do it nearly enough on short projects, but these projects are much more likely to launch if we’re around to help with the transition than if only the client is there.

On a related note, I get antsy about adding new functionality before the end of a project. This makes sense, of course. I don’t mind adding new reports and other reading-related functionality, but workflow tweaks are scary. When we’re planning future projects, we can consider similar risks for late-project tasks.

Things I want to improve for future projects:

• More Rails! More!
• More automated tests
• Track time estimates and actual time in order to improve estimation accuracy
• More launch planning
• More blog posts about what I’m learning
• Optimization

Things I want to reduce on future projects:

• Avoid documentation formatting problems – I guess that means Microsoft Office when working with clients who use it
• Must not give in to temptation to skip tests

For this project, I put a lot of time into writing tests. Now they’re paying off. User acceptance testing and beta testing is going to be limited thanks to some real-world project constraints, so we’ll have to catch more bugs ourselves.

I want to get to 100% test coverage. That will go a long way to increasing our confidence in the results.

We had neglected our tests in the beginning, so the autogenerated tests fell out of sync. A few weeks ago, I put in around six hours to get everything back into shape.

Once that was sorted out, I looked for ways to encourage us to keep the tests working. After briefly experimenting with autotest, I settled for using git precommit hooks to force us to run tests before checking code in. Fortunately, the other developer agreed with this idea, so I set up the same script on her system.

Rspec was great for detailed tests, but I missed the expressiveness of Cucumber. After a few more hours of work, I got our Cucumber tests to work again. I wrote a number of tests from different users’ perspectives, then added Cucumber to the pre-commit hook.

The next step in our testing journey was to set up code coverage analysis. It took a little hacking to get rcov to play nicely with cucumber and rspec, but I got that to work too. Now we can get a sense of what still needs to be tested, and we have clear goals: 100% of code covered by tests, and 100% of user stories matched up with tests too.

On another project, I’ve been working on automated tests with Selenium. I hate letting the clients run into bugs, and I hate breaking things I thought I’d already fixed. I also hate clicking around and doing repetitive actions. As we run into things I’ve broken and as I change things, I write tests.

For me, test-driven development is incredibly motivating: the carrot of seeing tests pass and coverage stats go up, the stick of embarrassment when things break. =) I can take advantage of that energy and change my development habits.

My next development goals related to testing:

• behaviour-driven development and business-user-compatible test cases
• 100% code coverage from day one
• integrating Selenium tests into our build scripts
• Selenium web driver screenshots to assist in keeping user documentation up to date
• writing automated tests for life – making progress!

From November:

We got the mail information for our Rails site, so I reconfigured the mail server. We’re doing a lot of development, and testing mail is much easier when you can send example mail addresses to one bucket of mail. Here’s how I set up the server to redirect everything addressed to @example.org to a test mail address.

First, I set the mail server up for local delivery only, and I confirmed that I could send mail from a local user to another local user account. Then I experimented with rewriting, using virtual_alias_maps to rewrite addresses to a specific account. I confirmed that worked. Then I checked the database to make sure we didn’t have live e-mail addresses, reconfigured the mail server to deliver Internet mail, and crossed my fingers. A few quick tests showed that mail was being delivered as planned – example.org mail routed to our temporary address, everything else delivered normally. We’ll see how this works!

Here’s our /etc/postfix/main.cf

smtpd_banner = $myhostname ESMTP $mail_name
biff = no
append_dot_mydomain = no
readme_directory = no
smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem
smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key
smtpd_use_tls=yes
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
myhostname = domain.goes.here
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
myorigin = /etc/mailname
mydestination = domains.go.here, example.org
relayhost =
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = all
default_transport = smtp
relay_transport = smtp
virtual_alias_maps = hash:/etc/postfix/virtual
inet_protocols = ipv4

And /etc/postfix/virtual:

@example.org change_to_real_email@example.com

Don’t forget to run postmap /etc/postfix/virtual; /etc/init.d/postfix reload after changing /etc/postfix/virtual and your configuration.

Tuesday, Dec 6: Launch project O. Wednesday, Dec 7: Launch project T. Thursday, Dec 8: Have insulation installed (W- taking care of coordinating with contractors). J-‘s with us too, so homework help will take up part of our evenings. There’s probably more, maybe, but we’ll see.

It’s a packed week, which means it’s even more important to slow down. Stress and hurry result in mistakes, which make things worse. Yes; slow down, write things out, follow the steps, and make time for notes, and I’ll make it through to the end of the week. Each minute will pass with or without me anyway, so I may as well make the most of it.

Writing is part of the way I keep sane. I notice that when I go for too long without writing, I feel a little fidgety, a bit unraveled, somewhat frayed around the edges. So, writing. There’s something about leaving myself these notes to mark the passage of time, punctuating the otherwise breathless stream with commas and periods and the occasional tangential paragraph.

Sleep. Sleep is important, too. I’ve been getting a decent amount of sleep.

Not stressing out is important. It’s particularly useful to remember that there isn’t that much more I can do this week to affect the way the next few days play out. We’ve prepared and prepared, and now we’ll see the results.

Oddly enough, thinking about the worst-case scenario helps. Even if both launches were toast, the contractors botched the insulation, and lots of other things happened, we’d adapt. Things would be okay.

Best case: what if things go without a hitch? Then, hypothetically looking back, I’d like to have worried just enough to improve my notes and productivity without worrying so much that the stress got in the way. Balance.

The more I get used to this, the easier it is to not get overwhelmed. =) It’s just one thing at a time. All I have to do is to do one thing well, and the next, and the next, without losing sight of my overarching priorities.