sacha chua living an awesome life

6079 comments
2357 subscribers
6213 on Twitter
Subscribe! Feed reader E-mail
« Older:
Newer: »

Growing as a developer: Automated tests

Posted on December 14th, 2011 by Sacha Chua

For this project, I put a lot of time into writing tests. Now they’re paying off. User acceptance testing and beta testing is going to be limited thanks to some real-world project constraints, so we’ll have to catch more bugs ourselves.

I want to get to 100% test coverage. That will go a long way to increasing our confidence in the results.

We had neglected our tests in the beginning, so the autogenerated tests fell out of sync. A few weeks ago, I put in around six hours to get everything back into shape.

Once that was sorted out, I looked for ways to encourage us to keep the tests working. After briefly experimenting with autotest, I settled for using git precommit hooks to force us to run tests before checking code in. Fortunately, the other developer agreed with this idea, so I set up the same script on her system.

Rspec was great for detailed tests, but I missed the expressiveness of Cucumber. After a few more hours of work, I got our Cucumber tests to work again. I wrote a number of tests from different users’ perspectives, then added Cucumber to the pre-commit hook.

The next step in our testing journey was to set up code coverage analysis. It took a little hacking to get rcov to play nicely with cucumber and rspec, but I got that to work too. Now we can get a sense of what still needs to be tested, and we have clear goals: 100% of code covered by tests, and 100% of user stories matched up with tests too.

On another project, I’ve been working on automated tests with Selenium. I hate letting the clients run into bugs, and I hate breaking things I thought I’d already fixed. I also hate clicking around and doing repetitive actions. As we run into things I’ve broken and as I change things, I write tests.

For me, test-driven development is incredibly motivating: the carrot of seeing tests pass and coverage stats go up, the stick of embarrassment when things break. =) I can take advantage of that energy and change my development habits.

My next development goals related to testing:

  • behaviour-driven development and business-user-compatible test cases
  • 100% code coverage from day one
  • integrating Selenium tests into our build scripts
  • Selenium web driver screenshots to assist in keeping user documentation up to date
  • writing automated tests for life – making progress!

More posts about: development, kaizen, work // 5 Comments »

Short URL: http://sachachua.com/blog/p/22639
  • http://blog.kennymeyer.net Kenny Meyer

    TDD is an awesome tool, and I’m happy to read that you’re considering TDD as helpful. At some places I worked TDD is no requirement, and in fact a lot them dedicated to software development don’t even know TDD or don’t consider this approach as important. That’s a shame.

    It’s really nice to hear the company you work actually allow you to use those tools, and they seem to even embrace them. That’s übercool. Really.

    I guess you are using testing tools in a Rails environment? There are so many of them around for Rails, that when it comes to choose the best tools, you could actually spend weeks of research. Personally, I also like Rspec, and not Cucumber so much, as rspec allows you write specifications in code, whereas in Cucumber writing specifications in human-language seems redundant to me.

    “Writing automated tests for life”… I bet I will enjoy your blog post about that topic in the future! Thanks for sharing your insights. Really.

  • http://sachachua.com Sacha Chua

    I love test-driven development, and hope all of my future projects involve automated tests one way or another. Not having tests is like not having version control – setting yourself up for a world of pain later! And if you’re going to write tests, you might as well have some tests up front so that you can understand the requirements and motivate yourself to check things off, right? =)

    I like both Rspec and Cucumber, even though Test::Unit and other tools are much more terse. I like Cucumber because my tests look like English, which makes them easier to write in the beginning. I like Rspec’s should-type matchers, which make it easier for me to talk through my tests. I typically start by writing high-level Cucumber tests with the @wip tag, then fleshing out additional functionality or deeper specs using Rspec.

    Automated tests for life – check out my first draft. I’m currently failing those tests: 2 overdue books (manga borrowed by J-); 36.8 hours of work in the last 7 days (last Friday was a short day for me, but I’m actually ahead of schedule this week; I should tweak the test so that it checks if I’m on track this week or over); an average of 9.04 hours of sleep over the last 7 days. Still, it’s good to have a standard, and maybe I’ll find a more visible way to hook the data into my life. My Selenium tests for checking credit card balances all pass, though, so that’s something! =)

    What kinds of tests would you like to have for your life?

  • http://www.trajano.net/ Archimedes Trajano

    Good job Sacha, keep it up.

    You have to know that TDD also has a dark side if done improperly and that is your application becomes less resilient to change because there is now an additional cost involved in maintaining the test scripts when there is a project change. Not really a big deal, but it needs to be costed as part of the delivery.

    Also note that once it leaves your team and gets transitioned to another group, unlike the application code, test code is rarely an important thing. Don’t get discouraged that the group flowing after you won’t maintain it. It’s their problem to deal with any issues anyway not yours.

    I have found that transitioning test maintenance skills isn’t easy, but in general transition projects are difficult, they can ask all the questions they want and you can prep everything you can think of but in the end scenario is likely (especially since you did TDD) is the people who got transitioned by the people you transitioned to will have to face the problem. (on the bright side at least it is one degree of separation from you).

  • http://blog.kennymeyer.net Kenny Meyer

    > Not having tests is like not having version control – setting yourself up for a world of pain later!
    *shudder* A project without VC… hard for me to imagine to work on such a project nowadays.

    > Automated tests for life – check out my first draft.
    Cucumber! That’s an awesome share! :)

    > What kinds of tests would you like to have for your life?
    - Quit smoking
    - Get 8 hours of sleep
    - Don’t spend money on “things nice to have”

    I will start with those, and see how I can make that a habit. :)

  • http://sachachua.com Sacha Chua

    Archie: Our high-level behavioral tests with Cucumber have been fairly forgiving of changes. I use I18n when defining steps, so we can change the wording of the interface without having to rejig our tests. If we change the way something is handled, I can change a few step definitions without rewriting lots of tests. If there are major spec changes (let’s totally change the workflow!), I think rejigging tests would be the least of our problems. ;)

    Kenny: It happens, even today! I was working on this PHP+JQuery real-time dashboard for a call center. Their server was Windows-based, so I ended up just copying files over, backing things up, and using a test directory for trying out changes. =( On another project of mine – Drupal this time – I only had FTP access to the server, so I had neither git push nor rsync even though I used git to manage my versions locally. I sorely missed my usual development workflow.

« Older:
Newer: »

Related posts

On This Day...

  • 2012: One to three, that’s all — One to three good pieces of work each day. That’s all I want to check off my list, and anything [...]
  • 2010: Android life so far — It’s been three weeks since I bought my Android phone, and I’m having lots of fun hacking it. Here’s what [...]
  • 2009: How I find and learn from mentors — Would you be willing to describe in more detail how those relationships came to be, and how you’ve gone about [...]
  • 2008: Cintiq 12WX, OpenSuse 11.0, and much pain and suffering — Let that be a lesson to me: Always blog solutions to problems. The cost of doing so is not just [...]
  • 2007: Tips for conference bloggers — Conference reports are a great way to help share knowledge and justify the expense of conference travel, but attendees are often [...]
  • 2006: Stuffing envelopes, writing cards… — <stretch> Halfway through my US/Canada 2006 letters. I’m limiting myself to 100 letters for now, although I *might* send more from the [...]
  • 2006: Learning from the best — I grew up with books and audiotapes of Tom Peters, Zig Ziglar, and Tom Hopkins. I can *still* hear Tom Hopkins [...]
  • 2006: Enthusiastic rapport with Emmanuel and Rob: movies that motivate — What a day, what a day! I went back to Second Cup for coffee with Emmanuel Lopez and Rob Schaumer at [...]
  • 2006: Kudos to Kevin Magee: sales and networking tips — I had a terrific conversation with Kevin Magee over coffee and chocolate chillers at Second Cup this morning. I met him very [...]
  • 2005: Missing — Finals were great. =) Now to figure out where the missing pieces are… I remember going back to my room this [...]
  • 2004: planner tweak: What am I supposed to be doing? — This snippet helps me keep track of what I’m supposed to be doing. I’ve bound it to F9 F9. Calling it [...]
  • 2004: PLUG Christmas Party — I would like to invite you to attend and celebrate with us in our upcoming Christmas party and General Assembly this [...]
  • 2004: Kanji sentence — 今年度はデジタル家電の需要の低迷などから１けたの伸びに減速する。 ことしどはデジタルかでんのじゅようのていめいなどからひとけたののびにげんそくする。 減速する。| [It] will slow down. 伸びに減速する。| Growth will slow down. １けたの伸びに減速する。| Growth will slow down to a 1-digit figure. 低迷などから１けたの伸びに減速する。| Because of sluggishness, [...]
  • 2003: chess.el — Another John Wiegley creation! Mumble mumble… ;)
  • 2003: Natural language processing — http://slashdot.org/article.pl?sid=03/12/11/2358209&mode=thread&tid=106&tid=126&tid=156&tid=185 From Slashdot: Zhang Le, a Chinese scientist working on Natural Language Processing has decided to pack the most important language analysis and [...]

Featured Categories

Like this? Subscribe using your feed reader or your e-mail client! Follow me on Twitter for quick updates, and add me on Google+. Contact me: I'd love to hear your thoughts, questions, and suggestions!


Copyright © 2001-2012 Sacha Chua (sacha@sachachua.com). Please feel free to reuse content under a Creative Commons Attribution Non-commercial Sharealike license unless otherwise noted.

Get the highlights as a PDF!

Stories from my Twenties: Highlights from a Decade of Blogging