Using Simpletest and spreadsheets to populate Drupal with data

One of the challenges of testing views or custom Drupal code is generating the right kind of data. Devel can generate hundreds of random nodes, but you might need more custom data than that. For example, on our project, we need to have test users, their content profiles, and nodes that follow a certain node reference structure. By creating a class that extends DrupalWebTestCase and provides convenience functions on top of drupalCreateNode, we can easily create test data as part of our test cases. Copying the code from drupalCreateUser and making our own version that uses roles and content profiles helps us set up the right users, too.

We wanted our tests and changes to use the same database tables used by the web interface, so we overrode the setUp methods to use the actual database. This not only makes the tests faster, it also makes them more useful for the web testing and demos.

Many of our test cases create the data they need. However, some test cases need even more complex structures that are similar from one test to another. Instead of creating and recreating them on each test, I’ve written another test case for populating the data. For example, PopulateTestUsers sets up about 30 users with different characteristics. I can then write other tests that assume PopulateTestUsers has been run and the sample users and nodes are available.

How do we generate the users and nodes without getting tangled in lots of PHP? Here’s a technique I picked up from Stuart Robertson, an IT architect with lots of good ideas. He fills in a spreadsheet with the values he wants test data to have. He then uses other columns to generate PHP that set the individual fields, and another column that generates PHP based on the generated PHP. For example, a formula to set a CCK value might look like this:

=IF(B3<>"",CONCATENATE("'field_widget_doohickey' => array(array('value' => '",B3,"')),"),"")

which turns a value of “foo” in B3 to

'field_widget_doohickey' => array(array('value' => 'foo'))

which is then something you can pass to the node creation function. To figure out the syntax for other node attributes, use var_dump or your favourite debugging tool to look at a node that has the information you want.

You might have the final generation like this:


where createWidget is a function you’ve defined to make things more readable. It would be a wrapper around drupalCreateNode that sets the type and does other things.

This spreadsheet makes it so much easier for us to work with our test data because we can refer to it to find test data matching criteria when designing our tests or trying things out using the web interface. Adding new test items is easy: just fill in the rows, copy the equations, and then copy the generated code and paste it into the test case.

Naming tip: Using a different naming convention makes it easy for me to use our custom-coded Drush testre command (run tests matching a regular expression) to run just the tests that populate data, or just the tests that assume data is there. Likewise, test cases that use SimpleTest’s web client ($this->get, etc.) have a different naming convention so that I can avoid running these slower tests when I just want to do a quick check.

Simpletest is a powerful tool. I’ve used it on every Drupal project I’ve worked on, even when I was the only one writing tests. Combined with a spreadsheet for generating structured test data, it’s a great help for development and demonstration, because we can set up or refresh complicated sets of users and nodes in little time. Well worth investing time to learn and use.

SCHEDULED: 2010-12-23 Thu 08:00

  • What a nice use case for spreadsheets!. I haven’t yet integrated SimpleTest to my dev workflow, but this one trick will certainly help me to have a nice setup for good testability.

    On a side note, spreadsheets are quite useful in many situations. I remember using a spreadsheet to transliterate allowed values for a field into safe machine names to compose the key|value pair (starting from the human-readable list); also, I once created a small tool to help me generate some long, repetitive html by using a visually nested structure and a basic parser that just did the trick.

  • Great idea! And how do you import the generated PHP from the spreadsheet into the test case?

  • infojunkie: That’s the easy part. Copy and paste. ;)

    Another discovery: writing custom spreadsheet functions for quoting, making lists, making arrays, and looking up node references makes it much easier to keep spreadsheet functions readable. ParamArray is the VBA way to handle a variable number of arguments.

  • Take a look at Sheetnode. Maybe you can move your spreadsheet-making to Drupal itself. It supports a lot of Excel functions, and you can write new functions using Javascript.