Running groups of Drupal tests from the command line

I’ve written about using drush to evaluate PHP statements in the Drupal context using the command line before, and it turns out that Drush is also quite useful for running Simpletest scripts. Drush comes with a module that allows you to display all the available tests with “drush test list”, run all the tests with “drush test run”, or run specified tests with “drush test run test1,test2″.

‘Course, I wanted to run groups of tests and tests matching regular expressions, so I defined two new commands:

drush test run re regular-expression
Run all tests matching a regular expression that uses ereg(..) to match.
Ex: drush test run re Example.*
drush test run group group1,group2…
Run all tests matching the given groups
Ex: drush test run group Example

Here’s the patch to make it happen:

Index: drush_simpletest.module
===================================================================
--- drush_simpletest.module	(revision 884)
+++ drush_simpletest.module	(working copy)
@@ -12,9 +12,13 @@
 function drush_simpletest_help($section) {
   switch ($section) {
       case 'drush:test run':
-        return t("Usage drush [options] test run.\n\nRun the specified specified unit tests. If  is omitted, all tests are run.  should be a list of classes separated by a comma. For example: PageCreationTest,PageViewTest.");
+        return t("Usage drush [options] test run .\n\nRun the specified unit tests. If  is omitted, all tests are run.  should be a list of classes separated by a comma. For example: PageCreationTest,PageViewTest.");
       case 'drush:test list':
         return t("Usage drush [options] test list.\n\nList the available tests. Use drush test run command to run them. ");
+      case 'drush:test group':
+        return t("Usage drush [options] test group .\n\nRun all unit tests in the specified groups. For example: drush test group Group1,Group2");
+      case 'drush:test re':
+        return t("Usage drush [options] test re .\n\nRun all unit tests matching this regular expression. For example: drush test re Page.*");
   }
 }
 
@@ -30,10 +34,18 @@
     'callback' => 'drush_test_list',
     'description' => 'List the available Simpletest test classes.',
   );
+  $items['test re'] = array(
+    'callback' => 'drush_test_re',
+    'description' => 'Run one or more Simpletest tests based on regular expressions.',
+  );
+  $items['test group'] = array(
+    'callback' => 'drush_test_group',
+    'description' => 'Run one or more Simpletest test groups.',
+  );
   return $items;
 }
 
-function drush_test_list() {
+function drush_test_get_list() {
   simpletest_load();
   // TODO: Refactor simpletest.module so we don't copy code from DrupalUnitTests
   $files = array();
@@ -60,6 +72,11 @@
       $rows[] = array($class, $info['name'], truncate_utf8($info['desc'], 30, TRUE, TRUE));
     }
   }
+  return $rows;
+}
+
+function drush_test_list() {
+  $rows = drush_test_get_list();
   return drush_print_table($rows, 0, TRUE);
 }
 
@@ -75,3 +92,31 @@
   }
   return $result;
 }
+
+function drush_test_re($expression) {
+  if (!$expression) {
+    die('You must specify a regular expression.');
+  }
+  $rows = drush_test_get_list();
+  $tests = array();
+  foreach ($rows as $row) {
+    if (ereg($expression, $row[0])) {
+      $tests[] = $row[0];
+    }
+  }
+  simpletest_run_tests($tests, 'text');
+  return $result;
+}
+
+function drush_test_group($groups) {
+  $rows = drush_test_get_list();
+  $tests = array();
+  $groups = explode(',', $groups);
+  foreach ($rows as $row) {
+    if (in_array($row[1], $groups)) {
+      $tests[] = $row[0];
+    }
+  }
+  simpletest_run_tests($tests, 'text');
+  return $result;
+}

That makes running tests so much easier and more fun!

Using Simpletest and spreadsheets to populate Drupal with data

One of the challenges of testing views or custom Drupal code is generating the right kind of data. Devel can generate hundreds of random nodes, but you might need more custom data than that. For example, on our project, we need to have test users, their content profiles, and nodes that follow a certain node reference structure. By creating a class that extends DrupalWebTestCase and provides convenience functions on top of drupalCreateNode, we can easily create test data as part of our test cases. Copying the code from drupalCreateUser and making our own version that uses roles and content profiles helps us set up the right users, too.

We wanted our tests and changes to use the same database tables used by the web interface, so we overrode the setUp methods to use the actual database. This not only makes the tests faster, it also makes them more useful for the web testing and demos.

Many of our test cases create the data they need. However, some test cases need even more complex structures that are similar from one test to another. Instead of creating and recreating them on each test, I’ve written another test case for populating the data. For example, PopulateTestUsers sets up about 30 users with different characteristics. I can then write other tests that assume PopulateTestUsers has been run and the sample users and nodes are available.

How do we generate the users and nodes without getting tangled in lots of PHP? Here’s a technique I picked up from Stuart Robertson, an IT architect with lots of good ideas. He fills in a spreadsheet with the values he wants test data to have. He then uses other columns to generate PHP that set the individual fields, and another column that generates PHP based on the generated PHP. For example, a formula to set a CCK value might look like this:

=IF(B3<>"",CONCATENATE("'field_widget_doohickey' => array(array('value' => '",B3,"')),"),"")

which turns a value of “foo” in B3 to

'field_widget_doohickey' => array(array('value' => 'foo'))

which is then something you can pass to the node creation function. To figure out the syntax for other node attributes, use var_dump or your favourite debugging tool to look at a node that has the information you want.

You might have the final generation like this:

=CONCATENATE("$this->createWidget('",A3,"',array(",E3,"));")

where createWidget is a function you’ve defined to make things more readable. It would be a wrapper around drupalCreateNode that sets the type and does other things.

This spreadsheet makes it so much easier for us to work with our test data because we can refer to it to find test data matching criteria when designing our tests or trying things out using the web interface. Adding new test items is easy: just fill in the rows, copy the equations, and then copy the generated code and paste it into the test case.

Naming tip: Using a different naming convention makes it easy for me to use our custom-coded Drush testre command (run tests matching a regular expression) to run just the tests that populate data, or just the tests that assume data is there. Likewise, test cases that use SimpleTest’s web client ($this->get, etc.) have a different naming convention so that I can avoid running these slower tests when I just want to do a quick check.

Simpletest is a powerful tool. I’ve used it on every Drupal project I’ve worked on, even when I was the only one writing tests. Combined with a spreadsheet for generating structured test data, it’s a great help for development and demonstration, because we can set up or refresh complicated sets of users and nodes in little time. Well worth investing time to learn and use.

SCHEDULED: 2010-12-23 Thu 08:00