Drupal, SimpleTest, and the node access API

Setting up Simpletest and Drush on Drupal 6.x:

  1. Download and enable Simpletest with drush dl simpletest; drush en -y simpletest
  2. Download simpletest.drush.inc to your ~/.drush/drush_extras directory. This version allows you to run a single test from the command-line.
  3. Create a custom module with a tests/ subdirectory, and write your tests in it. (See this Lullabot Simpletest tutorial.)

We’re starting another Drupal project. While the IT architect is working on clarifying the requirements, I volunteered to implement the risky parts so that we could get a better sense of what we needed to do.

The first major chunk of risk was fine-grained access control. Some users needed to be able to edit the nodes associated with other users, and some users needed to have partial access to nodes depending on how they were referenced by the node. Because there were many cases, I decided to start by writing unit tests.

SimpleTest was not as straightforward in Drupal 6.x as it was in Drupal 5.x. There were a few things that confused me before I figured things out.

I wondered why my queries were running off different table prefixes. I didn’t have some of the data I expected to have. It turns out that Simpletest now works on a separate Drupal instance by default, using a unique table prefix so that it doesn’t mess around with your regular database. I’m doing this on a test server and I want to be able to easily look up details using SQL, so I needed to add this to my test case:

class ExampleTestCase extends DrupalWebTestCase {
  function setUp() {
    global $base_url;
    $this->originalPrefix = $GLOBALS['db_prefix'];
  }
  function tearDown() { }
}

I also didn’t like how the built-in $this->drupalCreateUser took permissions instead of roles, and how it created custom roles each time. I created a function that looked up the role IDs using the {role} table, then added the role IDs and roles to the $edit['roles'] array before creating the user.

Lastly, I needed to add the Content Profile operations to my custom user creation function. I based this code on content_profile.test.

$this->drupalLogin($account);
// create a content_profile node
$edit = array(
  'title' => $account->name,
  'body'  => $this->randomName(),
);
$this->drupalGet('node/add');
$this->drupalPost('node/add/' . str_replace(' ', '-', $role), $edit, t('Save'));

It would’ve been even better to do this without going through the web interface, but it was fine for a quick hack.

I had the setup I wanted for writing test cases that checked user permissions. I wrote functions for checking if the user could accept an invitation (must be invited, must not already have accepted, and must be able to fit). SimpleTest made it easy to test each of the functions, allowing me to build and test blocks that I could then put together.

The code in content_permission.module turned out to be a good starting point for my field-level permissions, while the Drupal node access API made it easy to handle the user-association-related permissions even though I used node references instead of user references.

It was a good day of hacking. I wrote tests, then I wrote code, then I argued with the computer until my tests passed. ;) It was fun seeing my progress and knowing I wasn’t screwing up things I’d already solved.

If you’re writing Drupal code, I strongly recommend giving SimpleTest a try. Implementing hook_node_access_records and hook_node_grants is much easier when you can write a test to make sure the right records are showing up. (With the occasional use of node_access_acquire_grants to recalculate…) Otherwise-invisible Drupal code becomes easy to verify. The time you invest into writing tests will pay off throughout the project, and during future work as well. Have fun!

Drupal Features and Drush: updating our development workflow

I’m working with two other people on a Drupal project, so we’re coordinating our work through a Subversion source code repository. A lot has changed in Drupal since the days when I dived into the source code to figure out the code I needed in order to duplicate the configuration changes I made through the web interface (Drupal staging and deployment: it’s all code). Now, the Features module can export various configuration bits as a module that you can check into your source tree and enable on your site. It will even show you which settings you’ve overridden through the web interface, so you can regenerate the code and make sure everything’s included.

Drush (the Drupal shell) has some commands that make Drupal features even easier to use. For example, I use drush features-diff <feature_name> to see which settings I’ve changed, and drush features-update to re-export the settings to source code.

Because we’ll be using Features to share our changes instead of working off SQL backups, I need to make sure that I’ve included all the relevant components in the features I create. One way to test that is to use Backup and Migrate to save my configured database (just in case!), load a previous backup, enable the feature, and confirm that everything works as expected.

Tests using either SimpleTest or Selenium would be the best way to confirm everything is working, of course. When Stuart comes back on Monday, he can help us set up an environment using hudson as a continuous integration server. Stuart has set up Selenium tests before, and it might be possible to use simpletest with hudson also.

Drupal and Drush: Updating the database from the command-line

So now that we’re doing all our configuration changes in source code, it makes sense to automate database updates as much as I can. Here’s something I’ve added to drush_tools so that I can run all the schema changes from the command-line:

function drush_tools_update($command = '') {
  ob_start();
  require_once 'includes/install.inc';
  require_once 'update.php';
  $ret = ob_get_contents();
  drupal_load_updates();
  ob_end_clean();

  $list = module_list();
  $update_list = array();
  foreach ($list as $module) {
    $updates = drupal_get_schema_versions($module);
    if ($updates !== FALSE) {
      $latest = 0;
      $base = drupal_get_installed_schema_version($module);
      foreach ($updates as $update) {
        if ($update > $base) {
          if ($update > $latest) { $latest = $update; }
          $update_list[$module][] = $update;
        }
      }
      if ($latest) {
        sort($update_list[$module]);
        printf("%-30s %5d -> %5d (%s)\n", $module, $base, $latest, join(', ', $update_list[$module]));
      } else {
        printf("%-30s %5d\n", $module, $base);
      } 
    }      
  }
  if (count($update_list) == 0) return;
  if ($command != 'force' && !drush_confirm(t('Do you really want to continue?'))) {
    drush_die('Aborting.');
  }
  ob_start();
  foreach ($update_list as $module => $versions) {
    foreach ($versions as $v) {
      print "Running " . $module . "_update_" . $v . "\n";
      update_data($module, $v);
    }
  }
  $updates = ob_get_contents();

  cache_clear_all('*', 'cache', TRUE);
  cache_clear_all('*', 'cache_page', TRUE);
  cache_clear_all('*', 'cache_menu', TRUE);
  cache_clear_all('*', 'cache_filter', TRUE);
  drupal_clear_css_cache();
  ob_end_clean();
  $output = '';
  if (!empty($_SESSION['update_results'])) {
    $output .= "The following queries were executed:\n";
    foreach ($_SESSION['update_results'] as $module => $updates) {
      $output .= "\n" . $module . "\n--------------------------\n";
      foreach ($updates as $number => $queries) {
        $output .= 'Update #'. $number . ":\n";
        foreach ($queries as $query) {
          if ($query['success']) {
            $output .= "SUCCESS: " . $query['query'] . "\n";
          }
          else {
            $output .= "FAILURE: " . $query['query'] . "\n";
          }
        }
        if (!count($queries)) {
          $output .= "No queries\n";
        }
      }
    }
    $output .= "\n";
    print $output;
    unset($_SESSION['update_results']);
  }
}

Running groups of Drupal tests from the command line

I’ve written about using drush to evaluate PHP statements in the Drupal context using the command line before, and it turns out that Drush is also quite useful for running Simpletest scripts. Drush comes with a module that allows you to display all the available tests with “drush test list”, run all the tests with “drush test run”, or run specified tests with “drush test run test1,test2″.

‘Course, I wanted to run groups of tests and tests matching regular expressions, so I defined two new commands:

drush test run re regular-expression
Run all tests matching a regular expression that uses ereg(..) to match.
Ex: drush test run re Example.*
drush test run group group1,group2…
Run all tests matching the given groups
Ex: drush test run group Example

Here’s the patch to make it happen:

Index: drush_simpletest.module
===================================================================
--- drush_simpletest.module	(revision 884)
+++ drush_simpletest.module	(working copy)
@@ -12,9 +12,13 @@
 function drush_simpletest_help($section) {
   switch ($section) {
       case 'drush:test run':
-        return t("Usage drush [options] test run.\n\nRun the specified specified unit tests. If  is omitted, all tests are run.  should be a list of classes separated by a comma. For example: PageCreationTest,PageViewTest.");
+        return t("Usage drush [options] test run .\n\nRun the specified unit tests. If  is omitted, all tests are run.  should be a list of classes separated by a comma. For example: PageCreationTest,PageViewTest.");
       case 'drush:test list':
         return t("Usage drush [options] test list.\n\nList the available tests. Use drush test run command to run them. ");
+      case 'drush:test group':
+        return t("Usage drush [options] test group .\n\nRun all unit tests in the specified groups. For example: drush test group Group1,Group2");
+      case 'drush:test re':
+        return t("Usage drush [options] test re .\n\nRun all unit tests matching this regular expression. For example: drush test re Page.*");
   }
 }
 
@@ -30,10 +34,18 @@
     'callback' => 'drush_test_list',
     'description' => 'List the available Simpletest test classes.',
   );
+  $items['test re'] = array(
+    'callback' => 'drush_test_re',
+    'description' => 'Run one or more Simpletest tests based on regular expressions.',
+  );
+  $items['test group'] = array(
+    'callback' => 'drush_test_group',
+    'description' => 'Run one or more Simpletest test groups.',
+  );
   return $items;
 }
 
-function drush_test_list() {
+function drush_test_get_list() {
   simpletest_load();
   // TODO: Refactor simpletest.module so we don't copy code from DrupalUnitTests
   $files = array();
@@ -60,6 +72,11 @@
       $rows[] = array($class, $info['name'], truncate_utf8($info['desc'], 30, TRUE, TRUE));
     }
   }
+  return $rows;
+}
+
+function drush_test_list() {
+  $rows = drush_test_get_list();
   return drush_print_table($rows, 0, TRUE);
 }
 
@@ -75,3 +92,31 @@
   }
   return $result;
 }
+
+function drush_test_re($expression) {
+  if (!$expression) {
+    die('You must specify a regular expression.');
+  }
+  $rows = drush_test_get_list();
+  $tests = array();
+  foreach ($rows as $row) {
+    if (ereg($expression, $row[0])) {
+      $tests[] = $row[0];
+    }
+  }
+  simpletest_run_tests($tests, 'text');
+  return $result;
+}
+
+function drush_test_group($groups) {
+  $rows = drush_test_get_list();
+  $tests = array();
+  $groups = explode(',', $groups);
+  foreach ($rows as $row) {
+    if (in_array($row[1], $groups)) {
+      $tests[] = $row[0];
+    }
+  }
+  simpletest_run_tests($tests, 'text');
+  return $result;
+}

That makes running tests so much easier and more fun!

Drupal shell: quickly evaluating PHP statements in a Drupal context

I often find myself needing to variable_set something temporarily, just to try things out. The drush module provides a command-line interface that solves this problem with a little more hacking. To minimize the effect on my source tree, I’ve unpacked it into the sites directory for my local installation and enabled it in my test database. After I enabled the main drush module and the related modules, I tweaked drush_tools to include an insecure-but-useful eval command. Here it is:

--- sites/local.example.com/modules/drush/drush_tools.module.orig    2008-08-05 17:18:48.000000000 -0400
+++ sites/local.example.com/modules/drush/drush_tools.module    2008-08-05 17:18:55.000000000 -0400
@@ -46,6 +46,10 @@
     'callback' => 'drush_tools_sync',
     'description' => 'Rsync the Drupal tree to/from another server using ssh'
   );
+  $items['eval'] = array(
+    'callback' => 'drush_tools_eval',
+    'description' => 'Evaluate a command',
+  );
   return $items;
 }
 
@@ -156,3 +160,6 @@
   }
 }
 
+function drush_tools_eval($command) {
+  eval($command);
+}

I also added an alias to my ~/.bashrc along the lines of:

alias drush='php ~/drupal/sites/local.example.com/modules/drush/drush.php -r ~/drupal -l http://local.example.com'

where ~/drupal is my multisite Drupal directory root.

After I loaded the alias with “source ~/.bashrc”, I can now execute PHP statements in my Drupal context with commands like:

drush eval "variable_set('hello', 'world');"

Good stuff!