People found the 7-minute impromptu braindump I gave at last night’s Drupal Toronto meetup useful enough to vote it the best talk, so I thought I’d write it up. =)

Development

• Use source code control. No, seriously. Use it. Religiously. It’s going to save you someday. CVS is popular. Subversion is also easy to set up. It’s a must when you’re working with other people, but even if you’re on your own, it’s really handy to be able to go back to previous versions or to check your changes.
• Get to know your integrated development environment (IDE). For example, I use Eclipse + PHP Development Toolkit (PDT) + Xdebug, and the keyboard shortcuts save me so much time and make it easier for me to think about my code instead of thinking about editing my code. F3 to jump to a function definition, Ctrl-Shift-R to open a file by typing part of the filename, Ctrl-space to complete text… I’ve also gotten XDebug integration to work, so I can step through code and examine variables without putting var_dump everywhere. Know your tools.
• Add the XDebug extension to your PHP installation. Even if you don’t get XDebug integrated with Eclipse, you’ll like the new and improved var_dump and other debugging functions. I’ve heard good things about Zend Debugger, too.
• Put your entire Drupal directory into your project in the IDE, not just your site-specific code. You can even check your entire Drupal directory into your source code control repository. This makes it easier to look up functions in Drupal core, not just your sites/all/modules directory. You _could_ version-control just your site-specific code (profiles/, sites/, etc.), but set-up becomes a little more complicated.
• Use an installation profile to manage your configuration. Experiment with things using the administration interface, then create/update an installation profile that sets up a brand-new Drupal installation. This will save you lots of time and head-scratching when you need to reproduce your setup.
• Make starting from scratch easier, and do it often. I have a Makefile that drops my database and recreates it, resets my settings.php, and even opens up a Firefox browser that uses my installation profile to set everything up from scratch. Drupal 5: Copy your settings.php to settings-default.php or something like that before you install your system, then copy settings-default.php over settings.php when you want to refresh your setup. Drupal 6: Delete settings.php when you want to refresh your setup.

Firefox

• Use multiple profiles. You can start completely independent Firefox processes by using profiles. For example, you could have your default profile with all the bells and whistles, and a test profile with just FireBug. You can log in as different users, too. Use this command to start a new Firefox process: firefox -ProfileManager -no-remote
  
• Install and use FireBug. Best web development extension ever. I use this to examine the document object model, debug my CSS rules, experiment with my CSS and HTML on the fly, and monitor the network requests.
• Set up quick searches for drupal.org and php.net. You can define quick searches by creating bookmarks that use keywords and %s placeholders in Location:. Example: Do a quick Google-search of Drupal.org by adding a bookmark with Location: http://google.com/search?q=inurl:drupal.org+%s and Keyword: gd (short for Google Drupal). To use your new quick search, type Ctrl-L or Alt-D to get to the address bar, then type gd keywords. Set up ways to search php.net and drupal.org/project, too. =)

Multi-site / virtual hosts

• Simplify switching between your computer and your remote server. If you’ve set up your local system for virtual hosts and you have a remote server for testing, you’ll want to be able to switch between testing locally and testing remotely. Set up your local system to respond to all the domain names for the site (ex: example.com, foo.example.com, bar.example.com) and a local subdomain (ex: local.example.com). Set up your remote server to respond to all the domain names for the site (example.com, foo.example.com, bar.example.com) and a remote subdomain (ex: test.example.com). Then set up your /etc/hosts (c:\windows\system32\drivers\etc\hosts on Microsoft Windows) to look like this:

  127.0.0.1 localhost local.example.com
  remote.server.ip.address test.example.com
  
  127.0.0.1 example.com foo.example.com bar.example.com
  #remote.server.ip.address example.com foo.example.com bar.example.com
  

  local.example.com will always resolve to your computer, test.example.com will always resolve to the other computer, and everything else depends on which line is active. # comments out a line. When you want to switch to testing on the remote server, add # to the beginning of the line for 127.0.0.1 example.com… and remove # from the line for your remote server. If you want to automate this even further, write a script that swaps different /etc/hosts files around.

Hope that helps! I’d love to find out what you’ve done to tweak your environment. Feel free to share your tips here!

I’ve written about using drush to evaluate PHP statements in the Drupal context using the command line before, and it turns out that Drush is also quite useful for running Simpletest scripts. Drush comes with a module that allows you to display all the available tests with “drush test list”, run all the tests with “drush test run”, or run specified tests with “drush test run test1,test2”.

‘Course, I wanted to run groups of tests and tests matching regular expressions, so I defined two new commands:

drush test run re regular-expression

Run all tests matching a regular expression that uses ereg(..) to match.

Ex: drush test run re Example.*

drush test run group group1,group2…

Run all tests matching the given groups

Ex: drush test run group Example

Here’s the patch to make it happen:

Index: drush_simpletest.module
===================================================================
--- drush_simpletest.module	(revision 884)
+++ drush_simpletest.module	(working copy)
@@ -12,9 +12,13 @@
 function drush_simpletest_help($section) {
   switch ($section) {
       case 'drush:test run':
-        return t("Usage drush [options] test run.\n\nRun the specified specified unit tests. If  is omitted, all tests are run.  should be a list of classes separated by a comma. For example: PageCreationTest,PageViewTest.");
+        return t("Usage drush [options] test run .\n\nRun the specified unit tests. If  is omitted, all tests are run.  should be a list of classes separated by a comma. For example: PageCreationTest,PageViewTest.");
       case 'drush:test list':
         return t("Usage drush [options] test list.\n\nList the available tests. Use drush test run command to run them. ");
+      case 'drush:test group':
+        return t("Usage drush [options] test group .\n\nRun all unit tests in the specified groups. For example: drush test group Group1,Group2");
+      case 'drush:test re':
+        return t("Usage drush [options] test re .\n\nRun all unit tests matching this regular expression. For example: drush test re Page.*");
   }
 }
 
@@ -30,10 +34,18 @@
     'callback' => 'drush_test_list',
     'description' => 'List the available Simpletest test classes.',
   );
+  $items['test re'] = array(
+    'callback' => 'drush_test_re',
+    'description' => 'Run one or more Simpletest tests based on regular expressions.',
+  );
+  $items['test group'] = array(
+    'callback' => 'drush_test_group',
+    'description' => 'Run one or more Simpletest test groups.',
+  );
   return $items;
 }
 
-function drush_test_list() {
+function drush_test_get_list() {
   simpletest_load();
   // TODO: Refactor simpletest.module so we don't copy code from DrupalUnitTests
   $files = array();
@@ -60,6 +72,11 @@
       $rows[] = array($class, $info['name'], truncate_utf8($info['desc'], 30, TRUE, TRUE));
     }
   }
+  return $rows;
+}
+
+function drush_test_list() {
+  $rows = drush_test_get_list();
   return drush_print_table($rows, 0, TRUE);
 }
 
@@ -75,3 +92,31 @@
   }
   return $result;
 }
+
+function drush_test_re($expression) {
+  if (!$expression) {
+    die('You must specify a regular expression.');
+  }
+  $rows = drush_test_get_list();
+  $tests = array();
+  foreach ($rows as $row) {
+    if (ereg($expression, $row[0])) {
+      $tests[] = $row[0];
+    }
+  }
+  simpletest_run_tests($tests, 'text');
+  return $result;
+}
+
+function drush_test_group($groups) {
+  $rows = drush_test_get_list();
+  $tests = array();
+  $groups = explode(',', $groups);
+  foreach ($rows as $row) {
+    if (in_array($row[1], $groups)) {
+      $tests[] = $row[0];
+    }
+  }
+  simpletest_run_tests($tests, 'text');
+  return $result;
+}

That makes running tests so much easier and more fun!

To keep track of the bugfixes we’ll need to make for our next release, I’ve created a Subversion branch called branches/release-1. Development of new features will continue on trunk, but we’ll merge in the bugfixes from release-1 every so often.

There are three environments we deploy to:

Local

Developers should be able to easily test both versions on their local machines.

QA server

We should be able to deploy both versions to a publicly-accessible QA server for acceptance testing.

Production server

We should be able to deploy release-1 (and then later, release-2 and so on) to the production server, preferably after a lot of testing

Editorial changes happen on the production server, where our users update content. We would like to be able to take a snapshot of that database and use that to test our development code on the QA server or in our local development environments. Because we use Domain Access to serve multiple subdomains with shared content, it’s not just a matter of using mysqldump to back up the database and copy it over. We also need to replace URLs inside the database, and we need to override domain_root using the $conf array in settings.php.

I’m the only one running Linux, so the other developers don’t really benefit from the Makefiles I’ve defined or the tools I use. For the simpler build system we had before (all development on trunk), I wrote a deployment script that allowed users to:

• Download a stripped copy of the production database with the URLs changed for their local testing environment
• Deploy a stripped copy of the production database to the QA server
• Deploy a specified revision of the source code to the QA server
• Deploy a specified revision of the source code to the production server

The new deployment script needed to allow users to do the same, but for both branches of the code. Both branches of the code would be simultaneously available on the QA server, so the script would need to deploy the code to different directories.

After some fiddling around with the page design (because I care about making interfaces make sense!), I came up with something that looks like this:

-------
r988 | somegeek | 2008-09-15 13:00:00 -0500 (Mon, 15 Sep 2008) | 2 lines

Use dev.qa.example.com
------------
[more changelog entries go here]

------------------------------------------------------------------------
r986 | somegeek | 2008-09-15 11:03:38 -0500 (Mon, 15 Sep 2008) | 3 lines

Starting a branch for release-1
------------------------------------------------------------------------
[more changelog entries go here]

The deployment script allows the user to get a copy of the database, deploy a copy of the database, or deploy specific revisions of branches.

Because I was having a hard time figuring out how to do ssh key-based operations from Apache (which runs as a no-login user), I use two shell scripts to do the dirty work. One shell script connects to the production server, creates a partial backup, copies the information over, and does any necessary replacements. Another shell script takes a domain name and optionally a revision, and deploys the revision from the appropriate branch.

Here’s my totally small-scale PHP way to show the revisions log:

$dev_output = shell_exec("svn log $dev_url $details --limit 20");
$dev_revisions = preg_match_all('/r([0-9]+)/', $dev_output, $dev_matches);

where $dev_url is the URL of the trunk in Subversion, and $details contains the username and password specified as options for the Subversion command-line.

I’m going to see if I can get my regression tests running on the server that I’ve got my deployment script on. Wouldn’t that be awesome?

So now that we’re doing all our configuration changes in source code, it makes sense to automate database updates as much as I can. Here’s something I’ve added to drush_tools so that I can run all the schema changes from the command-line:

function drush_tools_update($command = '') {
  ob_start();
  require_once 'includes/install.inc';
  require_once 'update.php';
  $ret = ob_get_contents();
  drupal_load_updates();
  ob_end_clean();

  $list = module_list();
  $update_list = array();
  foreach ($list as $module) {
    $updates = drupal_get_schema_versions($module);
    if ($updates !== FALSE) {
      $latest = 0;
      $base = drupal_get_installed_schema_version($module);
      foreach ($updates as $update) {
        if ($update > $base) {
          if ($update > $latest) { $latest = $update; }
          $update_list[$module][] = $update;
        }
      }
      if ($latest) {
        sort($update_list[$module]);
        printf("%-30s %5d -> %5d (%s)\n", $module, $base, $latest, join(', ', $update_list[$module]));
      } else {
        printf("%-30s %5d\n", $module, $base);
      } 
    }      
  }
  if (count($update_list) == 0) return;
  if ($command != 'force' && !drush_confirm(t('Do you really want to continue?'))) {
    drush_die('Aborting.');
  }
  ob_start();
  foreach ($update_list as $module => $versions) {
    foreach ($versions as $v) {
      print "Running " . $module . "_update_" . $v . "\n";
      update_data($module, $v);
    }
  }
  $updates = ob_get_contents();

  cache_clear_all('*', 'cache', TRUE);
  cache_clear_all('*', 'cache_page', TRUE);
  cache_clear_all('*', 'cache_menu', TRUE);
  cache_clear_all('*', 'cache_filter', TRUE);
  drupal_clear_css_cache();
  ob_end_clean();
  $output = '';
  if (!empty($_SESSION['update_results'])) {
    $output .= "The following queries were executed:\n";
    foreach ($_SESSION['update_results'] as $module => $updates) {
      $output .= "\n" . $module . "\n--------------------------\n";
      foreach ($updates as $number => $queries) {
        $output .= 'Update #'. $number . ":\n";
        foreach ($queries as $query) {
          if ($query['success']) {
            $output .= "SUCCESS: " . $query['query'] . "\n";
          }
          else {
            $output .= "FAILURE: " . $query['query'] . "\n";
          }
        }
        if (!count($queries)) {
          $output .= "No queries\n";
        }
      }
    }
    $output .= "\n";
    print $output;
    unset($_SESSION['update_results']);
  }
}

• Avatar selection/upload during registration: Oddly enough, avatar_selection doesn’t handle this. You need the reg_with_pic module, too.
• Shared and per-site file directories: We’re segregating uploaded files by microsite, but we want to share some files as well. Hacking the shared path into file.inc is a kludge, but it works. Like so:

  --- file.inc    (revision 1046)
  +++ file.inc    (revision 1589)
  @@ -66,6 +66,11 @@
  else if (file_check_location($file_path . '/' . $dest, $file_path)) {
  return $file_path . '/' . $dest;
  }
  +  // Not found, check by going up one more
  +  else if (file_check_location($file_path . '/' . $dest, $file_path . '/../')) {
  +    return $file_path  . '/' . $dest;
  +  }
  +
  // File not found.
  return FALSE;
  }
  

• Group names with signed-up events: signup.module doesn’t make it easy to theme the list of events, so I created my own block based on the code in signup_list_user_signups. To determine the possible group a node came from, I check for the intersection between $node->og_groups and the array keys of $user->og_groups.
• Fixed panes: I couldn’t figure out a way to set a block as fixed in Panels 2, but a workaround is to define a custom layout that includes the block in the right place. Here’s an example:

  /**
   * Implements hook_panels_layouts
   */
  function mymodule_panels_layouts() {
    $items['my_panel'] = array(
      'title'  => t('My Panel Layout'),
      'icon'   => 'my_panel.png',
      'theme'  => 'my_panel',
      'css'    => 'my_panel.css',
      'panels' => array(
        'top' => t('Top'),
        'left' => t('Left'),
        'right' => t('Right'),
      ),
    )
    return $items
  }
  
  /**
   * This function uses heredoc notation to make it easier to convert
   * to a template.
   */
  function theme_my_panel($id, $content) {
    if ($id) {
      $idstr = " id='$id'"
    }
    // Pull in the Organic Groups - 0 block.
    $group_details = og_block('view', 0)
    $group_block = < <<EOT
    <div class="block block-og" id="block-og-0">               
      <h2 class="title">$group_details[subject]</h2>
      <div class="content">
      $group_details[content]
      </div>
      
  EOT
    $output = < <<EOT
  <div class="panel-2col-stacked clear-block panel-display" $idstr>
    <div class="panel-col-top panel-panel">
      <div class="inside">$content[top]</div>
    </div>
      <div class="center-wrapper">
      <div class="panel-col-first panel-panel">
      <div class="inside">
      $content[left]
      </div>
      </div>
      <div class="panel-col-last panel-panel">
      <div class="inside">
      $group_block
      $content[right]
      </div>
      </div>
    </div>
    <div class="panel-col-bottom panel-panel">
      <div class="inside">$content[bottom]</div>
    </div>
  EOT
    return $output;
  }
  

  To limit the choices to just that layout, put this in your installation profile:

    panels_load_include('common');
    $allowed_layouts = new panels_allowed_layouts();
    $allowed_layouts->allow_new = TRUE;
    $allowed_layouts->module_name = 'panels_common';
    $allowed_layouts->allowed_layout_settings = array(
      'threecol_33_34_33_stacked' => 0,
      'twocol' => 0,
      'flexible' => 0,
      'threecol_25_50_25' => 0,
      'threecol_25_50_25_stacked' => 0,
      'onecol' => 0,
      'threecol_33_34_33' => 0,
      'twocol_stacked' => 0,
      'twocol_bricks' => 0,
      'my_panel' => 1
    );
    $allowed_layouts->api_save();
  

• Organic Groups RSS feed: There’s an og_aggregator module, although it doesn’t quite fit in neatly with everything else.
• Organic Groups tag cloud: I was surprised to find that this wasn’t built in. I ended up defining my own block using a SQL query like this:

          if (($node = og_get_group_context()) && node_access('view', $node)) {
            $result = db_query(
              'SELECT COUNT(*) AS count, d.tid, d.name, d.vid FROM {term_data} d
               INNER JOIN {term_node} n ON (d.tid=n.tid)
               INNER JOIN {og_ancestry} og ON (og.nid=n.nid)
               WHERE og.group_nid=%d AND d.vid=0 GROUP BY d.tid, d.name, d.vid ORDER BY count DESC', $node->nid);
            $steps = 6;
            $tags = tagadelic_build_weighted_tags($result, $steps);
            $tags = tagadelic_sort_tags($tags);
            $output = theme('og_tagadelic_weighted', $node, $tags);
            $block['content'] = $output;
          }
  

  og_tagadelic_weighted was a custom theme function that constructed a link to a faceted_search using the og: and taxonomy: filters. I had to hack faceted_search, too. First I needed to enable the og filter, and then I needed to make taxonomy_facets.module recognize free tags (vid=0). Patch for taxonomy_facets.module:

  ===================================================================
  --- taxonomy_facets.module	(revision 1046)
  +++ taxonomy_facets.module	(revision 1582)
  @@ -147,6 +147,7 @@
         // and create facets from it.
         if ($found_text = search_query_extract($arg, 'taxonomy')) {
           $vocabularies = taxonomy_get_vocabularies();
  +        $vocabularies[0] = 1;
           // Extract separate facets
           $paths = explode(',', $found_text);
           foreach ($paths as $path_index => $tids) {
  

It’s a good thing I can find my way around sparsely-documented code with few or no examples… <laugh>

My session on Totally Rocking Your Development Environment has been accepted for DrupalCon 2009, hooray! Thanks!

This is great! And handy, because I’ve already promised to give an IBM-flavored version of the talk at the first community call for the newly-formed (or -revived, not sure) IBM Drupal community, which means I will have to have it all ready to go by two weeks from now instead of two months.

Two weeks from now is also when I’ll be giving a lecture on Enterprise 2.0 and knowledge management to Dorit Nevo’s MBA class at Schulich.

And I’ve volunteered to help organize or otherwise make these upcoming conferences awesomer: IBM Web 2.0 Summit, DrupalCampTO, Mesh.

And there’s LifeCampTO.

Busy, busy, busy.

Fortunately, talks are so much more fun to prepare when you think of them as learning opportunities. And I’ve volunteered to help conferences out with either things I know how to do well (say hi to people at registration desks, etc.) or that I’m interested in transforming/scaling (abstract submission, voting, schedules) or that I’m interested in learning (selling sponsorship, buying merchandise). And the conferences are a bit further out.

But “slew” is such a good word, because if I’m not careful and if I don’t intentionally slow down as I get into the busy-busy-busy times, then another sense (slew: past tense of slay) may figuratively kick in. That wouldn’t be fun at all.

It’ll all be great fun, though, and I’m sure I’ll learn tons! You’ll hear about all of it here, of course.

So if I’m slow at e-mail, you know why. =)