I’m starting the second phase of a Drupal development project, which means I get to write about all sorts of geeky things again. Hooray! So I’m investing some time into improving my environment set-up, and taking notes along the way.

This time, I’m going to try developing code in Eclipse instead of Emacs, although I’ll dip into Emacs occasionally if I need to do anything involving keyboard macros or custom automation. Setting up a good Eclipse environment will help me use XDebug for line-by-line debugging. var_dump> can only take me so far, and I still haven’t figured out how to properly use XDebug under Emacs. Configuring Eclipse will also help me help my coworkers, who tend to not be big Emacs fans. (Sigh.)

So here’s my current setup:

• A Linux server environment in VMWare, so that I can use all the Unix tools I like and so that I don’t have to fuss about with a WAMP stack
• Samba for sharing the source code between the Linux VM image and my Microsoft Windows laptop
• XDebug for debugging
• Eclipse and PDT for development

I like this because it allows me to edit files in Microsoft Windows or in Linux, and I can use step-by-step debugging instead of relying on var_dump.

Setting up Samba

Samba allows you to share folders on the network. Edit your smb.conf (mine’s in /etc/samba/) and uncomment/edit the following lines:

security = user

…

[homes]
   comment = Home Directories
   browseable = no
   read only = no
   valid users = %S

You may also need to use smbpasswd to set the user’s password.

Xdebug

Install php5-xdebug or whatever the Xdebug package is for PHP on your system. Edit xdebug.ini (mine’s in /etc/php5/conf.d) and add the following lines to the end:

[Xdebug]
xdebug.remote_enable=on
xdebug.remote_port=9000
xdebug.remote_handler=dbgp
xdebug.remote_autostart=1
xdebug.remote_connect_back=1

Warning: this allows debugging access from any computer that connects to it. Use this only on your development image. If you want to limit debugging access to a specific computer, remove the line that refers to remote_connect_back and replace it with this:

xdebug.remote_host=YOUR.IP.ADDRESS.HERE

Eclipse and PDT

I downloaded the all-in-one PHP Development Toolkit (PDT) from http://www.eclipse.org/pdt/, unpacked it, and imported my project. After struggling with Javascript and HTML validation, I ended up disabling most of those warnings. Then I set up a debug configuration that used Xdebug and the server in the VM image, and voila! Line by line debugging with the ability to look in variables. Hooray!

2011-05-31 Tue 17:37

The Drupal Calendar module is great. You can show view results in a decent-looking calendar easily. However, if you want to show more than what's provided out of the box, you need to do a bit of undocumented and confusing hacking.

Let's say that instead of displaying the items, you want to display the number of items. You need to implement hook_calendar_add_items, which, despite its name, actually overrides the items displayed. Your hook_calendar_add_items($view) should return an array of an array of items, which is also confusing, because the receiving function (template_preprocess_calendar in calendar/theme/theme.inc) doesn't actually do anything with the outermost array, or indeed with multiple entries – it simply takes the last array and sets the items to it.

Each item should be something like this:

$item = new stdClass();
$item->node_title = t('Hello world');
$item->calendar_start_date = $date;
$item->calendar_end_date = $date;
$items[] = $item;

and at the end, you do this:

return array($items);

Your hook_calendar_add_items gets the $view, but not the existing items, so you'll have to recalculate things using $view->result. Remember: if you return anything, your results will replace the items instead of being added to them.

Because hook_calendar_add_items doesn't actually do what it advertises, there are probably bugs in this area of code, and this behaviour might change in future releases of Calendar. Be careful when using it. However, it seems to be the easiest way to change Calendar behavior. Caveat developer.

Also useful: http://drupal.org/node/412418

2010-12-24 Fri 10:19

UPDATE: Fixed formatting. Thanks, Brock!

Two developers recently joined our team. Johnny has worked with Drupal before, and needs a little help getting used to Drupal 6 and Views 2. Elena is an IT architect who is new to both IBM and Drupal. She needs a lot more help getting started, because she doesn't know what things are called yet and she isn't yet accustomed to the Drupal way of doing things. For my part, I work on Workflow, node access, and other requirements that require deep Drupal hacking.

I'm learning to check on Elena more frequently and to help her break down tasks. Otherwise, she might get lost or stuck, because she might not yet know where things are or whether she's getting closer to an answer. I've made good progress on the things we've planned for this iteration, and I can invest the time into helping our new team members be more productive and learn more effectively.

Both Elena and Johnny have set up their debuggers in Eclipse, so they don't have to figure out the right places to insert var_dumps. Instead, they can trace through the relevant pieces of code, learning more about the structures and the flow of Drupal websites along the way.

Although I occasionally struggle to explain things I take for granted, I enjoy helping someone who's new to an area. It helps me remember the things people need to learn. For example, Elena's work on surveys requires her to learn about nodes, getting values from the $_REQUEST, loading nodes, working with CCK, altering forms, adding new form fields using the Form API, and using Drupal functions for links and text. We broke down the task into the following steps:

1. Create a CCK node type.
2. Use hook_form_alter to add some text to the form.
3. Load a node and fill the information in the form.
4. Get the extra node ID from the URL.
5. Adapt form_alter for the case where you're editing the node.

We've managed our planning well, so I don't feel overcommitted or stretched with the additional mentoring I've taken on. The time is an investment that will pay off both in the short-term as well as the long-term. If I can slow down and write more, then the investment can benefit to other people too.

I like this. It's certainly much better than leaving developers to flounder and work things out on their own, and I learn a lot in the process of helping. Maybe that will be one of my specialties: projects where other people are learning a lot on the fly.

One of the challenges of testing views or custom Drupal code is generating the right kind of data. Devel can generate hundreds of random nodes, but you might need more custom data than that. For example, on our project, we need to have test users, their content profiles, and nodes that follow a certain node reference structure. By creating a class that extends DrupalWebTestCase and provides convenience functions on top of drupalCreateNode, we can easily create test data as part of our test cases. Copying the code from drupalCreateUser and making our own version that uses roles and content profiles helps us set up the right users, too.

We wanted our tests and changes to use the same database tables used by the web interface, so we overrode the setUp methods to use the actual database. This not only makes the tests faster, it also makes them more useful for the web testing and demos.

Many of our test cases create the data they need. However, some test cases need even more complex structures that are similar from one test to another. Instead of creating and recreating them on each test, I've written another test case for populating the data. For example, PopulateTestUsers sets up about 30 users with different characteristics. I can then write other tests that assume PopulateTestUsers has been run and the sample users and nodes are available.

How do we generate the users and nodes without getting tangled in lots of PHP? Here's a technique I picked up from Stuart Robertson, an IT architect with lots of good ideas. He fills in a spreadsheet with the values he wants test data to have. He then uses other columns to generate PHP that set the individual fields, and another column that generates PHP based on the generated PHP. For example, a formula to set a CCK value might look like this:

=IF(B3<>"",CONCATENATE("'field_widget_doohickey' => array(array('value' => '",B3,"')),"),"")

which turns a value of “foo” in B3 to

'field_widget_doohickey' => array(array('value' => 'foo'))

which is then something you can pass to the node creation function. To figure out the syntax for other node attributes, use var_dump or your favourite debugging tool to look at a node that has the information you want.

You might have the final generation like this:

=CONCATENATE("$this->createWidget('",A3,"',array(",E3,"));")

where createWidget is a function you've defined to make things more readable. It would be a wrapper around drupalCreateNode that sets the type and does other things.

This spreadsheet makes it so much easier for us to work with our test data because we can refer to it to find test data matching criteria when designing our tests or trying things out using the web interface. Adding new test items is easy: just fill in the rows, copy the equations, and then copy the generated code and paste it into the test case.

Naming tip: Using a different naming convention makes it easy for me to use our custom-coded Drush testre command (run tests matching a regular expression) to run just the tests that populate data, or just the tests that assume data is there. Likewise, test cases that use SimpleTest's web client ($this->get, etc.) have a different naming convention so that I can avoid running these slower tests when I just want to do a quick check.

Simpletest is a powerful tool. I've used it on every Drupal project I've worked on, even when I was the only one writing tests. Combined with a spreadsheet for generating structured test data, it's a great help for development and demonstration, because we can set up or refresh complicated sets of users and nodes in little time. Well worth investing time to learn and use.

SCHEDULED: 2010-12-23 Thu 08:00

Me: Happiness is a test suite that passes.
@philiph: Do you practice test-driven development for your happiness?
Me: Why, yes, actually, I do. It gives me a tangible sense of accomplishment and minimizes my mouse-clicking. =)

Developers find their own balance of how much project structure works with them. Some people like seat-of-their-pants coding. Others want detailed architecture diagrams. Me, I’m getting the hang of agile development practices, and I really enjoy using them.

Test-driven development, for example. Yes, I could just plunge ahead and write Drupal code, and I could test it by clicking on buttons and typing into forms. I don’t particularly like using the mouse or doing repetitive actions, so I write tests for functionality and occasionally for web interaction. Tests also mean that  I can check small pieces of functionality before I have to build a web interface. And when something breaks – not if, but when – tests help me narrow down the error.

It’s so satisfying to see the tests pass, too.

There are tests that exercise functionality, and tests that set up test data just the way we like it so that we can demonstrate features or try things out using the web interface. One of my team members showed me a wonderful technique for repeatable, well-structured test data by using a spreadsheet to generate PHP code. I’ve been extending the pattern for other things.

Drupal + Simpletest is awesome. It can’t handle everything, but it makes my Drupal life better. Happy developers write happy code!

Setting up Simpletest and Drush on Drupal 6.x:

1. Download and enable Simpletest with drush dl simpletest; drush en -y simpletest
2. Download simpletest.drush.inc to your ~/.drush/drush_extras directory. This version allows you to run a single test from the command-line.
3. Create a custom module with a tests/ subdirectory, and write your tests in it. (See this Lullabot Simpletest tutorial.)

We're starting another Drupal project. While the IT architect is working on clarifying the requirements, I volunteered to implement the risky parts so that we could get a better sense of what we needed to do.

The first major chunk of risk was fine-grained access control. Some users needed to be able to edit the nodes associated with other users, and some users needed to have partial access to nodes depending on how they were referenced by the node. Because there were many cases, I decided to start by writing unit tests.

SimpleTest was not as straightforward in Drupal 6.x as it was in Drupal 5.x. There were a few things that confused me before I figured things out.

I wondered why my queries were running off different table prefixes. I didn't have some of the data I expected to have. It turns out that Simpletest now works on a separate Drupal instance by default, using a unique table prefix so that it doesn't mess around with your regular database. I'm doing this on a test server and I want to be able to easily look up details using SQL, so I needed to add this to my test case:

class ExampleTestCase extends DrupalWebTestCase {
  function setUp() {
    global $base_url;
    $this->originalPrefix = $GLOBALS['db_prefix'];
  }
  function tearDown() { }
}

I also didn't like how the built-in $this->drupalCreateUser took permissions instead of roles, and how it created custom roles each time. I created a function that looked up the role IDs using the {role} table, then added the role IDs and roles to the $edit['roles'] array before creating the user.

Lastly, I needed to add the Content Profile operations to my custom user creation function. I based this code on content_profile.test.

$this->drupalLogin($account);
// create a content_profile node
$edit = array(
  'title' => $account->name,
  'body'  => $this->randomName(),
);
$this->drupalGet('node/add');
$this->drupalPost('node/add/' . str_replace(' ', '-', $role), $edit, t('Save'));

It would've been even better to do this without going through the web interface, but it was fine for a quick hack.

I had the setup I wanted for writing test cases that checked user permissions. I wrote functions for checking if the user could accept an invitation (must be invited, must not already have accepted, and must be able to fit). SimpleTest made it easy to test each of the functions, allowing me to build and test blocks that I could then put together.

The code in content_permission.module turned out to be a good starting point for my field-level permissions, while the Drupal node access API made it easy to handle the user-association-related permissions even though I used node references instead of user references.

It was a good day of hacking. I wrote tests, then I wrote code, then I argued with the computer until my tests passed. ;) It was fun seeing my progress and knowing I wasn't screwing up things I'd already solved.

If you're writing Drupal code, I strongly recommend giving SimpleTest a try. Implementing hook_node_access_records and hook_node_grants is much easier when you can write a test to make sure the right records are showing up. (With the occasional use of node_access_acquire_grants to recalculate…) Otherwise-invisible Drupal code becomes easy to verify. The time you invest into writing tests will pay off throughout the project, and during future work as well. Have fun!

One of the best things about building websites with Drupal is that there are thousands of modules that help you quickly create functionality.

To set things up, you need to download Drush and add it to your path. For example, you might unpack it into /opt/drush and then add the following line to your ~/.bashrc:

PATH=/opt/drush:$PATH
export PATH

Reload your ~/.bashrc with source ~/.bashrc, and the drush command should become available. If you're on Microsoft Windows, it might need some more finagling. (Or you can just give up and use a virtual image of Linux to develop your Drupal websites. You'll probably end up much happier. ;) )

Once you've installed Drush, what can you do with it?

Drush is a huge time-saver. For example, I install dozens of modules in the course of building a Drupal website. Instead of copying the download link, changing to my sites/all/modules directory, pasting the download URL into my terminal window after wget, unpacking the file, deleting the archive, and then clicking through the various module enablement screens, I can just issue the following commands to download and enable the module.

drush dl modulename
drush en -y modulename

(The -y option means say yes to all the prompts.)

So much faster and easier. You can use these commands with several modules (module1 module2 module3), and you can use drush cli to start a shell that's optimized for Drush.

Drush is also useful if you've screwed up your Drupal installation and you need to disable themes or modules before things can work again. In the past, I'd go into the {system} table and carefully set the status of the offending row to 0. Now, that's just a drush dis modulename.

Drush has a bucketload of other useful commands, and drush help is well worth browsing. Give it a try!