On this page:
  • Subversion: Restore deleted files
  • Drupal: Deploying two branches to three systems
  • Deploying to servers

Subversion: Restore deleted files

If you accidentally delete a file and then commit the change, you can retrieve the file along with the rest of the history by using svn copy to copy it into your working directory. svn copy -r revision-number your-full-repository-path-to-the-file file

UPDATE: pgier suggested svn copy http://svn/url/to/file@revision-number file

Common Use-Cases for Merging

Drupal: Deploying two branches to three systems

To keep track of the bugfixes we’ll need to make for our next release, I’ve created a Subversion branch called branches/release-1. Development of new features will continue on trunk, but we’ll merge in the bugfixes from release-1 every so often.

There are three environments we deploy to:

Local
Developers should be able to easily test both versions on their local machines.
QA server
We should be able to deploy both versions to a publicly-accessible QA server for acceptance testing.
Production server
We should be able to deploy release-1 (and then later, release-2 and so on) to the production server, preferably after a lot of testing

Editorial changes happen on the production server, where our users update content. We would like to be able to take a snapshot of that database and use that to test our development code on the QA server or in our local development environments. Because we use Domain Access to serve multiple subdomains with shared content, it’s not just a matter of using mysqldump to back up the database and copy it over. We also need to replace URLs inside the database, and we need to override domain_root using the $conf array in settings.php.

I’m the only one running Linux, so the other developers don’t really benefit from the Makefiles I’ve defined or the tools I use. For the simpler build system we had before (all development on trunk), I wrote a deployment script that allowed users to:

  • Download a stripped copy of the production database with the URLs changed for their local testing environment
  • Deploy a stripped copy of the production database to the QA server
  • Deploy a specified revision of the source code to the QA server
  • Deploy a specified revision of the source code to the production server

The new deployment script needed to allow users to do the same, but for both branches of the code. Both branches of the code would be simultaneously available on the QA server, so the script would need to deploy the code to different directories.

After some fiddling around with the page design (because I care about making interfaces make sense!), I came up with something that looks like this:

  Development Release-1
Local Database
QA Database
QA Deployment
Production Deployment None
Changelog
-------
r988 | somegeek | 2008-09-15 13:00:00 -0500 (Mon, 15 Sep 2008) | 2 lines

Use dev.qa.example.com
------------
[more changelog entries go here]
------------------------------------------------------------------------
r986 | somegeek | 2008-09-15 11:03:38 -0500 (Mon, 15 Sep 2008) | 3 lines

Starting a branch for release-1
------------------------------------------------------------------------
[more changelog entries go here]

The deployment script allows the user to get a copy of the database, deploy a copy of the database, or deploy specific revisions of branches.

Because I was having a hard time figuring out how to do ssh key-based operations from Apache (which runs as a no-login user), I use two shell scripts to do the dirty work. One shell script connects to the production server, creates a partial backup, copies the information over, and does any necessary replacements. Another shell script takes a domain name and optionally a revision, and deploys the revision from the appropriate branch.

Here’s my totally small-scale PHP way to show the revisions log:

$dev_output = shell_exec("svn log $dev_url $details --limit 20");
$dev_revisions = preg_match_all('/r([0-9]+)/', $dev_output, $dev_matches);

where $dev_url is the URL of the trunk in Subversion, and $details contains the username and password specified as options for the Subversion command-line.

I’m going to see if I can get my regression tests running on the server that I’ve got my deployment script on. Wouldn’t that be awesome?

Deploying to servers

I’m heading to the Philippines tomorrow, and to make life easer for the two other Windows-based PHP developers on my team, I updated the web-based deployment script I mentioned in
Development kaizen: deployment and testing. I added the ability to push a specified revision to the production server. It took me less time than I thought it would (I love it when things Just Work!), so I decided to spend time documenting it just in case I ever need to do it again (almost certainly) or just in case it breaks while I’m away (hope not).

Behind the scenes, there are a number of moving parts:

  • Key-based authentication. Because I need to copy files and run commands on the production and QA servers non-interactively, I needed to set up key-based authentication using SSH. I’m somewhat nervous about using a passphrase-less key, but I couldn’t see a way to work around this.
  • Rsync. I use rsync over ssh to transfer files to the remote system. It’s good at efficiently transferring changed files. I couldn’t use –delete to get rid of old files, though, as our source tree does not include the complete system.
  • A shell script with the suid bit. The shell script is responsible for exporting the requested revision to a temporary directory, rsyncing it over to the selected host, and running a few commands on the server in order to reset file permissions and clear the cache. The suid bit is there so that it takes my identity and uses the key that I set up. I resorted to suid because I couldn’t figure out how to make sure that Apache had its own key. I tried associating it with the user that Apache ran as, but I kept running into “no tty”-type errors. The suid workaround solved the problem quickly.
  • A PHP script that displays a form and the last 20 revisions. The form includes a drop-down box of the revisions displayed, a button for deploying to QA, and a button for deploying to the server. When submitted, the script does some error-checking, then uses system to call the relevant shell script. The script determines the list of revisions by using shell_exec to store the output of svn log … -limit 20 in a string, then using preg_match_all to match all instances of /r([0-9]+)/. Seems to work.