Why automation matters to me

| geek, kaizen

The release engineer working on our Drupal project is happy. He manages the deployment of four Drupal projects (each with three multisites), and the project I’m on takes the least time to deploy.

In the beginning, it took two hours to deploy one set of Drupal multisite because of many manual configuration steps. The team had gotten it down to fifteen minutes thanks to node imports and other time-savers, but it still required someone to click on buttons, type in text, and upload files from the source code tree.

Shortly after I joined the team, I started automating as much as I could. This helped me because I could quickly test my configuration from a clean install. I also wrote a web-based deployment script based on the script I’d written for my previous Drupal project. This script made it easy for any team member to deploy a selected revision to the site and reinstall all the microsites. Any time a manual step entered into the installation process, I ruthlessly automated it. Now deployments take a single click (or three, if you want to select a revision other than the latest one), and the build process has shrunk from fifteen minutes to five minutes, with no manual interaction required.

So the release engineer is happy, the team’s happy, I’m happy, and we can quickly deploy and test new code.

I like automating repetitive tasks. It’s not just about showing off a clever shell script or Makefile (although my scripts are clever) or about saving myself time (although I do). It’s about freeing people up to do better work.

One of my best friends was on an internship at a large computer company. A large part of her work involved copying and pasting data between spreadsheets, a task she detested. However, the cost of getting interns to do the work had been lower than the cost of automating the process, so she was assigned to copy and paste the information. If she were more comfortable with programming, she might have automated it herself, but she wasn’t familiar enough with Microsoft Excel to do so. So she did the mind-numbing work. When she left her role, no progress had been made on automating the work, and she knew that the next intern would probably also need to copy and paste information manually.

I remember hearing her frustration with the kind of work she was doing. I wished I could do something about it. I couldn’t, but at least I can automate whatever routine tasks I come across so that people can work more effectively.

You can view 11 comments or e-mail me at sacha@sachachua.com.

11 comments

Joel J. Adamson

2010-06-22T00:23:58Z

My brother, who is exactly unlike me in that he has no computer training, and hates reading, once automated a system using Microsoft Excel macros when his supervisors had told him to use Liquid Paper! I once saw a research assistant filling in cells in a spreadsheet one by one with the same data; she did it really fast, but she did it a lot faster after I told her about "Fill Down."

I too worked myself out of a few jobs as a temp, but not like I could today with my automation skills. I agree with you: people would not see a problem with automation if they would stop holding themselves down by saying "I'm not a programmer."

<nod> That's why I like being technical. The world has plenty of programmers who can build to specifications (or at least try). It needs more people who can look at something, go, "Hmm, there _must_ be a better way of doing this," and make it happen.

Joel J. Adamson

2010-06-22T16:46:55Z

Incidentally, one of the reasons I recommend using Emacs is to "dupe" people into becoming programmers. When there's a way to automate repetitive tasks, like those encountered in editing, people see the opportunity for changing their way of thinking.

Mike Wilson

2010-06-22T16:53:49Z

At the new job I fight the automation battle daily. The willingness to work around "little problems" every single time they come up, as a matter of procedure, makes my brain boil.

"This should be a single-click/perl script/canned query!" comes out of my mouth at least 3-4 times a day.

The irony of course is that I've plateaued in my emacs use far below a 'power user' level, and rarely automate squat in it. Too funny.

Joel J. Adamson

2010-06-22T17:23:20Z

Hello Mike,

My Emacs usage goes in cycles. I'll go months without a single piece of elisp originating, then I will spend a whole day on it. It's never too late to start. One of the best things about Emacs is that you can use it at so many levels. You're not required to be a power user, but you can become one in less than a week at any time in your usage career.

Brilliant way to look at it, Joel. =) It's great to use tools with depth to them. And customizability! Oh my.

Mike: If you think of Emacs as part of an entire system - together with Perl, and shell scripts, and whatever else you've got - then it doesn't really matter what tool you use or how deeply you grok it, as long as you can figure out how to make stuff happen (and have fun along the way).

That said, you can pick up plenty of inspiration by reading through the manual for your tools, other people's tweaks and customizations (http://www.emacswiki.org and http://planet.emacsen.org are awesome), and even the source code (it's there!). A large part of being able to automate things is simply knowing what's out there. For example, automating cool stuff in Perl is much easier when you know about CPAN (http://www.cpan.org), how to search it (http://search.cpan.org), and the probability that someone has already contributed a module that does whatever you need (very close to 1). Ditto Emacs and the Emacswiki.

Joel J. Adamson

2010-06-22T18:30:47Z

I call that "entire system" Unix, but you can call it whatever you want ;)

Heh. I call that entire system life. I've had lots of fun experimenting delegating to freelancers and virtual assistants (it's amazing how talented other people are!), taking advantage of resources (hello, Toronto Public Library), making jigs to help simplify woodworking, and working with other people to make stuff happen. The broader your toolkit, the more you can do.

Mike Wilson

2010-06-22T18:34:51Z

I definitely need a pull forward. My comfort level actually what's thwarting me now. (Well, that and a nigh on 20 year old .emacs file.)

(you forgot the kungfu masters over at perlmonks :) )

Oh, yes. And Q&A forums like StackOverflow, and other great places to learn. Sometimes, when I'm pretty happy with my setup, I'll help out on forums or IRC channels. Helping with other people's questions gives me an excuse to learn more, and other people's cool ideas often make me think, "Hmm, I want to do that too!"

Mike Wilson

2010-06-22T19:42:16Z

See, this is why I love coming here. Now I' m all fired up.

Incidentally, right now I have 6 emacs processes running. XEmacs on xp and sessions on 3 different linux and sun machines with under couple different accounts.

It's inSANE.