Why automation matters to me

The release engineer working on our Drupal project is happy. He manages the deployment of four Drupal projects (each with three multisites), and the project I’m on takes the least time to deploy.

In the beginning, it took two hours to deploy one set of Drupal multisite because of many manual configuration steps. The team had gotten it down to fifteen minutes thanks to node imports and other time-savers, but it still required someone to click on buttons, type in text, and upload files from the source code tree.

Shortly after I joined the team, I started automating as much as I could. This helped me because I could quickly test my configuration from a clean install. I also wrote a web-based deployment script based on the script I’d written for my previous Drupal project. This script made it easy for any team member to deploy a selected revision to the site and reinstall all the microsites. Any time a manual step entered into the installation process, I ruthlessly automated it. Now deployments take a single click (or three, if you want to select a revision other than the latest one), and the build process has shrunk from fifteen minutes to five minutes, with no manual interaction required.

So the release engineer is happy, the team’s happy, I’m happy, and we can quickly deploy and test new code.

I like automating repetitive tasks. It’s not just about showing off a clever shell script or Makefile (although my scripts are clever) or about saving myself time (although I do). It’s about freeing people up to do better work.

One of my best friends was on an internship at a large computer company. A large part of her work involved copying and pasting data between spreadsheets, a task she detested. However, the cost of getting interns to do the work had been lower than the cost of automating the process, so she was assigned to copy and paste the information. If she were more comfortable with programming, she might have automated it herself, but she wasn’t familiar enough with Microsoft Excel to do so. So she did the mind-numbing work. When she left her role, no progress had been made on automating the work, and she knew that the next intern would probably also need to copy and paste information manually.

I remember hearing her frustration with the kind of work she was doing. I wished I could do something about it. I couldn’t, but at least I can automate whatever routine tasks I come across so that people can work more effectively.

One Pingback/Trackback

  • This’ll drive you nuts: I’m self-taught. No schooling to speak of. As such it was tremendously difficult to get a job programming back in the late 80s, early 90s. So I ended up with an awful lot of temp data-entry type jobs.

    Having some nontrivial skills under my belt I frequently set out to automate them. Most of the time (at least 6 of 10-11 instances) when I was able to actually automate the task I was immediately made redundant.

    Now, that’s the right solution for my employers (who included your current one actually) but it used to burn me something terrible.

    The happy ending is that someone eventually said “hey wait… you’d probably be good over here, since you’ve made yourself useless at this job.” But it was a rough few years.

    To non-IT staff, people who automate menial office tasks are truly terrifying entities. They represent disturbance to the established order.

    I suppose it’s what comes of believing in magic.

    • An autodidact! Awesome. =)

      If you’re working for an employer who sees you as cheap, unqualified labor, and you automate your trivial work, your employer (who may not have a lot of respect for you) may say, “Great! I can save some money by getting rid of that position.”

      If you work for an employer who sees you as skilled talent, and you automate your trivial work, your employer (who’s already starting with some respect for you) may say, “Great! What else can I have you do?”

      Kinda funny that way. Probably depends on the quality of employer. If I were your employer, I would immediately have assigned you to do something else that was costing me more money than it should, so that you could make it better. =)

      There’s also working for yourself, which means you reap even more benefits from automating things. =)

      I accept that by automating things and by learning as much as I can, I’ll probably make some people nervous. Maybe it’s because they don’t know how to do these things themselves. Maybe it’s because they’re worried that I’ll be a flash in the pan, that I’ll burn myself out. =) But I don’t want to do mediocre work just to fit in, not if I can save people time and effort and not if I can open opportunities for growth. And if I ever run into short-sighted people who’ll take the immediate savings of not needing to pay for my skills over the the long-term growth of applying my talents and energy to different opportunities, well, life is too short to work in that kind of situation when there are many better ones.

      Thanks for sharing, and don’t ever stop making magic.

  • My brother, who is exactly unlike me in that he has no computer training, and hates reading, once automated a system using Microsoft Excel macros when his supervisors had told him to use Liquid Paper! I once saw a research assistant filling in cells in a spreadsheet one by one with the same data; she did it really fast, but she did it a lot faster after I told her about “Fill Down.”

    I too worked myself out of a few jobs as a temp, but not like I could today with my automation skills. I agree with you: people would not see a problem with automation if they would stop holding themselves down by saying “I’m not a programmer.”

    • <nod> That’s why I like being technical. The world has plenty of programmers who can build to specifications (or at least try). It needs more people who can look at something, go, “Hmm, there _must_ be a better way of doing this,” and make it happen.

  • Pingback: Sacha Chua on Automation « Sex, math and programming()

  • Incidentally, one of the reasons I recommend using Emacs is to “dupe” people into becoming programmers. When there’s a way to automate repetitive tasks, like those encountered in editing, people see the opportunity for changing their way of thinking.

  • At the new job I fight the automation battle daily. The willingness to work around “little problems” every single time they come up, as a matter of procedure, makes my brain boil.

    “This should be a single-click/perl script/canned query!” comes out of my mouth at least 3-4 times a day.

    The irony of course is that I’ve plateaued in my emacs use far below a ‘power user’ level, and rarely automate squat in it. Too funny.

  • Hello Mike,

    My Emacs usage goes in cycles. I’ll go months without a single piece of elisp originating, then I will spend a whole day on it. It’s never too late to start. One of the best things about Emacs is that you can use it at so many levels. You’re not required to be a power user, but you can become one in less than a week at any time in your usage career.

  • Brilliant way to look at it, Joel. =) It’s great to use tools with depth to them. And customizability! Oh my.

    Mike: If you think of Emacs as part of an entire system – together with Perl, and shell scripts, and whatever else you’ve got – then it doesn’t really matter what tool you use or how deeply you grok it, as long as you can figure out how to make stuff happen (and have fun along the way).

    That said, you can pick up plenty of inspiration by reading through the manual for your tools, other people’s tweaks and customizations (http://www.emacswiki.org and http://planet.emacsen.org are awesome), and even the source code (it’s there!). A large part of being able to automate things is simply knowing what’s out there. For example, automating cool stuff in Perl is much easier when you know about CPAN (http://www.cpan.org), how to search it (http://search.cpan.org), and the probability that someone has already contributed a module that does whatever you need (very close to 1). Ditto Emacs and the Emacswiki.

  • I call that “entire system” Unix, but you can call it whatever you want ;)

    • Heh. I call that entire system life. I’ve had lots of fun experimenting delegating to freelancers and virtual assistants (it’s amazing how talented other people are!), taking advantage of resources (hello, Toronto Public Library), making jigs to help simplify woodworking, and working with other people to make stuff happen. The broader your toolkit, the more you can do.

  • I definitely need a pull forward. My comfort level actually what’s thwarting me now. (Well, that and a nigh on 20 year old .emacs file.)

    (you forgot the kungfu masters over at perlmonks :) )

  • Oh, yes. And Q&A forums like StackOverflow, and other great places to learn. Sometimes, when I’m pretty happy with my setup, I’ll help out on forums or IRC channels. Helping with other people’s questions gives me an excuse to learn more, and other people’s cool ideas often make me think, “Hmm, I want to do that too!”

  • See, this is why I love coming here. Now I’ m all fired up.

    Incidentally, right now I have 6 emacs processes running. XEmacs on xp and sessions on 3 different linux and sun machines with under couple different accounts.

    It’s inSANE.