November 17, 2009

Bulk view

Automating tedious wiki editing tasks with Emacs and w3m

I needed to update many of the links in our wiki because a team member left, so I had to reupload all of her files to a shared service and change all the URLs to point to the new files. Unfortunately, the file service didn’t send me the former URLs of the files, so that was going to be a manual process. Our wiki had 149 pages in it. Not fun.

After a few pages of editing (and correcting the occasional typo that crept in as I changed URLs), I decided to partially automate the process. Using a smidgen of Emacs Lisp, I created a function that pasted text into a temporary buffer, performed whatever automatic fixes it could make, prompted me for any URLs it didn’t recognize, remembered the old URL – new URL mapping I defined, and copied the text back.

The function looked somewhat like this:

(defvar sacha/wiki-links nil "Associative list of (old-url . new-url).")
(defun sacha/wiki-fix ()
  (interactive)
  (with-temp-buffer
    ;; Insert text from clipboard
    (yank)
    (goto-char (point-min))
    ;; Look for all the links 
    (while (re-search-forward
            "\\[\\([^|]+\\)|\\([^\]]+\\)\\]" nil t)
      ;; Check if it's one of the links I want to replace
      (if (or (string-match-p "viewpage" (match-string 2))
              (string-match-p "lsoohoo" (match-string 2)))
          (replace-match
           (save-match-data
             ;; Prompt and the entry to the map if it does not yet exist
             (unless (assoc (match-string 2) sacha/wiki-links)
               (add-to-list 'sacha/wiki-links
                            (cons (match-string 2)
                                  (read-string (concat (match-string 1)
                                                       "? ")))))
             ;; pick up the corresponding URL
             (cdr (assoc (match-string 2) sacha/wiki-links)))
           t t nil 2)))
    ;; Copy the text into the clipboard
    (kill-new (buffer-string))))
                   

I used M-x global-set-key to bind a convenient function key to it (F12, I think), and then it was just a matter of clicking on each page, clicking on Edit, typing Ctrl-C to copy the text, switching to Emacs, pressing F12, switching back to my browser, typing Ctrl-V, and saving the wiki page. I also added some lines (not shown here) to convert the previous wiki gardener’s full links to intrawiki links, change server URLs, and do other fun things.

I thought about fully automating it (somehow hooking into w3, perhaps?), but that seemed to be more trouble than needed. Besides, it was good to review all the pages.

As a result of this Emacs wizardry, processing all 149 wiki pages took me a few hours instead of a few days. Yay!

Of course, I finished the last wiki page, I found out that I needed to change the servers in the URL. I decided to go ahead and fully automate the darn thing.

I extracted a list of URLs for the wiki by viewing the tree version of the wiki index. It used Javascript, so I couldn’t just pull the URLs out of the source code. Fortunately, the Firebug plugin for Firefox lets me copy the rendered HTML, so I used that instead. Some judicious text-editing later (replace-regexp rocks), I had a list of URLs to the different pages. I knew I needed to put in some kind of delay when loading web pages. sleep-for let me spread out my requests so I didn’t hammer the server too badly. Reading the w3m.el source code turned up w3m-async-exec. Once I set that to nil, requesting web pages and running code on the results turned out to be straightforward. Selecting the right widgets was a bit of a hack (re-search-forward here, w3m-previous-anchor there), but hey, it worked. After confirming it by manually running it on a few pages, I left it merrily running in the background.

Here it is (some tweaking required):

(defun sacha/edit-wiki-page ()
  (interactive)
  (let ((buffer (current-buffer))
        (w3m-async-exec nil)
        (delay 5)) ;; number of seconds
    ;; While not at the end of the buffer
    (while (not (eobp))
      ;; Load the URL on the current line
      (w3m-browse-url
       (buffer-substring
        (line-beginning-position)
        (line-end-position)))
      ;; Look for the edit button
      (goto-char (point-min))
      (when (search-forward "Edit" nil t)
        ;; Click it
        (w3m-view-this-url)
        ;; Look for the Minor change checkbox
        (goto-char (point-min))
        (when (search-forward "Minor change" nil t)
          ;; The text area is the second widget back
          (w3m-previous-anchor 2)
          ;; Open the text area in a temporary buffer for editing
          (w3m-view-this-url)
          ;; Do the changes
          (while (re-search-forward "https?://example.com/path" nil t)
            (replace-match "http://path.example.com" t t nil 0))
          ;; Save the value
          (w3m-form-input-textarea-set)
          (when (search-backward "Save" nil t)
            (w3m-view-this-url))))
      (switch-to-buffer buffer)
      (forward-line)
      (sleep-for delay))))

I’m sure this kind of automation might be possible with lots of hacking in Mozilla Firefox, and I’ve seen great scripts for the Mac, too. But I know Emacs, I’m comfortable digging into source code, and I can make things work.

Awesome. =D

How I learned to stop worrying and love the webinar: Part 4: Taking the next steps

afteryourtalkThe only reason to give a presentation is to help people act or think differently. I’ve tried almost everything that could prod people to take the next step. I’ve distributed hand-outs summarizing the key points and next actions. I’ve given out worksheets. I’ve collected e-mail addresses and sent everyone a follow-up note with links to slides and what to do next. I’ve linked to short URLs on my slides to make it easy for people to take notes. I’ve even experimented with pairing people up so that they could follow up with each other. But it’s still a huge challenge to get people to think about a presentation after they’ve walked out the doors. After your talk, they all go their separate ways.

Virtual presentations are different. While I’m taking questions or after I wrap up, people can click on links in the text chat or type in the URL from my slides. They can download, review, and forward my slides right away. They can review the next steps. They can bookmark the page and return to it when they have questions. There are even systems that automatically track people’s interactions with the content, so I can e-mail them about updates.

The easier I make it for people to take the next small step, such as reviewing slides or planning their next actions, the more I support them in making the next big step, such as trying out social media tools. People’s access to mail and Internet can distract them during the presentation if you’re not engaging enough, but the same access can be powerful when you purposefully use it to guide people’s next actions.

This is the fourth part of How I learned to stop worrying and love the webinar.
Part 1: The best seats in the house
Part 2: From audience to participants
Part 3: Reading the room
Part 4: Taking the next steps
Next: Convenience and control