sacha chua living an awesome life

As previously mentioned, W- and I are re-typing parts of Albert Harkness’ 1822 textbook "An Easy Method for Beginners in Latin", which was digitized and uploaded to Google Books as a PDF of images. The non-searchable book was driving W- mad, so we’re re-typing up lessons. It’s a decent way to review, and I’m sure it will be a great resource for other people too.

Here’s what we have so far: An Easy Method for Beginners in Latin, Lessons 1-9

We’re starting off using Tiddlywiki because it’s a wiki system that W-’s been using a lot for his personal notes. He’s familiar with the markup. It’s not ideal because Google doesn’t index it, the file size is bigger than it needs to be (0.5MB!), and it’s Javascript-based. It’s a good start, though, and I should be able to convert the file to another format with a little scripting. My first instinct would be to start with Org Mode for Emacs, of course, but we already know what W- thinks of Emacs. ;)

Most of the text was easy to enter. Harkness is quite fond of footnotes, numbered sections, and lots of bold and italic formatting. We’re going to skip the illustrations for now.

Typing all of this in and using it as our own reference, though, we quickly ran into a limitation of the standard TiddlyWiki engine (and really, probably all wiki engines): you had to search for the exact word to find something. In order to find poēta, you had to type poēta, not poeta. That’s because ē and e are two different characters.

We wanted to keep the macrons as pronunciation and grammar guides. We didn’t want to require people to know or type letters with macrons. Hmm. Time to hack Tiddlywiki.

TiddlyWiki plugins use Javascript. I found a sample search plugin that showed me the basics of what I needed.

I considered two approaches:

1. Changing the search text to a regular expression that included macron versions of each vowel
2. Replacing all vowels in the Tiddler texts with non-macron vowels when searching

The first approach was cleaner and looked much more efficient, so I chose that route. If the search text contained a macron, I assumed the searcher knew what he or she was doing, so I left the text alone. If the text did not contain a macron, I replaced every vowel with a regular expression matching the macron equivalents. Here’s what that part of the code looked like:

s = s.replace(/(.)/g, "['/]*$1");
if (!s.match(macronPattern)) {
  // Replace the vowels with the corresponding macron matchers
  s = s.replace(/a/, "[aāĀA]");
  s = s.replace(/e/, "[eēĒE]");
  s = s.replace(/i/, "[iīĪI]");
  s = s.replace(/o/, "[oōŌO]");
  s = s.replace(/u/, "[uūŪU]");
}

That got me almost all the way there. I could search for most of the words using plain text (so poeta would find poēta and regina would find rēgīnae), but some words still couldn’t be found.

A further quirk of the textbook is that the characters in a word might be interrupted by formatting. For example, poēt<strong>am</strong> is written as =poēt”am”= in Tiddlywiki markup. So I also inserted a regular expression matching any number of ‘ or / (bold or italic markers when doubled) between each letter:

s = s.replace(/(.)/g, "['/]*$1");

It’s important to do this before the macron substitution, or you’ll have regexp classes inside other classes.

That’s the core of the macron search. Here’s what it looks like. I was so thrilled when I got all of this lined up! =)

And the source code:

// Macron Search Plugin
// (c) 2011 Sacha Chua - Creative Commons Attribution ShareAlike 3.0 License
// Based on http://devpad.tiddlyspot.com/#SimpleSearchPlugin by FND

if(!version.extensions.MacronSearchPlugin) { //# ensure that the plugin is only installed once
version.extensions.MacronSearchPlugin = { installed: true };

if(!config.extensions) { config.extensions = {}; }

config.extensions.MacronSearchPlugin = {
  heading: "Search Results",
  containerId: "searchResults",
  btnCloseLabel: "Close search",
  btnCloseTooltip: "dismiss search results",
  btnCloseId: "search_close",
  btnOpenLabel: "Open all search results",
  btnOpenTooltip: "Open all search results",
  btnOpenId: "search_open",

  displayResults: function(matches, query) {
    story.refreshAllTiddlers(true); // update highlighting within story tiddlers
    var el = document.getElementById(this.containerId);
    query = '"""' + query + '"""'; // prevent WikiLinks
    if(el) {
      removeChildren(el);
    } else { //# fallback: use displayArea as parent
      var container = document.getElementById("displayArea");
      el = document.createElement("div");
      el.id = this.containerId;
      el = container.insertBefore(el, container.firstChild);
    }
    var msg = "!" + this.heading + "\n";
    if(matches.length > 0) {
        msg += "''" + config.macros.search.successMsg.format([matches.length.toString(), query]) + ":''\n";
      this.results = [];
      for(var i = 0 ; i < matches.length; i++) {
        this.results.push(matches[i].title);
        msg += "* [[" + matches[i].title + "]]\n";
      }
    } else {
      msg += "''" + config.macros.search.failureMsg.format([query]) + "''\n"; // XXX: do not use bold here!?
    }
    wikify(msg, el);
    createTiddlyButton(el, "[" + this.btnCloseLabel + "]", this.btnCloseTooltip, config.extensions.MacronSearchPlugin.closeResults, "button", this.btnCloseId);
    if(matches.length > 0) { // XXX: redundant!?
      createTiddlyButton(el, "[" + this.btnOpenLabel + "]", this.btnOpenTooltip, config.extensions.MacronSearchPlugin.openAll, "button", this.btnOpenId);
    }
  },

  closeResults: function() {
    var el = document.getElementById(config.extensions.MacronSearchPlugin.containerId);
    removeNode(el);
    config.extensions.MacronSearchPlugin.results = null;
    highlightHack = null;
  },

  openAll: function(ev) {
    story.displayTiddlers(null, config.extensions.MacronSearchPlugin.results);
    return false;
  }
};

// override Story.search()
Story.prototype.search = function(text, useCaseSensitive, useRegExp) {
  var macronPattern = /[āĀēĒīĪōŌūŪ]/;
  var s = text;
  // Deal with bold and italics in the middle of words
  s = s.replace(/(.)/g, "['/]*$1");
  if (!s.match(macronPattern)) {
    // Replace the vowels with the corresponding macron matchers
    s = s.replace(/a/, "[aāĀA]");
    s = s.replace(/e/, "[eēĒE]");
    s = s.replace(/i/, "[iīĪI]");
    s = s.replace(/o/, "[oōŌO]");
    s = s.replace(/u/, "[uūŪU]");
  }
  var searchRegexp = new RegExp(s, "img");
  highlightHack = searchRegexp;
  var matches = store.search(searchRegexp, null, "excludeSearch");
  config.extensions.MacronSearchPlugin.displayResults(matches, text);
};

// override TiddlyWiki.search() to ignore macrons when searching
TiddlyWiki.prototype.search = function(s, sortField, excludeTag, match) {
    // Find out if the search string s has a macron
    var candidates = this.reverseLookup("tags", excludeTag, !!match);
    var matches = [];
    for(var t = 0; t < candidates.length; t++) {
        if (candidates[t].title.search(s) != -1 ||
            candidates[t].text.search(s) != -1) {
            matches.push(candidates[t]);
        }
    }
    return matches;
};

} //# end of "install only once"

To add this to your Tiddlywiki, create a new tiddler. Paste in the source code. Give it the systemConfig tag (the case is important). Save and reload your Tiddlywiki file, and it should be available.

It took me maybe 1.5 hours to research possible ways to do it and hack the search plugin together for Tiddlywiki. I’d never written a plugin for Tiddlywiki before, but I’ve worked with Javascript, and it was easy to pick up. I had a lot of fun coding it with W-, who supplied plenty of ideas and motivation. =) It’s fun geeking out!

Many websites use asynchronous Javascript and XML (AJAX) to provide all sorts of whizbang improvements, such as smooth interaction without page reloads. It took me a week to figure out how to do all the effects specified in our information architecture document: callouts, modal dialogs, in-page calendar navigation, and so on. I was pretty happy with what I did, considering it was my first serious work with JQuery.

Then my project manager said, “If I go to the event details page from the month view and hit back, it takes me to the day view instead.”

I said, “Welcome to the wonderful world of AJAX. This might be tough to fix.”

Making the back button work in AJAX applications requires a lot of effort. It doesn’t look like people have a nice and clean solution for it yet, although there are a number of libraries that try to address the situation.

Following the principle of progressive enhancement, I had built all the plain HTML functionality first, then layered the Javascript on top of it using jQuery callbacks. In addition to ensuring that the site still works even if Javascript is disabled, this approach also helps make sure that I have proper URLs for almost all the webpages involved. (I didn’t bother with explicitly transient pages like the year navigator or the day pop-up.)

I started with this Hijax-based approach, because it had the most documentation. I had problems getting it to behave, though, because my AJAX pages have other AJAX links that fail with the history-remote plugin. The history_remote plugin works by replacing all the links with the current page and a code (#remote-1, for example). When the back button is pressed, the library looks for the appropriate link and triggers the click event. This breaks down when the link isn’t actually on the first page. For example, when a user switches from a week to a month view, then goes to the next month, the plugin can’t find the link to the next month on the week view’s page, which is where the user started.

What I really needed to do is encode more information in the URL. If I encode information in the anchor portion of the URL (#…), I can use that to request the page and put that into the appropriate div. For example, if I pass in #transitions_content?new_path=connect/calendar/2009/05 , I might be able to parse that and put the content of new_path into the transitions_content div.

I started going down that rabbit-hole, and then I got myself thoroughly confused, so I decided that the best way would be to just rip out the major AJAX navigation and go back to the simple stuff (which fortunately still works).

Gah. Brain hurts!

Does anyone have a clean way to do this?

Junction brings the
power of the Ruby on Rails web MVC framework to JavaScript. It
makes it much easier to develop web-based applications that runs,
validate input, process data, and produce output in modern web
browsers like Mozilla Firefox. Well worth looking into!

私はコンピューターを操作するのがやさしいことがわかった。 I found it easy to operate the computer.

On Technorati: javascript

For the Javascript-challenged among us, myself included,
script.aculo.us is a godsend. This is
where you can pick up code for drag-and-drop lists and funky visual
effects. Neato.

その情報は私のコンピュータのファイルに入っている。 The information is in a file in my computer.

On Technorati: javascript

By popular demand we’ve implemented drag and drop
reordering of to-dos and notes in Backpack.

WAAAAAAAAAAAAAAAH! http://backpackit.com is _so_ cool!

Hooray for Javascript wizards. They totally rock.

E-Mail from Jason Fried

On Technorati: ajax, pim