On this page:

First steps towards Javascript testing

I know, I know, it’s about time I got the hang of this. Better late than never, right? =) Anyway, I spent some time going through tutorials for QUnit and Jasmine. For QUnit, I followed this Smashing Magazine tutorial on Javascript unit testing. I modified the code a little bit to add the Z timezone to the test data, since my tests initially didn’t pass.

test.html

<!DOCTYPE html>
<html>
<head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
    <title>Refactored date examples</title>
    <link rel="stylesheet" href="http://code.jquery.com/qunit/qunit-1.15.0.css" />
    <script src="http://code.jquery.com/qunit/qunit-1.15.0.js"></script>
    <script src="prettydate.js"></script>
    <script>
     test("prettydate.format", function() {
       function date(then, expected) {
         equal(prettyDate.format('2008/01/28 22:25:00Z', then), expected);
       }
       date("2008/01/28 22:24:30Z", "just now");
       date("2008/01/28 22:23:30Z", "1 minute ago");
       date("2008/01/28 21:23:30Z", "1 hour ago");
       date("2008/01/27 22:23:30Z", "Yesterday");
       date("2008/01/26 22:23:30Z", "2 days ago");
       date("2007/01/26 22:23:30Z", undefined);
     });
     function domtest(name, now, first, second) {
       test(name, function() {
         var links = document.getElementById('qunit-fixture').getElementsByTagName('a');
         equal(links[0].innerHTML, 'January 28th, 2008');
         equal(links[2].innerHTML, 'January 27th, 2008');
         prettyDate.update(now);
         equal(links[0].innerHTML, first);
         equal(links[2].innerHTML, second);
       });
     }

     domtest("prettyDate.update", '2008-01-28T22:25:00Z', '2 hours ago', 'Yesterday');
     domtest("prettyDate.update, one day later", '2008-01-29T22:25:00Z', 'Yesterday', '2 days ago');
    </script>
</head>
<body>
  <div id="qunit"></div>
  <div id="qunit-fixture">
    <ul>
      <li class="entry" id="post57">
        <p>blah blah blah…</p>
        <small class="extra">
          Posted <span class="time"><a href="/2008/01/blah/57/" title="2008-01-28T20:24:17Z">January 28th, 2008</a></span>
          by <span class="author"><a href="/john/">John Resig</a></span>
        </small>
      </li>
      <li class="entry" id="post57">
        <p>blah blah blah…</p>
        <small class="extra">
          Posted <span class="time"><a href="/2008/01/blah/57/" title="2008-01-27T22:24:17Z">January 27th, 2008</a></span>
          by <span class="author"><a href="/john/">John Resig</a></span>
        </small>
      </li>
    </ul>
  </div>
</body>
</html>

For practice, I converted the QUnit tests to Jasmine. The first part of the test was easy, but I wanted a clean way to do the HTML fixture-based tests for prettydate.update too. Jasmine-JQuery gives you a handy way to have HTML fixtures. Here’s what my code ended up as:

spec/DateSpec.js

describe("PrettyDate.format", function() {
    function checkDate(name, then, expected) {
        it(name, function() {
            expect(prettyDate.format('2008/01/28 22:25:00Z', then)).toEqual(expected);
        });
    }
    checkDate("should display recent times", '2008/01/28 22:24:30Z', 'just now');
    checkDate("should display times within a minute", '2008/01/28 22:23:30Z', '1 minute ago');
    checkDate("should display times within an hour", '2008/01/28 21:23:30Z', '1 hour ago');
    checkDate("should display times within a day", '2008/01/27 22:23:30Z', 'Yesterday');
    checkDate("should display times within two days", '2008/01/26 22:23:30Z', '2 days ago');
});
describe("PrettyDate.update", function() {
    function domtest(name, now, first, second) {
       it(name, function() {
           loadFixtures('test_fixture.html');
           var links = document.getElementById('qunit-fixture').getElementsByTagName('a');
           expect(links[0].innerHTML).toEqual('January 28th, 2008');
           expect(links[2].innerHTML).toEqual('January 27th, 2008');
           prettyDate.update(now);
           expect(links[0].innerHTML).toEqual(first);
           expect(links[2].innerHTML).toEqual(second);
       });
    }
    domtest("prettyDate.update", '2008-01-28T22:25:00Z', '2 hours ago', 'Yesterday');
    domtest("prettyDate.update, one day later", '2008-01-29T22:25:00Z', 'Yesterday', '2 days ago');
});

jasmine.html

<!DOCTYPE HTML>
<html>
<head>
  <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  <title>Jasmine Spec Runner v2.0.2</title>

  <link rel="shortcut icon" type="image/png" href="lib/jasmine-2.0.2/jasmine_favicon.png">
  <link rel="stylesheet" type="text/css" href="lib/jasmine-2.0.2/jasmine.css">

  <script type="text/javascript" src="https://code.jquery.com/jquery-2.1.1.min.js"></script>
  <script type="text/javascript" src="lib/jasmine-2.0.2/jasmine.js"></script>
  <script type="text/javascript" src="lib/jasmine-2.0.2/jasmine-html.js"></script>
  <script type="text/javascript" src="lib/jasmine-2.0.2/boot.js"></script>
  <script type="text/javascript" src="jasmine-jquery.js"></script>

  <!-- include source files here... -->
  <script type="text/javascript" src="prettydate.js"></script>

  <!-- include spec files here... -->
  <script type="text/javascript" src="spec/DateSpec.js"></script>

</head>

<body>
</body>
</html>

I’m looking forward to learning how to use Jasmine to test Angular applications, since behaviour-driven testing seems to be common practice there. Little steps! =)

Emacs: Evaluating Javascript and CSS in Chrome using Skewer Mode

I build a lot of quick prototypes, using Javascript and CSS to build little tools on top of the Jive social business platform. Since Javascript syntax errors could prevent the proper loading of the overview page customization screen and require me to reset the overview page through the admin console, my previous Javascript workflow involved copying and pasting code into Google Chrome’s developer console. Most of the time, I used narrow-to-region to focus on just the specific script block or set of functions I was working on, and I used js2-mode to handle syntax highlighting and indentation. Once the Javascript was sorted out, I’d widen to get back to the full HTML, JS, and CSS file, using web-mode for indentation.

Copying code between Javascript buffers and the developer console was a bit of a hassle. I’d use C-x h (mark-whole-buffer) to select the buffer, then C-w to copy it, change over to the Chrome window (possibly wading through a number of tabs and windows to find the right one), find the developer console, click in it, paste the code, and run it. My first step was to define a custom function that copied the whole buffer:

(defun sacha/copy-buffer ()
  "Copy buffer contents to kill ring."
  (interactive)
  (kill-new (buffer-substring-no-properties (point-min) (point-max))))
(global-set-key (kbd "C-c w") 'sacha/copy-buffer)

I still had to find the Chrome window and paste the code in, though.

My CSS workflow had its own challenges. I used Inspect Elements to look at CSS properties, and then I modified them on the fly. When I was happy with the rules I added or changed, I wrote the corresponding CSS code in my local file. Because I often ended up modifying several elements, it was hard to remember all the changes I needed to make, apply different sets of changes, or apply the changes after reloading the page. I used Stylish to make some of my changes persistent, but that still involved going back and forth between screens.

Since I’ll continue to work on web development over the next year (at least!), I thought I’d invest some time into improving my workflow. I’d seen several demos of Skewer Mode for Emacs, and I’d even given it a try a few times before. I hadn’t integrated it into my workflow yet, but it looked like it was definitely worth a try. Skewer allows you to interact with Google Chrome from Emacs. You can send HTML, CSS, and Javascript fragments to your browser.

If you use the included Greasemonkey-compatible script, you can even use this interactive capability on any website. I used the Tampermonkey extension for Google Chrome to run the script. When I tried it on the site I was working on, though, the https/http mismatch resulted in a content security error. It turns out that you need to run chrome --allow-running-insecure-content in order to let Chrome inject HTTPS sites with the scripts from the local HTTP server that Skewer Mode runs inside Emacs. If you had other Chrome sessions open, you’ll want to close them before starting up Chrome with that option. Once I sorted that out, it was easy to run skewer-setup, open a JS file, and start sending code to my browser.

I quickly became a fan of how C-c C-k (skewer-load-buffer in JS, skewer-css-eval-buffer in CSS) let me send my buffer to my browser. I narrowed my buffer to the parts I was working on, wrote some tests using console.assert(...), and kept the console visible as I coded. I periodically loaded the buffer to check whether my tests passed. I also liked having a proper file identifier and correct line numbers for errors. (It’s amazing how small things matter!)

Then to top it all off, I wanted a function that would prepare the source code for easy pasting into an HTML widget:

<script type="text/javascript">
// filename
</script>

Since Emacs is Emacs, you can do that. =)

(defvar sacha/javascript-test-regexp (concat (regexp-quote "/** Testing **/") "\\(.*\n\\)*")
  "Regular expression matching testing-related code to remove.
See `sacha/copy-javascript-region-or-buffer'.")

(defun sacha/copy-javascript-region-or-buffer (beg end)
  "Copy the active region or the buffer, wrapping it in script tags.
Add a comment with the current filename and skip test-related
code. See `sacha/javascript-test-regexp' to change the way
test-related code is detected."
  (interactive "r")
  (unless (region-active-p)
    (setq beg (point-min) end (point-max)))
  (kill-new
   (concat
    "<script type=\"text/javascript\">\n"
    (if (buffer-file-name) (concat "// " (file-name-nondirectory (buffer-file-name)) "\n") "")
    (replace-regexp-in-string
     sacha/javascript-test-regexp
     ""
     (buffer-substring (point-min) (point-max))
     nil)
    "\n</script>")))

(define-key js2-mode-map (kbd "C-c w") 'sacha/copy-javascript-region-or-buffer)

So now I can fiddle around with Javascript and CSS, send it to my browser with C-c C-k, and then use C-c w to wrap the Javascript in script tags and prepare it for copying.

Emacs!

My path for learning AngularJS

I’d been meaning to learn AngularJS for a while, and rapidly prototyping a data-binding-heavy Javascript application was the perfect excuse. The phonecat tutorial on the AngularJS site was a little too heavy-weight for me, although it would probably have been useful for learning how to Do Things Right. Simpler, from-scratch tutorials like AngularJS in 30 minutes and ng-newsletter were a little more useful for me. After I got the hang of setting things up and using a controller, I browsed through the AngularJS documentation and looked for different modules as I needed them.

Here’s the rough order I learned things in:

  1. Binding data with {{}}
  2. Retrieving data with $http (since I already had JSON handy from the NodeJS site I created)
  3. Iterating over data with ng-repeat
  4. Adding ng-click events
  5. Using ng-class
  6. Dependency injection
  7. Figuring out routing with ui-router
  8. Dividing things into multiple routers
  9. $interval and $timeout
  10. State change functions
  11. Resources (although I didn’t end up really using these)
  12. Directives

I still have to learn about filters, nested views, testing, proper file organization, and all sorts of other goodness. But yeah, AngularJS feels pretty good for my brain… =) Yay!

Imagining an index of sketchnotes

With sketchnotes gaining in popularity, I’m often curious about how other people drew a talk. TED talks are popular for sketchnoting practice, and sketchnoters are beginning to bump into each other at conferences as well.

There are many avenues to share or discover sketchnotes, such as The Sketchnote Handbook Flickr Pool and the wonderful graphic recordings at Ogilvy Notes. Sketchnote Army is a blog that features lots of sketchnotes, and Twitter searches turn up even more. But there isn’t really something that’ll help you bump into other sketchnotes of the same talk, or even sketchnotes of the same conference.

Are we at the point yet where multiple people might be sketchnoting something? For popular TED talks, yes, and many conferences might have sketchnoters in the crowd. I think it would be interesting to make it easier for people to find each other and compare notes.

So I registered sketchnoteindex.com and created a quick spreadsheet to get a feel for the data that would be good to capture and how we might want to organize it. (Prototype with the lowest-effort thing first!) In addition to indexing topics, I’d like to eventually build an image and visual metaphor index too, so we can see how different people have represented time. Text search would rock someday. In the meantime, I put together a quick text prototype as an excuse to learn more about the Ember.js framework, although I’m thiiiis close to chucking it all and using Emacs or a Ruby script to generate static HTML.

Some things to consider:

  • We want to avoid spam and build good data for navigation. I can start off manually indexing sketchnotes, and then open it up for submission (possibly with an assistant).
  • Many sketchnotes don’t indicate their licensing, so technically, they’d be all rights reserved. We can link to things, and include thumbnails if we have permission.
  • I can coordinate with Sketchnote Army (Mike / Binaebi) for submissions, and I can set up notifications for other sources.
  • Revenue model: Advertising? Flesh this out into a system where conferences can pay a small fee to have branding on their page? Do this as a volunteer because I want to learn more about sketchnotes along the way? Hmm…

Right now, Ember.js pulls the data off the CSV I exported from my Google Docs spreadsheet. That way, I don’t have to create an admin interface or anything else. I’m not actually using Ember.js’ features (aside from a little templating and a few models), so I may swap it for something else.

So this was about eight hours including data entry (300+ sketchnotes; I did it myself instead of delegating because I wanted to sketch the idea out quickly), going through the Ember.js tutorials, and fighting with Javascript.

"An Easy Method for Beginners in Latin" and macron-insensitive search for Tiddlywiki

As previously mentioned, W- and I are re-typing parts of Albert Harkness’ 1822 textbook "An Easy Method for Beginners in Latin", which was digitized and uploaded to Google Books as a PDF of images. The non-searchable book was driving W- mad, so we’re re-typing up lessons. It’s a decent way to review, and I’m sure it will be a great resource for other people too.

Here’s what we have so far: An Easy Method for Beginners in Latin, Lessons 1-9

We’re starting off using Tiddlywiki because it’s a wiki system that W-‘s been using a lot for his personal notes. He’s familiar with the markup. It’s not ideal because Google doesn’t index it, the file size is bigger than it needs to be (0.5MB!), and it’s Javascript-based. It’s a good start, though, and I should be able to convert the file to another format with a little scripting. My first instinct would be to start with Org Mode for Emacs, of course, but we already know what W- thinks of Emacs. ;)

Most of the text was easy to enter. Harkness is quite fond of footnotes, numbered sections, and lots of bold and italic formatting. We’re going to skip the illustrations for now.

Typing all of this in and using it as our own reference, though, we quickly ran into a limitation of the standard TiddlyWiki engine (and really, probably all wiki engines): you had to search for the exact word to find something. In order to find poēta, you had to type poēta, not poeta. That’s because ē and e are two different characters.

We wanted to keep the macrons as pronunciation and grammar guides. We didn’t want to require people to know or type letters with macrons. Hmm. Time to hack Tiddlywiki.

TiddlyWiki plugins use Javascript. I found a sample search plugin that showed me the basics of what I needed.

I considered two approaches:

  1. Changing the search text to a regular expression that included macron versions of each vowel
  2. Replacing all vowels in the Tiddler texts with non-macron vowels when searching

The first approach was cleaner and looked much more efficient, so I chose that route. If the search text contained a macron, I assumed the searcher knew what he or she was doing, so I left the text alone. If the text did not contain a macron, I replaced every vowel with a regular expression matching the macron equivalents. Here’s what that part of the code looked like:

s = s.replace(/(.)/g, "['/]*$1");
if (!s.match(macronPattern)) {
  // Replace the vowels with the corresponding macron matchers
  s = s.replace(/a/, "[aāĀA]");
  s = s.replace(/e/, "[eēĒE]");
  s = s.replace(/i/, "[iīĪI]");
  s = s.replace(/o/, "[oōŌO]");
  s = s.replace(/u/, "[uūŪU]");
}

That got me almost all the way there. I could search for most of the words using plain text (so poeta would find poēta and regina would find rēgīnae), but some words still couldn’t be found.

A further quirk of the textbook is that the characters in a word might be interrupted by formatting. For example, poēt<strong>am</strong> is written as =poēt”am”= in Tiddlywiki markup. So I also inserted a regular expression matching any number of ‘ or / (bold or italic markers when doubled) between each letter:

s = s.replace(/(.)/g, "['/]*$1");

It’s important to do this before the macron substitution, or you’ll have regexp classes inside other classes.

That’s the core of the macron search. Here’s what it looks like. I was so thrilled when I got all of this lined up! =)

image

And the source code:

// Macron Search Plugin
// (c) 2011 Sacha Chua - Creative Commons Attribution ShareAlike 3.0 License
// Based on http://devpad.tiddlyspot.com/#SimpleSearchPlugin by FND

if(!version.extensions.MacronSearchPlugin) { //# ensure that the plugin is only installed once
version.extensions.MacronSearchPlugin = { installed: true };

if(!config.extensions) { config.extensions = {}; }

config.extensions.MacronSearchPlugin = {
  heading: "Search Results",
  containerId: "searchResults",
  btnCloseLabel: "Close search",
  btnCloseTooltip: "dismiss search results",
  btnCloseId: "search_close",
  btnOpenLabel: "Open all search results",
  btnOpenTooltip: "Open all search results",
  btnOpenId: "search_open",

  displayResults: function(matches, query) {
    story.refreshAllTiddlers(true); // update highlighting within story tiddlers
    var el = document.getElementById(this.containerId);
    query = '"""' + query + '"""'; // prevent WikiLinks
    if(el) {
      removeChildren(el);
    } else { //# fallback: use displayArea as parent
      var container = document.getElementById("displayArea");
      el = document.createElement("div");
      el.id = this.containerId;
      el = container.insertBefore(el, container.firstChild);
    }
    var msg = "!" + this.heading + "\n";
    if(matches.length > 0) {
        msg += "''" + config.macros.search.successMsg.format([matches.length.toString(), query]) + ":''\n";
      this.results = [];
      for(var i = 0 ; i < matches.length; i++) {
        this.results.push(matches[i].title);
        msg += "* [[" + matches[i].title + "]]\n";
      }
    } else {
      msg += "''" + config.macros.search.failureMsg.format([query]) + "''\n"; // XXX: do not use bold here!?
    }
    wikify(msg, el);
    createTiddlyButton(el, "[" + this.btnCloseLabel + "]", this.btnCloseTooltip, config.extensions.MacronSearchPlugin.closeResults, "button", this.btnCloseId);
    if(matches.length > 0) { // XXX: redundant!?
      createTiddlyButton(el, "[" + this.btnOpenLabel + "]", this.btnOpenTooltip, config.extensions.MacronSearchPlugin.openAll, "button", this.btnOpenId);
    }
  },

  closeResults: function() {
    var el = document.getElementById(config.extensions.MacronSearchPlugin.containerId);
    removeNode(el);
    config.extensions.MacronSearchPlugin.results = null;
    highlightHack = null;
  },

  openAll: function(ev) {
    story.displayTiddlers(null, config.extensions.MacronSearchPlugin.results);
    return false;
  }
};

// override Story.search()
Story.prototype.search = function(text, useCaseSensitive, useRegExp) {
  var macronPattern = /[āĀēĒīĪōŌūŪ]/;
  var s = text;
  // Deal with bold and italics in the middle of words
  s = s.replace(/(.)/g, "['/]*$1");
  if (!s.match(macronPattern)) {
    // Replace the vowels with the corresponding macron matchers
    s = s.replace(/a/, "[aāĀA]");
    s = s.replace(/e/, "[eēĒE]");
    s = s.replace(/i/, "[iīĪI]");
    s = s.replace(/o/, "[oōŌO]");
    s = s.replace(/u/, "[uūŪU]");
  }
  var searchRegexp = new RegExp(s, "img");
  highlightHack = searchRegexp;
  var matches = store.search(searchRegexp, null, "excludeSearch");
  config.extensions.MacronSearchPlugin.displayResults(matches, text);
};

// override TiddlyWiki.search() to ignore macrons when searching
TiddlyWiki.prototype.search = function(s, sortField, excludeTag, match) {
    // Find out if the search string s has a macron
    var candidates = this.reverseLookup("tags", excludeTag, !!match);
    var matches = [];
    for(var t = 0; t < candidates.length; t++) {
        if (candidates[t].title.search(s) != -1 ||
            candidates[t].text.search(s) != -1) {
            matches.push(candidates[t]);
        }
    }
    return matches;
};

} //# end of "install only once"

To add this to your Tiddlywiki, create a new tiddler. Paste in the source code. Give it the systemConfig tag (the case is important). Save and reload your Tiddlywiki file, and it should be available.

It took me maybe 1.5 hours to research possible ways to do it and hack the search plugin together for Tiddlywiki. I’d never written a plugin for Tiddlywiki before, but I’ve worked with Javascript, and it was easy to pick up. I had a lot of fun coding it with W-, who supplied plenty of ideas and motivation. =) It’s fun geeking out!

Drupal in the trenches: AJAX history makes my brain hurt

Many websites use asynchronous Javascript and XML (AJAX) to provide all sorts of whizbang improvements, such as smooth interaction without page reloads. It took me a week to figure out how to do all the effects specified in our information architecture document: callouts, modal dialogs, in-page calendar navigation, and so on. I was pretty happy with what I did, considering it was my first serious work with JQuery.

Then my project manager said, “If I go to the event details page from the month view and hit back, it takes me to the day view instead.”

I said, “Welcome to the wonderful world of AJAX. This might be tough to fix.”

Making the back button work in AJAX applications requires a lot of effort. It doesn’t look like people have a nice and clean solution for it yet, although there are a number of libraries that try to address the situation.

Following the principle of progressive enhancement, I had built all the plain HTML functionality first, then layered the Javascript on top of it using jQuery callbacks. In addition to ensuring that the site still works even if Javascript is disabled, this approach also helps make sure that I have proper URLs for almost all the webpages involved. (I didn’t bother with explicitly transient pages like the year navigator or the day pop-up.)

I started with this Hijax-based approach, because it had the most documentation. I had problems getting it to behave, though, because my AJAX pages have other AJAX links that fail with the history-remote plugin. The history_remote plugin works by replacing all the links with the current page and a code (#remote-1, for example). When the back button is pressed, the library looks for the appropriate link and triggers the click event. This breaks down when the link isn’t actually on the first page. For example, when a user switches from a week to a month view, then goes to the next month, the plugin can’t find the link to the next month on the week view’s page, which is where the user started.

What I really needed to do is encode more information in the URL. If I encode information in the anchor portion of the URL (#…), I can use that to request the page and put that into the appropriate div. For example, if I pass in #transitions_content?new_path=connect/calendar/2009/05 , I might be able to parse that and put the content of new_path into the transitions_content div.

I started going down that rabbit-hole, and then I got myself thoroughly confused, so I decided that the best way would be to just rip out the major AJAX navigation and go back to the simple stuff (which fortunately still works).

Gah. Brain hurts!

Does anyone have a clean way to do this?