MacKuba

I’ve read a lot about good programming practices recently. I’ve read the “Pragmatic programmer” book (which is awesome, one of the most useful books I have read, seriously); I’ve watched a great presentation “Craftsmanship and ethics” by Robert C. Cooper. And it seems that everyone seems to emphasize that one thing that is extremely important to write good software is writing unit tests (and writing them “all the fucking time”, as that black dude said in that presentation ;). I must admit I still haven’t had the courage to switch to TDD (although I try it for single tasks from time to time), and my test coverage is nowhere near what it should be ― it ranges from 20% to 80% depending on the project and its layer. But I know it’s important and I’m working on it…

Anyway, one day I had a thought: even if I test all models and controllers thoroughly, am I not leaving something out? Didn’t I forget about something that is a quite important part of the application ― about my JavaScript code? After all, it’s code too, right? And sometimes it’s very important code; and not having any tests for it means the code is very fragile, it’s easy to break things, already fixed bugs may reappear again, and so on. Of course, JavaScript may be harder to test, because it’s sometimes very closely coupled to HTML, but at least some part of it could surely be tested.

But how do you unit test JavaScript?… I had no idea how to do this.

So I started googling, and I found that there are plenty of different unit test frameworks for JavaScript. Of course I couldn’t resist and I had to take a look at every single one and compare them to choose the best one :) (I heard that it’s called “maximiser” and that it’s bad…). The result of this looking is the list below.

The testing libraries can be divided into a few categories. First, there are two main kinds of syntax: most of the libraries use either the classic “assert” style inherited from JUnit (assertTrue(), assertEquals() and so on), or an RSpec-style syntax (using terms like “describe”, “it should” etc.). Since RSpec is my favourite test framework for Ruby code, I liked the second type more.

Also, tests can either be run in a browser, or from command-line, in a JavaScript engine that imitates the browser (mainly Rhino). Both option have their advantages: if you run tests in a browser, you’re testing your code in the real environment; the result reports also usually look better (maybe I’m weird, but if I’m going to look at something many times a day, I’d rather it looked good…). If you run tests from a console, you may not be able to test all the browser’s features, but it’s much easier to run tests automatically, e.g. on a CI server (since a CI server may not have a monitor attached to it, so it may not be able to run a browser at all…).

---

Update (2.04.2011): Things have changed a bit since then. Blue Ridge is officially not maintained anymore. There is however a new library called Jasmine, which is really good, uses RSpec style of writing tests, has support for mocks and a gem for Ruby/Rails integration.

---

The best

Here are the libraries which I liked most:

Screw.Unit ( + Blue Ridge plugin)

I’ll start with the winner, i.e. the one which I’m now using in my current project. The syntax looks very RSpec-like:

describe("Calculator", function() {
  it("should add numbers", function() {
    var calc = new Calculator();
    expect(calc.add(2, 2)).to(equal, 4);
  });
});

Of course it looks a bit more messy than in Ruby, because you have to use function() {} instead of do … end, but it’s good enough. And I don’t really have to type all that manually, because there is a Textmate bundle which makes it a lot easier. Screw.Unit also has such things as nested describe blocks, before and after functions in each describe, and there’s about 10 different matchers apart from equal that you can use (and you can add new ones easily).

But I didn’t get to the best part yet. The best part is a Rails plugin, previously known as javascript_testing, recently renamed to Blue Ridge. It combines several things together:

• Screw.Unit for testing
• Smoke, a mocking and stubbing library
• Rhino, a JavaScript interpreter engine in Java
• env.js, a great piece of code written by John Resig, which implements the whole DOM functionality on top of Rhino
• Rails generators which create some templates for you and make it easy to get started immediately

The result of all this is that you can easily write JavaScript unit tests, using mocks and stubs, and run them BOTH in a browser and on your CI server ― an ideal solution :) The only problem is that the Rhino/env.js engine doesn’t imitate the browser 100% correctly ― there are many bugs and some things are very hard to achieve (e.g. calculating offsets and sizes of elements), but env.js is under active development and things are getting better.

(Note: of course, you could integrate any of the in-browser test libraries with Rhino and env.js the same way it was done with Screw.Unit; but here it’s already done for you, so why bother?…)

JSpec (by Visionmedia)

This one is very different from the rest. What’s so special about it is that it uses a syntax that is not completely correct JavaScript. It uses some kind of preprocessor that first converts the tests to proper JavaScript, and then executes them. Thanks to that, the code looks more Ruby-like than in any other library:

describe 'Calculator'
  it 'should add numbers'
    var calc = new Calculator
    calc.add(2, 2).should.equal 4
  end
end

That’s right, no semicolons at the end, and no need for wrapping arguments in parentheses. I suspect that some JavaScript gurus would feel offended by this, and would say that it makes no sense to “fix” JavaScript by changing it into something it is not; I kind of like this syntax… Also, it has a Textmate bundle too, includes no end of matchers, and the results look just awesome:

The only reason I didn’t choose it is because the CI support was a requirement (client requested it), and I didn’t have time to try to make it work with env.js.

JSSpec

JSSpec is very similar to Screw.Unit. It has the same RSpec-style syntax:

describe("Calculator", function() {
  with(this) {
    it("should add numbers", function() {
      var calc = new Calculator();
      calc.add(2, 2).should(equal(4));
    });
  }
});

It’s also known to be able to run in Rhino. The Blue Ridge plugin has even included JSSpec previously, but later it switched to Screw.Unit, so I chose the latter…

jsUnitTest (+ jShoulda)

This is a member of the assert-style family. It’s based on the unittest library, which is a part of Scriptaculous and requires Prototype to run. jsUnitTest is a version of unittest that had all Prototype calls stripped from it and is now framework-independent. Tests run inside a browser. The code looks like this:

new Test.Unit.Runner({

  testCalculatorAdd: function() {
    with (this) {
      var calc = new Calculator();
      assertEqual(4, calc.add(2, 2));
    }
  }

});

And it can look like this if you install an additional library, jShoulda:

context('Calculator', {

  should('add numbers', function() {
    with (this) {
      var calc = new Calculator();
      assertEqual(4, calc.add(2, 2));
    }
  });

});

I think this is what I would choose if I liked the “assertEqual” style of assertions; but as I said, I don’t ;)

jqUnit

jqUnit, as you can guess from the name, is not framework-independent, it’s built on top of jQuery (which is not a problem for me… :) It’s one of those libraries which have a syntax that’s neither RSpec- nor JUnit-like. It runs inside a browser. Here’s how a test looks in jqUnit:

with (jqUnit) {

  module('calculator');
  test('add numbers', function() {
    var calc = new Calculator();
    equals(4, calc.add(2, 2));
  });

}

---

The rest

Just for completeness, here’s the rest of the libraries, with short descriptions:

• unittest
  As previously mentioned, it’s a part of Scriptaculous, dependent on Prototype. Runs in a browser, syntax is JUnit-style.
• jsAssertUnit
  Runs in a browser, more or less JUnit-style. Hasn’t been updated for ages…
• YUItest
  Part of the Yahoo UI framework ― I’m not sure if it can be used outside of it. Seems to me like it requires too much typing (YAHOO.util.Assert.areEqual etc.).
• Qooxdoo test framework
  Part of the Qooxdoo framework, but can be used without it. Runs in a browser, JUnit-style. Rather poorly documented.
• JSUnit (jsunit.net)
  Officially inspired by JUnit, runs in a browser. Hasn’t been updated recently.
• JSUnit (jsunit.berlios.de)
  Yeah, that’s a different one. Also inspired by JUnit, but this one runs only in Rhino, using Ant. Also not updated recently…
• Crosscheck
  JUnit-style, works in Rhino. Includes some kind of partial DOM implementation.
• Ecmaunit
  JUnit-style, not updated since 2005.
• Dojo Objective Harness
  Part of Dojo framework, but may be used without it. JUnit-style, it looks like it can run both in a browser and in Rhino.
• FireUnit
  Written by John Resig, works only in Firefox, and prints the results in Firebug’s console. Custom syntax (fireunit.ok(), fireunit.compare()).
• QUnit
  Requires jQuery; very similar to jqUnit.
• RhinoUnit
  Obviously, runs only in Rhino. Custom syntax (assert.that(2 + 2, eq(4)).
• JSpec (by Yehuda Katz)
  It has nothing to do with the JSpec that uses a preprocessor… This one was written by Yehuda, has a nice RSpec-like syntax, and prints results in Firebug console. It doesn’t seem to be developed further, looks like it’s just something that was written on one weekend and never really touched again…

---

Bonus: mocking and stubbing

When you’re writing tests, it’s sometimes good to be able to override some methods in other objects, to test the tested object in isolation; in JavaScript, this is a lot easier than even in Ruby, but a nice library can help anyway. I found several of them, and they all have comparable features and syntax. I could recommend almost any of them, it’s mostly a matter of taste ― I’m using Smoke because that’s what is integrated with Blue Ridge.

Here are the options, with code samples:

Smoke

new Smoke.Stub(Calculator, 'add').and_return(5);
Smoke.Mock(Calculator).should_receive('add').at_least('once').
  with_arguments(2, 2).and_return(5);

Amok

var mock = amok.mock(Calculator);
mock.should_receive('add').with_args(2, Number).times(1).and_return(5);

qMock

var mock = new Mock();
mock.expects(1).method('add').withArguments(2, 2).andReturn(5);

Jack

jack.expect("Calculator.add").exactly("1 time").
  withArguments(2, 2).mock(function() {
  …
  return sth;
});
jack.create("mock", ['add', 'substract']);
jack.expect("Calculator.add").exactly("1 time").
  whereArgument(0).isOneOf(2, 3, 4);

JSMock

Last updated 2007…

var factory = new MockControl();
var mock = factory.createMock(Calculator);
mock.expects().add(2, 2).andReturn(4);

MockMe

This one is actually a bit different. It says on the website that it’s a “spying” library, not a mocking library. The difference is that instead of specifying up front what will happen, you make the mock record what kinds of methods are called on it, and AFTER the execution you verify if it was called properly. I’m not really sure this is a better way of doing this…

mock(Calculator).andDo(function() {
  SomethingThatUsesCalculator.calculate("2 + 2");
  verify(Calculator.add)(2, 2);
});

So, happy unit testing ;)