MacKuba

Update: A new post from December 2010 with updated stats is available here.

---

For a few weeks I’ve been working on a new iPhone application.

Like most of other Cocoa apps I’ve written so far, this app also includes a JSON parser to load some kind of data from a server. The first thing the application does when it starts is connect to the JSON API, download a data file (about 100 KB) and parse it. This used to take about 10-15 seconds on the device, and I thought it was reasonable until I noticed that the HTTP response actually arrives after a second or so. So what was it doing for the rest of the time? I had to find out.

So I did some debugging, and it turned out that the 10 seconds are spent just on parsing the downloaded JSON file. That’s pretty bad… Like in all previous apps, I used an open source library BSJSONAdditions, which I knew wasn’t the fastest one available, but I never had any major problems with it before. On the other hand, I never tested it on a 100 KB file…

I knew there were a few other JSON parser libraries in ObjC, so I decided to make a small benchmark and see how well they all compared to the one I used. The other libraries I tried were: JSON Framework, TouchJSON and YAJL.

I ran the tests and here’s what I got: (average results from 5 runs)

Simulator

Framework	time [s]
BSJSONAdditions	0.186046
JSON Framework	0.046078
TouchJSON	0.037545
YAJL	0.017034

Device

Framework	time [s]
BSJSONAdditions	9.120360
JSON Framework	2.4779178
TouchJSON	1.9329024
YAJL	0.8663534

Pretty big difference, isn’t it? So it seems that:

• the library I used is, sadly, way slower than any of the rest (here, ~4 times slower than JSON Framework and more than 10× slower than YAJL!)
• JSON Framework and TouchJSON are more or less OK (though TouchJSON is a bit less convenient to use, as it requires you to convert a string to NSData first)
• YAJL is definitely the best one (it might be because it’s core is written in plain C)

Also, all the libraries are about 50× faster while running in the simulator than while running on the device. This means that even the slowest library will run much faster in the simulator than the fastest one does on the device. And because of that, you should test your app on the device since the beginning, because if you put this off until the end, you may suddenly realize 2 days before the planned release that your app runs so slow that it’s completely unusable. As Wil Shipley said during one presentation I watched, “simulator is essentially infinitely fast”, i.e. you should never assume that whatever runs smoothly on the simulator will run the same on the device – you always have to check.

---

If you’re interested, you can download the test project (365 KB).