Release the Kraken

September 14th, 2010

Editor’s note: On Sept. 14, Mozilla released a new JavaScript benchmark named Kraken. For more details, check out Rob Sayre’s announcement, reposted below.

We’re pleased to announce the first version of Kraken, a new browser benchmark. More than Sunspider, V8, and Dromaeo, Kraken focuses on realistic workloads and forward-looking applications. We believe that the benchmarks used in Kraken are better in terms of reflecting realistic workloads for pushing the edge of browser performance forward. These are the things that people are saying are too slow to do with open Web technologies today, and we want to have benchmarks that reflect progress against making these near-future apps universally available.

My Mozilla colleague Nicholas Nethercote pointed to Hennessy and Patterson’s “Computer Architecture” in the comments of a previous benchmark blog post on the state of benchmarks today. Hennessy and Patterson list five categories of benchmarks, from best to worst:

1. Real applications

2. Modified applications (eg. with I/O removed to make it CPU-bound)

3. Kernels (key fragments of real applications)

4. Toy benchmarks (eg. sieve of Erastosthenes)

5. Synthetic benchmarks (code created artificially to fit a profile of particular operations, eg. Dhrystone)

We think Kraken is a step in the right direction. Kraken will evolve quickly over the coming weeks and months as we build out its test suite and continue to push forward the capabilities of the open Web, as we make the workloads more realistic and varied. Of course, as with everything we do, Kraken will be free and open source and we welcome contributions and participation from the wider community.

Our testing indicates that Firefox 4 is currently more than 2.5X faster than Firefox 3.6 on Kraken 1.0. Try it out, and let us know how it works for you.