April 7th, 2010
We were hesitant to change the SunSpider content or harness much at all, since it’s been used for cross-version and cross-browser comparisons for so long. But it seems important to address the most significant issues. With that in mind, we have made a limited number of changes to the test harness and to the test content itself.
Dealing with Power Management
This resulted in our first SunSpider harness bug fix: bug 32505: In-browser SunSpider suffers excessive penalty under power management. In addition to lowering the delay between tests, we also greatly reduced the amount of loading that happens during the benchmark. All of the content is in one page. This further reduces the variability between test runs. And as an added bonus, the SunSpider test suite is now much faster to complete, as it no longer waits half a second between every single test.
Test Content Bug Fixes
SunSpider’s test content has some oddities. Out of the various problems reported, two stood out as affecting the accuracy of the test to some extent. First, the
string-base64 test had a bug that made it incorrect in Internet Explorer. The test has been updated, and should now give more accurate results in IE. Second, the
regexp-dna test had an issue specific to Gecko-based browsers such as Firefox. It too has been updated and should now be more accurate in Firefox.
Versioned Test Content
Since we changed the test content, we thought that some might be interested in running the “classic” SunSpider 0.9 content set under the new harness. We’d also like to be prepared to handle multiple versions of the test content in general, so we did just that. For this release, we made sunspider-0.9.1 the default content set (both command-line and hosted).
Minor Test Harness Tweaks
In the course of implementing these fixes, we decided some cleanup was in order. First, we made sure all of the HTML files in the browser-hosted version are valid HTML5. With HTML5 advancing and becoming more popular, it seemed like the right thing to do. The test will still work fine in older browsers. In addition, the older SunSpider harness processed JSON strings using somewhat ad-hoc techniques. We updated to use json2.js or browser-native JSON.parse, where available.
Try it Out
Last but not least, try it out, report your scores, any problems you run into, or any other interesting comments. You can run it with the latest content set right from that page, or visit the all versions page if you want to test an older content set.