When I wrote graphviz-java, I also used it as a benchmark to compare the execution speed of Chrome’s V8 and JDK’s Nashorn javascript engines. The result was clear: V8 was one to three orders of magnitudes faster than Nashorn.

Lately, I was working again on graphviz-java and Java 10 was released too. A good opportunity to see what happened in the last two and a half years.

First, I tried to run the tests under the same circumstances as back then. (Checking out graphviz-0.0.2 from git and using JDK 1.8.31) I could reproduce the results more or less. V8 was around 130 times faster than Nashorn.

All charts in this article have a logarithmic scale, performance differences are bigger than they might seem!

Comparing these numbers with the results of current runs gave really strange effects. It’s better to change just one variable a time. So I used different versions of Viz.js and kept everything else the same.

Running Viz.js 1.8.1 takes about an order of a magnitude longer than Viz.js 1.0.0! Part of this seems to be to avoid errors. But still, quite a surprise. Running the same tests with Nashorn gave me the next surprise.

With Nashorn, running Viz.js 1.8.1 takes around half the time than Viz.js 1.0.0! Apparently, Nashorn and V8 have totally different optimizing strategies so that the changes to Viz.js make it run slower on one engine and faster on the other.

Trading speed for correctness/stability seems to be a common topic here. Using an older version of J2V8 gave a similar picture:

The older version of J2V8 is faster then the newer version. However, when I tried to use an even older version, J2V8 just crashed. When I tried J2V8 3.1.6 with newer versions of Viz.js, it also crashed. In this case, the increased stability was definitely worth the moderate loss of performance.

But back to the thing I wanted to test in the beginning. Did Nashorn become faster with JDK 9 and 10?

JDK 9 takes around 60% the time JDK 8 needed. A nice gain. But interestingly, JDK 10 lost about half of this improvement again. I cannot even guess the reason for this, but JDK 11 is coming soon… The mythical Graal (more about it here) compiler/runtime on the other hand is three times faster than JDK 8! JDK 10 started including parts of it with JEP 317, but the Javascript engine is still the old one.

So now, we are ready for the last benchmark: Graal vs. V8:

V8 is only about twice as fast as Graal! Knowing that V8 is one of the fastest javascript engines, this is a great result and makes me wonder what the future will bring.
Not only for javascript and Java itself but for the JVM as a platform as Graal supports also Ruby, Python and R and seems to provide already a very competitive performance for them too.

Anyways, measuring is difficult, computer speed measuring more so, and benchmarking javascript engines even more. It seems that V8 has reached some performance plateau where improvements are only slow anymore. And it’s nice to see that the JVM is still pushed forward with new ideas to become an even better platform.

I stay tuned.