Lately, I was working again on graphviz-java and Java 10 was released too. A good opportunity to see what happened in the last two and a half years.
First, I tried to run the tests under the same circumstances as back then. (Checking out graphviz-0.0.2 from git and using JDK 1.8.31) I could reproduce the results more or less. V8 was around 130 times faster than Nashorn.
All charts in this article have a logarithmic scale, performance differences are bigger than they might seem!
Comparing these numbers with the results of current runs gave really strange effects. It’s better to change just one variable a time. So I used different versions of Viz.js and kept everything else the same.
Running Viz.js 1.8.1 takes about an order of a magnitude longer than Viz.js 1.0.0! Part of this seems to be to avoid errors. But still, quite a surprise. Running the same tests with Nashorn gave me the next surprise.
With Nashorn, running Viz.js 1.8.1 takes around half the time than Viz.js 1.0.0! Apparently, Nashorn and V8 have totally different optimizing strategies so that the changes to Viz.js make it run slower on one engine and faster on the other.
Trading speed for correctness/stability seems to be a common topic here. Using an older version of J2V8 gave a similar picture:
The older version of J2V8 is faster then the newer version. However, when I tried to use an even older version, J2V8 just crashed. When I tried J2V8 3.1.6 with newer versions of Viz.js, it also crashed. In this case, the increased stability was definitely worth the moderate loss of performance.
But back to the thing I wanted to test in the beginning. Did Nashorn become faster with JDK 9 and 10?
So now, we are ready for the last benchmark: Graal vs. V8:
I stay tuned.