Blogs >>
ECOOP 2016
Sun 17 - Fri 22 July 2016 Rome, Italy
Mon 18 Jul 2016 11:35 - 12:05 at Alfieri - Session 2 Chair(s): Tomas Kalibera

Virtual Machines (VMs) with Just-In-Time (JIT) compilers are traditionally thought to execute programs in two phases: first the \emph{warmup} phase determines which parts of a program would most benefit from dynamic compilation; after compilation has occurred the program is said to be at \emph{peak performance}. When measuring the performance of JIT compiling VMs, data collected during the warmup phase is generally discarded, placing the focus on peak performance. In this paper we run a number of small, deterministic benchmarks on a variety of well known VMs. In our experiment, less than one quarter of the benchmark/VM pairs conform to the traditional notion of warmup, and none of the VMs we tested consistently warms up in the traditional notion. This raises a number of questions about VM benchmarking, which are of interest to both VM authors and end users.

Preprint (icooolps-final12.pdf)833KiB