Michal Sojka, Czech Technical University in Prague
Stefan Wächtler, TU Dresden

Computers are often used to perform various measurements. Benchmarking and hardware performance evaluation is a typical example. In this kind of measurement the computer is used to measure the time of certain operation. The precision of hardware clocks in typically more than satisfying for this purpose but the measured time is also influenced by the operating system. In this paper, we evaluate how does the precision of measurements depend on the operating system.

We developed a set of benchmarks to measure the characteristics of memory subsystem, the cost of task preemption and the cost of task migration between CPU cores/sockets. We run the benchmarks on NOVA microkernel as well as on two different configurations of Linux, in all cases on x86 architecture. We captured the data from several tenths of hours of running experiments and compare the obtained results. The measured values are in all cases very similar, because they correspond the properties of the hardware. In addition to the absolute values, we also compare the variance calculated from multiple (typically 1000) runs of the experiments. The results vary depending on the experiment, but there is a clearly visible trend in all experiments - the microkernel offers the lowest variance, Debian kernel sits in the middle and the highest variance is produced with a minimalistic Linux kernel (allnoconfig manually enabled the needed options). The results seem realistic and in the paper we try to quantify the obtained differences.