Multicore OS benchmarks: we can do better

Current multicore OS benchmarks do not provide workloads that sufficiently reflect real-world use: they typically run a single application, whereas real workloads consist of multiple concurrent programs. In this paper we show that this lack of mixed workloads leads to benchmarks that do not fully exercise the OS and are therefore inadequate at predicting real-world behavior. This implies that effective multicore OS benchmarks must includemixed workloads, but the main design challenge is choosing an appropriate mix. We present a principled approachwhich treats benchmark design as an optimization problem. Our solution leads to a workload mix that uses as much of a system's resources as possible, while also selecting applications whose performance is most sensitive to the availability of those resources.