Wasm-R3 : creating executable benchmarks of WebAssembly binaries via record-reduce-replay

Abstract

WebAssembly is the newest language to arrive on the web and has now been implemented in all major browsers for several years. It features a compact binary format, making it fast to be loaded, decoded and run. To evaluate and improve the performance of WebAssembly engines, relevant executable benchmarks are required. Existing benchmarks such as PolyBenchC and Spec CPU have shortcomings in their relevancy, since they do not necessarily represent real-world WebAssembly applications well. To make the creation of such benchmarks faster and simpler, we develop Wasm-R3 an approach that has the capability of recording existing web applications and generate an executable benchmark from it. Wasm-R3’s workflow can be described in three phases: record, reduce and replay. In the record phase the instrumenter instruments the website’s WebAssembly code, a user then interacts with the website, which causes traces of the execution to be recorded. Since these traces are typically large, unnecessary information gets filtered out in the reduce phase. In the replay phase a replay generator takes these traces along with the original web application’s WebAssembly binary and generates a standalone executable benchmark from it. We evaluate Wasm-R3 by implementing it in Typescript and Rust to show that the generated benchmarks correctly mimic the behavior of the recorded application. We further demonstrate that replays can be generated in reasonable time by measuring a mean wall time of 8.651 seconds and that our benchmarks are portable across a variety of di↵erent WebAssembly engines.