Compressed streaming of sparse images

Abstract

Virtual reality applications can use Multiplane Images (MPIs) to render their virtual scenes. Scenes rendered like this consist of multiple layers, each containing objects visible at its corresponding depth level. Since many depth layers can exist, the amount of data required to transmit even one frame of such a scene can be enormous. This work evaluates the gains in encoding speed and file size achieved by packing only the occupied regions of the layered sparse image into a virtual image and compressing these virtual image sequences using established video codecs such as H.264 and H.265, VP9, and AV1, with both hardware and software implementations, while maintaining decent visual quality. The different encoders are evaluated regarding their encoding speed, file size, and quality. This research suggests that the H.264 FFmpeg software encoder using a tile size of 64 provides the best trade-off among all assessed metrics.