Computation of 3D audio using ambisonics for an immersive audio-visual 3D experience

Abstract

In this thesis an immersive audio-visual 3D experience was created in Unity using VR. This experience is based on high quality spatial audio and 360° video recordings. In order to process the audio without loss of quality, it was encoded into the Ambisonics surround sound format from which it was decoded to an optimal loudspeaker layout. Thus, the mathematical background for Ambisonics is presented in this thesis. The processing was done using the software REAPER and the IEM plug-in suite for Ambisonics. Additionally, since the audio and video had been recorded separately, they were synchronized not only in time but also in angle using a beamforming technique. The processed audio was combined with the synchronized video in a VR experience with gamification elements, which were created in Unity with the help of the Windows Mixed Reality feature packages. Finally, an experiment showing the quality of the angular synchronization was performed.