External tracking for head-mounted displays

Abstract

In many applications based on head-mounted displays (HMD), external devices are employed to identify and track specific objects within the surrounding environment. One such application is the tracking of faces within the surrounding environment. One potential benefit of this application is the development of user interfaces (UI) that adapt to the recognized faces. As most HMDs are equipped with a camera, it is feasible to integrate face recognition into the system. The Company Meta imposed restrictions on the functionality of the Quest 3’s camera for external developers, thereby precluding the use of the internal camera for facial recognition. In this work, a system was developed that utilizes a smartphone (Samsung Galaxy S24 Ultra), mounted on the Quest via a 3D-printed holder, to recognize faces within the surrounding environment. Subsequently, the positional data is transformed for the Quest to facilitate the optimal marking of the faces. In regard to data transmission, users may select either an Internet or Bluetooth connection. The results of the system are encouraging, demonstrating the successful recognition and transmission of faces. Nevertheless, the insufficient accuracy in certain instances when identifying faces represents a compelling motivation for the ongoing advancement of the system.