Paper-based Tangible User Interface as interaction in mixed reality

Abstract

In wearable mixed reality systems, hand interaction has several drawbacks of interact with virtual objects. In the thesis, we proposed the Paper Cube-based Tangible Interface (PCTI) which gave users an alternative to interact with a virtual object using a cheap and readily available tangible prop. We presented four different potential prototypes, and ultimately implemented the one that better met our requirements. During the implementation process, the image tracking technique that we employed in the interface could not let the interface achieve the expected performance. Therefore, we replaced it with optical tracking technology to improve the tracking accuracy. A comparative study was conducted to evaluate PCTI and hand interaction in HoloLens 2. The results revealed that PCTI on object manipulation was as efficient as hand interaction and provided a better user experience. Its usability performance was rated as good. Finally, we presented the current limitations of PCTI and had some suggestions for further improvement.