Browsing by Author "Steuerlein, Benedict"

Now showing 1 - 1 of 1

Open Access
An investigation into the simulation of capacitive fiducial markers using deep learning
(2021) Steuerlein, Benedict
Tangibles have shown to enrich the interaction space on touchscreens already in the early 2000s. On early tabletop installations, camera-based systems were used for tracking tangibles. On mainstream projected-capacitive screens, the ability to recognize objects other than fingers is limited. Lately, researchers have utilized deep learning to bring back the capabilities of recognizing conductive tangibles on capacitive screens. The main drawback of sufficiently working neural networks is that they require huge amounts of data for training, domain-specific knowledge for hyper-parameter tuning, and are often single-purpose networks. With this thesis, we propose a toolkit that allows designers and developers to train a deep learning recognizer that is purely trained on simulated data. Our toolkit makes use of a pre-trained Conditional Generative Adversarial Network that, based on sketches of the footprint of conductive tangibles, simulates the corresponding capacitive representation. Furthermore, we use this simulated data to train a deployable recognizer network. Therefore, using our toolkit, designers require no domain knowledge or need to collect data. Our evaluation shows that our approach can reliably recognize conductive fiducials with an average accuracy of 99.3% with a recognizer network solely trained on simulated data. Additionally, our recognizer architecture can predict the tangible's orientation with an average absolute error of 4.8°.