Accented character entry in XR

Abstract

Entering accented characters on physical keyboards presents considerable challenges in computing environments, requiring users to memorize complex key sequences that disrupt typing flow, particularly in multilingual settings. While research on efficient accented character entry remains generally limited, this thesis explores Augmented Reality passthrough environments as a potential solution to these challenges. AR video passthrough provides advantages by preserving direct visual contact with physical keyboards and hands while enabling virtual interface augmentation. This strategy takes advantage of users’ existing familiarity with physical keyboards and maintains the tactile feedback that typists rely upon, while allowing for enhanced interaction through spatial positioning of virtual interface elements. This research employed a two-phase experimental approach using a Meta Quest Pro headset with integrated eye tracking. The interaction technique requires users to press Alt plus a base letter to display accent options, direct their gaze toward the desired accented character in the displayed toolbar, and confirm selection through various mechanisms. A pilot study evaluated three confirmation mechanisms: KEYPRESS CONFIRMATION (secondary keypress), KEY RELEASE CONFIRMATION (key release triggers selection), and EYE BLINK CONFIRMATION. Based on pilot results favoring KEYPRESS CONFIRMATION for its reliability and user preference, the main study used this eye-gaze selection with KEYPRESS CONFIRMATION strategy to investigate three spatial positioning strategies for accent selection toolbars: KEYBOARD-PROXIMATE (near physical keyboard), TEXT-FIELD (adjacent to input area), and GAZE-RESPONSIVE (appearing at user’s gazelocation). Results demonstrate significant performance advantages for GAZE-RESPONSIVE and TEXTFIELD positioning over KEYBOARD-PROXIMATE placement. Accent character typing speed improved substantially for both preferred strategies compared to KEYBOARD-PROXIMATE positioning. System usability scores were significantly higher for GAZE-RESPONSIVE and TEXT-FIELD strategies. GAZE-RESPONSIVE positioning received the strongest user preference, while KEYBOARD-PROXIMATE positioning was consistently rejected as least preferred. These findings provide empirical evidence for optimal toolbar placement strategies in AR text input systems and contribute to future development of multimodal interfaces combining physical keyboards with eye-gaze selection.