Touchless mobile interaction on a smartwatch

Abstract

Smartwatches are being used more often and the market for them continues to grow. As a result, their sensors are constantly being improved, making it increasingly possible to control the device without touching the screen. Current interaction methods, such as tapping and swiping on the small touchscreen, can be imprecise, especially on the go. This creates a need for touchless interaction methods that offer more flexibility and accessibility in such scenarios. Additionally, touchless interaction differs from traditional gesture control in that it often relies on continuous tracking (e.g., arm or wrist movement) rather than discrete hand gestures. Our app, GesturePointerApp, can be controlled with a red circle (pointer), which moves across the screen according to movements of the arm and wrist. We conducted a study with the app to investigate how three different conditions-controlling the pointer while standing, while walking, and while walking with bags in both hands-affect smartwatch interaction and which types of gesture (e.g., tilting the forearm to zoom in or out, rotating the wrist to scroll a list up and down, etc.) are preferred in each context. Our results showed that participants considered walking with bags in both hands to be the most difficult condition. Furthermore, the majority of participants found the new touchless hover and selection methods to be a good alternative to existing touch interactions, such as tapping. For the selection method, users must hold the pointer over a bar for about 1.2 seconds to select the bar, and for the hover method, they moved the pointer over circles, which then display a number. Feedback on touchless map interactions was mixed; some participants saw them as a viable alternative to panning (navigation) and pinching (zooming) on a touch screen, while others did not. The map interactions involved the participant moving the map by moving the pointer, and after tapping the screen, they could zoom in and out, on the map, by tilting their arm. However, most agreed that they could be a useful supplement to touch interactions. For list interactions, such as scrolling the list by rotating the wrist or selecting an item by tilting the arm, participants generally preferred familiar touch-based methods such as swiping and scrolling. However, they noted that gesture-based alternatives could be useful in specific situations, such as when it is raining and the screen is wet or when wearing gloves. This thesis aimed to explore new interaction possibilities with a smartwatch, especially while on the move.