Pervasive interaction across displays

Abstract

Digital screens are becoming more and more ubiquitous. Resolution and size are increasing, and, at the same time, prices for displays are falling. Large display installations are increasingly appearing in public spaces as well as in home and office environments. We expect this trend to continue, making wall-size displays commonplace in the next decade. With this development, all three classes of devices described by Mark Weiser - pads, tabs, and boards - will be mainstream. Pads (tablets), tabs (smartphones), and boards (displays) let us show and interact with data in different situations, because each device class is optimized for a certain use case. Consequently, the use of multiple devices becomes common—for example, the use of second screens while watching TV is becoming the norm. However, the use of multiple devices requires seamless transitions between devices, mechanisms for exchanging data, and the ability to move content from one device to another and to remotely access or control the data. Back in 1998, Michael Beigle and his colleagues proposed dynamically and automatically distributing Web-based content to different output devices in a smart environment. A few years later, Roy Want and his colleagues suggested using interfaces in our environment to interact with our personal data. Because mobile devices or notebooks often provide only a small screen for output and limited input techniques, they proposed using office screens or public displays to create a more enjoyable user experience. They also argued for having physical access to private data. These examples highlight that research in ubiquitous computing was already early on exploring interaction across pervasive devices, displays, and content. Current products support both visions. On one hand, there are devices that provide options to present remote data on a screen in the environment with the control residing on the mobile device. On the other hand, there are means to easily present content from mobile devices on remote displays. There are now also many cloud-based products for interacting with data on multiple devices. For example, Dropbox provides access to all text documents and images. Spotify lets you enjoy your favorite music on smartphones, tablets, notebooks, and music systems. Furthermore, people are starting to use mobile devices as remote controls for large screens, smart TVs, or music systems. All these examples show that streaming and connecting different devices ubiquitously are key technologies for smart environments. Here, we present a few commercially available technologies supporting this and provide an outlook on how displays might become a service themselves.