CHS: Small: Capturing Multisensory Interactions in Cutaneous Displays

Massachusetts Institute Of Technology, Cambridge MA

Investigators

Abstract

There has been growing interest in understanding how tactile displays that stimulate the skin can be used to expand the avenues available for communication for users of mobile devices and wearable technology. At present, such devices rely heavily on vision to communicate. Tactile features are increasingly being implemented in smart watches and Android devices where they provide greater opportunities for communication beyond simple alerts and notifications. These technological developments are supported by an expanding body of research on how wearable devices such as rings, watches, bracelets and earpieces can be designed to function as tactile or thermal displays. The appeal of these devices is that they are always in direct contact with the skin, the cues are private and the sites are readily accessible if a user needs to make a response. Although much of the research on incorporating tactile cues in smart devices has focused on employing vibration to communicate, other sensory cues from the skin such as pressure, temperature and skin stretch have been explored. This work has demonstrated the feasibility of incorporating tactile and thermal feedback in a small device, but has typically focused on comparing single inputs rather than on understanding how these various signals can be combined to create a richer sensory experience that also expands the bandwidth for communication. By selectively combining vibration, pressure, skin stretch and thermal cues in a display it is anticipated that the information provided to a user can be augmented and potentially offload the overworked visual and auditory systems. The advantages of using tactile-thermal signals for communication include the large area of skin available to present information and the effectiveness of such stimulation in capturing our attention. The application of this work extends beyond assisting users of mobile devices and wearable technologies to providing those with visual or hearing impairments who make use of the sense of touch to compensate for their sensory loss with additional possibilities for communication. There have been few studies that have systematically explored the benefits and challenges of combining thermal and tactile feedback in a single display. Much of the previous work with thermo-tactile displays has evaluated these channels as independent sources of information, rather than as concurrent, synergistic inputs. The contribution of thermal displays to multi-sensory systems has not been investigated, although they show promise in that thermal inputs are known to influence tactile sensory processing and so the possibility exists that the dimensionality of information can be enhanced in some situations by the addition of thermal cues. This project is focused on understanding how cutaneous signals generated by tactile-thermal interfaces can be designed to communicate efficiently with users, in contexts in which we know tactile communication is effective. A framework that details how different tactile and thermal inputs can be combined or used in isolation in displays should increase the interface designers' ability to choose among modalities and assign functions and types of information to the channel best suited for their presentation. Such displays will be mounted at different sites on the body depending on the particular application and tasks that are being performed. User performance will be evaluated by measuring the discriminability of thermo-tactile cues, the information transfer associated with changing the dimensionality of the cues, their identification with concurrent workload, and the preference of users for particular cue characteristics (i.e., preferred body location such as hand, forearm, thigh, etc). This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.