CHS: Small: Auditory and Haptic Based Brain-Computer Interfaces Using In-Ear Electrodes

Georgia Tech Research Corporation, Atlanta GA

Investigators

Abstract

Brain Computer Interfaces (BCIs) are systems that allow people to control computers and other devices with brain signals alone. BCIs have the potential to improve the lives of many individuals with severe physical disabilities, by allowing them to communicate and control their environment without needing muscle movement or voice. After more than two decades of research, BCIs have become robust and reliable enough to consider them for mainstream applications, such as hands-free and voice-free control of devices. However, the most common BCIs require visual attention, which restricts their utility for people with visual impairments, or for mobile environments (such as driving) where diverting visual attention is dangerous or not possible. Intriguing alternatives to visual displays are auditory (sounds) or haptic (touch or sensations such as vibrations) based BCI interfaces, or multimodal BCI interfaces which are a combination of these. In addition to providing an alternate interface for people with visual impairments or for special purposes, nonvisual BCI control of devices could also be useful in everyday life, such as when responding to a text message in a movie theater without looking at a device screen or speaking. This project will extend the state of the art in BCIs by evolving the body of knowledge in auditory, haptic, and multimodal stimuli, which are relatively unexplored areas of the field. Additionally, the research will create and explore small, wearable in-ear electrodes to detect brain signals, and thus will contribute to the emerging field of mobile BCIs which will open possibilities for large numbers of mainstream users. Current BCI systems cover a wide and varying range of brain signals and recording technologies; this research focuses on "evoked-response" electroencephalograph (EEG) approaches, that is to say brain signals that are triggered due to a stimulus such as a sound, flashing light, or touch. To these ends, the project will study auditory and haptic cues to determine the best ways to map them to input choices. One of the biggest challenges with auditory and haptic interfaces is how to label a stimulus so it is meaningful to the BCI user. To address this problem, the project will explore novel methods of encoding the labeling and mapping of auditory and haptic stimuli into the stimuli themselves (in a manner analogous to how it is possible to label visual stimuli, such as when the target is a flashing letter so the user can determine the meaning of a cue by looking at it). The plan is to leverage research in sonification (which largely focuses on presenting data in auditory "displays") to devise techniques for encoding speech or patterned tones (such as Morse code) into audio cues in such a way that a user can simply listen to the cue to determine its meaning. The team will also experiment with a variety of multimodal approaches (combining visual, auditory, and haptic cues in a single system) in order to achieve higher accuracy than is possible with a single stimulus mode alone. Finally, the project will evaluate the effectiveness of in-ear EEG electrodes in detecting brain responses to auditory and haptic stimuli, experimenting with electrode design, placement within the ear, and various filters and classifiers to improve the signal-to-noise ratio. The results of the experiments in alternative stimuli will be combined with the optimized wearable electrode system, to create the first hands-free, voice-free, vision-free interfaces for mainstream users.