HCC: Human-Computer Interaction and Brain Measurement Using fNIR Spectroscopy

Tufts University, Medford MA

Investigators

Abstract

There has been much previous research in the general area of brain-computer interfaces (BCI), which has primarily aimed to help people with severe motor disabilities interact with their environment by translating their brain activity into specific device control signals. Users who are completely paralyzed ("locked-in") or those who lack muscle control (e.g., people with cerebral palsy or stroke victims) can use BCIs to answer simple questions, to control their environment, and for word processing. But the variety of brain imaging techniques that have been tried to date for BCIs have all suffered from serious drawbacks, so that communicating through such systems is currently time consuming and mentally demanding. Open research challenges concern the accuracy of BCIs (systems often misinterpret a user's intentions), and the information transfer rate of such systems (which is often too slow for use in real world settings). In this project, the PIs will build on their experience in designing, implementing, and evaluating non-command, adaptive user interfaces (e.g., based on eye movement), to advance BCI by bringing to bear emerging technology for measuring brain activity using functional near infrared (fNIR) spectroscopy coupled with machine learning to analyze user data in human-computer interaction. While fNIR technology is still in its infancy, the PIs expect its use as a real-time input to an adaptive interface to break new ground. The PIs believes that this combination will also lead to new, more objective methods for evaluating next generation interaction styles for HCI in general. The PIs' approach is a natural extension of and will exploit their prior work relating to the concept of reality-based interaction, which focuses on the ways that new interaction styles exploit the user's pre-existing skills and expectations from the real world more than trained computer skills, while at the same time helping to differentiate mental effort devoted to interface-related or syntactic aspects from that devoted to the underlying task or semantic aspects. Broader Impacts: This project will develop technologies of immediate benefit to users with motor disabilities. Additionally, the work will open up a new area of HCI research (namely, relating to interfaces that can dynamically adapt in real time to the user's cognitive load), and will advance the theory and evaluation of interaction styles in general.