I-Corps: Video Based Interface for the Live Labelling of Emotional Responses

New York University, New York NY

Investigators

Abstract

The broader impact/commercial potential of this I-Corps project is to provide an affordable, wearable device capable of detecting emotional states in real time. Currently, there are few methods to objectively evaluate emotional states in real time and in real-life settings. The proposed software and hardware platform can potentially be used by entertainment venues to enhance the customer experience to increase sales, improve venue management and safety, and support decision making in scheduling. The system can also be used to provide real time measurements of audience emotion and assess content preferences. This I-Corps project develops a software and hardware platform centered on a wearable device for the unconstrained, contactless and live detection of physiological correlates of peak emotion. The technology uses individualized signals along with video-sourced data to provide real-time measures of consumers' emotional state. The project develops an approach combining innovative data collection and analysis strategies with cutting-edge software and affordable, scalable hardware for contactless, image-based detection of emotional states in real-time. The technology provides a wireless, easy-to-deploy, and wearable device capable of intelligent processing with limited power, well-suited for large scale data acquisition in real-world environments. Finally, this project also innovates in data science by combining real-time physiological signals from individual consumers with aggregate video-recordings in order to model group engagement and predict behavior. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.