II-New: Flexible User Interaction Instrumentation for Ubiquitous and Immersive Computing Environments

University Of Maryland Baltimore County, Baltimore MD

Investigators

Abstract

Smart phones, networked devices, and commercially available augmented and virtual reality kits are making ubiquitous and immersive interactive systems increasingly commonplace. Designing these systems, and the applications that use them, requires the ability to study how system features and human behavior affect each other in natural contexts. This award will provide an interdisciplinary research team with equipment for capturing both individual and small group behavioral data in situ, including body motion, eye gaze, mental workload, and emotion. The infrastructure will support projects at the team's institution in a number of domains, including autonomous vehicle use by visually impaired users, augmented reality training for emergency medical responders, and collaborative scientific discovery in virtual visualization environments. A doctoral student with related research interests will coordinate management of and training on the infrastructure, developing both technical and research skills. The team will also use the equipment to provide enhanced research opportunities for undergraduates from a number of disciplines, institutions, and backgrounds. Much of the planned infrastructure uses next-generation versions of tools that the team already has expertise with. This reduces deployment risks and allows them to augment existing projects with new capabilities while enabling new directions for research. In most cases, this is a transformation from fixed lab-based sensing to unconstrained, mobile data collection in the field. The new capabilities include five main data sources. One is body motion data, which will be collected using an industry standard infrared-based motion capture system that can flexibly capture the movements of individuals or dyads. A second is location and gait capture, for both individuals and groups, through an unobtrusive, configurable system of floor-mounted force plates. A third is eye gaze data, collected through a portable headset that captures eye fixations and pupil dilation to support monitoring visual attention. A fourth is electroencephalogram (EEG) data, collected through a portable dry sensor-based headset, that can monitor workload, affect, and facial features as well as support prototyping of brain-computer interfaces (BCIs). A fifth is physiological data including temperature, pulse, arm motion, and arousal, collected through a inconspicuous wristband that includes photoplethysmography, accelerometer, and electrodermal sensors. Each individual data stream comes with accompanying analytic software; collectively, data will be managed through a commercially available tool for analyzing events synchronized across multiple parallel data streams. Key innovations of this award this award are its novel data integration strategies and cross-disciplinary application areas.