CAREER: Learning from Interfaces for Simulation Environments to Design Wearable Technologies and Mixed-Reality Testbeds for Disaster Response Teams

New Mexico State University, Las Cruces NM

Investigators

Abstract

This project's goal is to support disaster response teams -- particularly urban search and rescue -- by designing wearable technologies that meet their information needs. Wearables for disaster response are promising because responders' hands are often busy and their awareness must be on their surroundings rather than a screen, making it hard to use common computing devices like smartphones, tablets, or computers. A number of prototype wearable technologies exist that might help: head-up displays can show data alongside the physical world; audio headsets can enable communication with both teammates and computers; armbands can provide touch-based feedback and allow gestures for communication and control; sensors can detect location, motion, and aspects of people's physical and mental state. However, usable wearable systems and design guidance for building them is scarce in general and even scarcer in the context of disaster response. The key insight behind the proposed work is that interfaces for training simulations and computer games make heavy use of head-up displays and specialized controllers; further, these often share elements of real disaster response scenarios. This project will study the features and effectiveness of these interfaces to generate design guidelines for wearable computing systems. Working closely with the Texas Task Force One response team, the project team will create custom-built wearable systems to support their mission as well as purpose-built mixed reality training simulations that combine virtual simulation with physical-world settings. The team will validate those simulations with the disaster response partners and use them to test and improve both the wearable technologies and the design guidelines the team creates. Both the work itself and the lessons learned will be used to improve classes at the lead investigator's institution around designing mixed reality technologies and human-computer interaction (HCI); the lead, and his institution, are also committed to broadening participation in computing education and computing research. The work will start with a deep study of training practices and needs for disaster response teams. The team will closely collaborate with its existing task force partners, using ethnographic observation of their existing training practices and interviews with task force leaders to develop models of task force training requirements and design considerations for wearable systems to support them. These will be disseminated both to disaster response teams and the HCI community. In parallel, the team will extract design best practices from existing training simulations and computer games, focusing on those that align with disaster response scenarios and needs. Team members will analyze the interaction techniques and mechanics these systems use -- how they provide situation awareness through ancillary displays, interface controls for managing large numbers of units and functions, and communication facilities for multiple participants -- along with people's preferences for and ability to use them. This will lead to a design catalog of existing interfaces and best practices for designing wearable and mixed reality interfaces that, along with the requirements identified through studying the task force, will inform the design of both wearable interfaces and testbed training simulations for disaster response contexts. These design activities will involve regular communication with task force partners to get feedback on using the interfaces in real situations, helping to align both the technology and simulation designs with real disaster response needs. Finally, the team will evaluate both the interfaces and the developed testbeds through a number of small-scale deployments and experiments, looking at how they support situation awareness, efficiency, and communication, as well as their effect on stress and workload. Much of the work will be used to support both courses and outreach activities; these include developing cases and projects based on the work for an HCI class as well as leveraging the designed technologies to improve a course on mixed reality. These will also be used to support a mixed reality interface development outreach activity with the local community college to expose students from underrepresented groups to both research and to work with advanced computing technology.