ITR: Environment Management for Hybrid User Interfaces

Columbia University, New York NY

Investigators

Abstract

This is the first year funding of a three-year continuing award. Hybrid user interfaces combine together multiple displays and interaction devices to benefit from the advantages of each. For example, a hybrid user interface could be constructed in which multiple users view one or more common displays, such as a wall-mounted data visualization and a desk-top virtual workbench 3D model. At the same time, each user might see complementary private material, customized to her own information needs, and overlaid on and registered with the common displays---an augmented reality that is presented on personal, tracked, hand-held or head-worn see-through displays.This project addresses environment management, the task of managing large numbers of virtual objects on large numbers of displays in hybrid user interfaces. More complex than the tasks involved in current window management, environment management will be especially challenging if it is to address the needs of future mobile, collaborating users, whose proximity to other users, displays, and interaction devices may change rapidly and unpredictably as users move about. This work explores an alternative to direct manipulation approaches, in which knowledge-based environment management tools take over many low-level tasks to avoid overwhelming the user. The goal is to increase a user's effectiveness by making it possible for her to exert higher-level control over the layout and contents of her personal and shared work environment. This project will develop the underlying concepts for hybrid user interfaces and effective environment management facilities, and will design, demonstrate, and test research prototypes that embody these concepts. Special emphasis is placed on issues raised by collaborative, 3D augmented environments that exploit a wide range of displays (held, worn, and stationary), including head-tracked see-through displays that create augmented realities, in which virtual objects coexist in the same surrounding space as users and other physical objects. User interface design approaches that can promote shorter task performance time, lower error rate, or greater user satisfaction, will be vital for improving our ability to interact with information and with each other in the nontraditional world of collaborative mobile computing.