III: Medium: Collaborative Research: Situated Visual Information Spaces

Harvard University, Cambridge MA

Investigators

Abstract

The aim of this project is to enable people to effectively visualize information about the world in augmented reality. Augmented reality is potentially the next big social benefit from computer technologies, because it allows visual information to be embedded - or ‘situated’ - into the real world. This allows people using smartphones and smartglasses to see data around them in the correct real-world context. However, unlike when visualizing data on a regular computer or smartphone display, where a designer has complete control over how the application looks and feels, augmented reality visualizations are inherently overlaid on the real world. As such, visualizations must be capable of reacting to different real-world environments including dynamic scenes, and for there to be design recommendations that say how visualizations should react to different environments. This project will scientifically investigate visualization for augmented reality, study the efficacy of different approaches, create design recommendations, and then build a software system that can apply these recommendations to help design and run effective visualization applications. The proposed approach will be experimentally validated in the sports and healthcare domains. Situated visual information spaces fuse the digital information world with the physical world of objects, people, locations, and environments using augmented reality. To realize this, three scientific and design challenges will be tackled: (1) Situated visualization, interaction, and collaboration, which requires intuitive in-situ data visualizations, physical and digital interfaces for natural user interactions, and schemes for collaboration in augmented reality. Novel situated visual embedding methods will be studied for spatial and non-spatial data in dynamic environmental and situational contexts. These visualizations will automatically adapt to the physical environment, digital entities, users, and tasks while using perceptually and cognitively effective methods that do not overwhelm the user. (2) Design via constraints, where software reduces the increased complexity of creating visualizations that adapt to real-world environments. This software is aimed at visualization designers and evaluates guidelines as constraints, then balances these to provide recommendations for appropriate data and designs for the current environment. (3) Situated applications, where two wellness applications in healthcare and sports will be developed and evaluated in partnership with respective domain experts. Within them, these domains cover a spectrum of different techniques, tasks, and users. These applications will help to define an achievable research scope, drive it with motivated stakeholders, and present best-practices via use cases. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.