CRII: CHS: Concept-Driven Visual Analysis

Indiana University, Bloomington IN

Investigators

Abstract

Tools that create visualizations, or visual representations of large datasets, are increasingly important for making use of data in a number of domains from commerce to science. To date, most visualization tools have been designed to support open-ended exploration of patterns in data; though useful for some tasks, this exploratory model does not fit well when analysts have existing models or hypotheses. This project aims to support a "concept-driven" analysis style in which analysts can share their existing conceptual models with the system, which uses those models to generate visualizations that allow the analyst to explore places where the models and data disagree and develop revised models that reconcile those discrepancies. To do this, the research team will design a number of prototype techniques for communicating conceptual models, algorithms for selecting visualizations and data features that best match those models, and interfaces that highlight discrepancies and provide tools for analysts to dig into the data around them. If successful, these concept-driven analyses will provide better ways for scientists and other analysts with existing models to leverage data while reducing the risk of confirmation biases in which people choose analyses that don't show where their existing models are wrong. The project will also enable the research team to learn more about the ways people come to form and express expectations about data. Lastly, project will provide opportunities for graduate research training as well as tools to support K-12 outreach workshops that introduce younger students to data science. The project has two main activities. The first involves prototyping three elicitation techniques that prompt users to externalize their mental models and expectations about a dataset: free text expressions combined with natural language processing techniques that extract both variables of interest and implied relationships between them; concept mapping tools that allow users to graphically express relationships between entities, ideas, and concepts as node-link diagrams in which the nodes represent key aspects of the data and links represent suspected relationships between them; and tools for sketching expected relationships between variables using existing visualizations such as line charts and heatmaps. The team will also develop interfaces that encourage analysts to develop several alternative models to reduce the chance of confirmation bias. The second main activity is using the captured models to generate relevant visualizations that support discrepancy exploration. To do this, the team will first use a taxonomy of best practices for choosing visualizations that best fit the concepts and relationships represented in the models. They will then design interfaces that highlight discrepancies in both the visualizations (for instance, by highlighting data that badly fits a model) and the models (for instance, by highlighting links in a concept map that are not supported by the data) to call attention to inconsistencies. Both the elicitation and feedback interfaces will be refined through a series of semi-structured visual analysis studies in which participants use them to analyze data in domains of general interest such as socioeconomic indices, crime statistics, and health risks. The refined versions will then be used to compare the effectiveness of the concept-driven approach with more traditional exploratory approaches, as well as against both structured and unstructured workflows that interleave exploratory and concept-driven elements, in a series of lab studies using participants drawn from a number of scientific disciplines and a case study with scientific partners at Argonne National Laboratory. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.