CDS&E/Collaborative Research: A New Framework for Computational Model Validation

University Of Southern California, Los Angeles CA

Investigators

Abstract

Simulations play a key role in helping decision makers respond to complex societal problems, ranging from strategies for enhancing infrastructure resilience to natural hazards, to understanding financial markets, or managing air traffic. However, to be effective, individuals must select appropriate models and understand their limitations. This project conducts fundamental research to create a computationally efficient paradigm for model validation designed to guide decisions on model accuracy and validity. The resulting probabilistic framework will allow users to explore many families of models and to choose the ones that are the most useful. Advanced computational algorithms and architectures will enable the framework to leverage vast amounts of data from devices, platforms, sensors and systems and users to simultaneously evaluate multiple models. By developing an integrated framework of model falsification and Bayesian model selection, the project investigators give researchers and decision makers a powerful tool for effectively using models to understand and respond to emerging societal challenges and opportunities. Integrating Bayesian model selection with two model falsification phases forms a novel probabilistic framework for model validation: initial falsification eliminates unsuitable models that do not fit measured data, while final falsification selects the model classes with the most accurate simulations. New approaches to model falsification -- false discovery rate testing and model likelihood thresholds -- provide a cohesive framework that accommodates many types of dynamic response data. The approach is optimized and calibrated with testbed modeling problems such as NASA's benchmark for turbulence model validation and modeling a fully instrumented base isolated building. This work will transform the efficiency, accuracy and widespread applicability of model validation, allowing it to be used for previously intractable science and engineering applications. The researchers will explore modeling collaborations in promising areas including turbulence, material behavior, and biochemical reactions. The project will enhance awareness of model validation through education and outreach activities. Research results will be disseminated in journal publications and presentations at national and international conferences. Two graduate students will be mentored and research concepts and results will be incorporated into two graduate uncertainty quantification courses and an undergraduate engineering risk analysis class. The researchers will also incorporate research findings into their ongoing K-12 outreach efforts, including a model-building module targeting middle/high school STEM enrichment programs.