CAREER: Generative Physical Modeling for Computational Imaging Systems

University Of Texas At Austin, Austin TX

Investigators

Abstract

Imaging devices, from microscopes to medical-imaging scanners, have transformed science and diagnostic medicine by providing safe and noninvasive techniques for observing the environment and seeing inside the body. However, imaging-system design choices are often based on idealized operating conditions, resulting in highly promising "benchtop demonstrations" that quickly degrade when deployed outside of a controlled laboratory environment. This project aims to develop a framework for robust computational-imaging system design, where the data acquisition and data processing are jointly designed in tandem to address the mismatch between the idealized performance of physical systems and their real-world behavior. The research aims to enable reliable imaging in dynamically evolving clinical and scientific-research settings, for example by reducing acquisition times, imaging moving objects, and compensating for system imperfections. The project also involves the dissemination of results through tutorials, webinars, and reproducible code, as well as community-outreach initiatives based on hands-on interactive demos of computational-imaging systems with the goal of increasing participation in engineering and science. The objective of this project is to develop a machine-learning framework for robust computational-imaging system design with principled methods and theoretical performance guarantees. Central to the approach is the separation between modeling the physical system, which is governed by established imaging physics, and modeling the statistical prior knowledge, which is learned from imaging data. The project involves four research thrusts, each intended to tackle a specific problem foundational to computational-imaging system robustness and reliability: 1) adapting to dynamically changing operating conditions; 2) accounting for motion during image acquisition; 3) learning directly from noisy, sub-sampled measurements; and 4) improving resiliency to system imperfections. The project is founded on a fruitful synthesis between imaging physics, signal processing, optimization, and machine learning. Image acquisition and recovery will be formalized using newly developed deep generative physical models instead of poorly understood and non-generalizable black-box deep-learning methods. In addition to impacting applications including microscopic, medical, and automotive imaging, the work will also inform the design of non-imaging systems, such as in wireless communications, where similar challenges exist surrounding the deployment of deep-learning-based algorithms. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.