CAREER: Neural Computational Imaging - A Path Towards Seeing Through Scattering

University Of Maryland, College Park, College Park MD

Investigators

Abstract

Imaging through scattering is arguably the most important open problem in applied optics. If one could overcome scattering and other optical aberrations, one could (a) see through tissue to observe "biology in action" at cellular scale; (b) see through fog, smoke, and inclement weather to safely navigate in adverse conditions; (c) see through the atmosphere to allow ground-based telescopes to outperform James Webb for a fraction of the cost; and (d) see through thin fiber bundles to enable minimally invasive endoscopy. The physics of scattering is now relatively well-understood and the obstacles to effectively image through scattering are primarily computational in nature: Existing algorithms cannot efficiently disentangle measurements of scattered photons to recover the underlying structure of a time-varying three-dimensional scene hidden behind a scattering medium. This project develops a collection of signal processing and machine learning innovations to broadly address this challenge. This project also develops a portable imaging-through-scattering demonstrator that will be used to engage high-school and undergraduate students in STEM. It also supports the development of a new hands-on cross-disciplinary undergraduate computational-imaging course that will improve US workforce development. The overarching goal of this project is to develop a "neural computational imaging" framework capable of solving challenging, high-dimensional, non-stationary computational-imaging problems. The framework consists of three key innovations: First, functional neural signal representations will be used to capture and exploit a signal's low-dimensional structure and temporal regularity without explicit models. Second, neural forward operators will provide an interpretable, computationally efficient, and easy-to-calibrate approach to model non-stationary imaging inverse problems. Finally, self-supervised learning will be used to extract data-driven priors in imaging applications where ground-truth images/signals are not available. Collectively, these innovations are expected to provide breakthrough imaging-through-scattering capabilities for use in national defense, remote sensing, robotics, astronomy, microscopy, endoscopy, pathology, and other applications. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.