CCSS: Uncertainty-Aware Computational Imaging in the Wild: a Bayesian Deep Learning Approach in the Latent Space

Suny At Albany, Albany NY

Investigators

Abstract

From microscopy and tomography to biometrics and surveillance, computational imaging (CI) has greatly expanded mankind’s vision capability in various scientific and engineering fields. In contrast to hardware-based solutions that require expensive optics, data-driven or computational approaches powered by deep learning have fueled the development of AI-enabled computational imaging systems. The emerging paradigm of Bayesian deep learning endows the next-generation CI systems more flexibility to handle various uncertainties in the wild – no matter whether they are related to the adversarial conditions of image acquisition (e.g., moving platform or bad weather) or unintentional mistakes made by humans (e.g., the mal-functioned main mirror of Hubble Space Telescope after the launch into the space). Developing uncertainty-aware CI systems have a wide range of impact on both scientific exploration (e.g., imaging at extreme scales such as microscopic and astronomical) and our daily lives (e.g., smartphone applications). This project aims at taking a Bayesian deep learning (BDL) approach to modeling uncertainty in real-world imaging scenarios. An important new insight brought about by this project is to unify the flow-based uncertainty kernel estimation (for likelihood modeling) with memory-based uncertainty image generation in latent space (for prior modeling). Under a fully Bayesian framework but using reparameterization to simplify the modeling process, the proposed BDL greatly facilitates both degradation learning and image reconstruction in the realistic scenario when uncertainty is inevitable. The proposed research consists of three tasks: 1) Flow-based Nonuniform Kernel Estimation in the Latent Space; 2) Memory-enhanced Deep Generative Models for Image Reconstruction; and 3) Real-world Applications in Deep Tissue Imaging such as Superresolution Microscopy. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.