RI: Small: Learning to See Through Atmospheric Turbulence

Purdue University, West Lafayette IN

Investigators

Abstract

For a variety of long-range imaging systems in autonomous vehicles, surveillance, and defense, restoring images that are distorted by atmospheric turbulence is inevitable. However, unlike the better-known image restoration problems such as denoising and deblurring, recovering turbulence distorted images is considerably more difficult because of the physics involved. On one hand, the image formation process due to a turbulent medium is described by a sequence of wave equations of diffraction and phase distortion. The lack of a simple forward model makes the inverse problem difficult to formulate and solve. On the other hand, while deep learning algorithms have produced promising results in many disciplines, the disparity between these generic models and the specific turbulence physics makes the resulting methods lack generalizability, explainability, and robustness. The goal of this proposal is to bridge the gap between turbulence physics and deep learning algorithms. The approach is to ground the algorithmic designs on physics by developing new forward models, reconstruction algorithms, training schemes that improve consistency, and benchmark evaluation. By improving the image restoration capability, the project will enable a wide range of imaging applications and software products that, in turn, improve object detection, biometric analysis, and navigation. For mission-critical applications such as defense, the integration of physics and algorithms will provide more consistent and trustworthy information for decision-making. The project also trains next-generation imaging scientists that will provide the necessary workforce to the United States. To accomplish the goal of the project, four objectives will be pursued. (1) To develop a new forward model that has low complexity, adheres to physics, and is differentiable in the sense of backpropagation. The new model will fill the critical need for a viable turbulence simulator that can generate data at a large scale for training and testing. (2) To develop a new image restoration algorithm by integrating the forward model, lucky imaging, and end-to-end neural networks. Specifically, a new strategy for feature matching in the presence of turbulence and noise will be developed, and inverse optimization will be formulated via the concept of unrolled neural networks. It is anticipated that the new techniques will enable the imaging of small and moving objects. (3) To develop a new training scheme that improves the consistency of the algorithm from one turbulence condition to another, by optimally allocating the training samples according to the turbulence strengths. (4) To establish a benchmark evaluation system by building controllable experimental setups and collecting real data. On the education front, the project aims to promote the exchange of knowledge across physics and deep learning by developing tutorials in major computer vision and optics conferences; disseminating educational materials to the general public through classes and books; delivering codes and datasets to support reproducible research. The project will promote STEM education by offering image processing and machine learning to high school students. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.