Collaborative Research: CIF: Medium: Taming Deep Unsupervised Representation Learning in Imaging: Theory and Algorithms

Michigan State University, East Lansing MI

Investigators

Abstract

Deep neural networks are driving significant breakthroughs across engineering and scientific applications. Their success is often predicated on the availability of large and high-quality data sets. However, for many inverse problems or classification problems such as in computed tomography (CT), magnetic resonance imaging (MRI), and cryo-electron microscopy (cryo-EM), obtaining paired training data can be extremely difficult or expensive. The training data may also be limited or highly corrupted. While recent progress addresses these challenges via methods such as deep image prior or self-supervised learning, the training procedures therein are often ad hoc and the underlying mechanisms behind the approaches are far from well-understood, which are due to the lack of precise mathematical modeling and the lack of understanding of learned representations. To deal with these challenges, this project aims to develop robust unsupervised deep learning methods with rigorous guarantees in settings with limited and corrupted data by incorporating physical models and constraints that capture the intrinsic data structures and invariances effectively. The utility of the developed methods will be demonstrated on a variety of imaging applications, and the resulting findings and software will be widely disseminated. Furthermore, this project will develop a new educational program involving yearly virtual workshops with global participation and targeted K-12 outreach in southeast Michigan to enable improved participation in machine learning and computing programs, especially from underrepresented students. This project aims to develop robust, unsupervised deep representation learning methods with rigorous guarantees for application in inverse problems or classification problems in medical, industrial, and scientific imaging. The focus will be on learning deep models from limited and/or corrupted unpaired data. Deep representations will be designed and learned to capture the intrinsic structures and low-dimensionality of the data by leveraging ideas from traditional shallow learning methods (e.g., dictionary/transform learning). By understanding image characteristics captured during different stages of deep representation learning, physical models, priors, and constraints will be incorporated that enable deep networks to be learned efficiently from corrupted data. The deep representations will also be designed to be invariant to typical symmetry ambiguities such as translations and rotations. Based upon such principled modeling, efficient and robust optimization methods will be developed with theoretical guarantees for learning the intrinsic structures of the data, and the developed methods will be applied to a variety of imaging problems. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.