Online Dictionary Learning for Dependent and Multimodal Data Samples: Convergence, Complexity, and Applications

University Of Wisconsin-Madison, Madison WI

Investigators

Abstract

One of the remarkable human capabilities is the ability to extract essential patterns from a constantly evolving stream of information that shapes everyday decision-making. Online dictionary learning (ODL) is a mathematical formulation that emulates the human ability to extract patterns in real time. ODL has found fruitful applications in various domains such as text analysis, image reconstruction and denoising, medical imaging, and bioinformatics. However, existing theories and algorithms for ODL are facing significant challenges in coping with modern streaming data. This project will advance both the theoretical understanding and algorithmic capacities of existing ODL methods. More specifically, the project will address challenges in handling streaming data with multi-modal attributes, partial labels for further classification or inference tasks, and heterogeneous structure in the form of networks. This project will also involve interdisciplinary collaboration and provide research opportunities for students at all levels. The project aims to advance the theory and algorithms of ODL in the following aspects: 1) Obtain the worst-case rate of convergence and iteration complexity of generalized ODL algorithms to stationary points for a stream of structured signals under Markovian dependence; 2) Devise supervised ODL algorithms for learning class-discriminating dictionaries from labeled streaming data with provable convergence guarantees and rate of convergence; 3) Use the theory and algorithm for supervised ODL with tensor-valued signals to develop methods of supervised and temporal network dictionary learning, where the former will learn discriminative basis subgraphs from network data for network classification and denoising applications and the latter will learn basis subgraphs and their time-evolution for reconstructing given temporal or multilayer networks. A key element is the development of stochastic majorization-minimization type algorithms that can handle complex surrogate functions depending on data type using block-minimization and regularization techniques. This project will also provide students with research experiences in optimization, machine learning, and network science. Specific topics for undergraduate research experience will include generating a repository of optimal network dictionaries for various real-world networks, network-level regression and inference experiments with biological networks, and temporal brain network analysis. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.