Conference: Modeling Randomness in Neural Network Training: Mathematical, Statistical, and Numerical Guarantees

Rutgers University New Brunswick, New Brunswick NJ

Investigators

Abstract

Neural networks are at the heart of modern machine learning and artificial intelligence (ML/AI) systems. The rapid development of these technologies has led to rapid adoption across a variety of domains, particularly in speech processing, computer vision, and natural language processing. At the same time, the theoretical underpinnings of these statistical models are not yet fully understood. The question of how and why neural networks “work" can be approached from a variety of mathematical perspectives. One of the most promising mathematical tools for analysis of neural networks is random matrix theory, a field that has recently reached mathematical maturity and whose relevance and applicability to modeling, understanding, and characterizing a vast array of science and technology problems keeps growing every day. From principle component analysis and random growth processes to particle interactions and community detection in large networks, random matrices are now used to investigate and explain high-dimensional phenomena like concentration (the so- called "blessing of dimensionality" as opposed to the "curse of dimensionality"). Recent results in universality allow for usage of more complex, non-Gaussian models, sometimes even allowing for limited dependencies. This prompts the question: what can probability theory in general---and random matrix theory (RMT) in particular---tell us about neural networks, modern machine learning, and AI? Such fundamental insights could lead to novel approaches that simplify or improve the efficiency of neural network design and training. The DIMACS Center at Rutgers University will hold the Workshop on Modeling Randomness in Neural Network Training: Mathematical, Statistical, and Numerical Guarantees at Rutgers University on June 5–7, 2024. This is a multidisciplinary workshop to create bridges between the different mathematical and computational communities by bringing together researchers with a diverse set of perspectives on neural networks. Random matrix theory can be used to understand different phenomena in neural network training. The workshop will center around the following themes: understanding matrix-valued random processes that arise during neural network training, modeling/measuring uncertainty and designing estimators for training processes, and applications to these designs within optimization algorithms. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.