III: Small: Modular structures in the brain and artificial learningsystems: emergence and function

Massachusetts Institute Of Technology, Cambridge MA

Investigators

Abstract

Deep learning has made great strides, with artificial neural networks outperforming humans after training exhaustively on specialized tasks. However, rapid learning and flexible intelligence, hallmarks of biological brains, remain out of reach. In this proposal, we seek to understand an important feature of biological brains that will be critical for better and more interpretable artificial intelligence: the existence of modular architectures that are combined in rich ways to solve multiple problems with some shared sub-structure. We will study three aspects of modular architecture: 1) How observed modular structures in the brain might arise with minimal training, through simple and local constraints on how far a neuron can extend and how many synapses it can make; 2) What is the utility of a modular organization in neural circuits in solving tasks, in terms of robustness and the speed of learning, in the context of known neural circuits and function; 3) How the modular architectures and principles that lead to rapid modularization we learn about in 1)-2) can be imported into artificial neural networks to improve the learning speed and flexibility of machine intelligence. Thus, we seek to better understand brains, build a stronger dialogue between neuroscience and artificial intelligence, and use the resulting insights to improve machine intelligence. The grant will provide training opportunities to students and postdoctoral fellows in cutting-edge areas of strong interest for industry, government, and science, and we will focus on training a highly skilled and diverse workforce in these areas. Compositionality, i.e., the ability to learn and perform complex cognitive function by re-combining simpler sub-functions, is fundamental to the capabilities of the mind and is the basis for general intelligence. Underlying this ability is the presence of modular structures in the brain. Modular structures confer inherent advantages, such as increased stability to perturbations and faster learning. We will investigate mechanisms for the emergence of modularity in natural and artificial systems. Existing models for modularity are based primarily on top-down supervised learning, which requires large amounts of learning time and data. Our central hypothesis is that local constraints on connectivity in the brain provide strong prior biases towards modularization, and these lead to the rapid emergence of modular structure and improved function. First, we will use theoretical and computational tools to model low-level constraints to probe how they may drive modularization in neural circuits observed in the brain, including grid cells in the entorhinal cortex. Second, we will analyze the advantages conferred by modular organization in biological systems by studying the properties of high-capacity memory architectures directly inspired by the entorhinal-hippocampal circuit, with mEC-like modular subnetworks. Third, we will combine our developed modularization mechanisms and understanding of biological architectural circuitry to import similar advantages into artificial learning systems, creating artificial neural networks that achieve robust modular solutions to complex real-world tasks. Thus, our work will help to characterize how biological and artificial networks may spontaneously modularize to support robust and efficient inference and learning. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.