CAREER: An Embodied Intelligence Approach to Neural Architecture Search

University Of Vermont & State Agricultural College, Burlington VT

Investigators

Abstract

Recent advances in Machine Learning (ML) models like deep neural networks have shown immense promise to solve problems in a wide variety of fields, ranging from health and wellness to environmental science to national defense. Yet the practical use of these methods currently requires a great deal of ML-training and experience to implement due to their sensitivity to nonintuitive and complex configuration settings like the size and shape of deep neural networks. The subfield of Automated Machine Learning (AutoML) seeks to help reduce the barriers to entry by creating models and pipelines which automatically self-configure based on the needs of a given problem. Within AutoML, Neural Architecture Search (NAS) aims to automatically find the ideal structure of a deep neural network. The process of using ML to find the optimal shape and form of a deep neural network is roughly analogous to the well-studied evolutionary and developmental processes that create shape and form in biological creatures, or that automatically find the shape and form of robots in the field of Evolutionary Robotics. Despite the analogies between these subfields, and the outsized impact that work in AutoML may have on our ability to Harness the Data Revolution and make practical impacts across a wide variety of application areas, few examples of Neural Architecture Search algorithms have been inspired by methodologies, successes, and challenges in the evolution of development of embodied robots and animals. In this work, we propose to: (1) Systematically compare the quality of different neural network architectures by analyzing the amount of “embodied intelligence” in their network topologies. (2) Highlight a shortcoming of current “weight-sharing” approaches to NAS and demonstrate how an embodied perspective to brain-body co-optimization may improve search for high quality neural architectures. (3) Demonstrate how neural architectures that grow and prune their structures throughout training compare to static neural network architectures. (4) Create infrastructure to more easily integrate collaborations on real-world problems datasets into the teaching of machine learning and data science at the University of Vermont. (5) Create scientific communication materials for engaging non-STEM students in machine learning via interactive network visualizations and generative art. This project is jointly funded by the Electrical, Communications and Cyber Systems Division (ECCS) and the Established Program to Stimulate Competitive Research (EPSCoR). This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.