RI:Small: Neural Architecture Search with Deep Compositional Grammatical Structures

North Carolina State University, Raleigh NC

Investigators

Abstract

Artificial Intelligence (AI) technologies have recently shown great values in a wide range of applications such as self-driving cars, smart speaker, machine translation, robot autonomy and medical diagnosis. Computer vision (automatic analysis of images and videos) and natural language processing (automatic analysis of text) are two key pillars of AI. Deep learning artificial neural networks are the "brain" of many state-of-the-art AI systems in these two domains. However, much of the neural architectures are still hand-crafted to tailor to individual tasks in each domain, whereas learning in the human brains seems to be more task- and domain-agnostic. This project presents a principled framework to automatically learn neural architectures that are smaller, faster and better for both computer vision and natural language processing tasks. More specifically, the project explores methods of searching for neural architecture based on structural rules that compose smaller units to make bigger units, similar to how grammars in natural languages guides the way sentences are formed. This approach eliminates the efforts of manually engineering neural architectures. The success of this project will significantly advance AI systems in computer vision and natural language processing, thus moving forward other practical applications of AI technologies as well. This project will also integrate educational components by making significant connections with an interdisciplinary set of students and researchers in the Digital Humanities, and preparing demos to engage the K-12 and undergraduate communities. The project proposes three main tasks. The first is to unfold the space of neural architectures with deep compositional grammatical architectures; this allows for learning rich features of deep neural networks in a principled way. The second is to develop grammar-guided neural architecture search. The goal is to search smaller, faster and better deep compositional grammatical architectures by integrating differentiable search and reinforcement learning search in the unfolded space. The third task is to evaluate the proposed work applications. The searched architectures will be used as new feature backbones for existing state-of-the-art deep learning based systems commonly used in computer vision and NLP. The key innovations of this project include: a novel method of grammar-guided network architecture design and search for deep learning, and a unified feature backbone enabling effective and efficient feature exploration and exploitation in computer vision and NLP. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.