RI: Small: Unraveling and Building Top-Down Generators in Deep Convolutional Neural Networks

University Of California-San Diego, La Jolla CA

Investigators

Abstract

Deep learning has recently significantly advanced research fields that are closely related to artificial intelligence. The fundamental problem of knowledge representation however remains open and the role of top-down process in deep learning is yet not very clear. For example, to train a deep learning algorithm to detect simply the translation of a dog in an image, a data-driven way of training deep learning would require generating thousands of samples by moving the dog around in the image. However, a top-down model, if available, can directly detect translation using two variables along the axes. The main goal of this project is to explore a path to discover, learn, and build embedded deep learning models, accounting for a rich family of top-down spatial transformation and geometric composition in convolutional neural networks. The resulting models provide a transparent way of understanding the embedded top-down transformation process through neural network layers. The learned neurally-inspired top-down knowledge representation will benefit studies across multiple disciplines, including visual perception, brain sciences, cognitive modeling, and decision making. The current practice in deep learning, for example convolutional neural networks (CNN), is largely dominated by data-driven bottom-up approaches. While the performances of various applications using convolutional neural networks (CNN) are impressive, there nevertheless exists a big gap between what bottom CNN can offer and what comprehensive intelligence requires. These strongly bottom-up CNN characteristics leave a big room for one to provide deep learning with the ability to also incorporate top-down information for effective knowledge representation, network learning, cognitive modeling, and visual inference. This project is about building a roadmap towards developing top-down generators. This is done by unraveling the role of explicit top-down knowledge representation and propagation, by studying the feature flows produced inside the convolutional neural networks, by building robust analysis-by-synthesis methods that combine top-down and bottom-up processes, and by creating explicit generative models to assist a wide range of applications. The benefit of studying the top-down generators to a broad family of applications is greatly intriguing, including but not limited to: creating network internal data augmentation, building object detection, developing scene understanding systems; modeling compositional and contextual object configurations; and performing zero-shot learning.