RI: Small: Probabilistic Hierarchical Models for Multi-Task Visual Recognition

Carnegie Mellon University, Pittsburgh PA

Investigators

Abstract

This project studies biologically-inspired architectures for visual recognition. The human visual system can perform a remarkable number of tasks, from estimating the 3D shape of an object that is grasped to inferring subtle differences between two similar makes and models of cars. Such diverse sets of visual tasks are required of a range of autonomous agents, including self-driving cars or humanoid robotics. Such autonomous platforms have the potential to increase general welfare and health of the overall population. This project attempts to build a computational model capable of such diverse visual tasks. Motivated by biological evidence, this project explores the use of feedback logic to enable such computational reasoning. The project provides research opportunities for both undergraduate and graduate students and for increasing diversity in the fields of computer and human vision. This research focuses on development of a unified hierarchical probabilistic model that can be used to solve multiple fine-grained visual tasks. Feedforward hierarchical models, of which the most ubiquitous are Convolutional Neural Nets (CNNs), have demonstrated remarkable performance in recent history. This project introduces hierarchical models for vision-with-scrutiny tasks, such as 3D articulated pose estimation and part segmentation. Rather than focusing on increasing performance on established benchmark performance, this research provides a theoretical framework for analyzing bottom-up (feedforward) CNNs and imbuing them with novel top-down reasoning capabilities. It does so by exploring a link between three dominant but disparate paradigms for visual recognition: feedforward neural models, generative probabilistic models (Boltzmann machines), and discriminative latent-variable models (deformable part models). The models introduced in this proposal allow CNNs to be used for large-scale multi-task learning, where tasks span both coarse-grained tasks (such as rapid scene categorization) and fine-grained tasks (such as 3D articulated pose estimation). By addressing multiple fine-grained tasks with a single hierarchical architecture, resource requirements for memory and speed are vastly decreased, important for embedded visual perception applications such as autonomous robots and vehicles.