EAGER: A Metric Space Embedding of Object Fragments and Object Categories for Object Recognition and Segmentation

Brown University, Providence RI

Investigators

Abstract

Recent developments in augmenting appearance-based approaches to object recognition with shape have used local shape features in analogy to appearance features. However, past work on shape has shown that shape is much richer than a conglomerate of local features. Rather, shape is very high-dimensional and defies global embedding in a reasonably-dimensioned Euclidean space. Thus, Euclidean space concepts used in appearance-based recognition, such as formation of visual words from k-means, vocabulary trees, etc. are no longer applicable. This project is developing analogous concepts for efficient indexing with a large number of categories in the context of a metric space for shape. These concepts are being investigated in the context of an integrated bottom-up and top-down object recognition and segmentation framework. First, a top-down approach using a novel language for shape has already exceeded the state of the art in the ETHZ dataset. However, the prototypical shapes are manually selected. The project aims to use the concept of structural averaging to automatically form prototypical shapes. Second, a fragment-based bottom-up approach has shown state of the art performance for a one-category Weizmann Horse database. An extension to the use of more categories requires an organization of the object space and the space of object fragments. The project aims to capture the metric structure of both spaces using a proximity graph, which is then used for efficient indexing. These two developments will together enable an integrated approach where bottom-up methods narrow a selection of categories which are then examined by the top-down approach. Broader impacts include aerial tracking and recognition of vehicles for defense applications, segmentation of X-ray fluoroscopic images of the spine, and indexing into databases, e.g., trademarks.