Collaborative Research: Collaborative Learning for Multimodal Data

University Of Minnesota-Twin Cities, Minneapolis MN

Investigators

Abstract

A multimodal paradigm has become increasingly important given today's explosive growth of information, which often arises from, for instance, automatic image categorization and personalized prediction. Multimodal data has a wide spectrum of applications in medical diagnostics, social networking, multimedia, information filtering, personalized advertising, consumers' recommendations, virtually in any electronic commerce and entertainment platform. This research aims to develop statistical theory, methods, and computational tools to integrate multimodal data for prediction and description. The development will lead to the higher accuracy of learning, which will ultimately enhance information storage, sorting and filtering. Moreover, the research project has an education component to train graduate students in emerging areas. The research products will be disseminated through publications and presentations. The proposed research aims to develop statistical techniques to utilize conditional dependence structures for integrating multimodal data. It will proceed in the areas of collaborative learning and personalized prediction. In each area, regression, classification, and ranking will be performed collaboratively based on pairwise conditional dependencies between the response components, modeled by a directed graph or an undirected graph. Special efforts will be devoted to the joint learning of data of multiple modalities and extraction of latent structures with an adjustment for covariates. Target applications include image categorization and recommender systems, where the proposed techniques will be applied to understand the content of an image and to predict personalized preference over a large number of items. Furthermore, The research will develop computational tools and design methods that have desirable statistical properties.