RI: Medium: Collaborative Research: Learning to Su

University Of California-Los Angeles, Los Angeles CA

Investigators

Abstract

Today there is far more video being captured - by consumers, scientists, defense analysts, and others - than can ever be watched. With this explosion of video data comes a pressing need to develop automatic video summarization algorithms. Video summarization takes a long video as input and produces a short video as output, while preserving its information content as much as possible. As such, summarization techniques have great potential to make large video collections substantially more efficient to browse, search, disseminate, and facilitate communication. Such increased efficiency will play a vital role in many important application areas. For example, with reliable summarization systems, a primatologist gathering long videos of her animal subjects could quickly browse a week's worth of their activity before deciding where to inspect the data most closely. A young student searching YouTube to learn about Yellowstone National Park could see at a glance what content exists, much better than today's simple thumbnail images can depict. An intelligence agent could rapidly sift through reams of aerial video, reducing the resources required to analyze surveillance data to identify suspicious activities. This project develops new machine learning and computer vision algorithms for video summarization. Unsupervised methods, which are the cornerstone of nearly all existing approaches, have become increasingly limiting due to their reliance on hand-crafted heuristics. By instead posing video summarization as a supervised learning problem, this project investigates a markedly different formulation of the task. The research team is investigating four key new ideas: powerful probabilistic models for learning to select the optimal subset of video frames for summarization, semi-supervised learning models and co-summarization algorithms for leveraging the abundance of multiple related videos, algorithms for exploiting photos on the Web to improve summarization, and evaluation protocols that assess summaries in a way that aligns well with human comprehension. The broader impact of the proposed research includes practical tools for video summarization, scientific advances that appeal broadly to several communities, publicly disseminated research results, inter-disciplinarily trained graduate students, and outreach activities to engage young students in STEM education and career paths.