Virtual Videography

University Of Wisconsin-Madison, Madison WI

Investigators

Abstract

This is the first year funding of a three year continuing award. Digital video is revolutionizing communication. However, while the technology has provided a means to capture, store, and deliver video, it has not adequately addressed the issue of content creation. Effectively utilizing the medium of video is still a resource- and talent-intensive task. In this proposal, the PI will address this issue by developing Virtual Videography, systems that help automate the production of communicative video. He will develop methodologies that enable events to be recorded with minimal intrusion and processed without expert intervention into informative video. The plan for implementing Virtual Videography is based on the observation that a good portion of the challenge of videography is the range of disparate tasks that a videographer must be proficient at. This traditional decomposition of video into its subtasks suggests a scheme where each task is mimicked by a system component. The PI will develop a software architecture where the traditional video production process has practitioners replaced by software components, and the production pipeline is augmented by a centralized repository for not only video data, but also for annotations about the video's content. Such annotations, created by an image analysis component, will guide: (a) a computational cinematographer in pointing a virtual camera that synthetically created novel views using image-based rendering methods; (b) a virtual editor in selecting shots; and even (c) a virtual effects supervisor in using visual effects to add additional emphasis. The PI will focus his efforts on a specific domain: capturing and presenting university lectures. He will demonstrate a system that automatically processes video recorded by stationary cameras in a lecture hall into edited video, showing multiple viewpoints, editing, and special effects. He will not only develop the technology required to construct such systems, but will also evaluate the utility of the results. This project will contribute insight into how the advanced features of video may be used in pedagogy; it will provide useful tools for disseminating lectures and for educational science research, as well as making technical contributions to the many fields that the effort draws upon.