INTEGRATED SENSING: Light Field Compression and Streaming

Stanford University, Stanford CA

Investigators

Abstract

Light Field Compression and Streaming In image-based rendering, a large set of conventional 2-D images is used to render arbitrary views of a 3-D scene. The images capture the light field around the scene. Since hundreds or thousands of individual images are required, the data sets are huge, unless they are compressed. A particularly important application of light field compression is the streaming of light fields stored on an Internet media server. For interactive viewing, this requires a framework to decide, which packets of compressed light field data to send over the network, and at what time. This project proposes to develop and study new techniques for efficient light field compression and interactive streaming. For compression, a novel, elegant disparity-compensated 4-D wavelet transform will be investigated that avoids the re-sampling problem of previous disparity-compensated wavelet schemes by incorporating the disparity compensation into a sequence of lifting steps. For light field streaming, the proposed work will build on recent advances in rate-distortion optimized packet scheduling and extend the state-of-the-art to interactive 3-D viewing. The project is expected to have broad impact for networked 3-D interactive graphics applications, ranging from distance education and remote visualization of scientific and medical data sets, to Internet e-commerce and networked video games.