CIF: Medium: Understanding Quality of 3D Video with Applications in 3D Video Processing and Communications

University Of California-San Diego, La Jolla CA

Investigators

Abstract

Research on 3D image/video perception in the light of general principles of stereo processing in the human visual system is being used to derive 3D quality metrics for 3D video applications in order to deliver the best 3D experience. Human observers can detect differences in depth with high sensitivity, but limited precision. Moreover, while the visual system can represent fine details of the 2D image that are carried by high spatial frequency components (even when the image is rapidly changing), it can not track variations in depth with comparably high resolution in space or time. Thus the representation of stereoscopic depth is restricted both in bandwidth and in bit depth. Because of those limitations, some deviations from accuracy in the representation of depth at the retinal level are perceptually salient, and others less so. Measurements of perceived image fidelity across a range of spatial and temporal profiles for the depth signal are being used to guide the development of optimal video processing techniques, and to allow evaluation of the advantages and limitations of alternative 3D video coding algorithms such as multiview versus video+depth). Vision experiments investigate both perceived fidelity and perceived image quality in 3D video generated using a variety of encoding schemes. From those results quality metrics are developed and integrated into video processing and communications applications. A human-centric disparity estimation and view synthesis algorithm is being developed for video processing and communications applications; this can also be used to improve the performance of object detection, classification and tracking, and to generate multi views for autostereoscopic display, which finds applications in 3D enabled diagnostic medical imaging and surgical systems.