VEC: Small: Collaborative Research: Joint Compressive Spectral Imaging and 3D Ranging Sensing Using a Commodity Time-Of-Flight Range Sensor

University Of Delaware, Newark DE

Investigators

Abstract

The commercial Kinect input device uses both a camera and a time-of-flight depth sensor to capture 3d interactions ? initially for gaming, it has found novel applications, including autism diagnosis and allowing surgeons to control imaging in the operating room. This project aims to integrate the time-of-flight (TOF) depth with Coded Aperture Snapshot Spectral Imaging (CASSI) in a single sensor, giving 3d images not only of visible light, but extending into the infrared. This will have advantages of improving the resolution of the depth information, allowing the two sensors to work together to improve the quality of both. The result will be the first generation of 3D-based ?spectrophotographic? cameras for use on robotic platforms and as the heart of machine vision systems used in manufacturing and metrology. With their higher spatial and spectral resolution, these could open new and unexplored markets. This project develops a time-of-flight, spectral imaging camera by addressing the problems of (a) novel coded aperture design and optimization under light level constraints; (b) fast compressive inverse reconstruction algorithms for real-time implementations; (c) characterization on non-ideal optical elements and calibration mechanisms, and (d) super-resolution enhancement schemes. The PIs will design and assemble a proof-of-concept prototype camera that integrates a DMD array with a CMOS, TOF image sensor in a package comparable to a pico-projector light engine. In order to mitigate the computational complexity of CS inverse image reconstruction algorithms that often preclude their use in real-time implementations, the PIs will build on their history of developing coded aperture schemes that give rise to divide-and-conquer image reconstruction algorithms that can be implemented on GPU and other multi-core processors. This project also incorporates education activities including: (1) the development of a compressive optical imaging course with an emphasis on spectral and TOF modalities; (2) technical seminars at the annual Society of Hispanic Professional Engineers (SHPE) Conference; (3) open-source algorithms and measurement data for the scientific community.