VIDI: Visual Information Dissemination for Visually Impaired Individuals

University Of Massachusetts Amherst, Amherst MA

Investigators

Abstract

This is the first year funding of a three year continuing award. The PI's contention is that the field of computer vision has matured to the point where a completely portable system capable of finding signs and other textual events in the environment in close to real time, and then reading their contents aloud, is feasible. His goal in this project is to provide the visually impaired with a way to obtain crucial information from the environment that is now totally inaccessible (except through intervention of a sighted person). Running on a wearable computer connected to a gyroscopically referenced head mounted camera, the envisaged system will detect text and signs within a compact representation of continuous video of the user's surroundings in the form of a mosaic. The sign detection and recognition system will use a set of multi-scale features that have been shown to be invariant to changes in viewing angle and illumination. Recognition will be accomplished by building multi-resolution spatial networks of distributions of these features and matching them against a database of known (customized to locale) signs (e.g. international highway signs) using a coarse to fine matching technique that is both robust and efficient. Text will be detected using the same techniques, with the aid of a commercial OCR system. The PI will also examine the OCR's intermediate representations, and a word dictionary for tolerance to character recognition errors. He will leverage existing work on image retrieval from large databases using image content, face recognition, image indexing using color properties, and text detection and recognition in complex images. The work will be performed in three phases roughly corresponding to a year each. System design and evaluation will both be conducted in conjunction with Lighthouse International, which specializes in rehabilitation training for the visually impaired. In the first phase of the work, individual components, which include camera stabilization, mosaics, text and sign recognition, will be developed. In the second phase, a head mounted unit, attitude inference, a wearable computer, a camera, speech synthesizer and user interface model will be developed. Additionally, the use of log-polar cameras will be examined for fast computation of multi-scale features and for its effectiveness in simultaneously addressing the field of view and resolution issues. The third and last phase will consist of a series of user and system evaluations and refinements. There is no doubt that, if successful, this research will have a tremendous impact on and dramatically improve the lives of visually impaired individuals.