Vision Without Sight: Exploring the Environment with a Portable Camera

University Of California Santa Cruz, Santa Cruz CA

Investigators

Linked publications, trials & patents

Abstract

Vision without Sight: Exploring the Environment with a Portable Camera Project Summary As computer vision object recognition algorithms improve in accuracy and speed, and computers become more powerful and compact, it is becoming increasingly practical to implement such algorithms on portable devices such as camera-enabled cell phones. This mobile vision approach allows normally sighted users to identify objects, signs, places and other features in the environment simply by snapping a photo and waiting a few seconds for the results of the object recognition analysis. The approach holds great promise for blind or visually impaired (VI) users, who may have no other means of identifying important features that are undetectable by non-visual cues. However, in order for the approach to be practical for VI users, the interaction between the user and the environment using the camera must be properly facilitated. For instance, since the user may not know in advance which direction to point the camera towards a desired target, he or she must be able to pan the camera left and right to search for it, and receive rapid feedback whenever it is detected. Drawing on past experience of the PI and his colaborators on object recognition systems for VI users, we propose to study the use of mobile vision technologies for exploring features in the environment, specificaly examining the process of discovering these features and obtaining guidance towards them. Our main objectives are to investigate the strategies adopted by users of these technologies to expedite the exploration process, devise and test maximally effective user interfaces consistent with these strategies, and to assess and benchmark the efficiency of the technologies. The result will be a set of minimum design standards that will specify the system performance parameters, the user interface functionality and the operational strategies necessary for any mobile vision object recognition system for VI users.