A Non-Document Text and Display Reader for Visually Impaired Persons

Smith-Kettlewell Eye Research Institute, San Francisco CA

Investigators

Linked publications & trials

Abstract

DESCRIPTION (provided by applicant): The goal of this project is to develop a computer vision system based on standard camera cell phones to give blind and visually impaired persons the ability to read appliance displays and similar forms of non-document visual information. This ability is increasingly necessary to use everyday appliances such as microwave ovens and DVD players, and to perform many daily activities such as counting paper money. No access to this information is currently afforded by conventional text reading systems such as optical character recognition (OCR), which is intended for reading printed documents. Our proposed software runs on a standard, off-the- shelf camera phone and uses computer vision algorithms to analyze images taken by the user, to detect and read the text within each image, and to then read it aloud using synthesized speech. Preliminary feasibility studies indicate that current cellular phones easily exceed the minimum processing power required for these tasks. Initially, the software will read out three categories of symbols: LED/LCD appliance displays, product or user-defined barcodes, and denominations of paper money. Ultimately these functions will be integrated with other capabilities being developed under separate funding, such as reading a broad range of printed text (including signs), recognizing objects, and analyzing photographs and graphics, etc., all available as free or low-cost software downloads for any cell phone user. Our specific goals are to (1) gather a database of real images taken by blind and visually impaired persons of a variety of LED/LCD appliance displays, barcodes and US paper currency;(2) develop algorithms to process the images and extract the desired information;(3) implement the algorithms on a camera phone;and (4) conduct user testing to establish design parameters and optimize the human interface. PUBLIC HEALTH RELEVANCE: For blind and visually impaired persons, one of the most serious barriers to employment, economic self sufficiency and independence is insufficient access to the ever-increasing variety of devices and appliances in the home, workplace, school or university that incorporate visual LED/LCD displays, and to other types of text and symbolic information hitherto unaddressed by rehabilitation technology. The proposed research would result in an assistive technology system (with zero or minimal cost to users) to provide increased access to such display and non- document text information for the approximately 10 million Americans with significant vision impairments or blindness.