PFI-TT: A portable and real-time system for individuals with visual impairments to explore digital images using alternate feedback

Purdue University, West Lafayette IN

Investigators

Abstract

The broader impact/commercial potential of this Partnerships for Innovation-Technology Translation (PFI-TT) project is to develop a portable technology enabling individuals who are blind or visually impaired to have access to digital images in real time through alternate sensory modalities. Current methods to generate accessible images on tactile paper printouts are bulky, time-consuming to develop, cost-prohibitive, and require sighted assistance. The proposed technology presents the opportunity to promote the participation of students who are blind or visually impaired (BVI) and enable them to equally benefit from active learning experiences as their sighted peers, especially in the science, technology, engineering and mathematics (STEM) fields. The proposed system will allow instant access to images for students who are BVI, supporting natural and intuitive interaction with visual scientific data and enabling them to perform observation-based tasks. Successful commercialization of the proposed system will remove a huge obstacle in the STEM education of BVI students and will encourage them to pursue a career in STEM related fields. The system also has the potential to play a critical role in enhancing social interaction in the BVI community. The proposed project will allow the development of a portable, low-cost system which enables individuals who are BVI to understand images in real time. There is currently no commercially available tool for BVI individuals that is portable and allows real-time exploration of images. The novelty of the proposed system developed is three-fold. As a tablet accessory, it is portable. It also incorporates a multisensory feedback interface comprising modalities such as haptics, sound and vibration. Lastly, it is deployed with an intelligent computing unit that uses Computational Neural Networks to characterize visual information and optimally delivering it through various sensory modalities. Using a multisensory and intelligent interface with digital images poses a novel solution to increase efficiency of exploration and perception of visual data in real-time. The team at Purdue University will develop and beta test a portable image exploration system allowing BVI individuals to use a portable controller with supporting software to explore images in real-time. This will result in prototypes for beta testing to assess usability, performance, reliability, and overall user satisfaction. Results will be used to assess the overall impact of the proposed system. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.