Automated Tactilization of Graphical Images: Full Access to Math, Science, and Engineering for Blind Students

University Of Washington, Seattle WA

Investigators

Abstract

Graphical images (line graphs, bar charts, diagrams, illustrations, etc.) are prevalent in math, science, and engineering (MSE) textbooks at all educational levels. But while studies have shown that tactual perception is the best modality for comprehension of graphical images by people who are visually impaired, the graphical images found in textbooks typically aren't available in this format. Visually impaired students' lack of full access to the contents of textbooks impedes their learning, development, and success in MSE careers, areas in which individuals with disabilities are underrepresented. This project seeks to address this problem, by developing innovative ways to overcome obstacles to the timely translation of graphical images into a tactual format. The needs of two user communities will be addressed: transcribers who translate graphical images into tactual formats within low- and high-production environments; and students who are in MSE classes at the K-12 and postsecondary education levels, are blind, and read Braille. To these ends, the PI has assembled an interdisciplinary team with expertise in image processing, machine learning, IR, HCI, experiment design, and addressing the needs of students with disabilities. The PI and his team will design and develop the Tactile Graphics Assistant (TG Assistant), a set of plug-ins for Adobe Photoshop and Illustrator, which will support transcribers in transforming, as automatically and intelligently as possible, graphical images into a high-quality tactual form that can be reproduced and then used by students who are blind. Empirical studies will be conducted to better understand the perception of tactile graphics, and to inform the design of prediction models to estimate image comprehension time and comprehension accuracy, the application of machine learning techniques to classify images by their type, and the design of image processing algorithms to carry out the steps (appropriate for the image type) to translate an image into a tactual form. A user-centered design approach will be followed during the development of the TG Assistant. Project benefits will be documented by three proof-of-concept activities, wherein the TG Assistant will be used to provide access to textbook images to three students at the K-8, high school, and postsecondary education levels. Broader Impact. Tactual access to graphical images will improve blind students' learning, performance, retention, and potential to succeed in MSE careers. Project outcomes will include invaluable data on the tactual perception of tactile graphics, new image processing and classification algorithms, and a usable tool to streamline the translation of graphical images into a tactual format. Research findings on the translation of graphical images will be summarized as a set of guidelines and distributed to transcribers. The PI will make an effort to incorporate the guidelines and TG Assistant into transcriber training programs. The research team includes individuals from under-represented groups (women, minority, and disabled individuals.