CAREER: Fundamental Research, Design and Evaluation of Auditory and Multimodal Graphs

Georgia Tech Research Corporation, Atlanta GA

Investigators

Abstract

Regardless of advances in technology, a continual challenge to science, math, and engineering (SMET) education for blind students, and for their careers, is the task of examining and interpreting data, typically accomplished with visual graphs by sighted users. While there have been many projects aimed at enhancing SMET education and scientific access for the blind by adding sound in some manner, there has been very little systematic study of how, exactly, to create effective auditory graphs. Without such auditory display tools (in combination with visual graphs), there is little chance for visually impaired individuals to participate in science, especially when collaborative discussions are part of the learning or discovery process. In this project, the PI will conduct a program of research and empirical validation into the most effective ways to design and develop auditory graphs, train users to interpret them, and evaluate their effectiveness and appropriateness alongside visual graphs. The PI's preliminary findings indicate that both sighted and visually impaired students and scientists can make use of the auditory display of quantitative information, so long as the displays are sophisticated and well designed, the listeners are trained, and the designer is aware of any differences that may exist in how the various listeners tend to interpret what they hear. A multimodal display should be even more effective, especially when there are both sighted and blind participants in a classroom or research team. Building on these insights, the PI will first perform fundamental experiments aimed at determining the optimal methods for providing contextual cues to auditory graphs, analogous to the axis or tick marks in a visual graph. Once these auditory graph design guidelines have been determined, open source software will be developed to produce both auditory and visual graphs simultaneously, so that teachers or others working with data need not duplicate their efforts. The PI will then conduct experiments to determine the most effective ways to train listeners to interpret the auditory graphs, since it is crucial to overcome the lack of experience that most students, sighted or blind, have with auditory displays. Online training materials will also be developed in order to ensure that the teachers who are likely to create and utilize these sophisticated auditory graphs have the requisite knowledge and skills. The training methods and materials will be evaluated and validated in an experiment and focus group with teachers who work with sighted and visually impaired students. Finally, the efficacy of the enhanced auditory+visual graphs will be validated by comparing the learning outcomes for students who are taught science concepts with the aid of auditory vs. auditory+visual graphs. All of these studies will involve the participation of teachers and both visually impaired and sighted students. Broader Impact: Well-designed auditory displays should be invaluable for both sighted and visually impaired students. Having multiple formats for the data displays will allow all users to maximize learning, and have a shared basis for collaboration. The ongoing research projects will be tied in to university level classes, so that students can learn by taking part in the research efforts; the results and findings will then be rolled into the materials for subsequent classes in computing, psychology, and assistive technology. Visually impaired participants will be a major constituent in focus groups, experiments, and prototype evaluation; this will encourage them to become involved in future R & D efforts that focus on technologies of use and of interest to them. Project outcomes will be disseminated through the PI's various partner organizations and international research collaborators; web sites will make the software, training, and research materials freely available worldwide.