Speech Perception Training: Advanced Scoring and Feedback Methods

Seehear, Llc, Arlington VA

Investigators

Abstract

PROJECT SUMMARY Speech perception in noise is a pervasive, frustrating problem for older adults with mild-to-moderate hearing loss. Few of these adults qualify for or use hearing aids. Auditory speech recognition training is considered a possible intervention, but research has failed to deliver reliably effective training regimes to ameliorate speech recognition difficulties. Training to improve visual speech perception may be more effective, because (1) older adults may experience no significant declines in their ability to combine auditory and visual speech information, (2) visual speech information delivers significant benefit for overcoming acoustic noise interference, and (3) the benefit increases with better lipreading ability. Most individuals who have relied throughout their lives on auditory speech are not good lipreaders: Consequently, there is wide room for improvement in visual speech perception and audiovisual benefits under difficult listening conditions. Additionally, the few existing training systems available to older adults do not incorporate current knowledge about visual speech perception and the underlying mechanisms of perceptual learning. This Phase I project will further develop and test a visual speech training system. In Aim 1, we will port our existing laboratory lipreading training system to the cloud and perform an initial informal evaluation of its operation, and in Aim 2, we will formally test the efficacy of the cloud-based training with hearing-impaired adults, who have been screened and receive pre- and post-testing at George Washington University. A training system that presents sentence stimuli and carries out open set recognition response analyses and feedback was developed for this project. The core system components are phoneme-to-phoneme stimulus-response sequence alignment and computational modeling of the perceptual dissimilarities of visual spoken phonemes and words. The system can produce diverse types of feedback. Three will be compared in Aim 2, Consonant, Word, and whole Sentence feedback. During training, a video sentence is delivered, and participants type their open set response. The system analyzes the response. Consonant and Word feedback give feedback on correct and perceptually-near response words. Sentence feedback is the conventional printed whole sentence feedback. Aim-2 evaluation will determine whether there are significant differences across feedback approaches for lipreading and for audiovisual speech recognition in background noise. Individual differences in the benefits of each feedback type will be examined. Focus groups in Aims 1 and 2 will be convened to give the project team feedback on areas such as ease of system use, understanding of system instructions, and feedback formats. The project will lead straightforwardly to Phase II, which will include activities such as developing a larger stimulus database, including online audiovisual testing materials, and obtaining normative data on system efficacy across age groups and hearing loss levels.