Adaptive User-Guided Assistive Listening System

Rochester Institute Of Tech, Rochester NY

Investigators

Abstract

PI: Borkholder, D. A., Amuso, V. J., Eddins, D. A. Proposal Number: 0967732 The research objective of this proposal is to test the hypothesis that an assistive listening system with dynamic, user-controlled directional characteristics will improve communication for the hearing impaired significantly relative to traditional fixed-directional systems. The proposed study will remove barriers to the delivery of acoustic information to people who are deaf or hard of hearing, enhancing their participation in education, work, and social settings, and improving their quality of life. Interfering speech, reverberation, and temporally-fluctuating background noise have a profound impact on speech perception for the hearing impaired. Directional microphone arrays have been shown to improve key metrics of speech intelligibility, especially for the hearing impaired, but minimal user guidance capabilities limit achievable benefits in multi-talker situations. It is widely accepted that current personal assistive listening systems are inadequate in small-group, multi-talker settings. The proposed research will explore user controlled directivity coupled with speaker identification algorithms and tracking techniques from the radar and infrared tracking communities to enable an advanced user-guided assistive listening system. Acoustic beamforming and parallel processing offer opportunities for users to control a focal acoustic zone while attenuating noise sources outside of this zone. A hybrid pattern recognition scheme incorporating several techniques for parameter extraction and speaker identification will allow robust dynamic identification of speakers in multi-talker situations. Target tracking algorithm will dynamically steer the acoustic focal zone as the talker moves through the meeting area, relieving the listener of the burden of manual tracking. A user interface will provide spatial information for each identified talker, allowing the listener to independently select the desired talker, or manually steer the acoustic beam to a desired location. The study will involve exploration of the most effective parameters for user-control, algorithms for phased-array beamforming, speaker identification, and target tracking for this acoustic application, and quantitative hypothesis testing with hearing impaired listeners in controlled environments. This will be a collaborative effort between the Rochester Institute of Technology College of Engineering, College of Science, the National Technical Institute for the Deaf, the International Center for Hearing and Speech Research, and the University of Rochester - Medical Center Department of Otolaryngology. The research will involve engineering graduate students, AALANA and women undergraduate students, and hearing impaired individuals from NTID, allowing a unique opportunity to foster an appreciation of technology to aid persons with disabilities. Intellectual Merit The novelty of the proposed research lies in the integration of technologies and approaches from different fields to address a critical need of the hearing impaired, and testing the hypothesis that a shift from technology-control to user-control of parameters enhances communication By incorporating speaker identification with target tracking and user-guidance of an acoustic focal zone, the technology is expected to outperform current state-of-the-art assistive listening systems in the small group, multi-talker circumstance in terms of speech intelligibility metrics. The investigators bring a distinctive combination of expertise in the areas of biomedical systems, radar, psychoacoustics and assistive listening devices, providing a unique opportunity for interdisciplinary innovation in technology to aid the hearing impaired. Broader Impacts The proposed research will improve the ability of hard-of-hearing individuals to participate in education, work and social settings by enabling a presently unavailable level of user control critical for the multitalker situation. It will enhance education opportunities for the hearing impaired and may allow more students to enter the STEM disciplines. Demonstration that user control of assistive listening device parameters enhances communication could have a profound impact on the philosophy and approaches for other assistive listening devices such as hearing aids and cochlear implants. The proposed activities integrate research with education for graduate and undergraduate engineering students, under-represented minorities, and hearing impaired individuals. The results will be broadly disseminated to the engineering and disability communities through conferences and journals, and will leverage the expertise of Center on Access Technology for integration into the educational mission of this and other universities.