Gaze Steered Microphone Array for Understanding Speech in Noisy Areas

Advanced Medical Electronics Corporation, Maple Grove MN

Investigators

Abstract

DESCRIPTION (provided by applicant): Advanced Medical Electronics (AME) proposes a wireless hearing aid accessory that utilizes gaze tracking of the hearing aid user's eyes by a smartphone's front facing camera to electronically steer a microphone array with signal processing. The objective is to improve speech intelligibility in noisy environments. A common problem of hearing aid users is participating in conversations in a noisy environment such as at a workplace or restaurant. While hearing aids can improve speech intelligibility in certain types of noise, it is difficult to distinguish between multiple talkers where some might be noise and others are not. Two of the best solutions include moving the microphone closer to the source or using beamforming with microphone arrays. While companion microphones have been shown to be effective they have drawbacks that have limited their use. Companion microphones require an active involvement of the speakers to use microphones. There are many situations where this is impractical or socially awkward. There are also microphone array solutions but there are challenges with steering or pointing the array to the source of interest as well as making the array unobtrusive. Proposed is an innovative solution for using a microphone array that solves the issues of being cumbersome and slow to point, especially in a fast moving conversation of many people. The front facing camera (the side with the screen) on the smartphone will be utilized to detect the gaze direction of the hearing aid user's eyes. The microphone array, attached to the back of a smartphone, will be electronically steered with signal processing to collect sound from the direction the user is looking. The processed audio is then transmitted directly to the hearing aid via a digital wireless link. The user will simply hold the smartphone in a natural manor for someone using a smartphone and look at the person in a group they want to listen to. In order to demonstrate the feasibility of this solution in Phase I, AME will build a prototype system and the will test the prototype with hearing impaired individuals