Multimicrophone Adaptive Array Augmented with Visual Cueing

Advanced Medical Electronics Corporation, Maple Grove MN

Investigators

Abstract

DESCRIPTION (provided by applicant): Advanced Medical Electronics Corporation (AME) proposes to develop a smartphone-based audiovisual array that enables hearing aid users to converse with multiple speakers in reverberant environments with significant speech babble noise where their hearing aids do not function well. The array will consist of a smartphone, a smartphone accessory, and a smartphone software application. The smartphone multi-microphone audiovisual array accessory will be in a form factor that allows attachment to the back of the smartphone. The accessory will also contain a lower power radio by which it can transmit audio signals to compatible hearing aids. The smartphone software application will use the smartphone's built in camera to acquire images and perform real-time face detection using the built-in face detection support of the smartphone. The location of talking targets will be used to implement an audiovisual beamforming algorithm to improve the signal to noise ratio and consequently improve the user's speech intelligibility. Since the proposed array system leverages a handheld consumer electronic device, it will be portable and low cost. AME will design a production prototype in phase II and perform validation.