ITR: Dynamics-based Speech Segregation

Ohio State University Research Foundation -Do Not Use, Columbus OH

Investigators

Abstract

A typical auditory scene contains multiple simultaneous events, and a remarkable feat of the auditory nervous system is its ability to disentangle the acoustic mixture and group the acoustic energy from the same event. This fundamental process of auditory perception is called auditory scene analysis. Of particular importance in auditory scene analysis is the separation of speech from interfering sounds, or speech segregation. Speech segregation remains a largely unsolved problem in auditory engineering and speech technology. In this project, the P1 seeks to develop a dynamics-based system for speech segregation using perceptual and neural principles. Auditory grouping will be based on oscillatory correlation, whereby phases of neural oscillators encode the binding of auditory features. The investigation will consist of subsequent stages of computation, starting from simulated auditory periphery composed of cochlear filtering and hair cell transduction. A mid-level representation will be formed by computing auto- and cross-correlation of filter channels. A stage of segment formation then creates individual elements of a represented auditory scene, each of which is a dynamically evolving, connected time-frequency structure that may overlap with other elements. Operating on auditory segments from the segment formation stage, both simultaneous organization and sequential organization will be incorporated. For simultaneous organization, grouping will be based on periodicity, location, onset and offset analyses, while for sequential organization grouping will be based on pitch, spectral, and location continuities. In particular, two pitch maps corresponding to two ears and one location map will be computed for auditory organization. All of the employed grouping cues are consistent with perceptual principles of auditory scene analysis. These cues guide the connectivity of neural oscillator networks, which perform grouping and segregation of auditory segments. The proposed system will be evaluated using real recordings of speech and interfering sounds, where speech can be both voiced and unvoiced. The success of the system will be quantitatively assessed using two measures: changes in signal-to-noise ratio and speech recognition rate. This project is expected to make significant contributions to automatic speech recognition in unconstrained environments.