EAGER: Urban Sensing of Pedestrians through Integrated, Cost-Effective, and Scalable Audio Sensor Networks

Georgia Tech Research Corporation, Atlanta GA

Investigators

Abstract

This EArly-concept Grant for Exploratory Research (EAGER) project will investigate the usefulness of microphones for estimating pedestrian traffic. Recently, with the growing interest in active mobility and walkability, several cities are experimenting with various technologies to sense people. Pedestrian traffic estimation, which has been mostly based on video data analysis or infrared sensors, can be scaled up if microphone-based sensors are deemed equally effective. Acoustic sensor technology, while starting to be deployed for noise pollution measurement and the classification of urban noise sources, has not been explored in the pedestrian sensing context despite its considerable advantages in cost and power requirements. The main reason for the current underutilization of microphone-based sensors is the challenging task of analyzing the audio signal from a multitude of different sources. To address this hitherto unsolved challenge, technology developed for highly complex music audio signals will be adapted to the urban sensing context. Furthermore, a novel dataset comprising audio recordings with pedestrian count annotations will be curated and released to facilitate future research in this area. The project will also demonstrate how the data extracted from audio sensing can be used for pedestrian flow estimation in a small urban area. The project will experiment with a range of off-the-shelf hardware components in a pedestrian-heavy campus environment to investigate how far audio technology can be pushed to sense people, and to assess the possibilities for scalability. As the problem is particularly challenging due to high noise level and potentially low-level pedestrian sound, we speculate that state-of-the-art approaches in audio classification might not be powerful enough to solve the problem of pedestrian count estimation sufficiently well. Inspired by recent work with structured music representations learned through multi-task learning and supervised latent space regularization, a novel experimental regularization approach to representation learning for audio data is applied. This self-supervised learning approach to regularization supports structuring the latent space representation based on feature distances. The additional regularization loss term is derived from the distances of powerful task-relevant pre-trained features in current audio representation structures. This loss, computed for each pair of training data points, can be computed without data annotations as it is based solely on the feature distances between training data points. It enables implicitly imparting domain knowledge through the regularizing feature and thus improves the inductive bias of the network. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.