CI-P: Planning for AudioNet: A New Community Infrastructure for Audio Annotations for Acoustic Event Identification

International Computer Science Institute, Berkeley CA

Investigators

Abstract

This effort lays the groundwork for AudioNet, a public-domain corpus of audio labels for the nearly 800,000 videos in the open-access YFCC100M dataset. Audio information provides an important complement to visual information in the automatic analysis of video data, allowing systems to detect situations that may not be clearly identifiable from the visual stream alone. However, there are as yet no truly large-scale labeled audio datasets of the kind needed as input to build flexible, accurate analysis systems. Creating such a large-scale corpus will serve as an impetus for better multimedia algorithms to be developed by more researchers and computer science students, translating into an impact on the everyday life of the public at large. Social media videos are increasingly used for scientific research, as they provide an opportunity to observe and model many phenomena in the social sciences, economics, meteorology, and medicine. New capabilities for content analysis will therefore impact many scientific fields. In addition, audio analysis could be used in real-time security surveillance and in robotics applications like autonomous vehicles and household robots to aid and monitor the elderly. AudioNet is part of a multi-institution collaboration, the Multimedia Commons initiative, which is developing a variety of resources around the YFCC100M dataset of Creative Commons-licensed photos and videos. AudioNet is annotating the audio tracks from the YFCC100M videos, focusing on audio concepts. Audio concepts can be thought of as acoustic "objects": concrete, localizable units of sound like "crowd cheering" or "fire alarm". The approach will be modeled on ImageNet, an image dataset labeled and organized using the WordNet hierarchy of synsets (groups of synonyms); ImageNet has enabled major enabled advances in image processing. However, while ImageNet focuses largely on entities (noun synsets), audio data is inherently temporal. The label set for AudioNet will therefore focus on events and actions, though similarly organized using semantic resources like WordNet.