III: Small: MicSynth: Enhancing and Reconstructing Sound Scenes from Crowdsourced Recordings

University Of Illinois At Urbana-Champaign, Urbana IL

Investigators

Abstract

There is no doubt that we live in an environment that is massively recorded by multiple people at any point in time. Although we have the technology to combine such information in the visual space (e.g., with PhotoSynth), there is currently no good way to combine audio streams from multiple recordings of the same event. This projects fills that gap by developing new techniques in spectral decompositions and landmark-based localization methods to support taking large amounts of low-level audio recordings of the same event and resynthesize them as one high-quality version, eliminating the artifacts and noise of each individual recording while taking advantage of their strong points. This project aims to introduce new computational tools to combine uncurated recordings at a large scale, and produce information that no single recording can provide. Combining all available information and producing objective representations will enable effective sifting through data from massively recorded events (e.g., social unrest, natural disasters, historical moments) and focus on the needed information. The resulting tools will support creation of high-quality recordings from historical events that might not otherwise be well documented, by using the power of the crowds. The project web site (http://www.cs.illinois.edu/~paris/crowdmic) will provide access to the research results, including a service that allows the consolidation from user-submitted recordings, publication and source code in order to to stimulate activity in this field. Research results will also be incorporated in the development of classes on social and crowdsourcing aspects of audio and signal processing.