ITR: A User-centric Content-based Approach to Indexing, Query and Retrieval of Music through Signal Processing and Knowledge-based Methods

University Of Southern California, Los Angeles CA

Investigators

Abstract

This research project will develop methods for content-based indexing of music databases, using a combination of signal processing and knowledge-based methods, design of statistical algorithms for enabling queries using sung or hummed melodies, and design of robust search techniques for retrieving the queried information, especially in the presence of uncertainty. The research approach, which is based on statistical modeling, is user-centric and comprises three major components: (1) Representation and Indexing - musical information utilizing theories and knowledge about human musical intuition, and music perception and cognition. (2) Query Formulation and Interaction Modality - algorithm design for enabling interaction with music data through humming, a natural activity. (3) Search and Retrieval - algorithm design to match user query against the database that will be robust to uncertainties and errors in the query generation. The architecture will include a front-end recognizer, that converts the humming signal to notes using a statistical pattern recognition approach, which interfaces with a back-end music database that is indexed using perceptually viable features. The search process matching a query against indices is formulated as a statistical information retrieval problem. The project will employ progressively rich indexing representation including repeating patterns for recurring themes, chord, beat, and key information. Derivation of such music-theoretic knowledge will benefit from a principled approach of mathematically modeling tonality in music. The statistical framework allows for handling variability and uncertainty in query formulation and retrieval. It also enables providing for quality of solution in the query results. This work will contribute not only to the specific economically significant application area of music, but also to general information science knowledge on how to index and search humanly-meaningful patterns in complex sensory data. The project will also serve as a vehicle to foster cross-disciplinary graduate and undergraduate research and education.