SBIR Phase I: Realtime Automatic Accompaniment for Music Games

Khush Inc., Atlanta GA

Investigators

Abstract

This Small Business Innovation Research Phase I project investigates the technical and commercial feasibility of a realtime automatic accompaniment technology. The proposed innovation would allow a computer to improvise in realtime with a human musician. The intellectual merit of this proposal is in researching a novel computational framework for modeling musical interaction. Realtime improvisation is one of the most demanding cognitive tasks that humans undertake. This proposal seeks to model the sequential structure of melodies and realtime interaction between two improvising musicians. Although music is a creative act, often full of small surprises, it nevertheless is highly patterned. The prediction methods described in this proposal attempt to uncover this structure through context-dependent models. Realtime musical interaction in the context of tonal music is a relatively new area of research, and the cognitive and musical strategies by which musicians coordinate are complex. The research plan will explore whether this intuitive negotiation can be modeled using game theory, with musically appropriate ideas of payoff. The researchers will evaluate both methods using cross-entropy, an objective measure of how well the predictive distribution matches the actual sequence data. The objective of this proposal is to develop a RAA technology that facilitates technology-based music creation among people that are not musically trained and, in so doing, leads to a new class of self-expression products that represent a multi-billion dollar commercial opportunity. Further, the approach may have clinical applications in the areas of speech therapy for stroke patients suffering speech loss and children with developmental disorders.