CAREER: Performer-Centered Approaches to Computer-Assisted Music Making

University Of Southern California, Los Angeles CA

Investigators

Abstract

This research comprises projects that focus on two aspects of computer-assisted music making: expression synthesis using a driving (steering wheel and pedals) interface; and collaborative performance assessment. The common motivator in both cases is the performer (the pilot of the expression synthesis machine, or the ensemble musicians). The human performer is the engineer of expressive performances and creator of coordinated interpretations in ensemble playing. Incorporating the counsel of the performer is critical to the success of systems for generating, tracking and assessing single- or multi-player performances. A performance is the result of analysis, and decisions based on the analysis. This process will be studied through computational models of music perception and cognition, based in part on the PI's prior work relating to a representation for tonality called the Spiral Array model. Developing the quantitative tools for music analysis will form the foundation for the testbed applications. The Expression Synthesis Project (ESP) will support a driving interface that will enable non-experts to create expressive musical performances. Anecdotal evidence amongst musicians suggests that generating an expressive performance is very much like driving a car. Not everyone can play an instrument, but almost anyone can drive a car. In the first instantiation of the ESP interface, the pedals will allow the user to control the tempo (speed). The display will show landscape that directly map to musical content (extracted through computational analysis), so as to guide the expressive decisions. By relating preferred renditions of a piece to musical structure, we can better understand and explain human creativity through music performance. When the technology matures, plans to create an interactive ESP museum installation will commence. The Collaborative Performance Assessment (CPA) project will create metrics and systems for measuring temporal and contextual synchronization in collaborative performance. The most immediate application of the technology is to distributed collaborative performance. Ensemble playing requires coordinated decision-making amongst performers. In score-based music, the ensemble players' tempo and timing of expressive (musical) gestures must agree. In improvised music, the chords and transitions must also be synchronized. CPA tools will provide feedback to ensemble players to help them identify and solve problems so as to play together better. In remote collaborations, the tools will be used to quantify the effects of latency and presence, and to guide the design of distributed immersive performance systems. Broader Impacts: Expressive performance is usually only accessible to expert musicians after many years of practice. The ESP interface will allow expressive decisions to be explored long before (or without) the mastery of an instrument or a piece. Furthermore, a musical score is only an approximate representation of a composer's creation. Most composers are not expert performers, so the realization of their creation must be left to musicians' interpretations in live performance. The ESP system will allow composers to record their expressive intent precisely. The system will furthermore present a more intuitive interface for linking music to video and for creating coordinated music and picture in movies, and thus will inspire new entertainment technologies. CPA tools will give remote regions access to artistes, bands and orchestras that would normally not pass their way. They will enable distributed recordings to take place, allowing musicians to reduce their time on the road. The PI plans to organize performances and demonstrations showcasing distributed and local collaborations, to disseminate research results. With diminishing funding for the arts, technology-driven music applications such as those to be developed in this research will become a prime motivator for music education in schools.