Representation and Learning of Musical Expression in Melody

Indiana University, Bloomington IN

Investigators

Abstract

Proposal 0739563 ""Representation and Learning of Musical expression in Melody"" PI: Christopher S. Raphael Indiana University ABSTRACT Music exists on several levels. One view is in terms of the directly observable attributes of a musical score, such as notes and rhythms written in traditional symbolic music notation by a composer. Another view is in terms of the audio stream corresponding to an expressive rendering of the score. Between these two extremes--the atomic sub-components and an expressive performance--there are many levels. Only some of these have well-developed representations and analyses. The goal of this project is to develop a representation of an intermediate layer that can explain the association between the lowest note level and the top expressive level and to develop a system for expressive rendering of continuously controlled music that will begin with melody represented as a note list with an analysis of the harmonic structure and produce an expressive audio corresponding to a performance of it. To achieve this goal, the project aims to develop a representation for capturing the expressive elements of the notes--their prosodic function--for instance, their implicit musical direction and stress. The project will also study the relationship between this mid-level representation and continuous audio in an actual rendition of the music. For this aspect, the project will employ a ""Theramin"" model for audio that continuously modulates the pitch and amplitude of a sinusoidal tone. Such a model is capable of representing a variety of expressive musical elements including dynamics and vibrato. This project will pursue the following specific tasks: (1) development of a small corpus of expression-annotated music; (2) development of a system for automatically computing the prosodic annotation from the symbolic melody and harmony representations; and (3) development of a system for automatically generating the Theramin representations from prosodically annotated melody and harmony. The use of machine learning methods will be employed in this project to predict Theramin representations that resemble those of a training corpus; these will also be used in evaluation.