REU Site: Computational Methods for Understanding Music, Media, and Minds

University Of Rochester, Rochester NY

Investigators

Abstract

How can a computer learn to read an ancient musical score? What can methods from signal processing and natural language analysis tell us about the history of popular music? Can a computer system teach a person to better use prosody (the musical pattern of speech) in order to become a more effective public speaker? These are some of the questions that students will investigate in the University of Rochester's Research Experience for Undergraduates (REU) site. Students will explore an exciting, interdisciplinary research area that combines computer science, electrical engineering, cognitive science, and music. Each student will be mentored by two or more faculty members from the University's schools of engineering and music. Other activities of the REU site include workshops on career development; scholarship community colloquiums; Python programming for machine learning; and music-focused activities. The goals of this REU are to increase the diversity and broaden the horizons of students engaged in computer science research. The themes of music, digital media, and cognitive science will attract many students from groups under-represented in computer science. Students who are already majoring in computer science will discover that the research in the field is not limited to traditional engineering applications, but can address questions of art, culture, and human psychology. Students with experience in combining computer science with humanistic research are already in great demand in industry and academia, and will help define what it means to be a computer scientist in the 21st century. Students in the University's REU will engage in interdisciplinary research that combines machine learning, audio engineering, music theory, and cognitive science. These disciplines are united by their use of a common set of formal representations and computational methods; in particular, probabilistic models and machine learning. In the research activity, REU students will work on topics such as using machine learning and wide-spectrum imaging to recover lost ancient musical scores; working with cognitive scientists to understand how prosody makes a person a convincing public speaker; and developing algorithms for synchronizing hundreds of audio and video streams of an event to reconstruct the experience of live music performances. The University of Rochester's Department of Computer Science has a long history of contributions in machine learning, natural language processing, and computer vision, and the Department of Brain and Cognitive Science is in the top three nationally. The University's recently-founded audio engineering program is growing rapidly, and the Eastman School of Music is the nation's premiere music conservatory. Although the major objective of the REU is to encourage students to enter STEM graduate programs, many of the projects can be expected to lead to novel and publishable research in machine learning and audio processing.