CAREER: Enhancing Vocal Communication using Graphical Social Cues

University Of Illinois At Urbana-Champaign, Urbana IL

Investigators

Abstract

This proposal outlines a challenging goal of combining the ease of voice with the visual feedback of graphics to create a new communication medium. The project will create a graphical language for visualizing communication. Different versions of the graphical language will be used to mediate remote and co-located conversation, to create learning tools for acquiring new language skills and conducting speech therapy, to create new visualization techniques combining time and phase analysis, and to produce novel methods of archiving audio, speech, and voice. Intellectual Merit. The intellectual merit of this project is the consideration audio, speech, and voice processing from a perspective and goal different from that of traditional audio engineers. The project begins with simple yet effective techniques that have not yet been studied and progresses to developing new analysis and visualization techniques. The graphical languages will be evaluated through applications catered to their use. This research studies the use of social cues and signals that are more readily apparent in mediated interaction than in face-to-face interaction; the parameters in voice that are most effective and easy to interpret; the use of color, shape, and motion mapped onto these voice parameters. The research will help us understand how visualizing conversation publicly affects our vocal interaction when in co-located spaces and in remote spaces; and how much value is added by combining voice data with graphical data. Broader Impact. This research will alter vocal communication interaction in online environments and in physical co-located spaces. The act of archiving in real-time will have new consequences for voice communication and collaboration. The research will provide novel tools for helping users learn the subtleties of vocalization. These tools will be used for learning new languages and for speech therapy. The research extends the vocalization tools to tangible toy objects to encourage vocalization in children who have social impairments. The research will provide an alternate graphical method for archiving large bodies of audio that can be searched at a glance rather than with a search engine. The research results will be disseminated in new courses and in research publications.