CI-NEW: Collaborative Research: DialPort: Enabling Spoken Dialog Research with Real Data

Carnegie Mellon University, Pittsburgh PA

Investigators

Abstract

New interest in spoken dialog research has recently been generated by the advent of SIRI(TM). This application ignited the imagination of many who have begun to believe that speaking to automatic assistants is possible and useful. Soon everyone will need to use speech to communicate with technology, such as smart watches, which can't use a keyboard. Using speech will also give wider access to technology, for example to the elderly and the disabled. It will serve as an economic motivator, spurring new technology and technology-based products. The vision of the Dialog Portal project and community infrrastructure, DialPort, is that it will serve the spoken dialog community by being a central portal that will attract real users to existing dialog systems while also sparking the creation of new streams of spoken dialog data. It will distribute data and software and educate researchers about both, thus making it easier for new colleagues to access this domain. DialPort will link high school and undergraduate students (who have lived with the technology all of their lives, and can imagine how they would like to talk to technology), with technology experts who can help them implement their ideas. DialPort will enable the spoken dialog community to access an unprecedented amount of speech data, which is essential in creating successful dialog systems, as it is used to retrain them (adding data makes the statistical representations more precise) and to assess them (comparing systems helps find which novel techniques improve the systems) and to create new ones. In this manner, the contribution of DialPort will lead to both higher quality research and to widespread use of speech technology. The overarching goal of the DialPort project and community infrastructure is providing real users' speech in many different domains that will generate data and research platforms for the community. As a central portal it will attract the real users to spoken dialog systems that are linked to it. DialPort will relate high school and undergraduate students' ideas for novel, transformational applications of speech technology with the current spoken dialog community's technology. The resulting real applications will provide the streams of data from real users who are interested in repeatedly using an application because it helps them in some way (solving some task, or entertaining them). This will allow researchers to make their systems more robust. DialPort will also provide access to spoken dialog software and data and educate members of the community in their use. Unsolved basic research issues in spoken dialog include: signal processing in noise, recognition of groups of "difficult" users (like the elderly and non-native speakers), management of complex dialogs (i.e. in meetings and with agents), and the automatic use of meta-linguistic information such as prosody. DialPort will spark new applications that can be used as research platforms where these and other issues can be explored. One such platform has been provided in the past for bus information in Pittsburgh. Since then, research has evolved, necessitating real dialog systems in more challenging areas and on more devices. One site cannot create and run all of them. DialPort will help the community to create these systems, insuring that they meet their needs and obtaining consensus on a unified infrastructure. The DialPort project includes: (1) providing a central portal where real users can go to be connected to many spoken dialog systems, thus generating large amounts of real user speech; (2) conducting the yearly REAL Challenge to spark ideas for ways that speech will be used in the future, addressing high school and undergraduate students and educating them about spoken dialog systems; and (3) gathering tools that the community can use, ensuring their documentation and teaching researchers about their use.