RI: Small: Collaborative Reference in Open Domains

Rutgers University New Brunswick, New Brunswick NJ

Investigators

Abstract

Reference, the ability to single out objects in the world using linguistic expressions, is a fundamental part of communication. Cognitive scientists have showed that people collaborate on reference, for example by negotiating about how to describe things. This project lays the groundwork for computer systems that can do the same. Eventual applications range from talking robots that can communicate with people in real physical environments, to tutoring and decision-support systems that can give comprehensible explanations of specialized concepts. These applications are open domains, in the sense that one interlocutor, or both, may have no simple descriptor for an unfamiliar object. So the system has to have flexible strategies for describing things in words and inference mechanisms that acknowledge the possibility of a misunderstanding by either system or user. This project undertakes exploratory work on the communicative strategies and inference mechanisms required for collaborative reference in open domains. During the academic year, PI Stone works with a computer science student to extend a prototype system with additional methods for collaborating under uncertainty, drawing on the student's ongoing research in cognitive modeling, planning and learning. During the summer, Stone works with a three-member interdisciplinary team doing corpus analysis, grammar development and integration to characterize generic strategies for describing shape, using rough descriptions and part-whole relationships. The results are an open-source prototype with more general collaborative reference abilities along with specific hypotheses, informed by publishable linguistic and conceptual analyses, about how these abilities might affect the system's success in referring to unfamiliar objects.