Collaborative Research: SBE-UKRI: Spatial communication and computational efficiency

University Of Texas At Austin, Austin TX

Investigators

Abstract

A challenge of developing artificial intelligence (AI) that can work with and serve human needs is developing systems that can smoothly communicate with humans. An aspect of human communication that is effortless for humans, but that remains challenging for AI systems, is communication about space – that is, where people and things are and where they are going. Simple words like “this” and “there” help speakers point to objects, give directions, and share information. These words are important in every human language, and they are critical to get right for robotic systems that will inhabit human spaces and interact with human users. But these words are used in different ways across languages, and scientists do not yet fully understand why. This project quantitatively and experimentally measures how people talk about space using computer models and experiments. The researchers work with speakers of a variety of languages, analyze how people choose words to describe space, and build computer models that explain how this communication works. Other benefits to society include providing innovative educational opportunities that support workforce development for AI and other language technology industries. This research investigates how human languages express spatial relationships using deictic words (e.g., “this,” “that,” “here,” “there”). The project integrates cross-linguistic experimental data with information-theoretic computational models – specifically using the Information Bottleneck approach that was developed to explain artificial neural network behavior – to explore how people balance accuracy and simplicity when using spatial language. Using methods such as behavioral experiments, geospatial analysis, and machine learning, the project explores the cognitive and environmental pressures that shape how speakers refer to locations and objects. The models developed in this project also provide insights into how AI systems can more naturally interpret and produce spatial language, supporting improvements in areas such as robotics and AI more generally. This award is made possible through the NSF-UKRI lead agency opportunity. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.