Collaborative Research: CompCog: RI: Medium: Understanding human planning through AI-assisted analysis of a massive chess dataset

New York University, New York NY

Investigators

Abstract

Planning, the ability to mentally simulate the consequences of our actions. It is a crucial aspect of human intelligence – it lets us make choices that are more future-minded, strategize about how others might react to our decisions, and solve complex problems. However, studying planning in the laboratory has been challenging because it requires large amounts of data from highly motivated participants performing difficult tasks. This project addresses this challenge by studying how players select moves in the game of chess. Chess serves as a unique platform to explore how individuals form complex multi-step plans, acquire expertise over time, and develop sophisticated strategies through interaction with other players. Online chess platforms have recently gained popularity, leading to detailed records of billions of games that can be analyzed using computational methods. Furthermore, artificial intelligence (AI) systems capable of playing chess at superhuman levels provide a new tool we can use to study human planning and decision-making. By combining massive behavioral datasets and advanced AI, this project will facilitate the development and validation of new theories of human cognition, as well as the development of new AI systems. The broader social implications include the potential to develop better tutoring systems for complex tasks that require planning, improving human performance in these tasks, and providing insights about the limits of human planning and how they can be expanded that can be applied in various areas such as education, industry, and public welfare. The project has three distinct objectives. First, the datasets will be analyzed to explore long-standing questions about human planning, such as reliance on simplified representations, search pruning, and amortization. Second, the project will examine learning effects across individual longitudinal records to develop mathematical models of expertise development and its interplay with social interaction. Lastly, the findings from the first two objectives will be used to create a cognitive model capable of predicting individual moves in chess games. In addition to providing novel insights into human cognition, the project will improve methodologies for analyzing massive datasets of human behavior, creating an infrastructure that future studies can leverage. Additionally, the cognitive processes modeled in this project will be instantiated in computational terms, facilitating their incorporation into new AI algorithms. Ultimately, the project aims to contribute to cognitive science, AI, and the chess community, leading to broader impacts across academia, industry, and society at large. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.