A unified theory for perception of physical and social dynamics

University Of California-Los Angeles, Los Angeles CA

Investigators

Abstract

Humans spontaneously endow the world we see with physical and social meaning: we “see” not only physical interactions among inanimate objects, but also the intentional behaviors of interacting humans. This project will investigate how humans perceive and understand physical and social dynamics from the movements of objects and people. The larger goal is to explain how perception and reasoning operate together to guide behavior in dynamic situations. Findings from this project will help equip the next generation of intelligent systems with the visual reasoning ability required to operate in novel situations. Such systems will need to perform complex tasks that are challenging and time-consuming for humans. The project will integrate behavioral experiments with computational modeling to develop and test a unified theory. The theory specifies how force representations can be constructed directly from images and how the discovered force structures can be used to infer causal relations involved in dynamic stimuli. Specifically, Bayesian models will be developed with two basic computational components: (1) generative models to learn representations of latent force structures governing movements of inanimate objects and human agents in physical and social scenes, and (2) reasoning models to identify causal relations and hidden states of intentions based on spatial-temporal configurations of force structures. Computational models will be used to synthesize dynamic stimuli that instantiate a large range of physical and social situations. These stimuli will then be used to analyze human behaviors at a large scale. A series of psychophysical experiments and model comparisons will test the key computational components of the theory. The project will also provide excellent training opportunities for students who are interested in interdisciplinary research at the interface between cognitive science and artificial intelligence. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.