SBE-UKRI: Integrating vision and action through selection history

Brown University, Providence RI

Investigators

Abstract

Most real-world visual scenes are complex and crowded, where multiple objects compete for attention and goal-directed action. In daily life, for example, a person easily picks up a red apple from a grocery display containing many kinds of fruit. Successful interactions with such complex environments require seamless coordination among multiple mechanisms. In particular, mechanisms of attentional selection that help us make sense of the world work in unity with those that underlie action selection, allowing us to generate adaptive movements. This attention-action synergy is at the root of all complex behavior. However, researchers typically study these two mechanisms separately, without investigating interactions between them. This practice makes direct comparisons between the findings in attentional selection and action selection impossible. To overcome this gap, the goal of this proposal is to determine the interplay between mechanisms controlling attentional selection and action selection, particularly when recent selection history biases subsequent behavior. It will provide a basis for unifying theories of behavioral mechanisms involved in the selection of attention and action in a global frame. Specifically, the proposed project will focus on three understudied aspects of action selection: variation in action execution, effectors, and biomechanical costs, to determine their novel relations with the wealth of research on selection history of perceptual features. To ensure successful outcomes, an international collaboration between the two research teams, who will play complementary and synergistic roles, will be formed between experts in attention and motor control and computational modeling. This joint endeavor will advance our understanding of the interdependence between attention and action-driven mechanisms to eventually explain adaptive, real-world selection behavior. By including talented undergraduates including nationally selected underrepresented minorities, this proposal also intends to foster the integration of research, STEM and a broad education. Furthermore, outcomes can also contribute to significant technological innovations, including foundation for novel control architecture for robot arms and teleoperators, improving human-computer-interface such as cognitive neural prosthetics and the widely used touch-user-interfaces (e.g. smart phones). This project is jointly funded with the UK's Economic and Social Research Council (ESRC) under the SBE-UKRI Lead Agency collaborative research opportunity. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.