Using Color as a Model System for studying Perception, Thoughts, and Action

National Eye Institute

Investigators

Linked publications, trials & patents

Abstract

Research in the Section on Perception, Cognition and Action aims to understand the neural structures and operations that give rise to perception and cognition: how does the human brain generate knowledge about the stuff in the world? Towards this end, we engage two broad, interrelated lines of research. First, we study vision. And second, we do comparative studies to test hypotheses about homologies of brain anatomy and function. In the last year, we have used a combination of approaches, including magnetic resonance imaging and psychophysics, magnetoencephalography, and fMRI-guided neurophysiological recording. Our work often leverages color as a powerful tool to uncover how sense signals are transformed into representations that can guide thought and action. We have made headway on three specific projects. 1. Geometry of the neural representation of colors The geometry that describes the relationship among colors, and the neural mechanisms that support color vision, are unsettled. We used multivariate analyses of measurements of brain activity obtained with magnetoencephalography to reverse-engineer a geometry of the neural representation of color space. The analyses depend upon determining similarity relationships among the spatial patterns of neural responses to different colors and assessing how these relationships change in time. We evaluate the approach by relating the results to universal patterns in color naming, and we found that the greater precision in naming warm colors compared to cool colors was evident by an interaction of hue and lightness. Additional experiments showed that classifiers trained on responses to color words could decode color from data obtained using colored stimuli, but only at relatively long delays after stimulus onset. These results provide evidence that perceptual representations can give rise to semantic representations, but not the reverse. Taken together, the results uncover a dynamic geometry that provides neural correlates for color appearance and generates new hypotheses about the structure of color space. This work has been published (Rosenthal et al, Current Biology, 2020). 2. Temporal dynamics of the neural representation of hue and luminance-contrast polarity Hue and luminance contrast are basic visual features, yet the timing of the neural computations that extract them, and whether they depend on common neural circuits, is unknown. Using multivariate analyses of magnetoencephalography data, we show that hue and luminance-contrast polarity could be decoded from MEG data and, with lower accuracy, both features could be decoded across changes in the other feature. These results are consistent with the existence of both common and separable neural mechanisms. The decoding time course was earlier and more temporally precise for luminance polarity than hue, suggesting that luminance contrast is an updating signal that separates visual events. The timing differences were not dependent on the task performed by participants. Meanwhile, cross-temporal generalization was slightly greater for representations of hue compared to luminance polarity, providing a neural correlate of the preeminence of hue in perceptual grouping and memory. Finally, decoding of luminance polarity varied depending on the hues used to obtain training and testing data in a pattern suggesting that luminance contrast is mediated by both L-M and S cone mechanisms. This work has been posted to the pre-print server Biorxiv and is under review. 3. Color properties of face-selective neurons What role does color play in the neural representation of complex shapes? We approached the question by measuring color responses of face-selective neurons, using fMRI-guided microelectrode recording of the middle and anterior face patches of inferior temporal cortex (IT) in rhesus macaques. Face-selective cells responded weakly to pure color (equiluminant) photographs of faces. But many of the cells nonetheless showed a bias for warm colors when assessed using images that preserved the luminance contrast relationships of the original photographs. This bias was also found for non-face-selective neurons. Fourier analysis uncovered two components: the first harmonic, accounting for most of the tuning, was biased toward reddish colors, corresponding to the L>M pole of the L-M cardinal axis. The second harmonic showed a bias for modulation between blue and yellow colors axis, corresponding to the S-cone axis. To test what role face-selective cells play in behavior, we related the information content of the neural population with the distribution of face colors. The analyses show that face-selective cells are not optimally tuned to discriminate face colors, but are consistent with the idea that face-selective cells contribute selectively to processing the green-red contrast of faces. The research supports the hypothesis that color-specific information related to the discrimination of objects, including faces, is handled by neural circuits that are independent of shape-selective cortex, as captured by the multistage parallel processing framework of IT (Lafer-Sousa and Conway, 2013). This study is published (Duyck et al, eNeuro, 2021).