Using Color as a Model System for studying Perception, Thoughts, and Action

National Eye Institute

Investigators

Linked publications & trials

Abstract

Research in the Section on Perception, Cognition and Action aims to understand the neural structures and operations that give rise to perception and cognition: how does the human brain generate knowledge about the stuff in the world? Towards this end, we engage two broad, interrelated lines of research. First, we study vision. And second, we do comparative studies to test hypotheses about homologies of brain anatomy and function. In the last year, we have used a combination of approaches, including magnetic resonance imaging and psychophysics, magnetoencephalography, and fMRI-guided neurophysiological recording, as well as developing new ethologically relevant behavioral paradigms. Our work often leverages color as a powerful tool to uncover how sense signals are transformed into representations that can guide thought and action. We have made headway on six specific projects. 1. Temporal dynamics of the neural representation of hue and luminance-contrast polarity Hue and luminance contrast are basic visual features, yet the timing of the neural computations that extract them, and whether they depend on common neural circuits, is unknown. Using multivariate analyses of magnetoencephalography data, we show that hue and luminance-contrast polarity could be decoded from MEG data and, with lower accuracy, both features could be decoded across changes in the other feature. These results are consistent with the existence of both common and separable neural mechanisms. The decoding time course was earlier and more temporally precise for luminance polarity than hue, suggesting that luminance contrast is an updating signal that separates visual events. The timing differences were not dependent on the task performed by participants. Meanwhile, cross-temporal generalization was slightly greater for representations of hue compared to luminance polarity, providing a neural correlate of the preeminence of hue in perceptual grouping and memory. Finally, decoding of luminance polarity varied depending on the hues used to obtain training and testing data in a pattern suggesting that luminance contrast is mediated by both L-M and S cone mechanisms. This work has been published in Nature Communications (Hermann et al, Nat Comm, 2022). We are currently analyzing the data to test for asymmetries in decoding luminance increments versus decrements to test hypotheses about the relative role of ON versus OFF signals in organizing visual representations in visual cortex. We have also launched a new series of experiments that aims to recover the geometry of the neural representation of color with finer sampling of color space (again using MEG), and to relate these representations to individual differences in behavioral similarity judgements. This work was presented at the Annual Vision Sciences Society Meeting (2023) and the Human Brain Mapping meeting (2023). 2. Behavioral and Neural mechanisms supporting color-shape learning Humans with normal vision can exploit both color and shape to detect, identify, remember, and recognize objects. This ability is enabled because many objects not only have characteristic shapes but also predictable colors: when ripe, strawberries are red, pumpkins are orange, bananas are yellow. Such associations are also readily forged for artificial objects, such as stop signs (red), smurfs (blue), and school buses (yellow-orange). Here we ask how macaque monkeys, another primate with the same retinal color-encoding mechanisms as humans, learn about the colors and shapes of objects. Our motivation is ultimately to understand the neural mechanisms that encode and integrate color and shape information to support visual behavior. Progress on this project has involved the deployment of an ethologically relevant behavioral paradigm in which the animals engage, at will, with a touch-screen foraging task. The same subjects have also been scanned with fMRI to determine the brain structures supporting color shape learning. We presented preliminary results on the experiment at the annual Vision Sciences Society Meeting in May 2023. 3. Color categories in monkeys Color categorization appears to be a universal human phenomenon. Widespread variability in culture-specific color terminology suggests that color categories are linguistic in nature; however, fundamental similarities in color naming and color categorization across languages suggests that there may be some underlying structure which is universally inherent to human cognition and neurophysiology. Studying categories in non-linguistic animals allows us to pick apart the relative contributions of language and innate factors. In this project, we developed a method to test for evidence of categorical behavior without explicit categorical training, based on a task which has been extensively used in the investigation of working memory. In this task, the participant is shown a colored circle on screen, which they remember the color of, and then after a delay, they select a circle of matching color from a set of differently colored circles. We presented preliminary results on the experiment at the annual Vision Sciences Society Meeting in May 2023. 4. Building up visual representations Substantial information about the mechanisms of vision has been provided by measuring behavioral and neural responses to simple stimuli, such as gratings, following the linear-systems approach first used to characterize neural responses in the retina. In this project we build on this work with three aims: First, we seek to test the extent to which the macaque is a model of the human case, by using stimuli and a measurement technique (fMRI) that has been used to characterize brain responses in humans. Work comparing brain structure and function across macaques and humans is important not only for establishing the limits of the macaque as model system, but also for understanding the evolutionary history of primate brains. Second, we aim to assess the extent to which retinotopically defined regions (V1, V2, V3, and V4) and their visual field representations (upper versus lower visual field; left versus right visual field; central versus peripheral visual field) can be distinguished by responses to simple grating stimuli. This assessment is relevant not only to questions about the contribution of different areas to visual processing, but also to the on-going debate concerning the organization of V4 in monkeys, and the relationship of V4 in monkeys to V4 in humans. Finally, we want to establish an analysis pipeline and open-access data repository to compare macaque and human brain structure and function. Establishing the extent to which macaques and humans have similar brain organization and function will require work by many research groups using multiple approaches. We aim to contribute to this objective by making the data and analysis pipeline publicly accessible. This work has been published as a preprint and was presented at the Vision Sciences Society Meeting in 2023. 5. Comparing functional organization of macaque monkeys and humans The macaque monkey is often used as a model of the human. We are doing large scale fMRI data-driven comparison of macaque and human visual cortex, and using multivariate analysis methods to quantitatively establish functional homologies across the species. 6. V1 Mechanisms of vision at the center of gaze. We have developed methods to precisely track eye position at a resolution approximating the scale of photoreceptors. We are using this method together with array recordings in V1 to establish photoreceptor resolution spatiotemporal receptive field maps of V1 cells, to work out the mechanisms that build up V1 receptive fields from retinal inputs.