Multiple Scales of Stimulus Representation in the Human Brain

National Institute Of Mental Health

Investigators

Linked publications, trials & patents

Abstract

In FY25, we have pursued three lines of research: we have studied 1) how the brain creates visual representations of stimuli in the environment; 2) how visual representations interact with internal "endogenous" brain signals associated with factors such as arousal, attention, and emotion; and 3) how these visual representations change over time; specifically, how visual representations degrade following brain injury, and how they can be restored through intensive visual training. Studies were carried out under clinical protocols NCT00001360 and NCT01087281. 1) Characterizing stimulus representations in human visual cortex. A central goal in the laboratory is to understand the computations that give rise to neural selectivity for stimulus features such as orientation, spatial frequency, and direction of motion. Our research has focuses on these features because they are hypothesized to depend on a set of core 'canonical' computations. A foundational premise of our research is that understanding canonical computations in the visual system will yield important insights into neural function throughout the brain, in both healthy and diseased states. We have recently developed a population model of V1 that provides a computational framework for investigating orientation selectivity using non-invasive measurements in human subjects, such as fMRI and MEG (Gardner and Merriam, 2021). Using this framework, we have demonstrated that three theoretically distinct neural mechanisms may each contribute independently to orientation selectivity: stimulus vignetting, neural gain fields, and asymmetric surround suppression. Our modeling results shows that these mechanisms can easily be confused with one another. For example, attempts to measure surround suppression, which is thought to be a marker for excitatory/inhibitory imbalance in cortex, and implicated in a wide range of neuropsychiatric disorders, could in fact be confounded with other neural mechanisms. In FY2025, we have applied this computational approach to neural responses in mouse superior colliculus (SC) (Kuo, Gardner, Merriam, 2025). The SC is a deep brain structure that is intimately involved in a wide range of sensory (visual, auditory, multimodal), motor, and cognitive functions. Recently, two-photon imaging studies in rodent SC suggest that the SC is orientation selective, much like the primary visual cortex in primates. If valid, these new imaging studies suggest a fundamental difference between the rodent and primate visual systems. Using our modeling approach, we have shown that these putative orientation selective responses in the rodent colliculus may arise from circular center-surround receptive fields (rather than elongated V1-like receptive fields). Our results suggest that mouse SC may be more similar to primate SC than has been previously believed. 2) The role of internal brain state in modulating visual representations A second theme in the laboratory is to understand the influence of non-sensory cognitive signals in visual perception. We have identified a type of brain activity that reflects a subject's general engagement in a task. This task-related activity contributes prominently to brain hemodynamic responses measured with fMRI, is independent of external visual stimuli, and instead reflects internal brain states. Task-related activity appears to be distinct from spatial attention in that it is global in cortex rather than retinotopically-specific. We hypothesized that this activity is related to general arousal. To test this hypothesis, we developed an approach that enabled us to isolate task-related activity from visually-evoked activity, and systematically vary a number of critical parameters, such as task difficulty, the expected reward for correct performance, and the temporal predictability of task structure. We found that a widespread fMRI response tracks arousal level in each of these conditions, exhibiting a significance correlation with pupil size and other physiological correlates of arousal (e.g., heart rate variability). We have recently shown that measurements of pupil size can be decomposed into separate attention- and arousal-related components and that the arousal component closely tracks the task-related fMRI response. Understanding the neural mechanisms of global brain states has implications for the study of several neurological and psychiatric disorders, including schizophrenia, autism and Attention Deficit Hyperactivity Disorder (ADHD). 3) Visual representation following stroke-related brain damage and recovery. We have measured neural responses with fMRI from human patients who have lost part or all of their primarily visual cortex following stroke. We then applied our computational approach to evaluate the degree of orientation selectivity in the spared portions of visual cortex. Our results suggest that orientation selectivity may be computed from higher-order visual areas that receive direct input from the thalamus, thereby bypassing the damaged portion of visual cortex. Our results offer exciting clues regarding how the brain may compensate after losing a key processing node. Our work on understanding neural computations in the intact and lesioned visual system is vital in determining how core neural computations are altered in psychiatric and neurological disorders.