Information Processing by Visual System Neurons

National Eye Institute

Investigators

Linked publications & trials

Abstract

The visual cortex of the rhesus monkey can be divided into several functional areas. Areas V1, V2, and V4 each have a crucial role in visual form and color processing. We have addressed the question of how these areas function during visual working, or short-term memory. In a delayed match-to-sample (DMS) task, monkeys respond to the reappearance of the sample image, but ignore other images. Lesions in area TE of IT cortex interfere with DMS performance. Thus, it is reasonable to assume that TE neurons play some role in DMS, but what might that be? Success in this simple visual memory task requires coding, memory, recall, comparison and decision steps. TE neurons are selective for visual features, and so a role in image encoding has been assumed. Here we examine the possibility that TE neurons may actually hold a short-term memory trace during DMS tasks. We recorded activities of 35 TE neurons from two rhesus monkeys. Eight black and white patterns were used as test stimuli. Responses of neurons within TE fluctuated considerably across repeated presentations of a single stimulus. We measured the correlations between trial-by-trial fluctuations in different task phases (sample, nonmatch, and match). The sample and match response fluctuations correlated more strongly than sample and nonmatch fluctuations, even though the interval between sample and nonmatch was shorter than the interval between sample and match (median variance explained: sample vs. match = 7.3%; sample vs. nonmatch = 1.9%).Such trial-by-trial correlation between sample and match responses is strong evidence for local storage of the short-term memory trace. These data lead us to propose a new theory of the role of TE neurons in DMS tasks, in which the TE neurons actually store the short-term iconic memory trace. The average activities of TE neurons are selective for images. However, they also showed a very surprising result: the deviations of the sample responses correlated more strongly with the match than nonmatch deviations. There seems to be no way to explain these correlations if the TE neurons only encode the stimulus, because noise in encoding must be independent across different stimulus presentations, and thus can not preserve fluctuations across events. Here we propose a new hypothesis of iconic, short-term memory: these TE neurons hold the memory trace of the sample image in the strength of its synaptic inputs using a form of one-trial-learning. This population of neurons thus forms a matched filter (the best filter for detecting the presence of a known signal in white noise) for the sample image. The power in the responses of the population of TE neurons, but not in individual neurons, is higher for the match than for the nonmatch stimulus. This new theory is consistent with an idea we proposed in 1992 (Eskandar et al.), when we showed that responses of IT cortex neurons contained information about the sample and current images in a DMS task. Then, we fitted the responses of the neuron with a model that multiplied the encoding of the sample stimulus, recalled from a memory store, by the encoding of the current stimulus. This measured the correlation between the two images and formed the basis for a decision by a higher center. The new theory also uses a multiplicative model, but based on the evidence in the accompanying abstract posits that the memory trace is stored in the TE neurons themselves. This greatly simplifies the structure of the model needed to perform the DMS task, because the memory store, recall and correlation are all combined in the functions of one neuronal population.