Neural Coding of Visual Stimuli

National Institute Of Mental Health

Investigators

Linked publications & trials

Abstract

Primates, including old-world monkeys, can categorize images quickly based on similarity of visual features. The visual processing needed to carry out this type of categorization occurs as processing takes place in a sequence of visually sensitive brain areas called the ventral visual pathway. The neurons in the brain areas making up the ventral stream have receptive fields (RF) that become progressively larger as information passes sequentially through V1, V2, V4, area TEO, and finally area TE. We previously showed that both subregions of inferior temporal cortex, areas TEO and TE, contribute to visual categorization. Monkeys with TEO or TE removals show a deficit in categorization. The deficit is long lasting after the area TE removals, but the deficit recovers after several days of practice after bilateral TEO removals when they must categorize trial-unique images. To learn about the neural activity related to categorization, we recorded simultaneously from TE and TEO using implanted 96- or 64- channel arrays of electrodes (Utah arrays) while the two monkeys learned to distinguish cats from dogs. They were then tested with images of dogs and cats where the cats and dogs were mixed in different proportions, changing from cat to dog in gradual steps. We recorded the activity of many single neurons simultaneously as the monkeys learned to categorize the morphed images as more cat-like or dog-like across several days. We used an artificial neural network (a linear fully connected 2-layer, i.e., no hidden layer, network) for population-level decoding. The number of neurons that contributed to accurate decoding accuracy increased monotonically across the 7 testing days for one monkey and across the 8 days for the other monkey for TE neurons, paralleling what we had seen for the monkeys behavior. In TEO, decoding accuracy increased on the second day only, and showed no further improvement on subsequent days. At the end of this series, about 80% of TE neurons were contributing to decoding whereas only about 50% of the TEO neurons contributed. A further analysis of the neuronal activity showed that neurons in TE, not TEO, are increasingly modulated by image category during learning. These results show that neural activity in TE, but not TEO, correlates with learning to discriminate between visual categories. To learn more about how visual information is integrated to give rise to perception of objects, we are studying how the neurons in the visual system are connected. In the two subdivisions of IT, areas TE and TEO, the visual receptive fields of neurons extend across the midline, so that they include both contralateral and ipsilateral visual field representations. Previous studies reported that the representations of the ipsilateral visual fields of IT neurons depend on two sets of connections across the hemispheres, the anterior commissure and the corpus collosum. Combined transection of anterior commissure and splenium of the corpus collosum eliminates the visual response in the ipsilateral visual field. To study the source of the ipsilateral visual information, specifically which types of projections could contribute - feedforward, feedback, or contralateral - we injected nonreplicating viruses that induce retrograde transport of their gene products (FuGE) unilaterally into areas TE and TEO of two rhesus monkeys. This showed that area TE receives strong interhemispheric projections from contralateral TE, and weaker projections from contralateral TEO. Both sets of projections were mainly from layer III. The projections from contralateral V4 were very sparse. Area TEO received interhemispheric projections mostly from contralateral TEO. It also received significant projections from contralateral V2 and contralateral V4. The contralateral projecting neurons from TEO, V4 and V2 were located almost entirely in layer III. There were also sparse projections from contralateral TE to TEO, again, mainly originating in layer III. Thus it seems that area TE receives visual information about the ipsilateral visual field at the same hierarchical level, i.e., from area TE in the other hemisphere. In contrast, area TEO not only receives ipsilateral visual information from the other side of TEO, but also from feedforward projections of contralateral V2 and V4.