Measuring and modeling object similarity in the brain: combining conceptual and perceptual representations

University Of Rochester, Rochester NY

Investigators

Abstract

The brain uses similarity to generalize from the known to the unknown. For example, when a person encounters a new type of fruit and has to decide whether or not it is edible, the person must judge how similar it is to already known edible and inedible items. However, the type of similarity that is taken into account matters. A coconut can look like a rock (visual similarity), but for making a decision about edibility the fact that it hangs from a leafy tree (semantic similarity) is key. With funding from the National Science Foundation, Dr. Rajeev Raizada of Rochester University is investigating how the brain uses similarity to respond adaptively to changing circumstances. With an understanding of how types of similarity, such as visual and semantic similarity, are encoded in the brain, it should be possible to decode them from neural signals. In this project, Dr. Raizada is combining brain imaging with computational modeling and behavioral testing. He is developing novel methods of neural decoding to predict the similarity of brain patterns on the basis of computational models of the stimuli that people are perceiving. In addition, the methods are designed to investigate patterns across different people's brain activations. The novel computational methods being developed in the project could have significant broader impacts, for example, such techniques underpin brain-computer interfaces that attempt to restore communication to locked-in patients. Moreover, the modeling of semantic similarity in the brain has implications for disorders such as semantic dementia. There are also possible implications for technology. The brain responds flexibly to changing circumstances, but artificial systems, in contrast, are all too often brittle. When confronted with circumstances similar, but not identical, to familiar ones, they break down. Insights into how the brain generalizes from the known to the unknown have the potential to transform our knowledge of how the brain achieves its adaptability, opening up new avenues for endowing artificial systems with similar skills.