NSF East Asia and Pacific Summer Institute (EAPSI) for FY 2013 in Taiwan

Yu Chen-Ping, Stony Brook NY

Investigators

Abstract

This action funds Chen-Ping Yu of Stony Brook University to conduct a research project in the Computer and Information Science and Engineering area during the summer of 2013 at Perception and Attention Lab of National Taiwan University in Taipei, Taiwan. The project title is "Modeling Human Visual Attention using Object-based Attention." The host scientist is Dr. Su-Ling Yeh. This project is developing a visual attention model as measured by shift in gaze, which is based on object-based attention theory and takes object parts (proto-objects) into account in addition to the usual spatial features. Current visual attention models predict where people look next based on salient regions of a scene, such as noticeable objects or just features that are prominent somewhere in the image. This corresponds to spatially-based attention, such that attention is driven by interesting locations in the scene. However, recent work suggests that objects can serve as the attention selection basis, in a way that when a person is already attending at an object, changes that occur on the attending object is noticed faster than the same changes that occur away from the object, albeit the same spatial distances. This object-based attention has not yet been captured in predicting human visual attention, and it is the goal for this project to incorporate object-based attention processing with existing spatially-based models in order to predict gaze shift that are more consistent with human behavior. The research model predicts that the shift of eye fixation utilizes the theory of object-based attention, and it requires three major components for processing: objects, the associated object parts (proto-objects), and the saliency map. The top-down object information is being obtained by applying state-of-the-art object detectors in computer vision; proto-objects are being segmented by merging coherent pixel fragments (superpixels) that are similar in low level features; and the saliency map is being generated using multi-scale blob detector on the image, with blob strength denoting the degree of spatial saliency. Initial fixation is set at the most salient region, and whenever an object is selected as being fixated, object-based attention suggests that consequent fixations may prioritize on the associate proto-objects before moving to another spatial salient location, in contrast from existing models that may immediately release from the attending object if other spatial location is deemed more salient. The performance of the eye movement model will be validated against a human fixation dataset using 1000 images from the PASCAL VOC2008 dataset and 104 images from the SUN 2009 dataset. Broader impacts of an EAPSI fellowship include providing the Fellow a first-hand research experience outside the U.S.; an introduction to the science, science policy, and scientific infrastructure of the respective location; and an orientation to the society, culture and language. These activities meet the NSF goal to educate for international collaborations early in the career of its scientists, engineers, and educators, thus ensuring a globally aware U.S. scientific workforce. Furthermore, this project facilitates the learning exchange between the fields of Psychology (human vision) and Computer Science (computer vision). The model can also help and improve visual aids design for the general public, and the underrepresented groups that may require visual representations to be more intuitively perceived. This impact naturally leads to more partnerships between research, education, and the industry, which is all based on the better understanding of human visual attention.