Simulating the Human Visual "Find" Command

University Of Oregon Eugene, Eugene OR

Investigators

Abstract

For most computer users, the visual display provides the highest bandwidth of information throughput. However, the full potential of the visual display is often not achieved. Visual layouts are often poorly designed, minimally evaluated, and based primarily on the intuitions and introspection of the designer. Though some visual interface designers have developed a good sense for what works, most have a very limited understanding of human visual perception, the planning and execution of eye movements, and how visual design interacts with those processes. Part of the problem is that no empirically validated comprehensive theories of applied visual search have been developed. Visual search is rigorously studied in cognitive psychology and human factors, but typically one phenomena at a time; this leads to theories that are disjoint, disconnected from design guidelines, and that do not provide much practical guidance. No integrative theories are sufficiently well formed to provide useful explanations or predictions of how people search the visual layouts in web pages and computer software. In this project, the PI will develop a predictive theory of visual search based on ecologically valid scenarios. The comprehensive theory will identify the interaction among visual-perceptual, cognitive, semantic, and oculo-motor processing during visual search of realistic computer screen layouts. The theory will be embodied in a computational cognitive model and integrate the core processes used in visual interaction. The PI will compile the theory into SimFind, a simulated human visual "find" command. SimFind will model and display the fundamental human processing that occurs when a search is triggered. This will be a software module that will stand alone, but which could be integrated into existing predictive tools to provide an accurate and robust human "find" within a larger human simulation. The module will implement the processing, components, and behaviors that are fundamental to visual search. The module will provide analytical predictions, as well as visualizations, of human-computer visual interaction -- the seemingly invisible processes that direct a user's attention across a layout. The PI will evaluate and refine the theory behind the simulated visual search with empirical studies that will include eye tracking. Empirical data will evaluate how the theory accurately captures the cognitive strategies motivated by the structure of the layout, and the shifts of attention that occur in response to the structure, visual features, and content encountered during the search. Broader Impacts: This interdisciplinary project will synthesize psychological theory and engineering models to provide the research community with advanced integrative theory and the commercial world with a highly practical application. The project will illuminate the otherwise invisible processes that take place during visual interaction, and will be useful to visual interface designers, students of design, human factors researchers, and cognitive psychologists. It will illustrate and predict human behavior for searching web sites, online voting and census forms, and general-purpose human-computer visual layouts.