EAGER: Evaluating the Feasibility of Estimating Cognitive Load Using Microsaccadic Eye Motion and Oscillation in Pupil Diameter

Clemson University, Clemson SC

Investigators

Abstract

Systems that can detect and respond to their users' workload have the potential to improve both users' experiences and outcomes in many domains: students and teachers, drivers and pilots, rescue workers and soldiers might all benefit from systems that can detect when their jobs are too hard or easy and dynamically adapt the difficulty. Key to this promise is the ability to accurately estimate a person's workload without distracting them from their tasks. A promising, non-invasive approach is to estimate workload from eye movements, which are both known to correlate with cognitive activity and likely to become more widely available to computer systems as gaze tracking tools are developed for both commercial-grade web cameras and for devices that support augmented and virtual reality interfaces. However, commonly-proposed metrics for estimating workload, notably pupil diameter, have severe practical limitations around head motion, ambient light, and camera angle. Using a high-speed eye tracker and commonly used cognitive load and visual search tasks, this project will develop mathematical processing techniques to translate tiny eye movements ("micorsaccades") and pupil oscillation into workload estimates, then study the feasibility and accuracy of using those measures relative to pupil diameter and other ways of estimating workload. The work will take place in conjunction with several international partners, provide a student with international research experience, and help inform a future workshop on ubiquitous gaze sensing. The team will first develop and refine multiple candidate measures of cognitive load that focus on pupil motion and change. The first, Change in Pupil Diameter (relative to a baseline), represents commonly-proposed approaches. The second, the Index of Pupillary Activity, is based on a prior measure called the Index of Cognitive Activity derived from the oscillation frequency of pupil diameter ("pupil unrest"). It is likely to be better than raw pupil diameter, but still suffers from drawbacks around ambient light and off-axis distortion. The other is a technique based on microsaccadic activity that can be extracted from eye gaze positions rather than pupil diameter, ideally eliminating the drawbacks to diameter-based metrics. The team has already shown that their current versions of these metrics show promise in fixed-gaze tasks in which participants are asked to focus on a single, centralized fixation point while performing more and less difficult metal calculations. The next step is to generalize this work to fixed gaze at arbitrary angles, first by adapting the metrics to include techniques for off-axis pupil diameter compensation, then conducting similar workload tasks but where participants focus on one of a number of gaze targets located around the screen. This experiment to show that gaze can be shifted and off-center then sets up more realistic, unrestricted visual search tasks in which the team will ask participants to locate elevation-based terrain features in simulated maps that allow the team to control the location and workload involved in the search task.