Collaborative Research: GeoGaze: Gaze-Driven Adaptive Multimedia to Augment Geoscience Learning for Neurodiverse Learners

University Of Florida, Gainesville FL

Investigators

Abstract

Much of learning occurs today in multimedia environments. This project will design and test GeoGaze, a novel technology that uses eye tracking to augment learning with multimedia in real time. GeoGaze analyzes and predicts effective strategies for students based on differences in working memory capacity and changes the presentation of learning materials to help everyone view and learn information. Investigators will first study what visual attention strategies work best for learners with different levels of working memory capacity. This information will be used to build a gaze-driven technology to help each learner use the most effective strategies by adapting the learning materials in real time and for each learner. Examples of this include predicting where learners will look on the screen, what order they will explore the content, and then highlighting the most important elements in the sequence to be viewed or blurring the less relevant content before revealing when appropriate. The results of this project will help us understand how individual differences in visual attention and working memory capacity influence multimedia learning and how to adapt learning materials to improve learning in real time. This project seeks to engage more students in learning geoscience, a field not well represented by a diversity of students and researchers. Studies of multimedia learning traditionally focus on final learning outcomes but what happens during the learning process (e.g., visual attention strategies to integrate multimedia) is often unexplored. This project addresses these shortcomings to gain a robust perspective of how students with different working memory capacities learn authentic STEM content (geoscience) in multimedia environments, and how their cognition can be enhanced using gaze-driven adaptive learning technology, GeoGaze. This approach is based on a novel integration of behavioral and psychophysiological data as well as multi-layer backpropagation neural network analysis to predict media integration strategies. GeoGaze represents a novel gaze-driven technology for adapting multimedia learning in real time based on effective media integration strategies for each learner. This research will generate new empirical evidence for using real-time gaze-driven adaptation of learning materials to augment cognition and improve learning. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.