Research on the Use of Real-Time Tracking and Eye-Tracking Technology for Integrating Metacognition and Augmented Reality into Undergraduate Engineering Laboratories

University Of Missouri-Columbia, Columbia MO

Investigators

Abstract

This project aims to innovate Augmented Reality (AR) learning platforms in an undergraduate engineering laboratory. Recently, due to the COVID-19 pandemic, in-person engineering laboratories faced significant challenges in providing hands-on exercise. AR technology might be a solution to the current challenge. Unlike virtual reality, AR does not cover a physical world but mixes 3D virtual objects into physical objects to improve students’ laboratory experiences and spatial awareness. However, students faced difficulties navigating the AR device and have experienced a mismatch between computer-generated 3D images and physical objects while learning in an AR environment. To transform AR learning from a static and isolated experience into a dynamic self-study, this project includes research on using a real-time tracking sensor with a 3D full-body motion capture system to improve AR usability and detect student’s premature termination of learning using metacognitive monitoring feedback and eye-tracking technology. The primary goal of this project is to integrate real-time 3D motion and location tracking systems to meet a series of objectives to gain insight into scientific development and technological innovation in a location-based AR environment. The first objective is to create the next generation of an AR learning platform by integrating state-of-the-art indoor real-time location technology and a 3D full-body motion capture system. The second objective is to detect the early termination of learning by using metacognitive monitoring feedback and eye movement data. The researchers will apply advanced gaze behavior metrics, such as fractal analysis of pupil dilation and eye inter-fixation duration, to identify significant eye gaze behavior representing student’s premature termination of learning. The last objective is to implement the research findings to promote instructional improvement in an engineering laboratory on the impact of receiving metacognitive feedback on learning performance. The current project will address integrated research and education activities to enhance our knowledge of effective undergraduate engineering laboratory education using real-time 3D motion and location tracking capabilities. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.