Accelerating Professional Learning Through an Intelligent Performance Assessment System

Regents Of The University Of Michigan - Ann Arbor, Ann Arbor MI

Investigators

Abstract

Performance assessment in the workplace, whether a hospital, school, or lab setting, can be challenging. The purpose of this project is to enhance performance assessment in professional learning environments, particularly in surgical education, by developing novel, transparent, and rigorous statistical models. Medical education is increasingly using "micro" performance assessments that are collected over time and consist of one to three rating scales to evaluate a trainee's performance following a clinical activity. Despite their growing use, synthesizing multiple assessments into summary scores, integrating these summary scores with other evidence sources, and translating scores into acceleration, remediation, and certification decisions remain challenging. Identifying which trainees are struggling and why requires new ways of combining data to help the right individual at the right time and in the right ways. The approach developed in this project will help ensure that surgeons are better prepared for independent practice, ultimately leading to better patient outcomes. Overall, this project will contribute to a broader understanding of performance assessment in professional learning, enhancing public trust in newly certified professionals. This project aims to enhance performance assessment methodologies in surgical education and beyond using Bayesian inference networks. Bayesian networks are flexible and efficient probabilistic models that can be used to synthesize longitudinal data, enabling just-in-time evidentiary reasoning to drive educational decisions. In the context of surgery education, this project will summarize micro performance assessments using dynamic Bayesian networks, combine multiple sources of evidence into a quantitative professional profile using tiered Bayesian networks, and identity barriers and facilitators to collecting, analyzing, and reporting on multiple sources of assessment evidence using implementation science frameworks. Combining rigorous analytical techniques with implementation science will aid in identifying strategies to help ensure reliable assessment processes are carried out, score reports are useful to end-users, and educational interventions are tailored to meet the needs of developing professionals. This project is funded by the Research on Innovative Technologies for Enhanced Learning (RITEL) program that supports early-stage exploratory research in emerging technologies for teaching and learning. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.