Game-based Assessment: An Interdisciplinary Workshop Across Assessment, Business, and Education

University Of Minnesota-Twin Cities, Minneapolis MN

Investigators

Abstract

Game-based assessments (GBAs), defined as interactive experiences created through game design methods in which human knowledge, skills, abilities, and other characteristics (KSAOs) can be measured, and sometimes referred to as "assessment without tests", are becoming increasingly common across public and private organizations, education, and the military. Unfortunately, GBA research has been split across three fields -- organizational science, game science, and education -- without any systematic coordination or even much interaction between these silos. Meanwhile, GBA vendors have developed their own approaches to GBA with little guidance from researchers, sometimes creating biased and otherwise problematic assessments that are currently in use within the US education system, the hiring and promotion processes for US businesses, and the US military. Thus, the purpose of this workshop is to bring both researchers and vendors across applications domains together to create a roadmap for future research, to integrate knowledge across disciplines, and to connect researchers and vendors directly to facilitate basic research using real-world GBAs. We therefore recognize GBAs as a prime area reflecting two of NSF's "Big Ideas": Growing Convergence research in the context of the Future of Work at the Human-Technology Frontier. By bringing these fields together, we will create a research roadmap to better assess the complex KSAOs needed as work continues to evolve. We currently lack an integrative theoretical understanding or empirical evidence of the various cost-benefit trade-offs inherent to GBAs versus traditional measurement approaches, across relevant disciplines. There are two primary trade-offs commonly cited: 1) broader and more responsive KSAO measurement at the expense of focused measurement and 2) improved overall assessment utility despite increased development cost. Development of engaging and psychometrically meaningful GBAs to overcome these problems is more complex and multidimensional than survey development because prototype development requires large teams with diverse multidisciplinary expertise, yet researchers generally remain in tightly-defined disciplinary silos, developing parallel research literatures on GBA despite a general recognition of how this slows theory development. We aim to break across these disciplinary barriers with this workshop, to take the first steps towards an integrated research literature. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.