Situating Big Data: Assessing Game-Based STEM Learning in Context

University Of Wisconsin-Madison, Madison WI

Investigators

Abstract

This REAL project arises from the 2013 solicitation on Data-intensive Research to Improve Teaching and Learning. The intention of that effort is to bring together researchers from across disciplines to foster novel, transformative, multidisciplinary approaches to using the data in large education-related data sets to create actionable knowledge for improving STEM teaching and learning environments in the medium term and to revolutionize learning in the longer term. The project team aims to understand how to use data collected from the environment in which learning technologies are used to do the following: (1) allow automated assessment that takes the full range of classroom activities and discussions around use of the technology into account in providing customized feedback recommendations; (2) come to better understand how learning and the context in which it is happening interact; and, (3) provide theory-informed and evidence-based advice for refining learning approaches and activities. This will make it easier for teachers to manage ongoing assessment and to adapt classroom activities to learners' needs in learner-centered, project-based, and inquiry-driven learning environments. Results of this project will lay the foundations for making assessment regular, routine and ongoing and to take a fuller range of learning activities into account. This, in turn, will allow better personalization and ongoing feedback and scaffolding for learners. Results will enhance understanding of how to assess and foster not only disciplinary learning, but also disposition, identity development, and long-term participation. The PIs seek to integrate theories of situated cognition with analysis of big data. They will explore how to integrate clickstream data from technology with key forms of multimodal data describing the contexts in which the technology is being used, e.g., individual and group discourse (online and in-room), individual and curricular artifacts, classroom assessments, and school performance, to generate a data-driven methodology for: (1) understanding the learning happening in technology-rich learning environments; (2) assessing development and needs of individuals within those environments in ways that will suggest adaptations and scaffolding; and (3) investigating situated cognition. They aim to make it easier to manage ongoing assessment and to adapt classroom activities to learners' needs in learner-centered, project-based, and inquiry-driven learning environments. They will investigate how to (1) enable consideration of the full ecosystem of learning and data collected across it when assessing learning and engagement, and (2) identify what is working and not working to foster learning in a situation. They will demonstrate where and how useful data are situated in the learning ecology when learners are engaged in hands-on and discourse-rich learning activities, and how to use these data to assess effectiveness and impact of interventions. Their plan involves matching important patterns in hand-coded qualitative data to patterns of automatically collected data; this will allow them to identify the patterns in automated data collection that can be used as indicators of factors such as understanding, confusion, learning, and participation.