Contextualized Evaluation Framework for Advanced STEM MOOCs

Purdue University, West Lafayette IN

Investigators

Abstract

The Promoting Research and Innovation in Methodologies for Evaluation (PRIME) program supports research on evaluation with special emphasis on: (1) exploring innovative approaches for determining the impacts and usefulness of STEM education projects and programs; (2) building on and expanding the theoretical foundations for evaluating STEM education and workforce development initiatives, including translating and adapting approaches from other fields; and (3) growing the capacity and infrastructure of the evaluation field. The PI and co-PIs at Purdue University will undertake an important research study designed to create a new paradigm for evaluating STEM MOOCs that go beyond analyzing clickstream data and could be used for evidence-based decision making. The goal of this new paradigm is to contextualize evaluation metrics by taking into consideration learner characteristics, course characteristics, and stakeholder goals. It is envisioned that this new paradigm would guide the development of the next generation of STEM MOOCs. The proposed full-scale, three-year PRIME project will develop and test a contextualized evaluation framework on over 56,000 learners enrolled in 15 advanced STEM (nanotechnology-related) MOOCs offered through the Purdue X and edX partnership, in conjunction with nanoHUB-U and Boeing. The project will result in the creation of data collection instruments and an evaluation method that make sense of diverse forms of learner data for action-oriented decisions to support learning and institutional goals in providing STEM MOOCs. The research study will employ a multiphase mixed methods approach to address the following research questions: (1) What constructs contribute to learners? behavior and performance in advanced STEM MOOCs? (2) What information is needed to inform stakeholders decision-making regarding the value of offering advanced STEM MOOCs? How do stakeholders needs drive the design of MOOCs? (3) What is a context sensitive, generalizable framework of evaluation for advanced STEM MOOCs? The results from the research study have the potential to make significant theoretical contributions to the evaluation field by developing a novel evaluation framework that would be more effective for determining the value of offering STEM MOOCs and the impact on learning outcomes.