Tracking the Process of Data-Driven Decision Making: Exploring the Use of the Instructional Systems of Practice (ISOP) Framework to Transform Undergraduate STEM Education

University Of Wisconsin-Madison, Madison WI

Investigators

Abstract

This project is addressing the measurement of instructional practices and administrative decision-making concerning undergraduate STEM courses. In seeking to enhance the efficacy of pedagogical reforms, researchers and policymakers recommend that educators should utilize data-driven decision making (DDDM) systems. For this to work, the data must be robust, salient to local practice, and supported by adequate technical and administrative systems. Currently little is known about the nature of decision-making processes within STEM departments and pedagogical reform initiatives and this lack of high-quality data about instructional practice on individual campuses has retarded the use of DDDM at the postsecondary level. The goals of this study are to collect high quality data on instructional practice, to prepare reports based on these data for decision makers in higher education, and to subsequently examine the use of these data in decision making about pedagogical reforms. The core tool being used is the Instructional Systems of Practice (ISOP) framework. ISOP was developed as an approach to studying faculty teaching. It is designed to improve on existing data sources on teaching such as self-report surveys, unstructured observations, or student ratings. ISOP defines teaching to be a multi-dimensional practice comprised of course planning, classroom practice, and student interpretations of teaching efficacy. Classroom practice is studied using the Teaching Dimensions Observation Protocol (TDOP). STEM education leaders at three universities are committed to using the ISOP framework to help guide their decision-making. The participation of three universities is expected to define in greater detail the extent to which the ISOP framework can be used to enhance DDDM and efforts to transform undergraduate STEM education. The study will use of a longitudinal mixed methods case study design to study decision making and instructional practice over the course of three years at three research universities with active STEM education projects. Additionally this project is planning to provide professional development opportunities to administrators and STEM education leaders. Intellectual Merit: The ISOP framework is an innovative multidisciplinary approach that draws on established methodologies from cognitive science, naturalistic decision-making, and educational research. It does so in ways that promise to contribute new knowledge about the dynamics that underlie administrative decision-making and faculty practice, and provide better methods and measures for use in STEM education research. Broader Impacts: This research will determine to what extent the ISOP framework can be utilized in unique contexts engaged in undergraduate STEM education initiatives. It has the potential to be of significant value to researchers, educators and policymakers in the US.