Building Evaluation System Capacity for STEM Programs: Enhancing Capability and Advancing Practice

Cornell Univ - State: Awds Made Prior May 2010, Ithaca NY

Investigators

Abstract

This is an evaluation systems development project. It is based on the assumption that we know little about how evaluation systems are developed or can be themselves evaluated. This is especially so because of the complexity of evaluation demands given the range of current methods and approaches and contexts of evaluation. What is proposed is to use structured participatory mixed methods system approaches to: identify the needs that STEM educators and evaluators have for evaluation of their programs; develop a formal model of an evaluation system to address those needs; identify the major components of that system and how they might work together; and operationalize one or measures of evaluation capacity building that can be used to assess evaluation system development. The model and measures will be implemented and tested in a pilot project in a New York City that includes both formal and informal STEM education programs. The project is intended to contribute to our fundamental knowledge regarding the nature and structure of evaluation systems, how they can be developed more effectively and how to evaluate their development. By "evaluation system" the project means "an arrangement of various components - values and principles, methods, resources, and structures - designed to conduct and support an evaluation capability". Usually an evaluation system is a way to support many separate evaluation projects, and often with the express purpose of being able to integrate and compare the results of these distinct evaluations. The project does not claim there are no resources for evaluation. Rather it is that there are too many, or resources varying in quality and not screened or adapted for use in particular contexts. It is a lack not of resources but of systems for supporting evaluation. Thus, the proposed project is designed to: develop a model of STEM evaluation system needs and how they might be addressed; develop one or more measures or evaluation capacity that can be used to assess change efforts; and pilot test a capacity-building effort in two local STEM programs while simultaneously testing the evaluation capacity building assessments developed. Project partners have indicated support including the schools (the Biology Teachers Institute and the informal science Cornell Garden Mosaics project); STEM research groups at Cornell (nanobiotechnology project with education outreach, integrative graduate education and research traineeship project for scientists and engineers, science inquiry partnerships, and environmental inquiry IMD project.