Beyond Evaluation Use: Determining the Effect of Individual Project Participation on the Influence of STEM Overall Program Evaluations

University Of Minnesota-Twin Cities, Minneapolis MN

Investigators

Abstract

The project is designed to contrast the type and scope of influence of different approaches to program evaluation in order to increase the field's understanding of how evaluation results are used when evaluations are conducted for a large agency with both national and local implications. The project is grounded in two theoretical bases: 1) evaluation use, and 2) participation in evaluation. These are based on the theoretical work of Kirkhart, Greene, Gary and Mark, Patton, Burke, King and Lawrenz and Huffman. The primary theory of evaluation use is Kirkhart's three-dimensional influence of evaluation model - source of influence (process or results), intention (intended and unintended), and time (immediate, end-of-cycle, or long-term). Participation in evaluation draws on a continuum of Lawrenz and Huffman bounded by program evaluation conducted by an entity separate from the projects within the program to one when where the projects independently determine the evaluation procedures and what data to collect. ` The proposed research will investigate the continuum of participation and how it relates to use. The research questions are: 1) What patterns of participation exist in large, multi-site evaluation? 2)) To what extent do different levels of project participation in program evaluations result in different patterns of evaluation use? 3) What evaluation practices are most directly related to enhancing the influence of evaluations. Case studies will be carried out of four NSF Programs: ATE, LSC, CETP, and MSP RETAs. Within the case studies, methods will include artifact analysis, interview, surveys and site visits.