What Works for Whom in What Context: A Practical Guide to Conducting Evaluations With Diverse Populations?.

Science Museum Of Minnesota, Saint Paul MN

Investigators

Abstract

There is a need at NSF and in the field for a set of guidelines that would direct the evaluation process for projects whose purpose is to develop a more diverse STEM workforce. The purpose of this project is to create guidelines that would be used by PIs and proposal developers to write effective evaluation plans in their proposals; by evaluators to plan, implement and conduct rigorous project evaluations; and by Program Officers and reviewers to judge the merits of evaluation sections of proposals or to provide guidance for prospective or current projects. comprehensive guide, population-focused tip sheets, interactive internet materials and a wiki about conducting evaluations with diverse populations including considerations centering on gender, race/ethnicity, and disabilities. Consideration is also given to the setting of the evaluation (e.g., institutional type, program type). The materials will include information about designing, implementing and assess the quality of projects and activities that focus on broaden participation in STEM education, especially those targeting post-high school participants. The final products will be unique because they will specifically address evaluation issues related to diverse populations such as under represented minoritis, females, individuals with disabilities, and individuals in multiple categories. This project builds on previous NSF-funded work by the PIs individually and together. In addition, several specific NSF-funded or developed documents have led the way in formulating the idea and design for this project. Examples are the Framework for Evaluating Impacts of Broadening Participation Projects: A Report from a National Science Foundation Workshop, co edited by Beatriz Chu Clewell and Norman Fortenberry under Contract Number GS-10F-0482P by the Division of Research on Learning, the Indigenous Evaluation Framework Report by LaFrance and Nichols (Grant 0438720), and the Mixed Methods User-Friendly Handbook which was updated in 2010 (Contract REC 99-12175). The specific objectives of the project delineate the plan for the project, including a needs assessment with stakeholder groups, a literature review, the development and beta testing of materials, revision and field testing, and dissemination. The project includes an Advisory Board that has the expertise to provide important input to the process. The project also includes an external, independent evaluator and an evaluation plan that considers the progress and outcomes of the project. This project is a unique contribution to the fields of evaluation, broadening participation, and STEM education.