Lessons-Learned from GSE Extension Services Grantees, 2005-2009

American Association For The Advancement Of Science, Washington DC

Investigators

Abstract

Intellectual Merit: The American Association for the Advancement of Science (AAAS) Center for Advancing Science & Engineering Capacity will review experience with the Extension Services (EXT) component of the NSF Research on Gender in Science and Engineering (GSE) program. The purpose of the review is to compile lessons-learned among all project cohorts since 2005 and evaluate how the intended concept translated into field experience. Extension Services grantees are awarded up to $2.5 million over five years, a large investment for a relatively small program budget. The first two projects funded have passed a three and a half-year mark. Collectively seven projects represent over 17 years of experience in the field, testing the concept of this particular investment (each for five years). EXT projects are engaged in the process of changing educational values, practice, and knowledge, a process that is neither linear nor predictable. Models for assessing the success of new initiatives range from short and expedient, e.g. briefings, to formal, comprehensive evaluation of outcomes. As new initiatives, EXT's lack a standard template or control for gauging success. Periodically assessing lessons-learned is one approach for doing so. The proposed work is exploratory in combining methods of evaluation research (formal outcomes measurement), systems analysis (review and documentation of organizational processes and functions in a consistent manner), management review (leaders describe their experience), journalistic reporting (interesting stories and facts), and anthropology (how people explain what they are doing in situ). The method is inherently novel, interdisciplinary, and pragmatic. Among key inputs are interviews with project managers, project documentation, reports and reviews gathered via the third-year reverse site visit, analysis of the concept as articulated in annual program announcements, and interviews with the NSF Program Director. Interviews with grantees will employ an Interview Guide Approach (per the Kellogg Foundation Evaluation Handbook). The goal is to develop an overview of how the lead EXT teams interpreted the solicitation with guidance from the Program Director, how they designed their version of an Extension Service, and what happened when they confronted the realities of implementation. Interviews will be conducted by staff of the AAAS Capacity Center, who are experienced in multiple methods among those cited above, especially evaluation theory and methods, and who are familiar with NSF's objectives and operations, the GSE program, the community of practice and the knowledge base related to issues of women in S&E, and who have personally produced many similar reports for university, corporate, and federal clients. Broader Impacts: Lessons-learned about GSE's Extension Services will potentially enable the program to refine the solicitation, guiding new teams toward productive strategies in organizing their services and reaching various communities not currently participating. To the extent certain kinds of EXT events and relationships are found to be promising, they might be adapted elsewhere. Lessons-learned is not a substitute for rigorous formal project or program evaluation; it is expedient and pragmatic in real time, elucidating what is useful to others and NSF as "field experience," capturing significant experience as these large-scale projects evolve. Taken together, these lessons constitute a portfolio assessment of projects that can guide current and future performers as well as NSF management.