Categories and Criteria of Evaluation of Research Proposals in the Social Sciences and the Humanities

Harvard University, Cambridge MA

Investigators

Abstract

SES-0096889 Michele Lamont Princeton University The study will analyze the categories and criteria used by scholars to evaluate research proposals in the social sciences and humanities. Data will be gathered on the process used to evaluate research and training projects seeking funding from the American Council of Learned Societies, the Social Science Research Council, the Woodrow Wilson National Fellowship Foundation - Women's Studies program, the Princeton Society of Fellows, and an anonymous foundation in the social sciences. Of these programs, two are interdisciplinary competitions in the social sciences, two are interdisciplinary competitions in the humanities, and one is concerned specifically with funding research in the field of Women's Studies. The primary focus of the analysis is to explore (1) differences in how proposal in the social and sciences and humanities in general are evaluated, (2) investigate differences in the evaluation process of "interpretive and "empirical" disciplines, and (3) compare the criteria used in the elite competition, the Princeton Society of Fellows, with those used by other programs. The study asks: What categories of evaluation are used? How much weight is given to formal and informal categories of evaluation such as excellence, significance, originality, feasibility, social usefulness, political relevance, elegance, sophistication and objectivity? What criteria are used to assess proposals along these dimensions? Data will be collected through observing funding panels, interviews with panel members and an analysis of written evaluation of proposals submitted. Fieldnotes, interviews and written evaluations will be analyzed qualitatively and quantitatively using content analysis. Unlike most studies of the peer review process, this study is not aimed primarily at determining whether the evaluation process is fair, but rather at understanding the substance of how quality is assessed. It will contribute directly to promoting excellence in the social sciences and humanities. It is expected that empirical knowledge generated will aid experts in addressing normative questions, such as what should count for excellence in the fields under consideration. The project will also make a contribution to the study of distribution of grants and fellowships, a neglected topic in the sociology of higher education and sociology of knowledge.