Doctoral Dissertation Research: A Comparative Evaluation of International Models for Evaluating Federally-Funded Research

Western Michigan University, Kalamazoo MI

Investigators

Abstract

In the last few decades the evaluation of research has become a high-stakes enterprise. With increasing political governance and federal budgets often in the billions, the livelihood of individual researchers, research groups, departments, programs, and entire institutions often swing in the balance. Simultaneously, many of the longstanding principles and practices often lead to poor decisions about the actual or prospective merits of researchers and their research. This dissertation research describes, classifies, and comparatively evaluates national models used to evaluate research and allocate research funding in sixteen countries. These models vary widely in terms of how research is evaluated and financed, but nearly all share the common characteristic of relating funding to past performance. Independent, blinded panels of researchers and evaluators in two countries rated sixteen national models on more than 25 quality indicators. They then ranked the national models in terms of their validity, credibility, utility, cost-effectiveness, and ethicality. The results of the rankings show that the clear leaders are nations using large-scale research assessment exercises of various hues. Bulk funding and indicator-driven models received substantially lower ratings and thus ranked far lower in comparison to the assessment exercises. The project concludes with suggested implications for research evaluation practice and policy.