Assessment of Research Doctorate Programs: Methodology Study

National Academy Of Sciences, Washington DC

Investigators

Abstract

Assessment of the quality of doctoral programs and their faculty can be a powerful tool to improve doctoral education. For institutions, these assessments have been used in decisions to expand, contract, or merge programs. Potential graduate students use assessment results as part of the decision about where to apply and improved measurement could spur increased attention to the graduate student experience. State boards of higher education have used quality assessments to reallocate state resources. For administrators, the objective of "improving in the ratings" can be used to partially justify hiring and other personnel decisions. Scholars use ratings to examine the correlates of quality and as a yardstick to help them in designing policies to improve graduate education. These uses are consequential for all involved in doctoral education and the Conference Board of Associated Research Councils judges that a periodic review of the methodology of such assessment is essential given the significance of its use. The Conference Board of Associated Research Councils is composed of the American Council on Education, the American Council of Learned Societies, the National Research Council, and the Social Science Research Council. The National Research Council proposes to conduct a review of the methodology of assessment of research doctoral programs. The outcome of this review will be recommendations for the conduct of the next NRC assessment of doctoral programs. The methodology of assessing faculty reputation will be addressed, as will the construction of direct measures of effectiveness of the education of doctoral students. The study will examine the validity of measures of scholarly reputation and the extent to which they should be uniform across fields, the taxonomy of fields, and whether a broad reputational survey is the best way to assess research quality. It will also examine how to design and incorporate valid measures of the process of doctoral education and outcomes for recent graduates. A committee of experts consisting of statisticians, survey researchers, representatives of the major disciplines, and university administrators, will carry out the review. The committee will be assisted by information gathered from topical workshops, panels charged with focusing on taxonomy and measurement for each of the disciplines, and universities that have volunteered to collect data for prototype measures. The outcome of the study will be a report with recommendations designed to shape the next assessment of doctoral programs. Provided funding is obtained by summer of 2001, report completion is expected by February 2003. If the committee recommends a new assessment, it will take place in 2003-5 with data collected for the 2002-3 academic year. Total estimated cost for the project from July 2001 through February 2003 (20 months) is estimated to be $1.2 million. We are requesting $300,000 from NSF. Other prospective funders include the National Institutes of Health, U.S. Department of Agriculture, and the Sloan and Mellon Foundations.