Doctoral Dissertation Research: Managing the mixed messages of meta-analysis: How surgeons, policy makers, and judges cope with uncertainty

Northwestern University, Evanston IL

Investigators

Abstract

This doctoral dissertation research improvement award supports a study that examines how surgeons, policy makers, and judges interpret uncertain evidence. The goal of the study is to contribute to an understanding of how scientific evidence informs policymaking. The study focuses on meta-analysis, an increasingly authoritative form of evidence in which individual research studies are synthesized in a way that is said to facilitate scientific consensus but in doing so introduces new forms of uncertainty. For surgeons, the general conclusions of meta-analysis may be difficult to reconcile with the individual needs of patients; for policy makers, meta-analysis may back an intervention that worked at the sites where it was studied but is not compatible with the needs of their constituents; and for judges, meta-analysis may conflict with the court's adversarial process and written standards of evidence. By comparing these three cases, this research will contribute to a general understanding of how the organizational constraints, agendas, and values of these actors impacts how they interpret ambiguous evidence. The findings of this research may contribute to reforms in the way that decision makers in various domains evaluate scientific evidence in contentious debates. To study these cases, the researcher will interview policy makers and surgeons, and analyze archival documents, such as court transcripts. The use of mixed methods will permit an understanding of the social and historical context in which meta-analysis becomes a useful, if fraught, form of evidence in each of these domains. The intellectual merit of this project is that it takes up a classic question in science and technology studies, how scientists and their audiences manage epistemic uncertainty, in the light of a powerful new form of evidence across three heterogenous domains. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.