Designing a System for Improved Null Results Tracking: Berkeley, CA - December 2019

University Of California-Berkeley, Berkeley CA

Investigators

Abstract

This award provides support for a workshop on "Designing a System for Improved Null Results Tracking," to be held in December 2019 in conjunction with the Berkeley Initiative for Transparency in the Social Sciences Annual Meeting. The workshop will focus on the design of a sustainable, integrated, and multi-stakeholder system for improving the monitoring of and access to outputs from all funded research. Research projects that yield null findings are less likely to be published, yet still may serve as a valuable resource to both the public and the research community. While several mechanisms have begun to emerge to address this problem, these approaches tend not to be coordinated, and many institutions lack the capacity to adopt them. This workshop will convene an interdisciplinary group of research stakeholders to discuss existing and new approaches for improving the tracking of funded research outputs, with special consideration for projects that yield null results. The academic community, policymakers, and the public at large would benefit from more complete access to research results. Products for the broader community include recordings of live-streamed presentations, presentation slides, and a report synthesizing the workshop's proceedings and conclusions. This workshop will explore integrated approaches for addressing the inter-related problems of publication bias, questionable research practices, and the file drawer problem (where unpublished null or unexpected results remain locked away in researchers' file cabinets) in the social and health sciences. These problems can lead to incomplete and biased bodies of knowledge. Emphasis will be placed on striving for simplicity, lowering barriers to uptake of existing and new approaches, and avoiding high transaction costs for researchers and funders, as well as on ensuring compatibility with the values of scientific integrity and public interest. The event will begin with a discussion of design failures in how science is currently organized in research funding agencies and other institutions in order to build consensus within the group on priority issues to be addressed. Subsequent presentations and discussions will center on topics such as the effectiveness of study registries, the implementation of results repositories, how the benefits of reporting null results could be increased or the costs reduced, and strategies for research community monitoring of unreported results from registered studies. Participants also will deliberate about how such a system could feasibly be implemented and tested. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.