RR: Collaborative Research: Tracking Scientific Progress in Social Psychology

University Of California-Davis, Davis CA

Investigators

Abstract

A basic principle of the scientific method is that scientists are expected to update their theories in light of new evidence. Indeed, public investment in science is assumed to produce theoretical advances that lead to real-world changes such as better health care, more efficient technology, and a deeper understanding of human behavior. Recent findings, however, raise concerns about the robustness and reliability of scientific research. Some published findings cannot be replicated by other researchers, and thus fail to meet basic standards of scientific value. To address these concerns, the scientific community needs to (1) collect new evidence, testing and re-testing existing theories, and (2) update theories in response to this evidence. Psychological science has been a leader in initiating the first step, with multiple large-scale replication projects underway to test the robustness of existing theories. The current project complements these efforts by providing a real-world test of the second step, examining whether scientists adjust their theories in response to new evidence. Documenting how scientists respond to new evidence is fundamental to establishing the value of research. This research tracks psychological scientists' beliefs in psychological theories before and after the release of results from two large-scale replication projects: Many Labs 5 (ML5) and Registered Replication Reports (RRRs). These projects offer evidence that reaches the highest standards of scientific rigor: researchers commit to methodology and data analysis strategy ahead of time, data are made available to the public, and results are published regardless of outcome. If psychological science is progressing as it should, this high-quality evidence should lead researchers to update their theories. If such updating is not happening, this project will help to determine why this might be the case and whether some researchers might be better than others in updating their theories. By identifying factors that promote scientific progress, and those that stand in the way, this project should contribute broadly to improving the robustness and reliability of research across all fields of science.