Standard: Promoting Ethics in STEM with Transparency Initiatives that Work: An Interdisciplinary Investigation

Carnegie Mellon University, Pittsburgh PA

Investigators

Abstract

The production of valid scientific findings in STEM fields depends crucially on the ethical practices of scientists. Research integrity is receiving increased attention because of recent instances of data-fabrication, growing complaints about failure to replicate published results, and studies showing that researchers admit to engaging in questionable practices that undermine the validity of scientific results. To promote ethical practices, many universities and research institutions encourage researchers to be more transparent in the research process. Transparency initiatives include encouraging researchers to pre-register their studies - that is, to submit a study plan prior to beginning the research - or provide public proof of compliance with ethical standards. Little is known about how well such transparency initiatives work in terms of preventing questionable research practices. The current project explores when transparency policies work, when they may backfire, actually encouraging unethical behavior, and how to improve their design to make them more effective. It is crucial to investigate how initiatives designed to promote integrity interact with human behavior and to understand the contexts that cause people to bend rules while still thinking of themselves as honest. The current interdisciplinary project combines insights from psychology, behavioral economics, and machine learning to advance an understanding of how policies designed to promote transparency in research affect individuals' propensity to commit violations of ethical conduct. This project uses a unique combination of machine learning analysis of real-world data, survey data, and online experiments to explore the predictors and causes of unethical scientific conduct. One study examines pre-registered research plans from a large public database of clinical studies. Selected research plans are matched to the academic journal articles that resulted from the pre-registered studies, and articles that are retracted - withdrawn from the journal because of scientific problems with the work - are noted. The pre-registrations of publications that were vs. were not retracted are compared using computational methods at the intersection of natural language processing and machine learning. The goal of this analysis is to identify features of a pre-registered study plan (for example, being shorter or using more vague language) that predict that the resulting article will later be retracted. This analysis identifies when a well-intentioned intervention (pre-registration) fails to promote responsible research conduct. Another study surveys STEM researchers about causes of scientific misconduct and perceptions of transparency initiatives aimed at preventing it. This work is complemented by online experiments aimed at investigating the psychology of misconduct in a task where individuals have incentives to misreport information for private gains, which is analogous to the incentives scientists experience to tease significant results out of their data. These experiments investigate the conditions under which transparency policies that encourage individuals to provide evidence of ethical conduct might have unintended effects, increasing rather than decreasing individuals' tendency to violate ethical standards. A final set of experiments tests behavioral interventions that directly address the psychological drivers of misconduct identified in the previous studies. Taken together, this interdisciplinary project informs the design of transparency initiatives that work in promoting ethical conduct in STEM. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.