Research-Reporting Fidelity within Dietary Weight-Loss Supplement Scientific Literature

University Of Alabama At Birmingham, Birmingham AL

Investigators

Linked publications & trials

Abstract

? DESCRIPTION (provided by applicant): Research reporting fidelity refers to the integrity involved with making decisions about how to analyze and present data in scientific papers, talks, and press releases, such that our data and the methods used to generate them are described without artifice, without exaggeration, and with sufficient detail for other scientists t judge and replicate our methods. A current concern within health research is compromised research- reporting fidelity in many domains of inquiry. However, the literature on dietary weight-loss supplements may contain some of the most pervasive and egregious breaches. As a nation where the majority of individuals struggle with obesity, dietary weight-loss supplements are a popular resource used to lose weight. Any gap between evidence-based science and conclusions drawn about weight-loss supplements is of concern to biomedical research and to public health, given the number of studies conducted on these supplements and the number of persons taking such supplements. The objective of this proposal is to focus on two particular concerns: (1) a phenomenon commonly referred to as p-hacking defined as reanalyzing data using various tests until a p-value below the nominal alpha level is obtained, thus inflating the chances of obtaining false positive results, and (2) common errors in the calculation of effect sizes in meta-analytic research. This proposal aims to assess the magnitude of these problems, thereby laying a baseline for the evaluation of future progress in mitigating these problems, and to test an educational program to mitigate one of these problems. The first aim is to determine the prevalence of p-hacking within weight-loss supplement literature with unprecedented magnitude, speed, and efficiency by combining sophisticated statistical methods, such as multiple imputation and double sampling, as well as by employing the use of crowdsourcing. The second aim is to determine the frequency of effect size estimation errors that occur within peer-reviewed meta-analyses conducted on dietary weight-loss supplements, by systematically reviewing meta-analyses conducted on weight-loss supplements. The third aim is to develop and test the efficacy of three brief training modules for early career investigators designed to enhance effect-size estimation data reproducibility using weight-loss supplement meta-analyses, by conducting a simple randomized controlled trial. The methodological advances this proposal aims to foster will enhance the technical capabilities of future researchers who wish to employ the use of multiple imputation, double sampling, and crowdsourcing to investigate questions requiring the extraction of large amounts of detailed information from existing scientific literature. Additionally, the results obtained from this project will help improve the accuracy and reproducibility of dietary weight-loss supplement research, making accurate scientific information more accessible to the novice, the broader scientific community, regulatory agencies, medical decision makers, and for public health to help improve the health and life quality of persons taking, or considering taking, dietary supplements for weight loss.