Training Biomedical Research Teams for Rigor and Reproducibility in Data Science

University Of Michigan At Ann Arbor, Ann Arbor MI

Investigators

Abstract

We will develop a training program to shape the thinking, impart skills and tools for rigor and reproducibility in data- and AI-intensive biomedical science, and ensure the application of such skills and tools in a wide range of biomedical research. Our program includes a learning phase (bootcamp with collaborative learning) and a mentored implementation phase to strengthen our trainees’ research and enable them to teach their newly acquired skills at their institutions. Our short-term goal is to shape the thinking of biomedical researchers and equip them with skills and tools to improve the rigor and reproducibility of their research. Our long-term goal is to have a long-lasting impact on research rigor and reproducibility through the transfer of skills from our trainees to their trainees, improve research outcomes, and to strengthen the biomedical data science workforce. Research projects with long data manipulation and analytics / modeling pipelines face rigor and reproducibility challenges throughout their lifecycle. Despite the efforts of the research community to promote rigor and reproducibility, there lacks systematic training for researchers to build the technical know-how to achieve this in practice. Our program will focus on seven topics: 1) Ethical issues in biomedical data science. 2) Data management, representation, data sharing with confidentiality considerations, and metadata. 3) Rigorous statistical design. 4) Design and reporting of predictive modeling. 5) Reproducible workflow. 6) Meta-analysis. 7) The rigorous use of generative AI in biomedical research. Our program will bring positive impact to the biomedical research workforce in a broad range of research areas. We will train researchers who use a wide range of data (from -omics data all the way to population data) to address research questions at various scales.   Our program will focus on upskilling the current research workforce and will train teams of faculty (project PIs) and technical personnel. They both play critical roles to ensure rigor and reproducibility, but may approach this from different perspectives. Training them together will allow them to benefit from each other’s scientific expertise and technical skills and address rigor and reproducibility in a collaborative manner. We will use a combination of training components (lectures, small group intensive sessions and team projects) through an online adaptive learning tool to individualize the training so that trainees can choose to focus on topics that are most important to them. All activities in this program will be performed in accordance with Executive Order 14151 and associated agency guidance.