Precise Conditions for Permutation Tests to Control Multiple Testing Error Rates

Ohio State University, The, Columbus OH

Investigators

Abstract

The purpose of this project is to improve the control of multiple testing error rates. A popular approach to multiple testing is based on permutation. One motivation for the use of permutation-based testing is to reduce the conservativeness of the tests while still controlling error rates. Permutation-based tests are thought to achieve this by incorporating the multivariate distribution of the multiple test statistics. However, the hypotheses for which permutation testing is known to be appropriate are different from those usually considered in multiple testing. In this project, the investigators precisely describe model assumptions and conditions for permutation testing to control multiple testing error rates. In particular, the investigators study precise model assumptions connecting the marginal distributions of interest in multiple testing to the joint distributions known to be appropriate for permutation testing. While the methods studied in this project can be used in many fields, they will be particularly applicable to genetic association studies. Genome-Wide Association Studies (GWAS) are conducted to discover biological pathways and prognostic biomarkers, and to guide treatment. Some GWAS discoveries have failed to replicate, possibly due to statistical testing procedures lacking error rate control. In this project, the investigators study the conditions under which currently used methods are guaranteed to appropriately control statistical error rates. Such control will lead to better return on investments in developing drugs and treatments.