New Approaches to Sensitivity Analysis in Observational Studies

Regents Of The University Of Michigan - Ann Arbor, Ann Arbor MI

Investigators

Abstract

While randomized experiments remain the gold standard for elucidating cause and effect relations, countless societally important "what-if?" questions cannot be addressed through clinical trials for a litany of reasons, ranging from ethical concerns to logistical infeasibility. For this reason, observational studies, wherein the assignment of group status to individuals is outside the control of the researcher, often represent the only path forward for inferring causal effects. While observational data are often inexpensive to collect and plentiful, regrettably, they suffer from inescapable biases due to self-selection. In short, associations between group status and outcomes of interest need not reflect causal effects, as the groups being compared might have considerable differences on the basis of factors unavailable for adjustment. This project will develop new methods for sensitivity analysis in observational studies, which answer the question, "How much-unmeasured confounding would need to exist to overturn a study's finding of a causal effect?" Quantifying the robustness of observational findings to hidden bias will help frame the debate around the reliability of such studies, allowing researchers to highlight findings that are particularly resilient to lurking variables. This project provides both theoretical guidance on how to extract the most out of a sensitivity analysis and computationally tractable methods for making this guidance actionable. Moreover, when randomized experimentation is possible, the developed methods will help researchers use existing observational studies for hypothesis generation, enabling them to find sets of promising outcome variables whose causal effects may be verified through follow-up experimentation. This award includes support for work with graduate students. This project develops a new set of statistical methods for conducting sensitivity analyses after matching. These methods aim to overcome shortcomings of the existing approach, conferring computational, theoretical, and practical benefits. The project will provide a new approach to sensitivity analysis after matching called weighting-after-matching. The project will establish computational benefits, theoretical improvements in design sensitivity, and practical improvements in the power of a sensitivity analysis by using weighting-after-matching in lieu of the traditional unweighted approach. The project will also establish novel methods for sensitivity analysis with multiple outcome variables. These innovations will include a scalable multiple testing procedure for observational studies, facilitating exploratory analysis while providing control of the proportion of false discoveries, and methods for sensitivity analysis using weighting-after-matching for testing both sharp null hypotheses of no effect at all and hypotheses on average treatment effects. Finally, the project will establish previously unexplored benefits from using matching and weighting in combination, two modes of adjustment in observational studies commonly viewed as competitors. This will help bridge the divide between matching estimators and weighting estimators in the context of a sensitivity, in so doing providing a natural avenue for theoretical comparisons of these approaches. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.