Design and Interpretation of Observational Studies

University Of Pennsylvania, Philadelphia PA

Investigators

Abstract

This award is funded under the American Recovery and Reinvestment Act of 2009 (Public Law 111-5). An observational study is an empiric investigation of the effects of a treatment, policy, intervention or exposure which was not randomly assigned to subjects, as it would be in a randomized experiment. Observational studies are common in most fields that study people ? such as economics, education, medicine, and sociology - because harmful or unwanted treatments cannot be imposed on human subjects for experimental purposes. The central difficulty in an observational study is that, because treatments were not randomly assigned, the subjects receiving different treatments may not be comparable, so differing outcomes after treatment may not be effects caused by the treatment. If the treatment groups differ before treatment in ways that have been measured, there is an overt bias that often can be removed by adjustments, such as matching. However, if treatment groups differed before treatment in ways that were not measured, concern about hidden biases may arise. Hidden biases cannot generally be removed by adjustments and must be addressed by other means; these are the focus of the current proposal. Four projects are planned for the current project. The first component considers the use of sample splitting to guide design choices with a view to increasing the design sensitivity parameter in an observational study. This means setting aside a small portion of the study sample to improve the design sensitivity for the remaining study sample. The second component considers ways to implement Sir Ronald Fisher?s proposal to ?make your theories elaborate? in observational studies, including testing such theories without problems of multiplicity, without losses of power, but with suitable sensitivity analyses for unmeasured biases. The third component concerns the interpretation of sensitivity analyses for unmeasured biases in observational studies. Because sensitivity analyses refer to what has not been measured, they tend to become both complex and difficult to interpret; combating this tendency, an amplification of a sensitivity analysis maps a simple sensitivity analysis into the large set of its possible interpretations. The fourth component concerns situations in which most subjects are unaffected by treatment, but a few are strongly affected. It turns out that such situations can be highly insensitive to unobserved biases, and a thorough study of this phenomenon is proposed.