Multimodel Spaces for Robust Inference

Martingale Research Corporation, Plano TX

Investigators

Linked publications & trials

Abstract

DESCRIPTION (provided by applicant): Improving statistical methods to provide better inferences and new analytical capabilities for categorical regression models would be invaluable to the medical and health-related research communities. Presently, single regression models are used extensively to identify patterns of disease-related symptoms, screen for disorders, analyze the results of clinical trials, and for the assessment and justification of public health policies. However, while single model estimation and inference is widely used in health-related studies, such approaches neglect model uncertainty, thus abrogating the opportunity to: i) detect additional statistical regularities (e.g., treatment effects, risk factors), ii) improve the precision of statistical inferences for estimation and prediction/classification (e.g., patient screening, diagnosis), iii) control for overfitting (e.g., model selection bias), and iv) include different yet highly correlated risk factors. This Phase I study investigates the feasibility of combining robust estimators and specification analysis methods within a multimodel framework to create a robust multimodel estimation and inference technology that addresses the limitations of the single model approach. Robust multimodeling is a specific type of Frequentist Model Averaging (FMA) methodology. First, an important feature of this approach is that it provides robust confidence intervals on predictions and effect sizes averaged across multiple models, which simultaneously incorporate sources of uncertainty that arise from the presence of many different (yet equally appropriate) models of the same data generating process as well as sources of uncertainty resulting from sampling error. A second feature of our robust multimodeling approach is that it has a robust Bayesian Model Averaging (BMA) interpretation. Specifically, theoretical arguments establish that all inferences are robust with respect to the presence of model misspecification. Third, previous work in the BMA and FMA literature has tended to focus upon using the most probable models constrained within a model space by applying Occam's Window to identify a group of best models, rather than all possible models in computationally tractable model spaces. In this Phase I study, alternative strategies for multimodel estimation and inference involving large model spaces will be empirically studied with extensive simulations using realistic models on clinical trial datasets (NIDA-CTN, NIMH-STAR*D). Finally, Phase I feasibility results will provide the preliminary research and design for the Phase II prototype software and support technology dissemination through collaborative health-related research projects to establish the essential foundation for Phase III product commercialization.