Deep Learning for Survival Analysis, Causal Inference, and Conformal Inference,

University Of California-Davis, Davis CA

Investigators

Abstract

This research project will deploy deep learning to three mainstream research areas in statistics: survival analysis, causal inference, and conformal inference. Survival analysis studies event-time data, such as time to death, disease recovery, new employment, or equipment failure. Causal inference aims at assessing the causal effect of one variable (the treatment) on another variable (the outcome), while controlling for potential confounding factors. Conformal inference is a powerful tool to construct prediction intervals without relying on specific assumptions. These three areas have not yet fully benefited from the advantages that deep learning can bring, and the goal of this project is to fill this gap. This research falls squarely in the realm of data-centric AI (artificial intelligence) by tackling some of the key issues where Statistics can make major contributions in the age of AI. A major emphasis is the development of new methodology and theory that will be widely disseminated. The proposed approaches will be applied to various data, including data for breast cancer, hospital care, AIDS studies, and air quality. Codes for the algorithms developed in the project will be posted on CRAN for R or on github for Python. Student researchers will receive training in research, computing and communication skills. The research findings will be incorporated in graduate curricula, undergraduate research projects and short courses at workshops and will be presented at professional meetings. Project 1 (Deep Learning for Survival Data) will fill a void in deep survival analysis, referring to approaches that employ deep learning for the analysis of incomplete event-time data, by developing hypothesis testing procedures on two fronts: testing the significance of some specific covariates; and goodness of fit tests for survival models. A key feature of survival data is that they routinely involve incomplete observations, such as random right censoring, and therefore regression methods must be adjusted to account for such censorship. Deploying existing methods for inference and testing for deep learning approaches is challenging because of the ability of deep learning to detect the null model structure even while performing the optimal search in the full model. Consequently, conventional test statistics will vanish to zero asymptotically even under the null hypothesis. A new framework of hypothesis testing is thus needed to prevent the test statistic from approaching zero under the null hypothesis. To our knowledge, this is the first attempt to perform significance tests for censored survival data when deep learning is employed to model the risk function nonparametrically. Project 2 (Advances in Causal inference) addresses two problems of high relevance in causal inference: testing for continuous treatments and causal inference for censored survival data. Existing tests for continuous treatment effects fail to attain correct Type-I errors and therefore are not suitable when deploying deep learning. A new test procedure will be designed to resolve this problem with supporting theory. For survival outcomes, the conventional average treatment effect is shown to be ill suited for causal inference, motivating a new paradigm based on median or other quantiles to quantify treatment effects. Project 3 (A New and Improved Approach for Conformal Inference) explores a better conformal score that leads to improved conditional coverage probabilities compared to existing state-of-art score functions. The project will include the first theory for deep learning in conformal inference. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.