Machine learning methods in data science for clinical prediction and characterization

National Library Of Medicine

Investigators

Linked publications, trials & patents

Abstract

Clinical risk scores can degrade in new environments when the inputs are collected in a different way, and the problem may be magnified with changes to collection practices over time. To address unknown degradation levels of machine learning models that could in principle result in harm, we develop models robust to anticipated variations in time, by accounting for them during training, and by training suites of models that have known dependencies on reliable data elements. In other domains modeling approaches such as data augmentation, dropout, and equivariant graph neural networks, are used to improve model robustness to expected dataset shifts: e.g., substitutions, missingness, and translations, respectively. With this framework, we develop robust models specifically for the effects of time and censorship of electronic health records (EHR) data with applications to modeling inpatient and critical care processes and in national claims data for pharmacovigilance. For longitudinal tasks such as risk forecasting and reasoning about disease progression, EHR data quality can be improved by using both tabular and textual data, with the caveat that the textual data arrives after a delay. Better longitudinal models may be constructed if the concepts rather than the full text each are associated with times. While previous investigators have focused primarily on time vis-a-vis temporal relations, our work provides temporal alignment to tabular data to enrich forecasting data availability. To achieve this, we have investigated human-in-the-loop annotation and machine learning modeling, applied to a corpus of 300,000 de-identified notes and 70,000 patients. Our work draws upon and advances research in longitudinal visualization, large language modeling, and active learning. Our pilot studies in FY2023 demonstrate increased precision in identifying concept event times across 6,000 annotations and high levels of agreement with prior temporal relations work. The goal is to provide a high-quality and large-scale resource that enables improved clinical reasoning and longitudinal research. To serve clinical models that are fair with respect to equity and or equality, many machine learning models have been developed to optimize for predictive performance and some statistical measure of fairness. However, in applying these definitions to clinical settings, often the classical right-censoring problem arises, i.e., patients are lost to follow-up and the outcome is unknown. This not only transforms the classification (or regression) problem into a survival analysis problem, but it also prevents unbiased calculations of the statistical measures of fairness. Our work in FY2023 defines fairness measures amidst right-censoring and provides predictive algorithms to mitigate fairness at both the individual and group level. We introduce the algorithm fairness termsconcordance imparity and fair calibrationwhich capture desired notions of predictive performance and equity while being applicable to censored clinical data.