Artificial Intelligence in COVID-19, Procedural Medicine and Cancer

Clinical Center

Investigators

Linked publications, trials & patents

Abstract

A multidisciplinary multi-institute, public-private partnership tackled the goal of developing and validating AI tools and standardized methodologies for clinical dynamics and novel classification tools for medical imaging, voice analysis, data sharing, and detection and prevention of dynamic diseases. A public pipeline for classification of COVID-19 (vs Flu) on chest CT was deployed. The NIH and extended team were among the first to gather multi-national data and develop freeware public AI solutions based on COVID CTs for academic, researcher, and commercial developer use. A uniform and standardized methodology for automatic classification of disease based on imaging, voice spectrograms, text input, or information from wearables could expedite the pathway towards drug discovery, infection outbreak and migration, and non-invasive quantification of disease such as vasculitis, post-operative clots or atelectasis. The NIH team developed and helped publicly post COVID-19 data and tools on TCIA and MIDRC, including the largest (summer 2020) chest CT dataset posted for the 1st year of the pandemic. NVIDIA and NIH co-developed AI models that detected COVID-19, differentiated from influenza, fungal, or bacterial pneumonias as well as other entities. AI models were able to predict the later need for critical care therapies based upon an initial CT scan early on, at the initial point of care. The public-private multinational partnership also used "federated learning" to train an AI model in 8 nations and 20 institutions that was able to predict subsequent oxygen needs based upon the initial point-of-care chest X-ray alone (Nature Medicine). This demonstrated methodology for data collaboration protects while maintaining privacy and allowing the data itself to remain at the home institution. Federated learning can in this way overcome shortcomings in unbalanced source data for AI, by sharing "model weights" instead of the actual data. This enabling technique overcome data sharing gaps, thus showing that the data does not need to be fully shared, in order to build quality AI models from medical imaging. The team also showed that CT AI can track disease in a predictable fashion in the pre-symptomatic, asymptomatic, and pauci-symptomatic patient, and that the general dynamic curve of disease has dynamic curve lab correlates may be predictive and recapitulate available preclinical models. Correlation with zip codes or cell phone towers could theoretically predict disease outbreak and migration patterns. CT image processing and deep learning models provide quantifiable metrics to serve as a noninvasive biomarkers for pulmonary involvement in COVID-19. A MICCAI AI data challenge in COVID-19 was organized around the data that the team curated. The NIH multi-national dataset (>3000 CTs /4 nations) showed that CT may be positive days before PCR. Thus, the suggestion that CT could function as a targeted epidemiological tool to perhaps augment PCR and antibody testing in specific limited scenarios or better define patterns of spread. Early signal for Omicron correlatives also led to development of a classification model purely from voice audiograms / spectrograms with high performance metrics, which was not true for Alpha and Delta. Partnerships with the Trans-NIH working group has been forged, including NIAID IRF, NIBIB MIDCR, NCI, NCATS, and N3C. Centralized communication and discovery pathways for COVID-19-related data science that involves medical imaging like CT or chest x-ray is a common theme and goal, and provides a fertile ground for advancement of data science with broad scope impact in cancer and in interventional radiology and well outside of infectious diseases. NIH participated in publishing and disseminating methods for handling COVID-19 in the angiography suite, details about post-partum COVID, designed and characterized a disposable isolation device ("full body mask") that reduces contamination in health care settings such as transport of COVID positive patients, validated in vivo a miniature 3D printable ventilator for resource-starved pandemic settings, and deployed a camera with custom software to identify social distancing distances with a standard webcam. A clinical trial for training AI models for Omicron detection from public social media audio data was IRB approved. Smartphone tools for instant anonynmization of imaging data were developed. A smartphone app for point-of-care deployment was created for running inference on clinical PACS 2D imaging or for cloud transmittal. Further collaborations with N3C, industry, Oxford and IRF NIAID were developed for assessment of AI tools with PPG, wearables, and smartphone apps.