Artificial Intelligence in COVID-19, Procedural Medicine and Cancer

Clinical Center

Investigators

Linked publications, trials & patents

Abstract

A multidisciplinary multi-institute, public-private partnership tackled the goal of developing and validating AI tools and standardized methodologies for clinical dynamics and novel classification tools for medical imaging, voice analysis, data sharing, and detection and prevention of dynamic diseases. The NIH and extended team were among the first to gather multi-national data and develop freeware public AI solutions based on COVID CTs for academic, researcher, and commercial developer use. A uniform and standardized methodology for automatic classification of disease based on imaging, voice spectrograms, text input, or information from wearables could expedite the pathway towards drug discovery, infection outbreak and migration, and non-invasive quantification of disease such as vasculitis, post-operative clots or atelectasis, and segmentation of liver lesions or treatment zones. The NIH team developed and helped publicly post COVID-19 data and tools on TCIA and MIDRC, including the largest (summer 2020) chest CT dataset posted for the 1st year of the pandemic. NVIDIA and NIH co-developed AI models that detected COVID-19, differentiated from influenza, fungal, or bacterial pneumonias as well as other entities. AI models were able to predict the later need for critical care therapies based upon an initial CT scan early on, at the initial point of care. The public-private multinational partnership also used "federated learning" to train an AI model in 8 nations and 20 institutions that was able to predict subsequent oxygen needs based upon the initial point-of-care chest X-ray alone (Nature Medicine). This demonstrated methodology for data collaboration protects while maintaining privacy and allowing the data itself to remain at the home institution. Federated learning can in this way overcome shortcomings in unbalanced source data for AI, by sharing "model weights" instead of the actual data. This enabling technique overcome data sharing gaps, thus showing that the data does not need to be fully shared, in order to build quality AI models from medical imaging. The team also showed that CT AI can track disease in a predictable fashion in the pre-symptomatic, asymptomatic, and pauci-symptomatic patient, and that the general dynamic curve of disease has dynamic curve lab correlates may be predictive and recapitulate available preclinical models. Correlation with zip codes or cell phone towers could theoretically predict disease outbreak and migration patterns. CT image processing and deep learning models provide quantifiable metrics to serve as a noninvasive biomarkers for pulmonary involvement in COVID-19. A MICCAI AI data challenge in COVID-19 was organized around the data that the team curated. The NIH multi-national dataset (>3000 CTs /4 nations) showed that CT may be positive days before PCR. Thus, the suggestion that CT could function as a targeted epidemiological tool to perhaps augment PCR and antibody testing in specific limited scenarios or better define patterns of spread. Early signal for Omicron correlatives also led to development of a classification model purely from voice audiograms / spectrograms with high performance metrics, which was not true for Alpha and Delta. Partnerships with the Trans-NIH working group has been forged, including NIAID IRF, NIBIB MIDCR, NCI, NCATS, and N3C. Centralized communication and discovery pathways for COVID-19-related data science that involves medical imaging like CT or chest x-ray is a common theme and goal, and provides a fertile ground for advancement of data science with broad scope impact in cancer and in interventional radiology and well outside of infectious diseases. NIH participated in publishing and disseminating methods to perform spirometry with motion sensors, as well as voice as a vital sign for detection of disease. Further collaborations with industry, Oxford, AWS Strides, and Bridge to AI were developed for assessment of AI tools with PPG, wearables, and smartphone apps for voice AI. Public data posting of CT scans on public NCI TCIA websites were made in past years. AI deep learning models were made alongside of multiple industry partners, to educate on the serial temporal dynamics of diseases. AI deep learning models were built and shared for research purposes to automatically segment tumors, thermal ablation treatment zones, liver organ, or COVID-19 opacities, as well as try to classify COVID-19 or prostate cancer. NIH CC and NCI were among the first to gather multi-national data and develop freeware public AI solutions based on COVID CTs for both academic and commercial developer use in early years of the pandemic. Federated learning was piloted with academic and industry partners in several projects in Nature Medicine and JAMIA publications. The NIH team is working with commercial and academic partners to assess deep learning tools in cancer. Ongoing work will attempt to deploy voice models deployed on smartphones for pre-screening settings, and to assess voice input towards AI uses in healthcare. It was shown that pre-symptomatic CT AI can track disease in a predictable fashion. Prior work with extramural partners has demonstrated that federated learning can overcome shortcomings in unbalanced source data for imaging AI, and that the application of a specific federated learning technique can overcome the gap, thus showing that the data does not need to be shared in order to build quality AI models from medical imaging. This effort cross links with numerous campus efforts, including NCI/CCR efforts within NCI CCR MIB AI Resource, and extramural partners in Bridge to AI. CC/NCI team members also deployed a 3D-printed miniature ventilator in swine (now commercialized) as well as a disposable isolation bag device with in-line air filtration. Highly impactful AI models were developed and licensed towards the detection, characterization, and assessment of prostate cancer using MRI and MRI-US fusion biopsy. These models may have broad scope impact. Initial feasibility was begun for data curation of voice and wearable sensor data towards health care AI models. A large language model pathway for use of procedural imaging data and procedural endpoints were outlined with IR-GPT, and AI checklist for publications, and connections with MDRIC were initiated towards an AI toolkit for procedural medicine applications. A checklist for publications in AI was created for multiple simultaneous journal publication for the specialty of Interventional Radiology.