Artificial Intelligence for Infectious Disease Imaging

Clinical Center

Investigators

Linked publications, trials & patents

Abstract

The availability of non-human (NHP) CT data from SARS-CoV-2 animal imaging studies at the Integrated Research Facility (IRF) and human CT data from the COVID-19 pandemic has presented an opportunity to explore the interaction of combining NHP and human CT lung lesion data to enhance deep learning (DL) image segmentation capabilities for subjects with the same infection. To enhance DL model accuracies developed with the Deep learning Medical Imaging SegmenTation (DMIST) pipeline, we have been working on leveraging available NHP data to improve lung lesion segmentation. We are exploring different methodologies such as developing NHP pretrained models and deep learning inter-species transfer learning. Experimentation of this has shown promising results where pretrained models and models trained on a combination of NHP and human CT data outperform models which are only trained on human data. This is motivating as the IRF-Frederick has a large collection of animal imaging datasets that could be leveraged to improve segmentation model performances. Currently, there are multiple convolutional neural network (CNN) frameworks have been successful implemented in the DMIST pipeline and has led to utilizing the DMIST pipeline workflow to tune and explore model parameters for improving lung lesion segmentation in both NHP and human CT data. In-line with making improvements to DL segmentation models, efforts to add a model comparison module to the DMIST pipeline has helped expand DMIST capabilities and workflows. The addition of the module allows users with little to no programming knowledge to be able to compare different model parameters to one another utilizing a simple configuration file. This helps enable IRF-Frederick researchers to quickly iterate through not only the model development process but also the evaluation process by quickly comparing any number of models all on a single benchmark dataset and aggregate results. Monitoring the new state-of-the-art (SOTA) developed CNN models for medical imaging applications could significantly reduce efforts to develop these models inhouse. Currently, the U-NET, U-NET Transformer (UNET-R), and Shifted Window U-NET Transformer (Swin-UNETR) are implemented in the DMIST pipeline and have been successful for lung lesion segmentation in SARS-CoV-2 subjects. However, the predictive accuracies of the lung lesion models are still lacking due to challenges related to collecting annotations. We explored the nnUNET model architecture which has been shown to be successful for a wide variety of segmentation tasks on medical images by training the model on NHP lung lesion CT data. We see that this SOTA model architecture is outperformed by the Swin-UNETR. This demonstrations SOTA DL segmentation models can be successful for various segmentation tasks but it should be investigated to find the optimal solution to solve IRF-Frederick related image segmentation needs. The AI team has built up the software infrastructure of the lab, enabling future analyses to be completed faster, more consistently, and without the need to edit the base code. Particularly, major improvements were made to the Deep-learning Medical Image Segmentation Toolkit model Training pipeline (DMIST-Train). New capabilities were added including cross-validated model training, additional models, and editable parameters. One major new capability that was added was the ability to configure the entire training process from a configuration file. This allows people without knowledge of the inner workings of the pipeline to run deep learning model training and experiment with hyperparameters. Another major new capability is the ability to train a self-supervised model, a burgeoning technique that is at the forefront of deep learning research. Each of these new capabilities required a significant overhaul of the code but was done in a modular way such that future new additions will be easier. The AI team also built an entirely new toolkit for running classical machine learning. These were based on previously written scripts to run the experiments for the SARS-CoV-2 Classification study but were rebuilt to apply to any new structure dataset. By rebuilding the scripts into modules, it enables us to expand the scope of experiments we could run to include regression model training and feature selection within the cross-validation loop. The rebuild also means that new plots and capabilities added to the toolkit can be easily applied to enhance all studies across multiple datasets. Significant contributions were also made in the following areas: Multi-virus clustering project The CT scans from four distinct experiments involving 19 nonhuman primates (NHPs) exposed to the following viruses were investigated: Cowpox, Influenza A, Nipah and SARS-CoV-2 Lung abnormalities seen in the CT scans across multiple time points were examined. All CT protocols were controlled; however, NHP species, age, weight, dose, and route of inoculation varied across subjects. A radiology specialist on our AI team qualitatively graded each lobe-based lung scan using a standardized evaluation questionnaire. The qualifiers used were: GGO, (GGO + Consolidation, Nodular), (GGO, Nodular), No opacities, (GGO + Consolidation), (GGO, GGO + Consolidation), (GGO with septal thickening), (Nodular, GGO with septal thickening), each accompanied by a numerical value between 0 and 5 indicating the extent of the qualifier in the lung lobe. The qualitative features were one-hot encoded, then combined with the numerical features (the “extent”), feeding to the UMAP dimension reduction method, which reduced the dimensions to 2. These UMAP components (coordinates) were then fed to a K-means clustering algorithm, and different numbers of clusters were attempted to find the optimum number of clusters to describe the data on the scatter plot. We developed an understanding and modified the analysis pipeline so that the TensorFlow model was replaced by the DMIST model with the existing pipeline was thoroughly investigated. Further pipeline projects include an automated generation of summary reports for incoming computed tomography (CT) scans. These reports include subject demographic information, example images with interim organ segmentation shown in color as well as quantitative measures such as volume and density of the segmented regions. To solve the problem of differing fields-of-view in our CT scans, an auto-cropping feature for specific regions of interest (ROI) when segmenting medical images has been developed. This uses the TotalSegmentator algorithm to create an initial organ segmentation and then uses the centroid of that segmentation was the crop center for our DMIST model. In addition, radiomic features extracted from CT scans of the liver were investigated to select features relevant to Lassa virus disease severity. This work leveraged statistical analysis (t-tests) to select features that showed significant difference between baseline and terminal scans. These selected features were included in a correlation analysis with liver enzyme values.