Artificial Intelligence for Infectious Disease Imaging

Clinical Center

Investigators

Linked publications, trials & patents

Abstract

The availability of non-human (NHP) CT data from SARS-CoV-2 animal imaging studies at the Integrated Research Facility (IRF) and human CT data from the COVID-19 pandemic has presented an opportunity to explore the interaction of combining NHP and human CT lung lesion data to enhance deep learning (DL) image segmentation capabilities for subjects with the same infection. To enhance DL model accuracies developed with the Deep learning Medical Imaging SegmenTation (DMIST) pipeline, we have been working on leveraging available NHP data to improve lung lesion segmentation. We are exploring different methodologies such as developing NHP pretrained models and deep learning inter-species transfer learning. Experimentation of this has shown promising results where pretrained models and models trained on a combination of NHP and human CT data outperform models which are only trained on human data. This is motivating as the IRF-Frederick has a large collection of animal imaging datasets that could be leveraged to improve segmentation model performances. Currently, there are multiple convolutional neural network (CNN) frameworks have been successful implemented in the DMIST pipeline and has led to utilizing the DMIST pipeline workflow to tune and explore model parameters for improving lung lesion segmentation in both NHP and human CT data. In-line with making improvements to DL segmentation models, efforts to add a model comparison module to the DMIST pipeline has helped expand DMIST capabilities and workflows. The addition of the module allows users with little to no programming knowledge to be able to compare different model parameters to one another utilizing a simple configuration file. This helps enable IRF-Frederick researchers to quickly iterate through not only the model development process but also the evaluation process by quickly comparing any number of models all on a single benchmark dataset and aggregate results. Monitoring the new state-of-the-art (SOTA) developed CNN models for medical imaging applications could significantly reduce efforts to develop these models inhouse. Currently, the U-NET, U-NET Transformer (UNET-R), and Shifted Window U-NET Transformer (Swin-UNETR) are implemented in the DMIST pipeline and have been successful for lung lesion segmentation in SARS-CoV-2 subjects. However, the predictive accuracies of the lung lesion models are still lacking due to challenges related to collecting annotations. We explored the nnUNET model architecture which has been shown to be successful for a wide variety of segmentation tasks on medical images by training the model on NHP lung lesion CT data. We see that this SOTA model architecture is outperformed by the Swin-UNETR. This demonstrations SOTA DL segmentation models can be successful for various segmentation tasks but it should be investigated to find the optimal solution to solve IRF-Frederick related image segmentation needs. The AI team has built up the software infrastructure of the lab, enabling future analyses to be completed faster, more consistently, and without the need to edit the base code. Particularly, major improvements were made to the Deep-learning Medical Image Segmentation Toolkit model Training pipeline (DMIST-Train). New capabilities were added including cross-validated model training, additional models, and editable parameters. One major new capability that was added was the ability to configure the entire training process from a configuration file. This allows people without knowledge of the inner workings of the pipeline to run deep learning model training and experiment with hyperparameters. Another major new capability is the ability to train a self-supervised model, a burgeoning technique that is at the forefront of deep learning research. Each of these new capabilities required a significant overhaul of the code but was done in a modular way such that future new additions will be easier. The AI team also built an entirely new toolkit for running classical machine learning. These were based on previously written scripts to run the experiments for the SARS-CoV-2 Classification study but were rebuilt to be applicable to any new structure dataset. By rebuilding the scripts into modules, it enables us to expand the scope of experiments we could run to include regression model training and feature selection within the cross-validation loop. The rebuild also means that new plots and capabilities added to the toolkit can be easily applied to enhance all studies across multiple datasets. Significant contributions were also made to the groups Deep-learning Medical Image Segmentation Toolkit model deploy pipeline (DMIST-Deploy). Specific contributions include standardizing the Python environment setup process by creating conda and pip requirements/specification files, implementing bulk calculation of radiomic features from NIFTI filetypes, and adding additional metadata details to the output of radiomic feature calculation. The process for generating multiple training configuration files when optimizing parameters or performing a parameter grid search was automated. To enhance code quality, improvements were made to code readability and documentation by applying the standard Python formatting module "black" to the codebase, improving repository README files, and adding useful comments throughout the source code. Additionally, the AI team helped develop and implement best practices in collaborative software development on GitHub.