Machine learning and artificial intelligence research for clinical medical image processing

National Library Of Medicine

Investigators

Linked publications, trials & patents

Abstract

1. Machine Learning and Artificial Intelligence for Biomedical Images Automated computer-aided diagnostic (CADx) tools driven by ML/AI methods based on deep learning (DL) are designed to detect and differentiate disease in medical images to improve automated disease prediction and add efficiencies to human performance. Toward this, we focused our research on various medical image analysis tasks such as quality assessment, image enhancement, region of interest detection and segmentation, image classification and prediction interpretation. Several advances were made to address these topics through applications for diseases of interest. Novel works done this year include image quality assessment for cervical and oral cavity images and echocardiography images and videos. We also developed a novel unsupervised registration method for cervical cancer image sequences which resulted in a stabilized sequence toward improving visual (or automated) assessment of lesions. We developed a variety of novel ML methods and learning strategies toward improving their prediction performance. These included ensemble learning techniques which provide benefits from combining the predictions from different models and result in improved generalizability and overall accuracy. Other improvements in learning strategies were modality-specific pretraining of deep models. We also considered various other learning strategies, such as unsupervised, semi-supervised, self-supervised, deep-metric, multiple instance, and federated learning, to overcome small data set size, inadequate expert annotated labels, and case-control imbalance. Automated predictions for medical diagnostic applications demand uncertainty quantification to gain user trust which traditional ML models do not directly provide. Further, the training data might not contain the extent of disease exhibited by different populations and disease comorbidities. Therefore, we developed techniques to measure uncertainty and use human-in-the-loop expertise to actively learn new information using an open-world learning strategy to improve prediction capability. Case-control imbalance is well-known in medical image classification thereby biasing the predictions toward the majority class. We contributed toward advances in model calibration for alleviating these effects. We also benchmarked various state-of-the-art loss functions which are used in ML model training, systematically analyzed model performance, and proposed improved loss function selection strategies to counter prediction bias effects. 2. Disease-based ML/AI Research All software codes and data were made publicly available where possible. Chest X-ray bone suppression: Automated bone suppression methods would increase soft tissue visibility in chest X-rays (CXRs) and enhance automated disease detection. We developed DeBoNet, a DL algorithm to suppress bones in frontal CXRs. The DeBoNet was then applied to case and control standard digital CXR images. We observed that the model trained on bone-suppressed CXRs significantly outperformed the model trained on non-bone-suppressed images in detecting COVID-19 manifestations. Cardiovascular disease: Automated echocardiography (echo) analysis is benefited through use of machine learning for tasks such as image quality assessment, view classification, cardiac region segmentation, and quantification of diagnostic indices. We proposed a novel and efficient DL-based real-time system for echo analysis and quantification. It uses a self-supervised modality-specific representation. We evaluated the proposed system using four echo datasets. Cardiac indices extracted by the system had high agreement with experts. We also developed an open world active learning approach for echo view classification, where the network identifies images of unknown views. The system alerts the users to label unseen samples which are then integrated into the model thereby increasing the classifier robustness. Tuberculosis: Automated segmentation of tuberculosis (TB)-consistent lesions in CXRs using DL methods can help reduce radiologist effort and supplement clinical decision-making. In the first study of its kind, we evaluated the benefits of using fine-grained annotations of TB-consistent lesions toward training ensembles for semantically segmenting TB-consistent lesions in both original and bone-suppressed frontal CXRs. Results showed that the stacking ensemble demonstrated superior segmentation performance. In a separate study, we investigated the benefits of selecting an appropriate loss function and quantifying uncertainty in predictions for segmenting TB manifestations in CXRs. Highly uncertain cases are referred to an expert thereby adding reliability to the classifier. We were the first to also analyze lateral CXRs using an ensemble of modality-specific convolutional neural networks (CNN) and vision transformer models (ViT) and obtained significantly superior performance which was verified using attention maps to highlight the discriminative image regions. Cervical cancer: Colposcopic appearance is often evaluated based on static images that do not reveal the dynamics of acetowhitening. We compared the accuracy and reproducibility of colposcopic impression based on a single image at 1 minute after application of acetic acid versus a time-series of 17 sequential images over two minutes. Use of the time-series increased the proportion of images classified as normal, regardless of histology. However, substantial variation exists in visual assessment of colposcopic images using 17-image time series. For ML-based image evaluation, as a first step, we developed an image registration method to automatically spatially align dynamic images without the need for a manually-provided reference standard which improved over previously reported results. Cervical tissue ablation is an effective treatment approach for excising high-grade precancerous lesions. Following our previous work that automatically determined if a cervix was eligible for ablative treatment based on visual characteristics presented in the image, we investigated the use of an image augmenter followed by a customized classification CNN to overcome the challenges due to insufficient training data. We built the image augmenter using a CycleGAN model that was trained using three different datasets to ensure that the augmented images contain clinically significant morphological features. We gained a performance improvement in treatability eligibility classification. Oral cavity malignant lesion analysis: Oral cavity cancer is a common cancer that can result in significant impairments, and there is high mortality for the advanced stage. The final diagnosis is confirmed through histopathology, however high variability is observed among human experts in determining if a subject needs biopsy and identifying the correct biopsy location. Further, the disease can occur in different parts of the oral cavity. Toward developing an ML-based method that can help address these problems and reduce downstream classification errors, we automatically identify, with high accuracy, different anatomical sites in the oral cavity on the images that are verified using class activation maps obtained from both correct and incorrect predictions. Noting that a ruler is placed near a suspected lesion to indicate its location and as a physical size reference, we evaluated the performance of two deep-learning networks: ResNeSt and ViT, to automatically identify images with rulers. The findings were verified using heatmaps generated using three saliency methods. We also developed an automatic method for extracting the measurement information on the ruler which can help measure the lesion size. Our method is resilient to various ruler styles, visibility completeness, and overall image quality.