Machine learning and artificial intelligence research for clinical medical image processing

National Library Of Medicine

Investigators

Linked publications, trials & patents

Abstract

ML/AI driven automated computer-aided diagnostic (CADx) tools are designed to detect, localize, classify and grade disease in medical images to augment human-expert decision-making, add efficiencies, and improve overall performance. However, reliability of the predictions depends significantly on several training data characteristics which include quality, imbalance between cases and controls, volume, variety, and the truth standard . The prediction performance and reliability also depend on the AI architecture and frameworks used, foundation models used for initial training, use of multimodal data, how synthetic data is incorporated in training. Toward this, I focused my research on various medical image analysis tasks such as quality assessment, image enhancement, region of interest detection and segmentation, image classification and prediction interpretation. Several advances were made to address these topics through applications to diseases of interest. We continued our research on ensemble learning techniques and multimodal (text + image) learning which promise to provide benefits from combining the predictions from different models and result in improved generalizability and overall accuracy. We also added novel generative AI research to synthesize high quality images. Advances in these areas were embedded in various disease-detection driven ML/AI research efforts. My research contributed to advances in effectiveness of modality-specific pretext learning strengthened by image denoising and deblurring toward enhancing the classification of pediatric chest X-ray (CXR) images to distinguish normal from abnormal. We benchmarked our algorithm performance against a traditional transfer learning approach. Our findings reveal that models developed from CXR modality-specific pretext encoders substantially outperform the ImageNet-only pretrained model suggesting the method a viable alternative and promote further exploration of medical modality-specific technique for various medical imaging applications. I also studied the impact of data imbalance on AI prediction. Clinical data often has far fewer cases (minority class) compared to controls (majority class). It has been shown that synthetic image augmentation techniques can simulate clinical variability, leading to enhanced model performance. We evaluated the effectiveness of a text-guided image-to-image latent-diffusion model (LDM) for synthesizing disease-positive CXRs to augment a pediatric image dataset to improve classification performance. We studied two tasks-normal vs. pneumonia and normal vs. bronchopneumonia classification. My work showed that the augmentation significantly improved Youden's index (p<0.05) and markedly enhanced other metrics, indicating that data augmentation using LDM-synthesized images is an effective strategy for addressing class imbalance in medical image classification. Many AI algorithms are merely focused on a single anatomical target. Using the You Only Look Once (YOLO) object detection models we studied their potential for simultaneous detection of more than one anatomical organ. We evaluate their effectiveness in detecting lung and heart regions in CXRs simultaneously. Using a publicly available CXR dataset we evaluated the method on an internal and for generalizability evaluation used two external test sets. We showed that YOLOv9 models notably outperform YOLOv8 variants. We also showd further improvements in detection performance through ensemble approaches. My group also studied hallucination in deep learning (DL) classification, where DL models yield confidently erroneous predictions. We investigated whether binary classifiers are truly learning disease-specific features when distinguishing overlapping radiological presentations among pneumonia subtypes on chest X-ray (CXR) images. Specifically, we evaluated if uncertainty measure is a valuable tool in classifying signs of different pathogen-specific subtypes of pneumonia. We evaluated two binary classifiers to classify bacterial pneumonia and viral pneumonia, respectively, from normal CXRs. A third classifier explored the ability to distinguish bacterial from viral pneumonia presentation to highlight our concern regarding the observed hallucinations in the former cases. Our comprehensive analysis reveals that the normal/bacterial and normal/viral classifiers consistently and confidently misclassify the unseen pneumonia subtype to their respective disease class. These findings expose a critical limitation concerning the tendency of binary classifiers to hallucinate by relying on general pneumonia indicators rather than pathogen-specific patterns, thereby challenging their utility in clinical workflows. Expanding beyond images, my group studied deep learning (DL) algorithms that integrate multiple biomedical modalities. However, a majority of clinical data sets are unimodal and lack annotated reports paired with the images, thereby limiting the advance and use of multimodal DL algorithms. We proposed a novel strategy exploiting real and synthesized data in a multimodal architecture that encodes fine-grained text representations within image embeddings to create a robust representation of skin lesion data. Large language models (LLMs) were used to synthesize textual descriptions from image metadata that are subsequently paired with the original skin lesion images and used for model development. The architecture is evaluated on the classification of skin lesion images, considering nine internal and external data sources. The proposed multimodal representation outperforms the unimodal one on the classification of skin lesion images, achieving superior performance in every tested dataset. Oral cavity malignant lesion analysis: Oral cavity cancer is a common cancer that can result in significant impairments, and there is high mortality for the advanced stage. The final diagnosis is confirmed through histopathology, however high variability is observed among human experts in determining if a subject needs biopsy and identifying the correct biopsy location. Further, the disease can occur in different parts of the oral cavity. Toward developing an ML-based method that can help address these problems and reduce downstream classification errors, we automatically identify, with high accuracy, different anatomical sites in the oral cavity on the images that are verified using class activation maps obtained from both correct and incorrect predictions. We also use tabular data for predictions which informed us about the lesion characteristics that are key for detecting precancers. Several publications from this work are under review. Kaposi Sarcoma on Dark-skinned population: We developed methods to identify and grade dark skin and separate it from irrelevant background in the image. We also evaluated several methods to automatically locate potential KS lesions and then classify as them as being KS or not. Initial results in the study demonstrated improvement over human experts for detecting low-grade and high grade lesions. The results were comparable to human ambivalence for mid-grade lesions. Publication of this work is under review.