Integrating Radiologist Insights for Safe and Accurate AI-Assisted Prostate MRI Interpretation

University Of Delaware, Newark DE

Investigators

Abstract

Abstract: Advances in natural language processing can build on prior breakthroughs in image processing to offer clinicians new AI tools in the fight against prostate cancer, the second leading cause of cancer death. Current AI for interpreting multi­parametric MRI (mp­MRI) scans, the primary imaging modality in PCa, achieves high performance through training on visual encodings of human expertise (e.g. lesion annotations). Such models fail at translation to patient care because they do not derive and communicate their results through the standardized format accepted by clinicians, PI­RADS. The expertise encoded in PI­RADS reports offers a major resource for training AI to achieve clinical acceptance. The proposed research is needed to overcome two major gaps in knowledge necessary to develop advanced AI systems that can learn from both visual (annotations) and text (PI­RADS) exemplars. (1) Data availability: Public PCa data repositories provide mp­MRI scans and annotations but omit accompanying PI­RADS reports. (2) AI modeling: Existing AI approaches focus on image processing and lack the capacity to integrate complex radiologist expertise expressed through language. This study will test the hypothesis that PI­RADS reports can be made machine­readable and combined with visual data so that AI can be trained to interpret MRIs according to the reasoning processes of radiologists. Memorial Sloan Kettering (MSK) radiologists and University of Delaware (UD) researchers will collaborate to achieve the following aims. Aim 1): The MSK team will curate a comprehensive dataset by annotating 300 public and 50 MSK MRI scans with corresponding PI­RADS reports. The UD team will leverage GPT­4 to automatically extract radiologists’ reasoning processes, i.e., radiologist rationale, from each report. Aim 2): The UD team will develop a Prostate Vision­ Language Model (ProstateVLM), building on supportive pilot data to leverage medical foundation models and successfully integrate both images (MRIs and annotations) and text (rationales) in a uniform embedding space. ProstateVLM will be trained on the comprehensive dataset to accurately segment the prostate gland, anatomical zones, and lesions on MRI scans and align its segmentation with radiologist expertise. Aim 3): The UD team will evaluate whether inclusion of radiologist rationales from PI­RADS as training data improves MRI segmentation vs. training on images alone, measured by Dice Score, mAP, etc. The MSK team will evaluate whether ProstateVLM’s interpretations of MRIs align with those of radiologists, measured by a questionnaire scoring metrics for accuracy and completeness. To support future research, the curated dataset and ProstateVLM will be shared through the Cancer Imaging Archive (TCIA).