Machine learning for medical imaging: automated disease diagnosis and prognosis

National Library Of Medicine

Investigators

Linked publications & trials

Abstract

Deep learning, a class of machine learning algorithms, has showed impressive results in several of our recent studies this year. In addition to its application to natural language processing, we have also seen its success in our medical image analysis such as processing chest X-rays, CT images, and various kinds of retinal images for autonomous disease diagnosis and prognosis. Together with our clinical collaborators on the NIH campus, we have previously text-mined over 100,000 radiology reports where our algorithm generated weak training labels to enable the development of advanced deep learning methods for automatically reading and classifying chest X-ray images. This work resulted in the release to the scientific community of ChestX-ray14 (https://nihcc.app.box.com/v/ChestXray-NIHCC), one of the largest publicly available chest x-ray datasets with over 10,000 downloads. We have also conducted research to assist in the screening of age-related macular degeneration (AMD), a leading cause of vision loss in Americans 60 and older. By leveraging cutting-edge deep learning techniques and repurposing big imaging data from a major AMD clinical trial, we previously developed a novel data-driven approach (DeepSeeNet) for autonomous AMD diagnosis that exceeded the performance of human ophthalmologists (retinal specialists in this case). Such a result highlights the potential of deep learning systems to assist in early disease detection and enhance the clinical decision-making processes. In 2022, we continued this line of research with an emphasis on other diseases such as cataract and heart disease prediction with retinal scans and new imaging modalities in radiology such as magnetic resonance (MR) images. Cataract is the leading cause of legal blindness worldwide.1,2 Its prevalence is predicted to increase further in the coming decades because of aging population demographics in many countries. The diagnosis and severity classification of cataract typically require in-person evaluation by an ophthalmologist.1,6 This may limit the accessibility of cataract assessment, particularly for individuals living in remote areas or in countries with few ophthalmologists. In this context, automated approaches to cataract diagnosis and classification have important potential advantages, including speed and accessibility, as well as accuracy and consistency. With Dr. Chew and her colleagues at NEI, we trained deep learning models to detect and quantify nuclear sclerosis (NS; scale 0.97.1) from 45-degree slit-lamp photographs and cortical lens opacity (CLO; scale 0%100%) and posterior subcapsular cataract (PSC; scale 0%100%) from retroillumination photographs. Our approach DeepLensNet performance was compared with that of 14 ophthalmologists and 24 medical students. For the 2 most common types (NS and CLO), the accuracy was significantly superior to that of ophthalmologists; for the least common type (PSC), it was similar. In addition to automatic disease diagnosis, our deep learning approach can significantly contribute to clinical research. For example, our previous DeepSeeNet software played a critical role in the recent discovery of The Third Macular Risk Feature for Progression to Late Age-related Macular Degeneration (AMD) by automatically grading reticular pseudodrusen (RPD) on color fundus photographs in the Age-Related Eye Disease Study (AREDS) dataset. Recent studies have shown that biomarkers on retinal images, for example, retinal blood vessel density or tortuosity, are associated with cardiac function and may identify patients at risk of coronary artery disease. In this work we investigate the use of retinal images, alongside relevant patient metadata, to estimate left ventricular mass and left ventricular end-diastolic volume, and subsequently, predict incident myocardial infarction. We trained a multichannel variational autoencoder and a deep regressor model to estimate left ventricular mass (4.4 (32.30, 41.1)g) and left ventricular end-diastolic volume (3.02 (53.45, 59.49)ml) and predict risk of myocardial infarction (AUC=0.800.02, sensitivity=0.740.02, specificity=0.710.03) using just the retinal images and demographic data. Our results indicate that one could identify patients at high risk of future myocardial infarction from retinal imaging available in every optician and eye clinic. On radiology, we focused on the accurate and reliable detection of abnormal lymph nodes in magnetic resonance (MR) images, as such is very helpful for the diagnosis and treatment of numerous diseases. However, it is still a challenging task due to similar appearances between abnormal lymph nodes and other tissues. Together with Dr. Summers and his team at the NIH clinical center, we proposed a novel network based on an improved Mask R-CNN framework for the detection of abnormal lymph nodes in MR images. Instead of laboriously collecting large-scale pixel-wise annotated training data, pseudo masks generated from RECIST bookmarks on hand were utilized as the supervision. Different from the standard Mask R-CNN architecture, there are two main innovations in our proposed network: 1) global-local attention which encodes the global and local scale context for detection and utilizes the channel attention mechanism to extract more discriminative features and 2) multi-task uncertainty loss which adaptively weights multiple objective loss functions based on the uncertainty of each task to automatically search the optimal solution. For the experiments, we built a new abnormal lymph node dataset with 821 RECIST bookmarks of 41 different types of abnormal abdominal lymph nodes from 584 different patients. The experimental results showed the superior performance of our algorithm over compared state-of-the-art approaches.