Machine learning and artificial intelligence research for clinical medical image processing

National Library Of Medicine

Investigators

Linked publications, trials & patents

Abstract

1) Advances in ML/AI Methods for Medical Image-based Decision Making Automated computer-aided diagnostic (CADx) tools driven by automated AI methods based on deep learning are designed to detect and differentiate disease abnormalities using medical images with an aim to minimize clinical burden, reduce errors, and improve human performance. Medical image analysis techniques used in CADx systems typically include several tasks such as enhancement, segmentation, and classification. Traditionally, these tasks are implemented using separate deep learning models for separate tasks, which is not efficient because it involves unnecessary training repetitions, demands greater computational resources, and requires a relatively large amount of labeled data. The pipeline for segmentation comprises regions of interest (ROIs) detection stage and segmentation stage, which are independent of each other and typically performed using separate models. To obtain accurate results, the segmentation stage requires rich spatial features and sufficient receptive fields. To reduce computational complexity, we developed a multi-task training approach for medical image analysis, where individual tasks are fine-tuned simultaneously through relevant knowledge transfer using a unified modality-specific feature representation. We improved segmentation performance through an end-to-end Triple attention Network (TaNet) to extract rich semantic information at different levels of abstraction. Our results prove that the performance of target tasks in medical images is highly influenced by the utilized fine-tuning strategy. 2) Optical Image Analysis: Cervical Cancer Screening and Diagnostics Cervical cancer screening is done using a technique called visual inspection with acetic acid (VIA), during which dilute (3-5%) acetic acid (vinegar) is applied to the cervix during a vaginal speculum exam. Abnormal tissue temporarily appears white (acetowhitening). The test is a sensitive but non-specific sign of risk of underlying cervical precancer, the direct precursor to invasive cervical cancer. Even when performed by experienced health workers, VIA can be subjective and variable in distinguishing precancer from benign HPV-related changes and unrelated look-alike conditions, e.g., cervicitis. The clinical management depends on many factors including the age of the patient, parity, lesion size, and often requires confirmation of cervical intraepithelial neoplasia (CIN) by histological examination with subsequent surgical treatment of highgrade lesions (CIN 2 or CIN 3). We are developing a set of AI algorithms for cervical cancer screening and treatment. Our joint work has recently demonstrated that an AI algorithm, that we call automated visual evaluation (AVE), can predict disease better on digital cervical images than human practitioners and other laboratory tests. A central challenge in the development of a robust AI algorithm requires it to be agnostic to image quality variations due to human error, optical camera hardware, and automated image processing done by the device. Images from various cameras vary significantly in technical image characteristics. Additionally, the time at which an image is taken after the cervix is exposed to the acetic acid, and the extent of visibility of the squamo-columnar junction (SCJ) where cervical cancers arise also affect AVE decision. To enable portability and reliability of the AI algorithm, images from a broad variety of image capture devices, with labels indicating underlying truth of whether precancer is present, are being collected from around the world. While these big data sets aid training, they introduce domain shift problems and are highly imbalanced. They present a challenge for answering questions about number of images needed from various datasets to add new knowledge to the AI for various disease stages and comorbidities. Other research includes intelligent image transformation and data synthesis to reduce the variation, multi-dataset knowledge transfer, ensemble learning, active learning, and decision explanation. 3) Echocardiography Image Analysis: Sickle Cell Disease risk stratification Doppler echocardiography is valuable for the diagnosis and management of several cardiovascular diseases. We developed a novel and fully automated method to detect and analyze spectral Doppler waves used in assessment of diastolic function from mitral inflow, mitral annulus, and pulmonary pressure. Our deep learning methods use a self-supervised modality specific representation to enhance the learning of the target echo tasks on relatively small datasets. We also proposed a novel Trilateral Attention Network (TaNet) for realtime cardiac region segmentation. Our experimental results showed a consistent improvement in the performance of echocardiography analysis tasks with enhanced computational efficiency that charts a path toward its adoption in clinical practice. Specifically, our results show superior real-time performance in retrieving good quality echo from individual cardiac view, segmenting cardiac chambers with complex overlaps, and extracting cardiac indices that highly agree with the experts values. 4) Chest X-ray Image Analysis: TB screening We conducted several research efforts that use deep learning methods to improve abnormality detection in CXRs. In particular we focused on Tuberculosis (TB) screening. Although TB is a curable disease, with a decreasing mortality rate due to global efforts to improve TB control and treatment, multi-drug resistant TB is concerning. It is difficult to diagnose, and it takes more time for patients to recover. We are investigating the possibility of detecting drug-resistant TB in radiographs and other patient data, using machine learning and image analysis. Our methods achieved better than other results in the literature. We also showed that both clinical and radiological features are important. With the help of different augmentation techniques, usage of synthetic data and other publicly available sources, it was possible to achieve a higher discrimination between drug-resistant and drug-sensitive TB. The most recent work has been dealing with the generalization performance of the trained networks, such as the ability to perform equally well for data from different countries, and the explainability of results. For the latter, visualization techniques such as Grad-CAM have been used to create heatmaps showing the regions of interest, as identified by a trained network, overlayed over an X-ray. In other related research, we proposed a deep learning-based bone suppression model that identifies and removes occluding bony structures in frontal CXRs and helps improve TB detection. We also trained CXR modality-specific U-Net models to perform semantic segmentation of TB-consistent findings. We improved segmentation performance by augmenting the training data with weak disease localizations made by a classifier trained to detect abnormal CXR manifestations. We also demonstrated the use of iteratively pruned deep learning model ensembles for detecting pulmonary COVID-19 manifestations. Here, the best-performing individual models are iteratively pruned to reduce complexity and improve memory efficiency and are combined through different ensemble strategies to improve classification performance. 5) Microscopic Image Analysis: Malaria Screening The standard method for malaria diagnosis involves manual counting of blood cells and parasites under a light microscope, which is a tedious and error-prone process requiring expert knowledge. Our Malaria Screener app that runs on a smartphone attached to a microscope encapsulates research in ML/AI to detect and quantify parasites on both thick and thin smear images for both p. falciparum or p. vivax. It is currently being tested in a field study in Sudan.