TRIPODS+X:RES:Collaborative Research: Learning with Expert-In-The-Loop for Multimodal Weakly Labeled Data and an Application to Massive Scale Medical Imaging

University Of Pittsburgh, Pittsburgh PA

Investigators

Abstract

The methods developed in this project address pressing problems at the interface of machine learning and a life-sciences application. This work hopes to spur a rich variety of followup research that not only builds on the mathematical tools developed but also on the insights gained from the project's focus on interpretable and interactive machine learning. As a key application, implementation of the technology obtained from this research on medical images will enable automatic report generation. This technology could reduce the workload of radiologists significantly, and hence reduce human error and optimize resource utilization. Ultimately, reduction in human error or missing cases in radiology can greatly benefit patient well-being and care. More broadly, the work will enhance the ongoing adoption of data-driven thinking in healthcare and thereby help accelerate new discoveries. This project also impacts education, and involves intellectual and professional development of students at a variety of academic stages. Most applications of machine learning to medical imaging (and other human-centric tasks) focus on supervised learning, which demands a large amount of expensive labeled-data. This limitation recurs throughout applications, while real-world use of machine learning demands robustness as well as an ability to work with limited supervision. This project focuses on developing new machine learning tools for working with weak-supervision, while advancing state-of-the-art in interpretable and interactive learning. Moreover, attention to large-scale data and incorporation of domain knowledge is paid, whenever feasible. The project shall also apply the theoretical advances that it will make to a real-world medical imaging application. Theoretical advances of the proposed work will rely on tools from geometry, especially metric learning (including over infinite dimensional spaces), mathematical models motivated by optimal transport theory, as well as nonlinear representations based on neural networks as well as kernel methods. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.