DDRIG: The Algorithmic Translation of Expertise: Credible Knowledge and Machine Learning in Medicine

Cornell University, Ithaca NY

Investigators

Abstract

The use of artificial intelligence (AI) and machine learning (ML) to assist experts in making sophisticated professional decisions is well under way in such areas as medical diagnosis, drug development, and public health. This project focuses on an especially promising application of ML systems: their use to support medical diagnoses by analyzing images, such as CT scans and digitized pathology slides. This research will study the development of ML-based medical image analysis systems, tracing their production, application, and regulation. It will pay special attention to how medical experts and policymakers assess the credibility of the diagnostic suggestions that ML systems make. The research aims to contribute to the use of ML tools to improve the quality and accessibility of healthcare and to inform policymaking about the introduction of these technologies. Developing an ML system involves translating human expertise into a new algorithmic form. This study will investigate novel questions raised by this process about the credibility of diagnosis. How can medical experts evaluate the credibility of ML systems, given that the internal workings of these systems are complex and, to some extent, inscrutable? How might the rise of ML systems affect the credibility of human experts? How will understanding of expertise change when well-trained experts, historically the most credible judges of complex professional questions, find their judgments implicitly challenged by AI systems? To explore these questions, the investigators will conduct ethnography at two AI startups, conduct semi-structured interviews with engineers and clinicians, and analyze written materials. By analyzing negotiations over credible knowledge in this context, the project will provide insights about how the credibility of the human and the machine are assessed. Beyond its immediate implications for understanding the credibility of ML systems, the study aims to enrich scholarship in the sociology of expertise, medical sociology, data studies, and the governance of emerging technologies. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.