Administrative Supplement - Rapid Actionable Data for Opioid Response in Kentucky (RADOR-KY)

University Of Kentucky, Lexington KY

Investigators

Abstract

Abstract Systematic and algorithmic biases in machine learning (ML) modeling and underlying definitions for capturing opioid overdose may result in inaccuracies in burden measures for disparate groups, potentially leading to an ineffective and unequal distribution of harm reduction and prevention resources. Identifying and evaluating data and model biases and health disparities is critical to effective public health practice and research. This project is a supplement to RADOR-KY (Rapid Actionable Data for Opioid Response in Kentucky; 1-R01 DA057605-01). The RADOR-KY project will build a robust state-wide surveillance system for opioid use disorder (OUD) including opioid overdose, integrating multiple data sources to monitor and predict drug overdose mortality and morbidity. The system will be used by stakeholders to inform data-driven action, supporting the coordination and targeting of prevention and treatment efforts. As proposed in the parent grant for this supplement, the RADOR-KY system will integrate several data sources, including Emergency Medical Services (EMS) data, to develop machine learning predictive models and forecasting for opioid overdoses to inform public health and public safety agencies’ actions and planning. The proposed administrative supplement of RADR-KY will improve our understanding of the ethical aspects of these machine learning/artificial intelligence methods. EMS run data for opioid overdose surveillance is a promising new system that overcomes limitations of traditional data sources, such as prolonged delays and omission of non-clinical overdose events. While recent national standards have improved the structural components of EMS encounter data, the quality and completeness of such data still necessitate reliance on patient care narratives for case assertion. There have been a host of opioid overdose definitions proposed, typically focused on keyword matches or other rule-based criteria, with little emphasis on definition validation, comparative evaluations, or demographic parity. Critically, no previous models, whether machine learning or rule-based, have considered demographic fairness in their approaches. Leveraging our access to over 3.5 million EMS detailed encounter records access under RADOR-KY’s data use agreement, along with expert-labeled and extracted data, we aim to assess these proposed models against our own machine learning natural language processing classifier, particularly considering disparate populations. The specific aims are to 1) Evaluate potential bias in the opioid overdose data and definitions and identify suitable definitions for each specific sub-population; and 2) Identify, address, and generate bias-aware ML-ready datasets.