CAREER: Improving Real-world Performance of AI Biosignal Algorithms

Duke University, Durham NC

Investigators

Abstract

Artificial intelligence (AI)-based algorithms for processing individual biological data streams (biosignals) are the enabling technology underlying wearables like smartwatches and medical monitors that are pivotal for health monitoring in everyday life. Current algorithms, despite their utility in wearables for health monitoring, are hindered by performance discrepancies. This project addresses critical challenges surrounding data and constantly changing technologies, or drift, leading to reduced accuracy. The research focuses on evaluating how well algorithms perform across different people, various types of measurements, over time, and with technology updates. Outputs of this project promise to enhance the appropriateness and reliability of AI technologies for monitoring biosignals, offering breakthrough methods for reliable health monitoring outside of the clinic. The research plan unfolds in two primary thrusts. The first thrust develops robust techniques for assessing and reporting algorithm performance across different populations, with a particular focus on regression tasks involving continuous variables. This includes characterizing existing biosignal training dataset demographics, designing reporting standards, and implementing a theory-based method for quantitative evaluation of algorithmic appropriatness. An empirical analysis will assess appropriateness of key biosignal algorithms and datasets. The second thrust focuses on detecting and monitoring concept drift over time in biosignal data and algorithms, accounting for demographic shifts. This involves developing methods and metrics for concept drift monitoring, providing a nuanced understanding of how changes in training data composition impact the performance of AI-based biosignal algorithms. This work will result in gaining fundamental knowledge about errors and drift, advancing techniques for their detection and monitoring, and, ultimately, enhancing the reliable application of AI in biosignal algorithms for improved health outcomes. The project's scope also extends to an outreach and education plan, promoting increased access to biosignal monitoring devices and fostering education in STEM fields. This multifaceted approach ensures that the impact of the research transcends theoretical advancements, directly benefiting society through the improvement of wearables for health monitoring as well as the development of methods to enable more general AI oversight. Reflecting NSF’s statutory mission, this project will provide societal benefits through development of and education on trustworthy AI technologies and their applications to biosignal algorithms to improve health and wellness broadly. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.