CAREER: Accelerating Scientific Data Collection through Human-in-the-Loop Artificial Intelligence

University Of Hawaii, Honolulu

Investigators

Abstract

Scientists in domains such as psychology, marine science, and ecology spend a large amount of time, effort, and funding designing experiments and collecting large datasets. Although recent technological breakthroughs in artificial intelligence (AI) have dramatically impacted the process of scientific data analysis, these approaches have not yet had a similar impact on scientific data collection. This project explores improving efficiency of data collection by using AI to automatically direct data gathering efforts towards the most useful regions of the space. This is a big challenge; although AI systems can easily analyze large amounts of data in real-time, they lack an inherent understanding of the scientific domain (including objectives, background knowledge, and experience). On the other hand, human scientists and data collectors understand the domain but lack the ability to quickly process vast amounts of data. This project will develop principles and systems for real-time direction of scientific data collection with humans and AI systems working together, leveraging each other's strengths. The project will further our understanding of human-AI collaboration, resulting in new data collection algorithms and interaction paradigms that will help promote the progress of many scientific domains. The project will have a major impact on research and educational activity on Hawaii Island, developing new human-in-the-loop AI techniques that address problems of great local importance and leveraging this to drive increased interest in science and technology among community members and local undergraduate students. The technical aims of the project are divided into two thrusts. In the first, the investigators seek to understand how to build AI systems for data collection that ensure that their objective functions are well-aligned with human scientists. The project will explore better aligning objective functions through: 1) requesting and incorporating richer feedback in the form of key points and suggestions, and 2) inferring objective functions based on human behavior during the annotation process itself. In the second thrust, the investigators seek to understand how to design AI systems that efficiently direct human effort in real-time to optimize data collection. The project will study how real-time interaction can be improved through: 1) ensuring that notifications generated by the AI system are timed for maximum usefulness, and 2) building AI systems which learn to be a better teammate by observing in situ human behavior. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.