CAREER: Towards Human-Algorithm Collaborative Intelligence for Risk Assessments

University Of Florida, Gainesville FL

Investigators

Abstract

Human experts performing financial fraud risk assessments are more willing to delegate to Artificial Intelligence (AI) tasks that require mechanical and analytical intelligence. Conversely, they are less likely to entrust an AI with tasks that require intuitive and emotional intelligence, perceiving the technology to be very limited in those aspects. While machines have their shortcomings, humans do as well. Studies reveal that human novices and experts are both susceptible to judgment weaknesses, such as attentional and confirmation biases, that result in inconsistent decisions. Consequently there is a move towards building human-AI collaborations that can improve decision-making. Achieving higher utility with human-AI collaborations than with AI or humans alone is challenging. This project addresses several of the challenges by aligning the goals and activities of the human and AI participants through a shared mental model of the task. This will foster more accurately calibrated trust, and thus enable sound augmenting of human judgment in risky settings. The principals and methods developed will be applied to financial fraud risk assessment. The project aligns with the goal of enabling human-technology partnerships that augment performance. It includes outreach activities to foster an interest in science and technology research among undergraduate students. It also involves auditors and forensic experts to cultivate lifelong learning on AI-human partnerships. This project draws from team decision-making theory, which suggests that effective team performance requires members to hold a common or overlapping representation of the task requirements and procedures, referred to as a shared mental model. The shared mental model includes the knowledge or belief about causal relationships. The mental model will capture the human decision-makers' and AIs' perceived relationship between the cues in a company's data and fraud risk. The model will be informed by the human decision-makers' perception of risk and their process in ascertaining risk arrived at by analyzing behavior of the human expert interacting with visual data displays during risk assessment. It will incorporate AI feedback to the human decision-maker during risk assessments to enhance the team's performance. The modeling approach and resulting team process will result in AI systems that can learn from, and respond to decision-makers during risk assessments, and select the appropriate interventions to strengthen collaboration and augment decision-making. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.