FAI: Auditing and Ensuring Fairness in Hard-to-Identify Settings

Cornell University, Ithaca NY

Investigators

Abstract

The spread of artificial intelligence (AI) systems for high-stakes decision making gives rise to an urgent ethical and legal imperative to avoid discrimination and guarantee fairness with respect to protected classes. Example application domains include credit decisioning, personalized medicine, targeted policymaking, and sentence and bail setting. In these settings, fairness is often quantified by the decisions' disparate impacts on different groups. However, fundamental limits in the data available make both auditing disparities and eliminating them difficult or impossible. For instance, both in credit and insurance claims data, sensitive labels such as race are missing, and one never knows if a defendant detained pretrial would have fled. In both cases this missingness renders disparities unidentifiable. These identification issues not only arise in nearly every application where fairness is a concern, they also break most existing methods for fair AI and create an urgent gap between the theory and practice of fair AI. Addressing this gap, this project develops a robust theory and methodology for assessing and ensuring fairness in settings where fairness metrics are hard or impossible to pin down. Specifically, the project will develop: (a) fairness assessment methods that can reliably support credible conclusions in the face of fundamental identification limitations; (b) learning algorithms that robustly enforce fairness at the design stage even if fairness is unmeasurable. A key approach is recognizing the limits of identification and addressing it by considering the possible ranges of disparities that an algorithm may and may not induce in practice. The project identifies several discrete settings where fairness is hard or impossible to measure and that require separate treatment: unobserved protected class membership, personalized interventions, non-binary algorithmic outputs, decision-support algorithms, and non-ignorable selective labeling. By tackling these, the project stands to transform the assessment of disparity in real practical settings, where fairness is actually statistically difficult to pin down, and correspondingly impact how we ensure AI systems are fair and benefit everyone equally. This impact will also be directly achieved through direct collaborations with practitioners. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.