CRII: SHF: Testing Fairness in Human Decisions with Algorithmic Bias

Rochester Institute Of Tech, Rochester NY

Investigators

Abstract

Unfairness in human decisions has been a long-standing issue in our society that constantly threatens the equality rights of historically underrepresented groups. This issue has been exacerbated in decision making processes where a machine learning software learns from potentially unfair human decisions and makes potentially unfair predictions to guide future human decisions. One such example is the talent hiring process where a human resource person screens a list of candidate resumes ranked by a machine learning software to decide who deserves an interview. Human decision fairness is especially important in this scenario since (1) the machine learning software will inherit the bias when trained on unfair human decisions; (2) even with unbiased machine learning software, the final decisions can still be biased as long as the human resource person is biased. This project will approach this historically challenging issue from a different direction. It learns to model the human decisions with a machine learning software and utilizes the algorithmic bias of that software to detect bias in the human decisions. This work could potentially advance the equity process in many decision making activities such as talent hiring, credit card approval, and school admission. In contrast to most of the existing research focusing on improving algorithmic fairness, this work aims to isolate the algorithmic bias inherited from the training data (with human decisions as dependent variables) as an indicator for human decision unfairness. To do this, the project will use a novel technique to test fairness in human decisions. Specifically, machine learning models are trained on the human decisions under test with re-balanced class distribution in each demographic group, then tests the learned model with a regression test suite of comparative judgements. If the model fails the regression test, the human decisions it was trained on will be considered as unfair. While it is difficult to directly test whether a human has bias, it is easier to test whether a machine learning model makes biased predictions using comparative judgements. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.