EAGER: DCL: SaTC: Enabling Interdisciplinary Collaboration: Evaluating Bias In The Creation and Perception of GAN-Generated Faces

Haverford College, Haverford PA

Investigators

Abstract

Bad actors often use bots and fake profiles to attack individuals or groups and to undermine social harmony and collective movements. These fake profiles may use face images to signal human authenticity. Until recently it was possible to identify bad-faith actors via reverse image searches because many fake profiles used stock photos. Recent advances in machine learning-enabled general adversarial networks (GANs) have made it possible to create hyper-realistic faces of people who do not exist and cannot be identified. These faces can be animated and used to cause harm. To help develop more secure and trustworthy cyberspaces, it is critical to understand whether and how human perceivers (alone or with computational aids) can detect real vs. artificial faces, and how their detection strategies and outcomes differ across groups. This project investigates whether the GANs that generate faces are racially biased and whether this bias is manifested in differential detectability of ingroup vs. outgroup faces. The project tests the hypothesis that GANs are racially biased because the training dataset is itself biased, with White faces (especially White female faces) overrepresented. Furthermore, when tools are created to control what kind of face is generated, these tools may be racially biased as well because they are extracting biased parameters. These biased processes may result in GAN-generated faces that are more detectable to racial minority individuals vs. racial majority individuals. To test these hypotheses, the project is developing a training dataset of diverse faces, with annotations for dimensions of interest such as skin tone and gender. These annotations can be used to train a GAN with any number of checkpoints to examine how GAN-generated faces appear at different stages of creation. The project is examining how people perceive the generated faces at each stage of the GAN. This project is helping spur theoretical insights into how machine-learning works, and provides training in computer science and social psychology for a diverse group of undergraduate researchers. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.