I-Corps: Translation potential of synthetic data generation to audit face recognition systems

University Of Wisconsin-Madison, Madison WI

Investigators

Abstract

The broader impact of this I-Corps project is the development of infrastructure to promote robust, automated face recognition systems. To date, face recognition systems have broadly proliferated across various industries, including commercial and governmental domains. Automated face recognition enables many applications including identifying individuals on social media, locating missing persons, assisting law enforcement and surveillance activities, and authenticating personal identities. Unfortunately, there are still significant concerns which prevent automated face recognition by smaller organizations. This technology makes face recognition systems both auditable and finely tunable. These properties can potentially mitigate many of the concerns that have prevented widespread deployment. Consequently, face recognition deployments, if used in conjunction with this technology, will become more acceptable while increasing efficacy and improving fairness properties. This I-Corps project utilizes experiential learning coupled with a first-hand investigation of the industry ecosystem to assess the translation potential of the technology. This solution is based on the development of a system to generate synthetic face data, to be used to audit and tune face recognition systems. This technology is based on generating synthetic face data though novel systematization of text-to-image generative image architectures. Users can synthesize high-quality faces for different text-specified facial semantics. These generated faces may be subsequently used to assess face recognition model performance or to tune under-performing systems. Synthetically generated faces have a high degree of utility when natural face images are too expensive or are otherwise impossible to collect. The recent literature shows that face recognition systems, in practice, exhibit hard-to-detect conditional failure modes. These failure modes imply that face recognition systems are not robust to changes in inputs and have demographic disparities. This solution debugs failures in current face recognition systems through well-curated synthetic data. The approach to face recognition validation and tuning was preliminary verified by a human study. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.