CAREER: Resisting Automated Algorithmic Surveillance with Human-centered Adversarial Machine Learning

Carnegie Mellon University, Pittsburgh PA

Investigators

Abstract

This project studies adversarial machine learning technologies that will resist facial recognition and other forms of automated inferencing of personally-identifiable information in images that people share online. This approach includes methods to modify images in a manner that is difficult or impossible to detect with the human eye but that reliably reduce the accuracy of machine learning classifiers. Current adversarial machine learning techniques do allow end-users to perturb images that they intend to share online but these technologies have, to date, been developed and evaluated from a technology-centered perspective, rather than a human-centered perspective. It remains unclear whether consumers find the developed technologies useful and practical. This project re-designs these technologies to improve consumer acceptance. The project introduces a human-centered approach to designing and evaluating adversarial machine learning anti-surveillance technologies. Collaboration with advocacy organizations for populations that bear especially high risks from automated surveillance of images shared online enhance the educational and research impacts of the proposed work. The specific activities proposed involves the creation of a standardized set of measures through which developers of human-centered adversarial machine learning ask human evaluators to assess the quality and acceptability of perturbed images. When a standardized measure is created, a public repository of human quality assessments of adversarial machine learning anti-surveillance technologies will be created and seeded with evaluations of extant systems. The project also involves the creation of a differentiable-loss term that captures the aesthetic quality of adversarially perturbed images. The differentiable-loss term can be used to help generate adversarial examples that human users perceive to be high quality. An open-source implementation of this novel loss term for at least one popular machine learning library will make it easier for other researchers to use and build on this work in the creation of novel human-centered adversarial attacks against automated surveillance. The project also involves the use of co-design processes to develop a new and novel human-centered adversarial machine learning application that will allow end-users to touch-up images they choose to share online in a manner that is both aesthetically pleasing and that helps evade facial recognition and other forms of automated inferencing of personally-identifiable information. The proposed work also entails educational activities such as webinars and video lectures open to the public and organized in concert with advocacy organizations for populations at higher risk of automated surveillance, to improve public literacy and knowledge of how to protect images shared online from automated surveillance. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.