Collaborative Research: SaTC: CORE: Medium: Privacy Through Design: A Design Methodology to Promote the Creation of Privacy-Conscious Consumer AI

Carnegie Mellon University, Pittsburgh PA

Investigators

Abstract

This project centers consideration of privacy in the development of artificial intelligence (AI) technologies that directly impact consumers. Using AI can help systems adapt to specific people's goals and abilities; however, AI systems typically require collecting data and making guesses about their users, both of which can be intrusive and cause harms. For instance, AI systems sometimes make wrong inferences about personal, sensitive characteristics that can cause both psychological harm and affect people's access to systems; the data collected can also be used in unwanted ways, such as large facial recognition databases assembled without people's consent. These harms often happen, even when system designers are well-intentioned, because current design practice provides little specific guidance on how to reason about possible harms. This project will tackle this problem by creating design methods and guidelines that highlight potential privacy issues and design choices that often increase these risks. Student and industry researcher involvement in the development and evaluation of the methods will give the work both direct educational impact and increase the chance that future AI-based systems will make informed choices around privacy and safety risks. The specific method proposed is called Privacy through Design (PtD), a novel research methodology to help creators of consumer-facing AI technologies: (i) model how acute, use-case specific privacy concerns among end-users among stakeholders trade off against the envisioned utility or value of proposed AI concepts; and, (ii) understand how to (re-)design those concepts in a manner that respects stakeholders' privacy concerns of while retaining the envisioned utility of the design. Doing this work makes three main scientific contributions. The first is to develop a taxonomy of algorithmic privacy intrusions to operationalize the unique privacy harms entailed by consumer AI and map those harms onto the unique capabilities and requirements of AI systems. This second is to develop PtD using an iterative methodology incorporating experts and practitioners in industry and academia. The third is to formally evaluate how products developed through PtD compare to those developed through existing industry standards for designing consumer AI technologies. Two key envisioned outputs are a repository of design cases in which privacy concerns emerge and are resolved, and a guidebook with worksheets and recommendations to help creators of consumer AI technologies center consideration of privacy in their design processes. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.