Collaborative Research: SaTC: CORE: Large: Privacy-Preserving Abuse Prevention for Encrypted Communications Platforms

University Of Washington, Seattle WA

Investigators

Abstract

This project will address the immense challenge of mitigating abuse in communication services where interactions between the parties are private and fully encrypted. Such services have become popular for individual and group communications, and their security features help protect individual privacy and rights. But these platforms are also used for harmful and illegal purposes such as organizing violent activities or sharing child sexual abuse materials. In addition, users on these platforms are subject to abuse such as hate and harassment. The encryption used by such services makes detecting and blocking harmful content extremely difficult. This work will develop new trust and safety approaches to enable secure and trustworthy communications that preserve privacy while mitigating abuses. The project will aim to provide (1) technical advances in developing novel cryptographic tools and techniques to support mitigation of abuse; (2) human-centered advances in understanding perceptions and expectations of privacy and abuse mitigation, as well as creating novel designs for individual and community interactions; and (3) legal, policy, and regulatory advances to support and enable these abuse-mitigating features. The research effort is organized around two overlapping thrusts: algorithmic-driven approaches and community-driven approaches. The algorithmic approaches will focus on developing better cryptographic tools for privacy-aware abuse detection in encrypted settings, such as detection of viral, fast-spreading content. These designs will be informed by a human-centered approach to understanding people's privacy expectations, and supported by legal analyses that ensure tools are consistent with applicable privacy and content-moderation laws. In the second thrust, the community approaches will focus on providing communities with the tools they need to address abuse challenges in encrypted settings. Given the challenges and pitfalls of centralized approaches for abuse mitigation, the project will explore building distributed capabilities to support communities and groups on these platforms. An importantly ingredient is working with communities and community moderators to understand their needs, as well as guide design of legal and policy frameworks to support new approaches. Taken together, this project will address the challenge of abuse mitigation on encrypted platforms, while preserving privacy protections for individuals and communities. It will especially consider the perspectives of individuals and communities most in need of privacy and abuse protection. The work, if successful, should fuel new innovations in the design of encrypted messaging platforms, and in basic research in cryptography, human-centered design, and Internet law. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.