SaTC: CORE: Small: Securing Embeddings

Cornell University, Ithaca NY

Investigators

Abstract

This project investigates security and privacy of embeddings, a fundamental building block of artificial intelligence (AI) systems. Embeddings convert text, images, audio, and video into mathematical vector representations whose geometric relationships reflect the meaning of the inputs. Embeddings are a key component in modern search and information retrieval systems, large language models (LLMs), and other generative AI systems. An entire new industry of vector databases has emerged to provide solutions for large-scale storage and management of embeddings. The project's novelties are to tackle, for the first time, security and privacy vulnerabilities that are unique to embedding-based systems and to develop robust mitigations. This includes defending against attacks that exploit embeddings to manipulate LLMs, as well as attacks that extract confidential and private information from vector databases. The project's broader significance and importance include protecting emerging AI systems from adversaries, mentoring students at an early stage of their career, and technology transfer from academia to industry via open-source code releases. The technical approach of the project focuses on three key security and privacy problems related to embeddings. First, it studies inference-time attacks and defenses. This includes robustness to adversarial inputs, especially in real-world systems that operate on multi-modal inputs (e.g., text and images). Second, it investigates training-time attacks that aim to change semantic relationships encoded in the embeddings. Third, it develops new methods for measuring information leakage from embeddings in a variety of realistic scenarios. In summary, this project addresses a new research area of trustworthy AI and helps ensure that AI-based systems can be safely deployed in our digital society. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.