SaTC: CORE: Small: Enhancing Security and Mitigating Harm in AI-Generated Vision Language Models

University Of South Carolina At Columbia, Columbia SC

Investigators

Abstract

The extraordinary benefits of large generative AI models also come with a substantial risk of misuse and potential for harm. Given that roughly 3.2 billion images are uploaded daily on social networks and a rapidly growing percentage of these are AI generated, the need for robust multimodal harm prevention is more pressing now than ever. This project seeks to prevent harms associated with AI-generated vision language model content. Project techniques will be valuable in many domains and can help stakeholders in government, regulatory bodies, and policy making. This project will engage journalism and other students in the project. Project activities include undergraduate research internships and an annual AI summer camp for high school students. This project pursues three technical objectives. The first is a prompting framework for harmful content provenance in AI-generated vision language models with the use of a novel prompting method utilizing multimodal knowledge graphs, organized by who, what, when, where, and why semantic schema, and stored and optimized utilizing techniques such as joint embedding, contrastive learning, and negative sampling methods. A second objective is machine unlearning as a proactive measure to mitigate harms associated with AI-generated vision language models. The third objective is to blur segments of harmful images. The evaluation framework for the project includes automated metrics and human evaluations. The project will share open-source web codebase, datasets and demos that can be tested live. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.