PDaSP Track 2: Explainable Auditing of ML Models for Privacy Violations

Washington University, Saint Louis MO

Investigators

Abstract

The growing use of artificial intelligence in healthcare and medical research has created a difficult challenge: researchers need to share their computer models to advance scientific discovery, but these models can reveal private information about the patients whose data was used to create them. Organizations are often reluctant to share their computer models because of privacy risks, even though withholding these models prevents broader societal benefits from medical research. This creates a barrier to scientific collaboration that could otherwise lead to better treatments, improved public health outcomes, and medical breakthroughs. This project addresses this challenge by developing methods that allow organizations to safely share models trained on sensitive patient data without compromising individual privacy. Proposed research will result in new techniques for auditing models, certifying their privacy guarantees, and providing actionable tools to fix any identified issues. This work serves the national interest by advancing medical research and scientific discovery, enhancing national health and prosperity through improved healthcare technologies, supporting American competitiveness in artificial intelligence innovation, and enabling secure collaboration while protecting personal privacy rights. This project develops an end-to-end framework for privacy-preserving sharing of machine learning models trained on sensitive data. Despite growing interest in sharing models rather than raw data, machine learning models remain vulnerable to privacy attacks, such as membership inference attacks, which can reveal whether an individual's data was used during training. The research activities include four technical components. First, the project will evaluate the privacy properties of shared models by subjecting them to existing and newly tailored privacy attacks, establishing a foundational understanding of their vulnerabilities. Second, the team will develop formal privacy guarantees using methods like differential privacy and establish privacy-utility tradeoffs, creating privacy certificates for machine learning models that may include legal and usage constraints. Third, the project will design explainable auditing tools and privacy patching mechanisms such as machine unlearning to help developers mitigate risks without compromising model utility. Fourth, the research will build user-friendly tools to deploy these methods, focusing on real-world applicability in healthcare and biomedical research. The project will introduce a novel privacy-risk scoring system, enabling developers and regulators to assess the privacy risks associated with a given model. Unlike existing point solutions, this comprehensive framework integrates auditing, certification, and remediation into a unified system. Results will be disseminated through tools, publications, and educational modules to support broad adoption and training. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.