SaTC: CORE: Small: U.S.-Ireland R&D Partnership: Securing Machine Learning from Threats Exploiting Unused Model Parameters

University Of California-Riverside, Riverside CA

Investigators

Abstract

Machine learning (ML) models are driving an AI revolution that is transforming all areas of human life, with applications including healthcare, self-driving cars, and robotics. Anticipating the security vulnerabilities of ML is essential to improve the safety and trustworthiness of systems that depend on them. The project studies a new threat to ML models that exploits how these models are built and trained. Specifically, when ML models are trained, they configure a number of parameters as they learn patterns in data sets; large models can contain billions of parameters that help them as they learn complex tasks efficiently. However, once the model is trained, it is well known that only a subset of these parameters contribute to the model’s functionality. The remaining, unused, parameters have little effect on the performance of the model, and there are reasons to believe that malicious actors might be able to exploit unused parameters to harm the security and privacy of models. The goal of this research is to understand the security implications of the unused parameters of machine learning models. Since unused parameters do not affect the baseline model, their state can be manipulated by the attacker during training to install potentially malicious additional functionality, without being detected when the model is tested. The project characterizes this threat both experimentally and theoretically for different model types, and develops mitigation approaches against it, helping to secure ML models. The project will explore a number of coordinated research directions around exploiting unused parameters. First, the research empirically studies the capacity of ML models to store data covertly within the unused parameters without affecting the baseline model accuracy. Viewing the problem from an information theoretic perspective allows the project team to use tools from communication to reason about the capacity of the unused parameters to hold unintended functionality (in this case, covertly stored data). This in turn allows the use of ideas from game theory to analyze tradeoffs between optimizing malicious goals and preserving baseline functionality. The project also studies generalizations to emerging ML models including Large Language Models, and to additional attacks including a novel model hijacking attack. Having established the threats and characterized their properties, the project team will explore defenses and mitigations to improve the robustness of machine learning models against this new attack vector. These new attacks, optimizations, theoretical models, and mitigations represent the intellectual merits of the project. The broader impacts include improving the safety and trustworthiness of ML models, training graduate and undergraduate students, and developing new pedagogical material on ML safety. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.