Collaborative Research: NSF-BSF: SaTC: CORE: Small: Detecting malware with machine learning models efficiently and reliably

Duke University, Durham NC

Investigators

Abstract

Detecting malicious software (a.k.a. malware) is a common security challenge that machine learning (ML) is increasingly used to perform. While many types of malware detectors exist, this project focuses on improving those that identify malware without actually running it. In these types of detectors, an ML model is trained on many examples of malware and of benign software to learn how to classify new examples based on patterns in the computer instructions they contain. A specific threat to the use of such ML models is that attackers might use their understanding of a model's workings to develop malware variants that are particularly hard for the model to detect. This project seeks to develop ML models for malware detection that are reliable despite attackers' attempts to fool them, as well developing models that are more time- and space-efficient. The technical innovations of this project lie in basic research on techniques to quantifiably improve the resistance of malware detectors to being fooled, with greater efficiency than has been possible to date. These techniques include new approaches to combining multiple classifiers into ensembles; novel model architectures; leveraging methods that attribute classification outcomes to specific properties of the input; and improvements to ML training methods for the malware domain. This project will develop principles underlying the application of these techniques to improve ML models and empirically demonstrate the improvements they yield, using datasets of malware and benign executables. Where doing so is responsible (i.e., does not risk facilitating new attacks on operational systems), the advances of this project will be open-sourced for others to leverage in follow-on research; in other cases, the implementations will be distributed only by request to parties who are verifiable researchers or developers of anti-malware tools. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.