SaTC: CORE: Medium: Guarding Noisy Neighborhoods with Weak Detectors

University Of Texas At Austin, Austin TX

Investigators

Abstract

Malicious programs ("malware") are expensive and can put people's lives at risk. Unfortunately, automatic malware detection is difficult and many automated detection systems produce a large number of false alarms. In large enterprises, detectors may create millions of security log entries per day, deluging the human analysts with false alarms. This project is developing algorithmic and statistical techniques to automatically analyze these security logs and reduce the number that human analysts must review from millions to only tens or hundreds per day. The researcher's key insight is that attack patterns induce transient correlations across time and nodes (users or devices). The project explores the hypothesis that encoding and algorithmically exploiting such transient correlations can lead to tremendous dimensionality reductions of the multi-scale alert data, and allow statistically significant insights into malware activity. The team is exploring this hypothesis using two ideas. First, they have defined the idea of a "neighborhood" that allows filtering of alerts. A neighborhood is a set of nodes that shares an action attribute such as having visited a common website or received emails from the same source within a specific time window. Thus, neighborhoods are dynamic collections of nodes that are likely to be exposed to a similar attack vector, e.g., a compromised web-server or a malicious phishing email. The second idea is a statistical approach for composing local detectors that uses the feature vectors (FVs) leading to local detector alerts ("alert-FVs") instead of operating only on the original alert flags from nodes. This may allow a global detector to better separate true positive neighborhoods from false positive neighborhoods by comparing the distributional shape of alert-FVs from each neighborhood. The research team will perform experiments to evaluate their techniques using production-scale, real-time alert logs from a large commercial enterprise and the University of Texas' Information Security Office.