Collaborative Research: CT-ISG: Accurate Sampling of the Internet for Effective Anomaly Detection

Georgia Tech Research Corporation, Atlanta GA

Investigators

Abstract

Title: Collaborative Research: CT-ISG: Accurate Sampling of the Internet for Effective Anomaly Detection Abstract: Sampled traffic data has been increasingly used as input for anomaly detection systems, as the high link speeds make it impossible to examine each and every packet. This raises an important question of whether sampling has a (negative) impact on the accuracy/effectiveness of anomaly detection, and if so how to mitigate this effect. Intellectual Merit: This project systematically studies the question mentioned above from the following three angles. First, we will identify traffic features that are critical for a wide range of anomaly detection schemes and quantify how much they are distorted by various sampling schemes. Second, we will design new sampling or measurement techniques that preserve enough accuracy to support effective anomaly detection, while being cost-effective and light-weight. Third, we will study how to correlate the NetFlow samples obtained at the edge routers with the information-rich data generated using existing data streaming algorithms, for much better anomaly detection than pure sampling. The new scientific knowledge learned through this research will provide us with much better technologies to monitor large high-speed networks for anomalous behaviors. Broader impact: The results will be broadly disseminated through publications, invited talks and tutorials, and open-sourcing of software developed for this project. The PIs' collaboration with tier-1 ISP's will facilitate the transfer of technology from research environment to actual managing of production networks. Research results will be incorporated into information security curriculum. Both PIs have been actively engaging under-represented groups in research and higher education and will continue and expand these efforts.