II-EN: High-Performance Network Monitoring Infrastructure For Research in a Large-Scale Operational Environment

International Computer Science Institute, Berkeley CA

Investigators

Abstract

This award is funded under the American Recovery and Reinvestment Act of 2009 (Public Law 111-5). Developing security monitoring to robustly protect large sites against Internet attacks presents exceptionally difficult research challenges. There is a world of difference between detecting attackers in a small-scale environment such as a departmental LAN (as is often used for evaluation of academic studies) and doing so at the scale of a large site. Algorithms that work fine in the presence of a modicum of background traffic can be rendered completely useless when faced with two orders of magnitude more background traffic, for reasons of both performance and false positives induced by the much greater diversity. The overall objective of this proposal is to greatly enhance our UC Berkeley infrastructure that monitors the campus border traffic in order to facilitate the continuation of security research tied to the operational requirements of one of the largest academic network environments in the country. The proposed new cluster will serve as a powerful research platform for many future studies, providing unprecedented capabilities for analyzing a large-scale operational network in depth. Furthermore, on a technical level it will allow the principal investigators (PIs) to systematically assess the scalability of the our clusterized approach to larger network loads and determine what is required to provide in-depth monitoring capabilities for other environments. Intellectual merit: Intellectual Merit: The instrumentation infrastructure supported by this proposal will serve as the key enabler for a range of research otherwise not possible to undertake at an equivalent scale. These span: (1) detection algorithms that operate robustly in the presence of highly diverse background traffic, (2) indepth semantic analysis of the very broad range of modern network applications, (3) efficient recording of high-volume traffic streams for forensic analysis, (4) scalability assessment of clustering and multicore techniques for achieving high performance monitoring, and (5) ties with the UC Berkeley cybersecurity staff leading to investigation of new research problems that arise when deploying network defenses operationally. Broader impact: The ability to richly monitor large traffic streams in real-time has major implications for Internet security, as it is a key component for securing large Internet sites. This effort will enable a range of research directly grounded in the operations of a high-performance, high-volume site, a type of environment only very lightly addressed in the field due to its significant logistical and technical difficulties. The monitoring system will realize an order of magnitude more power for such analysis than to our knowledge any existing deployment provides. As such, it will serve not only as a platform for network security research at scales previously unattainable, but also as an exemplar for how others can construct and operate such systems. Thus, this effort has the potential both to enable new discoveries regarding protecting high-volume network environments, and to facilitate the broader use of such technology for better securing and operating high-speed networks. Finally, the infrastructure from this grant will provide doctoral students with an unparalleled opportunity for undertaking research in an environment unmatched by any other in the field.