IMR: MT: Tools for Programming Distributed Data-plane Measurements

Princeton University, Princeton NJ

Investigators

Abstract

Understanding the flow of traffic across key networks---what it is composed of and how it changes---is critical for improving modern information services. Traditionally, however, it has been difficult for researchers to develop new tools for dissecting this traffic and analyzing its characteristics, while taking care to maintain user privacy. Recently, though, the development of relatively cheap programmable switches has made it possible to develop diagnostic tools and place them directly inside the network, on the path through which traffic flows. In such a position, new tools have the potential to see all the internet traffic as it flows by, from a university campus to the broader internet, for instance, or along a corporate wide-area network or data center. Unfortunately, while it is possible to develop such tools, doing so is currently an incredibly difficult and error-prone process. To ameliorate this situation, the research team will develop Lucid, a new programming language and system that will facilitate the process of developing, debugging, and deploying network measurement tools in live programmable networks. The research team will deliver a compiler that translates high-level Lucid programs into lower-level code that execute in multiple places---directly on programmable switches, or in support, on servers connected to the network in question. In addition, the team will deliver a collection of reusable components that network measurement researchers can plug together to get started on a new idea quickly. To help teach researchers how to use the new language, the team is developing tutorials for major conferences in networking. To summarize, this project will impact the performance, reliability, and security of critical networks by facilitating the development of new measurement tools that can discover network optimization opportunities, detect failures, and rapidly recognize attacks that disrupt online services. Traditional measurement tools and datasets, while incredibly useful, have significant limitations in scale and coverage. Measurement researchers should capitalize on the exciting advances in programmable data planes to analyze Internet traffic and performance as packets traverse the network. Analyzing traffic directly in the data plane (e.g., network switches, routers) enables sophisticated analysis without sacrificing efficiency or divulging sensitive user information, and enterprise networks, such as university campuses, provide an excellent opportunity to use these programmable data planes in practice. However, programming the data plane is not easy. Existing languages, such as P4, are very low-level, have an extremely steep learning curve, and are notoriously difficult to work with (with seemingly legitimate programs often failing to compile). This project addresses these pain points by delivering new programming support in the form of Lucid, a high-level language designed to support cooperative measurement across multiple locations and device types. More specifically, the research team is developing compilers that will target both Intel Tofino programmable switches (via P4) and software servers (via eBPF). Using both kinds of devices, researchers will be able to develop and deploy a range of different kinds of distributed measurement tools. The research team will also develop an interpreter for the language so that interesting new research ideas may be developed and debugged prior to deployment. The infrastructure developed by the research team will also include a suite of libraries that implement key data structures and utilities useful in network measurement and in support of data privacy. To teach the community how to use our language, libraries, tools, and infrastructure, the team will develop documentation and tutorials. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.