CSR-SMA: Language Support for Data-Centric Systems Monitoring

Princeton University, Princeton NJ

Investigators

Abstract

Complex distributed systems must be monitored to find problems, record/archive system health, oversee system operation, detect malicious activity, and perform a myriad of other tasks. To improve the reliability, security, performance, ease-of-construction and maintaince of system monitors, this research develops a high-level, domain-specific language to specify the data that monitoring systems accumulate, archive and present to users. Using high-level specifications of distributed data sources, our compiler generates reliable, secure, and high-performance monitoring tools that concurrently fetch distributed data, archive (self-describing) data for later analysis, query data to troubleshoot problems, and display statistical data summaries to monitor real-time system health. To make a broad impact, the PIs will develop tools for monitoring distributed systems, including clusters and wide-area platforms such as PlanetLab, a global network research testbed with 400-450 nodes and 200-250 network experiments running at any given time. We will make the technology open-source, enabling others to use the monitoring system for their own projects. Additionally, we will also work with industrial partners to transfer our monitoring technology to industry. Finally, the language and compiler system being built will make a broad impact outside the networking community as the kind of ad hoc data found in monitoring systems also appears across the natural and social sciences, including biology, chemistry, physics and economics. Hence it will be possible to use the specification language to describe the formats of scientific data sets and to generate querying and visualization tools that help improve the productivity of a wide range of computational scientists.