CAREER: Diagnosing Distributed Systems with Provenance

Georgetown University, Washington DC

Investigators

Abstract

Despite the ubiquitous adoption of distributed systems, designing and deploying distributed systems remain challenging because of the ever-increasing scale, complexity, and unpredictability of system executions. In occurrences of unanticipated faults, system operators often find themselves needing to diagnose the systems to discover software bugs, misconfiguration or malicious intrusions. Emerging provenance techniques can construct an explanation of an observed system state, in the form of a directed dependency graph. However, the use of provenance is limited; they lack query interfaces that allow users to easily extract information from provenance data, as well as, techniques to fully leverage provenance data. System operators have to further analyze provenance data, and manually perform management actions such as system recovery and reconfiguration. This project introduces a framework that efficiently maintains and queries provenance data, and leverages the exposed dependency information for system diagnosis. The research will i) develop layered privacy-aware provenance maintenance that provides availability and security guarantees in untrusted environments, ii) introduce a SQL-like declarative query language for users to annotate and abridge provenance data, and iii) investigate various methods of leveraging provenance data in order to identify and control suspicious misbehavior. The work provides a holistic view of provenance support for practical applications that impact performance, privacy, availability and security. More broadly, this project will provide a novel approach towards enhancing the reliability and security of distributed systems in an automated manner. Tools developed in the project will be freely distributed, and research findings will be incorporated into networking and databases courses.