CAREER: Principled yet practical observability for a microservices-based cloud

Tufts University, Medford MA

Investigators

Abstract

Society relies on cloud-based software services built using the microservices architecture in almost every aspect of their everyday lives---e.g., to shop, watch movies, and work. Though the microservices architecture has many advantages, it has one critical drawback. Observing how user requests (e.g., to buy a book) are processed by services is extremely challenging because they involve myriad interactions among many simpler (micro)services. This lack of observability complicates important management tasks, such as problem diagnosis and resource management. Previous research has demonstrated the strong potential of distributed tracing---which captures graphs of how microservices interact to process requests---to provide microservice observability. But, results in real-world settings have been disappointing. This gap between potential and reality occurs because research efforts assume principled trace graphs that capture a variety of behaviors and have no data loss. But, in practice, services are never well-instrumented and data loss is common. The overarching goal of this proposal is to create a new tracing platform that automatically infers the data needed to make traces principled. Doing so will actualize distributed tracings' vast potential for microservice observability, improve the utility of existing tracing-based management tools, and enable transformative new tools. These outcomes will improve the resiliency and efficiency of the software services society depends on. Insights from this project will inform age-appropriate course material and projects in a college course on debugging cloud systems, a high-school research program, and a middle-school outreach program. This project proposes a novel tracing platform that automatically enriches span-based traces with two sets of primitives. 1) The happens-before concurrency/wait primitives and 2) the holes and holes covering primitives. The former allows requests' critical paths to be identified, enabling slack analyses, targeted performance debugging, and precise resource allocation decisions. The latter allows areas of data loss to be expressed in traces along with predictions of what work might execute in them. Since scheduling decisions may obfuscate causal structure, the project will investigate active probing methods to tease out concurrent and waiting relationships. To account for uncertainty, it will explore probabilistic data models to represent the primitives. The project will demonstrate the value of the primitives by modifying an existing auto-scaling solution and performance-debugging tool to use them. It will also demonstrate a new trace subgraph sampling approach made possible by the holes and holes covering primitive. The proposed platform, improved management tools, and inference methods will be publicly available to benefit the computer science and microservice observability communities. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.