AitF: The Fuzzy Log: A Unifying Abstraction for the Theory and Practice of Distributed Systems

Yale University, New Haven CT

Investigators

Abstract

The Fuzzy Log project seeks to democratize the design and development of complex distributed systems, accelerating innovation by allowing developers to focus on high-level application functionality instead of low-level protocol details. Examples of such complex systems include Software Defined Network controllers for the network, filesystem namespaces for storage, schedulers and allocators for big data run-times, and general-purpose coordination services. These distributed systems require large numbers of highly trained engineers and scientists to construct and operate them. Simplifying the design, development, deployment and debugging of such systems can drastically reduce the cost to create and operate massively scalable cloud services that are reliable and responsive. More broadly, the Fuzzy Log project will also act as an educational gestalt that combines distributed systems and theory to improve the state of the art in cloud computing. A Fuzzy Log is a partially ordered shared log that multiple clients can append to and read from concurrently. As in other shared log designs, applications can extract properties such as consistency, durability, and concurrency control from the Fuzzy Log. However, unlike a conventional shared log, a Fuzzy Log does not impose a total order over all entries. When clients append to the log, they specify dependencies to define a partial order; when they read from the log, the system returns entries in some sequence satisfying the partial order. Fuzzy Log applications are simple to design, implement, and debug, with full-fledged distributed systems realized in hundreds of lines of code. Fuzzy Log applications are also fast and scalable, extracting parallelism from workloads while imposing order only when strictly necessary.