CSR: Small: Co-Designing Distributed Coordination Systems and the Datacenter Network

University Of Washington, Seattle WA

Investigators

Abstract

Datacenter applications must remain continuously available and keep data consistent despite the fact that individual servers may fail. This is one of the fundamental challenges of distributed systems design: coordinating accesses to distributed services. Today's systems solve these problems with sophisticated distributed algorithms like Paxos and two-phase commit, yet these are frequently considered too expensive for practical use. This is because these algorithms are designed independently from -- and must make worst-case assumptions about -- the underlying network. This project will explore a new approach to designing distributed systems for the datacenter: co-designing distributed protocols with the network layer. This includes designing new network-level primitives to enable datacenter networks to provide stronger semantics, e.g., an approximately synchronous mode of execution. Then these primitives will be used to design high-performance distributed systems, including state machine replication protocols and transactional storage systems. This work will improve the ability of distributed systems to provide fault tolerance with high performance, increasing the robustness of critical datacenter infrastructure. It will also reduce the cost of strongly-consistent transactional storage, making it easier for application developers to reason about concurrent accesses to shared data, and thereby encourage innovation.