CRII: CSR: Efficient and Available Replication in Large-scale Datacenters

Ohio State University, The, Columbus OH

Investigators

Abstract

Replication is widely used in the IT industry to protect users' data against various kinds of errors, such as power loss, disk corruption, and network partition. It is generally the case that we need to pay a higher cost for stronger replication protocols that can tolerate more kinds of errors. This presents a challenging trade-off to developers who will sometimes choose to use weaker forms of replication, willing to take the risk of occasional data loss in exchange for low cost. This project presents how to reduce the cost of Paxos, a popular strong replication protocol in today's data centers, while preserving its other properties. While this topic has already drawn a continuous effort in academia, existing approaches usually give up certain useful properties of Paxos, such as availability, presenting a different but equally hard question to the developers. This work presents ThriftyPaxos, a replication protocol that can achieve the same properties as Paxos with lower cost. To reduce cost, ThriftyPaxos incorporates the idea of on-demand instantiation, which activates the minimal number of replicas when there are no failures, and activates backup replicas when the active ones fail. To solve the key limitation of on-demand instantiation, that the system is unavailable when the backup is rebuilding its state, ThriftyPaxos incorporates the idea of delayed recovery, which allows the system to proceed while recovering a backup replica in the background. Such design is motivated by the observation that when acceptors and learners, the two key components of Paxos, are decoupled, it is possible to design separate mechanisms to delay their recovery without blocking the system. A cheaper Paxos protocol will be appealing to both existing Paxos users and to those who are still using weaker protocols, because they will not have the same type of difficult choice between data consistency and cost. To realize such impact, the PI will apply ThriftyPaxos to popular open-source software, as well as publishing the source code of ThriftyPaxos. This project has impact to education as well. The code will be converted for use in a course project so that graduate and undergraduate students can build a similar system with reasonable effort in a course project.