CNS Core: Small: Optimizing Distributed Transactions on Emerging Hardware

University Of Washington, Seattle WA

Investigators

Abstract

Replicated transactional storage simplifies the programming and reasoning about distributed systems by providing a simple and powerful abstraction: atomic and durable execution of transactions that can survive node failures. As a consequence, many datacenter systems using transactional storage as a critical building block. This project revisits this research topic to address the opportunities and challenges posed by multiple hardware trends, e.g., large core counts on processors, programmable network interfaces, and low-latency storage. The project seeks to build distributed transactional storage systems using principles that reduce coordination and maximize network and storage performance. In particular, the project seeks to optimize distributed transactional storage by taking advantage of modern hardware features ranging from core-heavy servers to programmable network devices and tiered and highly-parallel durable storage. A key goal is to design distributed protocols that minimize cross-core coordination traffic to provide multi-core scalable solutions. Further, the project will distill remote access communication primitives, enabled by a programmable network interface, in order to optimize transaction processing. The project will also examine how to support durable data structures on heterogeneous storage that includes low-latency non-volatile memory. Datacenter applications that require support for transactional storage are used by literally billions of people around the globe daily. By improving the efficiency of distributed transactions and reducing latency/overheads, the project seeks to dramatically reduce the cost of existing datacenter-based services as well as make it much cheaper for new public services to be developed. Collaborators at hardware vendors are equal partners in this effort, providing access to new hardware technologies and assisting in project execution and technology transfer to the industry. The developed software will also enrich the distributed systems curriculum at all levels. For the broader community of users and the society at large, the project will periodically release the developed software to enable a rich set of distributed systems and high-performance datacenter applications. A public repository will be maintained until at least December 2025 at https://github.com/arvindkrish/dist-transactions/tree/master. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.