CSR: Medium: Rethinking Distributed SSD Storage Systems

Princeton University, Princeton NJ

Investigators

Abstract

Modern storage technologies, such as solid-state disks (SSDs), have increased their capacity by increasing density: representing more digital bits in the same physical media. To do so, their margin of error for representing data has shrunk, and the resulting bit error rate has risen. While disks employ error correcting codes to mask such bit errors, these error rates rise as disk blocks age, by being (over)written, and eventually the disk fails. Thus, as density has risen, disks' expected lifetimes have plunged. This research rethinks how to build scalable and cost-effective storage through a cross-layer design to the storage stack. For most datacenter applications, objects are already replicated between multiple devices, and this application-layer redundancy can be leveraged for increased reliability. This research thus explores a new approach for end-to-end reliability for SSD storage systems: datacenter flash devices should be far less reliable and push error visibility and correction up to the distributed application layer. The design challenges and implications of providing such a holistic, cross-layer approach are explored through three research thrusts: at the individual disk layer, at the local file system, and at the distributed storage system layer. By extending the lifecycle of disks by several orders of magnitude, this research has the potential for the adoption of new SSD storage technologies, significant cost savings, and enabling new applications to use SSDs. Due to the plunging lifecycles of newer generations of SSD technologies, large datacenter applications have been unable to integrate these new generations in a number of scenarios (e.g., workloads with heavy write rates, including databases and caches). Given this research's potential for orders of magnitude of reliability improvements, it provides a potential path out from this reliability cliff of the hardware. Code, data, and results relating to this research will be made available via a git repository, hosted publicly at https://github.com/princeton-sns/crosslayer-storage/. Materials will be available throughout the Award term and at least five years following it. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.