CSR: Medium: Collaborative Research: Data Center Scale Programmable Storage

University Of California-Santa Cruz, Santa Cruz CA

Investigators

Abstract

Historically, applications usually interface with persistent storage systems through protocols, abstractions and interfaces. Protocols define a series of actions that can be taken on the storage such as reading, or writing and have an implicit abstraction model such as blocks or objects. Interfaces such as general or dedicated storage networks transport protocol messages. While protocols and interfaces simplify storage system design, both impact performance by the use of abstraction models and limited operations. Emerging storage class memory has low latency and granular operations, greatly increasing the relative impact of the overhead of protocols, abstractions and interfaces. This project is re-imagining the storage interface as programmable storage, where the defined protocol involves sending encapsulated programs to the actual storage where their effect is applied. This will improve the performance of distributed systems as well as storage applications, both of which are central to Internet applications. The intellectual challenge in this project is the design of the new storage abstractions, insuring they are suitable for a variety of new storage media and that security is maintained while performance is improved. We are extending an existing storage system to include features of programmable storage while also supporting emerging consensus on low-level components used by distributed systems, such as persistent logs and transactional operations at the memory level. The project is using existing run-time code generation frameworks to insure that the programmable interface generates portable code that is also efficient. Part of the research effort is developing consensus among industry and academic researchers on the necessity and sufficiency of the storage abstractions we propose and developing educational materials to demonstrate how they should be used.