End-to-End Performance Management for Large Distributed Storage

University Of California-Santa Cruz, Santa Cruz CA

Investigators

Abstract

Storage systems for large distributed clusters of computer servers are themselves large and distributed. Their complexity and scale make it hard to ensure that applications using them get good, predictable performance. At the same time, shared access to the system from multiple applications, users, and internal system activities leads to a need for predictable performance. This research investigates mechanisms for improving storage system performance in large distributed storage systems through mechanisms that integrate the performance aspects of the path that I/O operations take through the system, from the application interface on the compute server, through the network, to the storate servers. The research focuses on five parts of the I/O path in a distributed storage system: I/O scheduling at the storage server, storage server cache management, client-to-server network flow control, client-to-server connection management, and client cache management.