CSR: Small: Elastic Soft State Cache as an OS Service

University Of California-Los Angeles, Los Angeles CA

Investigators

Abstract

To efficiently serve user requests, cloud applications cache soft-state data in memory. Data is considered “soft state” when it is helpful only for efficiency and losing it would not lead to functional issues. Soft state is ubiquitously used as it enables applications to trade off memory for better performance, and performance gains often grow with the amount of memory provisioned. Today’s systems provision memory statically to applications to prevent their cached data from growing unbounded and draining the server’s memory. It is extremely hard, however, for static memory provisioning to be able to address each application’s soft-state needs while simultaneously maximizing the use of global memory. Dynamic provisioning is challenging as current systems are unable to safely and efficiently add/remove memory between applications. This project builds a new virtual memory abstraction for soft-state caching (referred to as “soft memory’”), as well as a soft memory management system called Midas, enabling application developers to easily register their soft-state data with the Midas API and leave cache management and dynamic scaling to the OS kernel. Under Midas, soft-state caching is no longer application-specific; instead, it is provided by the OS kernel as a service. Modern society is relying increasingly on cloud platforms where memory is heavily underutilized. On the application side, most cloud applications need to cache intermediate results or soft-state data; more memory can significantly boost their throughput and reduce their latency. However, existing cloud systems provision memory statically, leading to memory waste. Simultaneously, the caching needs of many memory-hungry applications are not being satisfied. This project develops a new soft-state caching system, providing caching to all applications as an operating system service. Such a service can significantly improve memory utilization for the cloud, leading to monetary cost savings. In the meantime, it improves the performance of individual applications running on the cloud by letting them opportunistically use more memory, resulting in improved user experiences. The project also makes efforts in education and diversity by incorporating research into courses and recruiting and training junior researchers from underrepresented groups. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.