ITR-(ASE)-(dmc+int): Reconfigurable, Data-driven Resource Allocation in Complex Systems: Practice and Theoretical Foundations

College Of William And Mary, Williamsburg VA

Investigators

Abstract

Reconfigurable, data-driven resource allocation in complex systems: practice and theoretical foundations As web servers are developing into central components in the information infrastructure of our society, it becomes challenging to serve their ever-increasing and diversified customer population while ensuring high availability in a cost-effective way. The complexity of today's web servers systems and the variability of their workload often make effective resource allocation an elusive goal. This proposal seeks support for the development of a data-driven performance-engineering framework to automate the process of robust, workload-aware resource allocation and management in today's complex web server systems. The researcher's focus is on the development of better understanding of the workload resource demands, on the development and implementation of efficient methodologies for bottleneck identification and resource allocation at the system level, and on the development of efficient analytic methodologies for performance prediction. To meet the above targets the following research tasks will be accomplished: o A better understanding of the workload resource demands in web servers that serve dynamic pages will be obtained, focusing on identifying the different resource bottlenecks and the workload conditions under which these bottlenecks are triggered. o A data collection mechanism at the system level will be devised that will gather statistical information, which can prompt scheduler reconfigurations. This mechanism will provide a better understanding on what system and workload data, and at what level of detail, needs to be monitored at run-time to readily provide to the allocation policies information about the state of the system. o New, data-driven scheduling policies will be developed and will be implemented at the system level for the various bottleneck resources that will allow quick system recovery under transient overload conditions. o New theoretical results will allow modeling of the workload and resource allocation policies with compact and tractable models. These models will guide parameterization of the resource allocation policies. Intellectual Merit: The proposed research will advance science and engineering by integrating data and analytic models for the development and implementation on actual systems of both workload-aware and system-aware algorithms to modulate resource allocation in web servers serving dynamic pages under constantly changing workload conditions. The proposed research, even assuming that not all results are positive, will attempt to answer several fundamental questions for the development of cost-effective, autonomic systems. The theoretical contributions of this research will advance the state-of-the-art in modeling of complex systems that are subject to continuous and severe changes in workload intensities and demands. Broader Impact: The impact of this research will affect that state-of-the-practice in actual off-the-shelf systems via industrial collaborations, specifically Seagate Research, by providing algorithms and tools that can modulate and automate the process of resource allocation in complex environments. Through this project, the researcher will also be able to impact the education of several students, preparing them to better meet industry demands in the areas of performance modeling and resource allocation in complex environments.