CAREER: Self-Driving Database Management Systems

Carnegie Mellon University, Pittsburgh PA

Investigators

Abstract

Over the last four decades, both researchers and vendors have built advisory tools to assist human administrators in database management system (DBMS) tuning and physical design. All of this previous work, however, is incomplete because they require humans to make the final decisions about any changes and they are reactionary measures that fix problems after they occur. What is needed for a truly "self-driving" DBMS is a new software architecture that is explicitly designed for autonomous operation. With this, the DBMS will remove the need for humans to oversee and maintain the software. It also enables new optimizations that are important for modern high-performance DBMSs, but which are not possible today because the complexity of managing these systems has surpassed the abilities of human experts. Such a system will remove the human capital impediments of deploying databases and allow organizations in all facets of society (e.g., business, science, government) to more easily derive the benefits of data-driven decision-making applications. The techniques developed as part of this research are also applicable to other problem domains where autonomous operation could improve a software system's performance and efficiency, including both larger systems (e.g., distributed DBMSs, data centers) and smaller devices (e.g., mobile devices, IoT sensors). This project investigates techniques for self-driving DBMSs that combines state-of-the-art methods from database systems, machine learning (ML), and control theory. Achieving autonomous operation in a DBMS is now possible due to algorithmic advancements in ML, as well as improvements to storage and computation hardware. What makes this different than earlier attempts is that all aspects of the system are controlled by an integrated planning component that not only optimizes the system for the current workload but also predicts future workload trends before they occur so that the system can prepare itself accordingly. This work will produce on-line methods for discovering relevant optimization actions based on these workload forecast models, thereby enabling the planning component to converge to a better configuration with less training data. In addition to this, this research will study how to deploy these actions without hindering the DBMS's performance (e.g., downtime due to restarts) or causing incorrect behavior. The outcome will be a set of first principles for efficient self-driving DBMS architectures that can deploy these modifications and provide the necessary feedback for their integrated models. Such principles are timely in identifying issues that inhibit automation of existing systems and influencing the design of future DBMS architectures. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.