SHF: Small: Automatic Exploration and Analysis of Software Performance Responses

University Of California-Berkeley, Berkeley CA

Investigators

Abstract

Performance issues in software systems often result in negative user experience, highly publicized commercial failures, substantial monetary penalties, and even abandonment of multi-million dollar projects. Performance issues can cause a software system to run for an unexpectedly long time, or use an incredibly large amount of computer memory, potentially stalling the computer. Performance issues are often input-dependent, occurring only when the software is run with certain specific inputs. If malicious attackers find out the inputs on which programs have performance issues, they can force these inputs on the program and cause vital software systems to go down or become unavailable (i.e., a kind of denial-of-service attack). This project's goal is to build tools which can automatically identify the inputs on which a given software system has serious performance issues. These tools can then be used by programmers to identify performance issues in software before the software is vulnerable to attackers or deployed to the public. Successful completion of this project will help programmers identify and fix software performance issues early, thus increasing the efficiency, reliability, and security of software. A large body of research has focused on diagnosing performance problems by analyzing dynamically collected performance profiles. Almost all of these techniques assume the availability of test inputs for performance profiling. This project proposes to develop techniques to discover potential performance issues in real-world software systems automatically. Whole-system analysis, which is necessary for such input generation, does not scale for real-world software systems. The project addresses this scalability challenge in phases: a top-down approach to identify components that may have performance bottlenecks, a sophisticated exploration and generalization technique to learn worst-case behavior patterns, and a bottom-up approach for creating a performance-degrading input for the entire program. The technique will also create a performance regression test suite for a software system under test using a novel performance-coverage metric developed as part of the project. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.