CAREER: Toward Effective, Predictable, and Consistent Software Testing

University Of Washington, Seattle WA

Investigators

Abstract

Billions of people depend on software systems every day, but software remains buggy and vulnerable. Software testing is the predominant approach for software quality assurance (QA), and hundreds of thousands of developers are involved in QA activities on a daily basis. Developers heavily rely on intuition, experience, and familiarity with a code base when developing tests, and when reasoning about test effectiveness and consistency. However, experience varies widely, which has negative impacts on software quality and developer productivity. Addressing these challenges, this project develops practical end-to-end solutions for effective and consistent software testing that integrate well with existing QA practices. It aims to substantially increase the quality of the software that pervades modern technology, by providing developers with a framework and methodology for systematic software testing with quantifiable efficacy. The overall goal of this project is to advance software quality by making software testing more effective, predictable, and consistent. To achieve this goal, the proposed research co-develops a new mutation-based testing approach along with a methodology to quantify its efficacy. Specifically, this project (1) develops models for program context and test goal utility, (2) builds upon these models and develops a learning-based approach for context-sensitive predictions of test goal utility and ranking test goals, and (3) develops a probabilistic method that assesses how representative a set of test goals is for bugs encountered during software development. In addition to the research artifacts, this project will provide publicly available artifacts that enable training of a new generation of software engineers. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.