SHF: Small: Revisiting Assumptions of Regression Testing

University Of Illinois At Urbana-Champaign, Urbana IL

Investigators

Abstract

Regression testing is important as it checks that changes to software do not break previously working functionality. However, regression testing is expensive as it requires executing a large number of tests and inspecting their failed runs. To speed up regression testing, researchers have proposed many techniques, including test selection (which, given a set of tests and software changes, selects a subset of tests that are affected by the changes) and test-suite reduction (which identifies what tests can be removed from a test suite without substantially reducing its fault-detection capability). While some of those techniques have been successful in practice, there is a lot of opportunity to further improve regression testing by alleviating the assumptions upon which the existing techniques are built. Specifically, this project improves regression testing by revisiting these six assumptions: (1) tests are not deterministic (but depend on timing, environment, or concurrency), (2) code histories are not linear (but convoluted graphs of branches and merges), (3) test selection is relevant not only for large projects (but developers manually select tests even for small projects), (4) test-suite reduction can decrease fault-detection capability in one version (but can decrease even more in future versions), (5) tests depend not only on code (but also on non-code artifacts), and (6) tests depend not only on manually written artifacts (but also on automatically generated artifacts). The broader impacts of improving regression testing are to increase the speed of software development and improve the quality of developed software.