CRII: SHF: Regression Testing for Projects with Distributed Software Histories

University Of Texas At Austin, Austin TX

Investigators

Abstract

Developers practice regression testing -- running tests against each project commit -- to check that project changes do not break any functionality. While important, regression testing is expensive due to the number of tests and the number of commits. Regression test selection (RTS) techniques speed up regression testing by skipping to run tests that are not affected by recent changes, and regression test prioritization (RTP) techniques reorder tests to run failing tests faster. Existing regression techniques analyze only two adjacent commits, effectively assuming a linear software history. However, the revolution in version-control systems changed the shape of software histories. Distributed software histories are complex graphs of branches and merges, which do not match the simplistic view of the existing techniques. Although existing regression techniques can be run if a distributed software history is linearized, these techniques underutilize the data available in the distributed software history. To speed up regression testing, the PI proposes to design techniques that utilize the data available in the entire distributed software history. This project proposes four tasks to improve RTS and RTP: (1) non-adjacent commit reuse - discover likely optimal commit to be used in each analysis rather than always analyzing adjacent commits; (2) multi-commit analyses - design methods that analyze more than two commits rather than always analyzing only two commits; (3) command-aware methods - specialize the methods for various commands that create each commit rather than be command-unaware; and (4) unified implementation and evaluation - share the implementation and results among techniques, and evaluate the techniques on open-source and industrial projects. The broader impacts of improving RTS and RTP are to increase developers' productivity and reduce the resource usage during testing phases.