Observation-Based Software Testing

Case Western Reserve University, Cleveland OH

Investigators

Abstract

A new approach to software testing called "observation-based testing" is investigated. Observation-based testing is intended to reduce the cost of testing by minimizing the number of test cases that must be evaluated manually for conformance to requirements. It calls for profiling the executions induced by a set of potential test cases and then filtering the executions based on their profiles to identify a subset that warrants manual evaluation. The executions are filtered using multivariate analysis techniques. The subset of execution that is selected for evaluation is one whose profiles suggest is more likely to contain failures than are other subsets. In order to provide a methodology and set of tools for observation-based testing, alternative forms of execution are evaluated to determine which are most effective for identifying conditions associated with program failure; alternative procedures for filtering executions are evaluated to determine which are most effective for identifying failures among a set of executions; and alternative observation-based techniques for estimating software reliability are evaluated with respect to their efficiency. In conjunction with such methodological research, a prototype suite of tools is being developed to support observation-based testing.