Context

Executing various sequences of system functions in a system under test represents one of the primary techniques in software testing. The natural method for creating effective, consistent and efficient test sequences is to model the system under test and employ an algorithm to generate tests that satisfy a defined test coverage criterion. Several criteria for preferred test set properties can be defined. In addition, to optimize the test set from an economic viewpoint, the priorities of the various parts of the system model under test must be defined.

Objective

Using this prioritization, the test cases exercise the high-priority parts of the system under test by more path combinations than those with low priority (this prioritization can be combined with the test coverage criterion that determines how many path combinations of the individual parts of the system are tested). Evidence from the literature and our observations confirm that finding a universal algorithm that produces a test set with preferred properties for all test coverage criteria is a challenging task. Moreover, for different individual problem instances, different algorithms provide results with the best value of a preferred property. In this paper, we present a portfolio-based strategy to perform the best test selection.

Method

The proposed strategy first employs a set of current algorithms to generate test sets; then, a preferred property of each test set is assessed in terms of the selected criterion, and finally, the test set with the best value of a preferred property is chosen.

Results

The experimental results confirm the validity and usefulness of this strategy. For individual instances of 50 system under test models, different algorithms provided results having the best preferred property value; these results varied by the required test coverage level, the size of the priority parts of the model, and the selected test set preferred property criteria.

Conclusion

In addition to the used algorithms, the proposed strategy can be used to assess the optimality of different path-based testing algorithms and choose a suitable algorithm for the testing.