Combinatorial Testing Tools

The combinatorial testing techniques that I’ve discussed and published in the issue # 28 of Testing Experience Magazine article, are intended to solve a basic combinatorial testing problem:

Direct Combinatorial Test Problem: “In a software system accepting N input variables, each of which can take on different values, find the test set with the smallest number of test cases that guarantees (at least) coverage of all combinations (2-tuples) of all the values of the variables involved.”

The Pairwise technique and a large number of support tools have been developed to solve this problem. Once such a test set (the smallest possible) has been generated, you should run the test cases and detect (if present) all the defects that arise in the software under test. There is also a second problem, maybe less “popular” compared to the previous, as follows:

Reverse Combinatorial Test Problem: “Given a Test Set for which you do not know the method of generation (if any), calculate what percentage of nwise coverage the test set ensures, with nwise between 1 and the number variables in the test set”. An example: tests generated by automatic tools for which you have low or almost zero process control, or when the “test cases” are generated by the automatic flows that feed interfaces between different systems (think of a system that transmits accounting data from a system A to B); test data are – in general – excerpts from historical series over which you have no control. I’ve developed the Combinatorial Testing Tools, freely downloadable from http://www.opensource-combinatorial-soft-test-tools.net/ , to provide support to help solve both problems of reverse combinatorial testing problem previously stated.

More info on the article attached. The complete issue should be downloaded from the link  http://www.testingexperience.com/.

Attachments:
FileDescriptionFile sizeDownloadsCreated
Download this file (testingexperience_issue28_2014-12_Berta.pdf)Testing Experience # 28 - Combinatorial Testing ToolsArticle appered on issue # 28 of the Testing Experience Magazine675 kB5402014-12-16 09:59