An Empirical Comparison of Mutant Selection Approaches


Mutation analysis is a well-known method for measuring the quality of test suites. However, it is computationally inten- sive compared to other measures, which makes it hard to use in practice. Choosing a smaller subset of mutations to run is a simple approach that can alleviate this problem. Mu- tation operator selection has been heavily researched. Re- cently, researchers have found that sampling mutants can achieve accuracy and mutant reduction similar to operator selection. However, the empirical support for these conclu- sions has been limited, due to the small number of subject programs investigated. The best sampling technique is also an open problem. Our research compares a large number of sampling and operator selection criteria based on their ability to predict the full mutation score as well as the consistency of mu- tation reduction ratios achieved. Our results can be used to choose an appropriate mutation reduction technique by the reduction and level of delity to full mutation results required. We nd that all sampling approaches perform better than operator selection methods, when considering ability to pre- dict the full mutation score as well as the consistency of mutation reduction ratios achieved.

In Technical Report, School of Engineering and Computer Science,Oregon State University.