Fine-Grained Test Minimization

Author email: arashvhb@ece.ubc.ca
Tool name: Testler
Description: As a software system evolves, its test suite can accumulate redundancies over time. Test minimization aims at removing redundant test cases. However, current techniques remove whole test cases from the test suite using test adequacy criteria, such as code coverage. This has two limitations, namely (1) by removing a whole test case the corresponding test assertions are also lost, which can inhibit test suite effectiveness, (2) the issue of partly redundant test cases, i.e., tests with redundant test statements, is ignored. We propose a novel approach for fine-grained test case minimization. Our analysis is based on the inference of a test suite model that enables automated test reorganization within test cases. It enables removing redundancies at the test statement level, while preserving the coverage and test assertions of the test suite. We evaluated our approach, implemented in a tool called Testler, on the test suites of 15 open source projects. Our analysis shows that over 4,639 (24%) of the tests in these test suites are partly redundant, with over 11,819 redundant test statements in total. Our results show that Testler removes 43% of the redundant test statements, reducing the number of partly redundant tests by 52%. As a result, test suite execution time is reduced by up to 37% (20% on average), while maintaining the original statement coverage, branch coverage, test assertions, and fault detection capability.
Bibtex: @inproceedings{10.1145/3180155.3180203, author = {Vahabzadeh, Arash and Stocco, Andrea and Mesbah, Ali}, title = {Fine-Grained Test Minimization}, year = {2018}, isbn = {9781450356381}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3180155.3180203}, doi = {10.1145/3180155.3180203}, booktitle = {Proceedings of the 40th International Conference on Software Engineering}, pages = {210–221}, numpages = {12}, keywords = {test model, test reduction, test redundancy, test minimization}, location = {Gothenburg, Sweden}, series = {ICSE ’18} }
Link to public pdf: https://dl.acm.org/doi/10.1145/3180155.3180203
Link to tool webpage: https://github.com/saltlab/Testler
Link to demo: Not provided by authors
Category: None
Year and Conference: 2018, ICSE
Terms of use