Nemo: multi-criteria test-suite minimization with integer nonlinear programming

Author email: junwel1@uci.edu
Tool name: Nemo
Description: Multi-criteria test-suite minimization aims to remove redundant test cases from a test suite based on some criteria such as code coverage, while trying to optimally maintain the capability of the reduced suite based on other criteria such as fault-detection effectiveness. Existing techniques addressing this problem with integer linear programming claim to produce optimal solutions. However, the multi-criteria test-suite minimization problem is inherently nonlinear, due to the fact that test cases are often dependent on each other in terms of test-case criteria. In this paper, we propose a framework that formulates the multi-criteria test-suite minimization problem as an integer nonlinear programming problem. To solve this problem optimally, we programmatically transform this nonlinear problem into a linear one and then solve the problem using modern linear solvers. We have implemented our framework as a tool, called Nemo, that supports a number of modern linear and nonlinear solvers. We have evaluated Nemo with a publicly available dataset and minimization problems involving multiple criteria including statement coverage, fault-revealing capability, and test execution time. The experimental results show that Nemo can be used to efficiently find an optimal solution for multi-criteria test-suite minimization problems with modern solvers, and the optimal solutions outperform the suboptimal ones by up to 164.29% in terms of the criteria considered in the problem.
Bibtex: @inproceedings{10.1145/3180155.3180174, author = {Lin, Jun-Wei and Jabbarvand, Reyhaneh and Garcia, Joshua and Malek, Sam}, title = {Nemo: Multi-Criteria Test-Suite Minimization with Integer Nonlinear Programming}, year = {2018}, isbn = {9781450356381}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3180155.3180174}, doi = {10.1145/3180155.3180174}, booktitle = {Proceedings of the 40th International Conference on Software Engineering}, pages = {1039–1049}, numpages = {11}, keywords = {test-suite minimization, integer programming}, location = {Gothenburg, Sweden}, series = {ICSE ’18} }
Link to public pdf: https://dl.acm.org/doi/10.1145/3180155.3180174
Link to tool webpage: https://www.ics.uci.edu/~seal/projects/nemo/index.html
Link to demo: Not provided by authors
Category: None
Year and Conference: 2018, ICSE
Terms of use