Time to clean your test objectives

Author email: m.marconzzi@imperial.ac.uk
Tool name: Lclean
Description: Testing is the primary approach for detecting software defects. A major challenge faced by testers lies in crafting efficient test suites, able to detect a maximum number of bugs with manageable effort. To do so, they rely on coverage criteria, which define some precise test objectives to be covered. However, many common criteria specify a significant number of objectives that occur to be infeasible or redundant in practice, like covering dead code or semantically equal mutants. Such objectives are well-known to be harmful to the design of test suites, impacting both the efficiency and precision of the tester's effort. This work introduces a sound and scalable technique to prune out a significant part of the infeasible and redundant objectives produced by a panel of white-box criteria. In a nutshell, we reduce this task to proving the validity of logical assertions in the code under test. The technique is implemented in a tool that relies on weakest-precondition calculus and SMT solving for proving the assertions. The tool is built on top of the Frama-C verification platform, which we carefully tune for our specific scalability needs. The experiments reveal that the pruning capabilities of the tool can reduce the number of targeted test objectives in a program by up to 27% and scale to real programs of 200K lines, making it possible to automate a painstaking part of their current testing process.
Bibtex: @inproceedings{10.1145/3180155.3180191, author = {Marcozzi, Micha\"{e}l and Bardin, S\'{e}bastien and Kosmatov, Nikolai and Papadakis, Mike and Prevosto, Virgile and Correnson, Lo\"{\i}c}, title = {Time to Clean Your Test Objectives}, year = {2018}, isbn = {9781450356381}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3180155.3180191}, doi = {10.1145/3180155.3180191}, booktitle = {Proceedings of the 40th International Conference on Software Engineering}, pages = {456–467}, numpages = {12}, keywords = {infeasible objectives, coverage criteria, redundant objectives}, location = {Gothenburg, Sweden}, series = {ICSE ’18} }
Link to public pdf: https://dl.acm.org/doi/abs/10.1145/3180155.3180191
Link to tool webpage: https://sites.google.com/view/lclean
Link to demo: Not provided by authors
Category: None
Year and Conference: 2018, ICSE
Terms of use