Towards refactoring-aware regression test selection

Author email: kaiyuanw@utexas.edu
Tool name: REKs
Description: Regression testing checks that recent project changes do not break previously working functionality. Although important, regression testing is costly when changes are frequent. Regression test selection (RTS) optimizes regression testing by running only tests whose results might be affected by a change. Traditionally, RTS collects dependencies (e.g., on files) for each test and skips the tests, at a new project revision, whose dependencies did not change. Existing RTS techniques do not differentiate behavior-preserving transformations (i.e., refactorings) from other code changes. As a result, tests are run more frequently than necessary. We present the first step towards a refactoring-aware RTS technique, dubbed Reks, which skips tests affected only by behavior-preserving changes. Reks defines rules to update the test dependencies without running the tests. To ensure that Reks does not hide any bug introduced by the refactoring engines, we integrate Reks only in the pre-submit testing phase, which happens on the developers' machines. We evaluate Reks by measuring the savings in the testing effort. Specifically, we reproduce 100 refactoring tasks performed by developers of 37 projects on GitHub. Our results show that Reks would not run, on average, 33% of available tests (that would be run by a refactoring-unaware RTS technique). Additionally, we systematically run 27 refactoring types on ten projects. The results, based on 74,160 refactoring tasks, show that Reks would not run, on average, 16% of tests (max: 97% and SD: 24%). Finally, our results show that the Reks update rules are efficient.
Bibtex: @inproceedings{10.1145/3180155.3180254, author = {Wang, Kaiyuan and Zhu, Chenguang and Celik, Ahmet and Kim, Jongwook and Batory, Don and Gligoric, Milos}, title = {Towards Refactoring-Aware Regression Test Selection}, year = {2018}, isbn = {9781450356381}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3180155.3180254}, doi = {10.1145/3180155.3180254}, booktitle = {Proceedings of the 40th International Conference on Software Engineering}, pages = {233–244}, numpages = {12}, keywords = {regression test selection, behavior-preserving changes, Reks}, location = {Gothenburg, Sweden}, series = {ICSE ’18} }
Link to public pdf: https://dl.acm.org/doi/abs/10.1145/3180155.3180254
Link to tool webpage: http://cozy.ece.utexas.edu/reks/
Link to demo: Not provided by authors
Category: None
Year and Conference: 2018, ICSE
Terms of use