Please use this identifier to cite or link to this item: http://dx.doi.org/10.18419/opus-10378
Authors: Niedermayr, Rainer
Wagner, Stefan
Title: Is the stack distance between test case and method correlated with test effectiveness?
Issue Date: 2019
metadata.ubs.publikation.typ: Konferenzbeitrag
metadata.ubs.konferenzname: Evaluation and Assessment in Software Engineering (2019, Kopenhagen)
metadata.ubs.publikation.source: EASE '19 Proceedings of the Evaluation and Assessment on Software Engineering. New York, NY : ACM, 2019. - ISBN 978-1-4503-7145-2, S. 189-198
URI: http://elib.uni-stuttgart.de/handle/11682/10395
http://nbn-resolving.de/urn:nbn:de:bsz:93-opus-ds-103957
http://dx.doi.org/10.18419/opus-10378
ISBN: 978-1-4503-7145-2
metadata.ubs.bemerkung.extern: © 2019 Copyright held by the owner/author(s). Publication rights licensed to Association for Computing Machinery. This is the author’s version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in Evaluation and Assessment in Software Engineering (EASE ’19), April 15-17, 2019, Copenhagen, Denmark, https://doi.org/10.1145/3319008.3319021.
Abstract: Mutation testing is a means to assess the effectiveness of a test suite and its outcome is considered more meaningful than code coverage metrics. However, despite several optimizations, mutation testing requires a significant computational effort and has not been widely adopted in industry. Therefore, we study in this paper whether test effectiveness can be approximated using a more light-weight approach. We hypothesize that a test case is more likely to detect faults in methods that are close to the test case on the call stack than in methods that the test case accesses indirectly through many other methods. Based on this hypothesis, we propose the minimal stack distance between test case and method as a new test measure, which expresses how close any test case comes to a given method, and study its correlation with test effectiveness. We conducted an empirical study with 21 open-source projects, which comprise in total 1.8 million LOC, and show that a correlation exists between stack distance and test effectiveness. The correlation reaches a strength up to 0.58. We further show that a classifier using the minimal stack distance along with additional easily computable measures can predict the mutation testing result of a method with 92.9% precision and 93.4% recall. Hence, such a classifier can be taken into consideration as a light-weight alternative to mutation testing or as a preceding, less costly step to that.
Appears in Collections:05 Fakultät Informatik, Elektrotechnik und Informationstechnik

Files in This Item:
File Description SizeFormat 
niedermayr2019a.pdf631,17 kBAdobe PDFView/Open


Items in OPUS are protected by copyright, with all rights reserved, unless otherwise indicated.