05 Fakultät Informatik, Elektrotechnik und Informationstechnik
Permanent URI for this collectionhttps://elib.uni-stuttgart.de/handle/11682/6
Browse
4 results
Search Results
Item Open Access Towards practical application of mutation testing in industry : traditional versus extreme mutation testing(2022) Betka, Maik; Wagner, StefanMutation testing is a technique that changes code instructions to assess the quality of automated software tests. Industry has not broadly adopted the technique because execution and analysis times are too long and not considered worth the effort. To change this, a variation called “extreme mutation testing” emerged, which mutates whole methods instead of instructions. The extreme variant trades accuracy for speed gains and also provides pre‐analyzed results. In this study, we aim to analyze both techniques on their granularity levels, look for benefits when combining them, and find motivations when a developer considers killing mutants. For that, we conducted a case study in a company from the semiconductor industry. We mutated a large Java software project which is tested by more than 11,000 unit tests, analyzed the results, manually inspected more than 1000 mutants, and conducted a focus group with five developers of the software. Among other results, we provide the distribution of traditional across extreme mutants as well as qualitative coding results of our mutant inspection and focus group transcript. We conclude that the traditional approach can be similarly strategically applied as the extreme one and that motivations of developers to target mutants are mostly not code related.Item Open Access A fine-grained data set and analysis of tangling in bug fixing commits(2022) Herbold, Steffen; Trautsch, Alexander; Ledel, Benjamin; Aghamohammadi, Alireza; Ghaleb, Taher A.; Chahal, Kuljit Kaur; Bossenmaier, Tim; Nagaria, Bhaveet; Makedonski, Philip; Ahmadabadi, Matin Nili; Szabados, Kristof; Spieker, Helge; Madeja, Matej; Hoy, Nathaniel; Lenarduzzi, Valentina; Wang, Shangwen; Rodríguez-Pérez, Gema; Colomo-Palacios, Ricardo; Verdecchia, Roberto; Singh, Paramvir; Qin, Yihao; Chakroborti, Debasish; Davis, Willard; Walunj, Vijay; Wu, Hongjun; Marcilio, Diego; Alam, Omar; Aldaeej, Abdullah; Amit, Idan; Turhan, Burak; Eismann, Simon; Wickert, Anna-Katharina; Malavolta, Ivano; Sulír, Matúš; Fard, Fatemeh; Henley, Austin Z.; Kourtzanidis, Stratos; Tuzun, Eray; Treude, Christoph; Shamasbi, Simin Maleki; Pashchenko, Ivan; Wyrich, Marvin; Davis, James; Serebrenik, Alexander; Albrecht, Ella; Aktas, Ethem Utku; Strüber, Daniel; Erbel, JohannesContext: Tangled commits are changes to software that address multiple concerns at once. For researchers interested in bugs, tangled commits mean that they actually study not only bugs, but also other concerns irrelevant for the study of bugs. Objective: We want to improve our understanding of the prevalence of tangling and the types of changes that are tangled within bug fixing commits. Methods: We use a crowd sourcing approach for manual labeling to validate which changes contribute to bug fixes for each line in bug fixing commits. Each line is labeled by four participants. If at least three participants agree on the same label, we have consensus. Results: We estimate that between 17% and 32% of all changes in bug fixing commits modify the source code to fix the underlying problem. However, when we only consider changes to the production code files this ratio increases to 66% to 87%. We find that about 11% of lines are hard to label leading to active disagreements between participants. Due to confirmed tangling and the uncertainty in our data, we estimate that 3% to 47% of data is noisy without manual untangling, depending on the use case.Item Open Access Software product line testing : a systematic literature review(2024) Agh, Halimeh; Azamnouri, Aidin; Wagner, StefanA Software Product Line (SPL) is a software development paradigm in which a family of software products shares a set of core assets. Testing has a vital role in both single-system development and SPL development in identifying potential faults by examining the behavior of a product or products, but it is especially challenging in SPL. There have been many research contributions in the SPL testing field; therefore, assessing the current state of research and practice is necessary to understand the progress in testing practices and to identify the gap between required techniques and existing approaches. This paper aims to survey existing research on SPL testing to provide researchers and practitioners with up-to-date evidence and issues that enable further development of the field. To this end, we conducted a Systematic Literature Review (SLR) with seven research questions in which we identified and analyzed 118 studies dating from 2003 to 2022. The results indicate that the literature proposes many techniques for specific aspects (e.g., controlling cost/effort in SPL testing); however, other elements (e.g., regression testing and non-functional testing) still need to be covered by existing research. Furthermore, most approaches are evaluated by only one empirical method, most of which are academic evaluations. This may jeopardize the adoption of approaches in industry. The results of this study can help identify gaps in SPL testing since specific points of SPL Engineering still need to be addressed entirely.Item Open Access Do RESTful API design rules have an impact on the understandability of Web APIs?(2023) Bogner, Justus; Kotstein, Sebastian; Pfaff, TimoContext. Web APIs are one of the most used ways to expose application functionality on the Web, and their understandability is important for efficiently using the provided resources. While many API design rules exist, empirical evidence for the effectiveness of most rules is lacking. Objective. We therefore wanted to study 1) the impact of RESTful API design rules on understandability, 2) if rule violations are also perceived as more difficult to understand, and 3) if demographic attributes like REST-related experience have an influence on this. Method. We conducted a controlled Web-based experiment with 105 participants, from both industry and academia and with different levels of experience. Based on a hybrid between a crossover and a between-subjects design, we studied 12 design rules using API snippets in two complementary versions: one that adhered to a rule and one that was a violation of this rule. Participants answered comprehension questions and rated the perceived difficulty. Results. For 11 of the 12 rules, we found that violation performed significantly worse than rule for the comprehension tasks. Regarding the subjective ratings, we found significant differences for 9 of the 12 rules, meaning that most violations were subjectively rated as more difficult to understand. Demographics played no role in the comprehension performance for violation . Conclusions. Our results provide first empirical evidence for the importance of following design rules to improve the understandability of Web APIs, which is important for researchers, practitioners, and educators.