05 Fakultät Informatik, Elektrotechnik und Informationstechnik
Permanent URI for this collectionhttps://elib.uni-stuttgart.de/handle/11682/6
Browse
12 results
Search Results
Item Open Access On the impact of service-oriented patterns on software evolvability: a controlled experiment and metric-based analysis(2019) Bogner, Justus; Wagner, Stefan; Zimmermann, AlfredBackground: Design patterns are supposed to improve various quality attributes of software systems. However, there is controversial quantitative evidence of this impact. Especially for younger paradigms such as service- and Microservice-based systems, there is a lack of empirical studies. Objective: In this study, we focused on the effect of four service-based patterns - namely Process Abstraction, Service Façade, Decomposed Capability, and Event-Driven Messaging - on the evolvability of a system from the viewpoint of inexperienced developers. Method: We conducted a controlled experiment with Bachelor students (N = 69). Two functionally equivalent versions of a service-based web shop - one with patterns (treatment group), one without (control group) - had to be changed and extended in three tasks. We measured evolvability by the effectiveness and efficiency of the participants in these tasks. Additionally, we compared both system versions with nine structural maintainability metrics for size, granularity, complexity, cohesion, and coupling. Results: Both experiment groups were able to complete a similar number of tasks within the allowed 90 min. Median effectiveness was 1/3. Mean efficiency was 12% higher in the treatment group, but this difference was not statistically significant. Only for the third task, we found statistical support for accepting the alternative hypothesis that the pattern version led to higher efficiency. In the metric analysis, the pattern version had worse measurements for size and granularity while simultaneously having slightly better values for coupling metrics. Complexity and cohesion were not impacted. Interpretation: For the experiment, our analysis suggests that the difference in efficiency is stronger with more experienced participants and increased from task to task. With respect to the metrics, the patterns introduce additional volume in the system, but also seem to decrease coupling in some areas. Conclusions: Overall, there was no clear evidence for a decisive positive effect of using service-based patterns, neither for the student experiment nor for the metric analysis. This effect might only be visible in an experiment setting with higher initial effort to understand the system or with more experienced developers.Item Open Access An industrial case study on the evaluation of a safety engineering approach for software-intensive systems in the automotive domain(2016) Abdulkhaleq, Asim; Vöst, Sebastian; Wagner, Stefan; Thomas, JohnSafety remains one of the essential and vital aspects in today's automotive systems. These systems, however, become ever more complex and dependent on software which is responsible for most of their critical functions. Therefore, the software components need to be analysed and verified appropriately in the context of software safety. The complexity of software systems makes defining software safety requirements with traditional safety analysis techniques difficult. A new technique called STPA (Systems-Theoretic Process Analysis) based on system and control theory has been developed by Leveson to cope with complex systems. Based on STPA, we have developed a comprehensive software safety engineering approach in which the software and safety engineers integrate the analysis of software risks with their verification to recognize the software-related hazards and reduce the risks to a low level. In this paper, we explore and evaluate the application of our approach to a real industrial system in the automotive domain. The case study was conducted analysing the software controller of the Active Cruise Control System (ACC) of the BMW Group.Item Open Access Application performance management : measuring and optimizing the digital customer experience(Troisdorf : SIGS DATACOM GmbH, 2018) Hoorn, André van; Siegl, StefanNowadays, the success of most companies is determined by the quality of their IT services and application systems. To make sure that application systems provide the expected quality of service, it is crucial to have up-to-date information about the system and the user experience to detect problems and to be able to solve them effectively. Application performance management (APM) is a core IT operations discipline that aims to achieve an adequate level of performance during operations. APM comprises methods, techniques, and tools for i) continuously monitoring the state of an applications system and its usage, as well as for ii) detecting, diagnosing, and resolving performance-related problems using the monitored data. This book provides an introduction by covering a common conceptual foundation for APM. On top of the common foundation, we introduce today's tooling landscape and highlight current challenges and directions of this discipline.Item Open Access ContinuITy - automatisiertes Performance-Testen in der kontinuierlichen Softwareentwicklung : Abschlussbericht(2020) Angerstein, Tobias; Heger, Christoph; Hoorn, André van; Okanović, Dušan; Schulz, Henning; Siegl, Stefan; Wert, AlexanderZiel des Forschungsprojekts ContinuITy war die Entwicklung eines Ansatzes und entsprechender Werkzeugunterstützung zum automatisierten Performance-Testen - eingebettet in Prozesse und Infrastruktur der kontinuierlichen Softwareentwicklung. Ziel ist durch Ausnutzung von kontinuierlich aufgezeichneten Messdaten aus dem Produktivbetrieb, automatisiert effizientes und nachhaltiges Lasttesten zu gewährleisten und in die kontinuierliche Softwareentwicklung zu integrieren. Lasttests werden automatisiert aus Messdaten extrahiert und evolviert. Lasttests werden durch eine modulare Beschreibungssprache definiert und können durch zusätzliche Semantik - z. B. über Testart und -ziele - angereichert werden. Im Rahmen der Automatisierung des Softwareerstellungsprozesses (Continuous Delivery) erfolgt eine Auswahl relevanter Lasttests, die Erkennung von Regressionen und deren Diagnose. Basierend auf der Beschreibung der Aufgabenstellung und der Voraussetzungen fasst dieser Bericht die durchgeführten Aktivitäten und wesentlichen Ergebnisse zusammen.Item Open Access Will my tests tell me if I break this code?(2016) Niedermayr, Rainer; Juergens, Elmar; Wagner, StefanAutomated tests play an important role in software evolution because they can rapidly detect faults introduced during changes. In practice, code-coverage metrics are often used as criteria to evaluate the effectiveness of test suites with focus on regression faults. However, code coverage only expresses which portion of a system has been executed by tests, but not how effective the tests actually are in detecting regression faults. Our goal was to evaluate the validity of code coverage as a measure for test effectiveness. To do so, we conducted an empirical study in which we applied an extreme mutation testing approach to analyze the tests of open-source projects written in Java. We assessed the ratio of pseudo-tested methods (those tested in a way such that faults would not be detected) to all covered methods and judged their impact on the software project. The results show that the ratio of pseudo-tested methods is acceptable for unit tests but not for system tests (that execute large portions of the whole system). Therefore, we conclude that the coverage metric is only a valid effectiveness indicator for unit tests.Item Open Access Is the stack distance between test case and method correlated with test effectiveness?(2019) Niedermayr, Rainer; Wagner, StefanMutation testing is a means to assess the effectiveness of a test suite and its outcome is considered more meaningful than code coverage metrics. However, despite several optimizations, mutation testing requires a significant computational effort and has not been widely adopted in industry. Therefore, we study in this paper whether test effectiveness can be approximated using a more light-weight approach. We hypothesize that a test case is more likely to detect faults in methods that are close to the test case on the call stack than in methods that the test case accesses indirectly through many other methods. Based on this hypothesis, we propose the minimal stack distance between test case and method as a new test measure, which expresses how close any test case comes to a given method, and study its correlation with test effectiveness. We conducted an empirical study with 21 open-source projects, which comprise in total 1.8 million LOC, and show that a correlation exists between stack distance and test effectiveness. The correlation reaches a strength up to 0.58. We further show that a classifier using the minimal stack distance along with additional easily computable measures can predict the mutation testing result of a method with 92.9% precision and 93.4% recall. Hence, such a classifier can be taken into consideration as a light-weight alternative to mutation testing or as a preceding, less costly step to that.Item Open Access Too trivial to test? An inverse view on defect prediction to identify methods with low fault risk(2019) Niedermayr, Rainer; Röhm, Tobias; Wagner, StefanBackground: Test resources are usually limited and therefore it is often not possible to completely test an application before a release. To cope with the problem of scarce resources, development teams can apply defect prediction to identify fault-prone code regions. However, defect prediction tends to low precision in cross-project prediction scenarios. Aims: We take an inverse view on defect prediction and aim to identify methods that can be deferred when testing because they contain hardly any faults due to their code being “trivial”. We expect that characteristics of such methods might be project-independent, so that our approach could improve cross-project predictions. Method: We compute code metrics and apply association rule mining to create rules for identifying methods with low fault risk (LFR). We conduct an empirical study to assess our approach with six Java open-source projects containing precise fault data at the method level. Results: Our results show that inverse defect prediction can identify approx. 32–44% of the methods of a project to have a LFR; on average, they are about six times less likely to contain a fault than other methods. In cross-project predictions with larger, more diversified training sets, identified methods are even 11 times less likely to contain a fault. Conclusions: Inverse defect prediction supports the efficient allocation of test resources by identifying methods that can be treated with less priority in testing activities and is well applicable in cross-project prediction scenarios.Item Open Access Status quo in requirements engineering: a theory and a global family of surveys(2019) Wagner, Stefan; Méndez Fernández, Daniel; Felderer, Michael; Vetrò, Antonio; Kalinowski, Marcos; Wieringa, Roel; Pfahl, Dietmar; Conte, Tayana; Christiansson, Marie-Therese; Greer, Desmond; Lassenius, Casper; Männistö, Tomi; Nayebi, Maleknaz; Oivo, Markku; Penzenstadler, Birgit; Prikladnicki, Rafael; Ruhe, Guenther; Schekelmann, André; Sen, Sagar; Spínola, Rodrigo; Tuzcu, Ahmed; Vara, Jose Luis de la; Winkler, DieterRequirements Engineering (RE) has established itself as a software engineering discipline over the past decades. While researchers have been investigating the RE discipline with a plethora of empirical studies, attempts to systematically derive an empirical theory in context of the RE discipline have just recently been started. However, such a theory is needed if we are to define and motivate guidance in performing high quality RE research and practice. We aim at providing an empirical and externally valid foundation for a theory of RE practice, which helps software engineers establish effective and efficient RE processes in a problem-driven manner. We designed a survey instrument and an engineer-focused theory that was first piloted in Germany and, after making substantial modifications, has now been replicated in 10 countries worldwide. We have a theory in the form of a set of propositions inferred from our experiences and available studies, as well as the results from our pilot study in Germany. We evaluate the propositions with bootstrapped confidence intervals and derive potential explanations for the propositions. In this article, we report on the design of the family of surveys, its underlying theory, and the full results obtained from the replication studies conducted in 10 countries with participants from 228 organisations. Our results represent a substantial step forward towards developing an empirical theory of RE practice. The results reveal, for example, that there are no strong differences between organisations in different countries and regions, that interviews, facilitated meetings and prototyping are the most used elicitation techniques, that requirements are often documented textually, that traces between requirements and code or design documents are common, that requirements specifications themselves are rarely changed and that requirements engineering (process) improvement endeavours are mostly internally driven. Our study establishes a theory that can be used as starting point for many further studies for more detailed investigations. Practitioners can use the results as theory-supported guidance on selecting suitable RE methods and techniques.Item Open Access Mining valence, arousal, and dominance - possibilities for detecting burnout and productivity?(2016) Mäntylä, Mika; Adams, Bram; Destefanis, Giuseppe; Graziotin, Daniel; Ortu, MarcoSimilar to other industries, the software engineering domain is plagued by psychological diseases such as burnout, which lead developers to lose interest, exhibit lower activity and/or feel powerless. Prevention is essential for such diseases, which in turn requires early identification of symptoms. The emotional dimensions of Valence, Arousal and Dominance (VAD) are able to derive a person's interest (attraction), level of activation and perceived level of control for a particular situation from textual communication, such as emails. As an initial step towards identifying symptoms of productivity loss in software engineering, this paper explores the VAD metrics and their properties on 700,000 Jira issue reports containing over 2,000,000 comments, since issue reports keep track of a developer's progress on addressing bugs or new features. Using a general-purpose lexicon of 14,000 English words with known VAD scores, our results show that issue reports of different type (e.g., Feature Request vs. Bug) have a fair variation of Valence, while increase in issue priority (e.g., from Minor to Critical) typically increases Arousal. Furthermore, we show that as an issue's resolution time increases, so does the arousal of the individual the issue is assigned to. Finally, the resolution of an issue increases valence, especially for the issue Reporter and for quickly addressed issues. The existence of such relations between VAD and issue report activities shows promise that text mining in the future could offer an alternative way for work health assessment surveys.Item Open Access Interdisciplinary system courses - teaching agile systems engineering(2019) Seitz, Andreas; Avezum, Mariana; Bruegge, Bernd; Wagner, StefanWith the advent of technologies like the Internet of Things, Industry 4.0 and Cyber-Physical Systems, many software engineering courses turn into system engineering courses. Recent advances in technologies such as 3D printing and low-cost micro controllers enable to teach agile hard- and software co-design in system engineering courses. In this paper, we describe Interdisciplinary System Courses (ISC) - a teaching approach based on interdisciplinary projects, light-weight agile techniques and solving real problems by integrating industry customers. We describe our experiences from an exploratory case study where we applied ISC in a two-week international summer school with a customer from the aerospace industry. We derive a set of hypotheses on the effects of ISC.