05 Fakultät Informatik, Elektrotechnik und Informationstechnik
Permanent URI for this collectionhttps://elib.uni-stuttgart.de/handle/11682/6
Browse
13 results
Search Results
Item Open Access Empirical research plan: effects of sketching on program comprehension(2016) Baltes, Sebastian; Wagner, StefanSketching is an important means of communication in software engineering practice. Yet, there is little research investigating the use of sketches. We want to contribute a better understanding of sketching, in particular its use during program comprehension. We propose a controlled experiment to investigate the effectiveness and efficiency of program comprehension with the support of sketches as well as what sketches are used in what way.Item Open Access A comprehensive safety engineering approach for software-intensive systems based on STPA(2015) Abdulkhaleq, Asim; Wagner, Stefan; Leveson, NancyFormal verification and testing are complementary approaches which are used in the development process to verify the functional correctness of software. However, the correctness of software cannot ensure the safe operation of safety-critical software systems. The software must be verified against its safety requirements which are identified by safety analysis, to ensure that potential hazardous causes cannot occur. The complexity of software makes defining appropriate software safety requirements with traditional safety analysis techniques difficult. STPA (Systems-Theoretic Processes Analysis) is a unique safety analysis approach that has been developed to identify system hazards, including the software-related hazards. This paper presents a comprehensive safety engineering approach based on STPA, including software testing and model checking approaches for the purpose of developing safe software. The proposed approach can be embedded within a defined software engineering process or applied to existing software systems, allow software and safety engineers integrate the analysis of software risks with their verification. The application of the proposed approach is illustrated with an automotive software controller.Item Open Access On the impact of service-oriented patterns on software evolvability: a controlled experiment and metric-based analysis(2019) Bogner, Justus; Wagner, Stefan; Zimmermann, AlfredBackground: Design patterns are supposed to improve various quality attributes of software systems. However, there is controversial quantitative evidence of this impact. Especially for younger paradigms such as service- and Microservice-based systems, there is a lack of empirical studies. Objective: In this study, we focused on the effect of four service-based patterns - namely Process Abstraction, Service Façade, Decomposed Capability, and Event-Driven Messaging - on the evolvability of a system from the viewpoint of inexperienced developers. Method: We conducted a controlled experiment with Bachelor students (N = 69). Two functionally equivalent versions of a service-based web shop - one with patterns (treatment group), one without (control group) - had to be changed and extended in three tasks. We measured evolvability by the effectiveness and efficiency of the participants in these tasks. Additionally, we compared both system versions with nine structural maintainability metrics for size, granularity, complexity, cohesion, and coupling. Results: Both experiment groups were able to complete a similar number of tasks within the allowed 90 min. Median effectiveness was 1/3. Mean efficiency was 12% higher in the treatment group, but this difference was not statistically significant. Only for the third task, we found statistical support for accepting the alternative hypothesis that the pattern version led to higher efficiency. In the metric analysis, the pattern version had worse measurements for size and granularity while simultaneously having slightly better values for coupling metrics. Complexity and cohesion were not impacted. Interpretation: For the experiment, our analysis suggests that the difference in efficiency is stronger with more experienced participants and increased from task to task. With respect to the metrics, the patterns introduce additional volume in the system, but also seem to decrease coupling in some areas. Conclusions: Overall, there was no clear evidence for a decisive positive effect of using service-based patterns, neither for the student experiment nor for the metric analysis. This effect might only be visible in an experiment setting with higher initial effort to understand the system or with more experienced developers.Item Open Access The Quamoco product quality modelling and assessment approach(2012) Wagner, Stefan; Lochmann, Klaus; Heinemann, Lars; Kläs, Michael; Trendowicz, Adam; Plösch, Reinhold; Seidl, Andreas; Goeb, Andreas; Streit, JonathanPublished software quality models either provide abstract quality attributes or concrete quality assessments. There are no models that seamlessly integrate both aspects. In the project Quamoco, we built a comprehensive approach with the aim to close this gap. For this, we developed in several iterations a meta quality model specifying general concepts, a quality base model covering the most important quality factors and a quality assessment approach. The meta model introduces the new concept of a product factor, which bridges the gap between concrete measurements and abstract quality aspects. Product factors have measures and instruments to operationalise quality by measurements from manual inspection and tool analysis. The base model uses the ISO 25010 quality attributes, which we refine by 200 factors and 600 measures for Java and C# systems. We found in several empirical validations that the assessment results fit to the expectations of experts for the corresponding systems. The empirical analyses also showed that several of the correlations are statistically significant and that the maintainability part of the base model has the highest correlation, which fits to the fact that this part is the most comprehensive. Although we still see room for extending and improving the base model, it shows a high correspondence with expert opinions and hence is able to form the basis for repeatable and understandable quality assessments in practice.Item Open Access Interdisciplinary system courses - teaching agile systems engineering(2019) Seitz, Andreas; Avezum, Mariana; Bruegge, Bernd; Wagner, StefanWith the advent of technologies like the Internet of Things, Industry 4.0 and Cyber-Physical Systems, many software engineering courses turn into system engineering courses. Recent advances in technologies such as 3D printing and low-cost micro controllers enable to teach agile hard- and software co-design in system engineering courses. In this paper, we describe Interdisciplinary System Courses (ISC) - a teaching approach based on interdisciplinary projects, light-weight agile techniques and solving real problems by integrating industry customers. We describe our experiences from an exploratory case study where we applied ISC in a two-week international summer school with a customer from the aerospace industry. We derive a set of hypotheses on the effects of ISC.Item Open Access An industrial case study on the evaluation of a safety engineering approach for software-intensive systems in the automotive domain(2016) Abdulkhaleq, Asim; Vöst, Sebastian; Wagner, Stefan; Thomas, JohnSafety remains one of the essential and vital aspects in today's automotive systems. These systems, however, become ever more complex and dependent on software which is responsible for most of their critical functions. Therefore, the software components need to be analysed and verified appropriately in the context of software safety. The complexity of software systems makes defining software safety requirements with traditional safety analysis techniques difficult. A new technique called STPA (Systems-Theoretic Process Analysis) based on system and control theory has been developed by Leveson to cope with complex systems. Based on STPA, we have developed a comprehensive software safety engineering approach in which the software and safety engineers integrate the analysis of software risks with their verification to recognize the software-related hazards and reduce the risks to a low level. In this paper, we explore and evaluate the application of our approach to a real industrial system in the automotive domain. The case study was conducted analysing the software controller of the Active Cruise Control System (ACC) of the BMW Group.Item Open Access Naming the pain in requirements engineering: design of a global family of surveys and first results from Germany(2013) Méndez Fernández, Daniel; Wagner, StefanContext: For many years, we have observed industry struggling in defining a high quality requirements engineering (RE) and researchers trying to understand industrial expectations and problems. Although we are investigating the discipline with a plethora of empirical studies, those studies either concentrate on validating specific methods or on single companies or countries. Therefore, they allow only for limited empirical generalisations. Objective: To lay an empirical and generalisable foundation about the state of the practice in RE, we aim at a series of open and reproducible surveys that allow us to steer future research in a problem-driven manner. Method: We designed a globally distributed family of surveys in joint collaborations with different researchers from different countries. The instrument is based on an initial theory inferred from available studies. As a long-term goal, the survey will be regularly replicated to manifest a clear understanding on the status quo and practical needs in RE. In this paper, we present the design of the family of surveys and first results of its start in Germany. Results: Our first results contain responses from 30 German companies. The results are not yet generalisable, but already indicate several trends and problems. For instance, a commonly stated problem respondents see in their company standards are artefacts being underrepresented, and important problems they experience in their projects are incomplete and inconsistent requirements. Conclusion: The results suggest that the survey design and instrument are well-suited to be replicated and, thereby, to create a generalisable empirical basis of RE in practice.Item Open Access Towards guidelines for preventing critical requirements engineering problems(2016) Mafra, Priscilla; Kalinowski, Marcos; Méndez Ferández, Daniel; Felderer, Michael; Wagner, Stefan[Context] Problems in Requirements Engineering (RE) can lead to serious consequences during the software development lifecycle. [Goal] The goal of this paper is to propose empirically-based guidelines that can be used by different types of organisations according to their size (small, medium or large) and process model (agile or plan-driven) to help them in preventing such problems. [Method] We analysed data from a survey on RE problems answered by 228 organisations in 10 different countries. [Results] We identified the most critical RE problems, their causes and mitigation actions, organizing this information by clusters of size and process model. Finally, we analysed the causes and mitigation actions of the critical problems of each cluster to get further insights into how to prevent them. [Conclusions] Based on our results, we suggest preliminary guidelines for preventing critical RE problems in response to context characteristics of the companies.Item Open Access Will my tests tell me if I break this code?(2016) Niedermayr, Rainer; Juergens, Elmar; Wagner, StefanAutomated tests play an important role in software evolution because they can rapidly detect faults introduced during changes. In practice, code-coverage metrics are often used as criteria to evaluate the effectiveness of test suites with focus on regression faults. However, code coverage only expresses which portion of a system has been executed by tests, but not how effective the tests actually are in detecting regression faults. Our goal was to evaluate the validity of code coverage as a measure for test effectiveness. To do so, we conducted an empirical study in which we applied an extreme mutation testing approach to analyze the tests of open-source projects written in Java. We assessed the ratio of pseudo-tested methods (those tested in a way such that faults would not be detected) to all covered methods and judged their impact on the software project. The results show that the ratio of pseudo-tested methods is acceptable for unit tests but not for system tests (that execute large portions of the whole system). Therefore, we conclude that the coverage metric is only a valid effectiveness indicator for unit tests.Item Open Access Is the stack distance between test case and method correlated with test effectiveness?(2019) Niedermayr, Rainer; Wagner, StefanMutation testing is a means to assess the effectiveness of a test suite and its outcome is considered more meaningful than code coverage metrics. However, despite several optimizations, mutation testing requires a significant computational effort and has not been widely adopted in industry. Therefore, we study in this paper whether test effectiveness can be approximated using a more light-weight approach. We hypothesize that a test case is more likely to detect faults in methods that are close to the test case on the call stack than in methods that the test case accesses indirectly through many other methods. Based on this hypothesis, we propose the minimal stack distance between test case and method as a new test measure, which expresses how close any test case comes to a given method, and study its correlation with test effectiveness. We conducted an empirical study with 21 open-source projects, which comprise in total 1.8 million LOC, and show that a correlation exists between stack distance and test effectiveness. The correlation reaches a strength up to 0.58. We further show that a classifier using the minimal stack distance along with additional easily computable measures can predict the mutation testing result of a method with 92.9% precision and 93.4% recall. Hence, such a classifier can be taken into consideration as a light-weight alternative to mutation testing or as a preceding, less costly step to that.