Browsing by Author "Zahariev, Denis"

Now showing 1 - 2 of 2

Open Access
Controlled user study : usability and efficiency evaluation of the parallel performance catalogue extension for the Palladio-Bench
(2020) Zahariev, Denis
For the last two decades, the number of cores in modern CPUs has been steadily increasing. This enables a significant leap in the performance of modern software when the right parallel programming approaches and strategies are being used. One tool used by software performance engineers to examine and evaluate the performance and reliability of pieces of software is the Palladio-Bench. This tool allows its users to analyse these and many more Quality of Software (QoS) properties such as sizing, scalability, and load balancing, based only on graphical models of the software architecture. After various pieces of research showed that the Palladio-Bench does not fully support parallelism, and the modelling of parallel programming strategies, a new extension for the tool was developed. This new extension for the Palladio-Bench incorporates fundamental parallel programming approaches into its already existing toolkit. The researchers that proposed the extension also claimed that it has higher usability and better time efficiency than the standard modelling toolkit of the Palladio-Bench. However, they were not able to prove it since the extension was not yet developed. The purpose of this thesis is to put the supposed usability gains to the test. It compares the standard toolkit and the new extension in the context of the modelling of parallel behaviours. To support this study, a set of research questions was defined. The chosen research method was the conduction of a controlled empirical user study. Sixteen participants were recruited and split into two groups. Each group had to complete different modelling tasks with the standard toolkit and the extension. While they were working on the tasks, several metrics were recorded: task completion time, time spent in errors, number of errors, and usability evaluation. Afterwards, this data was statistically analysed and tested. The results of the analysis prove that the extension increases the usability and the time efficiency of the Palladio-Bench. A reduction in the time spent in errors and the number of errors, however, could not be proved.
Open Access
Supporting and verifying transient behavior specifications in chaos engineering
(2022) Zahariev, Denis
Context: Chaos Engineering is an approach for investigating the resilience of software systems, i.e. their ability to withstand unexpected events, adapt accordingly, and continue providing functionality. An integral part of the approach is continuous experimentation, expressed in continuously executing so-called Chaos Experiments. A Chaos Experiment consists of two crucial elements, namely a steady-state hypothesis and an anomaly injection. Traditionally, during the experimentation process, the steady-state hypothesis is verified at the start of an experiment, after which the anomaly is injected, followed by a second evaluation of the steady-state hypothesis. This Chaos Experimentation process is well suited for identifying whether a system is in a steady state after a failure is introduced. Problem: When applied, the traditional Chaos Experimentation approach can only verify whether the system is in a steady state without providing any information about the time between the state changes, e.g. the recovery of the system. The experimentation process does not explicitly allow the specification of hypotheses regarding the transient behavior, i.e. the behaviors experienced during the transition between steady states after a failure has been introduced. As a result, the experimentation process also does not support explicit verification of requirements on the transient behavior during the experiment, e.g, whether the response time stays below a given threshold all the time or whether a circuit breaker opens within a given time. Knowledge about such transient behaviors is beneficial for the stakeholders of the system. For example, assuming that high availability is of utmost importance for the business model of an application, a long period of recovery during which the application is unavailable after an unexpected failure could lead to considerable losses for various stakeholders. Objective: The first objective of the thesis is to examine how a transient behavior requirement can be specified in the context of Chaos Engineering and Chaos Experiments. The next goal of the thesis is to study how the Chaos Experimentation process can include a verification of transient behavior requirements. A further goal is to create a concept for an extended Chaos Engineering approach, which supports the specification of transient behavior hypotheses and their verification, and to implement the concept into a working prototype. After the prototype is developed, the last objective of the thesis is to evaluate the extended Chaos Engineering approach and its implementation. Method: In order to achieve the first objective of the thesis, formalisms capable of describing transient behaviors are examined with regard to their integration into Chaos Experiments. To accomplish the second objective, state-of-the-art Chaos Engineering tools are studied, and additionally, three expert interviews are conducted in order to elicit the requirements for such an extension of the Chaos Engineering approach. To accomplish the goal of creating a concept for the approach, the research results from the previous goals and the requirements elicited during the interviews are combined into a concept which is then implemented into a prototype. To reach the last goal of the thesis, the approach and the prototype are examined in three separate types of evaluation. Result: First, the results of the thesis include the requirements for an extended Chaos Engineering approach, supporting the specification and verification of transient behavior requirements. Furthermore, the results also include a concept for the realization of the extension. Moreover, the results also comprise a functioning prototype of the proposed approach and its evaluation. Conclusion: The proposed extension of the Chaos Engineering approach allows a deeper and more precise analysis of the resilience of software systems by enabling the specification and evaluation of more detailed and strict resilience requirements including transient behavior specifications. Furthermore, various stakeholders such as customers, operators, and developers benefit from the extended approach, allowing them to have stricter guarantees regarding the resilience of the application. Moreover, the prototype implementing the extended approach allows software engineers to easily adopt it and possibly extend it.