Evaluating the maintainability of variability concepts in cloud deployment technologies

Abstract

Context. Deployment (Automation) Technologies (DTs) allow automating the provisioning and management of cloud deployments. It is often required to use multiple technologies concurrently and define multiple variations of a deployment model to satisfy various user requirements such as cost or elasticity. Therefore, variability must be managed across multiple DTs. Problem. Managing variability across multiple DTs can lead to increased complexity, as DTs support different variability mechanisms of varying expressiveness. The Variable Deployment Metamodel (VDMM) introduces variability management concepts on top of the DT-agnostic Essential Deployment Metamodel (EDMM), which does not support variability. Thus, VDMM proposes improving maintainability by providing a single deployment model supporting variability. However, the effectiveness of VDMM in terms of these improvements has yet to be validated compared to other DTs. Objective. In this thesis, we identify and classify different variability concepts in DTs. This classification serves as a tool for understanding deployment variability’s implications on maintainability, supporting the primary objective of this thesis: to evaluate and compare the maintainability of variability concepts in VDMM and other popular DTs. Method. We derive a classification framework for variability concepts in cloud DTs from literature research and the analysis of the official documentation of DTs. Based on the Goal Question Metric Approach (GQM), we evaluate the maintainability of variability concepts in DTs by conducting a case study and measuring and evaluating the defined metrics. Result. Our classification framework contains three dimensions: (i) Variability Concepts define high-level concepts enabling variability in the deployment model, (ii) Variability Implementations are mechanisms used by DTs to implement these concepts, and (iii) Variability Properties define the properties of the implementations. The case study shows that DTs using General-Purpose Programming Languages (GPLs), such as Pulumi, are the most maintainable technologies supporting variability in the deployment model for our scenario. Furthermore, using the VDMMs internal pruning algorithm requires less maintenance work when implementing architectural changes than technologies such as Ansible, Terraform, and OpenStack Heat, which do not provide such algorithms. Conclusion. The classification framework aids the case study evaluation in understanding why variability in some DTs may be more maintainable than in others. The evaluation provides a set of key findings concerning the maintainability of variability concepts, such as the most efficient DTs and variability implementations for implementing architectural changes in our scenario. However, the evaluation also shows that the maintainability of variability depends on the use case and complexity of the implemented changes; for example, using expressions to implement configuration changes requires less work in our scenario.