Orchestrating data governance workloads as stateful services in cloud environments using Kubernetes Operator Framework

Abstract

Data is becoming the core corporate asset that will determine the business’s success. As a result, it is critical for governing enterprise data. Previously, the Enterprise Content Management (ECM) system was employed by many companies to manage and process their enterprise data, which is a monolithic data governance application. As the ECM system is typically deployed on bare metal or at most in a virtualized IT infrastructure, it lacks the flexibility to support Continuous Integration (CI) and Continuous Delivery (CD) cost-effectively. Cloud computing has gained popularity as a powerful platform for application deployment, owing to perceived benefits such as elasticity to fluctuating load and reduced operational costs as compared to running in traditional data centers. Therefore, it is promising to migrate the legacy ECM system into the cloud. The goal of this thesis is to orchestrate stateful database workloads in Kubernetes that are typical for ECM systems. For our concept verification, we included a comparison and analysis between traditional and comparable cloud native Relational Database Management System (RDBMS) using IBM DB2, PostgreSQL, CockRoachDB and Google Spanner. We proposed an implementation of the Monitor-Analyze-Plan-Execute (MAPE) concept using the Kubernetes Operator framework. With our prototype implementation, we proved that the Kubernetes operator is able to deploy a cluster for DB2 consisting of a read/write primary and up to three read-only members. Various experiments carried out on the prototype have evidenced its High Availability (HA), Disaster Recovery (DR) features as well as read scalability.