Adding value to object storage: integrating analytics with cloud storage back ends

Abstract

With the vast interest of customers in using the cloud infrastructure, cloud providers are going beyond limits to offer advanced functionalities. They try their utmost best to present the services in a way that makes the customers highly attracted and convince them about value and benefits of using such services. For this purpose, cloud providers need to have an access to customers’ data, hence customer-sensitive data stored in repositories should be transferred to the cloud. Object storages are one of the possible solutions for the implementation of repositories in cloud environments. However, due to the data being confidential and fragile, security and encryption mechanisms are required. The application of Enterprise Content Management (ECM) system highly relies on metadata, thus there is a need to keep metadata unencrypted while encrypting data itself. Therefore, cloud providers that are hosting ECM systems are forced to keep metadata unencrypted in order to satisfy the main functionalities of ECM systems on the cloud. Although other cloud providers can offer data encryption and unencrypted metadata as an option to their customers. This leads to the conclusion that enhancing object storages with analysis capabilities in ECM systems is more beneficial if it is done on top of unencrypted metadata. In this thesis I investigate how value can be added to such cloud storage services by only using access the metadata. I specifically focus on providing analytics functionality on metadata. This Master’s thesis aims at providing the means to efficiently analyze the metadata inside a cloud-based ECM system (OSECM) which uses Swift Object Store as its back end repository. I extended the OSECM system with required components by providing new modules that enable the retrieval of metadata from the object storage and the insertion of this metadata into a metadata warehouse. The importance of metadata replication in a distinct data warehouse offers the possibility of benefiting from SQL query capabilities for analysis purposes. Furthermore, an existing tool was integrated as the analysis component to offer the means for interaction with the underlying metadata warehouse and the user interface. Finally, after applying analysis queries, the results are presented on the user interface using the predefined set of visualization interfaces. The supported data structure for the visualization of the result are also defined in this work.