Middleware for Data Management in Multi-Cloud

Cloud storage is an essential service model of cloud computing that involves outsourcing the deployment, provisioning, and management of storage resources to external third-party providers. Additionally, multi-cloud (federated cloud) storage setups integrate and utilize storage resources, technologies, and services from multiple cloud storage providers. This approach has become increasingly popular and attractive for a wide range of cloud-based applications and services such as Software-as-a-Service (SaaS) and Internet-of-Things (IoT) applications.

However, this increased popularity also introduces substantial challenges for service providers, in particular, with respect to the complexity of managing a federated cloud storage setup. As such, service providers have to deal with the problem of lack of standardization (e.g., in terms of heterogeneous technologies, APIs, and data models). Furthermore, the extent of heterogeneity in cloud storage providers (different SLA guarantees, trust models) and the need to address different storage- and privacy-related requirements of the application lead inevitably to the problem of exploding implementation complexity.

Moreover, considering the limited trust in external third-party cloud storage providers, relying on traditional data protection strategies for providing secure data management services leads to significant performance and scalability degradation. Finally, the run-time dynamicity in cloud storage providers (e.g., in terms of performance characteristics, availability, offered SLAs, etc) contributes significantly to the problem of exploding management complexity as it demands continuous supervision and a sequence of manual actions to be performed by a human operator. Additionally, obliviousness to run-time dynamicity inherent in cloud providers (e.g., performance fluctuation, cost, availability, etc) may result in both sub-optimal data management decisions and costly SLA violations.

This dissertation focuses on the above-mentioned challenges through targeted contributions at the middleware level. As such, it involves the design of an effective middleware framework for coping with these key challenges in order to facilitate the wider adoption of a federated cloud storage setup. In particular, special attention is paid to aspects of performance overhead on the application and the required development, management, and integration cost.

In this regard, this dissertation makes five complementary contributions: (i) it provides a comprehensive trade-off analysis study between the performance impact and the application portability of existing Object-NoSQL Datastore Mapper (ONDM) frameworks, (ii) it describes an overarching and coherent middleware framework for federated cloud data management, (iii) it presents PERSIST, a middleware for data management in federated cloud storage setup, (iv) it introduces a scalable and reusable data protection strategy for secure data management, and finally (v) it presents SCOPE, a self-adaptive and autonomic middleware for SLA-aware data management.

These contributions have been validated and extensively evaluated in the context of two distinct industrial SaaS applications: a log management and a document processing service offering. We have performed a thorough evaluation of our contributions to assess the benefits in terms of reduced development and management effort and also to quantify the impact in terms of introduced performance overhead.