DTC A Dynamic Transaction Chopping Technique for Geo-Replicated Storage Services

Abstract : Replicating data across geo-distributed datacenters is usually necessary for large scale cloud services to achieve high locality, durability and availability. One of the major challenges in such geo-replicated data services lies in consistency maintenance, which usually suffers from long latency due to costly coordination across datacenters. Among others, transaction chopping is an effective and efficient approach to address this challenge. However, existing chopping is conducted statically during programming, which is stubborn and complex for developers. In this paper, we propose Dynamic Transaction Chopping (DTC), a novel technique that does transaction chopping and determines piecewise execution in a dynamic and automatic way. DTC mainly consists of two parts: a dynamic chopper to dynamically divide transactions into pieces according to the data partition scheme, and a conflict detection algorithm to check the safety of the dynamic chopping. Compared with existing techniques, DTC has several advantages: transparency to programmers, flexibility in conflict analysis, high degree of piecewise execution, and adaptability to data partition schemes. A prototype of DTC is implemented to verify the correctness of DTC and evaluate its performance. The experiment results show that our DTC technique can achieve much better performance than similar work.

 EXISTING SYSTEM :

 ? We address these challenges with a decentralized data popularity measurement scheme,which leverages existing state-of-the-art storage system architecture to identify hot data cluster-wide dynamically. ? However, existing approaches to dynamic replication for such mutable data remain centralized, thus incompatible with these systems. ? In this paper, we demonstrate that it is possible to integrate dynamic replication with the existing architecture of these storage systems, which enables us to leverage their existing, built-in algorithms to efficiently handle read and writes in geo-distributed environments. ? Determining when to create a new dynamic replica or delete an existing one based on the previous information is also challenging in a distributed setup.

 DISADVANTAGE :

 ? Without transactions, an application must carefully coordinate access to data to avoid race conditions, partial writes, overwrites, and other hard problems that cause erratic behavior. ? The developer can deal with this problem by logging and replaying actions which amounts to implementing rudimentary transactions—or garbage collecting dangling structures. ? Such failure is problematic because there may be committed transactions at s that were not yet replicated at other sites. ? A simple solution to this problem is to mark objects that caused the abort of slow commit and briefly delay access to them in subsequent fast commits: this delay would allow the next attempt of slow commit to succeed.

 PROPOSED SYSTEM :

 • The proposed solutions in the literature leverage the centralized metadata management of certain storage systems such as HDFS or GFS to allow the clients to locate the closest available replica of the data they want to access. • We acknowledge that using client votes has been proposed before in the context of replicated relational databases, specifically to ensure data consistency. • Dynamic replication proposes to solve this issue by dynamically replicating hot data as close as possible to the applications that access it. • Inspired by the P2P systems, proposes an adaptive decentralized file replication algorithm that achieves high query efficiency and high replica utilization at a significantly low cost.

 ADVANTAGE :

 ? An attractive storage choice for this setting is a key-value store, which provides good performance and reliability at low cost. ? We use Walter to build two applications and demonstrate the usefulness of its transactional guarantees. Our experience indicates that Walter transactions simplify application development and provide good performance. ? Transactions allow the data structure manipulations built into Redis to be implemented by the application, while providing competitive performance. ? In the mixed workload, performance is mostly determined by how many operations a transaction issues on average. ? The specification code is centralized to make it as simple as possible, whereas an implementation can be distributed, complex, and more efficient.