Partitioning Stateful Data Stream Applications in Dynamic Edge Cloud Environments
ABSTARCT :
Computation partitioning is an important technique to improve the application performance by selectively offloading some computations from the mobile devices to the nearby edge cloud. In a dynamic environment in which the network bandwidth to the edge cloud may change frequently, the partitioning of the computation needs to be updated accordingly. The frequent updating of partitioning leads to high state migration cost between the mobile side and edge cloud. However, existing works don’t take the state migration overhead into consideration. Consequently, the partitioning decisions may cause significant network congestion and increase overall completion time tremendously.In this paper, with considering the state migration overhead, we propose a set of novel algorithms to update the partitioning based on the changing network bandwidth. To the best of our knowledge, this is the first work on computation partitioning for stateful data stream applications in dynamic environments. The algorithms aim to alleviate the network congestion and minimize the make-span through selectively migrating state in dynamic edge cloud environments. Extensive simulations show our solution not only could selectively migrate state but also outperforms other classical benchmark algorithms in terms of make-span. The proposed model and algorithms will enrich the scheduling theory for stateful tasks, which has not been explored before.
EXISTING SYSTEM :
? It analyses and classifies existing work on exploiting elasticity to adapt resource allocation to match the demands of stream processing services.
? Previous work has surveyed stream processing solutions without a focus on how resource elasticity is addressed .
? The present work provides a more indepth analysis of existing solutions and discusses how they attempt to achieve resource elasticity.
? we elaborate on how existing work tries to tackle aspects of resource elasticity for data stream processing.
DISADVANTAGE :
In this paper, we develop a set of efficient algorithms to solve the problem of partitioning stateful data stream applications, with the aim of alleviating network congestion and minimizing the make-span through selectively migrating state.
? In particular, we design a novel algorithm, namely Score Matrix-based Heuristic (SM-H), to solve the one-shot problem, which updates the partitioning of the current arriving data frame when the edge network environment changes.
? SM-H uses a matrix to record the benefit score of adjusting the execution position of each module, and then always select the module with the greatest score to adjust.
PROPOSED SYSTEM :
• Several stream processing frameworks and tools have been proposed for carrying out analytical tasks in a scalable and efficient manner.
• Many tools employ a dataflow approach where incoming data results in data streams that are redirected through a directed graph of operators placed on distributed hosts that execute algebra-like operations or user-defined functions.
• Some frameworks, on the other hand, discretise incoming data streams by temporarily storing arriving data during small time windows and then performing micro-batch processing whereby triggering distributed computations on the previously stored data.
ADVANTAGE :
? We have done a group of simulations to evaluate the effect of input parameters to the performance. In the simulation, we generate a stateful data stream application with 40 modules.
? We consider a mobile device and an edge cloud server. The average execution time of modules on the mobile device and on the edge cloud server is 2.4s and 1.6s respectively.
? The number of network channels is 3 and each channel has the bandwidth 2MBps.
? As a benchmark naive approach, Re-partitioning has the worst performance. LS has slightly better performance than Repartitioning due to it schedules the state migration.
|