ONLINE DEEP REINFORCEMENT LEARNING FOR COMPUTATION OFFLOADING IN BLOCKCHAIN-EMPOWERED MOBILE EDGE COMPUTING

Abstract : Offloading computation-intensive tasks (e.g., blockchain consensus processes and data processing tasks) to the edge/cloud is a promising solution for blockchain-empowered mobile edge computing. However, the traditional offloading approaches (e.g., auction-based and game-theory approaches) fail to adjust the policy according to the changing environment and cannot achieve long-term performance. Moreover, the existing deep reinforcement learning-based offloading approaches suffer from the slow convergence caused by high-dimensional action space. In this paper, we propose a new model-free deep reinforcement learning-based online computation offloading approach for blockchain-empowered mobile edge computing in which both mining tasks and data processing tasks are considered. First, we formulate the online offloading problem as a Markov decision process by considering both the blockchain mining tasks and data processing tasks. Then, to maximize longterm offloading performance, we leverage deep reinforcement learning to accommodate highly dynamic environments and address the computational complexity. Furthermore, we introduce an adaptive genetic algorithm into the exploration of deep reinforcement learning to effectively avoid useless exploration and speed up the convergence without reducing performance. Lastly, our experimental results demonstrate that our algorithm can converge quickly and outperform three benchmark policies.

 EXISTING SYSTEM :

 ? Most of these existing works show how RL has been effectively applied to MEC based networks due to network dynamics. ? Random task generation and arrival exist for a mobile device since independent tasks are possibly sensed, collected, and generated by multiple types of applications stochastically. ? The existing DRL algorithms in terms of convergence, stability, and robustness, proposed a multiagent DRL-based cooperative computation offloading policy in the NOMA enabled MEC system with the aid of expert strategies, scatter networks, and hierarchical agents.

 DISADVANTAGE :

 ? The slow convergence caused by high-dimensional action space remains an urgent and challenging problem. ? In this paper, we study the computation offloading problem for both mining tasks and data processing tasks in multi-hop multi-user blockchain-empowered MEC. ? In our performance optimization problem, it is important to note that costs such as energy consumption and purchasing VMs should be paid immediately.

 PROPOSED SYSTEM :

 • To minimize the expected long-term cost of task delay and dropping penalty, the optimal offloading decision making strategy was proposed in with the aid of long short-term memory (LSTM), dueling DQN, and double DQN algorithms. • In, a low-complexity DRL-based task execution time and energy consumption optimization is proposed to avoid frequent offloading decisions and resource allocation recalculations, considering time-varying wireless channels and dynamic edge computational resources. • A context-aware attention mechanism is proposed for the online weight adjustment of each action value.

 ADVANTAGE :

 ? The performance of the algorithm is greatly affected by the convergence and accuracy of the deep Q-network (DQN). ? In this way, we can make the most of the evaluating role of critic network in exploration, avoiding useless exploration to a great extent without reducing the performance. ? A resource allocation algorithm based on deep Qlearning is proposed into optimize the performance of computation offloading in MEC. ? All these algorithms are constrained by the trade-off between efficiency and optimality.