A Deep Reinforcement Learning Approach for Composing Moving IoT Services

Abstract : We develop a novel framework for efficiently and effectively discovering crowdsourced services that move in close proximity to a user over a period of time. We introduce a moving crowdsourced service model which is modelled as a moving region. We propose a deep reinforcement learning-based composition approach to select and compose moving IoT services considering quality parameters. Additionally, we develop a parallel flock-based service discovery algorithm as a ground-truth to measure the accuracy of the proposed approach. The experiments on two real-world datasets verify the effectiveness and efficiency of the deep reinforcement learning-based approach.

 EXISTING SYSTEM :

 ? In this work, we propose the evolution of an existing Edge-IoT architecture to a new improved version in which SDN/NFV are used over the Edge-IoT capabilities. ? We will make an analysis of the related work existing in the field of the Internet of Things and Industrial Internet of Things. ? This sublayer inherits the existing components of the previous GECA 1.0 Edge Layer. That is, it is formed by the Edge nodes or Edge Gateways. Thus, these nodes act as a gateway between the IoT devices and the applications in the Cloud (Business Solution Layer). ? This sublayer includes all the functionalities that previously existed in the Business Solution Layer in GECA 1.0.

 DISADVANTAGE :

 ? In this paper, a task scheduling problem is studied in the EC scenario, and multiple tasks are scheduled to virtual machines (VMs) configured at the edge server by maximizing the longterm task satisfaction degree (LTSD). ? The problem is formulated as a Markov decision process (MDP) for which the state, action, state transition, and reward are designed. ? Two special problems must be addressed for task scheduling in edge computing: time scheduling and resource allocation. ? In, the joint optimization problem of task allocation and the time scheduling problem were formulated as mixed-integer programming (MIP), and the logicbased Benders decomposition (LBBD) approach was proposed to maximize the number of admitted tasks.

 PROPOSED SYSTEM :

 • The proposed new architecture contemplates the use of Deep Reinforcement Learning techniques for the implementation of the SDN controller. • In this sense, it is proposed the application of a Deep Q-Learning model for the management of virtual data flows in SDN/NFV in an Edge-IoT architecture according to the required quality of service. • The Global Edge Computing Architecture and the new mechanism proposed to manage data flows in Software Defined Networks based on GECA. • The GECA architecture on which this work is based also followed the hierarchical model before the update proposed in this paper. • Deep Q-Networks is proposed to manage the Network Function Virtualization in Edge-IoT scenarios.

 ADVANTAGE :

 ? To better evaluate the performance of the proposed DRL-based task scheduling algorithm, the FCFS algorithm and the SJF algorithm were selected as the two benchmark methods. ? The performance comparison of the proposed DRL-based task scheduling algorithm and the baselines toward the task popularity. ? It demonstrates that our proposed algorithm has good convergence performance. ? A heuristic algorithm was proposed in to address the energy-efficient and delay-sensitive task scheduling in IoT edge computing. ? The outcome of the cross-entropy multiplied by the expected discounted cumulative reward is used as the loss function to optimize the policy network parameter ?.