Deep Reinforcement Learning Based Resource Management for DNN Inference in Industrial IoT

Abstract : Performing deep neural network (DNN) inference in real time requires excessive network resources, which poses a great challenge to the resource-limited industrial Internet of things (IIoT) networks. To address the challenge, in this paper, we introduce an end-edge-cloud orchestration architecture, in which the inference task assignment and DNN model placement are flexibly coordinated. Specifically, the DNN models, trained and pre-stored in the cloud, are properly placed at the end and edge to perform DNN inference. To achieve efficient DNN inference, a multi-dimensional resource management problem is formulated to maximize the average inference accuracy while satisfying the strict delay requirements of inference tasks. Due to the mix-integer decision variables, it is difficult to directly solve the formulated problem. Thus, we transform the formulated problem into a Markov decision process which can be solved efficiently. Furthermore, a deep reinforcement learning based resource management scheme is proposed to make real-time optimal resource allocation decisions. Simulation results are provided to demonstrate that the proposed scheme can efficiently allocate the available spectrum, caching, and computing resources, and improve average inference accuracy by 31.4% compared with the deep deterministic policy gradient benchmark.

 EXISTING SYSTEM :

 ? It will be of fundamental importance to strengthen the existing theoretical foundations of DL with repeated experimental data. ? We discuss the existing solutions to IoT networks that leverage ML and DL algorithms with an emphasis on the major aspects of the resource management in IoT networks. ? Support Vector Machine (SVM), naive Bayes classifier, Random Forest, and Decision Tree (DT) are most commonly used supervised learning algorithms used for classification and modelling of the existing data sets. ? Another method is continuous learning in which an automated system is developed that continuously evaluates and retrains the existing models.

 DISADVANTAGE :

 ? The resource management in wireless networks with massive Internet of Things (IoT) users is one of the most crucial issues for the advancement of fifth-generation networks. ? This framework contains a DRL based RM problem with multiple constraints, such as the number of users, channel gains, signal noise ratio (SNR) issues, and power consumption levels. ? Therefore, to solve the most complicated RM problems in IoT networks, this paper proposes a DRL method, in which the state, action, and reward are important parameters that should be designed to generate an optimal policy. ? However, most of them were limited to designing and analyzing the DRL-based method in fixed base stations for solving the joint resource allocation problems.

 PROPOSED SYSTEM :

 • The proposed algorithm uses DL to optimize the transmission of different packets of variable sizes through multiple channels so that the overall network efficiency is maximized. • Cognitive networks have been an area of interest since last decade and many solutions have been proposed to solve different resource management problems in cognitive networks. • To date, many solutions have been proposed for efficient resource allocation in IoT networks using optimization and heuristics-based techniques. • In, the authors proposed a noncooperative game-based resource allocation mechanism for LTE-based IoT networks.

 ADVANTAGE :

 ? The performance of the proposed method is analyzed through different measurement metrics. ? Therefore, the evaluation result shows that the performance accuracy is better in DRL evaluation for both clusters for multi-action-based RM optimization in a real and dynamic environment. ? Therefore, in terms of managing resources for IoT users in each cluster, the proposed framework contributes to better performance. ? Compared to the performance in similar scenarios, all the previous methods (i.e., DRL, Q-learning+D3QN) achieved lower system accuracy and RMSE evaluation. ? Therefore, our proposed method shows better performance in terms of accuracy and RMSE testing time evaluations.