Multi-Objective Optimization for UAV-Assisted Wireless Powered IoT Networks Based on Extended DDPG Algorithm

Abstract : This paper studies an unmanned aerial vehicle (UAV)-assisted wireless powered IoT network, where a rotary-wing UAV adopts fly-hover-communicate protocol to successively visit IoT devices in demand. During the hovering periods, the UAV works on full-duplex mode to simultaneously collect data from the target device and charge other devices within its coverage. Practical propulsion power consumption model and non-linear energy harvesting model are taken into account. We formulate a multi-objective optimization problem to jointly optimize three objectives: maximization of sum data rate, maximization of total harvested energy and minimization of UAV’s energy consumption over a particular mission period. These three objectives are in conflict with each other partly and weight parameters are given to describe associated importance. Since IoT devices keep gathering information from the physical surrounding environment and their requirements to upload data change dynamically, online path planning of the UAV is required. In this paper, we apply deep reinforcement learning algorithm to achieve online decision. An extended deep deterministic policy gradient (DDPG) algorithm is proposed to learn control policies of UAV over multiple objectives. While training, the agent learns to produce optimal policies under given weights conditions on the basis of achieving timely data collection according to the requirement priority and avoiding devices’ data overflow. The verification results show that the proposed MODDPG (multi-objective DDPG) algorithm achieves joint optimization of three objectives and optimal policies can be adjusted according to weight parameters among optimization objectives.

 EXISTING SYSTEM :

 ? In, a battery charging policy and interference mitigation scheme were designed for use in UAV-aided wireless power IoT networks, in which a machine learning framework based on echo state networks was exploited to predict the energy consumption of nodes. ? However, a systematic study of UAV-enabled SWIPT for IoT networks is missing in the aforementioned research works. ? To tackle this problem, a joint 3D placement, beam pattern and power allocation scheme for the UAV can be properly designed. ? The energy-limited IoT devices may have communication requirement for the above considered scenario, such as uploading the collected information. ? However, the robustness of communication can not be guaranteed when the number of IoT devices increases.

 DISADVANTAGE :

 ? A deep reinforcement learning (DRL) approach is proposed for solving the continuous optimisation problem with time-varying channels in a centralised fashion. ? The EE problem is formulated for the downlink channel with the power restrictions and the RIS’s requirement. ? Through the numerical results, we demonstrate that our proposed methods efficiently solve the joint optimisation problem with the dynamic environmental setting and time-varying CSI and outperform the other benchmarks. ? We present the system model and problem formulation for the energy-efficient multi-UAV-assisted wireless communications with the support of the RIS.

 PROPOSED SYSTEM :

 • A deep deterministic policy gradient (DDPG)-based algorithm is proposed to achieve the correct prediction of the service requirement of devices (i.e., data transmission and battery charging) and, then, a dynamic path planning scheme is designed to maximize the energy efficiency (EE) of the system. • In, partial and binary computation offloading modes were proposed for investigating the computation rate maximization problem in a UAV-enabled mobile-edge computing (MEC) wireless-powered system. • A DDPG-based method is proposed to predict the service requirements of devices and, then, a dynamic path planning scheme is established.

 ADVANTAGE :

 ? To optimise the EE network performance, we propose a centralised DRL technique for jointly solving the power allocation at the UAVs and phase-shift matrix of the RIS. ? Owing to the intrinsic features of RIS and UAVs, the RISassisted UAV communications have been recently considered for enhancing network performance. ? In, the power allocation and the phase-shift optimisation algorithm was proposed for maximising the EE performance. ? By utilising both advantages of the UAV and the RIS, the network performance are significantly improved in terms of enhancing the received signal and mitigating the interference. ? To improve the network performance, we introduce the proximal policy optimisation (PPO) algorithm with a better sampling technique.