Machine-Learning Beam Tracking and Weight Optimization for mmWave Multi-UAV Links
Abstract : Millimeter-wave (mmWave) hybrid analog-digital beamforming is a promising approach to satisfy the low-latency constraint in multiple unmanned aerial vehicles (UAVs) systems, which serve as network infrastructure for flexible deployment. However, in highly dynamic multi-UAV environments, analog beam tracking becomes a critical challenge. The overhead of additional pilot transmission at the price of spectral efficiency is shown necessary to achieve high resilience in operation. An efficient method to deal with high dynamics of UAVs applies machine learning, particularly Q-learning, to analog beam tracking. The proposed Q-learning-based beam tracking scheme uses current/past observations to design rewards from environments to facilitate prediction, which significantly increases the efficiency of data transmission and beam switching.
Given the selected analog beams, the goal of digital beamforming is to maximize the SINR. The received pilot signals are utilized to approximate the desired signal and interference power values, which yield the SINR measurements as well as the optimal digital weights. Since the selected analog beams based on the received power do not guarantee the hybrid beamforming achieving the maximization SINR, we therefore reserve additional analog beams as candidates during the beam tracking. When the candidates include the ideal beams, the combination of analog beams with their digital weights achieving the maximum SINR consequently provides the optimal solution to the hybrid beamforming.
? We identify and empirically characterize the significant effect of the UAV airframe and the sub-optimal beam selection problem in existing standards using 802.11ad complaint Terragraph radios, in. In addition, we quantitatively relate these two effects, as well as beam misalignment caused by hovering with, (i) additional fading and (ii) resulting power fluctuations over time, by comparing collected data from static and UAV-to-Ground links.
? (2) We build and experimentally validate the first stochastic analytical UAV-to-Ground channel model that takes a systems-approach to estimate additional fading in mmWave links, complementing existing models, in Section 4.
One of the key performance indicators for dynamic beam tracking could be network resilience.
? In dynamic environments, the UAVs may have to switch the analog beams rapidly in order to stably provide the acceptable link quality.
? Given codebooks that consist of candidates for the analog beams, the work in presented a gradient-based algorithm to find a better beam next to the currently used beam, and in, the beam tracking problem is formulated as a multi-armed bandit problem.
? One can also use the extended Kalman filter to recursively track the beams based on the estimated angles of departure and arrival (AoDs/AoAs)
Connectivity-constrained trajectory optimization for UAVmounted relays and UAV-mounted base stations in cellular networks has been widely investigated in. In, a minimal delay trajectory design for UAV relays was proposed to ferry data from multiple sources to destination using reinforcement learning (RL) algorithm.
we evaluate the performance of the proposed DQN algorithm for our path planning problem. We compare our learning-based joint path planning and beam tracking method against learning-based path planning with heuristic exhaustive beam searching method
? In terms of high-resilience demand for the multi-UAV system, the numerical results of the proposed method show that balancing the exploration and exploitation can outperform the one using exploration only.
? Although Qlearning needs some space and efforts to record the experience in the Q-table, it makes actions depending on not only current observations but also the experience and rewards so that the performance is not completely dominated by the current observations, while the gradient-based method totally relies on them.
|