Rate-Adapted Decentralized Learning Over Wireless Networks
Abstract : This paper proposes a communication strategy for decentralized learning in wireless systems that employs adaptive modulation and coding capability. The main objective of this work is to address a critical issue in decentralized learning based on the cooperative stochastic gradient descent (C-SGD) over wireless systems: the relationship between the transmission rate and the network density influences the runtime performance of learning. We first present that a dense network topology does not necessarily benefit the iteration performance of learning than a sparse one. However, it tends to degrade the runtime performance because the dense network topology requires a low-rate transmission. Based on these findings, a communication strategy is proposed in which each node optimizes its transmission rate to minimize communication time during the C-SGD under the constraints of network density. We perform numerical simulations of an image classification task under both independent and identically distributed (i.i.d.) and non-i.i.d. settings. The simulation results reveal that the preferred setting for the network density depends on the channel conditions and the biases in the training samples. Furthermore, numerical simulations of an automatic modulation classification task indicate that the preferred setting is almost the same even if the training task is different.
? Several key wireless communication setups call for coordination capabilities between otherwise interfering transmitters.
? Coordination or cooperation can be achieved at the expense of channel state information exchange.
? Intuition has it that such message should convey a combination of channel state information (CGI) and possibly some power control decision-related information, although the optimal form of the message is still elusive.
? Another question lies in what optimal power control decision should be taken at each node, based on locally available CGI and the exchanged messages.
? This problem has motivated many researchers to investigate machine learning techniques that utilize distributed computing resources, such as multiple graphical processing units in one computer, numerous servers in a data center, or smartphones distributed over a city.
? There is a critical problem that must be considered before realizing decentralized machine learning in wireless systems.
? An outcome from the first step can be easily obtained by solving a simple combinatorial optimization problem.
? These examples show that the impact of network density on the training loss of the C-SGD increases as the number of iterations K increases.
• We propose a deep learning framework for power control scheme in decentralized networks considering limited message exchange between agents.
• The performance of the proposed scheme is compared with conventional power control schemes in terms of sum rate.
• The proposed DNN-based scheme outperforms the centralized WMMSE scheme when it is allowed to exchange information.
• We have proposed the deep learning based continuous power control scheme in a decentralized network where the information sharing between nodes are enabled.
? In contrast, low-rate transmissions enable denser network topologies, meaning that the training performance versus the number of iterations can be improved; however, the runtime performance deteriorates because the total communication time increases.
? However, the model-parameter sharing process tends to be a bottleneck in terms of runtime performance because the number of model parameters that must be communicated is often large.
? These factors heavily deteriorate the runtime performance of decentralized machine learning.
? Based on the findings, we propose a novel communication strategy for improving the runtime performance of decentralized learning in wireless systems.
|