Traffic Steering for Cellular-Enabled UAVs: A Federated Deep Reinforcement Learning Approach
Loading...
Access rights
openAccess
URL
Journal Title
Journal ISSN
Volume Title
A4 Artikkeli konferenssijulkaisussa
This publication is imported from Aalto University research portal.
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Other link related to publication (opens in new window)
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Other link related to publication (opens in new window)
Date
2023
Major/Subject
Mcode
Degree programme
Language
en
Pages
6
6230-6235
6230-6235
Series
ICC 2023 - IEEE International Conference on Communications, IEEE International Conference on Communications, Volume 2023-May
Abstract
This paper investigates the fundamental traffic steering issue for cellular-enabled unmanned aerial vehicles (UAVs), where each UAV needs to select one from different Mobile Network Operators (MNOs) to steer its traffic for improving the Quality-of-Service (QoS). To this end, we first formulate the issue as an optimization problem aiming to minimize the maximum outage probabilities of the UAVs. This problem is non-convex and non-linear, which is generally difficult to be solved. We propose a solution based on the framework of deep reinforcement learning (DRL) to solve it, in which we define the environment and the agent elements. Furthermore, to avoid sharing the learned experiences by the UAV in this solution, we further propose a federated deep reinforcement learning (FDRL)-based solution. Specifically, each UAV serves as a distributed agent to train separate model, and is then communicated to a special agent (dubbed coordinator) to aggregate all training models. Moreover, to optimize the aggregation process, we also introduce a FDRL with DRL-based aggregation (DRL2A) approach, in which the coordinator implements a DRL algorithm to learn optimal parameters of the aggregation. We consider deep Q-learning (DQN) algorithm for the distributed agents and Advantage Actor-Critic (A2C) for the coordinator. Simulation results are presented to validate the effectiveness of the proposed approach.Description
Publisher Copyright: © 2023 IEEE.
Keywords
Cellular Networks, Connection Steering, Deep Reinforcement Learning (DRL), FDRL with DRL-based Aggregation (DRL2A), Federated Deep Reinforcement Learning (FDRL), Unmanned Aerial Vehicles (UAVs)
Other note
Citation
Hellaoui, H, Yang, B, Taleb, T & Manner, J 2023, Traffic Steering for Cellular-Enabled UAVs: A Federated Deep Reinforcement Learning Approach . in M Zorzi, M Tao & W Saad (eds), ICC 2023 - IEEE International Conference on Communications : Sustainable Communications for Renaissance . IEEE International Conference on Communications, vol. 2023-May, IEEE, pp. 6230-6235, IEEE International Conference on Communications, Rome, Italy, 28/05/2023 . https://doi.org/10.1109/ICC45041.2023.10279441