Towards using Deep Reinforcement Learning for Connection Steering in Cellular UAVs

Loading...
Thumbnail Image
Journal Title
Journal ISSN
Volume Title
Conference article in proceedings
Date
2021
Major/Subject
Mcode
Degree programme
Language
en
Pages
6
Series
2021 IEEE Global Communications Conference, GLOBECOM 2021 - Proceedings
Abstract
This paper investigates the fundamental connection steering issue in cellular-enabled Unmanned Aerial Vehicles (UAVs), whereby a UAV steers the cellular connection across multiple Mobile Network Operators (MNOs) for ensuring enhanced Quality-of-Service (QoS). We first formulate the issue as an optimization problem for minimizing the maximum outage probability. This is a nonlinear and nonconvex problem that is generally difficult to be solved. To this end, we propose a new approach for solving the optimization problem based on Deep Reinforcement Learning (DRL), considering two important reinforcement learning algorithms (i.e., Deep Q-Learning (DQN) and Advantage Actor Critic (A2C)). Simulation results show that under the proposed approach, the UAVs can make optimal decisions to select the most suitable connection with MNOs for achieving the minimization of the maximum outage probability. Furthermore, the results also show that in our new approach, the A2C-based algorithm is better than the DQN-based one, especially when the number of MNOs increases, while the DQN-based algorithm can be executed in a shorter time.
Description
Funding Information: ACKNOWLEDGMENT This work was partially supported by the European Union’s Horizon 2020 Research and Innovation Program through the 5G!Drones Project under Grant No. 857031. It was also supported in part by the Academy of Finland 6Genesis project under Grant No. 318927. Publisher Copyright: © 2021 IEEE. | openaire: EC/H2020/857031/EU//5G!Drones
Keywords
5G, Beyond 5G, Connection Steering, Deep Reinforce-ment Learning, Mobile Networks, Unmanned Aerial Vehicles (UAVs)
Other note
Citation
Hellaoui , H , Yang , B & Taleb , T 2021 , Towards using Deep Reinforcement Learning for Connection Steering in Cellular UAVs . in 2021 IEEE Global Communications Conference, GLOBECOM 2021 - Proceedings . 2021 IEEE Global Communications Conference, GLOBECOM 2021 - Proceedings , IEEE , IEEE Global Communications Conference , Madrid , Spain , 07/12/2021 . https://doi.org/10.1109/GLOBECOM46510.2021.9685265