Reinforcement Learning Methodologies for RAN Slicing

No Thumbnail Available
Journal Title
Journal ISSN
Volume Title
Sähkötekniikan korkeakoulu | Master's thesis
Date
2024-01-22
Department
Major/Subject
Autonomous Systems
Mcode
ELEC3055
Degree programme
Master's Programme in ICT Innovation
Language
en
Pages
76
Series
Abstract
The rapid development of Radio Access Networks (RANs) from 1G to 5G has revolutionized telecommunications, ushering the world in a new era of connectivity. As a key component of the mobile network, RAN plays a crucial role in providing internet access to all personal devices, shaping the modern digital landscape. However, the increasing demand for a wide range of customer needs poses yet another challenge in the journey towards 5G and beyond. RAN Slicing addresses the performance bottleneck of RAN, substantially impacting user experience in terms of throughput, latency, and reliability. It utilizes virtualization to divide radio resources into separate, autonomous virtual networks, assigning resources dynamically to satisfy service requirements. The dynamic nature of RAN slicing requires innovative solutions. Deep Reinforcement Learning (DRL) agents have emerged as a solution to solve complex tasks that need complete automation. These agents adaptively distribute resources to network slices in response to changing factors, including channel quality, slice admission, service level agreements, and user density. This thesis builds upon prior research by addressing two key challenges in DRL algorithms: slow convergence and unstable exploration phases. The results emphasise the critical role of reward function selection and hyper-parameter tuning. A new reward function that outperforms previous approaches is introduced. Multiple algorithms are evaluated, with the Advantage Actor Critic one exhibiting superior performance in several cases. Furthermore, the significance of policy transfer is highlighted, whereby an expert base station’s policy is transferred to a learner base station using policy reuse, distillation, or a hybrid approach. This study confirms that these transfers accelerate convergence, with policy distillation and the hybrid approach demonstrating notable reliability.
Description
Supervisor
Hamalainen, Jyri
Thesis advisor
Ilter, Mehmet
Piiroinen, Markku
Keywords
cellular networks, RAN Slicing, 5G, 6G, deep reinforcement learning
Other note
Citation