Browsing by Author "Zhang, Junshan"

Now showing 1 - 4 of 4

Adaptive Cache Policy Optimization Through Deep Reinforcement Learning in Dynamic Cellular Networks
(2024) Srinivasan, Ashvin; Amidzade, Mohsen; Zhang, Junshan; Tirkkonen, Olav
A1 Alkuperäisartikkeli tieteellisessä aikakauslehdessä
We explore the use of caching both at the network edge and within User Equipment (UE) to alleviate traffic load of wireless networks. We develop a joint cache placement and delivery policy that maximizes the Quality of Service (QoS) while simultaneously minimizing backhaul load and UE power consumption, in the presence of an unknown time-variant file popularity. With file requests in a time slot being affected by download success in the previous slot, the caching system becomes a non-stationary Partial Observable Markov Decision Process (POMDP). We solve the problem in a deep reinforcement learning framework based on the Advantageous Actor-Critic (A2C) algorithm, comparing Feed Forward Neural Networks (FFNN) with a Long Short-Term Memory (LSTM) approach specifically designed to exploit the correlation of file popularity distribution across time slots. Simulation results show that using LSTM-based A2C outperforms FFNN-based A2C in terms of sample efficiency and optimality, demonstrating superior performance for the non-stationary POMDP problem. For caching at the UEs, we provide a distributed algorithm that reaches the objectives dictated by the agent controlling the network, with minimum energy consumption at the UEs, and minimum communication overhead.
Cache Policy Design via Reinforcement Learning for Cellular Networks in Non-Stationary Environment
(2023-10-23) Srinivasan, Ashvin; Amidzade, Mohsen; Zhang, Junshan; Tirkkonen, Olav
A4 Artikkeli konferenssijulkaisussa
We consider wireless caching both at the network edge and at User Equipment (UE) to alleviate traffic congestion, aiming to find a joint cache placement and delivery policy by maximizing the Quality of Service (QoS) while minimizing backhaul load and User Equipment (UE) power consumption. We assume unknown and time-variant file popularities which are affected by the UE cache content, leading to a non-stationary Partial Observable Markov Decision Process (POMDP). We address this problem in a deep reinforcement learning framework, employing Feed Forward Neural Network (FFNN) and Long Short Term Memory (LSTM) networks in conjunction with Advantageous Actor Critic (A2C) algorithm. LSTM exploits the correlation of the file popularity distribution across time slots to learn information of the dynamics of the environment and A2C algorithm is used due to its ability of handling continuous and high dimensional spaces. We leverage LSTM and A2C tools based on its virtue to find an optimal solution for the POMDP environment. Simulation results show that using LSTM-based A2C outperforms a FFNN-based A2C in terms of sample efficiency and optimality. An LSTM-based A2C gives a superior performance under the non-stationary POMDP paradigm.
IEEE Access Special Section Editorial: Recent Advances in Socially-Aware Mobile Networking
(2017) Peng, Mugen; Yang, Lei; Zhang, Junshan; Chen, Tao; Celentano, Ulrico; Roning, Juha; Ermolova, Natalia Y.; Tirkkonen, Olav
A1 Alkuperäisartikkeli tieteellisessä aikakauslehdessä
Mobile data traffic has been growing exponentially over the past few years. A report from Cisco shows that the mobile data traffic in 2014 grew 69 percent and was nearly 30 times the size of the entire global Internet in 2000 [item 1) in the Appendix]. One of the primary contributors to the explosive mobile traffic growth is the rapid proliferation of mobile social applications running on multimedia mobile devices (particularly smartphones). These sharp increases in mobile traffic (particularly from mobile social applications) are projected to continue in the foreseeable future. As mobile networks by and large are designed and deployed to meet people's social needs, people's behaviors and interactions in the social domain will shape their ways to access mobile services. Therefore, there is an urgent need to integrate social effects into the design of mobile networks.
Joint Cache Placement and Delivery Design using Reinforcement Learning for Cellular Networks
(2021-06-15) Amidzade, Mohsen; Al-Tous, Hanan; Tirkkonen, Olav; Zhang, Junshan
A4 Artikkeli konferenssijulkaisussa
We consider a reinforcement learning (RL) based joint cache placement and delivery (CPD) policy for cellular networks with limited caching capacity at both Base Stations (BSs) and User Equipments (UEs). The dynamics of file preferences of users is modeled by a Markov process. User requests are based on current preferences, and on the content of the user's cache. We assume probabilistic models for the cache placement at both the UEs and the BSs. When the network receives a request for an un-cached file, it fetches the file from the core network via a backhaul link. File delivery is based on network-level orthogonal multipoint multicasting transmissions. For this, all BSs caching a specific file transmit collaboratively in a dedicated resource. File reception depends on the state of the wireless channels. We design the CPD policy while taking into account the user Quality of Service and the backhaul load, and using an Actor-Critic RL framework with two neural networks. Simulation results are used to show the merits of the devised CPD policy.