ARL-RA: Efficient Resource Allocation in 5G Edge Networks: A Novel Intelligent Solution UsingApproximate Reinforcement Learning Algorithm

Document Type : Research Paper

Authors

1 Department of Computer Engineering, Semnan Branch, Islamic Azad University, Semnan, Iran

2 Department of Computer Engineering, University of Mohaghegh Ardabili, Ardabil, Iran

Abstract

The rapid proliferation of fifth-generation (5G) technology has resulted in a wide range of applications, posing challenges in managing network resources effectively and efficiently. To address these challenges, network slicing (NS) and Fog-Radio Access Networks (F-RAN) have emerged as key technologies, enabling the creation of isolated virtual networks on shared physical infrastructure to support high-bandwidth and low-latency communication. However, allocating network resources to latency-sensitive applications like self-driving cars and remote surgery, while ensuring a quality experience, is complex due to stringent latency requirements and limited availability. In this paper, we propose a novel approach, leveraging the Q-learning algorithm, specifically the Approximate Reinforcement Learning for dynamic resource allocation (RA-ARL), in the context of 5G environments. Our modified algorithm takes into account crucial network attributes, introduces innovations such as service type classification based on latency sensitivity, and considers time-varying resource conditions and service demands. We propose an RL model to optimize network utility, focusing on the F-RAN model. Our experimental results demonstrate the effectiveness of ARL-RA in terms of convergence, resource utilization, and the ability to handle user request rejections. This work contributes to the advancement of efficient and effective resource allocation in dynamic 5G networks, particularly for latency-sensitive applications with stringent quality requirements. 

Keywords

References

[1]     R. Atat, L. Liu, H. Chen, J. Wu, H. Li, and Y. Yi, "Enabling cyber‐physical communication in 5G cellular networks: challenges, spatial spectrum sensing, and cyber‐security," IET Cyber‐Physical Systems: Theory & Applications, vol. 2, no. 1, pp. 49-54, Apr. 2017.
[2]     A. A. Barakabitze, A. Ahmad, R. Mijumbi, and A. Hines, "5G network slicing using SDN and NFV: A survey of taxonomy, architectures and future challenges," Computer Networks, vol. 167, no. 7, p. 106984, Feb. 2020.
[3]     Y. D. Ahmad Sarlak "An Approach to Improve the Quality of Service in DTN and Non-DTN based VANET," Journal of Information Systems and Telecommunication, no. Issue 4, pp. 240 - 248, Jan. 2020.
[4]     H. Xiang, W. Zhou, M. Daneshmand, and M. Peng, "Network slicing in fog radio access networks: Issues and challenges," IEEE Communications Magazine, vol. 55, no. 12, pp. 110-116, Dec. 2017.
[5]     M. Khazaei, "Dynamic Tree- Based Routing: Applied in Wireless Sensor Network and IOT," Journal of Information Systems and Telecommunication, vol. Vol.10, No.3, pp. 191-200, Aug. 2022.
[6]     J. Kaur, M. A. Khan, M. Iftikhar, M. Imran, and Q. E. U. Haq, "Machine learning techniques for 5G and beyond," IEEE Access, vol. 9, no. 3, pp. 23472-23488, Jan. 2021.
[7]     M. E. Morocho-Cayamcela, H. Lee, and W. Lim, "Machine learning for 5G/B5G mobile and wireless communications: Potential, limitations, and future directions," IEEE access, vol. 7, no. 7. pp. 137184-137206, Sep. 2019.
[8]     H. F. Sajad Mohammadzadeh , Zohreh Dorrani, "Edge Detection and Identification using Deep Learning to Identify Vehicles," Journal of Information Systems and Telecommunication (JIST), vol. Vol.10, No.3, pp. 201-210, Aug. 2022.
[9]     J. Clifton and E. Laber, "Q-learning: Theory and applications," Annual Review of Statistics and Its Application, vol. 7, no. 3, pp. 279-301, Mar. 2020.
[10]   B. Han and H. D. Schotten, "Machine learning for network slicing resource management: A comprehensive survey," arXiv preprint arXiv:2001.07974, Jan. 2020.
[11]   M. S. Al-Abiad, M. Z. Hassan, and M. J. Hossain, "A joint reinforcement-learning enabled caching and cross-layer network code in F-RAN with D2D communications," IEEE Transactions on Communications, vol. 70, no. 7, pp. 4400-4416, Apr. 2022.
[12]   A. Nassar and Y. Yilmaz, "Reinforcement learning for adaptive resource allocation in fog RAN for IoT with heterogeneous latency requirements," IEEE Access, vol. 7, no. 2, pp. 128014-128025, Sep. 2019.
[13]   N. N. Khumalo, O. O. Oyerinde, and L. Mfupe, "Reinforcement learning-based resource management model for fog radio access network architectures in 5G," IEEE Access, vol. 9,no. 4,  pp. 12706-12716, Jan. 2021.
[14]   R. Aghazadeh, A. Shahidinejad, and M. Ghobaei‐Arani, "Proactive content caching in edge computing environment: A review," Software: Practice and Experience, vol. 53, no. 3, pp. 811-855, Mar. 2023.
[15]   Z. Cheng, M. Min, M. Liwang, L. Huang, and Z. Gao, "Multiagent DDPG-based joint task partitioning and power control in Fog computing networks," IEEE Internet of Things Journal, vol. 9, no. 1, pp. 104-116, June 2021.
[16]   A. Nassar and Y. Yilmaz, "Deep reinforcement learning for adaptive network slicing in 5G for intelligent vehicular systems and smart cities," IEEE Internet of Things Journal, vol. 9, no. 1, pp. 222-235, 2021.
[17]   Y. Zhou, M. Peng, S. Yan, and Y. Sun, "Deep reinforcement learning based coded caching scheme in fog radio access networks," in 2018 IEEE/CIC International Conference on Communications in China (ICCC Workshops), Aug. 2018: IEEE, pp. 309-313.
[18]   G. S. Rahman, M. Peng, S. Yan, and T. Dang, "Learning based joint cache and power allocation in fog radio access networks," IEEE Transactions on Vehicular Technology, vol. 69, no. 4, pp. 4401-4411, Feb. 2020.
[19]   P. Gazori, D. Rahbari, and M. Nickray, "Saving time and cost on the scheduling of fog-based IoT applications using deep reinforcement learning approach," Future Generation Computer Systems, vol. 110, no. 8, pp. 1098-1115, Sep. 2020.
[20]   A. O. Abdalrahman, D. Pilevarzadeh, S. Ghafouri, and A. Ghaffari, "The Application of Hybrid Krill Herd Artificial Hummingbird Algorithm for Scientific Workflow Scheduling in Fog Computing," Journal of Bionic Engineering, vol. 20, no. 5, pp. 2443-2464, May 2023.
[21]   M. Nematollahi, A. Ghaffari, and A. Mirzaei, "Task and resource allocation in the internet of things based on an improved version of the moth-flame optimization algorithm," Cluster Computing, Vol. 10, no. 5, pp. 1-23, June 2023.
[22]   M. Khani, S. Jamali, and M. K. Sohrabi, "Approximate Q-learning-based (AQL) network slicing in mobile edge-cloud for delay-sensitive services," The Journal of Supercomputing, 2023/09/09 2023, doi: 10.1007/s11227-023-05614-4.
[23]   C. E. Nelson et al., "In vivo genome editing improves muscle function in a mouse model of Duchenne muscular dystrophy," Science, vol. 351, no. 6271, pp. 403-407, Jan. 2016.
[24]         M. Khani, S. Jamali, and M. K. Sohrabi, "An Enhanced Deep Reinforcement Learning-based Slice Acceptance Control System (EDRL-SACS) for Cloud-Radio Access Network," Physical Communication, p. 102188, Sep. 2023.

Files

Share

How to cite

Statistics