Soft and Safe Network Slicing with Hierarchical Multi-Agent Reinforcement Learning
محورهای موضوعی : Pervasive computing
Zeinab Sasan
1
,
siavash khorsandi
2
1 - Amirkabir University of Technology
2 - Amirkabir University of Technology
کلید واژه: Hierarchical Multi-Agent Reinforcement Learning (HMARL), Soft Network Slicing, Safe Network Slicing, Resource Allocation, Multi-Agent Systems,
چکیده مقاله :
Network slicing has emerged as a key enabler for next-generation mobile networks, allowing multiple virtual network slices to coexist on shared physical infrastructure. Achieving high utilization while ensuring low dissatisfaction and safe operation across slices remains a challenging task, especially under dynamic and heterogeneous demands. In this paper, we propose a Hierarchical Multi-Agent Reinforcement Learning (HMARL) framework for soft and safe network slicing resource allocation. Our approach leverages a two-level learning architecture in which local agents manage slice-level decisions while a global agent coordinates overall resource distribution. At the lower layer, slice-level agents observe local conditions such as pending requests, deadlines, and resource demands and make admission or scheduling decisions accordingly. A meta-controller at the higher layer, adjusts inter-slice capacities and resource boosts to balance utilization, fairness, and safety across the entire system. We compare HMARL against Flat MARL, No-Sharing, and Random admission baselines to highlight its performance advantages. Extensive experiments show that HMARL achieves higher utilization and better objective function values while significantly reducing rejection ratios and dissatisfaction compared to other approaches. These results demonstrate that hierarchical reinforcement learning can provide a scalable, flexible, and safety-aware solution for resource management in soft network slicing environments improving both scalability and adaptability.
1. Chowdhury, M.Z., et al., 6G wireless communication systems: Applications, requirements, technologies, challenges, and research directions. IEEE Open Journal of the Communications Society, 2020. 1: p. 957-975.
2. Afolabi, I., et al., Network slicing and softwarization: A survey on principles, enabling technologies, and solutions. IEEE Communications Surveys & Tutorials, 2018. 20(3): p. 2429-2453.
3. Vassilaras, S., et al., The algorithmic aspects of network slicing. IEEE Communications Magazine, 2017. 55(8): p. 112-119.
4. Khan, L.U., et al., Network slicing: Recent advances, taxonomy, requirements, and open research challenges. IEEE Access, 2020. 8: p. 36009-36028.
5. Jiang, W., et al., The road towards 6G: A comprehensive survey. IEEE Open Journal of the Communications Society, 2021. 2: p. 334-366.
6. Sasan, Z., et al. Joint network slicing, routing, and in-network computing for energy-efficient 6g. in 2024 IEEE Wireless Communications and Networking Conference (WCNC). 2024. IEEE.
7. Gonzalez, A.J., et al. The isolation concept in the 5G network slicing. in 2020 European Conference on Networks and Communications (EuCNC). 2020. IEEE.
8. Sasan, Z. and S. Khorsandi. Slice-aware resource calendaring in cloud-based radio access networks. in 2022 30th International Conference on Electrical Engineering (ICEE). 2022. IEEE.
9. Zhang, S., et al., Hierarchical soft slicing to meet multi-dimensional QoS demand in cache-enabled vehicular networks. IEEE Transactions on Wireless Communications, 2020. 19(3): p. 2150-2162.
10. Wang, T., et al., LinkSlice: Fine-grained network slice enforcement based on deep reinforcement learning. IEEE Journal on Selected Areas in Communications, 2022. 40(8): p. 2378-2394.
11. Zhang, H., et al. A hard and soft hybrid slicing framework for service level agreement guarantee via deep reinforcement learning. in 2022 IEEE 95th Vehicular Technology Conference:(VTC2022-Spring). 2022. IEEE.
12. Kim, Y. and H. Lim, Multi-agent reinforcement learning-based resource management for end-to-end network slicing. IEEe Access, 2021. 9: p. 56178-56190.
13. Sulaiman, M., et al. Multi-agent deep reinforcement learning for slicing and admission control in 5G C-RAN. in NOMS 2022-2022 IEEE/IFIP network operations and management symposium. 2022. IEEE.
14. Mason, F., G. Nencioni, and A. Zanella. A multi-agent reinforcement learning architecture for network slicing orchestration. in 2021 19th Mediterranean Communication and Computer Networking Conference (MedComNet). 2021. IEEE.
15. Sasan, Z. and S. Khorsandi, Balancing resource utilization and slice dissatisfaction through dynamic soft slicing for 6G wireless networks. Scientific Reports, 2025. 15(1): p. 22987.
16. Li, J., et al., A hierarchical soft RAN slicing framework for differentiated service provisioning. IEEE Wireless Communications, 2020. 27(6): p. 90-97.
17. Shi, W., et al., Two-level soft RAN slicing for customized services in 5G-and-beyond wireless communications. IEEE Transactions on Industrial Informatics, 2021. 18(6): p. 4169-4179.
18. Ding, J., M. Zheng, and H. Yu, Soft Resource Slicing for Industrial Mixed Traffic in 5G Networks. IEEE/CAA Journal of Automatica Sinica, 2025. 12(2): p. 463-465.
19. Kaytaz, U., F. Sivrikaya, and S. Albayrak. Hierarchical deep reinforcement learning based dynamic RAN slicing for 5G V2X. in 2021 IEEE Global Communications Conference (GLOBECOM). 2021. IEEE.
20. Shao, Y., et al., Graph attention network-based multi-agent reinforcement learning for slicing resource management in dense cellular network. IEEE Transactions on vehicular Technology, 2021. 70(10): p. 10792-10803.
21. Toumi, N., M. Bagaa, and A. Ksentini. Hierarchical multi-agent deep reinforcement learning for SFC placement on multiple domains. in 2021 IEEE 46th Conference on Local Computer Networks (LCN). 2021. IEEE.
22. Chen, X., et al. Hierarchical Meta-Reinforcement Learning for Resource-Efficient Slicing in O-RAN. in GLOBECOM 2023-2023 IEEE Global Communications Conference. 2023. IEEE.
23. Akyıldız, H.A., et al., Hierarchical reinforcement learning based resource allocation for RAN slicing. IEEE Access, 2024. 12: p. 75818-75831.
24. Phyu, H.P., D. Naboulsi, and R. Stanica, ICE-CREAM: multI-agent fully CooperativE deCentRalizEd frAMework for Energy Efficiency in RAN Slicing. IEEE Transactions on Network and Service Management, 2025.
25. Hu, T., et al. Inter-cell slicing resource partitioning via coordinated multi-agent deep reinforcement learning. in ICC 2022-IEEE International Conference on Communications. 2022. IEEE.
26. Zhou, G., et al., Joint multi-objective optimization for radio access network slicing using multi-agent deep reinforcement learning. IEEE Transactions on Vehicular Technology, 2023. 72(9): p. 11828-11843.
27. Zhou, H., M. Elsayed, and M. Erol-Kantarci. RAN resource slicing in 5G using multi-agent correlated Q-learning. in 2021 IEEE 32nd Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC). 2021. IEEE.
