Soft and Safe Network Slicing with Hierarchical Multi-Agent Reinforcement Learning
Subject Areas : Pervasive computing
Zeinab Sasan
1
,
Siavash Khorsandi
2
1 -
2 -
Keywords: Hierarchical Multi-Agent Reinforcement Learning (HMARL), Soft Network Slicing, Safe Network Slicing, Resource Allocation, Multi-Agent Systems, Safe Reinforment Learning,
Abstract :
Network slicing has emerged as a key enabler for next-generation mobile networks, allowing multiple virtual network slices to coexist over shared physical infrastructure. Soft network slicing can improve resource utilization by enabling controlled inter-slice resource sharing; however, increased resource sharing may impose a higher burden on donor slices and consequently increase slice dissatisfaction. Achieving efficient resource utilization while controlling slice dissatisfaction and maintaining feasible resource allocation remains challenging under dynamic and heterogeneous traffic demands. In this paper, we propose a Hierarchical Multi-Agent Reinforcement Learning (HMARL) framework for online soft and safe network slicing. The proposed architecture operates at two decision levels: slice-level agents make local admission decisions, while a higher-level meta-controller coordinates inter-slice resource sharing through adaptive borrowing limits. A deterministic safety layer validates the proposed actions before resource allocation and ensures compliance with resource, continuity, and borrowing constraints. The proposed framework is evaluated against learning-based, non-sharing, random, greedy, and heuristic resource-allocation approaches. The training results show that HMARL reaches a stable operating regime with high normalized resource utilization and a low request rejection ratio. Across different scenario classes, HMARL achieves a favorable trade-off between resource utilization and slice dissatisfaction, resulting in competitive overall objective values relative to the considered alternatives. These results indicate that hierarchical coordination combined with deterministic feasibility enforcement provides an effective approach for adaptive resource management in soft network slicing environments.
[1] Chowdhury, M.Z., et al., 6G wireless communication systems: Applications, requirements, technologies, challenges, and research directions. IEEE Open Journal of the Communications Society, 2020. 1: p. 957-975.
[2] Afolabi, I., et al., Network slicing and softwarization: A survey on principles, enabling technologies, and solutions. IEEE Communications Surveys & Tutorials, 2018. 20(3): p. 2429-2453.
[3] Vassilaras, S., et al., The algorithmic aspects of network slicing. IEEE Communications Magazine, 2017. 55(8): p. 112-119.
[4] Khan, L.U., et al., Network slicing: Recent advances, taxonomy, requirements, and open research challenges. IEEE Access, 2020. 8: p. 36009-36028.
[5] Jiang, W., et al., The road towards 6G: A comprehensive survey. IEEE Open Journal of the Communications Society, 2021. 2: p. 334-366.
[6] Sasan, Z., et al. Joint network slicing, routing, and in-network computing for energy-efficient 6g. in 2024 IEEE Wireless Communications and Networking Conference (WCNC). 2024. IEEE.
[7] Gonzalez, A.J., et al. The isolation concept in the 5G network slicing. in 2020 European Conference on Networks and Communications (EuCNC). 2020. IEEE.
[8] Sasan, Z. and S. Khorsandi. Slice-aware resource calendaring in cloud-based radio access networks. in 2022 30th International Conference on Electrical Engineering (ICEE). 2022. IEEE.
[9] Zhang, S., et al., Hierarchical soft slicing to meet multi-dimensional QoS demand in cache-enabled vehicular networks. IEEE Transactions on Wireless Communications, 2020. 19(3): p. 2150-2162.
[10] Wang, T., et al., LinkSlice: Fine-grained network slice enforcement based on deep reinforcement learning. IEEE Journal on Selected Areas in Communications, 2022. 40(8): p. 2378-2394.
[11] Zhang, H., et al. A hard and soft hybrid slicing framework for service level agreement guarantee via deep reinforcement learning. in 2022 IEEE 95th Vehicular Technology Conference:(VTC2022-Spring). 2022. IEEE.
[12] Kim, Y. and H. Lim, Multi-agent reinforcement learning-based resource management for end-to-end network slicing. IEEe Access, 2021. 9: p. 56178-56190.
[13] Sulaiman, M., et al. Multi-agent deep reinforcement learning for slicing and admission control in 5G C-RAN. in NOMS 2022-2022 IEEE/IFIP network operations and management symposium. 2022. IEEE.
[14] Mason, F., G. Nencioni, and A. Zanella. A multi-agent reinforcement learning architecture for network slicing orchestration. in 2021 19th Mediterranean Communication and Computer Networking Conference (MedComNet). 2021. IEEE.
[15] Sasan, Z. and S. Khorsandi, Balancing resource utilization and slice dissatisfaction through dynamic soft slicing for 6G wireless networks. Scientific Reports, 2025. 15(1): p. 22987.
[16] Li, J., et al., A hierarchical soft RAN slicing framework for differentiated service provisioning. IEEE Wireless Communications, 2020. 27(6): p. 90-97.
[17] Shi, W., et al., Two-level soft RAN slicing for customized services in 5G-and-beyond wireless communications. IEEE Transactions on Industrial Informatics, 2021. 18(6): p. 4169-4179.
[18] Ding, J., M. Zheng, and H. Yu, Soft Resource Slicing for Industrial Mixed Traffic in 5G Networks. IEEE/CAA Journal of Automatica Sinica, 2025. 12(2): p. 463-465.
[19] Kaytaz, U., F. Sivrikaya, and S. Albayrak. Hierarchical deep reinforcement learning based dynamic RAN slicing for 5G V2X. in 2021 IEEE Global Communications Conference (GLOBECOM). 2021. IEEE.
[20] Shao, Y., et al., Graph attention network-based multi-agent reinforcement learning for slicing resource management in dense cellular network. IEEE Transactions on vehicular Technology, 2021. 70(10): p. 10792-10803.
[21] Toumi, N., M. Bagaa, and A. Ksentini. Hierarchical multi-agent deep reinforcement learning for SFC placement on multiple domains. in 2021 IEEE 46th Conference on Local Computer Networks (LCN). 2021. IEEE.
[22] Chen, X., et al. Hierarchical Meta-Reinforcement Learning for Resource-Efficient Slicing in O-RAN. in GLOBECOM 2023-2023 IEEE Global Communications Conference. 2023. IEEE.
[23] Akyıldız, H.A., et al., Hierarchical reinforcement learning based resource allocation for RAN slicing. IEEE Access, 2024. 12: p. 75818-75831.
[24] Phyu, H.P., D. Naboulsi, and R. Stanica, ICE-CREAM: multI-agent fully CooperativE deCentRalizEd frAMework for Energy Efficiency in RAN Slicing. IEEE Transactions on Network and Service Management, 2025.
[25] Hu, T., et al. Inter-cell slicing resource partitioning via coordinated multi-agent deep reinforcement learning. in ICC 2022-IEEE International Conference on Communications. 2022. IEEE.
[26] Zhou, G., et al., Joint multi-objective optimization for radio access network slicing using multi-agent deep reinforcement learning. IEEE Transactions on Vehicular Technology, 2023. 72(9): p. 11828-11843.
[27] Zhou, H., M. Elsayed, and M. Erol-Kantarci. RAN resource slicing in 5G using multi-agent correlated Q-learning. in 2021 IEEE 32nd Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC). 2021. IEEE.
