Abstract: With the development of satellite communication technology, satellite-terrestrial integrated networks (STINs), which integrate satellite networks and ground networks, can realize global seamless coverage of communication services. Confronting the intricacies of network dynamics, the resource heterogeneity, and the unpredictability of user mobility, dynamic resource allocation within networks faces formidable challenges. Digital twin (DT), as a new technique, can reflect a physical network to a virtual network to monitor, analyze, and optimize the physical networks. Nevertheless, in the process of constructing a DT model, the deployment location and resource allocation of DTs may adversely affect its performance. Therefore, we propose a STIN model, which alleviates the problem of insufficient single-layer deployment flexibility of the traditional edge network by deploying DTs in multi-layer nodes in a STIN. To address the challenge of deploying DTs in the network, we propose a multi-layer DT deployment problem in the STIN to reduce system delay. Then we adopt a multi-agent reinforcement learning (MARL) scheme to explore the optimal strategy of the DT multi-layer deployment problem. The implemented scheme demonstrates a notable reduction in system delay, as evidenced by simulation outcomes.

数字孪生驱动的星地融合网络中的自适应多层部署

[1]Barricelli BR, Casiraghi E, Fogli D, 2019. A survey on digital twin: definitions, characteristics, applications, and design implications. IEEE Access, 7:167653-167671.

[2]Bellavista P, Giannelli C, Mamei M, et al., 2021. Application-driven network-aware digital twin management in industrial edge environments. IEEE Trans Ind Inform, 17(11):7791-7801.

[3]Cao YR, Guo HZ, Liu JJ, et al., 2018. Optimal satellite gateway placement in space-ground integrated networks. IEEE Netw, 32(5):32-37.

[4]Chukhno O, Chukhno N, Araniti G, et al., 2022. Placement of social digital twins at the edge for beyond 5G IoT networks. IEEE Int Things J, 9(23):23927-23940.

[5]Dong R, She C, Hardjawana W, et al., 2019. Deep learning for hybrid 5G services in mobile edge computing systems: learn from a digital twin. IEEE Trans Wirel Commun, 18(10):4692-4707.

[6]Duong TQ, Nguyen LD, Bui TT, et al., 2023. Machine learning-aided real-time optimized multibeam for 6G integrated satellite-terrestrial networks: global coverage for mobile services. IEEE Netw, 37(2):86-93.

[7]Fan HL, Long J, Liu LM, et al., 2023. Dynamic digital twin and online scheduling for contact window resources in satellite network. IEEE Trans Ind Inform, 19(5):7217-7227.

[8]Fu S, Gao J, Zhao L, 2020. Integrated resource management for terrestrial-satellite systems. IEEE Trans Veh Technol, 69(3):3256-3266.

[9]Greff K, Srivastava RK, Koutník J, et al., 2017. LSTM: a search space odyssey. IEEE Trans Neur Netw Learn Syst, 28(10):2222-2232.

[10]Guo HZ, Li JY, Liu JJ, et al., 2022. A survey on space-air-ground-sea integrated network security in 6G. IEEE Commun Surv Tutor, 24(1):53-87.

[11]Guo Q, Tang FX, Kato N, 2023. Resource allocation for aerial assisted digital twin edge mobile network. IEEE J Sel Areas Commun, 41(10):3070-3079.

[12]Guo Q, Tang FX, Rodrigues TK, et al., 2024. Five disruptive technologies in 6G to support digital twin networks. IEEE Wirel Commun, 31(1):149-155.

[13]Guo ZY, Chen ZY, Liu P, et al., 2022. Multi-agent reinforcement learning-based distributed channel access for next generation wireless networks. IEEE J Sel Areas Commun, 40(5):1587-1599.

[14]Hui YL, Qiu Y, Su Z, et al., 2023. Digital twins for intelligent space-air-ground integrated vehicular network: challenges and solutions. IEEE Int Things Mag, 6(3):70-76.

[15]Ji Z, Wu S, Jiang CX, 2023. Cooperative multi-agent deep reinforcement learning for computation offloading in digital twin satellite edge networks. IEEE J Sel Areas Commun, 41(11):3414-3429.

[16]Jiang XY, Zhang T, Liu L, 2023. Research on satellite QoS routing algorithm based on digital twin. Proc 11^th Int Conf on Intelligent Computing and Wireless Optical Communications, p.118-122.

[17]Kato N, Fadlullah ZM, Tang FX, et al., 2019. Optimizing space-air-ground integrated networks by artificial intelligence. IEEE Wirel Commun, 26(4):140-147.

[18]Li XN, Zhang HJ, Zhou H, et al., 2023. Multi-agent DRL for resource allocation and cache design in terrestrial-satellite networks. IEEE Trans Wirel Commun, 22(8):5031-5042.

[19]Liu JJ, Shi YP, Zhao L, et al., 2018a. Joint placement of controllers and gateways in SDN-enabled 5G-satellite integrated network. IEEE J Sel Areas Commun, 36(2):221-232.

[20]Liu JJ, Shi YP, Fadlullah ZM, et al., 2018b. Space-air-ground integrated network: a survey. IEEE Commun Surv Tutor, 20(4):2714-2741.

[21]Liu T, Tang L, Wang WL, et al., 2022. Digital-twin-assisted task offloading based on edge collaboration in the digital twin edge network. IEEE Int Things J, 9(2):1427-1444.

[22]Lu YL, Maharjan S, Zhang Y, 2021a. Adaptive edge association for wireless digital twin networks in 6G. IEEE Int Things J, 8(22):16219-16230.

[23]Lu YL, Huang XH, Zhang K, et al., 2021b. Low-latency federated learning and blockchain for edge association in digital twin empowered 6G networks. IEEE Trans Ind Inform, 17(7):5098-5107.

[24]Mao BM, Zhou XM, Liu JJ, et al., 2024. Digital twin satellite networks toward 6G: motivations, challenges, and future perspectives. IEEE Netw, 38(1):54-60.

[25]Sun W, Zhang HB, Wang R, et al., 2020. Reducing offloading latency for digital twin edge networks in 6G. IEEE Trans Veh Technol, 69(10):12240-12251.

[26]Sunehag P, Lever G, Gruslys A, et al., 2018. Value-decomposition networks for cooperative multi-agent learning based on team reward. Proc 17^th Int Conf on Autonomous Agents and Multiagent Systems, p.2085-2087.

[27]Tan X, Zhou L, Wang HJ, et al., 2022. Cooperative multi-agent reinforcement-learning-based distributed dynamic spectrum access in cognitive radio networks. IEEE Int Things J, 9(19):19477-19488.

[28]Tang FX, Chen XH, Rodrigues TK, et al., 2022. Survey on digital twin edge networks (DITEN) toward 6G. IEEE Open J Commun Soc, 3:1360-1381.

[29]Tang M, Wong VWS, 2022. Deep reinforcement learning for task offloading in mobile edge computing systems. IEEE Trans Mob Comput, 21(6):1985-1997.

[30]Tang QQ, Fei ZS, Li B, et al., 2021. Computation offloading in LEO satellite networks with hybrid cloud and edge computing. IEEE Int Things J, 8(11):9164-9176.

[31]Tao F, Zhang H, Liu A, et al., 2019. Digital twin in industry: state-of-the-art. IEEE Trans Ind Inform, 15(4):2405-2415.

[32]Vaezi M, Noroozi K, Todd TD, et al., 2023. Digital twin placement for minimum application request delay with data age targets. IEEE Int Things J, 10(13):11547-11557.

[33]Wang GC, Zhou S, Zhang S, et al., 2020. SFC-based service provisioning for reconfigurable space-air-ground integrated networks. IEEE J Sel Areas Commun, 38(7):1478-1489.

[34]Wang P, Zhang JX, Zhang X, et al., 2020. Convergence of satellite and terrestrial networks: a comprehensive survey. IEEE Access, 8:5550-5588.

[35]Wu YW, Zhang K, Zhang Y, 2021. Digital twin networks: a survey. IEEE Int Things J, 8(18):13789-13804.

[36]Yao ZX, Xia SC, Li Y, et al., 2023. Cooperative task offloading and service caching for digital twin edge networks: a graph attention multi-agent reinforcement learning approach. IEEE J Sel Areas Commun, 41(11):3401-3413.

[37]Yin ZS, Cheng N, Luan TH, et al., 2023. DT-assisted multi-point symbiotic security in space-air-ground integrated networks. IEEE Trans Inform Forens Secur, 18:5721-5734.

[38]Zhang H, Luo TX, Wang QQ, 2023. Adaptive digital twin server deployment for dynamic edge networks in IoT system. IEEE/CIC Int Conf on Communications in China, p.1-6.

[39]Zhang JX, Wang KW, Li R, et al., 2023. MaCro: mega-constellations routing systems with multi-edge cross-domain features. IEEE Wirel Commun, 30(6):69-76.

[40]Zhang K, Cao JY, Zhang Y, 2022. Adaptive digital twin and multiagent deep reinforcement learning for vehicular edge computing and networks. IEEE Trans Ind Inform, 18(2):1405-1413.

[41]Zhang YD, Zhang HB, Lu YL, et al., 2024. Adaptive digital twin placement and transfer in wireless computing power network. IEEE Int Things J, 11(6):10924-10936.

[42]Zhang ZJ, Zhang WY, Tseng FH, 2019. Satellite mobile edge computing: improving QoS of high-speed satellite-terrestrial networks using edge computing techniques. IEEE Netw, 33(1):70-76.

[43]Zhang ZQ, Xiao Y, Ma Z, et al., 2019. 6G wireless networks: vision, requirements, architecture, and key technologies. IEEE Veh Technol Mag, 14(3):28-41.

[44]Zhao L, Wang CC, Zhao KL, et al., 2022. INTERLINK: a digital twin-assisted storage strategy for satellite-terrestrial networks. IEEE Trans Aerosp Electron Syst, 58(5):3746-3759.

[45]Zhou YK, Zhang R, Liu J, et al., 2023. A hierarchical digital twin network for satellite communication networks. IEEE Commun Mag, 61(11):104-110.

[46]Zhu XM, Jiang CX, Kuang LL, et al., 2019. Cooperative transmission in integrated terrestrial-satellite networks. IEEE Netw, 33(3):204-210.