Abstract: Recent studies have shown that system capacity is very important for cellular networks. In this paper, we consider maximizing the weighted sum-rate of the cellular network downlink and uplink, where each cell consists of a full-duplex (FD) base station (BS) and half-duplex (HD) users. Federated learning (FL) can train models in the absence of centralized data, which can achieve privacy protection of user data. A low Earth orbit (LEO) satellite edge computing system (LSECS) can be formed by placing the mobile edge computing (MEC) servers on LEO satellites, which greatly increases the processing capacities of the satellites. Therefore, we consider a combination of FL and MEC and propose an FL-based computation offloading algorithm to maximize the weighted sum-rate while ensuring the security of user data. We consider solving the sub-channel assignment and power allocation problems using deep reinforcement learning (DRL) algorithms with excellent global search capabilities. The simulation results show that our proposed algorithm achieves the maximum weighted sum-rate compared with the baseline algorithms and excellent convergence.

基于联邦深度强化学习的低轨卫星边缘计算系统计算卸载

[1]Alkhrijah Y, Camp J, Rajan D, 2023. Multi-band full duplex MAC protocol (MB-FDMAC). IEEE J Sel Areas Commun, 41(9):2864-2878.

[2]Chen H, Xiao M, Pang ZB, 2022. Satellite-based computing networks with federated learning. IEEE Wirel Commun, 29(1):78-84.

[3]Chen XM, Xu ZB, Shang L, 2023. Satellite Internet of Things: challenges, solutions, and development trends. Front Inform Technol Electron Eng, 24(7):935-944.

[4]Dai XY, Zhao C, Wang X, et al., 2022. Image-based traffic signal control via world models. Front Inform Technol Electron Eng, 23(12):1795-1813.

[5]El Houda ZA, Moudoud H, Brik B, 2024. Federated deep reinforcement learning for efficient jamming attack mitigation in O-RAN. IEEE Trans Veh Technol, 73(7):9334-9343.

[6]Fawaz H, Lahoud S, Helou ME, et al., 2023. Queue-aware resource allocation in full-duplex multi-cellular wireless networks. IEEE J Sel Areas Commun, 41(9):2852-2863.

[7]Fu H, Si WJ, Kim IM, 2023. Deep learning-based joint pilot design and channel estimation for OFDM systems. IEEE Trans Commun, 71(8):4577-4590.

[8]Gao YF, Ji Z, Zhao KL, et al., 2024. Game-based computation offloading and power allocation for LEO constellation networks in distributed and dynamic environment. IEEE Int Things J, 11(4):7040-7058.

[9]Han DJ, Hosseinalipour S, Love DJ, et al., 2024. Cooperative federated learning over ground-to-satellite integrated networks: joint local computation and data offloading. IEEE J Sel Areas Commun, 42(5):1080-1096.

[10]He ZY, Xu W, Shen H, et al., 2023. Full-duplex communication for ISAC: joint beamforming and power optimization. IEEE J Sel Areas Commun, 41(9):2920-2936.

[11]Jia M, Wu J, Zhang L, et al., 2023. Joint optimization communication and computing resource for LEO satellites with edge computing. Chin J Electron, 32(5):1011-1021.

[12]Jia M, Wu J, Guo Q, et al., 2024. Service-oriented SAGIN with pervasive intelligence for resource-constrained users. IEEE Netw, 38(2):79-86.

[13]Kamal M, Rashid I, Iqbal W, et al., 2023. Privacy and security federated reference architecture for Internet of Things. Front Inform Technol Electron Eng, 24(4):481-508.

[14]Kang YH, Zhu YF, Wang D, et al., 2024. Joint server selection and handover design for satellite-based federated learning using mean-field evolutionary approach. IEEE Trans Netw Sci Eng, 11(2):1655-1667.

[15]Liao Y, Yang ZJ, Yin ZS, et al., 2023. DQN-based adaptive MCS and SDM for 5G massive MIMO-OFDM downlink. IEEE Commun Lett, 27(1):185-189.

[16]Lim B, Vu M, 2023. Distributed multi-agent deep Q-learning for load balancing user association in dense networks. IEEE Wirel Commun Lett, 12(7):1120-1124.

[17]Liu PX, Jiang JM, Zhu GX, et al., 2022. Training time minimization for federated edge learning with optimized gradient quantization and bandwidth allocation. Front Inform Technol Electron Eng, 23(8):1247-1263.

[18]Lv ZH, Xiu WQ, 2020. Interaction of edge-cloud computing based on SDN and NFV for next generation IoT. IEEE Int Things J, 7(7):5706-5712.

[19]Razmi N, Matthiesen B, Dekorsy A, et al., 2022. Ground-assisted federated learning in LEO satellite constellations. IEEE Wirel Commun Lett, 11(4):717-721.

[20]Salim S, Moustafa N, Hassanian M, et al., 2024. Deep-federated-learning-based threat detection model for extreme satellite communications. IEEE Int Things J, 11(3):3853-3867.

[21]Sultan R, Shamseldeen A, 2024. Uplink-downlink cochannel interference cancellation in RIS-aided full-duplex networks. IEEE Syst J, 18(2):1220-1223.

[22]Sun YW, Duan BY, Su X, et al., 2023. Performance analysis on reconfigurable intelligent surface and network-controlled repeater in 3GPP release-18. Front Inform Technol Electron Eng, 24(12):1815-1828.

[23]Tang FX, Wen C, Chen XH, et al., 2023. Federated learning for intelligent transmission with space-air-ground integrated network toward 6G. IEEE Netw, 37(2):198-204.

[24]Teklu MB, Choi DY, Meng WX, 2024. Resource efficient full-duplex mode of transmissions under imperfect CSI. IEEE Trans Broadcast, 70(1):87-98.

[25]Tran DD, Sharma SK, Ha VN, et al., 2023. Multi-agent DRL approach for energy-efficient resource allocation in URLLC-enabled grant-free NOMA systems. IEEE Open J Commun Soc, 4:1470-1486.

[26]Uddin R, Kumar SAP, 2023. SDN-based federated learning approach for satellite-IoT framework to enhance data security and privacy in space communication. IEEE J Radio Freq Identif, 7:424-440.

[27]Vishnoi V, Budhiraja I, Gupta S, et al., 2023. A deep reinforcement learning scheme for sum rate and fairness maximization among D2D pairs underlaying cellular network with NOMA. IEEE Trans Veh Technol, 72(10):13506-13522.

[28]Wang Q, Chen XM, Qi Q, 2024. Energy-efficient design of satellite-terrestrial computing in 6G wireless networks. IEEE Trans Commun, 72(3):1759-1772.

[29]Wang ZJ, Gao WF, Li GH, et al., 2024. Path planning for unmanned aerial vehicle via off-policy reinforcement learning with enhanced exploration. IEEE Trans Emerg Top Comput Intell, 8(3):2625-2639.

[30]Wu J, Jia M, Zhang NT, et al., 2024. Multi-agent deep reinforcement learning-based computation offloading in LEO satellite edge computing system. IEEE Commun Lett, 28(10):2352-2356.

[31]Xiao Y, Song YQ, Liu J, 2023. Multi-agent deep reinforcement learning based resource allocation for ultra-reliable low-latency Internet of Controllable Things. IEEE Trans Wirel Commun, 22(8):5414-5430.

[32]Xu HT, Han SY, Li XH, et al., 2023. Anomaly traffic detection based on communication-efficient federated learning in space-air-ground integration network. IEEE Trans Wirel Commun, 22(12):9346-9360.

[33]Xu X, Li RP, Zhao ZF, et al., 2024. The gradient convergence bound of federated multi-agent reinforcement learning with efficient communication. IEEE Trans Wirel Commun, 23(1):507-528.

[34]Yu B, Qian C, Lee J, et al., 2023. Realizing high power full duplex in millimeter wave system: design, prototype and results. IEEE J Sel Areas Commun, 41(9):2893-2906.

[35]Zhao D, Zheng Z, Qi PF, et al., 2024. Resource allocation in multi-user cellular networks: a Transformer-based deep reinforcement learning approach. China Commun, 21(5):77-96.