Abstract: Due to the openness of the wireless propagation environment, wireless networks are highly susceptible to malicious jamming, which significantly impacts their legitimate communication performance. This study investigates a reconfigurable intelligent surface (RIS) assisted anti-jamming communication system. Specifically, the objective is to enhance the system’s anti-jamming performance by optimizing the transmitting power of the base station and the passive beamforming of the RIS. Taking into account the dynamic and unpredictable nature of a smart jammer, the problem of joint optimization of transmitting power and RIS reflection coefficients is modeled as a Markov decision process (MDP). To tackle the complex and coupled decision problem, we propose a learning framework based on the double deep Q-network (DDQN) to improve the system achievable rate and energy efficiency. Unlike most power-domain jamming mitigation methods that require information on the jamming power, the proposed DDQN algorithm is better able to adapt to dynamic and unknown environments without relying on the prior information about jamming power. Finally, simulation results demonstrate that the proposed algorithm outperforms multi-armed bandit (MAB) and deep Q-network (DQN) schemes in terms of the anti-jamming performance and energy efficiency.

可重构智能表面辅助的联合优化功率控制和被动波束赋形的抗干扰传输方法

[1]Arulkumaran K, Deisenroth MP, Brundage M, et al., 2017. Deep reinforcement learning: a brief survey. IEEE Signal Process Mag, 34(6):26-38.

[2]Basar E, di Renzo M, de Rosny J, et al., 2019. Wireless communications through reconfigurable intelligent surfaces. IEEE Access, 7:116753-116773.

[3]Björnson E, Özdogan Ö, Larsson EG, 2020. Intelligent reflecting surface versus decode-and-forward: how large surfaces are needed to beat relaying? IEEE Wirel Commun Lett, 9(2):244-248.

[4]Chang GY, Wang SY, Liu YX, 2017. A jamming-resistant channel hopping scheme for cognitive radio networks. IEEE Trans Wirel Commun, 16(10):6712-6725.

[5]di Renzo M, Zappone A, Debbah M, et al., 2020. Smart radio environments empowered by reconfigurable intelligent surfaces: how it works, state of research, and the road ahead. IEEE J Select Areas Commun, 38(11):2450-2525.

[6]Feng ZB, Ren GC, Chen J, et al., 2019. Power control in relay-assisted anti-jamming systems: a Bayesian three-layer Stackelberg game approach. IEEE Access, 7:14623-14636.

[7]Feng ZB, Luo YJ, Chen XQ, et al., 2020. A MAB-based discrete power control approach in anti-jamming relay communication via three-layer Stackelberg game. Proc 6^th Int Conf on Computer and Communications, p.267-272.

[8]Geng SQ, Li PK, Yin XZ, et al., 2022. The study on anti-jamming power control strategy based on Q-learning. Proc 7^th Int Conf on Intelligent Computing and Signal Processing, p.182-185.

[9]Guo HY, Liang YC, Chen J, et al., 2020. Weighted sum-rate maximization for reconfigurable intelligent surface aided wireless networks. IEEE Trans Wirel Commun, 19(5):3064-3076.

[10]He KM, Zhang XY, Ren SQ, et al., 2016. Deep residual learning for image recognition. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.770-778.

[11]Huang CW, Zappone A, Alexandropoulos GC, et al., 2019. Reconfigurable intelligent surfaces for energy efficiency in wireless communication. IEEE Trans Wirel Commun, 18(8):4157-4170.

[12]Jian MN, Alexandropoulos GC, Basar E, et al., 2022. Reconfigurable intelligent surfaces for wireless communications: overview of hardware designs, channel models, and estimation techniques. Intell Converg Netw, 3(1):1-32.

[13]Khawaja W, Ozdemir O, Yapici Y, et al., 2020. Coverage enhancement for NLOS mmWave links using passive reflectors. IEEE Open J Commun Soc, 1:263-281.

[14]Li XC, Chen JN, Ling X, et al., 2023. Deep reinforcement learning-based anti-jamming algorithm using dual action network. IEEE Trans Wirel Commun, 22(7):4625-4637.

[15]Luong NC, Hoang DT, Gong SM, et al., 2019. Applications of deep reinforcement learning in communications and networking: a survey. IEEE Commun Surv Tut, 21(4):3133-3174.

[16]Lyu L, Shen Y, Zhang SC, 2022. The advance of reinforcement learning and deep reinforcement learning. Proc IEEE Int Conf on Electrical Engineering, Big Data and Algorithms, p.644-648.

[17]Ma N, Xu K, Xia XC, et al., 2022. Reinforcement learning-based dynamic anti-jamming power control in UAV networks: an effective jamming signal strength based approach. IEEE Commun Lett, 26(10):2355-2359.

[18]Mnih V, Kavukcuoglu K, Silver D, et al., 2015. Human-level control through deep reinforcement learning. Nature, 518(7540):529-533.

[19]Ning WL, Huang XY, Yang K, et al., 2020. Reinforcement learning enabled cooperative spectrum sensing in cognitive radio networks. J Commun Netw, 22(1):12-22.

[20]Pirayesh H, Zeng HC, 2022. Jamming attacks and anti-jamming strategies in wireless networks: a comprehensive survey. IEEE Commun Surv Tut, 24(2):767-809.

[21]Ramachandran P, Zoph B, Le QV, 2017. Searching for activation functions. https://arxiv.org/abs/1710.05941

[22]Sharma H, Kumar N, Tekchandani R, 2023. Mitigating jamming attack in 5G heterogeneous networks: a federated deep reinforcement learning approach. IEEE Trans Veh Technol, 72(2):2439-2452.

[23]Shen ZX, Xu K, Xia XC, 2021. 2D fingerprinting-based localization for mmWave cell-free massive MIMO systems. IEEE Commun Lett, 25(11):3556-3560.

[24]Summers TA, Wilson SG, 1998. SNR mismatch and online estimation in turbo decoding. IEEE Trans Commun, 46(4):421-423.

[25]Sun YF, An K, Luo JS, et al., 2021. Intelligent reflecting surface enhanced secure transmission against both jamming and eavesdropping attacks. IEEE Trans Veh Technol, 70(10):11017-11022.

[26]Sutton RS, Barto AG, 2018. Reinforcement Learning: an Introduction. MIT Press, Cambridge, USA.

[27]Takizawa K, Sasaki S, Zhou J, et al., 2002. Online SNR estimation for parallel combinatorial SS systems in Nakagami fading channels. Proc Global Telecommunications Conf, p.1239-1243.

[28]Tang X, Wang DW, Zhang RN, et al., 2021. Jamming mitigation via aerial reconfigurable intelligent surface: passive beamforming and deployment optimization. IEEE Trans Veh Technol, 70(6):6232-6237.

[29]van Hasselt H, Guez A, Silver D, 2016. Deep reinforcement learning with double Q-learning. Proc 30^th AAAI Conf on Artificial Intelligence, p.2094-2100.

[30]Wang PL, Fang J, Yuan XJ, et al., 2020. Intelligent reflecting surface-assisted millimeter wave communications: joint active and passive precoding design. IEEE Trans Veh Technol, 69(12):14960-14973.

[31]Wang W, Zhang W, 2021. Joint beam training and positioning for intelligent reflecting surfaces assisted millimeter wave communications. IEEE Trans Wirel Commun, 20(10):6282-6297.

[32]Wang W, Zhang W, 2022a. Intelligent reflecting surface configurations for smart radio using deep reinforcement learning. IEEE J Select Areas Commun, 40(8):2335-2346.

[33]Wang W, Zhang W, 2022b. Jittering effects analysis and beam training design for UAV millimeter wave communications. IEEE Trans Wirel Commun, 21(5):3131-3146.

[34]Wei L, Huang CW, Alexandropoulos GC, et al., 2021. Channel estimation for RIS-empowered multi-user MISO wireless communications. IEEE Trans Commun, 69(6):4144-4157.

[35]Wu QQ, Zhang R, 2019. Intelligent reflecting surface enhanced wireless network via joint active and passive beamforming. IEEE Trans Wirel Commun, 18(11):5394-5409.

[36]Wu QQ, Zhang R, 2020. Towards smart and reconfigurable environment: intelligent reflecting surface aided wireless network. IEEE Commun Mag, 58(1):106-112.

[37]Xiao L, Hong SY, Xu SY, et al., 2022. IRS-aided energy-efficient secure WBAN transmission based on deep reinforcement learning. IEEE Trans Commun, 70(6):4162-4174.

[38]Xiao ZC, Gao B, Liu SC, et al., 2018. Learning based power control for mmWave massive MIMO against jamming. Proc IEEE Global Communications Conf, p.1-6.

[39]Xu JD, Yuen C, Huang CW, et al., 2023. Reconfiguring wireless environments via intelligent surfaces for 6G: reflection, modulation, and security. Sci China Inf Sci, 66(3):130304.

[40]Xu JW, Wang KH, Zhang X, et al., 2021. Anti-jamming strategy based on game theory in single-channel UAV communication network. Proc 6^th Int Conf on Fog and Mobile Edge Computing, p.1-7.

[41]Yang HL, Xiong ZH, Zhao J, et al., 2020. Intelligent reflecting surface assisted anti-jamming communications based on reinforcement learning. Proc IEEE Global Communications Conf, p.1-6.

[42]Yang HL, Xiong ZH, Zhao J, et al., 2021a. Deep reinforcement learning-based intelligent reflecting surface for secure wireless communications. IEEE Trans Wirel Commun, 20(1):375-388.

[43]Yang HL, Xiong ZH, Zhao J, et al., 2021b. Intelligent reflecting surface assisted anti-jamming communications: a fast reinforcement learning approach. IEEE Trans Wirel Commun, 20(3):1963-1974.

[44]Yu L, Li YS, Pan C, et al., 2017. Anti-jamming power control game for data packets transmission. Proc 17^th Int Conf on Communication Technology, p.1255-1259.

[45]Zhang SW, Zhang R, 2020. Capacity characterization for intelligent reflecting surface aided MIMO communication. IEEE J Select Areas Commun, 38(8):1823-1838.

[46]Zhang ZD, Zhang DX, Qiu RC, 2020. Deep reinforcement learning for power system applications: an overview. CSEE J Power Energy Syst, 6(1):213-225.