Abstract: Millimeter-wave (mmWave) communication is the key to increasing the demand for high data rates and low latency resulting from the rapid evolution of wireless communications, especially in the fifth generation (5G) of wireless communication systems and beyond. The mmWave band suffers from high path loss and obstacle blockage, significantly reducing the transmission range. Note that high-directional beams are required to perform well in the mmWave band. Hence, beam alignment is crucial for high-data-rate transmission between the transmitter (Tx) and the receiver (Rx). One of the drawbacks is getting an accurate beam alignment when the transceiver (Tx, Rx, or both) is mobile. Beam tracking plays a considerable role in 5G communications, especially in vehicular communications, due to the repeated change of the transceiver (Tx, Rx, or both) position. This work presents an overview of the different beam-tracking methods used in mmWave communications, focusing on hybrid beamforming techniques. We also compare the various tracking techniques in a recommendation table. This overview suggests that some tracking methods used in the sub-6-GHz band, such as least mean squares (LMS), recursive least squares (RLS), and Kalman filter, are unsuitable for the mmWave band (due to higher frequency and shorter coherence time), and it recommends faster tracking strategies.

面向毫米波无线通信的波束跟踪技术综述

[1]Abualhayja’a M, Hussein M, 2021. Comparative study of adaptive beamforming algorithms for smart antenna applications. Int Conf on Communications, Signal Processing, and their Applications, p.1-5.

[2]Ali A, González-Prelcic N, Ghosh A, 2020. Passive radar at the roadside unit to configure millimeter wave vehicle-to-infrastructure links. IEEE Trans Veh Technol, 69(12):14903-14917.

[3]Al-Ibadi M, Mahmood FE, 2022. Beam and channel tracking for 5G communication systems using adaptive filtering techniques: a comparison study. J Commun Softw Syst, 18(3):244-251.

[4]AlMahamid F, Grolinger K, 2022. Autonomous unmanned aerial vehicle navigation using reinforcement learning: a systematic review. Eng Appl Artif Intel, 115:105321.

[5]Anooz RSA, Pourrostam J, 2023. Beam alignment techniques classifications for mm-wave communications: a short review. AIP Conf Proc, 2977(1):020046.

[6]Anooz RSA, Pourrostam J, Al-Ibadi M, 2024a. Adaptive filters versus machine learning based beam tracking techniques for millimeter-wave wireless communications systems. IEEE Access, 12:165878-165888.

[7]Anooz RSA, Pourrostam J, Al-Ibadi M, 2024b. Performance evaluation of 2D and 3D beam and channel tracking using adaptive filtering techniques. Iran J Sci Technol Trans Electr Eng, 48(3):1361-1372.

[8]Anschel O, Baram N, Shimkin N, 2017. Averaged-DQN: variance reduction and stabilization for deep reinforcement learning. Proc 34^th Int Conf on Machine Learning p.176-185.

[9]Asi BA, Mohmood FE, AL-Ibadi MA, 2021. Beam tracking using least mean squares algorithm for single-cell massive MIMO communication system. IOP Conf Ser Mater Sci Eng, 1152:012004.

[10]Bae J, Lim SH, Yoo JH, et al., 2017. New beam tracking technique for millimeter wave-band communications. https://arxiv.org/abs/1702.00276

[11]Banday Y, Mohammad Rather G, Begh GR, 2019. Effect of atmospheric absorption on millimetre wave frequencies for 5G cellular networks. IET Commun, 13(3):265-270.

[12]Barati CN, Hosseini SA, Rangan S, 2015. Directional cell discovery in millimeter wave cellular networks. IEEE Trans Wirel Commun, 14(12):6664-6678.

[13]Burghal D, Abbasi NA, Molisch AF, 2019. A machine learning solution for beam tracking in mmwave systems. 53^rd Asilomar Conf on Signals, Systems, and Computers, p.173-177.

[14]Chen S, Samingan AK, Hanzo L, 2001. Support vector machine multiuser receiver for DS-CDMA signals in multipath channels. IEEE Trans Neur Netw, 12(3):604-611.

[15]Chen YJ, Chang DK, Zhang C, 2020. Autonomous tracking using a swarm of UAVs: a constrained multi-agent reinforcement learning approach. IEEE Trans Veh Technol, 69(11):13702-13717.

[16]Chiang HL, Chen KC, Rave W, 2021. Machine-learning beam tracking and weight optimization for mmWave multi-UAV links. IEEE Trans Wirel Commun, 20(8):5481-5494.

[17]Ciaramitaro G, Brambilla M, Nicoli M, 2023. Signalling design in sensor-assisted mmWave communications for cooperative driving. IEEE Open J Intell Transp Syst, 4:493-505.

[18]Cord M, Cunningham P, 2008. Machine Learning Techniques for Multimedia: Case Studies on Organization and Retrieval. Springer, Berlin, Germany.

[19]Dash M, Liu H, Yao J, 1997. Dimensionality reduction of unsupervised data. Proc 9^th IEEE Int Conf on Tools with Artificial Intelligence, p.532-539.

[20]De Donno D, Palacios J, Widmer J, 2017. Millimeter-wave beam training acceleration through low-complexity hybrid transceivers. IEEE Trans Wirel Commun, 16(6):3646-3660.

[21]Dehkordi SK, Kobayashi M, Caire G, 2021. Adaptive beam tracking based on recurrent neural networks for mmWave channels. 22^nd Int Workshop on Signal Processing Advances in Wireless Communications, p.1-5.

[22]El Ayach O, Rajagopal S, Abu-Surra S, 2014. Spatially sparse precoding in millimeter wave MIMO systems. IEEE Trans Wirel Commun, 13(3):1499-1513.

[23]Elbir AM, Mishra KV, 2020. Joint antenna selection and hybrid beamformer design using unquantized and quantized deep learning networks. IEEE Trans Wirel Commun, 19(3):1677-1688.

[24]Elsayed M, Erol-Kantarci M, Yanikomeroglu H, 2021. Transfer reinforcement learning for 5G new radio mmWave networks. IEEE Trans Wirel Commun, 20(5):2838-2849.

[25]Fodor IK, 2002. A survey of dimension reduction techniques. Meat Sci, 9:10-20.

[26]Foster DJ, Morris RG, Dayan P, 2000. A model of hippocampally dependent navigation, using the temporal difference learning rule. Hippocampus, 10(1):1-16.

[27]Gaudes CC, Santamaria I, Via J, 2007. Robust array beamforming with sidelobe control using support vector machines. IEEE Trans Signal Process, 55(2):574-584.

[28]Giordani M, Zanella A, Zorzi M, 2017. Millimeter wave communication in vehicular networks: challenges and opportunities. 6^th Int Conf on Modern Circuits and Systems Technologies, p.1-6.

[29]Goodfellow I, Bengio Y, 2016. Deep Learning. MIT Press, Cambridge, USA.

[30]Guo J, 2013. Backpropagation through time. https://docslib.org/doc/5485773/backpropagation-through-time

[31]Guo Y, Wang Z, Li M, et al., 2019. Machine learning based mmWave channel tracking in vehicular scenario. IEEE Int Conf on Communications Workshops, p.1-6.

[32]Gustafsson F, Hendeby G, 2012. Some relations between extended and unscented Kalman filters. IEEE Trans Signal Process, 60(2):545-555.

[33]Haghighatshoar S, Caire G, 2016. The beam alignment problem in mmWave wireless networks. 50^th Asilomar Conf on Signals, Systems and Computers, p.741-745.

[34]Han C, Duan S, 2019. Impact of atmospheric parameters on the propagated signal power of millimeter-wave bands based on real measurement data. IEEE Access, 7:113626-113641.

[35]Han C, Zhou S, Xiao L, 2019. A new multipath beam tracking algorithm for mmWave mobile communications. Proc Int Symp on Signal Processing Systems, p.61-66.

[36]Han Z, Yang Y, Wang W, 2022. RSSI map-based trajectory design for UGV against malicious radio source: a reinforcement learning approach. IEEE Trans Intell Transp Syst, 24(4):4641-4650.

[37]Hao L, Shu-kui Z, 2019. Moving object tracking algorithm based on improved Gaussian mixture model. 3^rd Int Conf on Electronic Information Technology and Computer Engineering, p.271-275.

[38]Hashemi M, Sabharwal A, Koksal CE, 2018. Efficient beam alignment in millimeter wave systems using contextual bandits. IEEE Conf on Computer Communications, p.2393-2401.

[39]Hassan K, Masarra M, Zwingelstein M, 2020. Channel estimation techniques for millimeter-wave communication systems: achievements and challenges. IEEE Open J Commun Soc, 1:1336-1363.

[40]Haykin S, 2013. Adaptive Filter Theory. Prentice Hall.

[41]He K, Zhang X, Ren S, 2015. Delving deep into rectifiers: surpassing human-level performance on ImageNet classification. IEEE Int Conf on Computer Vision, p.1026-1034.

[42]Heath RW, González-Prelcic N, Rangan S, 2016. An overview of signal processing techniques for millimeter wave MIMO systems. IEEE J Sel Top Signal Process, 10(3):436-453.

[43]Hochreiter S, 1998. The vanishing gradient problem during learning recurrent neural nets and problem solutions. Int J Unc Fuzz Knowl-Based Syst, 6(02):107-116.

[44]Huang H, Peng Y, Yang J, 2020. Fast beamforming design via deep learning. IEEE Trans Veh Technol, 69(1):1065-1069.

[45]Hur S, Kim T, Love DJ, 2013. Millimeter wave beamforming for wireless backhaul and access in small cell networks. IEEE Trans Commun, 61(10):4391-4403.

[46]Jeong MH, Lee TY, Jeon SB, 2021. Highway speed prediction using gated recurrent unit neural networks. Appl Sci, 11(7):3059.

[47]Jiang H, Gui R, Chen Z, 2019. An improved Sarsa(λ) reinforcement learning algorithm for wireless communication systems. IEEE Access, 7:115418-115427.

[48]Jin W, Zhang J, Wen CK, 2024. Low-complexity joint beamforming for RIS-assisted MU-MISO systems based on model-driven deep learning. IEEE Trans Wirel Commun, 23(7):6968-6982.

[49]Jog S, Wang J, Guan J, 2019. Many-to-many beam alignment in millimeter wave networks. 16^th USENIX Symp on Networked Systems Design and Implementation, p.783-800.

[50]Jolliffe IT, 2002. Principal component analysis for special types of data. In: Principal Component Analysis. Springer Series in Statistics. Springer, New York, NY, p.199-222.

[51]Kasun LLC, Yang Y, Huang GB, 2016. Dimension reduction with extreme learning machine. IEEE Trans Image Process, 25(8):3906-3918.

[52]Khan F, Pi J, 2011. mmWave mobile broadband (MMB): unleashing the 3–300GHz spectrum. 34^th IEEE Sarnoff Symp, p.1-6.

[53]Khan MM, Hossain S, Mozumdar P, 2022. A review on machine learning and deep learning for various antenna design applications. Heliyon, 8(4):e09317.

[54]Kim S, Han H, Kim N, 2019. Robust beam tracking algorithm for mmWave MIMO systems in mobile environments. IEEE 90^th Vehicular Technology Conf, p.1-5.

[55]Kim S, Kwon G, Park H, 2020. Q-learning-based low complexity beam tracking for mmWave beamforming system. Int Conf on Information and Communication Technology Convergence, p.1451-1455.

[56]Kokshoorn M, Chen H, Li Y, 2016. RACE: a rate adaptive channel estimation approach for millimeter wave MIMO systems. IEEE Global Communications Conf, p.1-6.

[57]Krizhevsky A, Sutskever I, Hinton GE, 2017. ImageNet classification with deep convolutional neural networks. Commun ACM, 60(6):84-90.

[58]Lad N, 2023. Graph neural networks for fast track finding.

[59]Li Y, Lu Y, Ai B, 2024. GNN-based beamforming for sum-rate maximization in MU-MISO networks. IEEE Trans Wirel Commun, 23(8):9251-9264.

[60]Liao X, Wu D, Wang Y, 2020. Dynamic spectrum access based on improved SARSA algorithm. IOP Conf Ser Mater Sci Eng, 768:072015.

[61]Lillicrap TP, Hunt JJ, Pritzel A, 2016. Continuous control with deep reinforcement learning. 4^th Int Conf on Learning Representations.

[62]Lim SH, Kim S, Shim B, 2020. Efficient beam training and sparse channel estimation for millimeter wave communications under mobility. IEEE Trans Commun, 68(10):6583-6596.

[63]Lim SH, Kim S, Shim B, 2021. Deep learning-based beam tracking for millimeter-wave communications under mobility. IEEE Trans Commun, 69(11):7458-7469.

[64]Liu C, Li M, Zhao L, 2021. Robust adaptive beam tracking for mobile millimetre wave communications. IEEE Trans Wirel Commun, 20(3):1918-1934.

[65]Liu F, Zhao P, Wang Z, 2019. EKF-based beam tracking for mmWave MIMO systems. IEEE Commun Lett, 23(12):2390-2393.

[66]Liu G, Zhou F, Yu P, 2020. Beam tracking based on unscented Kalman filter theory in millimeter wave communication systems for IoT. Int Wireless Communications and Mobile Computing, p.499-504.

[67]Liu Q, Zhang Z, Yang G, 2022. CSI feedback based on complex neural network for massive MIMO systems. IEEE Access, 10:78414-78422.

[68]Lu Z, Chen X, Wu J, 2023. Application of graph neural networks in dark photon search with visible decays at future beam dump experiment. 3^rd BenchCouncil Int Symp on Intelligent Computers, Algorithms, and Applications, p.254-263.

[69]Mavromatis I, Tassi A, Piechocki RJ, et al., 2017. mmWave system for future ITS: a MAC-layer approach for V2X beam steering. IEEE 86^th Vehicular Technology Conf, p.1-6.

[70]Mazin A, Elkourdi M, Gitlin RD, 2018. Comparative performance analysis of beam sweeping using a deep neural net and random starting point in mmWave 5G new radio. 9^th IEEE Annual Ubiquitous Computing, Electronics & Mobile Communication Conf, p.451-456.

[71]Mensch A, Blondel M, 2018. Differentiable dynamic programming for structured prediction and attention. Proc 35^th Int Conf on Machine Learning, p.3462-3471.

[72]Mizmizi M, Linsalata F, Brambilla M, 2022. Fastening the initial access in 5G NR sidelink for 6G V2X networks. Veh Commun, 33:100402.

[73]Mohamed Z, Ez-Zazi I, Arioua M, 2019. An overview of millimeter waves challenges in 5G vehicle-to-everything networks. https://hal.science/hal-02296900v1

[74]Molisch AF, Karttunen A, Wang R, 2016. Millimeter-wave channels in urban environments. 10^th European Conf on Antennas and Propagation, p.1-5.

[75]Mollel MS, Abubakar AI, Ozturk M, 2021. A survey of machine learning applications to handover management in 5G and beyond. IEEE Access, 9:45770-45802.

[76]Montella C, 2011. The Kalman filter and related algorithms: a literature review. https://www.researchgate.net/publication/236897001

[77]Naeem S, Ali A, Anam S, 2023. An unsupervised machine learning algorithms: comprehensive review. Int J Comput Digit Syst, 13(1):911-921.

[78]Narengerile, Thompson J, Patras P, 2022. Deep reinforcement learning-based beam training with energy and spectral efficiency maximisation for millimetre-wave channels. EURASIP J Wirel Commun Netw, 2022(1):110.

[79]Nawaz S, Hassan SA, 2020. Optimal beam separation in auxiliary beam pair-based initial access in mmWave D2D networks. IEEE 91^st Vehicular Technology Conf, p.1-5.

[80]Nawaz S, Hassan SA, Jung H, 2020. Auxiliary beam pair enabled initial access for mmWave D2D networks. Phys Commun, 39:101039.

[81]Niu Y, Li Y, Jin D, 2015. A survey of millimeter wave communications (mmWave) for 5G: opportunities and challenges. Wirel Netw, 21(8):2657-2676.

[82]Palacios J, De Donno D, Widmer J, 2017. Tracking mm-Wave channel dynamics: fast beam training strategies under mobility. IEEE Conf on Computer Communications, p.1-9.

[83]Pasek P, Kaniewski P, 2020. Unscented Kalman filter application in personal navigation. Radioelectronic Systems Conf, Article 114421C.

[84]Perfecto C, Del Ser J, Bennis M, 2017. Millimeter-wave V2V communications: distributed association and beam alignment. IEEE J Select Areas Commun, 35(9):2148-2162.

[85]Rangan S, Rappaport TS, Erkip E, 2014. Millimeter-wave cellular wireless networks: potentials and challenges. Proc IEEE, 102(3):366-385.

[86]Rappaport TS, Ben-Dor E, Murdock JN, et al., 2012. 38 GHz and 60 GHz angle-dependent propagation for cellular & peer-to-peer wireless communications. IEEE Int Conf on Communications, p.4568-4573.

[87]Rappaport TS, Sun S, Mayzus R, 2013. Millimeter wave mobile communications for 5G cellular: it will work! IEEE Access, 1:335-349.

[88]Rappaport TS, Xing Y, MacCartney GR, et al., 2017. Overview of millimeter wave communications for fifth-generation (5G) wireless networks—with a focus on propagation models. IEEE Trans Antenn Propag, 65(12):6213-6230.

[89]Rhudy MB, Salguero RA, Holappa K, 2017. A Kalman filtering tutorial for undergraduate students. Int J Comput Sci Eng Surv, 8(1):1-9.

[90]Ristic B, Arulampalam S, Gordon N, 2004. Beyond the Kalman Filter-Particle Filters for Tracking Applications. Artech House.

[91]Samimi MK, MacCartney GR, Sun S, 2016. 28 GHz millimeter-wave ultrawideband small-scale fading models in wireless channels. IEEE 83^rd Vehicular Technology Conf, p.1-6.

[92]Santamaria I, Via J, Gaudes C, 2004. Robust blind identification of SIMO channels: a support vector regression approach. IEEE Int Conf on Acoustics, Speech, and Signal Processing, Article V-673.

[93]Satyanarayana K, El-Hajjar M, Mourad AA, 2019. Deep learning aided fingerprint-based beam alignment for mmWave vehicular communication. IEEE Trans Veh Technol, 68(11):10858-10871.

[94]Shah SHA, Rangan S, 2022. Multi-cell multi-beam prediction using auto-encoder LSTM for mmWave systems. IEEE Trans Wirel Commun, 21(12):10366-10380.

[95]Shaham S, Kokshoorn M, Ding M, 2020. Extended Kalman filter beam tracking for millimeter wave vehicular communications. IEEE Int Conf on Communications Workshops, p.1-6.

[96]Shalal AA, Shareef OA, Taher HB, 2023. A comprehensive overview of machine learning-assisted antenna for modern wireless communication. Al-Salam J Eng Technol, 3(1):37-59.

[97]Shen LH, Chang TW, Feng KT, 2021. Design and implementation for deep learning based adjustable beamforming training for millimeter wave communication systems. IEEE Trans Veh Technol, 70(3):2413-2427.

[98]Shokri-Ghadikolaei H, Fischione C, Fodor G, 2015. Millimeter wave cellular networks: a MAC layer perspective. IEEE Trans Commun, 63(10):3437-3458.

[99]Simon D, 2006. Optimal State Estimation: Kalman, H_∞, and Nonlinear Approaches. John Wiley & Sons, Hoboken, USA.

[100]Simonyan K, Zisserman A, 2015. Very deep convolutional networks for large-scale image recognition. 3^rd Int Conf on Learning Representations.

[101]Singh S, Ziliotto F, Madhow U, 2009. Blockage and directivity in 60 GHz wireless personal area networks: from cross-layer model to multihop MAC design. IEEE J Select Areas Commun, 27(8):1400-1413.

[102]Srinivasan, 2020. Beam Prediction Using Deep Learning Methods for mmWave Communication. MS Thesis, California State University, Sacramento, USA.

[103]Tagliaferri D, Brambilla M, Nicoli M, 2021. Sensor-aided beamwidth and power control for next generation vehicular communications. IEEE Access, 9:56301-56317.

[104]Tan M, Le Q, 2019. EfficientNet: rethinking model scaling for convolutional neural networks. Proc 36^th Int Conf on Machine Learning, p.6105-6114.

[105]Tao J, Chen J, Xing J, 2020. Autoencoder neural network based intelligent hybrid beamforming design for mmWave massive MIMO systems. IEEE Trans Cogn Commun Netw, 6(3):1019-1030.

[106]Torkzaban N, Khojastepour MA, Baras JS, 2022. Multi-user beam alignment in presence of multi-path. 56^th Annual Conf on Information Sciences and Systems, p.224-229.

[107]Usama M, Qadir J, Raza A, 2019. Unsupervised machine learning for networking: techniques, applications, and research challenges. IEEE Access, 7:65579-65615.

[108]Va V, 2018. Beam Alignment for Millimeter Wave Vehicular Communications. PhD Dissemination, The University of Texas at Austin, Austin, USA.

[109]Va V, Vikalo H, Heath RW, 2016. Beam tracking for mobile millimeter wave communication systems. IEEE Global Conf on Signal and Information Processing, p.743-747.

[110]Vodopivec T, Samothrakis S, Ster B, 2017. On Monte Carlo tree search and reinforcement learning. J Artif Intell Res, 60:881-936.

[111]Wang A, Yin R, Yu G, 2019. Adaptive PCA based channel estimation and tracking for URA massive MIMO systems. IEEE Int Conf on Communications, p.1-6.

[112]Wang CX, Huang J, Wang H, 2020. 6G wireless channel measurements and models: trends and challenges. IEEE Veh Technol Mag, 15(4):22-32.

[113]Wang J, Lan Z, Pyo CW, 2009. Beam codebook based beamforming protocol for multi-Gbps millimeter-wave WPAN systems. IEEE J Select Areas Commun, 27(8):1390-1399.

[114]Wang P, Li Y, Song L, 2015. Multi-gigabit millimeter wave wireless communications for 5G: from fixed access to cellular networks. IEEE Commun Mag, 53(1):168-178.

[115]Wang R, Klaine PV, Onireti O, 2021. Deep learning enabled beam tracking for non-line of sight millimeter wave communications. IEEE Open J Commun Soc, 2:1710-1720.

[116]Wang X, Chen Z, Jiang B, 2022. Beyond greedy search: tracking by multi-agent reinforcement learning-based beam search. IEEE Trans Image Process, 31:6239-6254.

[117]Wang Y, Klautau A, Ribero M, 2019. mmWave vehicular beam selection with situational awareness using machine learning. IEEE Access, 7:87479-87493.

[118]Ward IR, Joyner J, Lickfold C, 2022. A practical tutorial on graph neural networks. ACM Comput Surv, 54(10s):205.

[119]Wu S, Ma B, Sun Z, 2021. Dynamic beam tracking for mobile users. J Phys Conf Ser, 1815(1):012025.

[120]Wu W, Cheng N, Zhang N, 2019. Fast mmwave beam alignment via correlated bandit learning. IEEE Trans Wirel Commun, 18(12):5894-5908.

[121]Yapıcı Y, 2018. Low-complexity adaptive beam and channel tracking for mobile mmWave communications. 52^nd Asilomar Conf on Signals, Systems, and Computers, p.572-576.

[122]Yu L, Shao X, Wei Y, 2018. Intelligent land-vehicle model transfer trajectory planning method based on deep reinforcement learning. Sensors, 18(9):2905.

[123]Zhang C, Guo D, Fan P, 2016. Tracking angles of departure and arrival in a mobile millimeter wave channel. IEEE Int Conf on Communications, p.1-6.

[124]Zhang J, Yu J, Tao D, 2018. Local deep-feature alignment for unsupervised dimension reduction. IEEE Trans Image Process, 27(5):2420-2432.

[125]Zhang J, Huang Y, Wang J, 2021. Intelligent interactive beam training for millimeter wave communications. IEEE Trans Wirel Commun, 20(3):2034-2048.

[126]Zhang L, Zhao H, Hou S, 2019. A survey on 5G millimeter wave communications for UAV-assisted wireless networks. IEEE Access, 7:117460-117504.

[127]Zhang L, Zhong W, Zhang J, 2022. mmWave beam tracking for V2I communication systems based on spectrum environment awareness. Symmetry, 14(4):677.

[128]Zhang Z, 1997. Parameter estimation techniques: a tutorial with application to conic fitting. Image Vision Comput, 15(1):59-76.

[129]Zhu D, Choi J, Heath RW, 2017. Auxiliary beam pair enabled AoD and AoA estimation in closed-loop large-scale millimeter-wave MIMO systems. IEEE Trans Wirel Commun, 16(7):4770-4785.