Abstract: Accurate estimation of the background error covariance matrix denoted as B remains a critical challenge in numerical weather prediction (NWP), directly influencing data assimilation (DA) performance and forecast accuracy. Although hybrid ensemble–variational (EnVar) methods combine static and flow-dependent matrices to improve assimilation, their effectiveness is constrained by empirically fixed weights. To address this limitation, we propose DRL-EnVar, an adaptive hybrid EnVar DA method enhanced with deep reinforcement learning. DRL-EnVar integrates deep learning (DL) components, including a novel cyclic convolution module to extract abstract features from data, and employs reinforcement learning (RL) to dynamically optimize hybrid weighting strategies. The system adaptively combines multiple ensemble-based flow-dependent matrices with one or more static matrices to construct a time-varying hybrid matrix B that better reflects real-time background errors. Experimental results demonstrate that DRL-EnVar performs better than the traditional ensemble Kalman filter (EnKF) and hybrid covariance DA (HCDA) methods, especially under sparse observations or transitional changes in state variables. It achieves competitive or superior assimilation accuracy with lower computational cost, and can be flexibly integrated into both three-dimensional variational assimilation (3DVar) and four-dimensional variational assimilation (4DVar) frameworks. Overall, DRL-EnVar offers a novel and efficient approach to adaptive DA, particularly valuable for improving forecast skill during transitional weather regimes.

DRL-EnVar：基于深度强化学习的自适应混合集合-变分资料同化方法

[1]Arulkumaran K, Deisenroth MP, Brundage M, et al., 2017. Deep reinforcement learning: a brief survey. IEEE Signal Process Mag, 34(6):26-38.

[2]Bannister RN, 2008a. A review of forecast error covariance statistics in atmospheric variational data assimilation. I: characteristics and measurements of forecast error covariances. Quart J Roy Meteor Soc, 134(637):1951-1970.

[3]Bannister RN, 2008b. A review of forecast error covariance statistics in atmospheric variational data assimilation. II: modelling the forecast error covariance statistics. Quart J Roy Meteor Soc, 134(637):1971-1996.

[4]Bannister RN, 2017. A review of operational methods of variational and ensemble-variational data assimilation. Quart J Roy Meteor Soc, 143(703):607-633.

[5]Bellman R, 1957. A Markovian decision process. J Math Mech, 6(5):679-684.

[6]Buehner M, Gauthier P, Liu Z, 2005. Evaluation of new estimates of background- and observation-error covariances for variational assimilation. Quart J Roy Meteor Soc, 131(613):3373-3383.

[7]Chen YD, Guo S, Meng DM, et al., 2020. The impact of optimal selected historical forecasting samples on hybrid ensemble-variational data assimilation. Atmos Res, 242:104980.

[8]Cho K, van Merriënboer B, Gulcehre C, et al., 2014. Learning phrase representations using RNN encoder–decoder for statistical machine translation. Proc Conf on Empirical Methods in Natural Language Processing, p.1724-1734.

[9]Cuomo S, Di Cola VS, Giampaolo F, et al., 2022. Scientific machine learning through physics-informed neural networks: where we are and what’s next. J Sci Comput, 92(3):88.

[10]Ding YH, Chan JCL, 2005. The East Asian summer monsoon: an overview. Meteor Atmos Phys, 89(1):117-142.

[11]Espeholt L, Soyer H, Munos R, et al., 2018. IMPALA: scalable distributed deep-RL with importance weighted actor-learner architectures. Proc 35^th Int Conf on Machine Learning, p.1406-1415.

[12]Gao ZH, Shi XJ, Wang H, et al., 2022. Earthformer: exploring space-time Transformers for Earth system forecasting. Proc 36^th Advances in Neural Information Processing Systems, p.25390-25403.

[13]Gaspari G, Cohn SE, 1999. Construction of correlation functions in two and three dimensions. Quart J Roy Meteor Soc, 125(554):723-757.

[14]Gasperoni NA, Wang XG, Wang YM, 2022. Using a cost-effective approach to increase background ensemble member size within the GSI-based EnVar system for improved radar analyses and forecasts of convective systems. Mon Wea Rev, 150(3):667-689.

[15]Gasperoni NA, Wang XG, Wang YM, 2023. Valid time shifting for an experimental RRFS convection-allowing EnVar data assimilation and forecast system: description and systematic evaluation in real time. Mon Wea Rev, 151(5):1229-1245.

[16]Glorot X, Bordes A, Bengio Y, 2011. Deep sparse rectifier neural networks. Proc 14^th Int Conf on Artificial Intelligence and Statistics, p.315-323.

[17]Gregor K, Danihelka I, Graves A, et al., 2015. DRAW: a recurrent neural network for image generation. Proc 32^nd Int Conf on Machine Learning, p.1462-1471.

[18]Hornik K, Stinchcombe M, White H, 1989. Multilayer feedforward networks are universal approximators. Neur Netw, 2(5):359-366.

[19]Houtekamer PL, Lefaivre L, Derome J, et al., 1996. A system simulation approach to ensemble prediction. Mon Wea Rev, 124(6):1225-1242.

[20]Huang B, Wang XG, 2018. On the use of cost-effective valid-time-shifting (VTS) method to increase ensemble size in the GFS hybrid 4DEnVar system. Mon Wea Rev, 146(9):2973-2998.

[21]Huang LL, Leng HZ, Song JQ, et al., 2025. An adaptive variance adjustment strategy for a static background error covariance matrix—part I: verification in the Lorenz-96 model. Appl Sci, 15(12):6399.

[22]James EP, Alexander CR, Dowell DC, et al., 2022. The high-resolution rapid refresh (HRRR): an hourly updating convection-allowing forecast model. Part II: forecast performance. Wea Forecast, 37(8):1397-1417.

[23]Johnson R, Zhang T, 2017. Deep pyramid convolutional neural networks for text categorization. Proc 55^th Annual Meeting of the Association for Computational Linguistics, p.562-570.

[24]Kaelbling LP, Littman ML, Moore AW, 1996. Reinforcement learning: a survey. J Artif Intell Res, 4:237-285.

[25]Kalman RE, 1960. A new approach to linear filtering and prediction problems. J Basic Eng, 82(1):35-45.

[26]Ketkar N, Moolayil J, 2021. Convolutional neural networks. In: Ketkar N, Moolayil (Eds.), Deep Learning with Python: Learn Best Practices of Deep Learning Models with PyTorch. Apress, Berkeley, CA, USA.

[27]Kurosawa K, Poterjoy J, 2023. A statistical hypothesis testing strategy for adaptively blending particle filters and ensemble Kalman filters for data assimilation. Mon Wea Rev, 151(1):105-125.

[28]Lam R, Sanchez-Gonzalez A, Willson M, et al., 2023. Learning skillful medium-range global weather forecasting. Science, 382(6677):1416-1421.

[29]LeCun Y, Bengio Y, Hinton G, 2015. Deep learning. Nature, 521(7553):436-444.

[30]Leng HZ, Song JQ, Yin FK, et al., 2013. Notes and correspondence on ensemble-based three-dimensional variational filters. J Zhejiang Univ SCIENCE C, 14(8):634-641.

[31]Lorenc AC, 2017. Improving ensemble covariances in hybrid variational data assimilation without increasing ensemble size. Quart J Roy Meteor Soc, 143(703):1062-1072.

[32]Lorenz EN, Emanuel KA, 1998. Optimal sites for supplementary weather observations: simulation with a small model. J Atmos Sci, 55(3):399-414.

[33]Mnih V, Kavukcuoglu K, Silver D, et al., 2015. Human-level control through deep reinforcement learning. Nature, 518(7540):529-533.

[34]Parrish DF, Derber JC, 1992. The national meteorological center’s spectral statistical-interpolation analysis system. Mon Wea Rev, 120(8):1747-1763.

[35]Qi CR, Su H, Mo K, et al., 2017. PointNet: deep learning on point sets for 3D classification and segmentation. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.77-85.

[36]Reichstein M, Camps-Valls G, Stevens B, et al., 2019. Deep learning and process understanding for data-driven Earth system science. Nature, 566(7743):195-204.

[37]Sainath TN, Vinyals O, Senior A, et al., 2015. Convolutional, long short-term memory, fully connected deep neural networks. IEEE Int Conf on Acoustics, Speech and Signal Processing, p.4580-4584.

[38]Sanz-Alonso D, Stuart A, Taeb A, 2023. Inverse problems and data assimilation. Cambridge University Press, New York, USA.

[39]Silver D, Schrittwieser J, Simonyan K, et al., 2017. Mastering the game of Go without human knowledge. Nature, 550(7676):354-359.

[40]Wang CC, Tsai CH, Jou BJD, et al., 2022. Time-lagged ensemble quantitative precipitation forecasts for three landfalling typhoons in the Philippines using the CReSS model, part II: verification using global precipitation measurement retrievals. Remote Sens, 14(20):5126.

[41]Wang YB, Min JZ, Chen YD, et al., 2017. Improving precipitation forecast with hybrid 3DVar and time-lagged ensembles in a heavy rainfall event. Atmos Res, 183:1-16.

[42]Yang Y, Wang XG, 2024. A comparison of 3DEnVar and 4DEnVar for convective-scale direct radar reflectivity data assimilation in the context of a filter and a smoother. Mon Wea Rev, 152(1):59-78.

[43]Yokota S, Banno T, Oigawa M, et al., 2024. JMA operational hourly hybrid 3DVar with singular vector-based mesoscale ensemble prediction system. J Meteor Soc Japan Ser II, 102(2):129-150.