Similar articles

Abstract: Accurate long-term power forecasting is important in the decision-making operation of the power grid and power consumption management of customers to ensure the power system’s reliable power supply and the grid economy’s reliable operation. However, most time-series forecasting models do not perform well in dealing with long-time-series prediction tasks with a large amount of data. To address this challenge, we propose a parallel time-series prediction model called LDformer. First, we combine Informer with long short-term memory (LSTM) to obtain deep representation abilities in the time series. Then, we propose a parallel encoder module to improve the robustness of the model and combine convolutional layers with an attention mechanism to avoid value redundancy in the attention mechanism. Finally, we propose a probabilistic sparse (ProbSparse) self-attention mechanism combined with uniDrop to reduce the computational overhead and mitigate the risk of losing some key connections in the sequence. Experimental results on five datasets show that LDformer outperforms the state-of-the-art methods for most of the cases when handling the different long-time-series prediction tasks.

LDformer：面向长期电力预测的并行神经网络模型

[1]Cao JS, Wang JH, 2019. Stock price forecasting model based on modified convolution neural network and financial time series analysis. Int J Commun Syst, 32(12):e3987.

[2]Chakraborty T, Chattopadhyay S, Ghosh I, 2019. Forecasting dengue epidemics using a hybrid methodology. Phys A Stat Mech Appl, 527:121266.

[3]Chen JF, Wang WM, Huang CM, 1995. Analysis of an adaptive time-series autoregressive moving-average (ARMA) model for short-term load forecasting. Electr Power Syst Res, 34(3):187-196.

[4]Chen XY, Sun LJ, 2022. Bayesian temporal factorization for multidimensional time series prediction. IEEE Trans Patt Anal Mach Intell, 44(9):4659-4673.

[5]Chuku C, Simpasa A, Oduor J, 2019. Intelligent forecasting of economic growth for developing economies. Int Econ, 159:74-93.

[6]Ciechulski T, Osowski S, 2021. High precision LSTM model for short-time load forecasting in power systems. Energies, 14(11):2983.

[7]Ding M, Zhou H, Xie H, et al., 2021. A time series model based on hybrid-kernel least-squares support vector machine for short-term wind power forecasting. ISA Trans, 108:58-68.

[8]Gai SX, Zeng XQ, Yuan TF, 2021. Parking volume forecast of railway station garages based on passenger behaviour analysis using the LSTM network. J Adv Transp, 2021:6688609.

[9]Guo SN, Lin YF, Li SJ, et al., 2019. Deep spatial-temporal 3D convolutional neural networks for traffic data forecasting. IEEE Trans Intell Transp Syst, 20(10):3913-3926.

[10]Han ZY, Zhao J, Leung H, et al., 2021. A review of deep learning models for time series prediction. IEEE Sens J, 21(6):7833-7848.

[11]Hosseini MK, Talebpour A, 2019. Traffic prediction using time-space diagram: a convolutional neural network approach. Transp Res Rec J Transp Res Board, 2673(7):425-435.

[12]Hu R, Chiu YC, Hsieh CW, 2020. Crowding prediction on mass rapid transit systems using a weighted bidirectional recurrent neural network. IET Intell Transp Syst, 14(3):196-203.

[13]Karevan Z, Suykens JAK, 2020. Transductive LSTM for time-series prediction: an application to weather forecasting. Neur Netw, 125:1-9.

[14]Khan MH, Muhammad NS, El-Shafie A, 2020. Wavelet based hybrid ANN-ARIMA models for meteorological drought forecasting. J Hydrol, 590:125380.

[15]Khandelwal U, He H, Qi P, et al., 2018. Sharp nearby, fuzzy far away: how neural language models use context. Proc 56^th Annual Meeting of the Association for Computational Linguistics, p.284-294.

[16]Kim SH, Lee G, Kwon GY, et al., 2018. Deep learning based on multi-decomposition for short-term load forecasting. Energies, 11(12):3433.

[17]Li SY, Jin XY, Xuan Y, et al., 2019. Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting. Proc 33^rd Int Conf on Neural Information Processing Systems, Article 471.

[18]Marcjasz G, Uniejewski B, Weron R, 2020. Probabilistic electricity price forecasting with NARX networks: combine point or probabilistic forecasts?Int J Forecast, 36(2):‍466-479.

[19]Miao KC, Han TT, Yao YQ, et al., 2020. Application of LSTM for short term fog forecasting based on meteorological elements. Neurocomputing, 408:285-291.

[20]Min K, Kim D, Park J, et al., 2019. RNN-based path prediction of obstacle vehicles with deep ensemble. IEEE Trans Veh Technol, 68(10):10252-10256.

[21]Nóbrega JP, Oliveira ALI, 2019. A sequential learning method with Kalman filter and extreme learning machine for regression and time series forecasting. Neurocomputing, 337:235-250.

[22]Ran XD, Shan ZG, Fang YF, et al., 2019. An LSTM-based method with attention mechanism for travel time prediction. Sensors, 19(4):861.

[23]Syriopoulos T, Tsatsaronis M, Karamanos I, 2021. Support vector machine algorithms: an application to ship price forecasting. Comput Econ, 57(1):55-8.

[24]Vaswani A, Shazeer N, Parmar N, et al., 2017. Attention is all you need. Proc 31^st Int Conf on Neural Information Processing Systems, p.6000-6010.

[25]Viccione G, Guarnaccia C, Mancini S, et al., 2020. On the use of ARIMA models for short-term water tank levels forecasting. Water Supply, 20(3):787-799.

[26]Wang C, Tian R, Hu J, et al., 2023. A trend graph attention network for traffic prediction. Inform Sci, 623:275-292.

[27]Wang ZP, Qu JF, Fang XY, et al., 2020. Prediction of early stabilization time of electrolytic capacitor based on ARIMA-Bi_LSTM hybrid model. Neurocomputing, 403:63-79.

[28]Wu N, Green B, Ben X, et al., 2020. Deep transformer models for time series forecasting: the influenza prevalence case. Proc 37^th Int Conf on Machine Learning, p.1-10.

[29]Wu Z, Wu LJ, Meng Q, et al., 2021. UniDrop: a simple yet effective technique to improve transformer without extra cost. Proc Conf of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, p.3865-3878.

[30]Xiao S, Yan JC, Farajtabar M, et al., 2019. Learning time series associated event sequences with recurrent point process networks. IEEE Trans Neur Netw Learn Syst, 30(10):3124-3136.

[31]Xie YY, Lou YS, 2019. Hydrological time series prediction by ARIMA-SVR combined model based on wavelet transform. Proc 3^rd Int Conf on Innovation in Artificial Intelligence, p.243-247.

[32]Xu DW, Wang YD, Jia LM, et al., 2017. Real-time road traffic state prediction based on ARIMA and Kalman filter. Front Inform Technol Electron Eng, 18(2):287-302.

[33]Zhang D, Ling JW, Wei ZH, et al., 2018. Long-term traffic speed prediction based on multiscale spatio-temporal feature learning network. IEEE Trans Intell Transp Syst, 20(10):3700-3709.

[34]Zhang L, Wang G, Giannakis GB, 2019. Real-time power system state estimation and forecasting via deep unrolled neural networks. IEEE Trans Signal Process, 67(15):4069-4077.

[35]Zhang TJ, Song S, Li SG, et al., 2019. Research on gas concentration prediction models based on LSTM multidimensional time series. Energies, 12(1):161.

[36]Zheng JH, Huang MF, 2020. Traffic flow forecast through time series analysis based on deep learning. IEEE Access, 8:82562-82570.

[37]Zhou HY, Zhang SH, Peng JQ, et al., 2021. Informer: beyond efficient transformer for long sequence time-series forecasting. Proc 35^th AAAI Conf on Artificial Intelligence, p.1-15.