Abstract: Diffusion models, a family of generative models based on deep learning, have become increasingly prominent in cutting-edge machine learning research. With distinguished performance in generating samples that resemble the observed data, diffusion models are widely used in image, video, and text synthesis nowadays. In recent years, the concept of diffusion has been extended to time-series applications, and many powerful models have been developed. Considering the deficiency of a methodical summary and discourse on these models, we provide this survey as an elementary resource for new researchers in this area and to provide inspiration to motivate future research. For better understanding, we include an introduction about the basics of diffusion models. Except for this, we primarily focus on diffusion-based methods for time-series forecasting, imputation, and generation, and present them, separately, in three individual sections. We also compare different methods for the same application and highlight their connections if applicable. Finally, we conclude with the common limitation of diffusion-based methods and highlight potential future research directions.

扩散模型在时间序列的应用综述

[1]Alcaraz JML, Strodthoff N, 2023. Diffusion-based time series imputation and forecasting with structured state space models. Trans Mach Learn Res.

[2]Anderson BDO, 1982. Reverse-time diffusion equation models. Stoch Process Their Appl, 12(3):313-326.

[3]Austin J, Johnson DD, Ho J, et al., 2021. Structured denoising diffusion models in discrete state-spaces. Proc 35^th Conf on Neural Information Processing Systems, p.17981-17993.

[4]Biloš M, Rasul K, Schneider A, et al., 2023. Modeling temporal data as continuous functions with stochastic process diffusion. Proc 40^th Int Conf on Machine Learning, p.2452-2470.

[5]Cao W, Wang D, Li J, et al., 2018. BRITS: bidirectional recurrent imputation for time series. Proc 32^nd Conf on Neural Information Processing Systems, p.6776-6786.

[6]Capel EH, Dumas J, 2023. Denoising diffusion probabilistic models for probabilistic energy forecasting.

[7]Chang P, Li HY, Quan SF, et al., 2023. TDSTF: transformer-based diffusion probabilistic model for sparse time series forecasting.

[8]Che ZP, Purushotham S, Cho K, et al., 2018. Recurrent neural networks for multivariate time series with missing values. Sci Rep, 8(1):6085.

[9]Chen T, 2023. On the importance of noise scheduling for diffusion models.

[10]Choi J, Choi H, Hwang J, et al., 2022. Graph neural controlled differential equations for traffic forecasting. Proc 36^th AAAI Conf on Artificial Intelligence, p.6367-6374.

[11]Chu C, Minami K, Fukumizu K, 2020. Smoothness and stability in GANs. Proc 8^th Int Conf on Learning Representations.

[12]Chung J, Gulcehre C, Bengio K, et al., 2014. Empirical evaluation of gated recurrent neural networks on sequence modeling.

[13]Coletta A, Gopalakrishan S, Borrajo D, et al., 2023. On the constrained time-series generation problem.

[14]Croitoru FA, Hondru V, Ionescu RT, et al., 2023. Diffusion models in vision: a survey. IEEE Trans Patt Anal Mach Intell, 45(9):10850-10869.

[15]Desai A, Freeman C, Wang ZH, et al., 2021. TimeVAE: a variational auto-encoder for multivariate time series generation.

[16]Dhariwal P, Nichol A, 2021. Diffusion models beat GANs on image synthesis. Proc 35^th Conf on Neural Information Processing Systems, p.8780-8794.

[17]Donahue C, McAuley JJ, Puckette MS, 2019. Adversarial audio synthesis. Proc 7^th Int Conf on Learning Representations.

[18]Esteban C, Hyland SL, Rätsch G, 2017. Real-valued (medical) time series generation with recurrent conditional GANs.

[19]Fortuin V, Baranchuk D, Rätsch G, et al., 2020. GP-VAE: deep probabilistic time series imputation. Proc 23^rd Int Conf on artificial intelligence and statistics, p.1651-1661.

[20]Goel K, Gu A, Donahue C, et al., 2022. It’s raw! Audio generation with state-space models. Proc 39^th Int Conf on Machine Learning, p.7616-7633.

[21]Gu A, Goel K, Ré C, 2022. Efficiently modeling long sequences with structured state spaces. Proc 10^th Int Conf on Learning Representations.

[22]Harvey W, Naderiparizi S, Masrani V, et al., 2022. Flexible diffusion modeling of long videos. Proc 36^th Conf on Neural Information Processing Systems, p.27953-27965.

[23]Ho J, Jain A, Abbeel P, 2020. Denoising diffusion probabilistic models. Proc 34^th Int Conf on Neural Information Processing Systems, p.6840-6851.

[24]Ho J, Saharia C, Chan W, et al., 2022a. Cascaded diffusion models for high fidelity image generation. J Mach Learn Res, 23(1):47.

[25]Ho J, Salimans T, Gritsenko A, et al., 2022b. Video diffusion models. Proc 36^th Conf on Neural Information Processing Systems, p.8633-8646.

[26]Hochreiter S, Schmidhuber J, 1997. Long short-term memory. Neur Comput, 9(8):1735-1780.

[27]Hyvärinen A, 2005. Estimation of non-normalized statistical models by score matching. J Mach Learn Res, 6(24):695-709.

[28]Kashif R, Abdul-Saboor S, Ingmar S, et al., 2021. Multi-variate probabilistic time series forecasting via conditioned normalizing flows.

[29]Kingma DP, Welling M, 2013. Auto-encoding variational Bayes.

[30]Kingma DP, Salimans T, Jozefowicz R, et al., 2016. Improved variational inference with inverse autoregressive flow. Proc 30^th Conf on Neural Information Processing Systems, p.29.

[31]Kong ZF, Ping W, Huang JJ, et al., 2021. DiffWave: a versatile diffusion model for audio synthesis. Proc 9^th Int Conf on Learning Representations.

[32]Li RK, Li XL, Gao SY, et al., 2023. Graph convolution recurrent denoising diffusion model for multivariate probabilistic temporal forecasting. Proc 19^th Int Conf on Advanced Data Mining and Applications.

[33]Li XL, Thickstun J, Gulrajani I, et al., 2022. Diffusion-LM improves controllable text generation. Proc 36^th Conf on Neural Information Processing Systems, p.4328-4343.

[34]Li Y, Lu XJ, Wang YQ, et al., 2022. Generative time series forecasting with diffusion, denoise, and disentanglement. Proc 36^th Conf on Neural Information Processing System, p.23009-23022.

[35]Li YG, Yu R, Shahabi C, et al., 2018. Diffusion convolutional recurrent neural network: data-driven traffic forecasting. Proc 6^th Int Conf on Learning Representations.

[36]Li YN, Chen ZZ, Zha DC, et al., 2021. Learning disentangled representations for time series.

[37]Lim H, Kim M, Park S, et al., 2023. Regular time-series generation using SGM.

[38]Liu DC, Wang J, Shang S, et al., 2022. MSDR: multi-step dependency relation networks for spatial temporal forecasting. Proc 28^th ACM SIGKDD Conf on Knowledge Discovery and Data Mining, p.1042-1050.

[39]Liu MZ, Huang H, Feng H, et al., 2023. PriSTI: a conditional diffusion framework for spatiotemporal imputation. Proc IEEE 39^th Int Conf on Data Engineering, p.1927-1939.

[40]Lugmayr A, Danelljan M, Romero A, et al., 2022. RePaint: inpainting using denoising diffusion probabilistic models. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.11451-11461.

[41]Luo C, 2022. Understanding diffusion models: a unified perspective.

[42]Luo YH, Cai XR, Zhang Y, et al., 2018. Multivariate time series imputation with generative adversarial networks. Proc 32^nd Conf on Neural Information Processing Systems, p.1603-1614.

[43]Mogren O, 2016. C-RNN-GAN: continuous recurrent neural networks with adversarial training.

[44]Mulyadi AW, Jun E, Suk HI, 2022. Uncertainty-aware variational-recurrent imputation network for clinical time series. IEEE Trans Cybern, 52(9):9684-9694.

[45]Neifar N, Ben-Hamadou A, Mdhaffar A, et al., 2023. Diff-ECG: a generalized probabilistic diffusion model for ECG signals synthesis.

[46]Nikolay S, Junyoung C, Mikolaj B, et al., 2022. Step-unrolled denoising autoencoders for text generation. Int Conf on Learning Representations.

[47]Osman MS, Abu-Mahfouz AM, Page PR, 2018. A survey on data imputation techniques: water distribution system as a use case. IEEE Access, 6:63279-63291.

[48]Rasul K, Seward C, Schuster I, et al., 2021. Autoregressive denoising diffusion models for multivariate probabilistic time series forecasting. Proc 38^th Int Conf on Machine Learning, p.8857-8868.

[49]Ronneberger O, Fischer P, Brox T, 2015. U-Net: convolutional networks for biomedical image segmentation. Proc 18^th Int Conf on Medical Image Computing and Computer-Assisted Intervention, p.234-241.

[50]Saremi S, Hyvärinen A, 2019. Neural empirical Bayes. J Mach Learn Res, 20(181):1-23.

[51]Seng DW, Lv FS, Liang ZY, et al., 2021. Forecasting traffic flows in irregular regions with multi-graph convolutional network and gated recurrent unit. Front Inform Technol Electron Eng, 22(9):1179-1193.

[52]Seo S, Arık S Ö, Yoon J, et al., 2021. Controlling neural networks with rule representations. Proc 35^th Conf on Neural Information Processing Systems, p.11196-11207.

[53]Shen LF, Kwok J, 2023. Non-autoregressive conditional diffusion models for time series prediction. Proc 40^th Int Conf on Machine Learning, p.31016-31029.

[54]Shu K, Zhao YC, Wu L, et al., 2023. Data augmentation for seizure prediction with generative diffusion model.

[55]Sikder MF, Ramachandranpillai R, Heintz F, 2023. Transfusion: generating long, high fidelity time series using diffusion models with transformers.

[56]Simeunović J, Schubnel B, Alet PJ, et al., 2022. Spatio-temporal graph neural networks for multi-site PV power forecasting. IEEE Trans Sustain Energy, 13(2):1210-1220.

[57]Sohl-Dickstein J, Weiss E, Maheswaranathan N, et al., 2015. Deep unsupervised learning using nonequilibrium thermodynamics. Proc 32^nd Int Conf on Machine Learning, p.2256-2265.

[58]Sønderby CK, Raiko T, Maaløe L, et al., 2016. Ladder variational autoencoders. Proc 30^th Int Conf on Neural Information Processing Systems, p.3745-3753.

[59]Song JM, Meng CL, Ermon S, 2021. Denoising diffusion implicit models. Proc 9^th Int Conf on Learning Representations.

[60]Song Y, Ermon S, 2019. Generative modeling by estimating gradients of the data distribution. Proc 33^rd Int Conf on Neural Information Processing Systems, p.11918-11930.

[61]Song Y, Garg S, Shi JX, et al., 2020. Sliced score matching: a scalable approach to density and score estimation. Proc 35^th Uncertainty in Artificial Intelligence Conf, p.574-584.

[62]Tashiro Y, Song JM, Song Y, et al., 2021. CSDI: conditional score-based diffusion models for probabilistic time series imputation. Proc 35^th Conf on Neural Information Processing Systems, p.24804-24816.

[63]Vahdat A, Kautz J, 2020. NVAE: a deep hierarchical variational autoencoder. Proc 34^th Int Conf on Neural Information Processing Systems, p.19667-19679.

[64]van den Oord A, Dieleman S, Zen HG, et al., 2016. WaveNet: a generative model for raw audio. 9^th ISCA Speech Synthesis Workshop, p.135.

[65]Vaswani A, Shazeer N, Parmar N, et al., 2017. Attention is all you need. Proc 31^st Int Conf on Neural Information Processing Systems, p.6000-6010.

[66]Vincent P, 2011. A connection between score matching and denoising autoencoders. Neur Comput, 23(7):1661-1674.

[67]Wang ZX, Wen QS, Zhang CL, et al., 2023. DiffLoad: uncertainty quantification in load forecasting with diffusion model.

[68]Wen HM, Lin YF, Xia YT, et al., 2023. DiffSTG: probabilistic spatio-temporal graph forecasting with denoising diffusion models.

[69]Wu HX, Xu JH, Wang JM, et al., 2021. Autoformer: decomposition transformers with auto-correlation for long-term series forecasting. Proc 35^th Conf on Neural Information Processing Systems, p.22419-22430.

[70]Xiao ZS, Kreis K, Vahdat A, 2022. Tackling the generative learning trilemma with denoising diffusion GANs. Proc 10^th Int Conf on Learning Representations.

[71]Xu DW, Wang YD, Jia LM, et al., 2017. Real-time road traffic state prediction based on ARIMA and Kalman filter. Front Inform Technol Electron Eng, 18(2):287-302.

[72]Xu TL, Wenliang LK, Munn M, et al., 2020. Cot-GAN: generating sequential data via causal optimal transport. Proc 34^th Conf on Neural Information Processing Systems, p.8798-8809.

[73]Yan TJ, Zhang HW, Zhou T, et al., 2021. ScoreGrad: multi-variate probabilistic time series forecasting with continuous energy-based generative models.

[74]Yang L, Zhang ZL, Song Y, et al., 2023. Diffusion models: a comprehensive survey of methods and applications.

[75]Yang RH, Srivastava P, Mandt S, 2022. Diffusion probabilistic modeling for video generation.

[76]Yang S, Sohl-Dickstein J, Kingma DP, et al., 2021. Score-based generative modeling through stochastic differential equations. Proc 9^th Int Conf on Learning Representations.

[77]Yi XW, Zheng Y, Zhang JB, et al., 2016. ST-MVL: filling missing values in geo-sensory time series data. Proc 25^th Int Joint Conf on Artificial Intelligence, p.2704-2710.

[78]Yi XW, Zhang JB, Wang ZY, et al., 2018. Deep distributed fusion network for air quality prediction. Proc 24^th ACM SIGKDD Int Conf on Knowledge Discovery & Data Mining, p.965-973.

[79]Yin H, Yang SQ, Zhu XQ, et al., 2015. Symbolic representation based on trend features for knowledge discovery in long time series. Front Inform Technol Electron Eng, 16(9):744-758.

[80]Yoon J, Zame WR, van der Schaar M, 2019. Estimating missing data in temporal data streams using multi-directional recurrent neural networks. IEEE Trans Biomed Eng, 66(5):1477-1490.

[81]Yu PY, Xie SR, Ma XJ, et al., 2022. Latent diffusion energy-based model for interpretable text modelling. Proc 39^th Int Conf on Machine Learning, p.25702-25720.

[82]Zhang HY, Cissé M, Dauphin YN, et al., 2018. Mixup: beyond empirical risk minimization. Proc 6^th Int Conf on Learning Representations.

[83]Zhang YF, Zhao ZD, Deng YJ, et al., 2021. ECGID: a human identification method based on adaptive particle swarm optimization and the bidirectional LSTM model. Front Inform Technol Electron Eng, 22(12):1641-1654.