CLC number: TP183
On-line Access: 2024-12-26
Received: 2023-11-17
Revision Accepted: 2024-02-26
Crosschecked: 2025-01-24
Cited: 0
Clicked: 1642
Yanping ZHU, Lei HUANG, Jixin CHEN, Shenyun WANG, Fayu WAN, Jianan CHEN. VG-DOCoT: a novel DO-Conv and transformer framework via VAE-GAN technique for EEG emotion recognition[J]. Frontiers of Information Technology & Electronic Engineering,in press.https://doi.org/10.1631/FITEE.2300781 @article{title="VG-DOCoT: a novel DO-Conv and transformer framework via VAE-GAN technique for EEG emotion recognition", %0 Journal Article TY - JOUR
VG-DOCoT:一种新颖的基于变分自动编码器-生成对抗网络技术、深度过参数化卷积和变换器框架的脑电情绪识别模型南京信息工程大学电子与信息工程学院,中国南京市,210044 摘要:人类情绪是反映个体当前生理和心理状态的复杂心理现象。情绪对人类的行为、认知、交流和决策产生了显著的影响。然而,目前的情绪识别方法在实际应用中往往存在性能不佳和可扩展性有限的问题。为此,我们提出一种新颖的脑电图(EEG)情绪识别框架VG-DOCoT,它基于深度过参数化卷积(DO-Conv)、变换器和变分自编码器-生成对抗网络(VAE-GAN)结构。具体来说,在预处理中,可以从EEG信号中提取微分熵(DE),以映射到时间、空间和频率信息中。为了增强训练数据,采用VAE-GAN进行数据增强。使用一种新颖的卷积模块DO-Conv替代传统的卷积层,以提高网络性能。在网络框架中引入了变换器结构,以揭示EEG信号中的全局依赖性。使用所提出的模型,在DEAP数据集上进行了二分类任务仿真,唤醒度和效价度的准确率分别达到92.52%和92.27%。另外,在SEED数据集上进行了三分类任务测试,包括中性、积极和消极三种情绪,获得的平均预测准确率为93.77%。所提出的方法显著提高了脑电情绪识别的准确率。 关键词组: Darkslateblue:Affiliate; Royal Blue:Author; Turquoise:Article
Reference[1]Aznan NKN, Atapour-Abarghouei A, Bonner S, et al., 2019. Simulating brain signals: creating synthetic EEG data via neural-based generative models for improved SSVEP classification. Int Joint Conf on Neural Networks, p.1-8. ![]() [2]Bahdanau D, Cho K, Bengio Y, 2015. Neural machine translation by jointly learning to align and translate. 3rd Int Conf on Learning Representations. ![]() [3]Bernat E, Bunce S, Shevrin H, 2001. Event-related brain potentials differentiate positive and negative mood adjectives during both supraliminal and subliminal visual processing. Int J Psychophysiol, 42(1):11-34. ![]() [4]Cao JM, Li YY, Sun MC, et al., 2022. Do-Conv: depthwise over-parameterized convolutional layer. IEEE Trans Image Process, 31:3726-3736. ![]() [5]Chao H, Dong L, 2021. Emotion recognition using three-dimensional feature and convolutional neural network from multichannel EEG signals. IEEE Sens J, 21(2):2024-2034. ![]() [6]Cheng J, Chen MY, Li C, et al., 2021. Emotion recognition from multi-channel EEG via deep forest. IEEE J Biomed Health Inform, 25(2):453-464. ![]() [7]Goodfellow I, Pouget-Abadie J, Mirza M, et al., 2020. Generative adversarial networks. Commun ACM, 63(11):139-144. ![]() [8]Guo JY, Cai Q, An JP, et al., 2022. A Transformer based neural network for emotion recognition and visualizations of crucial EEG channels. Phys A Stat Mech Appl, 603:127700. ![]() [9]Hu JF, Min JL, 2018. Automated detection of driver fatigue based on EEG signals using gradient boosting decision tree model. Cogn Neurodyn, 12(4):431-440. ![]() [10]Jenke R, Peer A, Buss M, 2014. Feature extraction and selection for emotion recognition from EEG. IEEE Trans Affect Comput, 5(3):327-339. ![]() [11]Kingma DP, Welling M, 2014. Auto-encoding variational Bayes. 2nd Int Conf on Learning Representations. ![]() [12]Koelstra S, Muhl C, Soleymani M, et al., 2012. DEAP: a database for emotion analysis; using physiological signals. IEEE Trans Affect Comput, 3(1):18-31. ![]() [13]Lan ZR, Sourina O, Wang LP, et al., 2016. Real-time EEG-based emotion monitoring using stable features. Vis Comput, 32(3):347-358. ![]() [14]Lew WCL, Wang D, Shylouskaya K, et al., 2020. EEG-based emotion recognition using spatial-temporal representation via Bi-GRU. 42nd Annual Int Conf of the IEEE Engineering in Medicine & Biology Society, p.116-119. ![]() [15]Li C, Lin XJ, Liu Y, et al., 2022. EEG-based emotion recognition via efficient convolutional neural network and contrastive learning. IEEE Sens J, 22(20):19608-19619. ![]() [16]Li JP, Zhang ZX, He HG, 2018. Hierarchical convolutional neural networks for EEG-based emotion recognition. Cogn Comput, 10(2):368-380. ![]() [17]Li SJ, Li W, Xing ZJ, et al., 2022. A personality-guided affective brain–computer interface for implementation of emotional intelligence in machines. Front Inform Technol Electron Eng, 23(8):1158-1173. ![]() [18]Li X, Song DW, Zhang P, et al., 2016. Emotion recognition from multi-channel EEG data through convolutional recurrent neural network. IEEE Int Conf on Bioinformatics and Biomedicine, p.352-359. ![]() [19]Li X, Zhang YZ, Tiwari P, et al., 2022. EEG based emotion recognition: a tutorial and review. ACM Comput Surv, 55(4):79. ![]() [20]Lin YP, Wang CH, Wu TL, et al., 2009. EEG-based emotion recognition in music listening: a comparison of schemes for multiclass support vector machine. IEEE Int Conf on Acoustics, Speech and Signal Processing, p.489-492. ![]() [21]Liu YJ, Yu MJ, Zhao GZ, et al., 2018. Real-time movie-induced discrete emotion recognition from EEG signals. IEEE Trans Affect Comput, 9(4):550-562. ![]() [22]Liu YS, Sourina O, 2014. EEG-based subject-dependent emotion recognition algorithm using fractal dimension. IEEE Int Conf on Systems, Man, and Cybernetics, p.3166-3171. ![]() [23]Mohammadi Z, Frounchi J, Amiri M, 2017. Wavelet-based emotion recognition system using EEG signal. Neur Comput Appl, 28(8):1985-1990. ![]() [24]Picard RW, 2000. Affective Computing. MIT Press, Cambridge, UK. ![]() [25]Salama ES, El-Khoribi RA, Shoman ME, et al., 2018. EEG-based emotion recognition using 3D convolutional neural networks. Int J Adv Comput Sci Appl, 9(8):329-337. ![]() [26]Song TF, Zheng WM, Song P, et al., 2020. EEG emotion recognition using dynamical graph convolutional neural networks. IEEE Trans Affect Comput, 11(3):532-541. ![]() [27]Sorkhabi MM, 2014. Emotion detection from EEG signals with continuous wavelet analyzing. Am J Comput Res Repos, 2(4):66-70. ![]() [28]Stam CJ, 2005. Nonlinear dynamical analysis of EEG and MEG: review of an emerging field. Clin Neurophysiol, 116(10):2266-2301. ![]() [29]Tang ZC, Li C, Wu JF, et al., 2019. Classification of EEG-based single-trial motor imagery tasks using a B-CSP method for BCI. Front Inform Technol Electron Eng, 20(8):1087-1098. ![]() [30]Tao W, Li C, Song RC, et al., 2023. EEG-based emotion recognition via channel-wise attention and self attention. IEEE Trans Affect Comput, 14(1):382-393. ![]() [31]Tripathi S, Acharya S, Sharma RD, et al., 2017. Using deep and convolutional neural networks for accurate emotion classification on DEAP data. Proc 31st AAAI Conf on Artificial Intelligence, p.4746-4752. ![]() [32]Vaswani A, Shazeer N, Parmar N, et al., 2017. Attention is all you need. 31st Int Conf on Neural Information Processing Systems, p.6000-6010. ![]() [33]Vijayan AE, Sen D, Sudheer AP, 2015. EEG-based emotion recognition using statistical measures and auto-regressive modeling. IEEE Int Conf on Computational Intelligence & Communication Technology, p.587-591. ![]() [34]Wang Q, Sourina O, Nguyen MK, 2011. Fractal dimension based neurofeedback in serious games. Vis Comput, 27(4):299-309. ![]() [35]Wang XW, Nie D, Lu BL, 2014. Emotional state classification from EEG data using machine learning approach. Neurocomputing, 129:94-106. ![]() [36]Wei C, Chen LL, Song ZZ, et al., 2020. EEG-based emotion recognition using simple recurrent units network and ensemble learning. Biomed Signal Process Contr, 58:101756. ![]() [37]Yang BH, He LF, Lin L, et al., 2015. Fast removal of ocular artifacts from electroencephalogram signals using spatial constraint independent component analysis based recursive least squares in brain-computer interface. Front Inform Technol Electron Eng, 16(6):486-496. ![]() [38]Yang Y, Gao Q, Song XL, et al., 2021. Facial expression and EEG fusion for investigating continuous emotions of deaf subjects. IEEE Sens J, 21(15):16894-16903. ![]() [39]Yang YL, Wu QF, Fu YZ, et al., 2018a. Continuous convolutional neural network with 3D input for EEG-based emotion recognition. 25th Int Conf on Neural Information Processing, p.433-443. ![]() [40]Yang YL, Wu QF, Qiu M, et al., 2018b. Emotion recognition from multi-channel EEG through parallel convolutional recurrent neural network. Int Joint Conf on Neural Networks, p.1-7. ![]() [41]Yang YX, Gao ZK, Wang XM, et al., 2018. A recurrence quantification analysis-based channel-frequency convolutional neural network for emotion recognition from EEG. Chaos, 28(8):085724. ![]() [42]Yin YQ, Zheng XW, Hu B, et al., 2021. EEG emotion recognition using fusion model of graph convolutional neural networks and LSTM. Appl Soft Comput, 100:106954. ![]() [43]Zhang DL, Yao LN, Zhang X, et al., 2018. Cascade and parallel convolutional recurrent neural networks on EEG-based intention recognition for brain computer interface. Proc 32nd AAAI Conf on Artificial Intelligence, p.1703-1710. ![]() [44]Zhang QQ, Liu Y, 2018. Improving brain computer interface performance by data augmentation with conditional deep convolutional generative adversarial networks. https://arxiv.org/abs/1806.07108 ![]() [45]Zhang T, Zheng WM, Cui Z, et al., 2019. Spatial–temporal recurrent neural network for emotion recognition. IEEE Trans Cybern, 49(3):839-847. ![]() [46]Zheng WL, Lu BL, 2015. Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks. IEEE Trans Auton Ment Dev, 7(3):162-175. ![]() [47]Zhong XY, Gu Y, Luo YT, et al., 2023. Bi-hemisphere asymmetric attention network: recognizing emotion from EEG signals based on the transformer. Appl Intell, 53(12):15278-15294. ![]() Journal of Zhejiang University-SCIENCE, 38 Zheda Road, Hangzhou
310027, China
Tel: +86-571-87952783; E-mail: cjzhang@zju.edu.cn Copyright © 2000 - 2025 Journal of Zhejiang University-SCIENCE |
Open peer comments: Debate/Discuss/Question/Opinion
<1>