|
Frontiers of Information Technology & Electronic Engineering
ISSN 2095-9184 (print), ISSN 2095-9230 (online)
2022 Vol.23 No.8 P.1158-1173
A personality-guided affective brain–computer interface for implementation of emotional intelligence in machines
Abstract: Affective brain–computer interfaces have become an increasingly important topic to achieve emotional intelligence in human–machine collaboration. However, due to the complexity of electroencephalogram (EEG) signals and the individual differences in emotional response, it is still a great challenge to design a reliable and effective model. Considering the influence of personality traits on emotional response, it would be helpful to integrate personality information and EEG signals for emotion recognition. This study proposes a personality-guided attention neural network that can use personality information to learn effective EEG representations for emotion recognition. Specifically, we first use a convolutional neural network to extract rich temporal and regional representations of EEG signals, and a special convolution kernel is designed to learn inter- and intra-regional correlations simultaneously. Second, inspired by the fact that electrodes within distinct brain scalp regions play different roles in emotion recognition, a personality-guided regional-attention mechanism is proposed to further explore the contributions of electrodes within a region and between regions. Finally, attention-based long short-term memory is designed to explore the temporal dynamics of EEG signals. Experiments on the AMIGOS dataset, which is a dataset for multimodal research for affect, personality traits, and mood on individuals and groups, show that the proposed method can significantly improve the performance of subject-independent emotion recognition and outperform state-of-the-art methods.
Key words: Electroencephalogram (EEG); Emotion recognition; Attention mechanism; Personality traits
兰州大学信息科学与工程学院,中国兰州市,730099
摘要:情感脑机接口(brain–computerinterfaces, BCIs)已成为在人机协作中实现情感智能的一个重要途径。然而,由于脑电图(electroencephalogram, EEG)信号的复杂性和情绪反应的个体差异性,设计一个可靠和有效的模型仍然是一个巨大挑战。考虑到不同人格特征的个体在情绪感知和反应过程中的差异,整合人格信息和脑电信号对情绪识别是有帮助的。鉴于此,提出一种人格引导的注意力神经网络,其可以利用人格信息学习更为有效的EEG表征以用于情感识别。具体来说,我们首先利用卷积神经网络提取脑电信号的时域和空域表征,进而设计一种特殊的卷积核同时学习大脑头皮不同区域间和区域内的EEG导联相关关系。其次,考虑到不同大脑头皮区域在情绪识别中可能发挥不同的作用,提出一种人格引导的区域注意力机制,以进一步探索区域内和区域间EEG导联的贡献。最后,设计一种基于注意力的长短期记忆网络(long short-term memory, LSTM)建模EEG信号的时域动态特征。在AMIGOS数据集(一个用于个人和群体的情感、人格特征和情绪多模态研究的数据集)的实验结果表明,本研究所提方法可以显著提升被试独立策略下情感识别的性能,并优于现有情感识别方法。
关键词组:
References:
Open peer comments: Debate/Discuss/Question/Opinion
<1>
DOI:
10.1631/FITEE.2100489
CLC number:
TP39
Download Full Text:
Downloaded:
3969
Download summary:
<Click Here>Downloaded:
345Clicked:
1831
Cited:
0
On-line Access:
2022-08-22
Received:
2021-10-14
Revision Accepted:
2022-08-29
Crosschecked:
2022-07-28