تشخیص احساسات مبتنی بر سیگنال‌های EEG به کمک یادگیری عمیق مبتنی بر حافظه کوتاه‌مدت ماندگار دوجهته و مکانیسم توجه
الموضوعات :سیدعابد حسینی 1 , محبوبه هوشمند 2
1 - گروه مهندسی برق، دانشگاه آزاد اسلامی واحد مشهد، ایران
2 - گروه مهندسی کامپیوتر، دانشگاه آزاد اسلامی واحد مشهد، ایران
الکلمات المفتاحية: تشخیص احساس, حافظه کوتاه‌مدت ماندگار دوجهته, سیگنال مغزی, مکانیسم توجه, یادگیری عمیق,
ملخص المقالة :
این پژوهش به تشخیص احساسات از روی سیگنال‌های EEG به کمک یادگیری عمیق مبتنی بر حافظه کوتاه‌مدت ماندگار (LSTM) دوجهته و مکانیسم توجه می‌پردازد. در این پژوهش از دو پایگاه داده SEED و DEAP برای تشخیص احساس استفاده شده است. داده SEED شامل سیگنال‌های EEG در 62 کانال متعلق به 15 شرکت‌کننده در سه دسته مختلف از احساسات مثبت، خنثی و منفی است. داده DEAP شامل سیگنال EEG در 32 کانال متعلق به 32 شرکت‌کننده در دو دسته از ظرفیت و برانگیختگی است. LSTM کارایی خود را در استخراج اطلاعات زمانی از سیگنال‌های فیزیولوژیکی طولانی نشان داده است. نوآوریهای این پژوهش شامل استفاده از یک تابع تلفات جدید و بهینه‌ساز بیزین برای یافتن نرخ یادگیری اولیه است. صحت روش پیشنهادی برای طبقه‌بندی احساسات در پایگاه داده SEED 72/96 درصد شده است. صحت روش پیشنهادی برای طبقه‌بندی احساس در دو دسته ظرفیت و برانگیختگی در پایگاه داده DEAP به‌ترتیب 9/94 و 1/97 درصد است. نهایتاً مقایسه نتایج به‌دست‌آمده با پژوهش‌های اخیر روی داده‌های یکسان، نشان از بهبود نسبتاً خوب روش پیشنهادی دارد.
[1] A. S. A. Hans and S. Rao, "A CNN-LSTM based deep neural networks for facial emotion detection in videos," International J. of Advances in Signal and Image Sciences, vol. 7, no. 1, pp. 11-20, Jun. 2021.
[2] L. Mou, et al., "Driver stress detection via multimodal fusion using attention-based CNN-LSTM," Expert Systems with Applications, vol. 173, Article ID: 114693, Jul. 2021.
[3] N. S. Suhaimi, J. Mountstephens, and J. Teo, "EEG-based emotion recognition: a state-of-the-art review of current trends and opportunities," Computational Intelligence and Neuroscience, vol. 2020, Article ID: 8875426, 16 Sept. 2020.
[4] Y. Kim, H. Lee, and E. M. Provost, "Deep learning for robust feature generation in audiovisual emotion recognition," in Proc. IEEE Int. Conf. on Acoustics, Speech and Signal Processing, pp. 3687-3691, Vancouver, Canada, 26-31 May 2013.
[5] C. Herrando, J. Jiménez-Martínez, M. J. Martín-De Hoyos, and E. Constantinides, "Emotional contagion triggered by online consumer reviews: evidence from a neuroscience study," J. of Retailing and Consumer Services, vol. 67, Article ID: 102973, Jul. 2022.
[6] M. Ali, A. H. Mosa, F. Al Machot, and K. Kyamakya, "EEG-based emotion recognition approach for e-healthcare applications," in Proc. 8th Int. Conf. on Ubiquitous and Future Networks, pp. 946-950, Vienna, Austria, 5-8 Jul. 2016.
[7] S. A. Hosseini, M. A. Khalilzadeh, and S. Changiz, "Emotional stress recognition system for affective computing based on bio-signals," J. of Biological Systems, vol. 18, no. spec01, pp. 101-114, 2010.
[8] A. Sakalle, P. Tomar, H. Bhardwaj, D. Acharya, and A. Bhardwaj, "A LSTM based deep learning network for recognizing emotions using wireless brainwave driven system," Expert Systems with Applications, vol. 173, Article ID: 114516, Jul. 2021.
[9] C. Li, Z. Bao, L. Li, and Z. Zhao, "Exploring temporal representations by leveraging attention-based bidirectional LSTM-RNNs for multi-modal emotion recognition," Information Processing & Management, vol. 57, no. 3, Article ID: 102185, May 2020.
[10] A. Bhattacharyya, R. K. Tripathy, L. Garg, and R. B. Pachori, "A novel multivariate-multiscale approach for computing EEG spectral and temporal complexity for human emotion recognition," IEEE Sensors J., vol. 21, no. 3, pp. 3579-3591, Feb. 2021.
[11] Y. Luo, et al., "EEG-based emotion classification using spiking neural networks," IEEE Access, vol. 8, pp. 46007-46016, 2020.
[12] Y. Wang, et al., "EEG-based emotion recognition with prototype-based data representation," in Proc. 41st Annual Int. Conf. of the IEEE Engineering in Medicine and Biology Society, pp. 684-689, Berlin, Germany, 23-27 Jul. 2019.
[13] M. M. Rahman, et al., "Recognition of human emotions using EEG signals: a review," Computers in Biology and Medicine, vol. 136 Article ID: 104696, Sept. 2021.
[14] J. Yang, X. Huang, H. Wu, and X. Yang, "EEG-based emotion classification based on bidirectional long short-term memory network," Procedia Computer Science, vol. 174, pp. 491-504, 2020.
[15] R. Andreasson, B. Alenljung, E. Billing, and R. Lowe, "Affective touch in human-robot interaction: conveying emotion to the nao robot," International J. of Social Robotics, vol. 10, no. 4, pp. 473-491, Dec. 2018.
[16] X. Wang, Y. Ren, Z. Luo, W. He, J. Hong, and Y. Huang, "Deep learning-based EEG emotion recognition: current trends and future perspectives," Frontiers in Psychology, vol. 14, Article ID: 1126994, Feb. 2023.
[17] M. K. Chowdary, J. Anitha, and D. J. Hemanth, "Emotion recognition from EEG signals using recurrent neural networks," Electronics, vol. 11, no. 15, Article ID: 2387, Jul. 2022.
[18] R. C. Dhingra and S. Ram Avtar Jaswal, "Emotion recognition based on EEG using DEAP dataset," European J. of Molecular & Clinical Medicine, vol. 8, no. 3, pp. 3509-3517, 2021.
[19] N. Zhuang, et al., "Emotion recognition from EEG signals using multidimensional information in EMD domain," BioMed Research International, vol. 2017, Article ID: 8317357, 2017.
[20] V. M. Joshi and R. B. Ghongade, "IDEA: intellect database for emotion analysis using EEG signal," J. of King Saud University-Computer and Information Sciences, vol. 34, no. 7, pp. 4433-4447, Jul. 2022.
[21] O. Atila and A. Şengür, "Attention guided 3D CNN-LSTM model for accurate speech based emotion recognition," Applied Acoustics, vol. 182, Article ID: 108260, Nov. 2021.
[22] X. Zheng and W. Chen, "An attention-based bi-LSTM method for visual object classification via EEG," Biomedical Signal Processing and Control, vol. 63, Article ID: 102174, Jan. 2021.
[23] D. Huang, et al., "Differences first in asymmetric brain: a bi-hemisphere discrepancy convolutional neural network for EEG emotion recognition," Neurocomputing, vol. 448, pp. 140-151, 11 Aug. 2021.
[24] S. Koelstra, et al., "Deap: a database for emotion analysis; using physiological signals," IEEE Trans. on Affective Computing, vol. 3, no. 1, pp. 18-31, Jun. 2011.
[25] M. Algarni, F. Saeed, T. Al-Hadhrami, F. Ghabban, and M. Al-Sarem, "Deep learning-based approach for emotion recognition using electroencephalography (EEG) signals using bi-directional long short-term memory (Bi-LSTM)," Sensors, vol. 22, no. 8, Article ID: 2976, Apr. 2022.
[26] W. L. Zheng and B. L. Lu, "Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks," IEEE Trans. on Autonomous Mental Development, vol. 7, no. 3, pp. 162-175, Sep. 2015.
[27] Q. Ma, M. Wang, L. Hu, L. Zhang, and Z. Hua, "A novel recurrent neural network to classify EEG signals for customers’ decision-making behavior prediction in brand extension scenario," Frontiers in Human Neuroscience, vol. 15, Article ID: 610890, Mar. 2021.
[28] S. Hochreiter and J. Schmidhuber, "Long short-term memory," Neural Computation, vol. 9, no. 8, pp. 1735-1780, Nov. 1997.
[29] S. Kumar, A. Sharma, and T. Tsunoda, "Brain wave classification using long short-term memory network based OPTICAL predictor," Scientific Reports, vol. 9, Article ID: 9153, Jun. 2019.
[30] M. Z. I. Ahmed and N. Sinha, "EEG-based emotion classification using LSTM under new paradigm," Biomedical Physics & Engineering Express, vol. 7, no. 6, Article ID: 065018, Sept. 2021.
[31] G. Liu and J. Guo, "Bidirectional LSTM with attention mechanism and convolutional layer for text classification," Neurocomputing, vol. 337, pp. 325-338, 14 Apr. 2019.
[32] J. C. Nunez, R. Cabido, J. J. Pantrigo, A. S. Montemayor, and J. F. Velez, "Convolutional neural networks and long short-term memory for skeleton-based human activity and hand gesture recognition," Pattern Recognition, vol. 76, pp. 80-94, Apr. 2018.
[33] K. Greff, R. K. Srivastava, J. Koutník, B. R. Steunebrink, and J. Schmidhuber, "LSTM: a search space odyssey," IEEE Trans. on Neural Networks and Learning Systems, vol. 28, no. 10, pp. 2222-2232, Mar. 2015.
[34] T. Shen, et al., "Disan: directional self-attention network for rnn/cnn-free language understanding," in Proc. of the AAAI Conf. on Artificial Intelligence, pp. 5446-5455, New Orleans, LA, USA, 2-7 Feb. 2018.
[35] S. Mirsamadi, E. Barsoum, and C. Zhang, "Automatic speech emotion recognition using recurrent neural networks with local attention," in Proc. IEEE Int. Conf. on Acoustics, Speech and Signal Processing, pp. 2227-2231, New Orleans, LA, USA, 5-9 Mar. 2017.
[36] R. Dutta and M. Majumder, "Attention-based bidirectional LSTM with embedding technique for classification of COVID-19 articles," Intelligent Decision Technologies, vol. 16, no. 1, pp. 205-215, Apr. 2022.
