التفاصيل البيبلوغرافية
| العنوان: |
面向深度学习的多模态情感识别研究进展. (Chinese) |
| Alternate Title: |
Survey of Deep Learning Based Multimodal Emotion Recognition. (English) |
| المؤلفون: |
赵小明, 杨轶娇, 张石清 |
| المصدر: |
Journal of Frontiers of Computer Science & Technology; Jul2022, Vol. 16 Issue 7, p1479-1503, 25p |
| مصطلحات موضوعية: |
ARTIFICIAL neural networks, AFFECTIVE computing, DEEP learning, EMOTION recognition, ARTIFICIAL intelligence, EMOTIONS |
| Abstract (English): |
Multimodal emotion recognition aims to recognize human emotional states through different modalities related to human emotion expression such as audio, vision, text, etc. This topic is of great importance in the fields of human- computer interaction, artificial intelligence, affective computing, etc., and has attracted much attention. In view of the great success of deep learning methods developed in recent years in various tasks, a variety of deep neural networks have been used to learn high-level emotional feature representations for multimodal emotion recognition. In order to systematically summarize the research advance of deep learning methods in the field of multimodal emotion recognition, this paper aims to present comprehensive analysis and summarization on recent multimodal emotion recognition literatures based on deep learning. First, the general framework of multimodal emotion recognition is given, and the commonly used multimodal emotional dataset is introduced. Then, the principle of representative deep learning techniques and its advance in recent years are briefly reviewed. Subsequently, this paper focuses on the advance of two key steps in multimodal emotion recognition: emotional feature extraction methods related to audio, vision, text, etc., including hand-crafted feature extraction and deep feature extraction; multimodal information fusion strategies integrating different modalities. Finally, the challenges and opportunities in this field are analyzed, and the future development direction is pointed out. [ABSTRACT FROM AUTHOR] |
| Abstract (Chinese): |
多模态情感识别是指通过与人类情感表达相关的语音、视觉、文本等不同模态信息来识别人的情感状 态。该研究在人机交互、人工智能、情感计算等领域有着重要的研究意义，备受研究者关注。鉴于近年来发展 起来的深度学习方法在各种任务中所取得的巨大成功，目前各种深度神经网络已被用于学习高层次的情感特 征表示，用于多模态情感识别。为了系统地总结深度学习方法在多模态情感识别领域中的研究现状，拟对近 年来面向深度学习的多模态情感识别研究文献进行分析与归纳。首先，给出了多模态情感识别的一般框架， 并介绍了常用的多模态情感数据集。然后，简要回顾了代表性深度学习技术的原理及其进展。随后，重点详 细介绍了多模态情感识别中的两个关键步骤的研究进展：与语音、视觉、文本等不同模态相关的情感特征提取 方法，包括手工特征和深度特征；融合不同模态信息的多模态信息融合策略。最后，分析了该领域面临的挑战 和机遇，并指出了未来的发展方向。 [ABSTRACT FROM AUTHOR] |
|
Copyright of Journal of Frontiers of Computer Science & Technology is the property of Beijing Journal of Computer Engineering & Applications Journal Co Ltd. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) |
| قاعدة البيانات: |
Complementary Index |