التفاصيل البيبلوغرافية
| العنوان: |
Adaptive cross-fusion learning for multi-modal gesture recognition |
| المؤلفون: |
Benjia Zhou, Jun Wan, Yanyan Liang, Guodong Guo |
| المصدر: |
Virtual Reality & Intelligent Hardware, Vol 3, Iss 3, Pp 235-247 (2021) |
| بيانات النشر: |
KeAi Communications Co., Ltd., 2021. |
| سنة النشر: |
2021 |
| المجموعة: |
LCC:Computer engineering. Computer hardware |
| مصطلحات موضوعية: |
Gesture recognition, Multi-modal fusion, RGB-D, Computer engineering. Computer hardware, TK7885-7895 |
| الوصف: |
Background: Gesture recognition has attracted significant attention because of its wide range of potential applications. Although multi-modal gesture recognition has made significant progress in recent years, a popular method still is simply fusing prediction scores at the end of each branch, which often ignores complementary features among different modalities in the early stage and does not fuse the complementary features into a more discriminative feature. Methods: This paper proposes an Adaptive Cross-modal Weighting (ACmW) scheme to exploit complementarity features from RGB-D data in this study. The scheme learns relations among different modalities by combining the features of different data streams. The proposed ACmW module contains two key functions: (1) fusing complementary features from multiple streams through an adaptive one-dimensional convolution; and (2) modeling the correlation of multi-stream complementary features in the time dimension. Through the effective combination of these two functional modules, the proposed ACmW can automatically analyze the relationship between the complementary features from different streams, and can fuse them in the spatial and temporal dimensions. Results: Extensive experiments validate the effectiveness of the proposed method, and show that our method outperforms state-of-the-art methods on IsoGD and NVGesture. |
| نوع الوثيقة: |
article |
| وصف الملف: |
electronic resource |
| اللغة: |
English |
| تدمد: |
2096-5796 |
| Relation: |
http://www.sciencedirect.com/science/article/pii/S2096579621000292; https://doaj.org/toc/2096-5796 |
| DOI: |
10.1016/j.vrih.2021.05.003 |
| URL الوصول: |
https://doaj.org/article/e379d95728e84eefa6ae49d9a2210b02 |
| رقم الانضمام: |
edsdoj.379d95728e84eefa6ae49d9a2210b02 |
| قاعدة البيانات: |
Directory of Open Access Journals |
Full Text Finder