التفاصيل البيبلوغرافية
| العنوان: |
Efficient supervised training of audio transformers for music representation learning |
| المؤلفون: |
Alonso Jiménez, Pablo, Serra, Xavier, Bogdanov, Dmitry |
| سنة النشر: |
2023 |
| المجموعة: |
UPF Digital Repository (Universitat Pompeu Fabra, Barcelona) |
| الوصف: |
This work has been accepted at the 24th International Society for Music Information Retrieval Conference (ISMIR 2023), at Milan, Italy. October 5-9, 2023. ; In this work, we address music representation learning using convolution-free transformers. We build on top of existing spectrogram-based audio transformers such as AST and train our models on a supervised task using patchout training similar to PaSST. In contrast to previous works, we study how specific design decisions affect downstream music tagging tasks instead of focusing on the training task. We assess the impact of initializing the models with different pre-trained weights, using various input audio segment lengths, using learned representations from different blocks and tokens of the transformer for downstream tasks, and applying patchout at inference to speed up feature extraction. We find that 1) initializing the model from ImageNet or AudioSet weights and using longer input segments are beneficial both for the training and downstream tasks, 2) the best representations for the considered downstream tasks are located in the middle blocks of the transformer, and 3) using patchout at inference allows faster processing than our convolutional baselines while maintaining superior performance. The resulting models, MAEST, 1 are publicly available and obtain the best performance among open models in music tagging tasks. ; This work has been supported by the Musical AI project - PID2019-111403GB-I00/AEI/10.13039/501100011033, funded by the Spanish Ministerio de Ciencia e Innovación and the Agencia Estatal de Investigación. |
| نوع الوثيقة: |
report |
| وصف الملف: |
application/pdf |
| اللغة: |
English |
| Relation: |
info:eu-repo/grantAgreement/ES/2PE/PID2019-111403GB-I00; http://hdl.handle.net/10230/58023 |
| الاتاحة: |
http://hdl.handle.net/10230/58023 |
| Rights: |
© P. Alonso-Jiménez, X. Serra, and D. Bogdanov. Licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0). Attribution: P. Alonso-Jiménez, X. Serra, and D. Bogdanov, “Efficient Supervised Training of Audio Transformers for Music Representation Learning”, in Proc. of the 24th Int. Society for Music Information Retrieval Conf., Milan, Italy, 2023. ; https://creativecommons.org/licenses/by/4.0/ ; info:eu-repo/semantics/openAccess |
| رقم الانضمام: |
edsbas.17548B0E |
| قاعدة البيانات: |
BASE |