Wav2vec2-based Paralinguistic Systems to Recognise Vocalised Emotions and Stuttering

Loading...
Thumbnail Image
Access rights
openAccess
Journal Title
Journal ISSN
Volume Title
A4 Artikkeli konferenssijulkaisussa
This publication is imported from Aalto University research portal.
View publication in the Research portal
View/Open full text file from the Research portal
Date
2022-10
Major/Subject
Mcode
Degree programme
Language
en
Pages
4
Series
Proceedings of the 30th ACM International Conference on Multimedia
Abstract
With the rapid advancement in automatic speech recognition and natural language understanding, a complementary field (paralin- guistics) emerged, focusing on the non-verbal content of speech. The ACM Multimedia 2022 Computational Paralinguistics Challenge introduced several exciting tasks of this field. In this work, we focus on tackling two Sub-Challenges using modern, pre-trained models called wav2vec2. Our experimental results demonstrated that wav2vec2 is an excellent tool for detecting the emotions behind vocalisations and recognising different types of stutterings. Albeit they achieve outstanding results on their own, our results demonstrated that wav2vec2-based systems could be further improved by ensembling them with other models. Our best systems outperformed the competition baselines by a considerable margin, achieving an unweighted average recall of 44.0 (absolute improvement of 6.6% over baseline) on the Vocalisation Sub-Challenge and 62.1 (absolute improvement of 21.7% over baseline) on the Stuttering Sub-Challenge.
Description
Keywords
Other note
Citation
Grósz, T, Porjazovski, D, Getman, Y, Kadiri, S & Kurimo, M 2022, Wav2vec2-based Paralinguistic Systems to Recognise Vocalised Emotions and Stuttering . in Proceedings of the 30th ACM International Conference on Multimedia . ACM, ACM International Conference on Multimedia, Lisboa, Portugal, 10/10/2022 . https://doi.org/10.1145/3503161.3551572