aalto1 untyped-item.component.html
Deep neural learning on weighted datasets utilizing label disagreement from crowdsourcing
Loading...
Access rights
openAccess
publishedVersion
URL
Journal Title
Journal ISSN
Volume Title
A1 Alkuperäisartikkeli tieteellisessä aikakauslehdessä
This publication is imported from Aalto University research portal.
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Unless otherwise stated, all rights belong to the author. You may download, display and print this publication for Your own personal use. Commercial use is prohibited.
Authors
Wang, Dongsheng
Tiwari, Prayag
Shorfuzzaman, Mohammad
Schmitt, Ingo
Date
Department
Major/Subject
Mcode
Degree programme
Language
en
Pages
7
Series
Computer Networks, Volume 196
Abstract
Experts and crowds can work together to generate high-quality datasets, but such collaboration is limited to a large-scale pool of data. In other words, training on a large-scale dataset depends more on crowdsourced datasets with aggregated labels than expert intensively checked labels. However, the limited amount of high-quality dataset can be used as an objective test dataset to build a connection between disagreement and aggregated labels. In this paper, we claim that the disagreement behind an aggregated label indicates more semantics (e.g. ambiguity or difficulty) of an instance than just spam or error assessment. We attempt to take advantage of the informativeness of disagreement to assist learning neural networks by computing a series of disagreement measurements and incorporating disagreement with distinct mechanisms. Experiments on two datasets demonstrate that the consideration of disagreement, treating training instances differently, can promisingly result in improved performance.
Description
| openaire: EC/H2020/101016775/EU//INTERVENE
Keywords
Other note
Citation
Wang, D, Tiwari, P, Shorfuzzaman, M & Schmitt, I 2021, 'Deep neural learning on weighted datasets utilizing label disagreement from crowdsourcing', Computer Networks, vol. 196, 108227. https://doi.org/10.1016/j.comnet.2021.108227