Tackling the Unannotated: Scene Graph Generation with Bias-Reduced Models
No Thumbnail Available
Access rights
openAccess
Journal Title
Journal ISSN
Volume Title
Conference article in proceedings
This publication is imported from Aalto University research portal.
View publication in the Research portal
Other link related to publication
View publication in the Research portal
Other link related to publication
Date
2020
Major/Subject
Mcode
Degree programme
Language
en
Pages
13
Series
Proceedings of the British Machine Vision Conference (BMVC)
Abstract
Predicting a scene graph that captures visual entities and their interactions in an image has been considered a crucial step towards full scene comprehension. Recent scene graph generation (SGG) models have shown their capability of capturing the most frequent relations among visual entities. However, the state-of-the-art results are still far from satisfactory, e.g. models can obtain 31% in overall recall R@100, whereas the likewise important mean class-wise recall mR@100 is only around 8% on Visual Genome (VG). The discrepancy between R and mR results urges to shift the focus from pursuing a high R to a high mR with a still competitive R. We suspect that the observed discrepancy stems from both the annotation bias and sparse annotations in VG, in which many visual entity pairs are either not annotated at all or only with a single relation when multiple ones could be valid. To address this particular issue, we propose a novel SGG training scheme that capitalizes on self-learned knowledge. It involves two relation classifiers, one offering a less biased setting for the other to base on. The proposed scheme can be applied to most of the existing SGG models and is straightforward to implement. We observe significant relative improvements in mR (between +6.6% and +20.4%) and competitive or better R (between -2.4% and 0.3%) across all standard SGG tasks.Description
| openaire: EC/H2020/780069/EU//MeMAD
Keywords
Other note
Citation
Wang , T-J J , Pehlivan , S & Laaksonen , J 2020 , Tackling the Unannotated: Scene Graph Generation with Bias-Reduced Models . in Proceedings of the British Machine Vision Conference (BMVC) . British Machine Vision Association , British Machine Vision Conference , Virtual, Online , United Kingdom , 07/09/2020 . < https://www.bmvc2020-conference.com/conference/papers/paper_0541.html >
