Anchor-Free Action Proposal Network with Uncertainty Estimation
Loading...
Access rights
openAccess
acceptedVersion
URL
Journal Title
Journal ISSN
Volume Title
A4 Artikkeli konferenssijulkaisussa
This publication is imported from Aalto University research portal.
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Other link related to publication (opens in new window)
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Other link related to publication (opens in new window)
Authors
Date
2023
Department
Major/Subject
Mcode
Degree programme
Language
en
Pages
6
Series
Proceedings - 2023 IEEE International Conference on Multimedia and Expo, ICME 2023, pp. 1853-1858, Proceedings - IEEE International Conference on Multimedia and Expo ; Volume 2023-July
Abstract
Proposal generation is a fundamental yet challenging task for two-stage temporal action detection pipelines. The task aims at predicting starting and ending boundaries of segments in realistic video sequences and action recognition methods cannot be directly applied to such videos due to their untrimmed nature. Most state-of-the-art models rely on temporal convolutional neural networks with pre-defined anchor segments. By eliminating anchors, we propose a lighter end-to-end trainable Anchor-Free Multiscale Transformer-based Generator (AMTG) model using local clues via video snippets. To improve effectiveness for temporal evaluation, we apply multiscale Transformer encoders to sequences with a bi-directional mask extension that simultaneously predicts boundary distances with uncertainties and various snippet-based local scores. Later, our model integrates local predictions to generate proposal candidates using the proposed scoring function. Experiments on the THUMOS14 and ActivityNet-1.3 benchmarks demonstrate the effectiveness of AMTG for the temporal proposal generation task.Description
Funding Information: This work has been funded by the Academy of Finland project numbers 329268 and 345791. The computational resources have been provided by the Aalto University’s Aalto Science-IT project and the CSC–IT Center for Science. Publisher Copyright: © 2023 IEEE.
Keywords
anchor-free, multiscale transformer network, temporal action proposals, two-stage detectors
Other note
Citation
Pehlivan, S & Laaksonen, J 2023, Anchor-Free Action Proposal Network with Uncertainty Estimation . in Proceedings - 2023 IEEE International Conference on Multimedia and Expo, ICME 2023 . Proceedings - IEEE International Conference on Multimedia and Expo, vol. 2023-July, IEEE, pp. 1853-1858, IEEE International Conference on Multimedia and Expo, Brisbane, Australia, 10/07/2023 . https://doi.org/10.1109/ICME55011.2023.00318