Realistic text replacement with non-uniform style conditioning
Loading...
Access rights
openAccess
publishedVersion
URL
Journal Title
Journal ISSN
Volume Title
A1 Alkuperäisartikkeli tieteellisessä aikakauslehdessä
This publication is imported from Aalto University research portal.
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Date
Major/Subject
Mcode
Degree programme
Language
en
Pages
9
Series
IEEE Access, Volume 9, pp. 92706-92714
Abstract
In this work, we study the possibility of realistic text replacement. The goal of realistic text replacement is to replace text present in the image with user-supplied text. The replacement should be performed in a way that will not allow distinguishing the resulting image from the original one. We achieve this goal by developing a novel non-uniform style conditioning layer and apply it to an encoder-decoder ResNet based architecture. The resulting model is a single-stage model, with no post-processing. We train the model with a combination of adversarial, style, content and L-1 losses. Qualitative and quantitative evaluations show that the model achieves realistic text replacement and outperforms existing approaches in multilingual and challenging scenarios. Quantitative evaluation is performed with direct metrics, like SSIM and PSNR, and proxy metrics based on the performance of a text recognition model. The proposed model has several potential applications in augmented reality.Description
Publisher Copyright: CCBY Copyright: Copyright 2021 Elsevier B.V., All rights reserved.
Keywords
Other note
Citation
Nerinovsky, A, Buzhinsky, I & Filchenkov, A 2021, 'Realistic text replacement with non-uniform style conditioning', IEEE Access, vol. 9, 9398684, pp. 92706-92714. https://doi.org/10.1109/ACCESS.2021.3071666