Realistic text replacement with non-uniform style conditioning

Loading...
Thumbnail Image
Journal Title
Journal ISSN
Volume Title
A1 Alkuperäisartikkeli tieteellisessä aikakauslehdessä
Date
2021
Major/Subject
Mcode
Degree programme
Language
en
Pages
9
92706-92714
Series
IEEE Access, Volume 9
Abstract
In this work, we study the possibility of realistic text replacement. The goal of realistic text replacement is to replace text present in the image with user-supplied text. The replacement should be performed in a way that will not allow distinguishing the resulting image from the original one. We achieve this goal by developing a novel non-uniform style conditioning layer and apply it to an encoder-decoder ResNet based architecture. The resulting model is a single-stage model, with no post-processing. We train the model with a combination of adversarial, style, content and L-1 losses. Qualitative and quantitative evaluations show that the model achieves realistic text replacement and outperforms existing approaches in multilingual and challenging scenarios. Quantitative evaluation is performed with direct metrics, like SSIM and PSNR, and proxy metrics based on the performance of a text recognition model. The proposed model has several potential applications in augmented reality.
Description
Publisher Copyright: CCBY Copyright: Copyright 2021 Elsevier B.V., All rights reserved.
Keywords
GAN, Style conditioning, Text replacement
Other note
Citation
Nerinovsky , A , Buzhinsky , I & Filchenkov , A 2021 , ' Realistic text replacement with non-uniform style conditioning ' , IEEE Access , vol. 9 , 9398684 , pp. 92706-92714 . https://doi.org/10.1109/ACCESS.2021.3071666