Multi-node Training for StyleGAN2

Thumbnail Image

Access rights

openAccess

URL

Journal Title

Journal ISSN

Volume Title

A4 Artikkeli konferenssijulkaisussa
This publication is imported from Aalto University research portal.
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Other link related to publication (opens in new window)

Authors

Date

2021

Department

Department of Computer Science

Major/Subject

Mcode

Degree programme

Language

en

Pages

8
677-684

Series

Pattern Recognition. ICPR International Workshops and Challenges, 2021, Proceedings, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Volume 12661 LNCS

Abstract

StyleGAN2 is a Tensorflow-based Generative Adversarial Network (GAN) framework that represents the state-of-the-art in generative image modelling. The current release of StyleGAN2 implements multi-GPU training via Tensorflow’s device contexts which limits data parallelism to a single node. In this work, a data-parallel multi-node training capability is implemented in StyleGAN2 via Horovod which enables harnessing the compute capability of larger cluster architectures. We demonstrate that the new Horovod-based communication outperforms the previous context approach on a single node. Furthermore, we demonstrate that the multi-node training does not compromise the accuracy of StyleGAN2 for a constant effective batch size. Finally, we report strong and weak scaling of the new implementation up to 64 NVIDIA Tesla A100 GPUs distributed across eight NVIDIA DGX A100 nodes, demonstrating the utility of the approach at scale.

Description

Publisher Copyright: © 2021, Springer Nature Switzerland AG. Copyright: Copyright 2021 Elsevier B.V., All rights reserved.

Keywords

GAN, GPU, Massively parallel architectures, Multi-node training, StyleGAN2

Other note

DOI

10.1007/978-3-030-68763-2_51

Citation

Loppi, N & Kynkäänniemi, T 2021, Multi-node Training for StyleGAN2 . in A Del Bimbo, R Cucchiara, S Sclaroff, G M Farinella, T Mei, M Bertini, H J Escalante & R Vezzani (eds), Pattern Recognition. ICPR International Workshops and Challenges, 2021, Proceedings . Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12661 LNCS, Springer, pp. 677-684, International Conference on Pattern Recognition, Milan, Italy, 10/01/2021 . https://doi.org/10.1007/978-3-030-68763-2_51

Permanent link to this item

https://urn.fi/URN:NBN:fi:aalto-2021111110102

Collections

[article-cris] Perustieteiden korkeakoulu / SCI