Guiding a Diffusion Model with a Bad Version of Itself
Loading...
Access rights
openAccess
URL
Journal Title
Journal ISSN
Volume Title
A4 Artikkeli konferenssijulkaisussa
This publication is imported from Aalto University research portal.
View publication in the Research portal (opens in new window)
Other link related to publication (opens in new window)
View publication in the Research portal (opens in new window)
Other link related to publication (opens in new window)
Date
Department
Major/Subject
Mcode
Degree programme
Language
en
Pages
Series
Advances in Neural Information Processing Systems 37 (NeurIPS 2024)
Abstract
The primary axes of interest in image-generating diffusion models are image quality, the amount of variation in the results, and how well the results align with a given condition, e.g., a class label or a text prompt. The popular classifier-free guidance approach uses an unconditional model to guide a conditional model, leading to simultaneously better prompt alignment and higher-quality images at the cost of reduced variation. These effects seem inherently entangled, and thus hard to control. We make the surprising observation that it is possible to obtain disentangled control over image quality without compromising the amount of variation by guiding generation using a smaller, less-trained version of the model itself rather than an unconditional model. This leads to significant improvements in ImageNet generation, setting record FIDs of 1.01 for 64x64 and 1.25 for 512x512, using publicly available networks. Furthermore, the method is also applicable to unconditional diffusion models, drastically improving their quality.Description
| openaire: EC/H2020/866435/EU//PIPE
Keywords
Other note
Citation
Karras, T, Aittala, M, Kynkäänniemi, T, Lehtinen, J, Aila, T & Laine, S 2025, Guiding a Diffusion Model with a Bad Version of Itself. in A Globerson, L Mackey, D Belgrave, A Fan, U Paquet, J Tomczak & C Zhang (eds), Advances in Neural Information Processing Systems 37 (NeurIPS 2024). Curran Associates Inc., Conference on Neural Information Processing Systems, Vancouver, Canada, 10/12/2024. < https://arxiv.org/abs/2406.02507 >