MixupE: Understanding and improving Mixup from directional derivative perspective
Loading...
Access rights
openAccess
publishedVersion
URL
Journal Title
Journal ISSN
Volume Title
A4 Artikkeli konferenssijulkaisussa
This publication is imported from Aalto University research portal.
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Other link related to publication (opens in new window)
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Other link related to publication (opens in new window)
Date
2023-08
Department
Major/Subject
Mcode
Degree programme
Language
en
Pages
Series
Proceedings of the 39th Conference on Uncertainty in Artificial Intelligence (UAI 2023), pp. 2597-2607, Proceedings of Machine Learning Research ; Volume 216
Abstract
Mixup is a popular data augmentation technique for training deep neural networks where additional samples are generated by linearly interpolating pairs of inputs and their labels. This technique is known to improve the generalization performance in many learning paradigms and applications. In this work, we first analyze Mixup and show that it implicitly regularizes infinitely many directional derivatives of all orders. Based on this new insight, we propose an improved version of Mixup, theoretically justified to deliver better generalization performance than the vanilla Mixup. To demonstrate the effectiveness of the proposed method, we conduct experiments across various domains such as images, tabular data, speech, and graphs. Our results show that the proposed method improves Mixup across multiple datasets using a variety of architectures, for instance, exhibiting an improvement over Mixup by 0.8% in ImageNet top-1 accuracy.Description
Keywords
Other note
Citation
Zou, Y, Verma, V, Mittal, S, Tang, W H, Pham, H, Kannala, J, Bengio, Y, Solin, A & Kawaguchi, K 2023, MixupE: Understanding and improving Mixup from directional derivative perspective . in Proceedings of the 39th Conference on Uncertainty in Artificial Intelligence (UAI 2023) . Proceedings of Machine Learning Research, vol. 216, JMLR, pp. 2597-2607, Conference on Uncertainty in Artificial Intelligence, Pittsburgh, Pennsylvania, United States, 31/07/2023 . < https://proceedings.mlr.press/v216/zou23a.html >