Efficient and Robust Algorithms for Extreme Multilabel Classification
Loading...
URL
Journal Title
Journal ISSN
Volume Title
School of Science |
Doctoral thesis (article-based)
| Defence date: 2024-11-29
Unless otherwise stated, all rights belong to the author. You may download, display and print this publication for Your own personal use. Commercial use is prohibited.
Authors
Date
2024
Major/Subject
Mcode
Degree programme
Language
en
Pages
72 + app. 74
Series
Aalto University publication series DOCTORAL THESES, 239/2024
Abstract
Extreme Multilabel Classification (XMC) refers to the problem of finding relevant labels from an extremely large label space, prevalent in applications such as recommender systems, web-scale document tagging, large language models, and question answering systems. This thesis investigates three fundamental challenges in XMC, namely storage and computational efficiency, robustness to data irregularities, and robustness against adversarial attacks. Concerning efficiency, this thesis highlights the high space and computational complexities of using meta classifiers for negative sampling in deep XMC models. It proposes a method utilizing Maximum Inner Product Search (MIPS), which achieves comparable accuracy to methods based on meta classifiers while reducing space and computational demands by eliminating the need to train and store meta classifiers. To address data irregularities, the thesis explores the use of unbiased estimates for tackling the missing labels problem and rebalanced loss functions to manage data imbalance. It discusses the practical optimization challenges related to unbiased estimates, namely non-convexity and non-lower-boundedness of unbiased loss functions. To overcome these issues, it proposes an alternative approach by employing convex surrogates for the unbiased 0-1 loss. Regarding robustness to adversarial attacks, the thesis first defines adversarial attacks within the multilabel context of XMC models for text classification. Then, it evaluates the robustness of XMC models, focusing on the pervasive data imbalance in XMC datasets, which highlights the high vulnerability of infrequent classes to adversarial attacks. Finally, the thesis explores adapting rebalanced convex surrogates, demonstrating their impact on significantly improving the robustness of infrequent classes against these attacks. Together, the findings advance the scalability, accuracy, and security of multilabel classification models in settings with extremely large label spaces.Description
Supervising professor
Marttinen, Pekka, Prof., Aalto University, Department of Computer Science, FinlandThesis advisor
Babbar, Rohit, Prof., University of Bath, United Kingdom / Aalto University, Department of Computer Science, FinlandKeywords
machine learning, deep learning, extreme multilabel classification, negative sampling, missing labels, data imbalance, adversarial attacks
Other note
Parts
-
[Publication 1]: Mohammadreza Qaraei, and Rohit Babbar. Meta-classifier free negative sampling for extreme multilabel classification. Machine Learning, volume 113, issue 2, pages 675–697, January 2024.
Full text in Acris/Aaltodoc: https://urn.fi/URN:NBN:fi:aalto-202401312176DOI: 10.1007/s10994-023-06468-w View at publisher
- [Publication 2]: Mohammadreza Qaraei, Sujay Khandagale, and Rohit Babbar. Why state-of-the-art deep learning barely works as good as a linear classifier in extreme multi-label text classification. In Proceedings of the European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, Online event, pages 223-228, October 2020. https://urn.fi/URN:NBN:fi:aalto-2021120110556.
-
[Publication 3]: Mohammadreza Qaraei, Erik Schultheis, Priyanshu Gupta, and Rohit Babbar. Convex surrogates for unbiased loss functions in extreme classification with missing labels. In WWW ’21: Proceedings of the Web Conference 2021, Ljubljana, pages 3711–3720, April 2021.
Full text in Acris/Aaltodoc: https://urn.fi/URN:NBN:fi:aalto-202108098273DOI: 10.1145/3442381.3450139 View at publisher
-
[Publication 4]: Mohammadreza Qaraei, and Rohit Babbar. Adversarial examples for extreme multilabel text classification. Machine Learning, volume 111, issue 12, pages 4539–4563, December 2022.
Full text in Acris/Aaltodoc: https://urn.fi/URN:NBN:fi:aalto-202212146935DOI: 10.1007/s10994-022-06263-z View at publisher