Efficient and Robust Algorithms for Extreme Multilabel Classification

Loading...
Thumbnail Image

URL

Journal Title

Journal ISSN

Volume Title

School of Science | Doctoral thesis (article-based) | Defence date: 2024-11-29

Date

2024

Major/Subject

Mcode

Degree programme

Language

en

Pages

72 + app. 74

Series

Aalto University publication series DOCTORAL THESES, 239/2024

Abstract

Extreme Multilabel Classification (XMC) refers to the problem of finding relevant labels from an extremely large label space, prevalent in applications such as recommender systems, web-scale document tagging, large language models, and question answering systems. This thesis investigates three fundamental challenges in XMC, namely storage and computational efficiency, robustness to data irregularities, and robustness against adversarial attacks. Concerning efficiency, this thesis highlights the high space and computational complexities of using meta classifiers for negative sampling in deep XMC models. It proposes a method utilizing Maximum Inner Product Search (MIPS), which achieves comparable accuracy to methods based on meta classifiers while reducing space and computational demands by eliminating the need to train and store meta classifiers. To address data irregularities, the thesis explores the use of unbiased estimates for tackling the missing labels problem and rebalanced loss functions to manage data imbalance. It discusses the practical optimization challenges related to unbiased estimates, namely non-convexity and non-lower-boundedness of unbiased loss functions. To overcome these issues, it proposes an alternative approach by employing convex surrogates for the unbiased 0-1 loss. Regarding robustness to adversarial attacks, the thesis first defines adversarial attacks within the multilabel context of XMC models for text classification. Then, it evaluates the robustness of XMC models, focusing on the pervasive data imbalance in XMC datasets, which highlights the high vulnerability of infrequent classes to adversarial attacks. Finally, the thesis explores adapting rebalanced convex surrogates, demonstrating their impact on significantly improving the robustness of infrequent classes against these attacks. Together, the findings advance the scalability, accuracy, and security of multilabel classification models in settings with extremely large label spaces.

Description

Supervising professor

Marttinen, Pekka, Prof., Aalto University, Department of Computer Science, Finland

Thesis advisor

Babbar, Rohit, Prof., University of Bath, United Kingdom / Aalto University, Department of Computer Science, Finland

Keywords

machine learning, deep learning, extreme multilabel classification, negative sampling, missing labels, data imbalance, adversarial attacks

Other note

Parts

  • [Publication 1]: Mohammadreza Qaraei, and Rohit Babbar. Meta-classifier free negative sampling for extreme multilabel classification. Machine Learning, volume 113, issue 2, pages 675–697, January 2024.
    DOI: 10.1007/s10994-023-06468-w View at publisher
  • [Publication 2]: Mohammadreza Qaraei, Sujay Khandagale, and Rohit Babbar. Why state-of-the-art deep learning barely works as good as a linear classifier in extreme multi-label text classification. In Proceedings of the European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, Online event, pages 223-228, October 2020. https://urn.fi/URN:NBN:fi:aalto-2021120110556.
  • [Publication 3]: Mohammadreza Qaraei, Erik Schultheis, Priyanshu Gupta, and Rohit Babbar. Convex surrogates for unbiased loss functions in extreme classification with missing labels. In WWW ’21: Proceedings of the Web Conference 2021, Ljubljana, pages 3711–3720, April 2021.
    DOI: 10.1145/3442381.3450139 View at publisher
  • [Publication 4]: Mohammadreza Qaraei, and Rohit Babbar. Adversarial examples for extreme multilabel text classification. Machine Learning, volume 111, issue 12, pages 4539–4563, December 2022.
    DOI: 10.1007/s10994-022-06263-z View at publisher

Citation