Multilabel Classification through Structured Output Learning - Methods and Applications
Loading...
Journal Title
Journal ISSN
Volume Title
School of Science |
Doctoral thesis (article-based)
| Defence date: 2015-03-27
Checking the digitized thesis and permission for publishing
Instructions for the author
Instructions for the author
Unless otherwise stated, all rights belong to the author. You may download, display and print this publication for Your own personal use. Commercial use is prohibited.
Author
Date
2015
Major/Subject
Mcode
Degree programme
Language
en
Pages
82 + app. 97
Series
Aalto University publication series DOCTORAL DISSERTATIONS, 28/2015
Abstract
Multilabel classification is an important topic in machine learning that arises naturally from many real world applications. For example, in document classification, a research article can be categorized as “science”, “drug discovery” and “genomics” at the same time. The goal of multilabel classification is to reliably predict multiple outputs for a given input. As multiple interdependent labels can be “on” and “off” simultaneously, the central problem in multilabel classification is how to best exploit the correlation between labels to make accurate predictions. Compared to the previous flat multilabel classification approaches which treat multiple labels as a flat vector, structured output learning relies on an output graph connecting multiple labels to model the correlation between labels in a comprehensive manner. The main question studied in this thesis is how to tackle multilabel classification through structured output learning. This thesis starts with an extensive review on the topic of classification learning including both single-label and multilabel classification. The first problem we address is how to solve the multilabel classification problem when the output graph is observed apriori. We discuss several well-established structured output learning algorithms and study the network response prediction problem within the context of social network analysis. As the current structured output learning algorithms rely on the output graph to exploit the dependency between labels, the second problem we address is how to use structured output learning when the output graph is not known. Specifically, we examine the potential of learning on a set of random output graphs when the “real” one is hidden. This problem is relevant as in most multilabel classification problems there does not exist any output graph that reveals the dependency between labels. The third problem we address is how to analyze the proposed learning algorithms in a theoretical manner. Specifically, we want to explain the intuition behind the proposed models and to study the generalization error. The main contributions of this thesis are several new learning algorithms that widen the applicability of structured output learning. For the problem with an observed output graph, the proposed algorithm “SPIN” is able to predict an optimal directed acyclic graph from an observed underlying network that best responses to an input. For general multilabel classification problems without any known output graph, we proposed several learning algorithms that combine a set of structured output learners built on random output graphs. In addition, we develop a joint learning and inference framework which is based on max-margin learning over a random sample of spanning trees. The theoretic analysis also guarantees the generalization error of the proposed methods.Description
Supervising professor
Rousu, Juho, Prof., Aalto University, Department of Computer Science, FinlandKeywords
machine learning, classification, structured prediction, large margin methods, graphical models, social network
Other note
Parts
- [Publication 1]: Hongyu Su, Aristides Gionis, Juho Rousu. Structured Prediction of Network Response. In Proceedings of the 31th International Conference on Machine Learning (ICML 2014), Beijing, China, 2014. Journal of Machine Learning Research (JMLR) W&CP volume 32:442-450, June 2014.
- [Publication 2]: Hongyu Su, Markus Heinonen, Juho Rousu. Multilabel Classification of Drug-like Molecules via Max-margin Conditional Random Fields. In Proceedings of the 5th International Conference on Pattern Recognition in Bioinformatics (PRIB 2010), Nijmegen, The Netherlands, 2010. Springer LNBI volume 6282:265-273, September 2010.
- [Publication 3]: Hongyu Su, Juho Rousu. Multi-task Drug Bioactivity Classification with Graph Labeling Ensembles. In Proceedings of the 6th International Conference on Pattern Recognition in Bioinformatics (PRIB 2011), Delft, The Netherlands, 2011. Springer LNBI volume 7035:157-167, November 2011.
-
[Publication 4]: Hongyu Su, Juho Rousu. Multilabel Classification through Random Graph Ensembles. Machine Learning, 26 Pages, September 2014.
DOI: 10.1007/s10994-014-5465-9 View at publisher
- [Publication 5]: Mario Marchand, Hongyu Su, Emilie Morvant, Juho Rousu, John Shawe-Taylor. Multilabel Structured Output Learning with Random Spanning Trees of Max-Margin Markov Networks. In Advances in Neural Information Processing Systems 27 (NIPS 2014), 873-881, December 2014.
