Building heterogeneous ensembles by pooling homogeneous ensembles
Loading...
Access rights
openAccess
URL
Journal Title
Journal ISSN
Volume Title
A1 Alkuperäisartikkeli tieteellisessä aikakauslehdessä
This publication is imported from Aalto University research portal.
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Other link related to publication (opens in new window)
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Other link related to publication (opens in new window)
Authors
Sabzevari, Maryam
Martínez-Muñoz, Gonzalo
Suárez, Alberto
Date
2021-10-13
Department
Major/Subject
Mcode
Degree programme
Language
en
Pages
8
Series
International Journal of Machine Learning and Cybernetics
Abstract
Heterogeneous ensembles consist of predictors of different types, which are likely to have different biases. If these biases are complementary, the combination of their decisions is beneficial and could be superior to homogeneous ensembles. In this paper, a family of heterogeneous ensembles is built by pooling classifiers from M homogeneous ensembles of different types of size T. Depending on the fraction of base classifiers of each type, a particular heterogeneous combination in this family is represented by a point in a regular simplex in M dimensions. The M vertices of this simplex represent the different homogeneous ensembles. A displacement away from one of these vertices effects a smooth transformation of the corresponding homogeneous ensemble into a heterogeneous one. The optimal composition of such heterogeneous ensemble can be determined using cross-validation or, if bootstrap samples are used to build the individual classifiers, out-of-bag data. The proposed heterogeneous ensemble building strategy, composed of neural networks, SVMs, and random trees (i.e. from a standard random forest), is analyzed in a comprehensive empirical analysis and compared to a benchmark of other heterogeneous and homogeneous ensembles. The achieved results illustrate the gains that can be achieved by the proposed ensemble creation method with respect to both homogeneous ensembles and to the tested heterogeneous building strategy at a fraction of the training cost.Description
Publisher Copyright: © 2021, The Author(s).
Keywords
Ensembles, Heterogeneous, Homogeneous, Optimal composition, Simplex
Other note
Citation
Sabzevari, M, Martínez-Muñoz, G & Suárez, A 2021, ' Building heterogeneous ensembles by pooling homogeneous ensembles ', International Journal of Machine Learning and Cybernetics, vol. 13, no. 2, pp. 551-558 . https://doi.org/10.1007/s13042-021-01442-1