Advances in Extreme Learning Machines

 |  Login

Show simple item record

dc.contributor Aalto-yliopisto fi
dc.contributor Aalto University en
dc.contributor.advisor Miche, Yoan, Dr., Aalto University, Department of Information and Computer Science, Finland
dc.contributor.author van Heeswijk, Mark
dc.date.accessioned 2015-04-08T09:00:31Z
dc.date.available 2015-04-08T09:00:31Z
dc.date.issued 2015
dc.identifier.isbn 978-952-60-6149-8 (electronic)
dc.identifier.isbn 978-952-60-6148-1 (printed)
dc.identifier.issn 1799-4942 (electronic)
dc.identifier.issn 1799-4934 (printed)
dc.identifier.issn 1799-4934 (ISSN-L)
dc.identifier.uri https://aaltodoc.aalto.fi/handle/123456789/15585
dc.description.abstract Nowadays, due to advances in technology, data is generated at an incredible pace, resulting in large data sets of ever-increasing size and dimensionality. Therefore, it is important to have efficient computational methods and machine learning algorithms that can handle such large data sets, such that they may be analyzed in reasonable time. One particular approach that has gained popularity in recent years is the Extreme Learning Machine (ELM), which is the name given to neural networks that employ randomization in their hidden layer, and that can be trained efficiently. This dissertation introduces several machine learning methods based on Extreme Learning Machines (ELMs) aimed at dealing with the challenges that modern data sets pose. The contributions follow three main directions.    Firstly, ensemble approaches based on ELM are developed, which adapt to context and can scale to large data. Due to their stochastic nature, different ELMs tend to make different mistakes when modeling data. This independence of their errors makes them good candidates for combining them in an ensemble model, which averages out these errors and results in a more accurate model. Adaptivity to a changing environment is introduced by adapting the linear combination of the models based on accuracy of the individual models over time. Scalability is achieved by exploiting the modularity of the ensemble model, and evaluating the models in parallel on multiple processor cores and graphics processor units. Secondly, the dissertation develops variable selection approaches based on ELM and Delta Test, that result in more accurate and efficient models. Scalability of variable selection using Delta Test is again achieved by accelerating it on GPU. Furthermore, a new variable selection method based on ELM is introduced, and shown to be a competitive alternative to other variable selection methods. Besides explicit variable selection methods, also a new weight scheme based on binary/ternary weights is developed for ELM. This weight scheme is shown to perform implicit variable selection, and results in increased robustness and accuracy at no increase in computational cost. Finally, the dissertation develops training algorithms for ELM that allow for a flexible trade-off between accuracy and computational time. The Compressive ELM is introduced, which allows for training the ELM in a reduced feature space. By selecting the dimension of the feature space, the practitioner can trade off accuracy for speed as required.    Overall, the resulting collection of proposed methods provides an efficient, accurate and flexible framework for solving large-scale supervised learning problems. The proposed methods are not limited to the particular types of ELMs and contexts in which they have been tested, and can easily be incorporated in new contexts and models. en
dc.format.extent 108 + app. 84
dc.format.mimetype application/pdf en
dc.language.iso en en
dc.publisher Aalto University en
dc.publisher Aalto-yliopisto fi
dc.relation.ispartofseries Aalto University publication series DOCTORAL DISSERTATIONS en
dc.relation.ispartofseries 43/2015
dc.relation.haspart [Publication 1]: Mark van Heeswijk, Yoan Miche, Tiina Lindh-Knuutila, Peter A.J. Hilbers, Timo Honkela, Erkki Oja, and Amaury Lendasse. Adaptive Ensemble Models of Extreme Learning Machines for Time Series Prediction. In LNCS 5769 - Artificial Neural Networks, ICANN’09: International Conference on Artificial Neural Networks, pp. 305-314, September 2009. doi:10.1007/978-3-642-04277-5_31.
dc.relation.haspart [Publication 2]: Mark van Heeswijk, Yoan Miche, Erkki Oja, and Amaury Lendasse. GPU-accelerated and parallelized ELM ensembles for large-scale regression. Neurocomputing, 74 (16): pp. 2430-2437, September 2011. doi:10.1016/j.neucom.2010.11.034.
dc.relation.haspart [Publication 3]: Benoît Frenay, Mark van Heeswijk, Yoan Miche, Michel Verleysen, and Amaury Lendasse. Feature selection for nonlinear models with extreme learning machines. Neurocomputing, 102, pp. 111-124, February 2013. doi:10.1016/j.neucom.2011.12.055.
dc.relation.haspart [Publication 4]: Alberto Guillén, Maribel García Arenas, Mark van Heeswijk, Dušan Sovilj, Amaury Lendasse, Luis Herrera, Hector Pomares and Ignacio Rojas. Fast Feature Selection in a GPU Cluster Using the Delta Test. Entropy, 16 (2): pp. 854-869, 2014. doi:10.3390/e16020854.
dc.relation.haspart [Publication 5]: Mark van Heeswijk, and Yoan Miche. Binary/Ternary Extreme Learning Machines. Neurocomputing, 149, pp. 187-197, February 2015. doi:10.1016/j.neucom.2014.01.072.
dc.relation.haspart [Publication 6]: Mark van Heeswijk, Amaury Lendasse, and Yoan Miche. Compressive ELM: Improved Models Through Exploiting Time-Accuracy Trade-offs. In CCIS 459 - Engineering Applications of Neural Networks, pp. 165-174, 2014. doi:10.1007/978-3-319-11071-4_16.
dc.subject.other Computer science en
dc.title Advances in Extreme Learning Machines en
dc.type G5 Artikkeliväitöskirja fi
dc.contributor.school Perustieteiden korkeakoulu fi
dc.contributor.school School of Science en
dc.contributor.department Tietojenkäsittelytieteen laitos fi
dc.contributor.department Department of Information and Computer Science en
dc.subject.keyword Extreme Learning Machine (ELM) en
dc.subject.keyword high-performance computing en
dc.subject.keyword ensemble models en
dc.subject.keyword variable selection en
dc.subject.keyword random projection en
dc.subject.keyword machine learning en
dc.identifier.urn URN:ISBN:978-952-60-6149-8
dc.type.dcmitype text en
dc.type.ontasot Doctoral dissertation (article-based) en
dc.type.ontasot Väitöskirja (artikkeli) fi
dc.contributor.supervisor Oja, Erkki, Aalto Distinguished Prof., Aalto University, Department of Information and Computer Science, Finland
dc.opn Wunsch, Donald C., Prof., Missouri University of Science & Technology, United States
dc.date.dateaccepted 2015-03-09
dc.contributor.lab Environmental and Industrial Machine Learning Group en
dc.contributor.lab Ympäristön ja teollisuuden alojen koneoppiminen fi
dc.rev Huang, Guang-Bin, Prof., Nanyang Technological University, Singapore
dc.rev Tapson, Jonathan, Prof., University of Western Sydney, Australia
dc.date.defence 2015-04-17


Files in this item

This item appears in the following Collection(s)

Show simple item record

Search archive


Advanced Search

article-iconSubmit a publication

Browse

My Account