Efficient hyperparameter tuning for kernel ridge regression with Bayesian optimization
Loading...
Access rights
openAccess
publishedVersion
URL
Journal Title
Journal ISSN
Volume Title
A1 Alkuperäisartikkeli tieteellisessä aikakauslehdessä
This publication is imported from Aalto University research portal.
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Other link related to publication (opens in new window)
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Other link related to publication (opens in new window)
Date
2021-09
Department
Major/Subject
Mcode
Degree programme
Language
en
Pages
14
Series
Machine Learning: Science and Technology, Volume 2, issue 3
Abstract
Machine learning methods usually depend on internal parameters-so called hyperparameters-that need to be optimized for best performance. Such optimization poses a burden on machine learning practitioners, requiring expert knowledge, intuition or computationally demanding brute-force parameter searches. We here assess three different hyperparameter selection methods: Grid search, random search and an efficient automated optimization technique based on Bayesian optimization (BO). We apply these methods to a machine learning problem based on kernel ridge regression in computational chemistry. Two different descriptors are employed to represent the atomic structure of organic molecules, one of which introduces its own set of hyperparameters to the method. We identify optimal hyperparameter configurations and infer entire prediction error landscapes in hyperparameter space that serve as visual guides for the hyperparameter performance. We further demonstrate that for an increasing number of hyperparameters, BO and random search become significantly more efficient in computational time than an exhaustive grid search, while delivering an equivalent or even better accuracy.Description
Funding Information: We gratefully acknowledge the CSC-IT Center for Science, Finland, and the Aalto Science-IT project for generous computational resources. This study has received funding from the Magnus Ehrnrooth and the Finnish Cultural Foundation as well as the Academy of Finland through project no. 316601 and through Flagship programme: Finnish Center for Artificial Intelligence FCAI. This article is based on work from COST Action 18234, supported by COST (European Cooperation in Science and Technology). Publisher Copyright: © 2021 IOP Publishing Ltd.
Keywords
Bayesian optimization, Chemical physics, Grid search, Hyperparameter tuning, Kernel ridge regression, Molecular descriptor, Random search
Other note
Citation
Stuke, A, Rinke, P & Todorovic, M 2021, ' Efficient hyperparameter tuning for kernel ridge regression with Bayesian optimization ', Machine Learning: Science and Technology, vol. 2, no. 3, 035022 . https://doi.org/10.1088/2632-2153/abee59