Training methods for climate and neural network models
Loading...
Journal Title
Journal ISSN
Volume Title
School of Science |
Doctoral thesis (article-based)
| Defence date: 2018-11-15
Unless otherwise stated, all rights belong to the author. You may download, display and print this publication for Your own personal use. Commercial use is prohibited.
Author
Date
2018
Major/Subject
Mcode
Degree programme
Language
en
Pages
56 + app. 76
Series
Aalto University publication series DOCTORAL DISSERTATIONS, 209/2018
Abstract
When modeling complex phenomena in nature and in technological systems, one is often faced with the task of tuning/calibrating the models. In such cases, there typically exists a need for model parameter (and/or meta-parameter) value tuning for more effective modeling performance. Often such cannot be done manually, and in the machine learning approach, the tuning is done in an algorithmic and data-driven manner, and is called model training. The thesis presents studies in which such methods are adopted, in the contexts of climate and artificial neural networks, and proposes novel techniques. One of the studies is on the suitability of a well-known machine learning method called Bayesian optimization (BO), for parametric tuning of chaotic systems such as climate and numerical weatherprediction (NWP) models. The obtained results show that BO is a suitable method for such tuning tasks. A major desiderata for a trained machine learning model is the ability to generalize well to unseen data, and thus the phenomena such as (so-called) under- and overfitting are to be avoided. In this context, adopting (so-called) regularization methods as part of the model training process has become a standard procedure. In this thesis, we introduce a regularization framework that is shown to have close connections with many existing state-of-the-art regularization approaches. An adversarial variant, derived from the proposed regularization framework, is used for solving a classification task, and the obtained results are compared to those of other regularization methods.Description
Supervising professor
Kaski, Kimmo, Prof., Aalto University, Department of Computer Science, FinlandThesis advisor
Kivinen, Jyri, Dr., Aalto University, Department of Computer Science, FinlandKeywords
chaotic systems, filtering, bayesian optimization
Other note
Parts
-
[Publication 1]: Solonen, A., Hakkarainen, J., Ilin, A., Abbas, M., Bibov, A. Estimating model error covariance matrix parameters in extended Kalman filtering. Nonlinear Processes in Geophysics, 21, 5, 919–927, 2014.
DOI: 10.5194/npg-21-919-2014 View at publisher
-
[Publication 2]: Abbas, M., Ilin, A., Solonen, M., Hakkarainen, J., Oja, E., Järvinen, H. Bayesian optimization for tuning chaotic systems. Nonlin. Processes Geophys. Discuss., 1, 1283–1312, 2014.
DOI: 10.5194/npgd-1-1283-2014 View at publisher
-
[Publication 3]: Abbas, M., Ilin, A., Solonen, M., Hakkarainen, J., Oja, E., Järvinen, H. Empirical evaluation of Bayesian optimization in parametric tuning of chaotic systems. Int. J. Uncertainty Quantification, 6, 6, 467–485, 2016.
DOI: 10.1615/Int.J.UncertaintyQuantification.2016016645 View at publisher
- [Publication 4]: Abbas, M., Kivinen, J., Raiko, T. Understanding regularization by virtual adversarial training, ladder networks and others. In International Conference on Learning Representations (ICLR) Workshop track, Puerto Rico, May 2016
