Rethinking Inference in Gaussian Processes: A Dual Parameterization Approach
Loading...
URL
Journal Title
Journal ISSN
Volume Title
School of Science |
Doctoral thesis (article-based)
| Defence date: 2024-11-22
Unless otherwise stated, all rights belong to the author. You may download, display and print this publication for Your own personal use. Commercial use is prohibited.
Authors
Date
2024
Major/Subject
Mcode
Degree programme
Language
en
Pages
59 + app. 53
Series
Aalto University publication series DOCTORAL THESES, 243/2024
Abstract
Uncertainty quantification is a vital aspect of machine learning, especially when accurate estimates of uncertainty are crucial for making informed decisions. Gaussian Processes (GPs), known for their versatility as function space priors, find wide-ranging applications in diverse fields, including climate modelling and epidemiology. GPs are particularly useful due to their non-linearity, allowing them to adapt to various data patterns and their ability to integrate domain-specific knowledge. As probabilistic models, they offer predictions and quantify the uncertainty within these predictions, an essential feature in scenarios demanding reliable forecasts. This thesis focuses on applying GPs to large-scale, non-Gaussian sequential data. Due to their non-parametric nature, GPs face increasing computational demands as data size grows. The requirement for a conjugate Gaussian likelihood for computational tractability presents further challenges. Therefore, it is common to use approximate inference for applying GPs to non-Gaussian likelihoods alongside scalable model formulations to handle complex data distributions in real- world applications. The theme connecting the papers is an innovative approach to parameterizing the optimization problem in approximate inference, centring on a forgotten parameterization termed the dual parameters. This fresh perspective offers methods to enhance the efficiency of GPs in applying them to large and complex datasets, particularly in the context of sequential data. This approach addresses the pivotal challenges of tractability and scalability inherent in GPs in the sequential setting. The concept of dual parameters serves as a unifying framework, linking all approximate inference techniques through their various likelihood approximations. In addition, the thesis shows the application of dual parameterization methods in a range of GP model formulations and problem settings. It introduces a new algorithm for inference and learning in non-Gaussian time series data and the sparse GP framework. The applications discussed extend to areas such as Bayesian optimization and continual learning, highlighting the adaptability and potential of GPs in contemporary machine learning.Description
Supervising professor
Solin, Arno, Prof., Aalto University, Department of Computer Science, FinlandKeywords
machine learning, Gaussian Processes, climate modelling, epidemiology
Other note
Parts
-
[Publication 1]: Paul E. Chang, William J. Wilkinson, Mohammad Emtiyaz Khan and Arno Solin. Fast Variational Learning in State-Space Gaussian Process Models. In 30th International Workshop on Machine Learning for Signal Processing (MLSP), Finland, pages 1-6, IEEE 2020.
DOI: 10.1109/MLSP49062.2020.9231560 View at publisher
- [Publication 2]: Wilkinson J. William, Paul E. Chang, Michael Andersen, and Arno Solin. State Space Expectation Propagation: Efficient Inference Schemes for Temporal Gaussian Processes. In International Conference on Machine Learning (ICML), Vienna, pages 10270-10281, PMLR 020. https://urn.fi/URN:NBN:fi:aalto-2020123160394.
- [Publication 3]: Vincent, Adam∗, Paul E. Chang∗, Mohammad Emtiyaz Khan, and Arno Solin. Dual Parameterization of Sparse Variational Gaussian Processes. In Advances in Neural Information Processing Systems 34 (NeurIPS), pages 11474-11486, Curran Associates Inc 2021. https://urn.fi/URN:NBN:fi:aalto-202202091869.
- [Publication 4]: Paul E. Chang∗, Prakhar Verma∗, S. T. John, Arno Solin, and Mohammad Emtiyaz Khan. Memory-Based Dual Gaussian Processes for Sequential Learning. In International Conference on Machine Learning, (ICML), Hawaii, pages 4035-4054, PMLR 2023. https://urn.fi/URN:NBN:fi:aalto-202309135861.