Differentially deep Gaussian processes
No Thumbnail Available
Journal Title
Journal ISSN
Volume Title
Perustieteiden korkeakoulu |
Master's thesis
Author
Date
2019-01-28
Department
Major/Subject
Machine Learning and Data Mining
Mcode
SCI3044
Degree programme
Master’s Programme in Computer, Communication and Information Sciences
Language
en
Pages
47
Series
Abstract
Many modern machine learning methods, including deep neural networks, utilize a discrete sequence of parametric transformations to learn complex functions. Neural network based approaches can be an attractive choice for many real-world problems especially because of their modular nature. Gaussian process based methods, on the other hand, pose function approximation as a probabilistic inference problem by specifying prior distributions on unknown functions. Further, these probabilistic non-linear models provide well-calibrated uncertainty estimates which can be useful in many applications. However, the flexibility of these models depends on the choice of the kernel; handcrafting problem-specific kernels can be difficult in practice. Recently, deep Gaussian processes, a way of stacking multiple layers of Gaussian processes, was proposed as a flexible way of expanding model capacity. In this thesis, we propose a novel probabilistic deep learning approach by formulating stochastic differential transformations or `flows' of inputs using Gaussian processes. This provides continuous-time `flows' as an alternative to the traditional approach of a discrete sequence of transformations using `layers'. Moreover, the proposed approach can also be seen as an approximation to very deep Gaussian processes with infinitesimal increments across layers. We also derive a scalable inference method based on variational sparse approximations for Gaussian processes. The proposed model shows excellent results on various experiments on real-world datasets, as compared to the other popular probabilistic approaches including deep Gaussian processes and Bayesian neural networks.Description
Supervisor
Kaski, SamuelThesis advisor
Heinonen, MarkusKeywords
Bayesian deep learning, Bayesian nonparametrics, Gaussian processes, stochastic methods
