Algorithms for approximate Bayesian inference with applications to astronomical data analysis

Thumbnail Image
Journal Title
Journal ISSN
Volume Title
Doctoral thesis (article-based)
Checking the digitized thesis and permission for publishing
Instructions for the author
Degree programme
Verkkokirja (2836 KB, 68 s.)
Bayesian inference is a theoretically well-founded and conceptually simple approach to data analysis. The computations in practical problems are anything but simple though, and thus approximations are almost always a necessity. The topic of this thesis is approximate Bayesian inference and its applications in three intertwined problem domains. Variational Bayesian learning is one type of approximate inference. Its main advantage is its computational efficiency compared to the much applied sampling based methods. Its main disadvantage, on the other hand, is the large amount of analytical work required to derive the necessary components for the algorithm. One part of this thesis reports on an effort to automate variational Bayesian learning of a certain class of models. The second part of the thesis is concerned with heteroscedastic modelling which is synonymous to variance modelling. Heteroscedastic models are particularly suitable for the Bayesian treatment as many of the traditional estimation methods do not produce satisfactory results for them. In the thesis, variance models and algorithms for estimating them are studied in two different contexts: in source separation and in regression. Astronomical applications constitute the third part of the thesis. Two problems are posed. One is concerned with the separation of stellar subpopulation spectra from observed galaxy spectra; the other is concerned with estimating the time-delays in gravitational lensing. Solutions to both of these problems are presented, which heavily rely on the machinery of approximate inference.
machine learning, data analysis, Bayesian inference, variational methods, blind source separation, nonnegative factor analysis, heteroscedasticity, predictive uncertainty, delay estimation, elliptical galaxies, gravitational lenses
Other note
  • [Publication 1]: Tapani Raiko, Harri Valpola, Markus Harva, and Juha Karhunen. 2007. Building blocks for variational Bayesian learning of latent variable models. Journal of Machine Learning Research, volume 8, pages 155-201.
  • [Publication 2]: Markus Harva, Tapani Raiko, Antti Honkela, Harri Valpola, and Juha Karhunen. 2005. Bayes Blocks: An implementation of the variational Bayesian building blocks framework. In: Proceedings of the 21st Conference on Uncertainty in Artificial Intelligence (UAI 2005), Edinburgh, Scotland, 26-29 July 2005, pages 259-266.
  • [Publication 3]: Harri Valpola, Markus Harva, and Juha Karhunen. 2004. Hierarchical models of variance sources. Signal Processing, volume 84, number 2, pages 267-282.
  • [Publication 4]: Markus Harva. 2007. A variational EM approach to predictive uncertainty. Neural Networks, volume 20, number 4, pages 550-558.
  • [Publication 5]: Markus Harva and Ata Kabán. 2007. Variational learning for rectified factor analysis. Signal Processing, volume 87, number 3, pages 509-527.
  • [Publication 6]: Louisa A. Nolan, Markus O. Harva, Ata Kabán, and Somak Raychaudhury. 2006. A data-driven Bayesian approach for finding young stellar populations in early-type galaxies from their ultraviolet–optical spectra. Monthly Notices of the Royal Astronomical Society, volume 366, number 1, pages 321-338.
  • [Publication 7]: Markus Harva and Somak Raychaudhury. 2008. Bayesian estimation of time delays between unevenly sampled signals. Neurocomputing, to appear.