Advanced source separation methods with applications to spatio-temporal datasets

No Thumbnail Available
Journal Title
Journal ISSN
Volume Title
Doctoral thesis (article-based)
Checking the digitized thesis and permission for publishing
Instructions for the author
Degree programme
120, [96]
Dissertations in computer and information science. Report D, 16
Latent variable models are useful tools for statistical data analysis in many applications. Examples of popular models include factor analysis, state-space models and independent component analysis. These types of models can be used for solving the source separation problem in which the latent variables should have a meaningful interpretation and represent the actual sources generating data. Source separation methods is the main focus of this work. Bayesian statistical theory provides a principled way to learn latent variable models and therefore to solve the source separation problem. The first part of this work studies variational Bayesian methods and their application to different latent variable models. The properties of variational Bayesian methods are investigated both theoretically and experimentally using linear source separation models. A new nonlinear factor analysis model which restricts the generative mapping to the practically important case of post-nonlinear mixtures is presented. The variational Bayesian approach to learning nonlinear state-space models is studied as well. This method is applied to the practical problem of detecting changes in the dynamics of complex nonlinear processes. The main drawback of Bayesian methods is their high computational burden. This complicates their use for exploratory data analysis in which observed data regularities often suggest what kind of models could be tried. Therefore, the second part of this work proposes several faster source separation algorithms implemented in a common algorithmic framework. The proposed approaches separate the sources by analyzing their spectral contents, decoupling their dynamic models or by optimizing their prominent variance structures. These algorithms are applied to spatio-temporal datasets containing global climate measurements from a long period of time.
Bayesian learning, blind source separation, global climate, denoising source separation, frequency-based separation, independent component analysis, independent subspace analysis, latent variable models, nonstationarity of variance, post-nonlinear mixing, unsupervised learning, variational methods
Other note
  • A. Ilin and H. Valpola. On the effect of the form of the posterior approximation in variational learning of ICA models. Neural Processing Letters, Vol. 22, No. 2, pages 183-204, October 2005. [article1.pdf] © 2005 Springer Science+Business Media. By permission.
  • A. Ilin, H. Valpola, and E. Oja. Nonlinear dynamical factor analysis for state change detection. IEEE Transactions on Neural Networks, Vol. 15, No. 3, pages 559-575, May 2004. [article2.pdf] © 2004 IEEE. By permission.
  • A. Ilin, S. Achard, and C. Jutten. Bayesian versus constrained structure approaches for source separation in post-nonlinear mixtures. In Proceedings of the International Joint Conference on Neural Networks (IJCNN 2004), pages 2181-2186, Budapest, Hungary, July 2004. [article3.pdf] © 2004 IEEE. By permission.
  • A. Ilin and A. Honkela. Post-nonlinear independent component analysis by variational Bayesian learning. In Proceedings of the Fifth International Conference on Independent Component Analysis and Blind Signal Separation (ICA 2004), pages 766-773, Granada, Spain, September 2004. [article4.pdf] © 2004 Springer Science+Business Media. By permission.
  • A. Ilin, H. Valpola, and E. Oja. Semiblind source separation of climate data detects El Niño as the component with the highest interannual variability. In Proceedings of the International Joint Conference on Neural Networks (IJCNN 2005), pages 1722-1727, Montréal, Québec, Canada, August 2005. [article5.pdf] © 2005 IEEE. By permission.
  • A. Ilin and H. Valpola. Frequency-based separation of climate signals. In Proceedings of the 9th European Conference on Principles and Practice of Knowledge Discovery in Databases (PKDD 2005), pages 519-526, Porto, Portugal, October 2005. [article6.pdf] © 2005 Springer Science+Business Media. By permission.
  • A. Ilin, H. Valpola, and E. Oja. Exploratory analysis of climate data using source separation methods. Neural Networks, Vol. 19, No. 2, pages 155-167, March 2006. [article7.pdf] © 2006 Elsevier Science. By permission.
  • A. Ilin. Independent dynamics subspace analysis. In Proceedings of the 14th European Symposium on Artificial Neural Networks (ESANN 2006), pages 345-350, Bruges, Belgium, April 2006. [article8.pdf] © 2006 d-side publications. By permission.
  • A. Ilin, H. Valpola, and E. Oja. Extraction of components with structured variance. In Proceedings of the IEEE World Congress on Computational Intelligence (WCCI 2006), pages 10528-10535, Vancouver, BC, Canada, July 2006. [article9.pdf] © 2006 IEEE. By permission.
Permanent link to this item