Modeling of mutual dependencies
Loading...
Journal Title
Journal ISSN
Volume Title
Doctoral thesis (article-based)
Checking the digitized thesis and permission for publishing
Instructions for the author
Instructions for the author
Unless otherwise stated, all rights belong to the author. You may download, display and print this publication for Your own personal use. Commercial use is prohibited.
Author
Date
2008
Department
Major/Subject
Mcode
Degree programme
Language
en
Pages
Verkkokirja (839 KB, 69 s.)
Series
Abstract
Data analysis means applying computational models to analyzing large collections of data, such as video signals, text collections, or measurements of gene activities in human cells. Unsupervised or exploratory data analysis refers to a subtask of data analysis, in which the goal is to find novel knowledge based on only the data. A central challenge in unsupervised data analysis is separating relevant and irrelevant information from each other. In this thesis, novel solutions to focusing on more relevant findings are presented. Measurement noise is one source of irrelevant information. If we have several measurements of the same objects, the noise can be suppressed by averaging over the measurements. Simple averaging is, however, only possible when the measurements share a common representation. In this thesis, we show how irrelevant information can be suppressed or ignored also in cases where the measurements come from different kinds of sensors or sources, such as video and audio recordings of the same scene. For combining the measurements, we use mutual dependencies between them. Measures of dependency, such as mutual information, characterize commonalities between two sets of measurements. Two measurements can hence be combined to reduce irrelevant variation by finding new representations for the objects so that the representations are maximally dependent. The combination is optimal, given the assumption that what is in common between the measurements is more relevant than information specific to any one of the sources. Several practical models for the task are introduced. In particular, novel Bayesian generative models, including a Bayesian version of the classical method of canonical correlation analysis, are given. Bayesian modeling is especially justified approach to learning from small data sets. Hence, generative models can be used to extract dependencies in a more reliable manner in, for example, medical applications, where obtaining a large number of samples is difficult. Also, novel non-Bayesian models are presented: Dependent component analysis finds linear projections which capture more general dependencies than earlier methods. Mutual dependencies can also be used for supervising traditional unsupervised learning methods. The learning metrics principle describes how a new distance metric focusing on relevant information can be derived based on the dependency between the measurements and a supervising signal. In this thesis, the approximations and optimization methods required for using the learning metrics principle are improved.Description
Keywords
canonical correlation analysis, clustering, data fusion, exploratory data analysis, probabilistic modeling, learning metrics, mutual dependency, mutual information
Other note
Parts
- [Publication 1]: Arto Klami and Samuel Kaski. 2005. Non-parametric dependent components. In: Proceedings of the 30th IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2005). Philadelphia, PA, USA. 18-23 March 2005. Piscataway, NJ, IEEE, pages V-209 - V-212. © 2005 IEEE. By permission.
- [Publication 2]: Abhishek Tripathi, Arto Klami, and Samuel Kaski. 2008. Simple integrative preprocessing preserves what is shared in data sources. BMC Bioinformatics, volume 9, 111. © 2008 by authors.
- [Publication 3]: Arto Klami and Samuel Kaski. 2006. Generative models that discover dependencies between data sets. In: S. McLoone, T. Adali, J. Larsen, and M. Van Hulle (editors). Machine Learning for Signal Processing XVI. Piscataway, NJ, IEEE, pages 123-128. © 2006 IEEE. By permission.
- [Publication 4]: Arto Klami and Samuel Kaski. 2008. Probabilistic approach to detecting dependencies between data sets. Neurocomputing, to appear. © 2008 by authors and © 2008 Elsevier Science. By permission.
- [Publication 5]: Arto Klami and Samuel Kaski. 2007. Local dependent components. In: Zoubin Ghahramani (editor). Proceedings of the 24th International Conference on Machine Learning (ICML 2007). Corvallis, OR, USA. 20-24 June 2007. Madison, WI, Omnipress, pages 425-433. © 2007 by authors.
- [Publication 6]: Jaakko Peltonen, Arto Klami, and Samuel Kaski. 2004. Improved learning of Riemannian metrics for exploratory analysis. Neural Networks, volume 17, numbers 8-9, pages 1087-1100. © 2004 Elsevier Science. By permission.
- [Publication 7]: Samuel Kaski, Janne Sinkkonen, and Arto Klami. 2005. Discriminative clustering. Neurocomputing, volume 69, numbers 1-3, pages 18-41. © 2005 Elsevier Science. By permission.
- [Errata file]: Errata of publication 6
