Advances in Nonnegative Matrix Decomposition with Application to Cluster Analysis

Loading...
Thumbnail Image
Journal Title
Journal ISSN
Volume Title
School of Science | Doctoral thesis (article-based) | Defence date: 2014-09-19
Checking the digitized thesis and permission for publishing
Instructions for the author
Date
2014
Major/Subject
Mcode
Degree programme
Language
en
Pages
94 + app. 94
Series
Aalto University publication series DOCTORAL DISSERTATIONS, 127/2014
Abstract
Nonnegative Matrix Factorization (NMF) has found a wide variety of applications in machine learning and data mining. NMF seeks to approximate a nonnegative data matrix by a product of several low-rank factorizing matrices, some of which are constrained to be nonnegative. Such additive nature often results in parts-based representation of the data, which is a desired property especially for cluster analysis.  This thesis presents advances in NMF with application in cluster analysis. It reviews a class of higher-order NMF methods called Quadratic Nonnegative Matrix Factorization (QNMF). QNMF differs from most existing NMF methods in that some of its factorizing matrices occur twice in the approximation. The thesis also reviews a structural matrix decomposition method based on Data-Cluster-Data (DCD) random walk. DCD goes beyond matrix factorization and has a solid probabilistic interpretation by forming the approximation with cluster assigning probabilities only. Besides, the Kullback-Leibler divergence adopted by DCD is advantageous in handling sparse similarities for cluster analysis.  Multiplicative update algorithms have been commonly used for optimizing NMF objectives, since they naturally maintain the nonnegativity constraint of the factorizing matrix and require no user-specified parameters. In this work, an adaptive multiplicative update algorithm is proposed to increase the convergence speed of QNMF objectives.  Initialization conditions play a key role in cluster analysis. In this thesis, a comprehensive initialization strategy is proposed to improve the clustering performance by combining a set of base clustering methods. The proposed method can better accommodate clustering methods that need a careful initialization such as the DCD.  The proposed methods have been tested on various real-world datasets, such as text documents, face images, protein, etc. In particular, the proposed approach has been applied to the cluster analysis of emotional data.
Description
Supervising professor
Oja, Erkki, Aalto Distinguished Prof., Aalto University, Department of Information and Computer Science, Finland
Thesis advisor
Yang, Zhirong, Dr., Aalto University, Department of Information and Computer Science, Finland
Keywords
nonnegative matrix factorization, cluster analysis, multiplicative update rule, constrained optimization, initialization condition, image classification and retrieval, affective computing, image emotion
Other note
Parts
  • [Publication 1]: Zhirong Yang, He Zhang, Zhijian Yuan, and Erkki Oja. Kullback-Leibler divergence for nonnegative matrix factorization. In Proceedings of 21st International Conference on Artificial Neural Networks (ICANN), pages 250–257, Espoo, Finland, June 2011.
    DOI: 10.1007/978-3-642-21735-7_31 View at publisher
  • [Publication 2]: He Zhang, Tele Hao, Zhirong Yang, and Erkki Oja. Pairwise clustering with t-PLSI. In Proceedings of 22nd International Conference on Artificial Neural Networks (ICANN), pages 411–418, Lausanne, Switzerland, September 2012.
    DOI: 10.1007/978-3-642-33266-1_51 View at publisher
  • [Publication 3]: Zhirong Yang, He Zhang, and Erkki Oja. Online Projective Nonnegative Matrix Factorization for large datasets. In Proceedings of 19th International Conference on Neural Information Processing (ICONIP), pages 285–290, Doha, Qatar, November 2012.
    DOI: 10.1007/978-3-642-34487-9_35 View at publisher
  • [Publication 4]: He Zhang, Zhirong Yang, and Erkki Oja. Adaptive multiplicative updates for projective nonnegative matrix factorization. In Proceedings of 19th International Conference on Neural Information Processing (ICONIP), pages 277–284, Doha, Qatar, November 2012.
    DOI: 10.1007/978-3-642-34487-9_34 View at publisher
  • [Publication 5]: He Zhang, Zhirong Yang, and Erkki Oja. Adaptive Multiplicative Updates for quadratic nonnegative matrix factorization. Neurocomputing, 134: 206–213, 2014.
    DOI: 10.1016/j.neucom.2013.06.047 View at publisher
  • [Publication 6]: He Zhang, Zhirong Yang, and Erkki Oja. Improving cluster analysis by co-initializations. Pattern Recognition Letters, 45: 71–77, 2014. DOI:
    DOI: 10.1016/j.patrec.2014.03.001 View at publisher
  • [Publication 7]: He Zhang, Teemu Ruokolainen, Jorma Laaksonen, Christina Hochleitner, and Rudolf Traunmüller. Gaze- and speech-enhanced content-based image retrieval in image tagging. In Proceedings of 21st International Conference on Artificial Neural Networks (ICANN), pages 373–380, Espoo, Finland, June 2011.
    DOI: 10.1007/978-3-642-21738-8_48 View at publisher
  • [Publication 8]: He Zhang, Eimontas Augilius, Timo Honkela, Jorma Laaksonen, Hannes Gamper, and Henok Alene. Analyzing emotional semantics of abstract art using low-level image features. In Proceedings of 10th International Symposium on Intelligent Data Analysis (IDA), pages 413–423, Porto, Portugal, October 2011.
    DOI: 10.1007/978-3-642-24800-9_38 View at publisher
  • [Publication 9]: He Zhang, Zhirong Yang, Mehmet Gönen, Markus Koskela, Jorma Laaksonen, Timo Honkela, and Erkki Oja. Affective abstract image classification and retrieval using multiple kernel learning. In Proceedings of 20th International Conference on Neural Information Processing (ICONIP), pages 166–175, Daegu, South Korea, November 2013.
    DOI: 10.1007/978-3-642-42051-1_22 View at publisher
Citation