Neural Variational Sparse Topic Model for Sparse Explainable Text Representation
Loading...
Access rights
openAccess
URL
Journal Title
Journal ISSN
Volume Title
A1 Alkuperäisartikkeli tieteellisessä aikakauslehdessä
This publication is imported from Aalto University research portal.
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Other link related to publication (opens in new window)
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Other link related to publication (opens in new window)
Date
2021-09
Department
Major/Subject
Mcode
Degree programme
Language
en
Pages
15
1-15
1-15
Series
Information Processing and Management, Volume 58, issue 5
Abstract
Texts are the major information carrier for internet users, from which learning the latent representations has important research and practical value. Neural topic models have been proposed and have great performance in extracting interpretable latent topics and representations of texts. However, there remain two major limitations: 1) these methods generally ignore the contextual information of texts and have limited feature representation ability due to the shallow feed-forward network architecture, 2) Sparsity of the representations in topic semantic space is ignored. To address these issues, in this paper, we propose a semantic reinforcement neural variational sparse topic model (SR-NSTM) towards explainable and sparse latent text representation learning. Compared with existing neural topic models, SR-NSTM models the generative process of texts with probabilistic distributions parameterized with neural networks and incorporates Bi-directional LSTM to embed contextual information at the document level. It achieves sparse posterior representations over documents and words with zero-mean Laplace distribution and topics with sparsemax. Moreover, we propose a supervised extension of SR-NSTM via adding the max-margin posterior regularization to tackle the supervised tasks. The neural variational inference method is utilized to learn our models efficiently. Experimental results on Web Snippets, 20Newsgroups, BBC, and Biomedical datasets demonstrate that the contextual information and revisiting generative process can improve the performance, leading to the competitive performance of our models in learning coherent topics and explainable sparse representations for texts.Description
| openaire: EC/H2020/101016775/EU//INTERVENE
Keywords
Neural Variational Inference, Neural Sparse Topic Model, Explainable Text Representation
Other note
Citation
Xie, Q, Tiwari, P, Gupta, D, Huang, J & Peng, M 2021, ' Neural Variational Sparse Topic Model for Sparse Explainable Text Representation ', Information Processing and Management, vol. 58, no. 5, 102614, pp. 1-15 . https://doi.org/10.1016/j.ipm.2021.102614