aalto1 untyped-item.component.html
Asynchronous stochastic Quasi-Newton MCMC for non-convex optimization supplementary document
Loading...
Access rights
openAccess
publishedVersion
URL
Journal Title
Journal ISSN
Volume Title
A4 Artikkeli konferenssijulkaisussa
This publication is imported from Aalto University research portal.
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Other link related to publication (opens in new window)
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Other link related to publication (opens in new window)
Unless otherwise stated, all rights belong to the author. You may download, display and print this publication for Your own personal use. Commercial use is prohibited.
Date
Department
Major/Subject
Mcode
Degree programme
Language
en
Pages
8
Series
35th International Conference on Machine Learning, ICML 2018, Volume 11, pp. 4674-4683, Proceedings of Machine Learning Research ; Volume 80
Abstract
Recent studies have illustrated that stochastic gradient Markov Chain Monte Carlo techniques have a strong potential in non-convex optimization, where local and global convergence guarantees can be shown under certain conditions. By building up on this recent theory, in this study, we develop an asynchronous-parallel stochastic L-BFGS algorithm for non-convex optimization. The proposed algorithm is suitable for both distributed and shared-memory settings. We provide formal theoretical analysis and show that the proposed method achieves an ergodic convergence rate of {equation Presented} (N being the total number of iterations) and it can achieve a linear speedup under certain conditions. We perform several experiments on both synthetic and real datasets. The results support our theory and show that the proposed algorithm provides a significant speedup over the recently proposed synchronous distributed L-BFGS algorithm.
Description
Keywords
Other note
Citation
Simsekli, U, Yildiz, C, Nguyen, T H, Richard, G & Cemgil, A T 2018, Asynchronous stochastic Quasi-Newton MCMC for non-convex optimization supplementary document. in A Krause & J Dy (eds), 35th International Conference on Machine Learning, ICML 2018. vol. 11, Proceedings of Machine Learning Research, vol. 80, International Machine Learning Society, pp. 4674-4683, International Conference on Machine Learning, Stockholm, Sweden, 10/07/2018. < http://proceedings.mlr.press/v80/simsekli18a.html >