Distributed Bayesian matrix factorization with limited communication

Thumbnail Image

Access rights

openAccess
publishedVersion

URL

Journal Title

Journal ISSN

Volume Title

A1 Alkuperäisartikkeli tieteellisessä aikakauslehdessä
This publication is imported from Aalto University research portal.
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Other link related to publication (opens in new window)

Authors

Date

2019-01-01

Department

Probabilistic Machine Learning
Centre of Excellence in Computational Inference, COIN
Department of Computer Science

Major/Subject

Mcode

Degree programme

Language

en

Pages

1-26

Series

Machine Learning

Abstract

Bayesian matrix factorization (BMF) is a powerful tool for producing low-rank representations of matrices and for predicting missing values and providing confidence intervals. Scaling up the posterior inference for massive-scale matrices is challenging and requires distributing both data and computation over many workers, making communication the main computational bottleneck. Embarrassingly parallel inference would remove the communication needed, by using completely independent computations on different data subsets, but it suffers from the inherent unidentifiability of BMF solutions. We introduce a hierarchical decomposition of the joint posterior distribution, which couples the subset inferences, allowing for embarrassingly parallel computations in a sequence of at most three stages. Using an efficient approximate implementation, we show improvements empirically on both real and simulated data. Our distributed approach is able to achieve a speed-up of almost an order of magnitude over the full posterior, with a negligible effect on predictive accuracy. Our method outperforms state-of-the-art embarrassingly parallel MCMC methods in accuracy, and achieves results competitive to other available distributed and parallel implementations of BMF.

Description

| openaire: EC/H2020/671555/EU//ExCAPE

Keywords

Bayesian matrix factorization, Distributed inference, Embarrassingly parallel MCMC, Posterior propagation, Software, Artificial Intelligence, 113 Computer and information sciences

Other note

DOI

10.1007/s10994-019-05778-2

Citation

Qin , X , Blomstedt , P , Leppäaho , E , Parviainen , P & Kaski , S 2019 , ' Distributed Bayesian matrix factorization with limited communication ' , Machine Learning , pp. 1-26 . https://doi.org/10.1007/s10994-019-05778-2

Permanent link to this item

https://urn.fi/URN:NBN:fi:aalto-201905062840

Collections

[article-cris] Perustieteiden korkeakoulu / SCI