Expectation Propagation as a Way of Life

 |  Login

Show simple item record

dc.contributor Aalto-yliopisto fi
dc.contributor Aalto University en
dc.contributor.author Vehtari, Aki
dc.contributor.author Gelman, Andrew
dc.contributor.author Sivula, Tuomas
dc.contributor.author Jylanki, Pasi
dc.contributor.author Tran, Dustin
dc.contributor.author Sahai, Swupnil
dc.contributor.author Blomstedt, Paul
dc.contributor.author Cunningham, John P.
dc.contributor.author Schiminovich, David
dc.contributor.author Robert, Christian P.
dc.date.accessioned 2020-03-13T15:25:09Z
dc.date.available 2020-03-13T15:25:09Z
dc.date.issued 2020
dc.identifier.citation Vehtari , A , Gelman , A , Sivula , T , Jylanki , P , Tran , D , Sahai , S , Blomstedt , P , Cunningham , J P , Schiminovich , D & Robert , C P 2020 , ' Expectation Propagation as a Way of Life : A Framework for Bayesian Inference on Partitioned Data ' , Journal of Machine Learning Research , vol. 21 , pp. 1-53 . en
dc.identifier.issn 1532-4435
dc.identifier.issn 1533-7928
dc.identifier.other PURE UUID: 9a6301b2-1ef9-450b-a379-43d90b283a61
dc.identifier.other PURE ITEMURL: https://research.aalto.fi/en/publications/expectation-propagation-as-a-way-of-life(9a6301b2-1ef9-450b-a379-43d90b283a61).html
dc.identifier.other PURE LINK: http://www.jmlr.org/papers/v21/18-817.html
dc.identifier.other PURE FILEURL: https://research.aalto.fi/files/41457095/Vehtari_Expectation_Propagation.18_817.pdf
dc.identifier.uri https://aaltodoc.aalto.fi/handle/123456789/43450
dc.description.abstract A common divide-and-conquer approach for Bayesian computation with big data is to partition the data, perform local inference for each piece separately, and combine the results to obtain a global posterior approximation. While being conceptually and computationally appealing, this method involves the problematic need to also split the prior for the local inferences; these weakened priors may not provide enough regularization for each separate computation, thus eliminating one of the key advantages of Bayesian methods. To resolve this dilemma while still retaining the generalizability of the underlying local inference method, we apply the idea of expectation propagation (EP) as a framework for distributed Bayesian inference. The central idea is to iteratively update approximations to the local likelihoods given the state of the other approximations and the prior. The present paper has two roles: we review the steps that are needed to keep EP algorithms numerically stable, and we suggest a general approach, inspired by EP, for approaching data partitioning problems in a way that achieves the computational benefits of parallelism while allowing each local update to make use of relevant information from the other sites. In addition, we demonstrate how the method can be applied in a hierarchical context to make use of partitioning of both data and parameters. The paper describes a general algorithmic framework, rather than a specific algorithm, and presents an example implementation for it. en
dc.format.extent 53
dc.format.extent 1-53
dc.format.mimetype application/pdf
dc.language.iso en en
dc.publisher MICROTOME PUBL
dc.relation.ispartofseries Journal of Machine Learning Research en
dc.relation.ispartofseries Volume 21 en
dc.rights openAccess en
dc.subject.other 113 Computer and information sciences en
dc.title Expectation Propagation as a Way of Life en
dc.type A1 Alkuperäisartikkeli tieteellisessä aikakauslehdessä fi
dc.description.version Peer reviewed en
dc.contributor.department Probabilistic Machine Learning
dc.contributor.department Columbia University
dc.contributor.department Centre of Excellence in Computational Inference, COIN
dc.contributor.department Department of Computer Science en
dc.subject.keyword Bayesian computation
dc.subject.keyword data partitioning
dc.subject.keyword expectation propagation
dc.subject.keyword hierarchical models
dc.subject.keyword statistical computing
dc.subject.keyword PRECISION MATRIX
dc.subject.keyword CLASSIFICATION
dc.subject.keyword LIKELIHOOD
dc.subject.keyword REGRESSION
dc.subject.keyword MODELS
dc.subject.keyword 113 Computer and information sciences
dc.identifier.urn URN:NBN:fi:aalto-202003132491
dc.type.version publishedVersion

Files in this item

Files Size Format View

There are no open access files associated with this item.

This item appears in the following Collection(s)

Show simple item record

Search archive

Advanced Search

article-iconSubmit a publication