Bayes-Newton Methods for Approximate Bayesian Inference with PSD Guarantees

Simple item page
dc.contributorAalto-yliopistofi
dc.contributorAalto Universityen
dc.contributor.authorWilkinson, Williamen_US
dc.contributor.authorSärkkä, Simoen_US
dc.contributor.authorSolin, Arnoen_US
dc.contributor.departmentDepartment of Computer Scienceen
dc.contributor.departmentDepartment of Electrical Engineering and Automationen
dc.contributor.groupauthorHelsinki Institute for Information Technology (HIIT)en
dc.contributor.groupauthorSensor Informatics and Medical Technologyen
dc.contributor.groupauthorComputer Science Professorsen
dc.contributor.groupauthorComputer Science - Artificial Intelligence and Machine Learning (AIML)en
dc.contributor.groupauthorProfessorship Solin A.en
dc.date.accessioned2023-06-05T04:41:55Z
dc.date.available2023-06-05T04:41:55Z
dc.date.issued2023-03en_US
dc.description.abstractWe formulate natural gradient variational inference (VI), expectation propagation (EP), and posterior linearisation (PL) as extensions of Newton's method for optimising the parameters of a Bayesian posterior distribution. This viewpoint explicitly casts inference algorithms under the framework of numerical optimisation. We show that common approximations to Newton's method from the optimisation literature, namely Gauss-Newton and quasi-Newton methods (e.g., the BFGS algorithm), are still valid under this 'Bayes-Newton' framework. This leads to a suite of novel algorithms which are guaranteed to result in positive semi-definite (PSD) covariance matrices, unlike standard VI and EP. Our unifying viewpoint provides new insights into the connections between various inference schemes. All the presented methods apply to any model with a Gaussian prior and non-conjugate likelihood, which we demonstrate with (sparse) Gaussian processes and state space models.en
dc.description.versionPeer revieweden
dc.format.extent1−50
dc.format.mimetypeapplication/pdfen_US
dc.identifier.citationWilkinson, W, Särkkä, S & Solin, A 2023, ' Bayes-Newton Methods for Approximate Bayesian Inference with PSD Guarantees ', Journal of Machine Learning Research, vol. 24, pp. 1−50 . < https://www.jmlr.org/papers/v24/21-1298.html >en
dc.identifier.issn1532-4435
dc.identifier.issn1533-7928
dc.identifier.otherPURE UUID: d051a2cd-8994-4026-97bc-5a00050254a6en_US
dc.identifier.otherPURE ITEMURL: https://research.aalto.fi/en/publications/d051a2cd-8994-4026-97bc-5a00050254a6en_US
dc.identifier.otherPURE LINK: https://www.jmlr.org/papers/v24/21-1298.htmlen_US
dc.identifier.otherPURE FILEURL: https://research.aalto.fi/files/112116216/SCI_Wilkinson_etal_JMLR_2023.pdfen_US
dc.identifier.urihttps://aaltodoc.aalto.fi/handle/123456789/121226
dc.identifier.urnURN:NBN:fi:aalto-202306053607
dc.language.isoenen
dc.publisherMICROTOME PUBL
dc.relation.ispartofseriesJournal of Machine Learning Researchen
dc.relation.ispartofseriesVolume 24en
dc.rightsopenAccessen
dc.titleBayes-Newton Methods for Approximate Bayesian Inference with PSD Guaranteesen
dc.typeA1 Alkuperäisartikkeli tieteellisessä aikakauslehdessäfi
dc.type.versionpublishedVersion
Files
Collections
[article-cris] Perustieteiden korkeakoulu / SCI