Incorporating functional summary information in Bayesian neural networks using a Dirichlet process likelihood approach

dc.contributorAalto-yliopistofi
dc.contributorAalto Universityen
dc.contributor.authorRaj, Vishnuen_US
dc.contributor.authorCui, Tianyuen_US
dc.contributor.authorHeinonen, Markusen_US
dc.contributor.authorMarttinen, Pekkaen_US
dc.contributor.departmentDepartment of Computer Scienceen_US
dc.contributor.departmentProbabilistic Machine Learningen_US
dc.contributor.departmentComputer Science Professorsen_US
dc.contributor.editorRuiz, Franciscoen_US
dc.contributor.editorDy, Jenniferen_US
dc.contributor.editorvan de Meent, Jan-Willemen_US
dc.date.accessioned2023-06-05T04:41:08Z
dc.date.available2023-06-05T04:41:08Z
dc.date.issued2023en_US
dc.description| openaire: EC/H2020/101016775/EU//INTERVENE
dc.description.abstractBayesian neural networks (BNNs) can account for both aleatoric and epistemic uncertainty. However, in BNNs the priors are often specified over the weights which rarely reflects true prior knowledge in large and complex neural network architectures. We present a simple approach to incorporate prior knowledge in BNNs based on external summary information about the predicted classification probabilities for a given dataset. The available summary information is incorporated as augmented data and modeled with a Dirichlet process, and we derive the corresponding Summary Evidence Lower BOund. The approach is founded on Bayesian principles, and all hyperparameters have a proper probabilistic interpretation. We show how the method can inform the model about task difficulty and class imbalance. Extensive experiments show that, with negligible computational overhead, our method parallels and in many cases outperforms popular alternatives in accuracy, uncertainty calibration, and robustness against corruptions with both balanced and imbalanced data.en
dc.description.versionPeer revieweden
dc.format.extent6741-6763
dc.format.mimetypeapplication/pdfen_US
dc.identifier.citationRaj , V , Cui , T , Heinonen , M & Marttinen , P 2023 , Incorporating functional summary information in Bayesian neural networks using a Dirichlet process likelihood approach . in F Ruiz , J Dy & J-W van de Meent (eds) , Proceedings of The 26th International Conference on Artificial Intelligence and Statistics (AISTATS) 2023 . Proceedings of Machine Learning Research , vol. 206 , JMLR , pp. 6741-6763 , International Conference on Artificial Intelligence and Statistics , Valencia , Spain , 25/04/2023 . < https://proceedings.mlr.press/v206/raj23a.html >en
dc.identifier.issn2640-3498
dc.identifier.otherPURE UUID: 55fc0a20-4d20-4336-87e7-6fcfdba16ce2en_US
dc.identifier.otherPURE ITEMURL: https://research.aalto.fi/en/publications/55fc0a20-4d20-4336-87e7-6fcfdba16ce2en_US
dc.identifier.otherPURE LINK: https://proceedings.mlr.press/v206/raj23a.htmlen_US
dc.identifier.otherPURE FILEURL: https://research.aalto.fi/files/112113971/SCI_Raj_etal_AISTATS_2023.pdfen_US
dc.identifier.urihttps://aaltodoc.aalto.fi/handle/123456789/121212
dc.identifier.urnURN:NBN:fi:aalto-202306053594
dc.language.isoenen
dc.publisherJMLR
dc.relationinfo:eu-repo/grantAgreement/EC/H2020/101016775/EU//INTERVENEen_US
dc.relation.ispartofInternational Conference on Artificial Intelligence and Statisticsen
dc.relation.ispartofseriesProceedings of The 26th International Conference on Artificial Intelligence and Statistics (AISTATS) 2023en
dc.relation.ispartofseriesProceedings of Machine Learning Researchen
dc.relation.ispartofseriesVolume 206en
dc.rightsopenAccessen
dc.titleIncorporating functional summary information in Bayesian neural networks using a Dirichlet process likelihood approachen
dc.typeConference article in proceedingsfi
dc.type.versionpublishedVersion
Files