Does the magic of BERT apply to medical code assignment? A quantitative study

Simple item page
dc.contributorAalto-yliopistofi
dc.contributorAalto Universityen
dc.contributor.authorJi, Shaoxiongen_US
dc.contributor.authorHölttä, Mattien_US
dc.contributor.authorMarttinen, Pekkaen_US
dc.contributor.departmentDepartment of Computer Scienceen
dc.contributor.groupauthorProfessorship Marttinen P.en
dc.contributor.groupauthorComputer Science Professorsen
dc.contributor.groupauthorComputer Science - Artificial Intelligence and Machine Learning (AIML)en
dc.contributor.organizationDepartment of Computer Scienceen_US
dc.contributor.organizationAalto Universityen_US
dc.date.accessioned2021-11-04T05:05:34Z
dc.date.available2021-11-04T05:05:34Z
dc.date.issued2021-12en_US
dc.description| openaire: EC/H2020/101016775/EU//INTERVENE
dc.description.abstractUnsupervised pretraining is an integral part of many natural language processing systems, and transfer learning with language models has achieved remarkable results in downstream tasks. In the clinical application of medical code assignment, diagnosis and procedure codes are inferred from lengthy clinical notes such as hospital discharge summaries. However, it is not clear if pretrained models are useful for medical code prediction without further architecture engineering. This paper conducts a comprehensive quantitative analysis of various contextualized language models' performances, pretrained in different domains, for medical code assignment from clinical notes. We propose a hierarchical fine-tuning architecture to capture interactions between distant words and adopt label-wise attention to exploit label information. Contrary to current trends, we demonstrate that a carefully trained classical CNN outperforms attention-based models on a MIMIC-III subset with frequent codes. Our empirical findings suggest directions for building robust medical code assignment models.en
dc.description.versionPeer revieweden
dc.format.mimetypeapplication/pdfen_US
dc.identifier.citationJi, S, Hölttä, M & Marttinen, P 2021, ' Does the magic of BERT apply to medical code assignment? A quantitative study ', Computers in Biology and Medicine, vol. 139, 104998 . https://doi.org/10.1016/j.compbiomed.2021.104998en
dc.identifier.doi10.1016/j.compbiomed.2021.104998en_US
dc.identifier.issn0010-4825
dc.identifier.otherPURE UUID: af2bb4b9-dbca-41fc-8981-c21e9e93dbfden_US
dc.identifier.otherPURE ITEMURL: https://research.aalto.fi/en/publications/af2bb4b9-dbca-41fc-8981-c21e9e93dbfden_US
dc.identifier.otherPURE LINK: http://www.scopus.com/inward/record.url?scp=85118487160&partnerID=8YFLogxKen_US
dc.identifier.otherPURE FILEURL: https://research.aalto.fi/files/76096743/Does_the_magic_of_BERT_apply_to_medical_code_assignment_A_quantitative_study.pdfen_US
dc.identifier.urihttps://aaltodoc.aalto.fi/handle/123456789/110832
dc.identifier.urnURN:NBN:fi:aalto-2021110410005
dc.language.isoenen
dc.publisherElsevier Limited
dc.relationinfo:eu-repo/grantAgreement/EC/H2020/101016775/EU//INTERVENEen_US
dc.relation.ispartofseriesCOMPUTERS IN BIOLOGY AND MEDICINEen
dc.rightsopenAccessen
dc.titleDoes the magic of BERT apply to medical code assignment? A quantitative studyen
dc.typeA1 Alkuperäisartikkeli tieteellisessä aikakauslehdessäfi
Files
Collections
[article-cris] Perustieteiden korkeakoulu / SCI