Using Large Language Models to Enhance Programming Error Messages

Simple item page
dc.contributorAalto-yliopistofi
dc.contributorAalto Universityen
dc.contributor.authorLeinonen, Juhoen_US
dc.contributor.authorHellas, Artoen_US
dc.contributor.authorSarsa, Samien_US
dc.contributor.authorReeves, Brenten_US
dc.contributor.authorDenny, Paulen_US
dc.contributor.authorPrather, Jamesen_US
dc.contributor.authorBecker, Brett A.en_US
dc.contributor.departmentDepartment of Computer Scienceen
dc.contributor.groupauthorComputer Science Lecturersen
dc.contributor.groupauthorComputer Science - Computing education research and educational technology (CER)en
dc.contributor.groupauthorLecturer Hellas Arto groupen
dc.contributor.organizationLecturer Hellas Arto groupen_US
dc.contributor.organizationAbilene Christian Universityen_US
dc.contributor.organizationUniversity of Aucklanden_US
dc.contributor.organizationUniversity College Dublinen_US
dc.date.accessioned2023-08-11T07:20:26Z
dc.date.available2023-08-11T07:20:26Z
dc.date.issued2023-03-02en_US
dc.description.abstractA key part of learning to program is learning to understand programming error messages. They can be hard to interpret and identifying the cause of errors can be time-consuming. One factor in this challenge is that the messages are typically intended for an audience that already knows how to program, or even for programming environments that then use the information to highlight areas in code. Researchers have been working on making these errors more novice friendly since the 1960s, however progress has been slow. The present work contributes to this stream of research by using large language models to enhance programming error messages with explanations of the errors and suggestions on how to fix them. Large language models can be used to create useful and novice-friendly enhancements to programming error messages that sometimes surpass the original programming error messages in interpretability and actionability. These results provide further evidence of the benefits of large language models for computing educators, highlighting their use in areas known to be challenging for students. We further discuss the benefits and downsides of large language models and highlight future streams of research for enhancing programming error messages.en
dc.description.versionPeer revieweden
dc.format.extent7
dc.format.extent563–569
dc.format.mimetypeapplication/pdfen_US
dc.identifier.citationLeinonen, J, Hellas, A, Sarsa, S, Reeves, B, Denny, P, Prather, J & Becker, B A 2023, Using Large Language Models to Enhance Programming Error Messages . in SIGCSE 2023 - Proceedings of the 54th ACM Technical Symposium on Computer Science Education . ACM, pp. 563–569, ACM Technical Symposium on Computer Science Education, Toronto, Canada, 15/03/2023 . https://doi.org/10.1145/3545945.3569770en
dc.identifier.doi10.1145/3545945.3569770en_US
dc.identifier.isbn978-1-4503-9431-4
dc.identifier.otherPURE UUID: 0b7e4a04-61b7-482b-97c4-918427fba5faen_US
dc.identifier.otherPURE ITEMURL: https://research.aalto.fi/en/publications/0b7e4a04-61b7-482b-97c4-918427fba5faen_US
dc.identifier.otherPURE LINK: http://www.scopus.com/inward/record.url?scp=85148727941&partnerID=8YFLogxKen_US
dc.identifier.otherPURE FILEURL: https://research.aalto.fi/files/117592299/SCI_Leinonen_etal_Using_Large_Language_Models_to_Enhance_Programming_Error_Messages_2023.pdfen_US
dc.identifier.urihttps://aaltodoc.aalto.fi/handle/123456789/122324
dc.identifier.urnURN:NBN:fi:aalto-202308114673
dc.language.isoenen
dc.relation.ispartofACM Technical Symposium on Computer Science Educationen
dc.relation.ispartofseriesSIGCSE 2023 - Proceedings of the 54th ACM Technical Symposium on Computer Science Educationen
dc.rightsopenAccessen
dc.titleUsing Large Language Models to Enhance Programming Error Messagesen
dc.typeA4 Artikkeli konferenssijulkaisussafi
dc.type.versionpublishedVersion
Files
Collections
[article-cris] Perustieteiden korkeakoulu / SCI