LLM-itation is the Sincerest Form of Data: Generating Synthetic Buggy Code Submissions for Computing Education
Loading...
Access rights
openAccess
CC BY
CC BY
publishedVersion
URL
Journal Title
Journal ISSN
Volume Title
A4 Artikkeli konferenssijulkaisussa
This publication is imported from Aalto University research portal.
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Unless otherwise stated, all rights belong to the author. You may download, display and print this publication for Your own personal use. Commercial use is prohibited.
Date
Department
Major/Subject
Mcode
Degree programme
Language
en
Pages
8
Series
ACE 2025 - Proceedings of the 27th Australasian Computing Education Conference, Held in conjunction with, pp. 56-63
Abstract
There is a great need for data in computing education research. Data is needed to understand how students behave, to train models of student behavior to optimally support students, and to develop and validate new assessment tools and learning analytics techniques. However, relatively few computing education datasets are shared openly, often due to privacy regulations and issues in making sure the data is anonymous. Large language models (LLMs) offer a promising approach to create large-scale, privacy-preserving synthetic data, which can be used to explore various aspects of student learning, develop and test educational technologies, and support research in areas where collecting real student data may be challenging or impractical. This work explores generating synthetic buggy code submissions for introductory programming exercises using GPT-4o. We compare the distribution of test case failures between synthetic and real student data from two courses to analyze the accuracy of the synthetic data in mimicking real student data. Our findings suggest that LLMs can be used to generate synthetic incorrect submissions that are not significantly different from real student data with regard to test case failure distributions. Our research contributes to the development of reliable synthetic datasets for computing education research and teaching, potentially accelerating progress in the field while preserving student privacy.Description
Publisher Copyright: © 2025 Copyright held by the owner/author(s).
Other note
Citation
Leinonen, J, Denny, P, Kiljunen, O, MacNeil, S, Sarsa, S & Hellas, A 2025, LLM-itation is the Sincerest Form of Data: Generating Synthetic Buggy Code Submissions for Computing Education. in C Seton & Simon (eds), ACE 2025 - Proceedings of the 27th Australasian Computing Education Conference, Held in conjunction with. ACM, pp. 56-63, Australasian Computing Education Conference, Brisbane, Australia, 12/02/2025. https://doi.org/10.1145/3716640.3716647