Automatic Generation of Programming Exercises and Code Explanations Using Large Language Models
Loading...
Access rights
openAccess
URL
Journal Title
Journal ISSN
Volume Title
A4 Artikkeli konferenssijulkaisussa
This publication is imported from Aalto University research portal.
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Other link related to publication (opens in new window)
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Other link related to publication (opens in new window)
Date
2022-08-03
Department
Major/Subject
Mcode
Degree programme
Language
en
Pages
17
27-43
27-43
Series
ICER 2022 - Proceedings of the 2022 ACM Conference on International Computing Education Research
Abstract
This article explores the natural language generation capabilities of large language models with application to the production of two types of learning resources common in programming courses. Using OpenAI Codex as the large language model, we create programming exercises (including sample solutions and test cases) and code explanations, assessing these qualitatively and quantitatively. Our results suggest that the majority of the automatically generated content is both novel and sensible, and in some cases ready to use as is. When creating exercises we find that it is remarkably easy to influence both the programming concepts and the contextual themes they contain, simply by supplying keywords as input to the model. Our analysis suggests that there is significant value in massive generative machine learning models as a tool for instructors, although there remains a need for some oversight to ensure the quality of the generated content before it is delivered to students. We further discuss the implications of OpenAI Codex and similar tools for introductory programming education and highlight future research streams that have the potential to improve the quality of the educational experience for both teachers and students alike.Description
Keywords
Other note
Citation
Sarsa, S, Denny, P, Hellas, A & Leinonen, J 2022, Automatic Generation of Programming Exercises and Code Explanations Using Large Language Models . in ICER 2022 - Proceedings of the 2022 ACM Conference on International Computing Education Research . ACM, pp. 27-43, ACM Conference on International Computing Education Research, Lugano, Switzerland, 07/08/2022 . https://doi.org/10.1145/3501385.3543957