On the Opportunities of Large Language Models for Programming Process Data
Loading...
Access rights
openAccess
CC BY
CC BY
publishedVersion
URL
Journal Title
Journal ISSN
Volume Title
A4 Artikkeli konferenssijulkaisussa
This publication is imported from Aalto University research portal.
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Unless otherwise stated, all rights belong to the author. You may download, display and print this publication for Your own personal use. Commercial use is prohibited.
Authors
Date
Department
Major/Subject
Mcode
Degree programme
Language
en
Pages
9
Series
ACE 2025 - Proceedings of the 27th Australasian Computing Education Conference, Held in conjunction with, pp. 105-113
Abstract
Computing educators and researchers have long used programming process data to understand how students construct programs and address challenges. Despite its potential, fully automated feedback systems remain underexplored. The emergence of Large Language Models (LLMs) offers new opportunities for analyzing programming data and providing formative feedback. This study explores using LLMs to summarize programming processes and deliver formative feedback. A case study analyzed keystroke-level data from an introductory programming course, processed into code snapshots. Three state-of-the-art LLMs - Claude 3 Opus, GPT-4 Turbo, and LLaMa2 70B Chat - were evaluated for their feedback capabilities. Results show LLMs effectively provide tailored feedback, emphasizing incremental development, algorithmic planning, and code readability. Our findings highlight the potential of combining keystroke data with LLMs to automate formative feedback, showing that the computing education research and practice community is again one step closer to automating formative programming process feedback.Description
Publisher Copyright: © 2025 Copyright held by the owner/author(s).
Other note
Citation
Edwards, J, Hellas, A & Leinonen, J 2025, On the Opportunities of Large Language Models for Programming Process Data. in ACE 2025 - Proceedings of the 27th Australasian Computing Education Conference, Held in conjunction with. ACM, pp. 105-113, Australasian Computing Education Conference, Brisbane, Australia, 12/02/2025. https://doi.org/10.1145/3716640.3716652