Speeding Up Automated Assessment of Programming Exercises
Loading...
Access rights
openAccess
URL
Journal Title
Journal ISSN
Volume Title
A4 Artikkeli konferenssijulkaisussa
This publication is imported from Aalto University research portal.
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Other link related to publication (opens in new window)
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Other link related to publication (opens in new window)
Date
2022
Department
Major/Subject
Mcode
Degree programme
Language
en
Pages
7
1-7
1-7
Series
Proceedings of the 2022 Conference on United Kingdom & Ireland Computing Education Research
Abstract
Introductory programming courses around the world use automatic assessment. Automatic assessment for programming code is typically performed via unit tests which require computation time to execute, at times in significant amounts, leading to computation costs and delay in feedback to students. We present a step-based approach for speeding up automated assessment to address the issue, consisting of (1) a cache of past programming exercise submissions and their associated test results to avoid retesting equivalent new submissions; (2) static analysis to detect e.g. infinite loops (heuristically) ; (3) a machine learning model to evaluate programs without running them ; and (4) a traditional set of unit tests. When a student submits code for an exercise, the code is evaluated sequentially through each step, providing feedback to the student at the earliest possible time, reducing the need to run tests. We evaluate the impact of the proposed approach using data collected from an introductory programming course and demonstrate a considerable reduction in the number of exercise submissions that require running the tests (up to 80. Using the approach leads to faster feedback in a more sustainable way, and also provides opportunities for precise non-exercise specific feedback in steps (2) and (3).Description
Keywords
machine learning, automatic assessment, automated assessment, feedback, sustainability, static analysis, source code, educational data mining
Other note
Citation
Sarsa, S, Leinonen, J, Koutcheme, C & Hellas, A 2022, Speeding Up Automated Assessment of Programming Exercises . in Proceedings of the 2022 Conference on United Kingdom & Ireland Computing Education Research ., 3, ACM, pp. 1-7, United Kingdom and Ireland Computing Education Research Conference, Dublin, Ireland, 01/09/2022 . https://doi.org/10.1145/3555009.3555013