Piloting Natural Language Generation for Personalized Progress Feedback

Abstract

Full research paper—We describe the results of a pilot study wherein we applied simple natural language generation methods to produce automated feedback for students of an online course based on student high-level progress data. Experimenting with both personalized and non-personalized feedback, we show that such feedback can be easily produced given access to even rudimentary data regarding student assignment submissions and their correctness. Our results suggest that students perceive automatically generated feedback generally positively and believe it to be useful. Our results also indicate that minor personalization and stylistic alterations in the feedback can have meaningful effects on how the feedback is interacted with and perceived. In particular, we observe that personalized feedback is perceived as being slightly easier to understand and as being better aligned with their progress. Students also felt better about the personalized feedback in comparison to non-personalized feedback. We conclude that the automated generation of personalized textual feedback shows promise as a low-threshold way of increasing student satisfaction. Further research is needed to assess the effect of different types of automated personalized feedback on student performance and behavior.