From Prompts to Propositions : A Logic-Based Lens on Student-LLM Interactions
Loading...
Access rights
openAccess
CC BY
CC BY
publishedVersion
URL
Journal Title
Journal ISSN
Volume Title
A4 Artikkeli konferenssijulkaisussa
This publication is imported from Aalto University research portal.
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Unless otherwise stated, all rights belong to the author. You may download, display and print this publication for Your own personal use. Commercial use is prohibited.
Date
Department
Major/Subject
Mcode
Degree programme
Language
en
Pages
11
Series
Proceedings of 25th International Conference on Computing Education Research, Koli Calling 2025, pp. 1-11
Abstract
Background and Context. The increasing integration of large language models (LLMs) in computing education presents an emerging challenge in understanding how students use LLMs and craft prompts to solve computational tasks. Prior research has used both qualitative and quantitative methods to analyze prompting behavior, but these approaches lack scalability or fail to effectively capture the semantic evolution of prompts.Objective. In this paper, we investigate whether students' prompts can be systematically analyzed using propositional logic constraints. We examine whether this approach can identify patterns in prompt evolution, detect struggling students, and provide insights into effective and ineffective strategies.Method. We introduce Prompt2Proposition, a novel framework that translates students' prompts into propositional logic constraints. The constraints are able to represent the intent of the prompts in succinct and quantifiable ways. We used this approach to analyze a dataset of 1,872 prompts from 203 students solving programming tasks.Findings. We find that while successful and unsuccessful attempts tend to use a similar number of constraints overall, when students fail, they often modify their prompts more significantly, shifting problem-solving strategies midway. We also identify points where specific interventions could be most helpful to students for refining their prompts.Implications. This work offers a new and scalable way to detect students who struggle in solving natural language programming tasks. This work could be extended to investigate more complex tasks and integrated into programming tools to provide real-time support.Description
Publisher Copyright: © 2025 Copyright held by the owner/author(s).
Other note
Citation
Alfageeh, A, Zarkouei, S A K, Nam, D, Prol, D, Amoozadeh, M, Chattopadhyay, S, Prather, J, Denny, P, Leinonen, J, Hilton, M, Ragavan, S S & Alipour, A 2025, From Prompts to Propositions : A Logic-Based Lens on Student-LLM Interactions. in J Leinonen & R Duran (eds), Proceedings of 25th International Conference on Computing Education Research, Koli Calling 2025., 16, ACM, pp. 1-11, Koli Calling - International Conference on Computing Education Research, Lieksa, Finland, 11/11/2025. https://doi.org/10.1145/3769994.3770004