aalto1 untyped-item.component.html
DxHF: Providing High-Quality Human Feedback for LLM Alignment via Interactive Decomposition
Loading...
Access rights
openAccess
CC BY
CC BY
Creative Commons license
Except where otherwised noted, this item's license is described as openAccess
publishedVersion
URL
Journal Title
Journal ISSN
Volume Title
A4 Artikkeli konferenssijulkaisussa
This publication is imported from Aalto University research portal.
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Unless otherwise stated, all rights belong to the author. You may download, display and print this publication for Your own personal use. Commercial use is prohibited.
Date
Major/Subject
Mcode
Degree programme
Language
en
Pages
14
Series
UIST 2025 - Proceedings of the 38th Annual ACM Symposium on User Interface Software and Technology
Abstract
Human preferences are widely used to align large language mod- els (LLMs) through methods such as reinforcement learning from human feedback (RLHF). However, the current user interfaces re- quire annotators to compare text paragraphs, which is cognitively challenging when the texts are long or unfamiliar. This paper con- tributes by studying the decomposition principle as an approach to improving the quality of human feedback for LLM alignment. This approach breaks down the text into individual claims instead of directly comparing two long-form text responses. Based on the principle, we build a novel user interface DxHF. It enhances the comparison process by showing decomposed claims, visually encod- ing the relevance of claims to the conversation and linking similar claims. This allows users to skim through key information and identify differences for better and quicker judgment. Our technical evaluation shows evidence that decomposition generally improves feedback accuracy regarding the ground truth, particularly for users with uncertainty. A crowdsourcing study with 160 participants indi- cates that using DxHF improves feedback accuracy by an average of 5%, although it increases the average feedback time by 18 seconds. Notably, accuracy is significantly higher in situations where users have less certainty. The finding of the study highlights the potential of HCI as an effective method for improving human-AI alignment.
Description
| openaire: EC/HE/101141916/EU//Artificial User
Other note
Citation
Shi, D, Cheng, F, Weinkauf, T, Oulasvirta, A & Mennatallah, E-A 2025, DxHF: Providing High-Quality Human Feedback for LLM Alignment via Interactive Decomposition. in A Bianchi, E L Glassman, W E Mackay, S Zhao, I Oakley & J Kim (eds), UIST 2025 - Proceedings of the 38th Annual ACM Symposium on User Interface Software and Technology., 123, ACM, ACM Symposium on User Interface Software and Technology, Busan, Korea, Republic of, 28/09/2025. https://doi.org/10.1145/3746059.3747600