How Suitable Is Your Naturalistic Dataset for Theory-based User Modeling?

Loading...
Thumbnail Image
Journal Title
Journal ISSN
Volume Title
Conference article in proceedings
Date
2022-04-07
Major/Subject
Mcode
Degree programme
Language
en
Pages
12
179-190
Series
UMAP2022 - Proceedings of the 30th ACM Conference on User Modeling, Adaptation and Personalization
Abstract
Theory-based, or "white-box,"models come with a major benefit that makes them appealing for deployment in user modeling: their parameters are interpretable. However, most theory-based models have been developed in controlled settings, in which researchers determine the experimental design. In contrast, real-world application of these models demands setups that are beyond developer control. In non-experimental, naturalistic settings, the tasks with which users are presented may be very limited, and it is not clear that model parameters can be reliably inferred. This paper describes a technique for assessing whether a naturalistic dataset is suitable for use with a theory-based model. The proposed parameter recovery technique can warn against possible over-confidence in inferred model parameters. This technique also can be used to study conditions under which parameter inference is feasible. The method is demonstrated for two models of decision-making under risk with naturalistic data from a turn-based game.
Description
Funding Information: This work was supported by Business Finland (MINERAL project), the Finnish Center for Artificial Intelligence (FCAI) and Academy of Finland projects Human Automata (Project ID: 328813) and BAD (Project ID: 318559). We would also like to thank Next Games for their assistance. Publisher Copyright: © 2022 Owner/Author.
Keywords
naturalistic data, parameter recovery, risky choice, theory-based models, user modeling
Other note
Citation
Putkonen , A , Nioche , A , Tanskanen , V , Klami , A & Oulasvirta , A 2022 , How Suitable Is Your Naturalistic Dataset for Theory-based User Modeling? in UMAP2022 - Proceedings of the 30th ACM Conference on User Modeling, Adaptation and Personalization . UMAP2022 - Proceedings of the 30th ACM Conference on User Modeling, Adaptation and Personalization , ACM , pp. 179-190 , Conference on User Modeling, Adaptation and Personalization , Virtual, Online , Spain , 04/07/2022 . https://doi.org/10.1145/3503252.3531322