aalto1 untyped-item.component.html
Measuring digital intervention user experience with a novel ecological momentary assessment (EMA) method, CORTO
Loading...
Access rights
openAccess
publishedVersion
URL
Journal Title
Journal ISSN
Volume Title
A1 Alkuperäisartikkeli tieteellisessä aikakauslehdessä
This publication is imported from Aalto University research portal.
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Unless otherwise stated, all rights belong to the author. You may download, display and print this publication for Your own personal use. Commercial use is prohibited.
Date
Major/Subject
Mcode
Degree programme
Language
en
Pages
17
Series
Internet Interventions, Volume 35, pp. 1-17
Abstract
Digital interventions often suffer from low usage, which may reflect insufficient attention to user experience. Moreover, the existing evaluation methods have limited applicability in the remote study of user experience of complex interventions that have expansive content and that are used over an extensive period of time. To alleviate these challenges, we describe here a novel qualitative Ecological Momentary Assessment (EMA) method: the CORTO method (Contextual, One-item, Repeated, Timely, Open-ended). We used it to gather digital intervention user experience data from Finnish adults (n = 184) who lived with interview-confirmed major depressive disorder (MDD) and took part in a randomized controlled trial (RCT) that studied the efficacy of a novel 12-week game-based digital intervention for depression. A second dataset on user experience was gathered with retrospective interviews (n = 22). We inductively coded the CORTO method and retrospective interview data, which led to four user experience categories: (1) contextual use, (2) interaction-elicited emotional experience, (3) usability, and (4) technical issues. Then, we used the created user experience categories and Template Analysis to analyze both datasets together, and reported the results qualitatively. Finally, we compared the two datasets with each other. We found that the data generated with the CORTO method offered more insights into usability and technical categories than the interview data that particularly illustrated the contextual use. The emotional valence of the interview data was more positive compared with the CORTO data. Both the CORTO and interview data detected 55 % of the micro-level categories; 20 % of micro-level categories were only detected by the CORTO data and 25 % only by the interview data. We found that the during-intervention user experience measurement with the CORTO method can provide intervention-specific insights, and thereby further the iterative user-centered intervention development. Overall, these findings highlight the impact of evaluation methods on the categories and qualities of insights acquired in intervention research.
Description
Funding Information: LL and VRB are funded by grants from Jane and Aatos Erkko Foundation and Technology Industries of Finland Centennial, and Business Finland ( 42173/31/2020 ) awarded to JMP. VMK is funded by the Academy of Finland grant ( 353267 ) and the European Research Council under the European Union 's Horizon Europe research and innovation programme grant ( 101042052 ). Publisher Copyright: © 2024 The Authors
Other note
Citation
Lukka, L, Karhulahti, V M, Bergman, V R & Palva, J M 2024, 'Measuring digital intervention user experience with a novel ecological momentary assessment (EMA) method, CORTO', Internet Interventions, vol. 35, 100706, pp. 1-17. https://doi.org/10.1016/j.invent.2023.100706