Designing experiment objects for Continuous Experimentation: An exploratory multiple-case study
Loading...
Access rights
openAccess
CC BY
CC BY
publishedVersion
URL
Journal Title
Journal ISSN
Volume Title
A4 Artikkeli konferenssijulkaisussa
This publication is imported from Aalto University research portal.
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Unless otherwise stated, all rights belong to the author. You may download, display and print this publication for Your own personal use. Commercial use is prohibited.
Authors
Date
Department
Major/Subject
Mcode
Degree programme
Language
en
Pages
Series
Proceedings of the 29th International Conference on Evaluation and Assessment in Software Engineering, pp. 158-168
Abstract
Continuous experimentation relies on the design and implementation of experiment objects: artefacts that are used as the treatment or stimulus in a software product experiment. To be used in practice, experiment objects must have traits that allow drawing correct inferences regarding a new product, while simultaneously being resource-efficient to produce. In this paper, we investigate what kinds of experiment objects companies use, how they are selected, what factors influence the selection, and how they evolve over time. We present a multiple-case study based on semi-structured interviews with representatives from eight companies. Four kinds of experiment objects were found: prototypes, minimum viable products, proofs-of-concept, and wireframes. The results show that deciding what kind of experiment object to develop is a collaborative and customer-oriented process. As experimentation proceeds, experiment objects can be thrown away, modified for new experiments, or even become products. Experiment objects were understood not only as artefacts in product experiments, but blended with purposes familiar from prototyping and user interface and experience design, and they were used as vehicles for customer communication. The study shows that experiment objects are created for a variety of purposes and understandings of experimentation. Guidelines for creating and managing effective experiment objects for different variations of continuous experimentation are called for.Description
Other note
Citation
Amasya, B & Fagerholm, F 2025, Designing experiment objects for Continuous Experimentation: An exploratory multiple-case study. in M A Babar, A Tosun, S Wagner & V Stray (eds), Proceedings of the 29th International Conference on Evaluation and Assessment in Software Engineering. ACM, pp. 158-168, International Conference on Evaluation and Assessment in Software Engineering, Istanbul, Türkiye, 17/06/2025. https://doi.org/10.1145/3756681.3756944