The Cross-Sequential Approach: A Short-Term Method for Studying Long-Term User Experience

Abstract

We propose a novel cross-sequential approach to accelerating the collection of user feedback while providing a broad perspective by analyzing samples of users at various stages of service and product experience. User experience changes over time as the relationship between user and product evolves. Understanding temporal changes is important for designing positive long-term user experience and supporting customer loyalty. However, companies and researchers face time and cost constraints, which is why longitudinal studies tracking user experience over an extended period of time are extremely rare. The cross-sequential approach we propose reduces these barriers without compromising user-feedback quality. We tested the cross-sequential approach in evaluating the user experience of a commercial Internet security service and identified user experience spanning seven months in only seven weeks with two online surveys. We compared the cross-sequential approach to the cross-sectional approach commonly used in the industry. The cross-sectional approach identified differences between cohorts of 252 users. The cross-sequential approach, on the other hand, allowed us to identify changes in individual users' user experience. These changes could also be tracked to different usage phases. This information could guide the design of better long-term user experience for the service. The positive and negative changes in user enjoyment over time are relevant as they were reflected in users' loyalty to the service, replicating results from longitudinal studies. Thus, the cross-sequential approach can be especially useful when quick, cost-effective user feedback is needed regarding extended periods of using services or products.