Browsing by Author "Oulasvirta, Antti"

Filter results by typing the first few letters
Now showing 1 - 20 of 186
  • Aalto Interface Metrics (AIM): A service and codebase for computational GUI evaluation
    (2018-10-11) Oulasvirta, Antti; De Pascale, Samuli; Koch, Janin; Langerak, Thomas; Jokinen, Jussi; Todi, Kashyap; Laine, Markku; Kristhombuge, Manoj; Zhu, Yuxi; Miniukovich, Aliaksei; Palmas, Gregorio; Weinkauf, Tino
     Poster
    Aalto Interface Metrics (AIM) pools several empirically validated models and metrics of user perception and attention into an easy-to-use online service for the evaluation of graphical user interface (GUI) designs. Users input a GUI design via URL, and select from a list of 17 different metrics covering aspects ranging from visual clutter to visual learnability. AIM presents detailed breakdowns, visualizations, and statistical comparisons, enabling designers and practitioners to detect shortcomings and possible improvements. The web service and code repository are available at interfacemetrics.aalto.fi.
  • Ability-based optimization of touchscreen interactions
    (2018-01-01) Sarcar, Sayan; Jokinen, Jussi P.P.; Oulasvirta, Antti; Wang, Zhenxin; Silpasuwanchai, Chaklam; Ren, Xiangshi
     A1 Alkuperäisartikkeli tieteellisessä aikakauslehdessä
    Ability-based optimization is a computational approach for improving interface designs for users with sensorimotor and cognitive impairments. Designs are created by an optimizer, evaluated against task-specific cognitive models, and adapted to individual abilities. The approach does not necessitate extensive data collection and could be applied both automatically and manually by users, designers, or caretakers. As a first step, the authors present optimized touchscreen layouts for users with tremor and dyslexia that potentially improve text-entry speed and reduce error.
  • Active learning and interactive training for retinal image classification
    (2018-06-18) Sahlsten, Jaakko
     Perustieteiden korkeakoulu | Master's thesis
    The goal of this study is to investigate application of deep learning and human-computer interaction on diagnosing diabetic retinopathy from colour fundus images. We apply deep learning and study the effects of network pretraining, active learning and a personalised annotation on a private dataset. Diabetic retinopathy is a global issue with increasing number of patients and screening cases each year. In the current trend, an increasing amount of fundus images are scanned which in turn require diagnosis of diabetic retinopathy and other eye diseases, constituting in a major expenditure in an ophthalmologist's time. To aid and speed up the increasing diagnosis and annotation tasks, a machine learning solution is suggested for automatic diagnosis of diabetic retinopathy from colour fundus images. The State-of-the-art deep neural network has been demonstrated to achieve the same performance as opthamologists in the diagnosing referable diabetic retinopathy, while trained on tens of thousands of colour fundus images and associated labels. In this work the State-of-the-art model was deployed using smaller dataset. The model was trained from random initialisation and from pretrained weights from training on ImageNet dataset. Fine-tuning the pretrained network was compared to a network trained from scratch on two test sets. Fine-tuned model had area under receiving operator characteristic (ROCAUC) of 0.965 and 0.921, and model trained from random initialisation had ROCAUC of 0.962 and 0.879. Active learning is a well-studied subfield of machine learning and has been applied successfully. However, there is limited literate on applying it to high-dimensional data with deep neural networks. In this work, recent active learning solutions were applied to diabetic retinopathy classification in order to reduce required size of dataset to achieve required opthamologists performance in classifying referable diabetic retinopathy when applied in screening. The solution achieved the threshold with 8700 images compared to randomly sampled requiring 10500 images. A model was developed attempting to learn the user preferences in annotation with the help of pretrained network. The trained model was compared to a reference model with no human feedback and evaluated on subjective and objective performance. Tool was tested anecdotally which showed that it was able to subjective gradability to some extent. However, the tool did not provide additional benefits in subjective classification of retinopathy.
  • AdaM: Adapting Multi-User Interfaces for Collaborative Environments in Real-Time
    (2018-04) Park, Seonwook; Gebhardt, Christoph; Rädle, Roman; Feit, Anna Maria; Vrzakova, Hana; Dayama, Niraj Ramesh; Yeo, Hui-Shyong; Klokmose, Clemens N; Quigley, Aaron; Oulasvirta, Antti
     A4 Artikkeli konferenssijulkaisussa
    Developing cross-device multi-user interfaces (UIs) is a challenging problem. There are numerous ways in which content and interactivity can be distributed. However, good solutions must consider multiple users, their roles, their preferences and access rights, as well as device capabilities. Manual and rule-based solutions are tedious to create and do not scale to larger problems nor do they adapt to dynamic changes, such as users leaving or joining an activity. In this paper, we cast the problem of UI distribution as an assignment problem and propose to solve it using combinatorial optimization. We present a mixed integer programming formulation which allows real-time applications in dynamically changing collaborative settings. It optimizes the allocation of UI elements based on device capabilities, user roles, preferences, and access rights. We present a proof-of-concept designer-in-the-loop tool, allowing for quick solution exploration. Finally, we compare our approach to traditional paper prototyping in a lab study.
  • Adapting User Interfaces with Model-based Reinforcement Learning
    (2021-05-06) Todi, Kashyap; Leiva, Luis; Bailly, Gilles; Oulasvirta, Antti
     A4 Artikkeli konferenssijulkaisussa
    Adapting an interface requires taking into account both the positive and negative effects that changes may have on the user. A carelessly picked adaptation may impose high costs to the user – for example, due to surprise or relearning effort – or "trap" the process to a suboptimal design immaturely. However, effects on users are hard to predict as they depend on factors that are latent and evolve over the course of interaction. We propose a novel approach for adaptive user interfaces that yields a conservative adaptation policy: It finds beneficial changes when there are such and avoids changes when there are none. Our model-based reinforcement learning method plans sequences of adaptations and consults predictive HCI models to estimate their effects. We present empirical and simulation results from the case of adaptive menus, showing that the method outperforms both a non-adaptive and a frequency-based policy.
  • Adaptive feature guidance: Modelling visual search with graphical layouts
    (2020-04-01) Jokinen, Jussi P.P.; Wang, Zhenxin; Sarcar, Sayan; Oulasvirta, Antti; Ren, Xiangshi
     A1 Alkuperäisartikkeli tieteellisessä aikakauslehdessä
    We present a computational model of visual search on graphical layouts. It assumes that the visual system is maximising expected utility when choosing where to fixate next. Three utility estimates are available for each visual search target: one by unguided perception only, and two, where perception is guided by long-term memory (location or visual feature). The system is adaptive, starting to rely more upon long-term memory when its estimates improve with experience. However, it needs to relapse back to perception-guided search if the layout changes. The model provides a tool for practitioners to evaluate how easy it is to find an item for a novice or an expert, and what happens if a layout is changed. The model suggests, for example, that (1) layouts that are visually homogeneous are harder to learn and more vulnerable to changes, (2) elements that are visually salient are easier to search and more robust to changes, and (3) moving a non-salient element far away from original location is particularly damaging. The model provided a good match with human data in a study with realistic graphical layouts.
  • An adaptive model of gaze-based selection
    (2021-05-06) Chen, Xiuli; Acharya, Aditya; Oulasvirta, Antti; Howes, Andrew
     A4 Artikkeli konferenssijulkaisussa
    Gaze-based selection has received signifcant academic attention over a number of years. While advances have been made, it is possible that further progress could be made if there were a deeper understanding of the adaptive nature of the mechanisms that guide eye movement and vision. Control of eye movement typically results in a sequence of movements (saccades) and fxations followed by a dwell' at a target and a selection. To shed light on how these sequences are planned, this paper presents a computational model of the control of eye movements in gaze-based selection.We formulate the model as an optimal sequential planning problem bounded by the limits of the human visual and motor systems and use reinforcement learning to approximate optimal solutions. The model accurately replicates earlier results on the efects of target size and distance and captures a number of other aspects of performance. The model can be used to predict number of fxations and duration required to make a gaze-based selection. The future development of the model is discussed.
  • AI-Assisted for Modeling Multitasking Driver
    (2022-12-12) Firooz, Hossein
     Perustieteiden korkeakoulu | Master's thesis
    Driving a vehicle is one of the most complicated tasks for the artificial intelligence algorithms to learn and perform well. One of the approaches to tackle this problem towards creating fully autonomous cars is to understand the human driver. The human driver behind the wheel, acts as a multitask agent whose main task is driving, but also interacts with other passengers, in-car entertainment and information systems, or her mobile devices. In this thesis, we will create an AI agent using Reinforcement Learning algorithms to model the multitasking driver behavior.
  • Älykkäät oppimisjärjestelmät
    (2020-12-04) Tuovinen, Katariina
     Sähkötekniikan korkeakoulu | Bachelor's thesis
  • Amortized Inference with User Simulations
    (2023-04-19) Moon, Hee Seung; Oulasvirta, Antti; Lee, Byungjoo
     A4 Artikkeli konferenssijulkaisussa
    There have been significant advances in simulation models predicting human behavior across various interactive tasks. One issue remains, however: identifying the parameter values that best describe an individual user. These parameters often express personal cognitive and physiological characteristics, and inferring their exact values has significant effects on individual-level predictions. Still, the high complexity of simulation models usually causes parameter inference to consume prohibitively large amounts of time, as much as days per user. We investigated amortized inference for its potential to reduce inference time dramatically, to mere tens of milliseconds. Its principle is to pre-train a neural proxy model for probabilistic inference, using synthetic data simulated from a range of parameter combinations. From examining the efficiency and prediction performance of amortized inference in three challenging cases that involve real-world data (menu search, point-and-click, and touchscreen typing), the paper demonstrates that an amortized-inference approach permits analyzing large-scale datasets by means of simulation models. It also addresses emerging opportunities and challenges in applying amortized inference in HCI.
  • Analyzing Finnish Customer Feedback with Natural Language Processing
    (2019-06-17) Kiviniemi-Ghonim, Anastasia
     Sähkötekniikan korkeakoulu | Master's thesis
    Nowadays companies receive a large number of customer feedback from different channels. Handling customer feedback is essential in terms of maintaining customer satisfaction, listening to the needs and suggestions, and establishing a closer relationship. Natural Language Processing is a rapidly growing field that provides methods for processing and analyzing unstructured textual data. One of the methods for analyzing the semantics of the text is called sentiment analysis. The main task of sentiment analysis is to identify the polarity of the text. This thesis is focused on building a sentiment analysis algorithm for Finnish language using different tools required for text processing steps. The aim was to analyze customer feedback of Telia Finland customers, study the correlation between star ratings and corresponding written comments and perform entity analysis in order to examine a reason for negative reviews within disconnection orders. In addition, the performance of sentiment analysis algorithm was evaluated, and it was estimated that it’s accurate enough for intended usage. The results showed a positive moderate correlation between customer ratings and sentiment scores. Neutrally scored reviews resulted to be majorly negative. Entity analysis showed that most of the complaints within channel package disconnection were associated with the difficulty of provided service and termination of free channel packages.
  • Approaching Aesthetics on User Interface and Interaction Design
    (2018-11-19) Wang, Chen; Sarcar, Sayan; Kurosu, Masaaki; Bardzell, Jeffrey; Oulasvirta, Antti; Miniukovich, Aliaksei; Ren, Xiangshi
     A4 Artikkeli konferenssijulkaisussa
    Although the HCI community inevitably contributes to engagement via beauty according to the attention paid to known and yet to be discovered principles of aesthetics for digital interface design, it is lacking an epistemological corpus which should include the notion, human factors and the quantification of aesthetic aspects. The aim of the proposed workshop is to discuss these issues in order to strengthen aesthetic studies specifically for HCI and related fields. We want to create a forum for discussing, drafting and promoting the foundations for disciplined aesthetics design within the HCI community. We thus welcome contributions such as theories, methodologies, evaluation methods, and potential applications regarding effective aesthetics for HCI and related fields. Concretely, we aim to (i) map the present state-of-art of aesthetic research in HCI, (ii) build a multidisciplinary community of experts, and (iii) raise the profile of this aesthetics research area within HCI community.
  • Approaching Engagement towards Human-Engaged Computing
    (2018-04) Salehzadeh Niksirat, Kavous; Sarcar, Sayan; Sun, Huatong; Law, Effie LC; Clemmensen, Torkil; Bardzell, Jeffrey; Oulasvirta, Antti; Silpasuwanchai, Chaklam; Light, Ann; Ren, Xiangshi
     Abstract
    Debates regarding the nature and role of HCI research and practice have intensified in recent years, given the ever increasingly intertwined relations between humans and technologies. The framework of Human-Engaged Computing (HEC) was proposed and developed over a series of scholarly workshops to complement mainstream HCI models by leveraging synergy between humans and computers with its key notion of "engagement". Previous workshop meetings found "engagement" to be a constructive and extendable notion through which to investigate synergized human-computer relationships, but many aspects concerning the core concept remain underexplored. This SIG aims to tackle the notion of engagement considered through discussions of four thematic threads. It will bring together HCI practitioners and researchers from different disciplines including Humanities, Design, Positive Psychology, Communication and Media Studies, Neuroscience, Philosophy and Eastern Studies, to share and discuss relevant knowledge and insights and identify new research opportunities and future directions.
  • Artificial Intelligence in public employment services: chatbot as a personal coach
    (2022-10-17) Ma, Chengwei
     Perustieteiden korkeakoulu | Master's thesis
    In Finland, the government is undergoing a number of reforms in order to promote employment and improve the services for job seekers. These reforms require municipalities to take more responsibilities to guide individual job seekers. In this case, current staff of municipalities (personal coaches) would have more workload, resulting in low motivation and long-term communication to serve job seekers. Besides, job seekers would be disappointed with the services. AI has been widely used in many public sectors all over the world to facilitate the work flow of public services and reduce pressure on human resource. Therefore, this thesis aims to investigate the possibility of applying AI chatbots, as personal coaches, to offer employment services that fulfill the user needs of job seekers, and thus reduce workloads of human career coaches and bring better services. This study was done under a research collaboration between the Department of Design, Aalto University, and the City of Espoo. Its research process included two parts: User Research and User Testing. The former part investigated citizens' attitudes and expectations towards applying AI in public employment services, through methods of online survey, co-design workshop, and guerrilla interviews. The latter one focused on the validation of the feasibility of applying AI chatbots in public employment services through a usability test on a chatbot prototype. Results from the User Research part showed that citizens had an overall positive attitude and expected AI could offer them services of job search, networking, career guidance and so on, while the User Testing results showed that AI chatbot could meet citizens' expectations and needs, and feasibly work as a personal coach in public employment services.
  • Audio and Text Conditioned Abstract Sound Synthesis through Human-AI Interaction
    (2023-01-23) Hassinen, Heidi
     Perustieteiden korkeakoulu | Master's thesis
    Recent trends in computational creativity research have drawn attention to multi- modal models relating data from two or more modalities, such as text, image and audio. Even though multimodal models have been demonstrated as a successful approach to text-conditioned image generation, such models have not been as studied for generative tasks in the audio domain. This work attempts to fill the gap by study- ing audio and text conditioned abstract sound synthesis based on the multimodal AudioCLIP model. By creating sound abstractions from user input, the studied synthesis algorithm aims to allow such human-computer co-exploration with artificial intelligence (AI) adaptable to artistic work. As a computational creativity support tool, the studied algorithm is among the few tools offering AI-based ideation for professional composers. This work evaluates qualitatively the suitability of the suggested abstract sound synthesis algorithm for co-creative ideation. While different approaches to abstract sound synthesis were compared with experiments, the quality of the synthesized sounds and their usefulness to artistic work were evaluated in a user study with professional composers. As one of the main findings, the research reveals that the AudioCLIP model is not effective enough for the studied multimodal generative task. However, the synthesis-by-optimization approach adapted from an exemplary study is able to create sounds that are interesting to professional composers. Ensuring stronger resemblance between user-provided input and generated results and providing users more control for steering the system is among the topics worth further research. In addition, further research is motivated by professional composers’ need for automatic tools to replace the manual work of idea generation from inspirational examples, as observed in the user study.
  • AUIT - the Adaptive User Interfaces Toolkit for Designing XR Applications
    (2022-10-29) Evangelista Belo, João Marcelo; Lystbæk, Mathias N.; Feit, Anna Maria; Pfeuffer, Ken; Kán, Peter; Oulasvirta, Antti; Grønbæk, Kaj
     A4 Artikkeli konferenssijulkaisussa
    Adaptive user interfaces can improve experiences in Extended Reality (XR) applications by adapting interface elements according to the user's context. Although extensive work explores different adaptation policies, XR creators often struggle with their implementation, which involves laborious manual scripting. The few available tools are underdeveloped for realistic XR settings where it is often necessary to consider conflicting aspects that affect an adaptation. We fill this gap by presenting AUIT, a toolkit that facilitates the design of optimization-based adaptation policies. AUIT allows creators to flexibly combine policies that address common objectives in XR applications, such as element reachability, visibility, and consistency. Instead of using rules or scripts, specifying adaptation policies via adaptation objectives simplifies the design process and enables creative exploration of adaptations. After creators decide which adaptation objectives to use, a multi-objective solver finds appropriate adaptations in real-time. A study showed that AUIT allowed creators of XR applications to quickly and easily create high-quality adaptations.
  • AutoGain: Gain Function Adaptation with Submovement Efficiency Optimization
    (2020-04-21) Lee, Byungjoo; Nancel, Mathieu; Kim, Sunjun; Oulasvirta, Antti
     A4 Artikkeli konferenssijulkaisussa
    A well-designed control-to-display gain function can improve pointing performance with indirect pointing devices like trackpads. However, the design of gain functions is challenging and mostly based on trial and error. AutoGain is a novel method to individualize a gain function for indirect pointing devices in contexts where cursor trajectories can be tracked. It gradually improves pointing efficiency by using a novel submovement-level tracking+optimization technique that minimizes aiming error (undershooting/overshooting) for each submovement. We first show that AutoGain can produce, from scratch, gain functions with performance comparable to commercial designs, in less than a half-hour of active use. Second, we demonstrate AutoGain’s applicability to emerging input devices (here, a Leap Motion controller) with no reference gain functions. Third, a one-month longitudinal study of normal computer use with AutoGain showed performance improvements from participants’ default functions.
  • Bayesian methods in interaction design (tutorial)
    (2020-03-17) Williamson, John; Oulasvirta, Antti; Kristensson, Per Ola
     A4 Artikkeli konferenssijulkaisussa
    This tutorial introduces Bayesian computational approaches to interaction and design. Bayesian methods offer a powerful approach for interactive settings with uncertainty and noise. This course introduces the theory and practice of computational Bayesian interaction, covering inference of user data and design/adaptation of interface features based around probabilistic inference. The tutorial is built around hands-on Python programming with modern computational tools, interleaved with theory and practical examples grounded in problems of wide interest in human-computer interaction.
  • Big data visualization and analytics: Future research challenges and emerging applications
    (2020) Andrienko, Gennady; Andrienko, Natalia; Drucker, Steven; Fekete, Jean Daniel; Fisher, Danyel; Idreos, Stavros; Kraska, Tim; Li, Guoliang; Ma, Kwan Liu; Mackinlay, Jock D.; Oulasvirta, Antti; Schreck, Tobias; Schmann, Heidrun; Stonebraker, Michael; Auber, David; Bikakis, Nikos; Chrysanthis, Panos K.; Papastefanatos, George; Sharaf, Mohamed A.
     A4 Artikkeli konferenssijulkaisussa
    In the context of data visualization and analytics, this report outlines some of the challenges and emerging applications that arise in the Big Data era. In particularly, fourteen distinguished scientists from academia and industry, and diverse related communities, i.e., Information Visualization, Human-Computer Interaction, Machine Learning, Data management & Mining, and Computer Graphics have been invited to express their opinions.
  • Boxer: A multimodal collision technique for virtual objects
    (2017-11-03) Lee, Byungjoo; Deng, Qiao; Hoggan, Eve; Oulasvirta, Antti
     A4 Artikkeli konferenssijulkaisussa
    Virtual collision techniques are interaction techniques for invoking discrete events in a virtual scene, e.g. throwing, pushing, or pulling an object with a pointer. The conventional approach involves detecting collisions as soon as the pointer makes contact with the object. Furthermore, in general, motor patterns can only be adjusted based on visual feedback. The paper presents a multimodal technique based on the principle that collisions should be aligned with the most salient sensory feedback. Boxer (1) triggers a collision at the moment where the pointer's speed reaches a minimum after first contact and (2) is synchronized with vibrotactile stimuli presented to the hand controlling the pointer. Boxer was compared with the conventional technique in two user studies (with temporal pointing and virtual batting). Boxer improved spatial precision in collisions by 26.7% while accuracy was compromised under some task conditions. No difference was found in temporal precision. Possibilities for improving virtual collision techniques are discussed.