Browsing by Author "Daee, Pedram"

Now showing 1 - 12 of 12

Entity Recommendation for Everyday Digital Tasks
(2021-10) Jacucci, Giulio; Daee, Pedram; Vuong, Tung; Andolina, Salvatore; Klouche, Khalil; Sjöberg, Mats; Ruotsalo, Tuukka; Kaski, Samuel
A1 Alkuperäisartikkeli tieteellisessä aikakauslehdessä
Recommender systems can support everyday digital tasks by retrieving and recommending useful information contextually. This is becoming increasingly relevant in services and operating systems. Previous research often focuses on specific recommendation tasks with data captured from interactions with an individual application. The quality of recommendations is also often evaluated addressing only computational measures of accuracy, without investigating the usefulness of recommendations in realistic tasks. The aim of this work is to synthesize the research in this area through a novel approach by (1) demonstrating comprehensive digital activity monitoring, (2) introducing entity-based computing and interaction, and (3) investigating the previously overlooked usefulness of entity recommendations and their actual impact on user behavior in real tasks. The methodology exploits context from screen frames recorded every 2 seconds to recommend information entities related to the current task. We embodied this methodology in an interactive system and investigated the relevance and influence of the recommended entities in a study with participants resuming their real-world tasks after a 14-day monitoring phase. Results show that the recommendations allowed participants to find more relevant entities than in a control without the system. In addition, the recommended entities were also used in the actual tasks. In the discussion, we reflect on a research agenda for entity recommendation in context, revisiting comprehensive monitoring to include the physical world, considering entities as actionable recommendations, capturing drifting intent and routines, and considering explainability and transparency of recommendations, ethics, and ownership of data.
EntityBot: Supporting everyday digital tasks with entity recommendations
(2021-09-13) Vuong, Tung; Andolina, Salvatore; Jacucci, Giulio; Daee, Pedram; Klouche, Khalil; Sjöberg, Mats; Ruotsalo, Tuukka; Kaski, Samuel
A4 Artikkeli konferenssijulkaisussa
Everyday digital tasks can highly benefit from systems that recommend the right information to use at the right time. However, existing solutions typically support only specific applications and tasks. In this demo, we showcase EntityBot, a system that captures context across application boundaries and recommends information entities related to the current task. The user's digital activity is continuously monitored by capturing all content on the computer screen using optical character recognition. This includes all applications and services being used and specific to individuals' computer usages such as instant messaging, emailing, web browsing, and word processing. A linear model is then applied to detect the user's task context to retrieve entities such as applications, documents, contact information, and several keywords determining the task. The system has been evaluated with real-world tasks, demonstrating that the recommendation had an impact on the tasks and led to high user satisfaction.
The good lie: humans steering interactive AI’s behaviour to reach a common goal
(2021-05-17) Colella, Fabio
Perustieteiden korkeakoulu | Master's thesis
The importance of human interaction in machine learning is nowadays becoming more and more recognised. While automatic Machine Learning is undoubtedly powerful, increasing evidence is showing that many scenarios require or benefit from interaction with a human expert. This work focuses on interactive optimisation, in a setting where the user and the system need to collaborate to reach a common goal. The system is an optimiser that tries to find the maximum of a function, but cannot directly access the function. In fact, it needs to ask the user, who can see the function entirely, the function value at each point. This scenario was chosen as it mimics the dynamics of real interactive recommender systems, but avoids its complexity, hence rendering it a flexible model to approximate many similar situations of this kind. Not only it will be shown that most users, after learning a model of the system, start providing feedback different from the true value; but also that such behaviour is beneficial in reaching the common objective. Besides, we will discuss how to model users to capture and predict such behaviour in order to reach the maximum even earlier.
Human Strategic Steering Improves Performance of Interactive Optimization
(2020-07-07) Colella, Fabio; Daee, Pedram; Jokinen, Jussi; Oulasvirta, Antti; Kaski, Samuel
A4 Artikkeli konferenssijulkaisussa
A central concern in an interactive intelligent system is optimization of its actions, to be maximally helpful to its human user. In recommender systems for instance, the action is to choose what to recommend, and the optimization task is to recommend items the user prefers. The optimization is done based on earlier user's feedback (e.g. "likes" and "dislikes"), and the algorithms assume the feedback to be faithful. That is, when the user clicks “like,” they actually prefer the item. We argue that this fundamental assumption can be extensively violated by human users, who are not passive feedback sources. Instead, they are in control, actively steering the system towards their goal. To verify this hypothesis, that humans steer and are able to improve performance by steering, we designed a function optimization task where a human and an optimization algorithm collaborate to find the maximum of a 1-dimensional function. At each iteration, the optimization algorithm queries the user for the value of a hidden function f at a point x, and the user, who sees the hidden function, provides an answer about f(x). Our study on 21 participants shows that users who understand how the optimization works, strategically provide biased answers (answers not equal to f(x)), which results in the algorithm finding the optimum significantly faster. Our work highlights that next-generation intelligent systems will need user models capable of helping users who steer systems to pursue their goals.
Improving genomics-based predictions for precision medicine through active elicitation of expert knowledge
(2018-06-27) Sundin, Iiris; Peltola, Tomi; Micallef, Luana; Afrabandpey, Homayun; Soare, Marta; Majumder, Muntasir Mamun; Daee, Pedram; He, Chen; Serim, Baris; Havulinna, Aki; Heckman, Caroline; Jacucci, Giulio; Marttinen, Pekka; Kaski, Samuel
A1 Alkuperäisartikkeli tieteellisessä aikakauslehdessä
Motivation Precision medicine requires the ability to predict the efficacies of different treatments for a given individual using high-dimensional genomic measurements. However, identifying predictive features remains a challenge when the sample size is small. Incorporating expert knowledge offers a promising approach to improve predictions, but collecting such knowledge is laborious if the number of candidate features is very large. Results: We introduce a probabilistic framework to incorporate expert feedback about the impact of genomic measurements on the outcome of interest and present a novel approach to collect the feedback efficiently, based on Bayesian experimental design. The new approach outperformed other recent alternatives in two medical applications: prediction of metabolic traits and prediction of sensitivity of cancer cells to different drugs, both using genomic features as predictors. Furthermore, the intelligent approach to collect feedback reduced the workload of the expert to approximately 11%, compared to a baseline approach. Availability and implementation: Source code implementing the introduced computational methods is freely available at https://github.com/AaltoPML/knowledge-elicitation-for-precision-medicine. Supplementary information: Supplementary data are available at Bioinformatics online.
Integrating neurophysiologic relevance feedback in intent modeling for information retrieval
(2019-03-12) Jacucci, Giulio; Barral, Oswald; Daee, Pedram; Wenzel, Markus; Serim, Baris; Ruotsalo, Tuukka; Pluchino, Patrik; Freeman, Jonathan; Gamberini, Luciano; Kaski, Samuel; Blankertz, Benjamin
A1 Alkuperäisartikkeli tieteellisessä aikakauslehdessä
The use of implicit relevance feedback from neurophysiology could deliver effortless information retrieval. However, both computing neurophysiologic responses and retrieving documents are characterized by uncertainty because of noisy signals and incomplete or inconsistent representations of the data. We present the first-of-its-kind, fully integrated information retrieval system that makes use of online implicit relevance feedback generated from brain activity as measured through electroencephalography (EEG), and eye movements. The findings of the evaluation experiment (N = 16) show that we are able to compute online neurophysiology-based relevance feedback with performance significantly better than chance in complex data domains and realistic search tasks. We contribute by demonstrating how to integrate in interactive intent modeling this inherently noisy implicit relevance feedback combined with scarce explicit feedback. Although experimental measures of task performance did not allow us to demonstrate how the classification outcomes translated into search task performance, the experiment proved that our approach is able to generate relevance feedback from brain signals and eye movements in a realistic scenario, thus providing promising implications for future work in neuroadaptive information retrieval (IR).
Interactive Intent Modeling Based on Probabilistic Sparse Models
(2017-04-03) Liao, Yi-Ping
Perustieteiden korkeakoulu | Master's thesis
In the exploratory search system, the user interacts with the system by providing feedback on the relevance of the recommended documents and keywords. It is often that the user is unfamiliar with the topic she is investigating, so the system should be able to help her form a precise query and explore the information space. Typically, the exploratory search process is modeled as a contextual bandit problem, a sequential learning algorithm which adopts the recommendation strategy based on user's feedback, aiming at suggesting more precise keywords and retrieving the most relevant documents with minimum user interactions. One big challenge in the exploratory search is that the corpus in which a bandit algorithm explores is huge while the feedback from the user is always scarce, leading to a non-trivial learning problem with large dimensionality and limited observations. In this thesis, I tackle this challenge by adopting Bayesian linear regression with spike and slab priors which enforce sparsity on the feature space, so the bandit algorithm could narrow down the search to the most relevant documents. I incorporate the Expectation Propagation algorithm to approximate the posterior distribution of the sparse model, Thompson sampling to address the exploration-exploitation dilemma, and Topic model to discover the structure of the documents which could provide group information that can further constrain the search space to specific topics. To assess the models, I simulate the user behavior in an exploratory search process and compare the model coefficients learned by linear models using Gaussian prior and, spike-and-slab prior with or without group information. Several performance metrics are also evaluated. Empirically, the spike-and-slab with or without group information perform similarly and outperform Gaussian prior which does not encourage sparsity. The learned model coefficients justify the assumption that most of the coefficients do not contribute to the model. Besides, the model of group spike-and-slab prior has fewer coefficients needed to be estimated than spike-and-slab prior without group information, and potentially could be applied to larger corpora.
Knowledge elicitation via sequential probabilistic inference for high-dimensional prediction
(2017-07-12) Daee, Pedram; Peltola, Tomi; Soare, Marta; Kaski, Samuel
A1 Alkuperäisartikkeli tieteellisessä aikakauslehdessä
Prediction in a small-sized sample with a large number of covariates, the “small n, large p” problem, is challenging. This setting is encountered in multiple applications, such as in precision medicine, where obtaining additional data can be extremely costly or even impossible, and extensive research effort has recently been dedicated to finding principled solutions for accurate prediction. However, a valuable source of additional information, domain experts, has not yet been efficiently exploited. We formulate knowledge elicitation generally as a probabilistic inference process, where expert knowledge is sequentially queried to improve predictions. In the specific case of sparse linear regression, where we assume the expert has knowledge about the relevance of the covariates, or of values of the regression coefficients, we propose an algorithm and computational approximation for fast and efficient interaction, which sequentially identifies the most informative features on which to query expert knowledge. Evaluations of the proposed method in experiments with simulated and real users show improved prediction accuracy already with a small effort from the expert.
Probabilistic Expert Knowledge Elicitation of Feature Relevances in Sparse Linear Regression
(2016) Daee, Pedram; Peltola, Tomi; Soare, Marta; Kaski, Samuel
Other contribution
Probabilistic user modelling methods for improving human-in-the-loop machine learning for prediction
(2021) Daee, Pedram
School of Science | Doctoral dissertation (article-based)
In many machine learning applications and in particular those with only few training data, human involvement in the form of data provider or expert of the task is crucial. However, human interaction with a machine learning model is constrained by (i) the interaction channels, i.e., how human knowledge can be applied in the model, and (ii) the interaction budget, i.e., how much the user is willing to interact with the model. This thesis presents new methods to improve these constraints in human-in-the-loop machine learning. The core idea of the thesis is to jointly model the available data with a model of the human user, i.e., the user model, in a unified probabilistic model and then perform sequential probabilistic inference on the joint model to design improved interaction. The thesis contributes on two types of prediction tasks. The first task is expert knowledge elicitation for high-dimensional prediction. Experts in a field usually have information beyond training data which can help to improve the prediction performance. User models, as priors and likelihood functions, are proposed to directly connect expert knowledge about the relevance of parameters to a model responsible for prediction. The user model can account for complex user behaviour such as users updating their knowledge during the interaction. Furthermore, sequential experimental design on the joint model is employed to query the most informative expert knowledge earlier to minimize the amount of interaction. The second task is personalized recommendation where the goal is to predict the most relevant item for a user with as few interactions as possible. The interactions are based on user relevance feedback on the recommendations. The thesis proposes user models that are able to receive and integrate feedback on multiple domains and sources by providing a joint probabilistic model connecting all feedback types. Sequential inference on the joint model, using Thompson sampling, was employed to find the targeted recommendation with minimum interaction. Simulated experiments and user studies in both tasks demonstrate improved prediction performance only after few interactions with the users. The research highlights the benefits of joint probabilistic modelling of the user and prediction model in interactive tasks.
User controlled Exploration and Exploitaion Search in Multi-Armed Bandits Using ToM(theory of mind)
(2021-08-23) Bhat, Aalok
Perustieteiden korkeakoulu | Master's thesis
The interactive Information Retrieval (IR) system rely on user relevance feedback to update the set of recommendation. However, the user intent while providing such feedback need not necessarily mean relevant/irrelevant. These kinds of feedbacks, if not accounted properly, lead to misinterpretation and biases the learning of the user intent. As users increasingly interact with AI system, they form a metal model of it. And thus they use this model to steer the AI towards their true intent by providing relevance feedback such that it would yield them the desired result. This thesis propose an system which lets users directly interact with the AI's theory of mind and there by aid in steering it towards their true intent. This is achieved by providing an interaction component along with relevance feedback using which the users can control the AI better. The thesis also discusses a metric using which a user can steer the AI towards his/her true intent. Preliminary examination of result shows that such kind of interactive component can use used in interactive Information Retrieval system to speed up the process of information retrieval.
User Modelling for Avoiding Overfitting in Interactive Knowledge Elicitation for Prediction
(2018-03-08) Daee, Pedram; Peltola, Tomi; Vehtari, Aki; Kaski, Samuel
A4 Artikkeli konferenssijulkaisussa
In human-in-the-loop machine learning, the user provides information beyond that in the training data. Many algorithms and user interfaces have been designed to optimize and facilitate this human--machine interaction; however, fewer studies have addressed the potential defects the designs can cause. Effective interaction often requires exposing the user to the training data or its statistics. The design of the system is then critical, as this can lead to double use of data and overfitting, if the user reinforces noisy patterns in the data. We propose a user modelling methodology, by assuming simple rational behaviour, to correct the problem. We show, in a user study with 48 participants, that the method improves predictive performance in a sparse linear regression sentiment analysis task, where graded user knowledge on feature relevance is elicited. We believe that the key idea of inferring user knowledge with probabilistic user models has general applicability in guarding against overfitting and improving interactive machine learning.