Browsing by Author "Klami, Arto"

Now showing 1 - 16 of 16

Bayesian exponential family projections
(2010) Virtanen, Seppo Juhani
School of Electrical Engineering | Master's thesis
Exploratory data analysis stands for extracting useful information from data sets. Machine learning methods automate this process by fitting models to data. It is essential to provide all available background knowledge for building such models. Principal component analysis is a standard method for exploratory data analysis. Recently its probabilistic interpretation has illustrated that it is only suitable for a specific type of data. Extension of principal component analysis to the exponential family removes this problem. In this thesis a general model family suitable for the analysis of multiple data sources is presented by building on the exponential family principal component analysis. The unifying framework contains as special cases methods suitable for unsupervised and supervised learning. While earlier methods have mainly relied on maximum likelihood inference, in this thesis Bayesian modelling is chosen. In Bayesian modelling background knowledge is utilized in the form of prior distributions. In this thesis, a general prior distribution is proposed that takes distribution-specific constraints into account. Multiple contributions to modelling, inference and model interpretation are introduced. With empirical experiments it is demonstrated how the proposed methods outperform traditional methods.
Flexible prior elicitation via the prior predictive distribution
(2020) Hartmann, Marcelo; Agiashvili, Georgi; Bürkner, Paul; Klami, Arto
A4 Artikkeli konferenssijulkaisussa
The prior distribution for the unknown model parameters plays a crucial role in the process of statistical inference based on Bayesian methods. However, specifying suitable priors is often difficult even when detailed prior knowledge is available in principle. The challenge is to express quantitative information in the form of a probability distribution. Prior elicitation addresses this question by extracting subjective information from an expert and transforming it into a valid prior. Most existing methods, however, require information to be provided on the unobservable parameters, whose effect on the data generating process is often complicated and hard to understand. We propose an alternative approach that only requires knowledge about the observable outcomes - knowledge which is often much easier for experts to provide. Building upon a principled statistical framework, our approach utilizes the prior predictive distribution implied by the model to automatically transform experts judgements about plausible outcome values to suitable priors on the parameters. We also provide computational strategies to perform inference and guidelines to facilitate practical use.
How Suitable Is Your Naturalistic Dataset for Theory-based User Modeling?
(2022-04-07) Putkonen, Aini; Nioche, Aurélien; Tanskanen, Ville; Klami, Arto; Oulasvirta, Antti
A4 Artikkeli konferenssijulkaisussa
Theory-based, or "white-box,"models come with a major benefit that makes them appealing for deployment in user modeling: their parameters are interpretable. However, most theory-based models have been developed in controlled settings, in which researchers determine the experimental design. In contrast, real-world application of these models demands setups that are beyond developer control. In non-experimental, naturalistic settings, the tasks with which users are presented may be very limited, and it is not clear that model parameters can be reliably inferred. This paper describes a technique for assessing whether a naturalistic dataset is suitable for use with a theory-based model. The proposed parameter recovery technique can warn against possible over-confidence in inferred model parameters. This technique also can be used to study conditions under which parameter inference is feasible. The method is demonstrated for two models of decision-making under risk with naturalistic data from a turn-based game.
Locally linear robust Bayesian dependency modeling of co-occurrence data
(2010) Viinikanoja, Jaakko
School of Science | Master's thesis
Traditional experimental design in, for instance, neuroscience is usually set up so that one needs to explain the observed signal in terms of few controlled covariates. In this setting relatively simple techniques, based on, for instance, averaging, are sufficient for data analysis. However, the analysis becomes considerably more complicated in less artificial scenarios where the cushion from experimental design is lost. The main contribution of this thesis is a novel data-driven model for analysis of natural signals resulting from such less restrictive experimental designs. More precisely we consider a task of extracting a relevant signal from observed natural signal which also includes structured noise. Traditional unsupervised methods, such as PCA, do not work in this setting because the variability of structured noise can be significant. Hence, supervision is necessary, but usually ground-truth supervision is not available or is prohibitively expensive to implement. The developed novel model is based on recently introduced approach where the supervision is learned by modelling mutual dependencies between paired multi-source data. Essentially, the assumption is that if the phenomena of interest are manifested in multiple signals, the extracted relevant signals should be statistically dependent. Canonical Correlation Analysis (CCA) is the established baseline method for extracting mutually dependent parts from multi-source data. However, plain CCA has multiple problems in dealing with natural signals. For instance, the inference is neither reliable nor robust and the signals are required to be stationary. Developed from the probabilistic interpretation of CCA, our novel model addresses these issues by switching to a mixture formulation which uses robust Bayesian inference.
Mikrosirumittausten välisten riippuvuuksien haku kanonisen korrelaatioanalyysin laajennuksien avulla
(2008) Virtanen, Seppo
Elektroniikan, tietoliikenteen ja automaation tiedekunta | Bachelor's thesis
Modeling of mutual dependencies
(2008) Klami, Arto
Doctoral dissertation (article-based)
Data analysis means applying computational models to analyzing large collections of data, such as video signals, text collections, or measurements of gene activities in human cells. Unsupervised or exploratory data analysis refers to a subtask of data analysis, in which the goal is to find novel knowledge based on only the data. A central challenge in unsupervised data analysis is separating relevant and irrelevant information from each other. In this thesis, novel solutions to focusing on more relevant findings are presented. Measurement noise is one source of irrelevant information. If we have several measurements of the same objects, the noise can be suppressed by averaging over the measurements. Simple averaging is, however, only possible when the measurements share a common representation. In this thesis, we show how irrelevant information can be suppressed or ignored also in cases where the measurements come from different kinds of sensors or sources, such as video and audio recordings of the same scene. For combining the measurements, we use mutual dependencies between them. Measures of dependency, such as mutual information, characterize commonalities between two sets of measurements. Two measurements can hence be combined to reduce irrelevant variation by finding new representations for the objects so that the representations are maximally dependent. The combination is optimal, given the assumption that what is in common between the measurements is more relevant than information specific to any one of the sources. Several practical models for the task are introduced. In particular, novel Bayesian generative models, including a Bayesian version of the classical method of canonical correlation analysis, are given. Bayesian modeling is especially justified approach to learning from small data sets. Hence, generative models can be used to extract dependencies in a more reliable manner in, for example, medical applications, where obtaining a large number of samples is difficult. Also, novel non-Bayesian models are presented: Dependent component analysis finds linear projections which capture more general dependencies than earlier methods. Mutual dependencies can also be used for supervising traditional unsupervised learning methods. The learning metrics principle describes how a new distance metric focusing on relevant information can be derived based on the dependency between the measurements and a supervising signal. In this thesis, the approximations and optimization methods required for using the learning metrics principle are improved.
Modeling Risky Choices in Unknown Environments
(2021-05-01) Tanskanen, Ville; Rajani, Chang; Afrabandpey, Homayun; Putkonen, Aini; Nioche, Aurélien; Klami, Arto
A4 Artikkeli konferenssijulkaisussa
Decision-theoretic models explain human behavior in choice problems involving uncertainty, in terms of individual tendencies such as risk aversion. However, many classical models of risk require knowing the distribution of possible outcomes (rewards) for all options, limiting their applicability outside of controlled experiments. We study the task of learning such models in contexts where the modeler does not know the distributions but instead can only observe the choices and their outcomes for a user familiar with the decision problems, for example a skilled player playing a digital game. We propose a framework combining two separate components, one for modeling the unknown decision-making environment and another for the risk behavior. By using environment models capable of learning distributions we are able to infer classical models of decision-making under risk from observations of the user’s choices and outcomes alone, and we also demonstrate alternative models for predictive purposes. We validate the approach on artificial data and demonstrate a practical use case in modeling risk attitudes of professional esports teams.
Prior knowledge elicitation: The past, present, and future
(2024-12) Mikkola, Petrus; Martin, Osvaldo; Halasinamara Chandramouli, Suyog; Hartmann, Marcelo; Abril Pla, Oriol; Thomas, Owen; Pesonen, Henri; Corander, Jukka; Vehtari, Aki; Kaski, Samuel; Burkner, Paul-Christian; Klami, Arto
A1 Alkuperäisartikkeli tieteellisessä aikakauslehdessä
Specification of the prior distribution for a Bayesian model is a central part of the Bayesian workflow for data analysis, but it is often difficult even for statistical experts. In principle, prior elicitation transforms domain knowledge of various kinds into well-defined prior distributions, and offers a solution to the prior specification problem. In practice, however, we are still fairly far from having usable prior elicitation tools that could significantly influence the way we build probabilistic models in academia and industry. We lack elicitation methods that integrate well into the Bayesian workflow and perform elicitation efficiently in terms of costs of time and effort. We even lack a comprehensive theoretical framework for understanding different facets of the prior elicitation problem. Why are we not widely using prior elicitation? We analyse the state of the art by identifying a range of key aspects of prior knowledge elicitation, from properties of the modelling task and the nature of the priors to the form of interaction with the expert. The existing prior elicitation literature is reviewed and categorized in these terms. This allows recognizing under-studied directions in prior elicitation research, finally leading to a proposal of several new avenues to improve prior elicitation methodology.
A Proactive Interface for Image Retrieval
(2009) Kozma, László
Helsinki University of Technology | Master's thesis
This thesis studies interfaces for browsing and searching for images. A novel gaze-based interface was developed which attempts to tailor the set of available choices according to the interest of the user. The system is integrated with PicSOM, a content-based image retrieval engine and is able to interact with various eye tracking devices. It can be used in online exploration of large image databases. Users interact with the system through a zooming interface inspired by the concept of infinite-desktops and Dasher, a tool for predictive text entry. The system computes on-line predictions of relevance of images based on implicit feedback, and when the user zooms in, the images predicted to be the most relevant are brought out. The key novelty is that the relevance feedback is inferred from implicit cues obtained in real-time from the gaze pattern, using an estimator learned during a separate training phase. It is found that there is sufficient amount of information in the gaze patterns to make the estimated relevance feedback a viable choice to complement or ultimately even replace explicit feedback by pointing-and-clicking, although the accuracy is not as good as with explicit feedback. The reliability of the relevance prediction is evaluated first as a stand-alone module, then in integration with the full system. For this purpose we carried out eye tracking experiments with test subjects using our software. The thesis describes the design and implementation of the image navigation system, and an evaluation of its performance. In addition, alternative approaches were explored. A simpler variant of the interface uses clicking or explicit selection with eye movements as feedback. An interface using a static collage-view has also been developed.
Proceedings of ICANN/PASCAL2 Challenge: MEG Mind Reading
(2011) Klami, Arto
School of Science | D4 Julkaistu kehittämis- tai tutkimusraportti tai -selvitys
This report summarizes the outcomes of the MEG mind reading challenge organized in conjunction with the International Conference on Artificial Neural Networks (ICANN) 2011, sponsored by the PASCAL2 Challenge Programme. The challenge task was to infer from brain activity, measured with MEG, the type of a video stimulus shown to the subject. Successful solutions would then allow determining, for example, whether a subject is watching football or a comedy film based on short single-trial recording of brain activity. The challenge was organized to study the feasibility of such decoding tasks, to discuss related machine learning methods and solutions, and to promote the awareness of the interesting problem formulation. Furthermore, the challenge produced a publicly available data set that can be used for future benchmarking of decoding methods. The best participants reached high accuracy in the task, demonstrating successful brain decoding. The collection contains an full description of the challenge and its results, as well as technical descriptions of the best three submissions written by the participants.
Regularized Discriminative Clustering
(2003) Klami, Arto
Helsinki University of Technology | Master's thesis
Ryhmäfaktorianalyysi neurotiedesovelluksissa
(2012-09-19) Remes, Sami
Perustieteiden korkeakoulu | Bachelor's thesis
Scalable Crop Yield Prediction with Sentinel-2 Time Series and Temporal Convolutional Network
(2022-09) Yli-Heikkila, Maria; Wittke, Samantha; Luotamo, Markku; Puttonen, Eetu; Sulkava, Mika; Pellikka, Petri; Heiskanen, Janne; Klami, Arto
A1 Alkuperäisartikkeli tieteellisessä aikakauslehdessä
One of the precepts of food security is the proper functioning of the global food markets. This calls for open and timely intelligence on crop production on an agroclimatically meaningful territorial scale. We propose an operationally suitable method for large-scale in-season crop yield estimations from a satellite image time series (SITS) for statistical production. As an object-based method, it is spatially scalable from parcel to regional scale, making it useful for prediction tasks in which the reference data are available only at a coarser level, such as counties. We show that deep learning-based temporal convolutional network (TCN) outperforms the classical machine learning method random forests and produces more accurate results overall than published national crop forecasts. Our novel contribution is to show that mean-aggregated regional predictions with histogram-based features calculated from farm-level observations perform better than other tested approaches. In addition, TCN is robust to the presence of cloudy pixels, suggesting TCN can learn cloud masking from the data. The temporal compositing of information do not improve prediction performance. This indicates that with end-to-end learning less preprocessing in SITS tasks seems viable.
Transfer Learning with Group Factor Analysis
(2013) Leppäaho, Eemeli
Perustieteiden korkeakoulu | Master's thesis
Modern measuring techniques allow us to get more and more data in less time and cheaper price. When analyzing data, one sample might be the gene expression of a cell or the activity of a human brain at a certain time, consisting of tens of thousands of features. Often we have much fewer samples than features, and simple methods will overfit the data. Factor models are designed to model this kind of high-dimensional data via a lower dimensional factor space. Factor analysis is the simplest factor model: it reconstructs each feature in the data as a weighted sum of the hidden factors (components). In this thesis I examine group factor analysis (GFA), which is an extension of factor analysis for multiple data sets. High-dimensional data can often be naturally divided to different groups (views), which GFA uses as prior information by inferring the component activities for views instead of single features. This property combined with an automatic system for the component activity determination results in a powerful factor model. In this thesis, GFA is extended to explicitly model hidden relations between different data views. This is done by generating their component activity matrix in two alternative ways: as samples of a multivariate normal distribution and as a product of two low-rank matrices. Both the extensions are solved via variational Bayesian inference, and are shown to model data with accuracy comparable to GFA. For data with many views low-rank GFA is the most accurate model. Additionally the problem of small number of samples is dealt with two transfer learning setups: one being able to take advantage of background data with samples or features shared with target data, and the other introducing a novel transfer learning setup. It is shown, using both artificial and real data, that both of these setups allow us to form a better model when suitable background data is available. The real data consists of drug response profiles measured on cell lines using two different microarray platforms.
Using dependencies to pair samples for multi-view learning
(2008) Tripathi, Abhishek; Klami, Arto; Kaski, Samuel
Faculty of Information and Natural Sciences | D4 Julkaistu kehittämis- tai tutkimusraportti taikka -selvitys
Several data analysis tools such as (kernel) canonical correlation analysis and various multi-view learning methods require paired observations in two data sets. We study the problem of inferring such pairing for data sets with no known one-to-one pairing. The pairing is found by an iterative algorithm that alternates between searching for feature representations that reveal statistical dependencies between the data sets, and finding the best pairs for the samples. The method is applied on pairing probe sets of two different microarray platforms.
Virtual laboratories : Transforming research with AI
(2024-08-27) Klami, Arto; Damoulas, Theo; Engkvist, Ola; Rinke, Patrick; Kaski, Samuel
A1 Alkuperäisartikkeli tieteellisessä aikakauslehdessä
New scientific knowledge is needed more urgently than ever, to address global challenges such as climate change, sustainability, health, and societal well-being. Could artificial intelligence (AI) accelerate science to meet these global challenges in time? AI is already revolutionizing individual scientific disciplines, but we argue here that it could be more holistic and encompassing. We introduce the concept of virtual laboratories as a new perspective on scientific knowledge generation and a means to incentivize new AI research and development. Despite the often perceived domain-specific research practices and inherent tacit knowledge, we argue that many elements of the research process recur across scientific domains and that even common software platforms for serving different domains may be possible. We outline how virtual laboratories will make it easier for AI researchers to contribute to a broad range of scientific domains, and highlight the mutual benefits virtual laboratories offer to both AI and domain scientists.