Browsing by Author "Solin, Arno"
Now showing 1 - 20 of 66
Results Per Page
Sort Options
Item Aivojen toiminnallisten yhteyksien tutkiminen wavelet-analyysin ja oskillaattorimallin avulla(2013-05-12) Hurme, Max; Solin, Arno; Sähkötekniikan korkeakoulu; Turunen, MarkusItem Applying Efficient Global Optimization on Industrial Use Cases(2020-08-18) Cheng, Yuhao; Frisco, Pierluigi; Maas, Martijn; Perustieteiden korkeakoulu; Solin, ArnoItem Applying machine learning to automatic incident detection from software log output(2019-06-17) Rantala, Aapo; Viitanen, Tuomas; Pitkäranta, Tapio; Sähkötekniikan korkeakoulu; Solin, ArnoIn order to secure a profitable business, stores must have enough products on shelves to offer to customers but shelf life of the products as well as available inventory space should also be considered. Optimizing the product flow from stores to customers is a critical part in supply chain management and is necessary to retain a competitive edge. RELEX Solutions offers a software platform for supply chain management. The software offered by RELEX allows store managers to calculate demand forecasts and order proposals for each product. Calculations in the software produce log files that contain information about the current calculation. Occasionally there are errors in these calculations which are also logged in the log file. Currently a designated support team at RELEX goes through the error messages and decides what actions need to be taken according to the criticality of each error. This thesis investigates the possibility of using a machine learning system to separate critical and non-critical issues. The raw text data in the form of error messages is first preprocessed so it is suitable for a machine learning system. The preprocessing stage includes separating the text data into individual words and filtering out irrelevant terms. Two different feature representations were studied. Selected algorithms for the text classification were support vector machines and a naive Bayes classifier. A systematic testing approach was constructed in order to find the best performing classifier. It was found that the original data set had a strong class imbalance that deteriorated the results. A balanced data set was constructed and the models were able to obtain a better classification performance with this set. However even with a balanced data set the classifier accuracy was only around $60\%$ with both algorithms. This study concluded that the structure of the error messages makes the classification challenging. The error messages do not have enough separating features, they are too messy and the error messages can look similar even if they stem from a different issue. Future research should focus on improving the error messages. In this thesis the numerical data was filtered out but it could be interesting to study the effect of numbers on log classification. Different algorithms should also be researched in the future.Item Bayes-Newton Methods for Approximate Bayesian Inference with PSD Guarantees(MICROTOME PUBL, 2023-03) Wilkinson, William; Särkkä, Simo; Solin, Arno; Department of Computer Science; Department of Electrical Engineering and Automation; Helsinki Institute for Information Technology (HIIT); Sensor Informatics and Medical Technology; Computer Science Professors; Computer Science - Artificial Intelligence and Machine Learning (AIML); Professorship Solin A.We formulate natural gradient variational inference (VI), expectation propagation (EP), and posterior linearisation (PL) as extensions of Newton's method for optimising the parameters of a Bayesian posterior distribution. This viewpoint explicitly casts inference algorithms under the framework of numerical optimisation. We show that common approximations to Newton's method from the optimisation literature, namely Gauss-Newton and quasi-Newton methods (e.g., the BFGS algorithm), are still valid under this 'Bayes-Newton' framework. This leads to a suite of novel algorithms which are guaranteed to result in positive semi-definite (PSD) covariance matrices, unlike standard VI and EP. Our unifying viewpoint provides new insights into the connections between various inference schemes. All the presented methods apply to any model with a Gaussian prior and non-conjugate likelihood, which we demonstrate with (sparse) Gaussian processes and state space models.Item Benchmark of Self-supervised Graph Neural Networks(2022-07-29) Wang, Haishan; Verma, Vikas; Perustieteiden korkeakoulu; Solin, ArnoA graph is an abstract data structure with abundant applications, such as social networks, biochemical molecules, and traffic maps. Graph neural networks (GNNs), deep learning tools which adapt to irregular non-Euclidean space, are designed for such graph data with heavy reliance on manual labels. Learning generalizable and reliable representation for unlabeled graph-structured data has become an attractive and trending task in academia because of the promising application scenarios. Recently, numerous SSL-GNN algorithms have been proposed with success on this task. However, the proposed methods are often evaluated with different architecture and evaluation processes on different small-scale datasets, resulting in unreliable model comparisons. To address this problem, this thesis aims to build a benchmark with a unified framework, a standard evaluation process, and replaceable blocks. In this thesis, a benchmark of SSL-GNNs algorithms is created with the implementation of 9 state-of-art algorithms. These algorithms are compared on this benchmark with consistent settings: shared structure of the GNN encoder, pre-training and fine-tuning scheme, and a unified evaluation protocol. Each model is pre-trained on large-scale datasets: ZINC-15 with two million molecular data and then fine-tuned on eight biophysical downstream datasets for the graph classification task. The experiment results support that two of the nine algorithms outperform others under the benchmark set. Furthermore, the comparison between algorithms also shows the correlation between the pre-training dataset and certain fine-tuning datasets, and the correlation is analyzed by the model mechanisms. The implemented benchmark and discoveries in this thesis are expected to promote transfer learning on graph representation learning.Item Combining pseudo-point and state space approximations for sum-separable Gaussian processes(PMLR, 2021) Tebbutt, Will; Solin, Arno; Turner, Richard E.; Department of Computer Science; Computer Science Professors; Computer Science - Artificial Intelligence and Machine Learning (AIML); Professorship Solin A.; University of CambridgeGaussian processes (GPs) are important probabilistic tools for inference and learning in spatio-temporal modelling problems such as those in climate science and epidemiology. However, existing GP approximations do not simultaneously support large numbers of off-the-grid spatial data-points and long time-series which is a hallmark of many applications. Pseudo-point approximations, one of the gold-standard methods for scaling GPs to large data sets, are well suited for handling off-the-grid spatial data. However, they cannot handle long temporal observation horizons effectively reverting to cubic computational scaling in the time dimension. State space GP approximations are well suited to handling temporal data, if the temporal GP prior admits a Markov form, leading to linear complexity in the number of temporal observations, but have a cubic spatial cost and cannot handle off-the-grid spatial data. In this work we show that there is a simple and elegant way to combine pseudo-point methods with the state space GP approximation framework to get the best of both worlds. The approach hinges on a surprising conditional independence property which applies to space–time separable GPs. We demonstrate empirically that the combined approach is more scalable and applicable to a greater range of spatio-temporal problems than either method on its own.Item Computationally efficient Bayesian learning of Gaussian process state space models(PMLR, 2016) Svensson, Andreas; Solin, Arno; Särkkä, Simo; Schön, Thomas B.; Department of Computer Science; Department of Electrical Engineering and Automation; Professorship Solin A.; Helsinki Institute for Information Technology (HIIT); Sensor Informatics and Medical Technology; Uppsala UniversityGaussian processes allow for flexible specification of prior assumptions of unknown dynamics in state space models. We present a procedure for efficient Bayesian learning in Gaussian process state space models, where the representation is formed by projecting the problem onto a set of approximate eigenfunctions derived from the prior covariance structure. Learning under this family of models can be conducted using a carefully crafted particle MCMC algorithm. This scheme is computationally efficient and yet allows for a fully Bayesian treatment of the problem. Compared to conventional system identification tools or existing learning methods, we show competitive performance and reliable quantification of uncertainties in the model.Item Computationally inferred genealogical networks uncover long-term trends in assortative mating(2018) Malmi, Eric; Gionis, Aristides; Solin, Arno; Department of Computer Science; School services,SCI; Adj. Prof. Gionis Aris group; Professorship Solin A.; Helsinki Institute for Information Technology (HIIT)Genealogical networks, also known as family trees or population pedigrees, are commonly studied by genealogists wanting to know about their ancestry, but they also provide a valuable resource for disciplines such as digital demography, genetics, and computational social science. These networks are typically constructed by hand through a very time-consuming process, which requires comparing large numbers of historical records manually. We develop computational methods for automatically inferring large-scale genealogical networks. A comparison with human-constructed networks attests to the accuracy of the proposed methods. To demonstrate the applicability of the inferred large-scale genealogical networks, we present a longitudinal analysis on the mating patterns observed in a network. This analysis shows a consistent tendency of people choosing a spouse with a similar socioeconomic status, a phenomenon known as assortative mating. Interestingly, we do not observe this tendency to consistently decrease (nor increase) over our study period of 150 years.Item Cubature Integration Methods in Non-Linear Kalman Filtering and Smoothing(2010) Solin, Arno; Särkkä, Simo; Informaatio- ja luonnontieteiden tiedekunta; Ehtamo, HarriItem Data-efficient meta-learning with Bayesian deep learning(2020-12-14) Irwanto, Christabella; Solin, Arno; Perustieteiden korkeakoulu; Solin, ArnoAs machine learning is increasingly used in real-world systems, two key methods for function approximation have gained traction: deep learning with neural networks, and Bayesian inference with Gaussian processes (GPs). The two methods have distinct but complementary properties, and Neural Processes (NPs) have emerged as a desirable approach that combines the merits of both. NPs naturally fall within the meta-learning regime, which is loosely inspired by how humans can learn new tasks from just a few examples, leveraging partially related tasks. As such, we compare NPs to a simple extension of GPs to the meta-learning setting ("Meta-GP"), augmenting the GPs with neural networks via an existing framework known as "deep kernel learning" or "deep mean learning". Using a few regression benchmarks from both synthetic and real-world one-dimensional data, the Meta-GP outperforms NP in both accuracy and uncertainty calibration. Whether deep kernel or mean learning for the GP is more appropriate, is shown to depend on whether the alternative handcrafted kernel or mean is unsuitable for the data, as expected. Future work may lie in more rigorous dataset benchmarks, as well as the inclusion of more baseline methods in time series forecasting. It also may be worth considering when it is desirable to forego a valid stochastic process in place of an approximate Neural Process, such as when there is more data, and why a valid stochastic process is desirable for applications.Item Databehandling av enskilda cellers RNA-sekvensering med hjälp av neurala nätverk(2020-11-30) Aspelin, Arthur; Solin, Arno; Sähkötekniikan korkeakoulu; Turunen, MarkusItem Deep Learning & Graph Clustering for Maritime Logistics: Predicting Destination and Expected Time of Arrival for Vessels Across Europe(2020-08-18) Orgaz Expósito, Álvaro; Poikonen, Jussi; Perustieteiden korkeakoulu; Solin, ArnoIn recent years, the need for improving operational processes internationally has drastically increased in the maritime logistics field. The lack of streamlined systems that provide reliable information about real-time maritime traffic for the main agents across countries, such as ports operators and ships authorities, has prompted several research questions. In this work, we propose Deep learning and Machine Learning based methods for (i) clustering ports across Europe using their maritime traffic connectivity, (ii) predicting the next destination of vessels, and (iii) forecasting their expected voyage duration. Several experiments based on public AIS data are developed to analyse and verify these methods, and the results of these experiments indicate that the proposed models achieve the state-of-the-art predictive performance considering the wide geographical scope of the problem across all over Europe. Furthermore, a big advantage of the proposed methods respect to other solutions is that the input data configuration and the intrinsic nature of the models enable the users to predict the aforementioned targets about the next destination of vessels right after they arrive at any European port, instead of waiting for the information given by the first submitted AIS messages once their corresponding next voyage has started. When deployed into production, the resulting system will help maritime industry agents to enhance their real-time situational awareness and operational planning.Item Deep learning based speed estimation for constraining strapdown inertial navigation on smartphones(2018) Cortes Reina, Santiago; Solin, Arno; Kannala, Juho; Department of Computer Science; Professorship Kannala Juho; Professorship Solin A.Strapdown inertial navigation systems are sensitive to the quality of the data provided by the accelerometer and gyroscope. Low-grade IMUs in handheld smart-devices pose a problem for inertial odometry on these devices. We propose a scheme for constraining the inertial odometry problem by complementing non-linear state estimation by a CNN-based deep-learning model for inferring the momentary speed based on a window of IMU samples. We show the feasibility of the model using a wide range of data from an iPhone, and present proof-of-concept results for how the model can be combined with an inertial navigation system for three-dimensional inertial navigation.Item Deep Learning for Error Modeling of Tractor-semitrailer Dynamics Model(2022-10-17) Teng, Rui; Takkoush, Mohamed; Helfrich, Thorsten; Perustieteiden korkeakoulu; Solin, ArnoIn the field of autonomous driving, vehicle models are the basis for a variety of research. Vehicle models provide simulated data describing trajectories and dynamics of vehicles, and the property that the utility of vehicle models is independent of real vehicles makes it possible to obtain a large amount of data quickly. Efforts have been made surrounding the construction and validation of vehicle models. However, currently, vehicle models could produce simulation errors under certain circumstances such as extreme speed driving and complex steering. Our solution to the simulation error problem is to predict the error and combine vehicle model simulation and error prediction. In this paper, we propose to apply deep learning to build point prediction error models and uncertainty-aware error models. Valid error models that pass validation tests are expected to offset residual between vehicle models and real systems, which allows using of non-accurate vehicle models. Besides, error prediction is useful to collect cases that have high simulation errors, which are important to analyze and fix vehicle models. The above methods are applied to a tractor-semitrailer dynamics model based on physical principles, provided by Volvo Autonomous Solutions. Involved data is collected from the corresponding tractor-semitrailer, under various driving contexts, and over diverse manoeuvres. Statistics-based evaluation results show that deep learning is potential in vehicle model error modeling, although the uncertainty-aware error model trained for the tractor-semitrailer fails in the validation testItem Deep Learning with Uncertainty Quantification for Emitter Classification(2020-01-20) Meronen, Lassi; Holter, Henrik; Perustieteiden korkeakoulu; Solin, ArnoEmitter classification is a crucial method of target recognition in electronic warfare. The recent popularity and success of deep learning models have made them an appealing method to be applied to emitter classification. However, the lack of reliable uncertainty estimates for deep learning models risks making overconfident incorrect decisions with irreversible consequences. The goal of this thesis is to compare different uncertainty estimation methods for deep learning models, applied to emitter classification tasks. The compared methods are Monte Carlo dropout, deep ensembles, Bayesian neural networks and noise contrastive priors. The neural network structure for all models is a convolutional residual neural network and the emitter classification data is generated through simulations. The results show that uncertainty estimates of Monte Carlo dropout, deep ensembles and Bayesian neural networks mainly differ from each other near decision boundaries and all these methods provide overconfident predictions outside of the training distribution. Changing the activation function of the last layer before the classification layer in the neural network, from a rectifier to a pulsed activation function, significantly helps to improve uncertainty estimates for samples outside of the training data distribution. Using noise contrastive priors further helps to increase the uncertainty estimates near the border of the training distribution, especially when used together with the pulsed activation function. The best overall uncertainty estimates were achieved using deep ensembles with the pulsed activation function and added noise contrastive priors. These methods are not limited to emitter classification and similar results can be expected in other applications.Item Deep learning-based doctor recommendations for public staffing emergency shifts(2020-10-20) Gardemeister, Joni; Niemenoja, Oskar; Perustieteiden korkeakoulu; Solin, ArnoItem Depth Estimation from Images using Convolutional Neural Networks(2019-05-13) Ylipiha, Tuomas; Solin, Arno; Perustieteiden korkeakoulu; Hyvönen, EeroItem Direct Methods in Visual Odometry(2019-01-16) Eriksson, Mathias; Solin, Arno; Perustieteiden korkeakoulu; Eero HyvönenItem Dual parameterization of sparse variational Gaussian processes(Morgan Kaufmann Publishers, 2021) Adam, Vincent; Chang, Paul; Khan, Mohammad Emtiyaz; Solin, Arno; Department of Computer Science; Professorship Solin A.; Computer Science Professors; Computer Science - Artificial Intelligence and Machine Learning (AIML); RIKEN Center for Advanced Intelligence ProjectSparse variational Gaussian process (SVGP) methods are a common choice for non-conjugate Gaussian process inference because of their computational benefits. In this paper, we improve their computational efficiency by using a dual parameterization where each data example is assigned dual parameters, similarly to site parameters used in expectation propagation. Our dual parameterization speeds-up inference using natural gradient descent, and provides a tighter evidence lower bound for hyperparameter learning. The approach has the same memory cost as the current SVGP methods, but it is faster and more accurate.Item Dynamic retrospective filtering of physiological noise in BOLD fMRI: DRIFTER(Elsevier BV, 2012) Särkkä, Simo; Solin, Arno; Nummenmaa, Aapo; Vehtari, Aki; Auranen, Toni; Vanni, Simo; Lin, Fa-Hsuan; Neurotieteen ja lääketieteellisen tekniikan laitos; Department of Neuroscience and Biomedical Engineering; Tietotekniikan laitos; Department of Computer Science; Sähkötekniikan ja automaation laitos; Department of Electrical Engineering and Automation; Perustieteiden korkeakoulu; School of Science; Sähkötekniikan korkeakoulu; School of Electrical EngineeringIn this article we introduce the DRIFTER algorithm, which is a new model based Bayesian method for retrospective elimination of physiological noise from functional magnetic resonance imaging (fMRI) data. In the method, we first estimate the frequency trajectories of the physiological signals with the interacting multiple models (IMM) filter algorithm. The frequency trajectories can be estimated from external reference signals, or if the temporal resolution is high enough, from the fMRI data. The estimated frequency trajectories are then used in a state space model in combination of a Kalman filter (KF) and Rauch–Tung–Striebel (RTS) smoother, which separates the signal into an activation related cleaned signal, physiological noise, and white measurement noise components. Using experimental data, we show that the method outperforms the RETROICOR algorithm if the shape and amplitude of the physiological signals change over time.