Browsing by Author "Heinonen, Markus"

Filter results by typing the first few letters
Now showing 1 - 20 of 52
  • AbODE: Ab initio antibody design using conjoined ODEs
    (2023-07) Verma, Yogesh; Heinonen, Markus; Garg, Vikas
     A4 Artikkeli konferenssijulkaisussa
    Antibodies are Y-shaped proteins that neutralize pathogens and constitute the core of our adaptive immune system. De novo generation of new antibodies that target specific antigens holds the key to accelerating vaccine discovery. However, this co-design of the amino acid sequence and the 3D structure subsumes and accentuates, some central challenges from multiple tasks including protein folding (sequence to structure), inverse folding (structure to sequence), and docking (binding). We strive to surmount these challenges with a new generative model AbODE that extends graph PDEs to accommodate both contextual information and external interactions. Unlike existing approaches, AbODE uses a single round of full-shot decoding, and elicits continuous differential attention that encapsulates, and evolves with, latent interactions within the antibody as well as those involving the antigen. We unravel fundamental connections between AbODE and temporal networks as well as graph-matching networks. The proposed model significantly outperforms existing methods on standard metrics across benchmarks.
  • Balancing Imbalanced Toxicity Models : Using MolBERT with Focal Loss
    (2025) Masood, Muhammad Arslan; Kaski, Samuel; Ceulemans, Hugo; Herman, Dorota; Heinonen, Markus
     A4 Artikkeli konferenssijulkaisussa
    Drug-induced liver injury (DILI) presents a multifaceted challenge, influenced by interconnected biological mechanisms. Current DILI datasets are characterized by small sizes and high imbalance, posing difficulties in learning robust representations and accurate modeling. To address these challenges, we trained a multi-modal multi-task model integrating preclinical histopathologies, biochemistry (blood markers), and clinical DILI-related adverse drug reactions (ADRs). Leveraging pretrained BERT models, we extracted representations covering a broad chemical space, facilitating robust learning in both frozen and fine-tuned settings. To address imbalanced data, we explored weighted Binary Cross-Entropy (w-BCE) and weighted Focal Loss (w-FL) . Our results demonstrate that the frozen BERT model consistently enhances performance across all metrics and modalities with weighted loss functions compared to their non-weighted counterparts. However, the efficacy of fine-tuning BERT varies across modalities, yielding inconclusive results. In summary, the incorporation of BERT features with weighted loss functions demonstrates advantages, while the efficacy of fine-tuning remains uncertain.
  • Bayesian Inference for Optimal Transport with Stochastic Cost
    (2021) Mallasto, Anton; Heinonen, Markus; Kaski, Samuel
     A4 Artikkeli konferenssijulkaisussa
    In machine learning and computer vision, optimal transport has had significant success in learning generative models and defining metric distances between structured and stochastic data objects, that can be cast as probability measures. The key element of optimal transport is the so called lifting of an exact cost (distance) function, defined on the sample space, to a cost (distance) between probability measures over the sample space. However, in many real life applications the cost is stochastic: e.g., the unpredictable traffic flow affects the cost of transportation between a factory and an outlet. To take this stochasticity into account, we introduce a Bayesian framework for inferring the optimal transport plan distribution induced by the stochastic cost, allowing for a principled way to include prior information and to model the induced stochasticity on the transport plans. Additionally, we tailor an HMC method to sample from the resulting transport plan posterior distribution.
  • Bayesian metabolic flux analysis reveals intracellular flux couplings
    (2019-07-15) Heinonen, Markus; Osmala, Maria; Mannerström, Henrik; Wallenius, Janne; Kaski, Samuel; Rousu, Juho; Lähdesmäki, Harri
     A1 Alkuperäisartikkeli tieteellisessä aikakauslehdessä
    Motivation: Metabolic flux balance analysis (FBA) is a standard tool in analyzing metabolic reaction rates compatible with measurements, steady-state and the metabolic reaction network stoichiometry. Flux analysis methods commonly place model assumptions on fluxes due to the convenience of formulating the problem as a linear programing model, while many methods do not consider the inherent uncertainty in flux estimates. Results: We introduce a novel paradigm of Bayesian metabolic flux analysis that models the reactions of the whole genome-scale cellular system in probabilistic terms, and can infer the full flux vector distribution of genome-scale metabolic systems based on exchange and intracellular (e.g. 13C) flux measurements, steady-state assumptions, and objective function assumptions. The Bayesian model couples all fluxes jointly together in a simple truncated multivariate posterior distribution, which reveals informative flux couplings. Our model is a plug-in replacement to conventional metabolic balance methods, such as FBA. Our experiments indicate that we can characterize the genome-scale flux covariances, reveal flux couplings, and determine more intracellular unobserved fluxes in Clostridium acetobutylicum from 13C data than flux variability analysis.
  • Puukerrostalokortteli Tampereen Isokuuseen - puujulkisivuverhoukset asuinkerrostalojen arkkitehtuurissa
    (2013) Heinonen, Markus
     School of Arts, Design and Architecture | Master's thesis
    In Finland, the need to seek sustainable architectural solutions has motivated renewed interest in the use of wood as a building material. However, building long-term structures with wood poses special challenges in environments experiencing extreme climatic variation, such as the Nordic countries. As part of a collective town planning project organized by the city of Tampere, this thesis sought to develop design principles for a cluster of multi-storey, wooden apartment blocks to be located in the new Isokuusi district of Tampere. The project area was based on the general plan for Isokuusi drawn up by the B&M architectural office. The buildings had an overall floor area of 7450 m2, which in addition to the apartments included 500 m2 of retail and work spaces as well as 300 m2 of hobby rooms, clubrooms and saunas for common use. The buildings were massed to take advantage of the compass orientation as well as the surrounding urban scene, and were designed to stand 3-5 stories on the northern edge and 2-3 stories on the southern edge of the plot. The only exception was a building of eight stories with one stairwell located at the western corner of the plot. The roofs tilt systematically towards the inner courtyard, thus enabling the apartments to gain exposure sunlight. In addition, the storm waters can be infiltrated within the plot. The proposed housing design provides pleasant entrances with pass-through staircases. Emphasized with red ochre paint, the retractable entrances divide the building masses into smaller parts and enable light to pass from two directions into the small adjacent apartments. Common-use balconies are located beside the entrances on each storey. The timber-framed walls are designed as elements. The intermediate floors are made of either ribbed slabs or, in the case of wet spaces or staircases, short spans of cross-laminated timber (CLT). To guide the design of wooden facades, a literature review was carried out. The long-term durability of wooden facades can be promoted by structural protection (eaves, overhangs, canopies and plinths), coatings, regular maintenance, using thick timber (at least 28 mm) and heat treating the wood. The carbon emissions during the life cycle of wooden claddings are mostly affected by the choice of the coating system. In addition the design principles of four existing wooden facades were examined and their functionality was evaluated based on their present state. Calculations based on weather statistics were made to study the intensity of weather stresses for different compass directions. In order to ensure that all facades would weather at the same rate, the southern facades, subjected to higher weather stresses, were designed to be heat-treated wood, oriented vertically and coated with light colours. The facades were designed to be individual elements that are separately attached onto exterior wall elements, thus allowing the layout of the seam lines to vary freely and the elements to be removed for maintenance. Wooden facades last long when they are designed well, made carefully and maintained with regularity.
  • Building and Managing Go-To-Market Partnerships in New Technology Ventures: A Case Study of Virtual Reality headsets
    (2018-10-03) Jokinen, Miika
     Perustieteiden korkeakoulu | Master's thesis
  • Chemistry-Based Modeling on Phenotype-Based Drug-Induced Liver Injury Annotation : From Public to Proprietary Data
    (2023-08-21) Moein, Mohammad; Heinonen, Markus; Mesens, Natalie; Chamanza, Ronnie; Amuzie, Chidozie; Will, Yvonne; Ceulemans, Hugo; Kaski, Samuel; Herman, Dorota
     A1 Alkuperäisartikkeli tieteellisessä aikakauslehdessä
    Drug-induced liver injury (DILI) is an important safety concern and a major reason to remove a drug from the market. Advancements in recent machine learning methods have led to a wide range of in silico models for DILI predictive methods based on molecule chemical structures (fingerprints). Existing publicly available DILI data sets used for model building are based on the interpretation of drug labels or patient case reports, resulting in a typical binary clinical DILI annotation. We developed a novel phenotype-based annotation to process hepatotoxicity information extracted from repeated dose in vivo preclinical toxicology studies using INHAND annotation to provide a more informative and reliable data set for machine learning algorithms. This work resulted in a data set of 430 unique compounds covering diverse liver pathology findings which were utilized to develop multiple DILI prediction models trained on the publicly available data (TG-GATEs) using the compound’s fingerprint. We demonstrate that the TG-GATEs compounds DILI labels can be predicted well and how the differences between TG-GATEs and the external test compounds (Johnson & Johnson) impact the model generalization performance.
  • Continuous-time Model-based Reinforcement Learning
    (2021-07-21) Yildiz, Cagatay; Heinonen, Markus; Lähdesmäki, Harri
     A4 Artikkeli konferenssijulkaisussa
    Model-based reinforcement learning (MBRL) approaches rely on discrete-time state transition models whereas physical systems and the vast majority of control tasks operate in continuous-time. To avoid time-discretization approximation of the underlying process, we propose a continuous-time MBRL framework based on a novel actor-critic method. Our approach also infers the unknown state evolution differentials with Bayesian neural ordinary differential equations (ODE) to account for epistemic uncertainty. We implement and test our method on a new ODE-RL suite that explicitly solves continuous-time control systems. Our experiments illustrate that the model is robust against irregular and noisy data, and can solve classic control problems in a sample-efficient manner.
  • De-randomizing MCMC dynamics with the diffusion Stein operator
    (2021) Shen, Zheyang; Heinonen, Markus; Kaski, Samuel
     A4 Artikkeli konferenssijulkaisussa
    Approximate Bayesian inference estimates descriptors of an intractable target distribution - in essence, an optimization problem within a family of distributions. For example, Langevin dynamics (LD) extracts asymptotically exact samples from a diffusion process because the time evolution of its marginal distributions constitutes a curve that minimizes the KL-divergence via steepest descent in the Wasserstein space. Parallel to LD, Stein variational gradient descent (SVGD) similarly minimizes the KL, albeit endowed with a novel Stein-Wasserstein distance, by deterministically transporting a set of particle samples, thus de-randomizes the stochastic diffusion process. We propose de-randomized kernel-based particle samplers to all diffusion-based samplers known as MCMC dynamics. Following previous work in interpreting MCMC dynamics, we equip the Stein-Wasserstein space with a fiber-Riemannian Poisson structure, with the capacity of characterizing a fiber-gradient Hamiltonian flow that simulates MCMC dynamics. Such dynamics discretizes into generalized SVGD (GSVGD), a Stein-type deterministic particle sampler, with particle updates coinciding with applying the diffusion Stein operator to a kernel function. We demonstrate empirically that GSVGD can de-randomize complex MCMC dynamics, which combine the advantages of auxiliary momentum variables and Riemannian structure, while maintaining the high sample quality from an interacting particle system.
  • Deep convolutional Gaussian processes
    (2019-03-11) Blomqvist, Kenneth
     Perustieteiden korkeakoulu | Master's thesis
    Convolutional neural networks have achieved unparalleled results on various machine learning tasks such as image classification, speech recognition, image segmentation, machine translation and many others. Modern neural network architectures have millions of parameters. This makes them prone to overfitting and sensitive to out-of-sample noise. As they are relatively practical to train, these issues can often be counteracted using massive amounts of training data. They have also been found to be prone to adversarial attacks. Developing methods which are well-regularized and could learn complicated functions without using massive amounts of data could enable us to deploy machine learning methods in settings where heaps of data are not available. Gaussian processes are known as a well-regularized statistical method which works beautifully for simple regression and classification tasks with a small number of training examples. Achieving such properties in deep models would be greatly beneficial. In this thesis we develop a deep Gaussian process model with convolutional structure which we call the deep convolutional Gaussian process. It is a method for modelling hierarchical combination of local features using Gaussian process mappings structured in a hierarchical manner. We compare our method on the MNIST and CIFAR-10 image classification tasks against other successful approaches. On the CIFAR-10 dataset, we achieve a more than 10\% improvement in test classification accuracy over other Gaussian process based methods.
  • Deep learning with differential Gaussian process flows
    (2019-04) Hegde, Pashupati; Heinonen, Markus; Lähdesmäki, Harri; Kaski, Samuel
     A4 Artikkeli konferenssijulkaisussa
    We propose a novel deep learning paradigm of differential flows that learn a stochastic differential equation transformations of inputs prior to a standard classification or regression function. The key property of differential Gaussian processes is the warping of inputs through infinitely deep, but infinitesimal, differential fields, that generalise discrete layers into a dynamical system. We demonstrate state-of-the-art results that exceed the performance of deep Gaussian processes and neural networks
  • Differentially deep Gaussian processes
    (2019-01-28) Hegde, Pashupati
     Perustieteiden korkeakoulu | Master's thesis
    Many modern machine learning methods, including deep neural networks, utilize a discrete sequence of parametric transformations to learn complex functions. Neural network based approaches can be an attractive choice for many real-world problems especially because of their modular nature. Gaussian process based methods, on the other hand, pose function approximation as a probabilistic inference problem by specifying prior distributions on unknown functions. Further, these probabilistic non-linear models provide well-calibrated uncertainty estimates which can be useful in many applications. However, the flexibility of these models depends on the choice of the kernel; handcrafting problem-specific kernels can be difficult in practice. Recently, deep Gaussian processes, a way of stacking multiple layers of Gaussian processes, was proposed as a flexible way of expanding model capacity. In this thesis, we propose a novel probabilistic deep learning approach by formulating stochastic differential transformations or `flows' of inputs using Gaussian processes. This provides continuous-time `flows' as an alternative to the traditional approach of a discrete sequence of transformations using `layers'. Moreover, the proposed approach can also be seen as an approximation to very deep Gaussian processes with infinitesimal increments across layers. We also derive a scalable inference method based on variational sparse approximations for Gaussian processes. The proposed model shows excellent results on various experiments on real-world datasets, as compared to the other popular probabilistic approaches including deep Gaussian processes and Bayesian neural networks.
  • Elämänkaariasumisen vaikutus paikallisyhteisöön ja asuntomarkkinoihin
    (2008) Heinonen, Markus
     Insinööritieteiden ja arkkitehtuurin tiedekunta | Bachelor's thesis
  • Evolving-Graph Gaussian Processes
    (2021-07) Blanco-Mulero, David; Heinonen, Markus; Kyrki, Ville
     A4 Artikkeli konferenssijulkaisussa
    Graph Gaussian Processes (GGPs) provide a dataefficient solution on graph structured domains. Existing approaches have focused on static structures, whereas many real graph data represent a dynamic structure, limiting the applications of GGPs. To overcome this we propose evolvingGraph Gaussian Processes (e-GGPs). The proposed method is capable of learning the transition function of graph vertices over time with a neighbourhood kernel to model the connectivity and interaction changes between vertices. We assess the performance of our method on time-series regression problems where graphs evolve over time. We demonstrate the benefits of e-GGPs over static graph Gaussian Process approaches.
  • Genome wide analysis of protein production load in Trichoderma reesei
    (2016-06-28) Pakula, Tiina M.; Nygren, Heli; Barth, Dorothee; Heinonen, Markus; Castillo, Sandra; Penttilä, Merja; Arvas, Mikko
     A1 Alkuperäisartikkeli tieteellisessä aikakauslehdessä
    Background: The filamentous fungus Trichoderma reesei (teleomorph Hypocrea jecorina) is a widely used industrial host organism for protein production. In industrial cultivations, it can produce over 100 g/l of extracellular protein, mostly constituting of cellulases and hemicellulases. In order to improve protein production of T. reesei the transcriptional regulation of cellulases and secretory pathway factors have been extensively studied. However, the metabolism of T. reesei under protein production conditions has not received much attention. Results: To understand the physiology and metabolism of T. reesei under protein production conditions we carried out a well-controlled bioreactor experiment with extensive analysis. We used minimal media to make the data amenable for modelling and three strain pairs to cover different protein production levels. With RNA-sequencing transcriptomics we detected the concentration of the carbon source as the most important determinant of the transcriptome. As the major transcriptional response concomitant to protein production we detected the induction of selected genes that were putatively regulated by xyr1 and were related to protein transport, amino acid metabolism and transcriptional regulation. We found novel metabolic responses such as production of glycerol and a cellotriose-like compound. We then used this cultivation data for flux balance analysis of T. reesei metabolism and demonstrate for the first time the use of genome wide stoichiometric metabolic modelling for T. reesei. We show that our model can predict protein production rate and provides novel insight into the metabolism of protein production. We also provide this unprecedented cultivation and transcriptomics data set for future modelling efforts. Conclusions: The use of stoichiometric modelling can open a novel path for the improvement of protein production in T. reesei. Based on this we propose sulphur assimilation as a major limiting factor of protein production. As an organism with exceptional protein production capabilities modelling of T. reesei can provide novel insight also to other less productive organisms.
  • Harmonizable mixture kernels with variational Fourier features
    (2019-05) Shen, Zheyang; Heinonen, Markus; Kaski, Samuel
     A4 Artikkeli konferenssijulkaisussa
    The expressive power of Gaussian processes depends heavily on the choice of kernel. In this work we propose the novel harmonizable mixture kernel (HMK), a family of expressive, interpretable, non-stationary kernels derived from mixture models on the generalized spectral representation. As a theoretically sound treatment of non-stationary kernels, HMK supports harmonizable covariances, a wide subset of kernels including all stationary and many non-stationary covariances. We also propose variational Fourier features, an inter-domain sparse GP inference framework that offers a representative set of 'inducing frequencies'. We show that harmonizable mixture kernels interpolate between local patterns, and that variational Fourier features offers a robust kernel learning framework for the new kernel family.
  • Human-in-the-loop active learning for goal-oriented molecule generation
    (2024-12-09) Nahal, Yasmine; Menke, Janosch; Martinelli, Julien; Heinonen, Markus; Kabeshov, Mikhail; Janet, Jon Paul; Nittinger, Eva; Engkvist, Ola; Kaski, Samuel
     A1 Alkuperäisartikkeli tieteellisessä aikakauslehdessä
    Machine learning (ML) systems have enabled the modelling of quantitative structure–property relationships (QSPR) and structure-activity relationships (QSAR) using existing experimental data to predict target properties for new molecules. These property predictors hold significant potential in accelerating drug discovery by guiding generative artificial intelligence (AI) agents to explore desired chemical spaces. However, they often struggle to generalize due to the limited scope of the training data. When optimized by generative agents, this limitation can result in the generation of molecules with artificially high predicted probabilities of satisfying target properties, which subsequently fail experimental validation. To address this challenge, we propose an adaptive approach that integrates active learning (AL) and iterative feedback to refine property predictors, thereby improving the outcomes of their optimization by generative AI agents. Our method leverages the Expected Predictive Information Gain (EPIG) criterion to select additional molecules for evaluation by an oracle. This process aims to provide the greatest reduction in predictive uncertainty, enabling more accurate model evaluations of subsequently generated molecules. Recognizing the impracticality of immediate wet-lab or physics-based experiments due to time and logistical constraints, we propose leveraging human experts for their cost-effectiveness and domain knowledge to effectively augment property predictors, bridging gaps in the limited training data. Empirical evaluations through both simulated and real humanin-the-loop experiments demonstrate that our approach refines property predictors to better align with oracle assessments. Additionally, we observe improved accuracy of predicted properties as well as improved drug-likeness among the top-ranking generated molecules.
  • Human-in-the-loop assisted de novo molecular design
    (2022-12) Sundin, Iiris; Voronov, Alexey; Xiao, Haoping; Papadopoulos, Kostas; Bjerrum, Esben Jannik; Heinonen, Markus; Patronov, Atanas; Kaski, Samuel; Engkvist, Ola
     A1 Alkuperäisartikkeli tieteellisessä aikakauslehdessä
    A de novo molecular design workflow can be used together with technologies such as reinforcement learning to navigate the chemical space. A bottleneck in the workflow that remains to be solved is how to integrate human feedback in the exploration of the chemical space to optimize molecules. A human drug designer still needs to design the goal, expressed as a scoring function for the molecules that captures the designer’s implicit knowledge about the optimization task. Little support for this task exists and, consequently, a chemist usually resorts to iteratively building the objective function of multi-parameter optimization (MPO) in de novo design. We propose a principled approach to use human-in-the-loop machine learning to help the chemist to adapt the MPO scoring function to better match their goal. An advantage is that the method can learn the scoring function directly from the user’s feedback while they browse the output of the molecule generator, instead of the current manual tuning of the scoring function with trial and error. The proposed method uses a probabilistic model that captures the user’s idea and uncertainty about the scoring function, and it uses active learning to interact with the user. We present two case studies for this: In the first use-case, the parameters of an MPO are learned, and in the second use-case a non-parametric component of the scoring function to capture human domain knowledge is developed. The results show the effectiveness of the methods in two simulated example cases with an oracle, achieving significant improvement in less than 200 feedback queries, for the goals of a high QED score and identifying potent molecules for the DRD2 receptor, respectively. We further demonstrate the performance gains with a medicinal chemist interacting with the system. Graphical Abstract: [Figure not available: see fulltext.].
  • Incorporating functional summary information in Bayesian neural networks using a Dirichlet process likelihood approach
    (2023) Raj, Vishnu; Cui, Tianyu; Heinonen, Markus; Marttinen, Pekka
     A4 Artikkeli konferenssijulkaisussa
    Bayesian neural networks (BNNs) can account for both aleatoric and epistemic uncertainty. However, in BNNs the priors are often specified over the weights which rarely reflects true prior knowledge in large and complex neural network architectures. We present a simple approach to incorporate prior knowledge in BNNs based on external summary information about the predicted classification probabilities for a given dataset. The available summary information is incorporated as augmented data and modeled with a Dirichlet process, and we derive the corresponding Summary Evidence Lower BOund. The approach is founded on Bayesian principles, and all hyperparameters have a proper probabilistic interpretation. We show how the method can inform the model about task difficulty and class imbalance. Extensive experiments show that, with negligible computational overhead, our method parallels and in many cases outperforms popular alternatives in accuracy, uncertainty calibration, and robustness against corruptions with both balanced and imbalanced data.
  • Inferenssimenetelmät Bayesilaisissa syväoppimismalleissa
    (2021-12-19) Rotko, Samu
     Sähkötekniikan korkeakoulu | Bachelor's thesis