Advances in physics-informed deep learning

Thumbnail Image

Files

isbn9789526424408.pdf (10.92 MB)

URL

Journal Title

Journal ISSN

Volume Title

School of Science | Doctoral thesis (article-based) | Defence date: 2025-04-22
Unless otherwise stated, all rights belong to the author. You may download, display and print this publication for Your own personal use. Commercial use is prohibited.

Authors

Date

2025

Department

Tietotekniikan laitos
Department of Computer Science

Major/Subject

Mcode

Degree programme

Language

en

Pages

75 + app. 63

Series

Aalto University publication series Doctoral Theses, 51/2025

Abstract

Accurate models of physical systems play a fundamental role in numerous scientific and industrial fields. Traditional models, grounded in physical laws and expressed through partial differential equations (PDEs), are powerful yet limited. These traditional techniques often exhibit slow inference, rely on oversimplified assumptions, and tend to overlook or underutilize the data gathered from the modeled system. While data-driven approaches like neural networks can overcome these limitations, the data collection is typically too scarce to support purely data-driven methods. This thesis addresses the challenge of modeling physical systems with data-driven techniques when the data from the system is limited and scarce. This research work advances modeling methodologies by embedding prior knowledge of the governing physics into various stages of the modeling procedure. First, the thesis proposes using diffusion-based generative models as probabilistic surrogates to recover unobserved states of physical systems, utilizing PDEs in data generation. In addition to accurate reconstructions, the model is able to produce multiple plausible solutions for non-identifiable systems. Second, the thesis proposes a neural network architecture that mimics the structure of a PDE solver. The proposed architecture meets low data requirements of industrial settings and shows great potential for monitoring real-world chemical reactors. Third, the thesis presents novel algorithms for solving PDEs with neural networks by applying PDE information as a loss term during training. Unlike traditional methods, the developed algorithms incorporate available measurements and yield more accurate solutions with more stable solving procedures. Finally, the thesis introduces an algorithm based on pre-trained large language models to discover the analytical equations that govern the observed data. The results of the thesis demonstrate that neural networks can be effectively applied to physical system modeling with a handful of measurements through effective utilization of prior knowledge. In a broader context, this research opens new opportunities for the successful application of data-driven models in scientific and industrial contexts.

Description

Supervising professor

Marttinen, Pekka, Assoc. Prof., Aalto University, Department of Computer Science, Finland

Thesis advisor

Ilin, Alexander, Dr., Aalto University, Department of Computer Science, Finland, and Chief Scientific Officer, System 2 AI, Finland

Keywords

neural networks, physics-informed neural networks, differential equations, physical system modelling, sample efficient modelling, prior knowledge incorporation, industrial applications of deep learning

Other note

Parts

  • [Publication 1]: Katsiaryna Haitsiukevich, Samuli Bergman, Cesar de Araujo Filho, Francesco Corona, Alexander Ilin. A Grid-Structured Model of Tubular Reactors. In 2021 IEEE 19th International Conference on Industrial Informatics (INDIN), Palma de Mallorca, Spain, pp. 1–6, July 2021.
    Full text in Acris/Aaltodoc: https://urn.fi/URN:NBN:fi:aalto-2021123111074
    DOI: 10.1109/INDIN45523.2021.9557382 View at publisher
  • [Publication 2]: Katsiaryna Haitsiukevich, Alexander Ilin. Learning Trajectories of Hamiltonian Systems with Neural Networks. In Artificial Neural Networks and Machin Learning – ICANN 2022: 31st International Conference on Artificial Neural Networks, Bristol, United Kingdom, pp. 562–573, September 2022.
    Full text in Acris/Aaltodoc: https://urn.fi/URN:NBN:fi:aalto-202401041029
    DOI: 10.1007/978-3-031-15919-0_47 View at publisher
  • [Publication 3]: Katsiaryna Haitsiukevich, Alexander Ilin. Improved Training of Physics-Informed Neural Networks with Model Ensembles. In 2023 International Joint Conference on Neural Networks (IJCNN), Gold Coast, Australia, pp. 1–8, June 2023.
    DOI: 10.1109/IJCNN54540.2023.10191822 View at publisher
  • [Publication 4]: Katsiaryna Haitsiukevich, Onur Poyraz, Pekka Marttinen, Alexander Ilin. Diffusion models as probabilistic neural operators for recovering unobserved states of dynamical systems. In IEEE International Workshop on Machine Learning for Signal Processing (MLSP), London, United Kingdom, September 2024.
    Full text in Acris/Aaltodoc: https://urn.fi/URN:NBN:fi:aalto-202412177888
    DOI: 10.1109/MLSP58920.2024.10734762 View at publisher
  • [Publication 5]: Matteo Merler*, Katsiaryna Haitsiukevich*, Nicola Dainese*, Pekka Marttinen. In-Context Symbolic Regression: Leveraging Large Language Models for Function Discovery. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop), Bangkok, Thailand, pp. 589–606, August 2024.
    DOI: 10.18653/v1/2024.acl-srw.49 View at publisher

Citation

Permanent link to this item

https://urn.fi/URN:ISBN:978-952-64-2440-8

Collections

[diss] Perustieteiden korkeakoulu / SCI