aalto1 untyped-item.component.html
Statistical methods for sequential decision making and inference
Loading...
URL
Journal Title
Journal ISSN
Volume Title
School of Electrical Engineering |
Doctoral thesis (article-based)
| Defence date: 2026-05-08
Authors
Date
Major/Subject
Mcode
Degree programme
Language
en
Pages
84 + app. 72
Series
Aalto University publication series Doctoral Theses, 106/2026
Abstract
Sequential decision making under uncertainty is a fundamental challenge across engineering and science. In these problems, decision makers must infer hidden states of the world and take actions based on them to maximize some expected utility over a horizon. Finding optimal behavior is notoriously difficult: each decision requires reasoning about (i) the immediate utility obtained, (ii) how the decision’s outcome improves knowledge of the hidden state, and (iii) how this improved state estimate could yield better utility over remaining decisions. Most existing algorithms for sequential decision making rely on restrictive approximations to make this otherwise computationally intractable problem tractable.
This thesis develops a unified inference and learning framework that addresses these challenges by leveraging the connection between stochastic optimal control and probabilistic inference. Using this framework, we build algorithms that reason over future decisions and observations without resorting to common approximations, and amortize this behavior into history dependent policies that can be deployed in real-time systems. We show that different sequential decision problems---namely optimal control of Markov decision processes (MDPs) and partially observed MDPs, and sequential Bayesian experimental design (BED)---can be framed as maximum likelihood estimation in Feynman-Kac models, providing a unified conceptual framework for understanding superficially distinct problems. We then solve these decision problems by introducing nested sequential Monte Carlo (SMC) algorithms that efficiently plan in the space of decisions and observations. The generality of this SMC framework enables applications to nonlinear, non-Gaussian, and non-Markovian settings.
Beyond decision making, we extend this probabilistic inference perspective to numerical computation. We develop a novel Bayesian probabilistic numerical algorithm for solving time dependent nonlinear partial differential equations (PDEs). Collectively, this thesis provides a unified theoretical perspective and a suite of advanced computational tools for optimal decision making and inference in dynamical systems.
Description
Supervising professor
Särkkä, Simo, Prof., Aalto University, Department of Electrical Engineering and Automation, FinlandThesis advisor
Särkkä, Simo, Prof., Aalto University, Department of Electrical Engineering and Automation, FinlandOther note
Parts
- [Publication 1]: Sahel Iqbal, Adrien Corenflos, Simo Särkkä, Hany Abulsamad. Nesting particle filters for experimental design in dynamical systems. In Proceedings of the 41st International Conference on Machine Learning (ICML), Vienna, Austria, Pages 21047–21068, July 2024.
Full text in Acris/Aaltodoc: https://urn.fi/URN:NBN:fi:aalto-202410026606
- [Publication 2]: Hany Abulsamad, Sahel Iqbal, Simo Särkkä. Sequential Monte Carlo for policy optimization in continuous POMDPs. In Proceedings of the 39th Annual Conference on Neural Information Processing Systems (NeurIPS), San Diego, United States of America, Pages 1–22, December 2025.
- [Publication 3]: Sahel Iqbal, Hany Abdulsamad, Tripp Cator, Ulisses Braga-Neto, Simo Särkkä. Parallel-in-time probabilistic solutions for time-dependent nonlinear partial differential equations. In Proceedings of the IEEE International Workshop on Machine Learning for Signal Processing (MLSP), London, United Kingdom, Pages 1–6, September 2024.
Full text in Acris/Aaltodoc: https://urn.fi/URN:NBN:fi:aalto-202411267462DOI: 10.1109/MLSP58920.2024.10734739 View at publisher