Continuous-time Model-based Reinforcement Learning

Loading...
Thumbnail Image
Access rights
openAccess
Journal Title
Journal ISSN
Volume Title
Conference article in proceedings
Date
2021-07-21
Major/Subject
Mcode
Degree programme
Language
en
Pages
12009-12018
Series
Proceedings of the 38th International Conference on Machine Learning, ICML 2021, Proceedings of Machine Learning Research, Volume 139
Abstract
Model-based reinforcement learning (MBRL) approaches rely on discrete-time state transition models whereas physical systems and the vast majority of control tasks operate in continuous-time. To avoid time-discretization approximation of the underlying process, we propose a continuous-time MBRL framework based on a novel actor-critic method. Our approach also infers the unknown state evolution differentials with Bayesian neural ordinary differential equations (ODE) to account for epistemic uncertainty. We implement and test our method on a new ODE-RL suite that explicitly solves continuous-time control systems. Our experiments illustrate that the model is robust against irregular and noisy data, and can solve classic control problems in a sample-efficient manner.
Description
Keywords
Other note
Citation
Yildiz , C , Heinonen , M & Lähdesmäki , H 2021 , Continuous-time Model-based Reinforcement Learning . in Proceedings of the 38th International Conference on Machine Learning, ICML 2021 . Proceedings of Machine Learning Research , vol. 139 , JMLR , pp. 12009-12018 , International Conference on Machine Learning , Virtual, Online , 18/07/2021 . < https://proceedings.mlr.press/v139/yildiz21a.html >