Continuous-time Model-based Reinforcement Learning

Proceedings of the 38th International Conference on Machine Learning, ICML 2021, pp. 12009-12018, Proceedings of Machine Learning Research ; Volume 139

Abstract

Model-based reinforcement learning (MBRL) approaches rely on discrete-time state transition models whereas physical systems and the vast majority of control tasks operate in continuous-time. To avoid time-discretization approximation of the underlying process, we propose a continuous-time MBRL framework based on a novel actor-critic method. Our approach also infers the unknown state evolution differentials with Bayesian neural ordinary differential equations (ODE) to account for epistemic uncertainty. We implement and test our method on a new ODE-RL suite that explicitly solves continuous-time control systems. Our experiments illustrate that the model is robust against irregular and noisy data, and can solve classic control problems in a sample-efficient manner.

Citation

Yildiz, C, Heinonen, M & Lähdesmäki, H 2021, Continuous-time Model-based Reinforcement Learning . in Proceedings of the 38th International Conference on Machine Learning, ICML 2021 . Proceedings of Machine Learning Research, vol. 139, JMLR, pp. 12009-12018, International Conference on Machine Learning, Virtual, Online, 18/07/2021 . < https://proceedings.mlr.press/v139/yildiz21a.html >

Permanent link to this item

https://urn.fi/URN:NBN:fi:aalto-202108258419

Collections

[article-cris] Perustieteiden korkeakoulu / SCI

Show all metadata

Continuous-time Model-based Reinforcement Learning

Access rights

URL

Journal Title

Journal ISSN

Volume Title

Authors

Date

Department

Major/Subject

Mcode

Degree programme

Language

Pages

Series

Abstract

Description

Keywords

Other note

Citation

Permanent link to this item

Collections