Model-Based Reinforcement Learning via Stochastic Hybrid Models
Loading...
Access rights
openAccess
URL
Journal Title
Journal ISSN
Volume Title
A1 Alkuperäisartikkeli tieteellisessä aikakauslehdessä
This publication is imported from Aalto University research portal.
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Authors
Date
2023
Major/Subject
Mcode
Degree programme
Language
en
Pages
16
155-170
155-170
Series
IEEE Open Journal of Control Systems, Volume 2
Abstract
Optimal control of general nonlinear systems is a central challenge in automation. Enabled by powerful function approximators, data-driven approaches to control have recently successfully tackled challenging applications. However, such methods often obscure the structure of dynamics and control behind black-box over-parameterized representations, thus limiting our ability to understand closed-loop behavior. This paper adopts a hybrid-system view of nonlinear modeling and control that lends an explicit hierarchical structure to the problem and breaks down complex dynamics into simpler localized units. We consider a sequence modeling paradigm that captures the temporal structure of the data and derive an expectation-maximization (EM) algorithm that automatically decomposes nonlinear dynamics into stochastic piecewise affine models with nonlinear transition boundaries. Furthermore, we show that these time-series models naturally admit a closed-loop extension that we use to extract local polynomial feedback controllers from nonlinear experts via behavioral cloning. Finally, we introduce a novel hybrid relative entropy policy search (Hb-REPS) technique that incorporates the hierarchical nature of hybrid models and optimizes a set of time-invariant piecewise feedback controllers derived from a piecewise polynomial approximation of a global state-value function.Description
Keywords
Hidden Markov models, Switches, Behavioral sciences, Bayes methods, Stochastic processes, Nonlinear dynamical systems, Reinforcement learning
Other note
Citation
Abdulsamad, H & Peters, J 2023, ' Model-Based Reinforcement Learning via Stochastic Hybrid Models ', IEEE Open Journal of Control Systems, vol. 2, 10128705, pp. 155-170 . https://doi.org/10.1109/OJCSYS.2023.3277308