Imitation-guided Multimodal Policy Generation from Behaviourally Diverse Demonstrations
Loading...
Access rights
openAccess
acceptedVersion
URL
Journal Title
Journal ISSN
Volume Title
A4 Artikkeli konferenssijulkaisussa
This publication is imported from Aalto University research portal.
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Date
Major/Subject
Mcode
Degree programme
Language
en
Pages
8
Series
2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1675-1682, Proceedings of the IEEE/RSJ international conference on intelligent robots and systems
Abstract
Learning policies from multiple demonstrators is often difficult because different individuals perform the same task differently due to hidden factors such as preferences. In the context of policy learning, this leads to multimodal policies. Existing policy learning methods often converge to a single solution mode, failing to capture the diversity in the solution space. In this paper, we introduce an imitation-guided reinforcement learning framework to solve the multimodal policy learning problem from a limited number of state-only demonstrations. Then, we propose LfBD (Learning from Behaviourally diverse Demonstration), an algorithm that builds a parameterised solution space to capture the variability in the behaviour space defined by demonstrations. To this end, we define a projection function based on the state density distributions from demonstrations to define such space. Our goal is not only to learn how to solve the task as the human demonstrator but also to extrapolate beyond the provided demonstrations. In addition, we show that with our method, we can perform a post-hoc policy search in the built solution space to recover policies that satisfy specific constraints or to find a policy that matches a given (state-only) behaviour.Description
Other note
Citation
Zhu, S, Kaushik, R, Kaski, S & Kyrki, V 2023, Imitation-guided Multimodal Policy Generation from Behaviourally Diverse Demonstrations. in 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Proceedings of the IEEE/RSJ international conference on intelligent robots and systems, IEEE, pp. 1675-1682, IEEE/RSJ International Conference on Intelligent Robots and Systems, Detroit, Michigan, United States, 01/10/2023. https://doi.org/10.1109/IROS55552.2023.10341403