Transferring Generalizable Motor Primitives From Simulation to Real World

Loading...
Thumbnail Image
Journal Title
Journal ISSN
Volume Title
A1 Alkuperäisartikkeli tieteellisessä aikakauslehdessä
This publication is imported from Aalto University research portal.
View publication in the Research portal
View/Open full text file from the Research portal
Date
2019-04
Major/Subject
Mcode
Degree programme
Language
en
Pages
8
2172-2179
Series
IEEE Robotics and Automation Letters, Volume 4, issue 2
Abstract
Reinforcement learning provides robots with an autonomous learning framework where a skill can he learned by exploration. Exploration in real world is, however, inherently unsafe and time consuming, and causes wear and tear. To address these, learning policies in simulation and then transferring them to physical systems has been proposed. In this letter, we propose a novel sample-efficient transfer approach, which is agnostic to the dynamics of a simulated system and combines it with incremental learning. Instead of transferring a single control policy, we transfer a generalizable contextual policy generated in simulation using one or few samples from real world to a target global model, which can generate policies across parameterized real-world situations. We studied the generalization capability of the incremental transfer framework using MuJoCo physics engine and KUKA LBR 4+. Experiments with ball-in-a-cup and basketball tasks demonstrated that the target model improved the generalization capability beyond the direct use of the source model indicating the effectiveness of the proposed framework. Experiments also indicated that the transfer capability depends on the generalization capability of the corresponding source model, similarity between source and target environment, and number of samples used for transferring.
Description
Keywords
Learning and Adaptive Systems, Model Learning for Control, Task analysis, Context modeling, Databases, Uncertainty, Robots, Friction
Other note
Citation
Hazara , M & Kyrki , V 2019 , ' Transferring Generalizable Motor Primitives From Simulation to Real World ' , IEEE Robotics and Automation Letters , vol. 4 , no. 2 , pp. 2172-2179 . https://doi.org/10.1109/LRA.2019.2900768