DROPO: Sim-to-real transfer with offline domain randomization
Loading...
Access rights
openAccess
Journal Title
Journal ISSN
Volume Title
A1 Alkuperäisartikkeli tieteellisessä aikakauslehdessä
This publication is imported from Aalto University research portal.
View publication in the Research portal
View/Open full text file from the Research portal
Other link related to publication
View publication in the Research portal
View/Open full text file from the Research portal
Other link related to publication
Date
2023-08
Major/Subject
Mcode
Degree programme
Language
en
Pages
15
Series
Robotics and Autonomous Systems, Volume 166
Abstract
In recent years, domain randomization over dynamics parameters has gained a lot of traction as a method for sim-to-real transfer of reinforcement learning policies in robotic manipulation; however, finding optimal randomization distributions can be difficult. In this paper, we introduce DROPO, a novel method for estimating domain randomization distributions for safe sim-to-real transfer. Unlike prior work, DROPO only requires a limited, precollected offline dataset of trajectories, and explicitly models parameter uncertainty to match real data using a likelihood-based approach. We demonstrate that DROPO is capable of recovering dynamic parameter distributions in simulation and finding a distribution capable of compensating for an unmodeled phenomenon. We also evaluate the method in two zero-shot sim-to-real transfer scenarios, showing successful domain transfer and improved performance over prior methods.Description
Funding Information: This work was supported by Academy of Finland grants 317020 and 328399 . We acknowledge the computational resources generously provided by CSC – IT Center for Science, Finland, and by the Aalto Science-IT project. Publisher Copyright: © 2023 The Author(s)
Keywords
Domain randomization, Reinforcement learning, Robot learning, Transfer learning
Other note
Citation
Tiboni, G, Arndt, K & Kyrki, V 2023, ' DROPO: Sim-to-real transfer with offline domain randomization ', Robotics and Autonomous Systems, vol. 166, 104432 . https://doi.org/10.1016/j.robot.2023.104432
