Imitation-Enhanced Reinforcement Learning With Privileged Smooth Transition for Hexapod Locomotion
Loading...
Access rights
openAccess
acceptedVersion
URL
Journal Title
Journal ISSN
Volume Title
A1 Alkuperäisartikkeli tieteellisessä aikakauslehdessä
This publication is imported from Aalto University research portal.
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Other link related to publication (opens in new window)
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Other link related to publication (opens in new window)
Date
Major/Subject
Mcode
Degree programme
Language
en
Pages
8
Series
IEEE Robotics and Automation Letters, Volume 10, issue 1, pp. 350-357
Abstract
Deep reinforcement learning (DRL) methods have shown significant promise in controlling the movement of quadruped robots. However, for systems like hexapod robots, which feature a higher-dimensional action space, it remains challenging for an agent to devise an effective control strategy directly. Currently, no hexapod robots have demonstrated highly dynamic motion. To address this, we propose imitation-enhanced reinforcement learning (IERL), a two-stage approach enabling hexapod robots to achieve dynamic motion through direct control using RL methods. Initially, imitation learning (IL) replicates a basic positional control method, creating a pre-trained policy for basic locomotion. Subsequently, the parameters from this model are utilized as the starting point for the reinforcement learning process to train the agent. Moreover, we incorporate a smooth transition (ST) method to make IERL overcome the changes in network inputs between two stages, and adaptable to various complex network architectures incorporating latent features. Extensive simulations and real-world experiments confirm that our method effectively tackles the high-dimensional action space challenges of hexapod robots, significantly enhancing learning efficiency and enabling more natural, efficient, and dynamic movements compared to existing methods.Description
Publisher Copyright: © 2016 IEEE.
Other note
Citation
Zhang, Z, Liu, T, Ding, L, Wang, H, Xu, P, Yang, H, Gao, H, Deng, Z & Pajarinen, J 2025, 'Imitation-Enhanced Reinforcement Learning With Privileged Smooth Transition for Hexapod Locomotion', IEEE Robotics and Automation Letters, vol. 10, no. 1, pp. 350-357. https://doi.org/10.1109/LRA.2024.3497754