Hierarchical Reinforcement Learning with Uncertainty-Guided Diffusional Subgoals
Loading...
Access rights
openAccess
CC BY
CC BY
publishedVersion
URL
Journal Title
Journal ISSN
Volume Title
A4 Artikkeli konferenssijulkaisussa
This publication is imported from Aalto University research portal.
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Other link related to publication (opens in new window)
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Other link related to publication (opens in new window)
Unless otherwise stated, all rights belong to the author. You may download, display and print this publication for Your own personal use. Commercial use is prohibited.
Date
Major/Subject
Mcode
Degree programme
Language
en
Pages
23
Series
Proceedings of Machine Learning Research, Volume 267, pp. 64750-64772
Abstract
Hierarchical reinforcement learning (HRL) learns to make decisions on multiple levels of temporal abstraction. A key challenge in HRL is that the low-level policy changes over time, making it difficult for the high-level policy to generate effective subgoals. To address this issue, the high-level policy must capture a complex subgoal distribution while also accounting for uncertainty in its estimates. We propose an approach that trains a conditional diffusion model regularized by a Gaussian Process (GP) prior to generate a complex variety of subgoals while leveraging principled GP uncertainty quantification. Building on this framework, we develop a strategy that selects subgoals from both the diffusion policy and GP’s predictive mean. Our approach outperforms prior HRL methods in both sample efficiency and performance on challenging continuous control benchmarks.Description
Publisher Copyright: © 2025 by the author(s).
Keywords
Other note
Citation
Wang, V H, Wang, T & Pajarinen, J 2025, 'Hierarchical Reinforcement Learning with Uncertainty-Guided Diffusional Subgoals', Proceedings of Machine Learning Research, vol. 267, pp. 64750-64772. < https://raw.githubusercontent.com/mlresearch/v267/main/assets/wang25dh/wang25dh.pdf >