Learning Task-Agnostic Action Spaces for Movement Optimization

Thumbnail Image

Access rights

openAccess
publishedVersion

URL

Journal Title

Journal ISSN

Volume Title

A1 Alkuperäisartikkeli tieteellisessä aikakauslehdessä
This publication is imported from Aalto University research portal.
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)

Authors

Date

2022-12

Department

Department of Computer Science
Department of Media

Major/Subject

Mcode

Degree programme

Language

en

Pages

Series

IEEE Transactions on Visualization and Computer Graphics, Volume 28, issue 12, pp. 4700-4712

Abstract

We propose a novel method for exploring the dynamics of physically based animated characters, and learning a task-agnostic action space that makes movement optimization easier. Like several previous papers, we parameterize actions as target states, and learn a short-horizon goal-conditioned low-level control policy that drives the agent's state towards the targets. Our novel contribution is that with our exploration data, we are able to learn the low-level policy in a generic manner and without any reference movement data. Trained once for each agent or simulation environment, the policy improves the efficiency of optimizing both trajectories and high-level policies across multiple tasks and optimization algorithms. We also contribute novel visualizations that show how using target states as actions makes optimized trajectories more robust to disturbances; this manifests as wider optima that are easy to find. Due to its simplicity and generality, our proposed approach should provide a building block that can improve a large variety of movement optimization methods and applications.

Description

Publisher Copyright: CCBY

Keywords

action space, Aerospace electronics, hierarchical reinforcement learning, movement optimization, Optimization, policy optimization, Reinforcement learning, Splines (mathematics), Task analysis, Training, trajectory optimization, Trajectory optimization

Other note

DOI

10.1109/TVCG.2021.3100095

Citation

Babadi, A, Van de Panne, M, Liu, C & Hamalainen, P 2022, 'Learning Task-Agnostic Action Spaces for Movement Optimization', IEEE Transactions on Visualization and Computer Graphics, vol. 28, no. 12, pp. 4700-4712. https://doi.org/10.1109/TVCG.2021.3100095

Permanent link to this item

https://urn.fi/URN:NBN:fi:aalto-202212146890

Collections

[article-cris] Perustieteiden korkeakoulu / SCI