Learning High-Risk High-Precision Motion Control

Thumbnail Image
Access rights
openAccess
Journal Title
Journal ISSN
Volume Title
A4 Artikkeli konferenssijulkaisussa
This publication is imported from Aalto University research portal.
View publication in the Research portal
View/Open full text file from the Research portal
Other link related to publication
Author
Date
2022-11-03
Department
Department of Computer Science
Department of Art and Media
Major/Subject
Mcode
Degree programme
Language
en
Pages
10
1-10
Series
MIG '22: Proceedings of the 15th ACM SIGGRAPH Conference on Motion, Interaction and Games
Abstract
Deep reinforcement learning (DRL) algorithms for movement control are typically evaluated and benchmarked on sequential decision tasks where imprecise actions may be corrected with later actions, thus allowing high returns with noisy actions. In contrast, we focus on an under-researched class of high-risk, high-precision motion control problems where actions carry irreversible outcomes, driving sharp peaks and ridges to plague the state-action reward landscape. Using computational pool as a representative example of such problems, we propose and evaluate State-Conditioned Shooting (SCOOT), a novel DRL algorithm that builds on advantage-weighted regression (AWR) with three key modifications: 1) Performing policy optimization only using elite samples, allowing the policy to better latch on to the rare high-reward action samples; 2) Utilizing a mixture-of-experts (MoE) policy, to allow switching between reward landscape modes depending on the state; 3) Adding a distance regularization term and a learning curriculum to encourage exploring diverse strategies before adapting to the most advantageous samples. We showcase our features’ performance in learning physically-based billiard shots demonstrating high action precision and discovering multiple shot strategies for a given ball configuration.
Description
Keywords
Other note
DOI
10.1145/3561975.3562943
Citation
Kim, N H, Kirjonen, M & Hämäläinen, P 2022, Learning High-Risk High-Precision Motion Control . in S N Spencer (ed.), MIG '22: Proceedings of the 15th ACM SIGGRAPH Conference on Motion, Interaction and Games ., 9, ACM, pp. 1-10, ACM SIGGRAPH Conference on Motion, Interaction and Games, Guanajuato, Mexico, 03/11/2022 . https://doi.org/10.1145/3561975.3562943
Permanent link to this item
https://urn.fi/URN:NBN:fi:aalto-202212076807
Collections
[article-cris] Perustieteiden korkeakoulu / SCI