Reader: Model-based language-instructed reinforcement learning

Thumbnail Image

Access rights

openAccess
publishedVersion

URL

Journal Title

Journal ISSN

Volume Title

A4 Artikkeli konferenssijulkaisussa
This publication is imported from Aalto University research portal.
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Other link related to publication (opens in new window)

Authors

Date

2023

Department

Department of Computer Science

Major/Subject

Mcode

Degree programme

Language

en

Pages

Series

Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 16583–16599

Abstract

We explore how we can build accurate world models, which are partially specified by language, and how we can plan with them in the face of novelty and uncertainty. We propose the first model-based reinforcement learning approach to tackle the environment Read To Fight Monsters (Zhong et al., 2019), a grounded policy learning problem. In RTFM an agent has to reason over a set of rules and a goal, both described in a language manual, and the observations, while taking into account the uncertainty arising from the stochasticity of the environment, in order to generalize successfully its policy to test episodes. We demonstrate the superior performance and sample efficiency of our model-based approach to the existing model-free SOTA agents in eight variants of RTFM. Furthermore, we show how the agent’s plans can be inspected, which represents progress towards more interpretable agents.

Description

Keywords

Other note

Citation

Dainese, N, Marttinen, P & Ilin, A 2023, Reader: Model-based language-instructed reinforcement learning . in H Bouamor, J Pino & K Bali (eds), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing . Association for Computational Linguistics, pp. 16583–16599, Conference on Empirical Methods in Natural Language Processing, Singapore, Singapore, 06/12/2023 . < https://aclanthology.org/2023.emnlp-main.1032 >

Permanent link to this item

https://urn.fi/URN:NBN:fi:aalto-202401041175

Collections

[article-cris] Perustieteiden korkeakoulu / SCI