Evaluation of reinforcement learning and model predictive control for apartment heating with heat pump and water storage tank

Thumbnail Image

Files

1-s2.0-S2352152X26001738-main.pdf (1.8 MB)

Access rights

openAccess
CC BY
publishedVersion

URL

Journal Title

Journal ISSN

Volume Title

A1 Alkuperäisartikkeli tieteellisessä aikakauslehdessä
This publication is imported from Aalto University research portal.
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Unless otherwise stated, all rights belong to the author. You may download, display and print this publication for Your own personal use. Commercial use is prohibited.

Authors

Date

2026-03-30

Department

Department of Energy and Mechanical Engineering

Major/Subject

Mcode

Degree programme

Language

en

Pages

11

Series

Journal of Energy Storage, Volume 152

Abstract

Optimising apartment heating systems is becoming increasingly crucial due to growing use of heat pumps and heat storages. This study compares reinforcement learning (RL) and model predictive control (MPC) to optimise and control a simulated apartment heating system. We explore various RL designs, including binary and continuous action spaces, different reward functions, and three storage sizes. We find that MPC outperforms RL when the optimisation period aligns with the actual problem. This is shown in the lower electricity costs of MPC compared to RL with small and base storage sizes. However, the two methods yielded similar electricity costs with the large storage size. We also find that the RL design significantly affects its performance and robustness. The RL model with binary actions and a reward function promoting the active use of storage and profit maximisation outperforms other RL configurations. In turn, a reward function representing the actual problem of cost minimisation was found to be ineffective for agent training. Future studies that compare the two methods for the optimisation of heating systems with long-term storage could focus on extending the prediction horizon or enhancing the terminal cost term in MPC.

Description

Publisher Copyright: © 2026 The Authors

Keywords

Apartment space heating, Heat pump, Heat storage, Model predictive control, Proximal policy optimisation, Reinforcement learning

Other note

DOI

10.1016/j.est.2026.120509

Citation

Tarvainen, S, Hirvijoki, E & Laukkanen, T 2026, 'Evaluation of reinforcement learning and model predictive control for apartment heating with heat pump and water storage tank', Journal of Energy Storage, vol. 152, 120509. https://doi.org/10.1016/j.est.2026.120509

Permanent link to this item

https://urn.fi/URN:NBN:fi:aalto-202602042395

Collections

[article-cris] Palvelut / Services