Safe reinforcement learning based optimization of a cruise ship energy management system

Abstract

The cruise ship industry is a large emitter of greenhouse gases. Multiple aspects must be tackled to help make the industry more sustainable and adhere to the net-zero emission goal set by the International Maritime Organization (IMO). One aspect is the optimization of the energy management of the ship, which can lead to large gains in energy savings. Traditional optimization approaches, such as linear optimization struggle to scale to large stochastic problem spaces such as that of a ship energy system. Modern approaches, such as reinforcement learning, are shown to be more effective at the task at hand. However, these approaches often lack a direct focus on safety, thus making them impractical for real-world applications.

This thesis presents work in the development of a safe reinforcement learning (RL) system for ship energy optimization. As part of the system, a ship's microgrid environment was developed alongside a reinforcement learning agent. The results showcase the potential of RL approaches for energy optimization as opposed to alternative approaches. The inclusion of safety-focused methods, such as utilizing a safety shield and a large language model-generated reward function, was shown to be effective at enhancing safety, reducing the number of safety violations by 87% when the agent was trained and tested in a surrogate model.