Development of a Reinforcement Learning Agent for Advanced Process Control in Dynamic Simulations

Abstract

Established advanced process control methods, such as model predictive control, remain the industry standard for optimal control of industrial chemical processes. Traditional approaches for optimising operational indices often rely on first principles models of complex process dynamics, which are strongly affected by stochastic process disturbances and require expensive online computation. The fourth industrial revolution has introduced a new generation of data-based approaches to process optimisation, including reinforcement learning as a potential tool for advanced process control.

The aim of this thesis is to develop a reinforcement learning based advanced process control solution to optimally control chemical processes in simulated environments. Firstly, an appropriate reinforcement learning framework is established to facilitate the development of a solution. Subsequently, a reinforcement learning agent for advanced process control is developed over iterative proof-of-concept stages while gradually increasing the problem complexity.

The results of the final case study demonstrate that the developed reinforcement learning solution is able to handle the optimal control of an industrial process simulation. Specifically, a control policy trained by the developed algorithm learns the simultaneous regulation of multiple interacting process variables through trial-and-error, accounting for stochastic system disturbances robustly. The experimental results validate reinforcement learning as a potential approach for the optimal control of industrial processes based on aptly defined reward functions governing the control objective. The findings motivate future research into extended applications of reinforcement learning based process optimisation in a burgeoning domain of control engineering.