Non-linear and probabilistic state discretization functions for enhanced discrete reinforcement learning: Application on wind turbine pitch control

Discrete reinforcement learning (RL) offers interpretability advantages compared to continuous reinforcement learning. Neural networks used in continuous RL behave like black boxes that do not show an obvious relationship between the system states and the actions of the controller. In contrast, the long-term reward table used in discrete RL, the Q-table, is easily interpretable. At a glance, it allows us to see which actions are optimal for each state of the system. Although this is a significant advantage, as a counterpart RL faces the problem of the explosion of the state–space: the computational complexity of the problem increases rapidly with the number of states in the system. In a controller based on RL, improving the system’s precision means increasing the number of states and, therefore, the computational complexity. To address this problem in the specific context of wind turbine pitch control, this article proposes new methods for discretizing the system state. Unlike the generally used uniform discretization, the use of nonlinear and probabilistic techniques is proposed, showing significant improvements in learning and controller performance. Results in this wind turbine case study show that all discretizations outperform the uniform baseline, with the A-law method achieving the best overall convergence speed and lowest steady-state error, while preserving interpretability.