Reactive Power Control of a Converter in a Hardware-Based Environment Using Deep Reinforcement Learning

Bokker, Ode; Schlachter, Henning; Beutel, Vanessa; Geißendörfer, Stefan; von Maydell, Karsten

doi:10.3390/en16010078

Open AccessFeature PaperArticle

Reactive Power Control of a Converter in a Hardware-Based Environment Using Deep Reinforcement Learning

by

Ode Bokker

^*

,

Henning Schlachter

,

Vanessa Beutel

,

Stefan Geißendörfer

and

Karsten von Maydell

German Aerospace Center (DLR), Institute of Networked Energy Systems, Carl-von-Ossietzky-Str. 15, 26129 Oldenburg, Germany

^*

Author to whom correspondence should be addressed.

Energies 2023, 16(1), 78; https://doi.org/10.3390/en16010078

Submission received: 22 November 2022 / Revised: 10 December 2022 / Accepted: 13 December 2022 / Published: 21 December 2022

(This article belongs to the Section A1: Smart Grids and Microgrids)

Download

Browse Figures

Versions Notes

Abstract

Due to the increasing penetration of the power grid with renewable, distributed energy resources, new strategies for voltage stabilization in low voltage distribution grids must be developed. One approach to autonomous voltage control is to apply reinforcement learning (RL) for reactive power injection by converters. In this work, to implement a secure test environment including real hardware influences for such intelligent algorithms, a power hardware-in-the-loop (PHIL) approach is used to combine a virtually simulated grid with real hardware devices to emulate as realistic grid states as possible. The PHIL environment is validated through the identification of system limits and analysis of deviations to a software model of the test grid. Finally, an adaptive volt–var control algorithm using RL is implemented to control reactive power injection of a real converter within the test environment. Despite facing more difficult conditions in the hardware than in the software environment, the algorithm is successfully integrated to control the voltage at a grid connection point in a low voltage grid. Thus, the proposed study underlines the potential to use RL in the voltage stabilization of future power grids.

Keywords: power grid; reactive power; voltage control; power hardware-in-the-loop

Share and Cite

MDPI and ACS Style

Bokker, O.; Schlachter, H.; Beutel, V.; Geißendörfer, S.; von Maydell, K. Reactive Power Control of a Converter in a Hardware-Based Environment Using Deep Reinforcement Learning. Energies 2023, 16, 78. https://doi.org/10.3390/en16010078

AMA Style

Bokker O, Schlachter H, Beutel V, Geißendörfer S, von Maydell K. Reactive Power Control of a Converter in a Hardware-Based Environment Using Deep Reinforcement Learning. Energies. 2023; 16(1):78. https://doi.org/10.3390/en16010078

Chicago/Turabian Style

Bokker, Ode, Henning Schlachter, Vanessa Beutel, Stefan Geißendörfer, and Karsten von Maydell. 2023. "Reactive Power Control of a Converter in a Hardware-Based Environment Using Deep Reinforcement Learning" Energies 16, no. 1: 78. https://doi.org/10.3390/en16010078

APA Style

Bokker, O., Schlachter, H., Beutel, V., Geißendörfer, S., & von Maydell, K. (2023). Reactive Power Control of a Converter in a Hardware-Based Environment Using Deep Reinforcement Learning. Energies, 16(1), 78. https://doi.org/10.3390/en16010078

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Menu

Reactive Power Control of a Converter in a Hardware-Based Environment Using Deep Reinforcement Learning

Abstract

Share and Cite

Article Metrics

Article Access Statistics

Further Information

Guidelines

MDPI Initiatives

Follow MDPI