**Optimisation, Optimal Control and Nonlinear Dynamics in Electrical Power, Energy Storage and Renewable Energy Systems**

Editors

**Victor M. Becerra Ahmed Rachid**

MDPI • Basel • Beijing • Wuhan • Barcelona • Belgrade • Manchester • Tokyo • Cluj • Tianjin

*Editors* Victor M. Becerra University of Portsmouth UK

Ahmed Rachid University of Picardie Jules Verne France

*Editorial Office* MDPI St. Alban-Anlage 66 4052 Basel, Switzerland

This is a reprint of articles from the Topic published online in the open access journals *Energies* (ISSN 1996-1073), *Electronics* (ISSN 2079-9292), *Batteries* (ISSN 2313-0105), and *Electricity* (ISSN 2673-4826) (available at: https://www.mdpi.com/topics/optimisation).

For citation purposes, cite each article independently as indicated on the article page online and as indicated below:

LastName, A.A.; LastName, B.B.; LastName, C.C. Article Title. *Journal Name* **Year**, *Volume Number*, Page Range.

**ISBN 978-3-0365-5585-0 (Hbk) ISBN 978-3-0365-5586-7 (PDF)**

Cover image courtesy of Victor M. Becerra

© 2022 by the authors. Articles in this book are Open Access and distributed under the Creative Commons Attribution (CC BY) license, which allows users to download, copy and build upon published articles, as long as the author and publisher are properly credited, which ensures maximum dissemination and a wider impact of our publications.

The book as a whole is distributed by MDPI under the terms and conditions of the Creative Commons license CC BY-NC-ND.

## **Contents**



## **About the Editors**

### **Victor M. Becerra**

Professor Victor M. Becerra is currently a Professor of Power Systems Engineering at the School of Energy and Electronic Engineering, University of Portsmouth, UK, where he has been since 2015. He obtained his PhD in Control Engineering from the City University of London in 1994. During 1989–1991, he carried out power systems analysis and control systems design at C.V.G. Edelca, Caracas, Venezuela. Between 1994 and 1999, he was a post-doctoral Research Fellow at the Control Engineering Research Centre at the City University of London. During the period between 2000 and 2015, he was an academic at the School of Systems Engineering, University of Reading, UK, where he became a Professor of Automatic Control in 2012. His current research interests consider a range of issues related to power systems and smart grids, as well as the methods and applications of automatic control. During his career, he has received funding for research and innovation from the EPSRC, Innovate UK, the Royal Academy of Engineering, the European Space Agency, and the EU. He has published over 150 refereed papers, one research monograph and two edited volumes. Professor Becerra is a Fellow of the IET and a Senior Member of the IEEE.

#### **Ahmed Rachid**

Ahmed Rachid received an engineering degree in Cybernetics in 1983, an MBA in 1985, a PhD in electrical engineering in 1986 and a Habilitation degree in 1991 from the University of Nancy (France). He has been teaching since 1984 in different universities and engineering schools mainly in the fields of control engineering and signal processing. Since 1992, he has been a professor at the University of Picardie Jules Verne (France), where he created the institution's first laboratory in Automatic Systems. His current research includes processes and systems modeling, simulation, control, diagnosis and observation with applications in blast furnaces, vehicle dynamics, diesel engines, induction electrical drives, stepper motors, greenhouses, magnetic levitation, pantograph-catenary, co-generation, electric vehicles, solar power systems, etc. He has supervised 24 PhD theses and has co-authored 2 patents, 2 books and over 150 scientific papers. He has organized and chaired several international conferences and has coordinated various European research projects and industrial R&D contracts.

## *Article* **Hardware Implementation of Novel Shade Dispersion PV Reconfiguration Technique to Enhance Maximum Power under Partial Shading Conditions**

**Vinaya Chandrakant Chavan <sup>1</sup> , Suresh Mikkili 1,\* and Tomonobu Senjyu <sup>2</sup>**


**Abstract:** Partial shade condition is a significant factor contributing to the PV panel performance in mismatch losses and power generation. The technique suggested in this study allows the physical rearrangement of the PV panel to distribute the shade on the entire PV array. MPPT, selecting suitable inverter topology, or PV panel reconfiguration enhances the performance of the PV panel. This study proposes a new shade dispersing method, novel shade dispersion (NSD). It compares the performance of the NSD method with conventional configurations (CCs). This research article models and simulates 6 × 6 PV array configurations such as Series-Parallel (SP), Total-Cross-Tide (TCT), Bridge-Linked (BL), Honey-Comb (HC), and the newly proposed NSD method under nonshading and nine different partial shading cases. The performance indices used for comparative analysis are global maximum power points, efficiency, power enhancement, open circuit voltage, short circuit current, and number of crests. The Soltech 1 STH-215-P PV module was selected in the MATLAB/Simulink environment to simulate PV array arrangements. Hardware experiments validate the performance of the NSD method.

**Keywords:** partial shading; conventional configuration; reconfiguration techniques

## **1. Introduction**

Renewable energy has become one of the most critical factors to preserve the climate and the earth's assets for the coming generation. Power has a crucial part in the financial growth and the welfare of a nation. From India's perspective, solar energy increases power generation and generates energy reliably considering ecological, communal, and financially beneficial properties. India has significant potential, availability, and other features of solar power. Therefore, the government also emphasizes the promotion of photovoltaic generation in the Indian power sector [1]. Series and parallel connected modules form an array to obtain power per requirement. PV material, temperature, irradiance, dust, array structure, maximum power point tracking (MPPT) method, converter topology, etc., are the different parameters responsible for the performance of PV array [2,3]. This uncertainty affects the estimation of a PV cell's series and shunt resistances (namely, Rs and Rsh). Ref. [4] provided the uncertainty analysis of the measured currents, voltages, temperatures, and irradiances. At a constant temperature, the output current increases with an increase in irradiance with negligible effect on the voltage. Voltage is substantially reduced at constant irradiance, and the current slightly increases with increased temperature up to the rated temperature [5]. Any object that obstructs the sun rays' path to the PV panel or uneven falling sunlight on the earth creates a partial shading condition (PSC) [6]. A bypass diode is connected antiparallel to the PV module to avoid the hot spot issues created by PSC. Under PSC, the difference between the minimum irradiance received by non-overlapped

**Citation:** Chavan, V.C.; Mikkili, S.; Senjyu, T. Hardware Implementation of Novel Shade Dispersion PV Reconfiguration Technique to Enhance Maximum Power under Partial Shading Conditions. *Energies* **2022**, *15*, 3515. https://doi.org/ 10.3390/en15103515

Academic Editor: Alberto Reatti

Received: 7 April 2022 Accepted: 9 May 2022 Published: 11 May 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

and overlapped solar cells determines the overcurrent [7]. A diode adds antiparallel either to a PV module or a set of PV series modules [8]. It inserts the number of peaks in the PV curve, among which the rise consisting of the highest value is known as the global maximum power point (GMPP). Traditional MPPT struggles to obtain true MPP in rapidly changing PSCs. In this area, extensive research considers improved conventional methods, new metaheuristic methods, etc., which increases the complexity and cost of the MPPT system [9]. Ref. [10] performed the experimental validation of TCT with the standard deviation of PSC and found a better configuration. Results demonstrated that 15% and 25% reduced irradiance. The power yield degraded to 4% and 7.5%. Refs. [11,12] explored the performance of series (S), parallel (P), SP, TCT, BL, and HC PV topologies and concluded, without considering bypass diode BL, and with bypass diode TCT generates maximum power. Reconfiguration is an effective way to improve power generation [13]. This was classified into two broad categories: electrical and physical reconfiguration. Both methods have their boons and downsides [14]. Ref. [15] proposed the maximum and minimum algorithm dynamic reconfiguration method, which uses a double pole double throw switch to change the electrical connections. Ref. [16] provided the dynamic array reconfiguration for water pumping applications. The authors of [17] investigated efficiency and robustness of the dynamic array reconfiguration for hybrid PV system application. Refs. [15–17] efficiently used the switch matrix to reconfigure the PV modules in the array for different applications. Ref. [18] proposed an electrical repositioning on a 4 × 5 arrangement with 12 switches to make adaptive cross ties to enhance the power generation. For case I the proposed method improved GMPP, fill factor (FF), and power enhancement (PE) compared with SP, and minimized power losses (PL). It was found best at 291.3 W, 0.586, 2.71%, 55.6 W, and 83.97%, for TCT in MATLAB/Simulink and experimentation with the same performance index under all PSCs. Complexity, sensor requirement, switches, and their controlling structure, sensor speed dependency, cost, etc., are the significant limitations of dynamic reconfiguration. Therefore, many researchers go for static array reconfiguration. The authors of [19] proposed and demonstrated a Sudoku-based physical reconfiguration system on a 9 × 9 array. It improves the power generation from 4 to 26% for different PSCs. The authors of [20] proposed an improved Sudoku method on a 9 × 9 array and compared it to the CC. Ref. [21] offered a magic square method and compared it with TCT, Sudoku, and the optimal Sudoku method on an 81 modules array structure. Ref. [22] proposed the zigzag method and compared it with the TCT and Sudoku method on a 6 × 6 PV array. Ref. [23] related competence square and the dominance square method with TCT on a 9 × 9 PV array. Ref. [24] proposed a clue-based skyscraper puzzle method and compared it with TCT, Sudoku method, and the puzzle-based competence square method. This study reviews many static reconfiguration (SR) methods and analyzes their performance. SR is a robust, simple, less costly, and effective way to deal with partial shading issues and improve the performance of the PV array. Still, these methods have some limitations.

The novelty of work is:

Electrical networks persist similarly to TCT and the physical position of units in the PV array changed.


• The proposed scheme was analyzed under ten shading cases. The results validated with SP, TCT, BL, and HC compared the global maximum power point (GMPP), mislead power (MP), output power (%O/P), efficiency, fill factor, and % power enhancement for NSD under-considered shading pattern. Figure 1 gives the workflow of this paper. same column, maintaining the least possible wiring; • The proposed scheme was analyzed under ten shading cases. The results validated with SP, TCT, BL, and HC compared the global maximum power point (GMPP), mislead power (MP), output power (%O/P), efficiency, fill factor, and % power enhancement for NSD under-considered shading pattern. Figure 1 gives the workflow of this paper.

ray. This method considers magic sub-matrix shifting to reposition the panels in the

*Energies* **2022**, *15*, x FOR PEER REVIEW 3 of 18

**Figure 1.** Representation of PV array connection structures under partial shading conditions with performance evaluation parameters. **Figure 1.** Representation of PV array connection structures under partial shading conditions with performance evaluation parameters.

#### **2. Modeling and Simulation of NSD Method 2. Modeling and Simulation of NSD Method**

This section explains the modeling and simulation of conventional PV array configurations and the proposed novel shade dispersion (NSD) method in MATLAB/Simulink. The panel electrical parameters has mentioned in Appendix A. The flowchart for panel repositioning is provided in Figure 2. The present work consists of 36 modules placed in six rows and six column patterns (shown in Figure 3a) for conventional configurations and the NSD method (Figure 3b). The TCT connection-based MATLAB simulation is shown in Figure 4. Mathematical modeling for output voltage, output current, and output power of all the traditional PV array structures are provided in Table 1. Here, the module, row, and array output voltages are V୨, Vୖ, and V, respectively. The module, string, and array output currents are I୨, Iୗ, and I. P is the PV generation of the array. A novel shade dispersion method is proposed to reconfigure the panel of a variety to improve the performance through shade dispersion. In the proposed technique, electrical connections remain the same as in TCT [20]; only the physical location of the modules changes. It uses voltage, current, and power equations for TCT, as This section explains the modeling and simulation of conventional PV array configurations and the proposed novel shade dispersion (NSD) method in MATLAB/Simulink. The panel electrical parameters has mentioned in Appendix A. The flowchart for panel repositioning is provided in Figure 2. The present work consists of 36 modules placed in six rows and six column patterns (shown in Figure 3a) for conventional configurations and the NSD method (Figure 3b). The TCT connection-based MATLAB simulation is shown in Figure 4. Mathematical modeling for output voltage, output current, and output power of all the traditional PV array structures are provided in Table 1. Here, the module, row, and array output voltages are V<sup>j</sup> , VR, and VPV, respectively. The module, string, and array output currents are I<sup>j</sup> , IS, and IPV. PPV is the PV generation of the array. A novel shade dispersion method is proposed to reconfigure the panel of a variety to improve the performance through shade dispersion. In the proposed technique, electrical connections remain the same as in TCT [20]; only the physical location of the modules changes. It uses voltage, current, and power equations for TCT, as mentioned in Table 1. The basic principle used for this method is row current equalization. It proposes a magic submatrix

*Energies* **2022**, *15*, x FOR PEER REVIEW 4 of 18

Step I: Formation of the submatrix

Step II: Formation of the primary matrix

the same row and vacant column serially;

the magic number 21;

fifth step is achieved.

is 6;

mentioned in Table 1. The basic principle used for this method is row current equalization. It proposes a magic submatrix arrangement with the magic number 21 and its logi-

1. The submatrix is proposed so that rows and column multiplication of the submatrix

2. Therefore, numbers 1 to 6 are placed in the 2 × 3 submatrix to summation equal to

3. The submatrix placing is started in the 1st row, 1st column position. The next submatrix is placed at the next (3rd) row, next (2nd) column position. This process con-

4. Submatrix rows are interchanged, and that submatrix starts to fill in vacant places of the primary matrix, starting from the next column of previously filled areas. Here, the interchanged submatrix is filled from 1st row, 4th column. The next submatrix is placed in the 3rd row and 4th column. The last column of the submatrix is placed in

cal shifting to reposition the panels within a column, as shown in Figure 3b. The steps involved to build the proposed 6 × 6 NSD matrix are:

tinues until the last (6th) row is filled by the considered submatrix;

arrangement with the magic number 21 and its logical shifting to reposition the panels within a column, as shown in Figure 3b. 5. It is performed so that the summation of row numbers, as well as column numbers, is equal to the magic number 21, as shown in Figure 3b; 6. The process continues until vacant places in the primary matrix are filled, and the

**Figure 2.** Flowchart for panel repositioning of NSD method (**a**) Step 1: submatrix formation; (**b**) Step 2: main matrix formation. **Figure 2.** Flowchart for panel repositioning of NSD method (**a**) Step 1: submatrix formation; (**b**) Step 2: main matrix formation.

C1 C2 C3 C4 C5 C6 R1 **111111** R2 **222222** R3 **333333** R4 **444444** R5 **555555** R6 **666666**

> **264 531**

Submatrix

**21**

(a)

(b)

**Figure 3.** Panel positions in PV array (**a**) conventional methods, (**b**) NSD method.

**11 12 13 14 15 16 21 22 23 24 25 26 31 32 33 34 35 36 41 42 43 44 45 46 51 52 53 54 55 56 61 62 63 64 65 66**

C1 C2 C3 C4 C5 C6 R1 **264531** 21 R2 **531264** 21 R3 **126453** 21 R4 **453126** 21 R5 **312645** 21 R6 **645312** 21 21 21 21 21 21 21

Step 2: main matrix formation.

(**b**) **Figure 2.** Flowchart for panel repositioning of NSD method (**a**) Step 1: submatrix formation; (**b**)

**Stop**

**Yes** 

**Are all the places filled?**

**Yes Yes** 

**No No**

**Start**

**Select square matrix of a×a**

**Start placing submatrix from 1st row, 1st column**

> **Is 'a' prime number**

> > **Place submatrix of m×n from next row, next column of previous submatrix**

**No**

**Is 'ath ' row filled**

**Place next submatrix element rowwise staring from next row and jump by skipping one column of previous submatrix**

**Yes** 

**No**

**Is 'ath ' row filled**

**Figure 3.** Panel positions in PV array (**a**) conventional methods, (**b**) NSD method. **Figure 3.** Panel positions in PV array (**a**) conventional methods, (**b**) NSD method.

**Figure 4.** TCT-based MATLAB simulation for the NSD method. **Figure 4.** TCT-based MATLAB simulation for the NSD method.

**Table 1.** The output voltage, current, and power for conventional PV array configurations. **Table 1.** The output voltage, current, and power for conventional PV array configurations.


**HC** V = V୨ ୨ୀଵ = 6V୨ I = Iୗଵ Iୗଶ Iୗଷ Iୗସ Iୗହ Iୗ = 6Iୗ P = 36V୨Iୗ The steps involved to build the proposed 6 × 6 NSD matrix are: Step I: Formation of the submatrix


global maximum power point (GMPP), mislead nine PSCs as power (MP), output power (%O/P), efficiency, fill factor, and % power enhancement of NSD. The %PE was calcu-

• Case II to case IV have group 1 of 24 panels with 1000 W irradiance; groups 2, 3, and 4 have 200 W, 500 W, and 700 W irradiance, respectively, with four panels in each

• In case V, 10 modules have PSCs, group 1 with 1000 W has 26 panels. Group 2, with 200 W, has four modules, and groups 3 and 4 have 500 W, 700 W irradiance, respec-

• Case VI has groups 1, 2, 3, and 4 with 1000 W, 200 W, 500 W, and 700 W irradiance

• In case VII, 24 panels came in group 1. Groups 2 and 3, have 200 W and 700 W irradiance, respectively. Group 4 has two panels that have 700 W irradiance; • In case VIII, groups 1, 2, 3, and 4 have five panels in each group of 1000 W, 200 W,

500 W, and 700 W irradiance with 16, 8, 7, and five panels, respectively;

non-shading condition), the performance of the NSD method is the same as the CC (i.e., without reconfiguration), as shown in Figure 7a. The performance indicators are the Step II: Formation of the primary matrix

on 28, 3, 1, and 4 modules, respectively;

group;

tively, with three panels;


#### **3. Partial Shadings Conditions and Shade Dispersion Effect**

For CCs the shade does not scatter, as shown in Figure 5. Through NSD, the partial shading condition spread on a complete array, as shown in Figure 6. Under case I (i.e., non-shading condition), the performance of the NSD method is the same as the CC (i.e., without reconfiguration), as shown in Figure 7a. The performance indicators are the global maximum power point (GMPP), mislead nine PSCs as power (MP), output power (%O/P), efficiency, fill factor, and % power enhancement of NSD. The %PE was calculated for NSD and compared with all the CCs. For case I, all 36 modules of the PV array have 1000 W irradiance; the remaining PSCs have four groups of irradiance. *Energies* **2022**, *15*, x FOR PEER REVIEW 7 of 18 • In case IX, only five modules are partially shaded, and 31 modules have 1000 W irradiance in group 1. It has two 200 W modules in group 2, one 500 W module in group 3 and two 700 W irradiance panels in group 4; • In case X, with 1000 W, group 1 has 25 panels. Group 2, with 200 W has four panels, and groups 3 and 4 have 500 W and 700 W irradiance, respectively, with three panels. *Energies* **2022**, *15*, x FOR PEER REVIEW 7 of 18 • In case IX, only five modules are partially shaded, and 31 modules have 1000 W irradiance in group 1. It has two 200 W modules in group 2, one 500 W module in group 3 and two 700 W irradiance panels in group 4; • In case X, with 1000 W, group 1 has 25 panels. Group 2, with 200 W has four panels, and groups 3 and 4 have 500 W and 700 W irradiance, respectively, with three pan-

**Figure 5.** Partial shading conditions (**a**) case II, (**b**) case III, (**c**)case IV, (**d**) case V, (**e**) case VI, (**f**) case VII, (**g**) case VIII, (**h**) case IX, (**i**) case X. **Figure 5.** Partial shading conditions (**a**) case II, (**b**) case III, (**c**)case IV, (**d**) case V, (**e**) case VI, (**f**) case VII, (**g**) case VIII, (**h**) case IX, (**i**) case X. (g) (h) (i) **Figure 5.** Partial shading conditions (**a**) case II, (**b**) case III, (**c**)case IV, (**d**) case V, (**e**) case VI, (**f**) case VII, (**g**) case VIII, (**h**) case IX, (**i**) case X.

**1000 1000 1000 500 200 1000**

**1000 200 700 1000 1000 1000**

**1000 700 1000 1000 200 1000**

VI, (**f**) case VII, (**g**)case VIII, (**h**) case IX, (**i**) case X. **Figure 6.** Shade spreading due to NSD method (**a**) case II, (**b**) case III, (**c**)case IV, (**d**) case V, (**e)** case VI, (**f**) case VII, (**g**)case VIII, (**h**) case IX, (**i**) case X. **Figure 6.** Shade spreading due to NSD method (**a**) case II, (**b**) case III, (**c**)case IV, (**d**) case V, (**e)** case VI, (**f**) case VII, (**g**)case VIII, (**h**) case IX, (**i**) case X.

**Figure 7.** PV curves under various irradiance levels (**a**) case I, (**b**) case II, (**c**)case III, (**d**) case IV, (**e**) case V, (**f**) case VI, (**g**)case VII, (**h**) case VIII, (**i**) case IX, (**j**) case X. **Figure 7.** PV curves under various irradiance levels (**a**) case I, (**b**) case II, (**c**)case III, (**d**) case IV, (**e**) case V, (**f**) case VI, (**g**)case VII, (**h**) case VIII, (**i**) case IX, (**j**) case X.


#### **4. Result and Discussion**

The performance evaluation of PV array CCs and proposed reconfiguration was performed under various shading situations on MATLAB/Simulink. The assessment has two segments, (1) power-voltage profile and (2) computational parameters.

#### *4.1. Power-Voltage Profiles under Various PSC*

The sketches in Figure 7 represent the variation in PV curves for different conventional configurations and the proposed NSD method of reconfiguration under various irradiance levels. Under case I of the non-shading condition, all the panels have the same standard irradiance of 1000 W. Figure 7a highlights that all the configuration and NSD methods have the same ideal performance under case I with only one peak on the PV curve.

Under case II, conventional configurations have two peaks on the PV curve, while the NSD method has only one extreme. TCT, BL, and HC have identical extreme power, and it is more than SP, while for NSD, the maximum capacity is much more enhanced, as revealed in Figure 7b. Figure 7c indicates that SP has the lowest power output for case III. It offers continuous improvement for BL, HC, TCT, and NSD. NSD offers 6234 W as the maximum power output. In Figure 7d, SP has three peaks, and other methods have two heights. BL and HC show improved performance over SP but lesser than TCT and NSD. NSD provides the highest output of 6234 W, as shown. Figure 7e portrays that BL and HC have six peaks, SP and TCT have five peaks, while NSD is reduced to four peaks and the highest power generation equal to 6303 W for case V. In Figure 7f case VI, HC and SP have the same maximum power generation as BL and TCT with three peaks; in contrast, the NSD has the highest power generation of 6646 W with three small deep peaks. Figure 7g showcases the case VII PV curves for SP, TCT, BL, HC, and NSD. NSD has a single top with 6133 W as the highest power generation. In Figure 7h SP and TCT have six and four peaks. BL, HC, and NSD structures have three peaks with the highest power, 5160 W, for the NSD method. Figure 7i portrays case IX. Under this shade, TCT has maximum power extraction of 6563 W in conventional configuration while NSD has improved power extraction with 6953 W.

#### *4.2. Computational Parameters*

The performances of all CC and NSD were investigated in the following six computational parameters, as observed in Appendix B, which highlights the efficacy of NSD over CC. Tables 2–6 give the performance of SP, TCT, BL, HC and NSD method under ten PSCs respectively.


**Table 2.** SP performance under PSC.

**Table 3.** TCT performance under PSC.


**Table 4.** BL performance under PSC.


**Table 5.** HC performance under PSC.



**Table 6.** NSD performance under PSC.

### 4.2.1. Global Maximum Power Point (GMPP)

The PV curve has multiple peaks due to PSC, as shown in Figure 8. GMPP is the maximum power of all the crests. Figure 8a indicates that GMPP for all CCs and NSD have concentrated 7665 W. For case II, TCT, BL, and HC provide equal power at 5039 W in CC, and NSD improves it up to 6234 W. Case III also NSD has GMPP at 6234 W. Under case IV in CC, TCT provides maximum output at 6028 W while NSD has 6211 W. Under case V, NSD has 6303 W as a GMPP; this is 1201 W more than HC, which offers the highest power of all CCs. Under case VI to case X in CC, TCT offers the uppermost GMPP while NSD has GMPP at 6646 W, 6133 W, 5160 W, 6953 W, and 6350 W for respective PSC. It is 812 W, 1325 W, 988 W, 389 W, and 257 W more than TCT under the same PSCs.

#### 4.2.2. Mislead Power (MP)

Figure 8b shows that the proposed NSD method provides the least misleading power as the red NSD trace is at the innermost on the radar pattern. Case I does not have any MP. From case II to case X it is 1431 W, 1431.8 W, 1454 W, 1363 W, 1020 W, 1533 W, 2506 W, 713 W, and 1315 W continuously. NSD reduces the MP by about 11 to 46% under these shading cases.

#### 4.2.3. Percent Output (%O/P)

Figure 8c Highlights that all considered structure provides 100% output for case I. NSD provides 81% O/P, the maximum from cases II, III, and IV. In CC, HC provides the highest %O/P at 66.5% under case V, while NSD improves it to 82.69%. In CC under case VI to case X, TCT offers the uppermost %O/P, and NSD improved it by 10.5%, 17.3%, 12.9%, 5%, and 3.4% compared with TCT for the respective shading cases.

#### 4.2.4. Efficiency (η) −

Figure 8d shows the NSD structure is more efficient than TCT, which has the same electrical connections; under-considered PSCs. The radar pattern representation in Figure 8d clearly shows that NSD offers efficiency near 21%. It is approximately equal to the PV array efficiency for case I.

#### 4.2.5. Fill Factor (FF)−

Figure 8e highlights that the proposed method offers 2–5% improvement in FF for cases III, IV, IX, and X; 15.19% more FF for case V, 8% improvement for case VI, and 20–25% enhancement for case II, VII, and VIII, compared with maximum FFs in CC.

#### 4.2.6. Percent Power Enhancement (%PE)

The %PE ensures the power increment by the proposed reconfiguration. Figure 8f shows that NSD improves power extraction compared with CC. Compared with SP, TCT, BL, and HC, NSD has a 25.9, 15.79, 17.44, and 16.45% power boost.

4.2.5. Fill Factor (FF)<sup>−</sup>

4.2.6. Percent Power Enhancement (%PE)

Figure 8e highlights that the proposed method offers 2–5% improvement in FF for cases III, IV, IX, and X; 15.19% more FF for case V, 8% improvement for case VI, and 20–

The %PE ensures the power increment by the proposed reconfiguration. Figure 8f

25% enhancement for case II, VII, and VIII, compared with maximum FFs in CC.

BL, and HC, NSD has a 25.9, 15.79, 17.44, and 16.45% power boost.

**Figure 8.** Performance evaluation parameter (**a**) GMPP (W), (**b**) MP(W), (**c**)%O/P, (**d**) η, (**e**) FF, (**f**) **Figure 8.** Performance evaluation parameter (**a**) GMPP (W), (**b**) MP(W), (**c**)%O/P, (**d**) η, (**e**) FF, (**f**) %PE.

#### **5. Experimental Implementation and Result Analysis 5. Experimental Implementation and Result Analysis**

#### *5.1. Experimental Setup 5.1. Experimental Setup*

%PE.

Towards real-time implementation of the proposed NSD method, a prototype laboratory experiment with 16 PV modules and a 4 × 4 array structure was conducted on the PV chroma 62050 H-600 S simulator (Taiwan). The chroma 63203 E DC (Taiwan) electronic load was used to develop an experimental setup, as shown in Figure 9. The electrical parameters for each 5 Watt panel are, VOC = 21.6 V, VM = 17.5 V, ISC = 0.3 A, IM = 0.29 V. Under non-shading conditions, this PV array provides 80 W power generation. Towards real-time implementation of the proposed NSD method, a prototype laboratory experiment with 16 PV modules and a 4 × 4 array structure was conducted on the PV chroma 62050 H-600 S simulator (Taiwan). The chroma 63203 E DC (Taiwan) electronic load was used to develop an experimental setup, as shown in Figure 9. The electrical parameters for each 5 Watt panel are, VOC = 21.6 V, V<sup>M</sup> = 17.5 V, ISC = 0.3 A, I<sup>M</sup> = 0.29 V. Under non-shading conditions, this PV array provides 80 W power generation.

> As shown in Figures 10 and 11, two types of irradiance circumstances were examined in the experiment. The shading patterns considered for experimentation were horizontal (case I) and triangular corner (case II). These are similar to case II and case V in Section 2 but with different irradiance levels and array sizes. The shade concentrates before reconfiguring this array structure, as shown in Figure 10, for all the conventional configurations. After applying the proposed NSD reconfiguration, the shadow was distributed on the array, as shown in Figure 11.

**Figure 9.** Experimental setup. **Figure 9.** Experimental setup. configurations. After applying the proposed NSD reconfiguration, the shadow was distributed on the array, as shown in Figure 11. configurations. After applying the proposed NSD reconfiguration, the shadow was distributed on the array, as shown in Figure 11.


*Energies* **2022**, *15*, x FOR PEER REVIEW 13 of 18

*Energies* **2022**, *15*, x FOR PEER REVIEW 13 of 18

**1000 1000 1000 1000 1000 1000 1000 1000 Figure 10.** PSC before reconfiguration (**a**) case I, (**b**) case II. **Figure 10.** PSC before reconfiguration (**a**) case I, (**b**) case II. **Figure 10.** PSC before reconfiguration (**a**) case I, (**b**) case II.


**200 1000 1000 1000 500 1000 700 1000 Figure 11.** PSC after reconfiguration (**a**) case I, (**b**) case II. **Figure 11.** PSC after reconfiguration (**a**) case I, (**b**) case II. **Figure 11.** PSC after reconfiguration (**a**) case I, (**b**) case II.

#### **1000 300 1000 1000** *5.2. Experimental Result Analysis 5.2. Experimental Result Analysis 5.2. Experimental Result Analysis*

**1000 1000 1000 500 1000 1000 400 1000 700 500 1000 1000 300 1000 1000 1000** a) b) To perform the analysis of conventional configurations, SP, BL, HC, TCT, and the proposed NSD reconfiguration, the PV curve nature was obtained by changing the load resistance through DC Electronic load from 4000 Ω to 1 Ω. The curve nature was observed on an HDO4034A (USA) 350 MHz high-definition oscilloscope. To perform the analysis of conventional configurations, SP, BL, HC, TCT, and the proposed NSD reconfiguration, the PV curve nature was obtained by changing the load resistance through DC Electronic load from 4000 Ω to 1 Ω. The curve nature was observed on an HDO4034A (USA) 350 MHz high-definition oscilloscope. To perform the analysis of conventional configurations, SP, BL, HC, TCT, and the proposed NSD reconfiguration, the PV curve nature was obtained by changing the load resistance through DC Electronic load from 4000 Ω to 1 Ω. The curve nature was observed on an HDO4034A (USA) 350 MHz high-definition oscilloscope.

**1000 700 1000 1000**

#### **Figure 11.** PSC after reconfiguration (**a**) case I, (**b**) case II. 5.2.1. PSC Case I

*5.2. Experimental Result Analysis*  To perform the analysis of conventional configurations, SP, BL, HC, TCT, and the proposed NSD reconfiguration, the PV curve nature was obtained by changing the load resistance through DC Electronic load from 4000 Ω to 1 Ω. The curve nature was observed Figure 12 depicts the PV curves for each conventional configuration, NSD reconfiguration scheme, and the global MPP (the dot on the PV curve) under PSC case I. The PV curve shows GMPP for SP, BL, HC, TCT, and NSD at 59.22 W, 49.79 W, 49.79 W, 59.23 W, and 66.57 W, respectively. Moreover, SP and TCT have two peaks on the PV curve, BL and HC have three heights, and NSD reconfiguration reduces the number of peaks.

on an HDO4034A (USA) 350 MHz high-definition oscilloscope.

Figure 12 depicts the PV curves for each conventional configuration, NSD reconfiguration scheme, and the global MPP (the dot on the PV curve) under PSC case I. The PV curve shows GMPP for SP, BL, HC, TCT, and NSD at 59.22 W, 49.79 W, 49.79 W, 59.23 W, and 66.57 W, respectively. Moreover, SP and TCT have two peaks on the PV curve, BL and HC have three heights, and NSD reconfiguration reduces the number of peaks.

**Figure 12.** PV curves under PSC case I. **Figure 12.** PV curves under PSC case I.

#### 5.2.2. PSC Case II 5.2.2. PSC Case II

5.2.1. PSC Case I

Under this shading condition, Figure 13 shows that all conventional configurations have three peak PV curves. In contrast, the proposed NSD reconfiguration has one minor local peak and GMPP at a higher level with an improved curve nature. SP, BL, HC, TCT, and NSD provide GMPP at 52.92 W, 54.63 W, 53.75 W, 56.04 W, and 70.03 W, respectively. Figure 13 highlights that there is much improvement in the PV curve nature under this shading condition and GMPP location. Under this shading condition, Figure 13 shows that all conventional configurations have three peak PV curves. In contrast, the proposed NSD reconfiguration has one minor local peak and GMPP at a higher level with an improved curve nature. SP, BL, HC, TCT, and NSD provide GMPP at 52.92 W, 54.63 W, 53.75 W, 56.04 W, and 70.03 W, respectively. Figure 13 highlights that there is much improvement in the PV curve nature under this shading condition and GMPP location. *Energies* **2022**, *15*, x FOR PEER REVIEW 15 of 18

**Figure 13.** PV curves under PSC case II. **Figure 13.** PV curves under PSC case II.

#### **6. Conclusions 6. Conclusions**

spectively;

From the discussion mentioned above, this study concludes that: From the discussion mentioned above, this study concludes that:

• In partial shading cases, for the least irradiating case VIII and most irradiating case IX, NSD has maximum power extraction of 5160 W and 6952.8 W, respectively; the

• Under PSC for the considered PV array, TCT has the highest GMPP, %O/P, efficiency, FF, %PE, and least %MP at 5683 W, 74%, 18.54%, 0.5784, and 28.73%. Compared with TCT, NSD has 705.77 W, 9.2%, and 0.1139 enhancement in GMPP, %O/P,

• The results obtained from the first simulation show that the NSD method achieved the most significant output power and overall performance compared with all con-

• Moreover, experimental research was carried out. The suggested NSD PV array reconfiguration design beats the other conventional PV array configurations (SP, BL, HC, and TCT) in power increase and several peaks. The experimental setup vali-

**Author Contributions:** Conceptualization—V.C.C. and S.M.; Methodology, Software, Validation—V.C.C., S.M. and T.S.; Investigation, Resources, Writing—Original Draft Preparation, Writing—Review and Editing, Visualization; Supervision; Project Administration, V.C.C., S.M. and T.S.

**Funding:** The authors gratefully acknowledge the support offered by the Science and Engineering Research Board (SERB), Department of Science and Technology, Government of India, under the

dated the influence of panel shifting on various PV array topologies.

ventional configurations under nine partial shading cases;

All authors have read and agreed to the published version of the manuscript.

and FF, and a 2.41% reduction in MP;

Grant number: EEQ/2021/000294 for this research work. **Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable. **Data Availability Statement:** Not applicable.


**Author Contributions:** Conceptualization—V.C.C. and S.M.; Methodology, Software, Validation— V.C.C., S.M. and T.S.; Investigation, Resources, Writing—Original Draft Preparation, Writing—Review and Editing, Visualization; Supervision; Project Administration, V.C.C., S.M. and T.S. All authors have read and agreed to the published version of the manuscript.

**Funding:** The authors gratefully acknowledge the support offered by the Science and Engineering Research Board (SERB), Department of Science and Technology, Government of India, under the Grant number: EEQ/2021/000294 for this research work.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** Not applicable.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **Appendix A**

**Table A1.** PV Module electrical characteristics.


#### **Appendix B**

**Definitions**:

• **Global Maximum Power Point (GMPP):** GMPP is the single maximum point on the PV curve.

#### GMPP = max(multiple power Peaks)

• **Mislead Power (MP)**: MP denotes the difference between GMPP in the non-shading (NS) condition and GMPP in the shaded (PSC). It creates the hot spot effect and, due to that, power loss. It is provided by:

MP = GMPPNS − GMPPPSC

• **Percent Output (%O/P):** The %O/P is calculated as the maximum power ratio under PSC to extreme power under non-shading (NS) conditions. It provides how much energy is generated under PSC compared with power at standard irradiance. It reaches the %O/P of the PV array under PSC and NS conditions. It provides how much percentage of PSC affects the PV array output.

$$\% \text{O/P} = \frac{\text{P}\_{\text{max}(\text{PSC})}}{\text{P}\_{\text{max}(\text{NS})}}$$

• **Efficiency (**η**):** It is provided by the ratio output power (power generation) POut to input power (solar insolation) Pin On the PV panel. It is provided by:

$$\eta = \frac{\mathcal{P}\_{\text{Out}}}{\mathcal{P}\_{\text{in}}}$$

• **Fill Factor (FF):** The maximum power generated at PSC to the total rated capacity of a solar PV array is FF. It is also known as the usefulness of PV array. It is always 0 to 1. It is provided by:

$$\text{FF} = \frac{\text{V}\_{\text{M}}\text{I}\_{\text{M}}}{\text{V}\_{\text{OC}}\text{Is}}$$

• **Percent Power Enhancement (%PE)**: %PE is provided by the difference between the GMPP for the proposed NSD method and the GMPP for conventional configuration (CC) under partial shading conditions. It provides how much power is enhanced by NSD compared with CC under various irradiance conditions.

$$\% \text{PE} = \frac{\text{GMPP}\_{\text{NSD}} - \text{GMPP}\_{\text{CC}}}{\text{GMPP}\_{\text{CC}}}$$

#### **References**


## *Article* **Multiobjective Load Dispatch for Coal-Fired Power Plants under Renewable-Energy Accommodation Based on a Nondominated-Sorting Grey Wolf Optimizer Algorithm**

**Yue Cao 1,\*, Tao Li <sup>1</sup> , Tianyu He <sup>1</sup> , Yuwei Wei <sup>2</sup> , Ming Li <sup>3</sup> and Fengqi Si <sup>1</sup>**


**Abstract:** Coal-fired power plants are widely used to achieve a power balance in grids with renewable energy, which leads to new requirements for speediness in load dispatch. This paper presents a nondominated-sorting grey wolf optimizer algorithm (NSGWO) for the multiobjective load dispatch of coal-fired power plants that employed efficient nondominated sorting, a reference-point selection strategy, and a simulated binary crossover operator. The optimization results of the benchmark functions indicated that the NSGWO algorithm had a better accuracy and a better distribution than the traditional multiobjective grey wolf optimizer algorithm. Regarding the load dispatch of economy, environmental protection, and speediness strategies, the NSGWO had the best performance of all the simulated algorithms. The optimal-compromise solutions of the economy and speediness strategies of the NSGWO algorithm had a good distribution, which elucidated that this novel algorithm was favorable to allowing coal-fired power plants to accommodate renewable energy.

**Keywords:** load dispatch; multiobjective optimization; grey wolf optimizer; nondominated sorting; coal-fired power plant

## **1. Introduction**

Modern coal-fired power plants are clean and highly efficient, and their emissions of NOx and coal consumption are controlled at remarkably low levels [1]. Regarding the national energy distribution in China, coal-fired power plants still need to output power for basic loads in the grid. A coal-fired power plant may have different units (such as water-cooling and air-cooling units [2]) and units of various scales. Differences in unit characteristics may result in variable performances; therefore, it is necessary to optimize the operation strategy of coal-fired power plants with multiple units on the basis of both the economy and environmental-protection objectives. Furthermore, regarding the longterm objective of carbon neutrality [3], coal-fired power plants will play a new role in the power grid. Considering the development of renewable energy including solar power and wind power, coal-fired power plants, which have a strong capability to accommodate renewable energy, are widely used to achieve a power balance in power grids. In terms of the uncertainty of renewable energy [4], it is necessary to operate coal-fired power plants under conditions of rapid load change. Therefore, the optimal operation of coal-fired power plants should be further investigated while considering the factors of the economy, environmental protection, and speediness.

**Citation:** Cao, Y.; Li, T.; He, T.; Wei, Y.; Li, M.; Si, F. Multiobjective Load Dispatch for Coal-Fired Power Plants under Renewable-Energy Accommodation Based on a Nondominated-Sorting Grey Wolf Optimizer Algorithm. *Energies* **2022**, *15*, 2915. https://doi.org/10.3390/ en15082915

Academic Editor: Elyas Rakhshani

Received: 21 March 2022 Accepted: 13 April 2022 Published: 15 April 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

#### *1.1. Load-Dispatch Strategy*

Units of various types and scales in a coal-fired power plant usually receive the load command from the grid, which limits the optimal operation of the coal-fired power plant. Regarding the load-dispatch strategy, the coal-fired power plant receives the total load command from the grid, and then dispatches it to different units [5]. Based on previously mentioned factors, namely the economy, environmental protection, and speediness, the load dispatch of a coal-fired power plant is a multiobjective optimization of load command between different units.

Several researchers have employed optimal load-dispatch strategies for power generation systems. Dai et al. [6] proposed a multiobjective economic load-dispatch method for large coal-fired power plants using a hierarchical clustering and retrieval strategy based on the fuzzy c-means clustering algorithm. They concluded that the data-mining-based method was able to achieve plant-level optimal load dispatching while meeting the actual requirements of the grid. Ishraque et al. [7] introduced various optimization strategies of load dispatch for microgrid power systems with renewable energy sources. They considered that the proposed load-following method was the best dispatch strategy to achieve lower operating costs and lower pollutant emissions. Cui et al. [8] considered that the heat load and power load could be optimized simultaneously in microgrids. Regarding the differences in real-time response of heat load and power load, they performed short-term and long-term simulations and elucidated that the total cost could be reduced, as could the peak load of the power grid. Li et al. [9] also focused on investigating multiobjective load dispatch between the heat load and power load, and concluded that the proposed method could balance the production cost and pollutant-emission objectives under the fluctuation of the thermoelectric load demand. Xu et al. [10] introduced the load-dispatch strategy to save coal for coal-fired power plants using a support-vector machine. Their results indicated that the proposed load-dispatch strategy performed better than the normal separate load demand on a single unit in a coal-fired power plant. In addition, novel loaddispatch strategies have been studied for similar power-generation systems. Liao et al. [11] investigated the short-term load-dispatching method for a hydropower plant with multiple turbines, and simulation results indicated that it could achieve optimal operation while simultaneously meeting the practical operation requirements of the hydraulic and electric constraints. Jamal et al. [12] introduced a bioinspired computational heuristic algorithm for dispatch loads for a power-generation system that included coal-fired thermal-generating units and wind-power units. They considered that the proposed algorithm had advantages in accuracy, convergence, and robustness in solving load-dispatch problems. Furthermore, Bie et al. [13] conducted a load-dispatch investigation based on the uncertainty of renewable energy, and the results showed that the real-time load-dispatch model could calculate the load command while considering the primary and secondary regulation of the grid.

In general, much research has been published on load-dispatch optimization, but few studies considered the present application of a coal-fired power plant with renewableenergy accommodation, which is significant in terms of the multiobjective load dispatch of modern coal-fired power plants.

#### *1.2. Multiobjective Optimization Algorithms*

Among the load-dispatch strategies, researchers consider the optimization algorithm to be the key technology needed to achieve the objectives of energy saving and reducing pollutant emissions. In addition to the traditional optimization methods (including dynamic programming, Lagrange relaxation, and the exhaustive method), intelligent optimization algorithms have already been introduced in recent investigations.

Intelligent optimization algorithms usually employ an evolutionary mechanism or heuristic algorithm to enhance the convergence speed and robustness ability of the optimal solution procedure. Ghasemi et al. [14] proposed a differential evolution algorithm (DE) based on the mathematical model of sociopolitical evolution. They performed comparative simulations to verify the effectiveness of the algorithm, and concluded that this algorithm

was a reliable tool for solving the load-dispatch problem. Neto et al. [15] proposed a method that combined a differential-evolution algorithm and the greedy randomized adaptive search procedure algorithm. The simulation results showed that the algorithm could solve the nonsmooth load-dispatch problem by considering the effect of valve-point loading. Secui et al. [16] proposed a new heuristic algorithm called modified symbiosis (MSOS) that could solve the large-scale-unit load-optimization problem with a valve-point effect. They concluded that the proposed algorithm had a good performance based on the simulation results of five power-generation systems of different constraints and dimensions.

Furthermore, other heuristic algorithms, including the bat algorithm and the population extremum algorithm, have been introduced to solve the multiobjective optimization of load dispatch. Kavousi-Fard et al. [17] proposed an improved bat algorithm to solve the nonconvex load-optimization problem, which was then examined on IEEE test systems to show its high abilities and satisfying performance. Chen et al. [18] used the feasible-solution method to deal with constraints, and proposed a constrained multiobjective population extremal optimization algorithm called CMOPEO-EED. They considered the proposed algorithm to be a viable option to solve the load-dispatch problem with renewable power generation.

Particle-swarm optimization (PSO) is another commonly used algorithm in multiobjective optimization. Hosseinnezhad et al. [19] used an improved PSO algorithm (θ-PSO) to solve the economic load-optimization problem that could easily deal with various constraints. The findings of this study elucidated that the advantage of computational efficiency of the θ-PSO algorithm could be a promising alternative approach to solving the load-dispatch problem in practical power systems. Regarding the combined advantages of the PSO algorithm and other intelligent algorithms, researchers have investigated hybrid algorithms based on PSO. Narimani et al. [20] proposed a hybrid algorithm based on the PSO algorithm and leapfrog algorithm for multiregional economic and environmentalprotection load distribution. They verified the effectiveness of the hybrid algorithm using different test systems to balance the multiobjectives of generation costs and environmental issues. Zou et al. [21] proposed a new global particle-swarm-optimization algorithm, which adopted an updated method that relied on the global optimal particle to guide all particle-search activities and a slightly perturbed randomization method for particle flight trajectories with uniform distribution. Mandal et al. [22] introduced variable acceleration coefficients into the PSO algorithm and then proposed a self-organized hierarchical particleswarm-optimization algorithm. They found that the results of the proposed method were superior in terms of fuel costs, emission output, and losses.

In general, intelligent optimization algorithms have advantages in solving the multiobjective optimization problems of load dispatch at power plants, though their application under the conditions of renewable energy accommodation should be further investigated.

#### *1.3. Grey Wolf Optimizer*

Former studies show that various intelligent algorithms have been employed to solve the multioptimization issues in power-generation systems and similar applications. However, considering the rapid load-change conditions present under renewable-energy accommodation, the load dispatch of current coal-fired power plants requires rapid and accurate convergence.

The grey wolf optimizer (GWO) is a swarm-intelligence optimization algorithm proposed by Mirjalili et al. [23]. Compared with other natural heuristic optimization algorithms such as particle-swarm optimization and differential evolution, GWO has attracted attention due to its fast convergence speed, strong robustness, and high solution accuracy with a simple structure. Zhang et al. [24] used a GWO to plan the path of an unmanned combat aerial vehicle, and they concluded that it was more competent than other state-of-the-art evolutionary algorithms in terms of quality, speed, and stability of solutions. Although the grey wolf algorithm has superior performance on most data sets, it may not have the best performance on all. In view of the defects of a GWO, many scholars have made

relevant improvements to the performance of the GWO algorithm. Madhiarasan et al. [25] optimized the initial population of a GWO to improve the diversity of the population. Saremi et al. [26] improved the search mechanism of a GWO. The results indicated that the proposed GWO algorithm was able to significantly improve the performance of multiobjective optimization. Long et al. [27] proposed an updating method for the key parameter of a GWO in the application of constrained optimization problems. They considered that the simulations for both benchmark functions and engineering applications could demonstrate the good performance of the proposed GWO algorithm. Moreover, hybrid optimization algorithms combining GWO with other intelligent optimization algorithms have already been introduced by researchers [28–30]. However, few scholars have conducted in-depth research on the evolution mechanism of the GWO for the present application of loaddispatch optimization. Song et al. [31] introduced a GWO to solve the economic emission problem of two different power systems, and used a price penalty factor to combine the goals of economic dispatch and emission dispatch into a single function. Wong et al. [32] solved the economic scheduling problem under practical constraints with a GWO and tested them on two test systems with practical constraints. Moradi et al. [33] used a GWO to solve nonlinear and nonconvex economic scheduling problems while considering the valve-point effect and transfer loss, and compared the obtained results with some existing heuristic methods. Jangir et al. [34] investigated the economic constrained-emission dispatch problem based on a nondominated-sorting GWO, and they considered that it could be used in a power grid with wind power. However, they did not consider the speediness requirement of units in the coal-fired power plant, which indicated that the solving procedure of nondominated-sorting GWO should be further optimized.

The above shows that limited work has been conducted on the multiobjective optimization algorithm while considering the rapid and accurate convergence requirements under the unit rapid load-change condition. The more constraints that are considered, the more complex the load-dispatch optimization will be. Therefore, further research on the GWO needs to be carried out according to the specific problems of the multiobjective optimization of the load dispatch of coal-fired power plants.

#### *1.4. Present Investigation*

On the basis of the previous literature review, although much research has been published on load-dispatch optimization, few studies considered the present application of a coal-fired power plant with a renewable-energy accommodation. Moreover, limited work has been conducted on the multiobjective optimization algorithm that considered the rapid and accurate convergence requirements under the unit's rapid load-change condition. Therefore, an investigation into the multiobjective optimization of the load dispatch of a coal-fired power plant was conducted using a novel grey wolf algorithm.

In the present paper, the proposed nondominated-sorting grey wolf algorithm showed that it may help solve various optimization problems in similar applications. The organization of this study was arranged as follows. Section 2 shows the methods for the multiobjective optimization problem of load dispatch. Section 3 details the solving procedure of the novel grey wolf algorithm. Section 4 presents the simulation results and further discussion. Section 5 presents the conclusions of this study.

#### **2. Methods**

A theoretical model for the multiobjective optimization of the load dispatch of a coalfired power plant, as well as optimization constraints and performance evaluation indices, were proposed in order to achieve the requirements related to the economy, environmental protection, and speediness.

#### *2.1. Multiobjective Optimization Problem of a Coal-Fired Power Plant*

#### 2.1.1. Problem Description

The load-dispatch problem of a coal-fired power plant can generally be described as per Equation (1):

$$\begin{cases} \min \mathbf{F}(\mathbf{x}) = \min \left( \mathbf{f}\_1(\mathbf{x}), \mathbf{f}\_2(\mathbf{x}), \dots, \mathbf{f}\_m(\mathbf{x}) \right)^T\\ \text{s.t.} \begin{cases} \mathbf{g}\_i(\mathbf{x}) \le \mathbf{0}, i = 1, 2, \dots, p\\ \mathbf{h}\_j(\mathbf{x}) = \mathbf{0}, j = 1, 2, \dots, q \end{cases} \end{cases} \tag{1}$$

where F(*x*) is the objective function of the economy performance, and the sub-functions represent the economy characteristics of units in the coal-fired power plant. Further, g*<sup>i</sup>* (*x*) and h*j*(*x*) represent inequality constraints and equality constraints, respectively. Specifically, pollutant emission is an inequality constraint when it is not selected as an optimization objective.

For a coal-fired power plant, the relationship between the pollutant emission g*<sup>i</sup>* and the unit load *P<sup>i</sup>* can be expressed as the quadratic function relationship shown below

$$\mathbf{g}\_i(P\_i) = \mathbf{a}\_i P\_i^2 + \beta\_i P\_i + \gamma\_i \tag{2}$$

where *α<sup>i</sup>* , *β<sup>i</sup> ,* and *γ<sup>i</sup>* are the emission-characteristic coefficients of the unit.

Regarding the requirements of the relevant national policies of environmental protection, it is necessary to employ the environmental protection index as an optimization object instead of an inequality constraint. Thus, the objective function of load dispatch when considering environmental protection is defined as shown in Equation (3):

$$\min \mathcal{G} = \min \sum\_{i=1}^{N} \mathbf{g}\_i(P\_i) \tag{3}$$

where G represents the pollutant discharge of the whole plant.

The power grid has specific requirements for the load response rate of coal-fired power plants, specially under the condition of renewable-energy accommodation. Therefore, the speediness of load variation should be considered in the load dispatch of coal-fired power plants. The minimum time consumption of load dispatch under ideal conditions *Tideal* is shown in Equation (4):

$$T\_{ideal} = \left| P\_D - \sum\_{i=1}^{N} P\_{i,now} \right| / \sum\_{i=1}^{N} v\_i \tag{4}$$

where *P<sup>D</sup>* is the total load command of all units, MW; and *v<sup>i</sup>* represents the load changing speed of unit *i*, MW/min.

The sum of squares of the differences between the load-dispatch times of each unit and the minimum time under ideal conditions T(*P*) can be calculated as follows:

$$\text{T}(P) = \sum\_{i=1}^{N} \left( (P\_i - P\_{i,now}) / v\_i - T\_{ideal} \right)^2 \tag{5}$$

Therefore, combining former indicators with the economic indicators, the multiobjective optimization of load dispatch for a coal-fired power plant can be described as shown in Equation (6):

$$\begin{array}{l} \min(F, G, T), \min(F, G), \min(F, T) \\ \text{s.t.} \begin{cases} \sum\_{i=1}^{N} P\_i - P\_D \\ P\_i^{\min} \le P\_i \le P\_i^{\max} \\ \left| P\_i - P\_i^{\max} \right| / t\_i \le v\_i \end{cases} \end{array} \tag{6}$$

#### 2.1.2. Constraint-Violation Degree

The constraint-violation degree is introduced to show whether an individual *x* violates the *i*-th inequality constraint or the *j*-th equality constraint, which is expressed in Equations (7) and (8):

$$\text{Con}\_i(\mathbf{x}) = \max\{\mathbf{g}\_i(\mathbf{x}), \mathbf{0}\}, 1 \le i \le p \tag{7}$$

$$\mathcal{Con}\_j(\mathbf{x}) = \max\{h\_j(\mathbf{x}), 0\}, 1 \le j \le q \tag{8}$$

Thus, the total constraint-violation degree of individual *x* is shown in Equation (9):

$$\nu(\mathbf{x}) = \sum\_{i=1}^{p} \mathbb{C}on\_i(\mathbf{x}) + \sum\_{j=1}^{q} \mathbb{C}on\_j(\mathbf{x}) \tag{9}$$

In the multiobjective optimization of the load dispatch for a coal-fired power plant, the equation constraint describes that the sum of the loads of each unit must be equal to the total load command, while the inequality constraint shows the load-regulation range.

#### 2.1.3. Feasible and Infeasible Solutions

When an individual *x* satisfies both the inequality constraints and the equality constraints in the load-dispatch model, this individual is called a feasible solution; otherwise, it is an infeasible solution. The region composed of all feasible solutions is called the feasible region Ω, and the set composed of infeasible solutions is called the infeasible region.

#### 2.1.4. Dominance

For the two specific solutions *x*<sup>1</sup> and *x*2, the economy, speediness, and environmentalprotection indicators corresponding to solution *x*<sup>1</sup> are better than those corresponding to solution *x*2. Thus, solution *x*<sup>1</sup> is called a nondominated solution, while solution *x<sup>2</sup>* is called a dominant solution, which can be written as *x*<sup>1</sup> *x*2.

All individuals in the solution set that cannot be dominated by other individuals are called nondominated solutions, and this set of nondominated solutions is called the Pareto front (PF) of optimization solutions.

#### *2.2. Constraints for Multiobjective Optimization*

Constraint processing should be conducted to balance the relationship between the objective function and the constraints. In this paper, the penalty-function method was used to deal with constraints. By adding the penalty function *F*(*x*, *M*) into the objective function, the constrained optimization problem was transformed into an unconstrained optimization problem. Parameter *M* in the penalty function represents a large positive number, called the penalty factor, which acts as a penalty for the solution that violates the constraints. Using the external penalty function, the multiobjective function with the penalty function is expressed as follows:

$$\min F(\mathbf{x}, M) = \min \left( f\_1(\mathbf{x}) + M\_1 v(\mathbf{x}), f\_2(\mathbf{x}) + M\_2 v(\mathbf{x}), \dots, f\_m(\mathbf{x}) + M\_m v(\mathbf{x}) \right) \tag{10}$$

#### *2.3. Evaluation Index of Multiobjective Optimization*

For the multiobjective optimization of a load dispatch, evaluation indices were employed to compare the simulation results, which may have shown the convergence performance and comprehensive performance of the optimization algorithm.

#### 2.3.1. Convergence Evaluation Index

The generational distance (*GD*) was introduced to evaluate the convergence performance, which represented the degree of the solution set *P* to the real Pareto frontier *P\**, as shown in Equation (11):

$$GD(P, P^\*) = \frac{1}{|P|} \sqrt{\sum\_{i=1}^{|P|} d\_i^2} \tag{11}$$

where |*P*| is the number of solutions in *P*, and *d<sup>i</sup>* represents the Euclidean distance from the *i*-th solution in *P* to the nearest solution in *P\**. Therefore, a small *GD* value meant the convergence performance was better.

#### 2.3.2. Comprehensive Evaluation Index

The inverse generational distance (*IGD*) was used to represent the degree of the real Pareto frontier *P\** to the solution set *P* [35]. It evaluated the convergence and diversity of the algorithm simultaneously, which was defined as:

$$IGD(P, P^\*) = \frac{1}{|P^\*|} \sqrt{\sum\_{i=1}^{|P^\*|} d\_i'^2} \tag{12}$$

where *d* 0 *i* 2 represents the nearest Euclidean distance from the *i*-th individual in *P\** to the solution in *P*. Here, a small *IGD* showed that the optimization algorithm was better in both the convergence performance and the diversity of solutions.

#### 2.3.3. Optimal-Compromise Solution

For the multiobjective optimization of a load dispatch, the fuzzy set theory is suggested to find an optimal-compromise solution from the Pareto fronts. The fuzzy membership degree for the load-dispatch optimization can be expressed as:

$$\varphi\_{pi} = \begin{cases} 1 & f\_{pi} \le f\_i^{\min} \\ (f\_i^{\max} - f\_{pi}) / (f\_i^{\max} - f\_i^{\min}) & f\_i^{\min} < f\_{pi} < f\_i^{\max} \\ 0 & f\_{pi} \ge f\_i^{\max} \end{cases} \tag{13}$$

where *ϕpi* represents the satisfaction of the *i*-th objective of the *p*-th optimal solution, *i* ∈ {1, 2, . . . , *m*}; *fpi* represents the value of the *i*-th objective of the *p*-th optimal solution; and *f* max *i* and *f* min *i* represent the maximum and minimum values of the *i*-th objective in all optimal solutions, respectively. When *ϕpi* was equal to 1, it meant that the individual was completely satisfied with the *i*-th target value.

The standardized satisfaction calculation method of each solution *p* in the nondominated solution set is shown in Equation (14), where the solution corresponding to the largest *ϕpi* was the optimal-compromise solution:

$$\varphi\_p = \sum\_{k=1}^{m} \varphi\_{pk} / \sum\_{p=1}^{N} \sum\_{k=1}^{m} \varphi\_{pk} \tag{14}$$

Regarding the previous objectives and constraints of the multiobjective optimization of the load dispatch for a coal-fired power plant, the sequence diagram is shown in Figure 1. Figure 1.

Regarding the previous objectives and constraints of the multiobjective optimization of the load dispatch for a coal-fired power plant, the sequence diagram is shown in

**Figure 1.** Sequence diagram of multiobjective optimization of load dispatch of coal-fired power **Figure 1.** Sequence diagram of multiobjective optimization of load dispatch of coal-fired power plant.

#### plant. **3. Solving Procedure of the Optimization Algorithm**

**3. Solving Procedure of the Optimization Algorithm**  The multiobjective grey wolf algorithm was employed to find the load-dispatchoptimization solution for a coal-fired power plant. However, the issues of local optimum and insufficient stability may limit the performance of this optimization algorithm. In addition, the influence of the alpha wolf may result in a poor distribution of the obtained solutions., and the traditional nondominated sorting may consume more time in the iteration process. Considering the multiobjective optimization of a load dispatch with multiple complex constrains, a nondominated-sorting grey wolf optimizer (NSGWO) algorithm was proposed that employed efficient nondominated sorting (ENS), a reference-The multiobjective grey wolf algorithm was employed to find the load-dispatchoptimization solution for a coal-fired power plant. However, the issues of local optimum and insufficient stability may limit the performance of this optimization algorithm. In addition, the influence of the alpha wolf may result in a poor distribution of the obtained solutions., and the traditional nondominated sorting may consume more time in the iteration process. Considering the multiobjective optimization of a load dispatch with multiple complex constrains, a nondominated-sorting grey wolf optimizer (NSGWO) algorithm was proposed that employed efficient nondominated sorting (ENS), a referencepoint selection strategy, and a simulated binary crossover operator.

#### point selection strategy, and a simulated binary crossover operator. *3.1. Fast Nondominated Sorting Based on ENS*

lowing equation:

*3.1. Fast Nondominated Sorting Based on ENS*  Fast nondominated sorting is a cyclic stratification process. If this process is conducted in sequence, it will require too much time to converge. In fact, the frontier num-Fast nondominated sorting is a cyclic stratification process. If this process is conducted in sequence, it will require too much time to converge. In fact, the frontier number of individuals is determined by the dominated individual, which could infer the following equation:

$$front(p) = 1 + \max\_{q \in Q} front(q) \tag{15}$$

q Q *front front* (p) 1 max (q) <sup>∈</sup> = + (15) where *f ront*(*p*) represents the number of the frontier where individual *p* is located, and *Q* is the set composed of all individuals that dominate individual *p*.

where *front p*( ) represents the number of the frontier where individual *p* is located, and *Q* is the set composed of all individuals that dominate individual *p*. The first step of the ENS was to arrange all individuals in the population according to their first-dimension target values from small to large. If the first-dimension target values were the same, we compared their second-dimension index values, and so on. In the sorted population, it was impossible for the individual in the latter order to domi-The first step of the ENS was to arrange all individuals in the population according to their first-dimension target values from small to large. If the first-dimension target values were the same, we compared their second-dimension index values, and so on. In the sorted population, it was impossible for the individual in the latter order to dominate the individual in front of it. This meant that the individual that dominated *p* must have been assigned to the frontier when examining *p*. The number of the frontier where *p* is located was calculated using Equation (15).

nate the individual in front of it. This meant that the individual that dominated *p* must have been assigned to the frontier when examining *p*. The number of the frontier where *p* is located was calculated using Equation (15). On this basis, starting from the first frontier *F*1, it was judged in turn whether each frontier contained an individual that dominated *p*. The first frontier that does not contain the dominant *p* was the frontier where *p* resides. If there were individuals dominating p up to the last frontier *F<sup>k</sup>* , then *p* was assigned to a new frontier *Fk*+1. If all individuals were mutually nondominant, then a nondominant comparison was required between them. In this case, since there was only one frontier, the complexity was *Tworst* = *m* · *N*(*N* −

1)/2 = *O*(*mN*<sup>2</sup> ). The algorithm could achieve the best performance if the population satisfied the following conditions: there were <sup>√</sup> *N* frontiers in total, and each frontier contained <sup>√</sup> *N* individuals. Moreover, an individual was dominated by all individuals whose frontier number was smaller than its under this condition. For a specific individual *p*, if it belonged to frontier *F<sup>i</sup>* , it was necessary to find an individual dominating it in each frontier *Fi*(*j* < *i*). This indicated that at least *i* − 1 nondominated comparisons were required. For all <sup>√</sup> *N* individuals in *F<sup>i</sup>* , a total of <sup>√</sup> *N*(*i* + 1) comparisons were required. In addition, all individuals of √ *F<sup>i</sup>* needed to be compared pairwise, which meant that a total of *N*( √ *N* − 1)/2 comparisons were required. Considering all frontiers, the best period of complexity is shown in Equation (16):

$$T\_{\rm best} = m \cdot \sum\_{i=1}^{\sqrt{N}} \left[ \sqrt{N}(i-1) + \sqrt{N}(\sqrt{N}-1)/2 \right] = m \cdot \left[ N(\sqrt{N}-1)/2 + N(\sqrt{N}-1)/2 \right] = O(\text{MN}^{1.5}) \tag{16}$$

#### *3.2. Selection Strategy Based on Reference Point*

To select *N* individuals from the set of offspring and parents R*<sup>t</sup>* = *Pt*∪*Q<sup>t</sup>* , it should firstly divide R*<sup>t</sup>* into multiple nondominated layers (*F*1, *F*2, . . . , *F<sup>k</sup>* ) by nondominated sorting. Then, a new population *S<sup>t</sup>* is established from the start of *F<sup>1</sup>* until its size is equal to or exceeds *N* for the first time. In order to maintain the diversity of solutions, a selection strategy based on reference point was used to select *n<sup>L</sup>* = *N* − ∑ *L*−1 *<sup>i</sup>*=<sup>1</sup> *N<sup>i</sup>* individuals, where *Ni* is the number of individuals in the *i*-th frontier, and *n<sup>L</sup>* is the number of individuals that needed to be selected from the last frontier satisfying the former requirements. Moreover, the systematic approach was introduced to generate a set of uniform weight vectors {*λ*1, *λ*2, . . . , *λR*} so that the reference points could be determined. In order to compare the target values of each dimension, it was necessary to adaptively normalize the target value of each dimension. The ideal point <sup>−</sup> z = (*z* min 1 , *z* min 2 , . . . , *z* min *<sup>m</sup>* ) of defining population *S<sup>t</sup>* was composed of the minimum value *z* min 1 , *i* = 1, 2, . . . , *m* of each dimension of *S<sup>t</sup>* . Based on the ideal point, the objective function could be transformed to *f* 0 *i* (*x*) = *fi*(*x*) − *z* min *i* . Then, a hyperplane could be built by extra points obtained from the objective function; this process can be expressed as:

$$\text{ASF}(\mathbf{x}, w) = \max\_{i=1}^{m} f\_i'(\mathbf{x}) / w\_{i\nu} \mathbf{x} \in \mathbb{S}\_t \tag{17}$$

$$Z^{i, \text{max}} = s: \text{argmin}\_{\text{s} \in \text{S}\_{\text{I}}} ASE(s, w), w = (\tau, \dots, \tau), \tau = 10^{-6} \tag{18}$$

For the *i*-th target, an extra target vector *z <sup>i</sup>*,max was generated. Thus, *m* extra vectors were generated by *m* targets which further constituted an *m*-dimensional linear hyperplane, and then the intercept *a<sup>i</sup>* , *i* = 1, 2, . . . , *m* could be calculated. The objective function can be normalized as shown in Equation (19):

$$f\_i^n(\mathbf{x}) = (f\_i'(\mathbf{x}) - z\_i^{\min}) / (a\_i - z\_i^{\min}) \tag{19}$$

#### *3.3. Simulated Binary Crossover Operator*

Under the influence of the top three levels of wolves, a GWO easily falls into a local optimum. Therefore, a simulated binary crossover (SBX) operator was introduced in this paper. Assuming that the two parent individuals are *x* 1 (*x* 1 1 , *x* 1 2 , . . . *x* 1 *n* ) and *x* 2 (*x* 2 1 , *x* 2 2 , . . . *x* 2 *n* ), the SBX operator could generate the offspring individuals *c* 1 (*c* 1 1 , *c* 1 2 , . . . *c* 1 *n* ) and *c* 2 (*c* 2 1 , *c* 2 2 , . . . *c* 2 *n* ) using the following equations:

$$\begin{cases} \ c\_i^1 = 0.5 \times \left[ (1+\beta) \cdot x\_i^1 + (1-\beta) \cdot x\_i^2 \right] \\\ c\_i^2 = 0.5 \times \left[ (1-\beta) \cdot x\_i^1 + (1+\beta) \cdot x\_i^2 \right] \end{cases} \tag{20}$$

where *β* is randomly determined by the distribution factor *η* according to Equation (21):

$$\beta = \begin{cases} \left(rand \times 2\right)^{1/(1+\eta)}, rand \le 0.5\\ \left(1/\left(2 - rand \times 2\right)\right)^{1/(1+\eta)}, rand > 0.5 \end{cases} \tag{21}$$

With the increase in iterations, the probability of generating individuals according to the grey wolf hunting mechanism increased, thereby enhancing the development ability and convergence of the algorithm. In this paper, the variable a was controlled by a nonlinear decreasing strategy to balance the exploration and development capabilities of the algorithm, as shown in Equation (22): With the increase in iterations, the probability of generating individuals according to the grey wolf hunting mechanism increased, thereby enhancing the development ability and convergence of the algorithm. In this paper, the variable a was controlled by a nonlinear decreasing strategy to balance the exploration and development capabilities of the algorithm, as shown in Equation (22):

Under the influence of the top three levels of wolves, a GWO easily falls into a local optimum. Therefore, a simulated binary crossover (SBX) operator was introduced in this

1 2 ( , ,... ) *<sup>n</sup> x xx x* , the SBX operator could generate the offspring individuals

1 12 2 12 0.5 [(1 ) (1 ) ] 0.5 [(1 ) (1 ) ] *i ii i ii*

 β

1/(1 )

+

η

(1/ (2 2)) , 0.5

*rand rand*

 β

(20)

*c xx c xx* β

β

where *β* is randomly determined by the distribution factor *η* according to Equation (21): 1/(1 )

( 2) , 0.5

+

η

−× >

*rand rand*

× ≤ <sup>=</sup>

1 2 ( , ,... ) *<sup>n</sup> ccc c* using the following equations:

 = × + +− = × − ++

paper. Assuming that the two parent individuals are 111 1

$$a = 2(1 - (t/iter)^2) \tag{22}$$

1 2 ( , ,... ) *<sup>n</sup> x xx x* and

(21)

where *iter* is the maximum number of iterations. where *iter* is the maximum number of iterations.

β

*Energies* **2022**, *15*, x FOR PEER REVIEW 10 of 20

*3.3. Simulated Binary Crossover Operator* 

1 2 ( , ,... ) *<sup>n</sup> ccc c* and 222 2

222 2

111 1

#### *3.4. Solving Procedure of NSGWO 3.4. Solving Procedure of NSGWO*

The solving procedure of the proposed nondominated-sorting grey wolf optimizer algorithm is presented in Figure 2. There were five specific steps of this algorithm, which are detailed as follows. The solving procedure of the proposed nondominated-sorting grey wolf optimizer algorithm is presented in Figure 2. There were five specific steps of this algorithm, which are detailed as follows.

**Figure 2.** Solving procedure of nondominated-sorting grey wolf optimizer algorithm. **Figure 2.** Solving procedure of nondominated-sorting grey wolf optimizer algorithm.

Step 1: Set the number *N* of individuals in the grey wolf population and the maximum number of iterations *iter*. Initialize parameters *a*, *A,* and *C*. Step 1: Set the number *N* of individuals in the grey wolf population and the maximum number of iterations *iter*. Initialize parameters *a*, *A*, and *C*.

Step 2: Initialize the grey wolf population *P*, and generate a reference point *Z*.

Step 2: Initialize the grey wolf population *P*, and generate a reference point *Z*. Step 3: Calculate the target value of each dimension of individuals in the popula-Step 3: Calculate the target value of each dimension of individuals in the population.

tion. Step 4: Perform the nondominated sorting based on ENS, and select the first three levels of wolves *α*, *β*, *δ*.

Step 5: Update the population *P<sup>t</sup>* and generate the offspring *Q<sup>t</sup>* . When the random number is greater than the set probability threshold, binary crossover mutation is conducted. Otherwise, update the population based on the evolution mechanism of GWO.

Step 6: Select the first *N* individuals based on the reference point in the merged population R*<sup>t</sup>* = *Pt*∪*Q<sup>t</sup>* , and then generate a new population *S<sup>t</sup>* .

Step 7: If the maximum number of iterations is not reached, repeat steps (2–4). Otherwise, output the optimization results.

#### **4. Results and Discussion**

#### *4.1. NSGWO Algorithm Performance Test*

In order to test and evaluate the performance of the NSGWO algorithm, three benchmark functions [36] were employed in this paper, the detailed structures of which are shown in Table 1.


**Table 1.** Benchmark functions for multiobjective optimization test of NSGWO algorithm.

Then, test simulations were conducted for the multiobjective optimization algorithm; the results for the *GD* and *IGD* values of the former benchmark functions are presented in Tables 2 and 3, respectively. Here, the results of the multiobjective grey wolf optimizer (MOGWO) and the multiobjective particle-swarm optimizer (MOPSO) are also shown for performance comparisons. For the unimodal function ZDT1, the *IGD* value of the NSGWO optimization result was 0.00690, which was an order of magnitude smaller than the results of other optimization algorithms, indicating that the improved algorithm had a strong ability to jump out of the local optimum. The UF series functions were all multimodal functions, which provided many local optimal solutions, and therefore the requirements for the algorithm were relatively high. Regardless of the *IGD* value or the *GD* value, the results of the improved algorithm were better than those of the other algorithms in the UF4 function. However, the *IGD* of the improved algorithm was smaller, as was the difference between the maximum and minimum values of the *GD* value, indicating that the improved algorithm had a better stability. The above test results showed that the improved algorithm NSGWO had good stability and the ability to jump out of the local optimum.

**Table 2.** *GD* values of optimization results for benchmark functions.



**Table 3.** *IGD* values of optimization results for benchmark functions.

Figure 3 shows the Pareto fronts of optimization results of the benchmark function, in which the horizontal and vertical coordinates represent a certain target value of the benchmark function. For the ZDT1 function, the solutions of MOGWO and NSGWO were close to the real Pareto fronts. For the UF2 function, most of the MOGWO solutions were far from the real Pareto fronts, while most of the NSGWO solutions were concentrated on the real Pareto fronts. Moreover, the results indicated that the UF4 function was relatively complex. Although the optimization results of the NSGWO algorithm in Figure 3c were not close to the real Pareto fronts, they were still better than those of MOGWO. In general, the NSGWO had better accuracy and a better distribution of the obtained solutions. *Energies* **2022**, *15*, x FOR PEER REVIEW 13 of 20

**Figure 3.** Pareto fronts of multiobjective optimization algorithms for benchmark functions. (**a**) ZDT1; (**b**) UF2; (**c**) UF4. **Figure 3.** Pareto fronts of multiobjective optimization algorithms for benchmark functions. (**a**) ZDT1; (**b**) UF2; (**c**) UF4.

#### *4.2. Multiobjective Optimization of Load Dispatch Based on NSGWO 4.2. Multiobjective Optimization of Load Dispatch Based on NSGWO*

The NSGWO algorithm was employed in the multiobjective optimization of the load dispatch for coal-fired power plants while considering the economy, environmental protection, and speediness. To better show the performance of the NSGWO algorithm in load dispatch, four intelligent algorithms, including the real-number coding genetic algorithm (RCGA), particle-swarm optimizer (PSO), modified nondominated-sorting genetic algorithm (NSGA-II), and MOGWO, were employed in the following simulations. Their configurations in terms of the population size and maximum number of iterations are summarized in Table 4, and were selected through analyses of presimulations. The NSGWO algorithm was employed in the multiobjective optimization of the load dispatch for coal-fired power plants while considering the economy, environmental protection, and speediness. To better show the performance of the NSGWO algorithm in load dispatch, four intelligent algorithms, including the real-number coding genetic algorithm (RCGA), particle-swarm optimizer (PSO), modified nondominated-sorting genetic algorithm (NSGA-II), and MOGWO, were employed in the following simulations. Their configurations in terms of the population size and maximum number of iterations are summarized in Table 4, and were selected through analyses of presimulations.

**Table 4.** Configurations of NSGWO and other intelligent algorithms used in simulations of multi-

**Simulation I** <sup>1</sup> **Simulation II** <sup>1</sup> **Simulation III** <sup>1</sup> **NSGWO RCGA PSO NSGWO NSGA-II MOGWO NSOGWO MOGWO** 

1 Simulations I-III represent the optimization of economy and environmental protection, the opti-

The NSGWO algorithm was employed for the multiobjective optimization of the load dispatch for coal-fired power plants while considering economy and environmental protection. The penalty function was used in the following simulations to deal with the equality constraints in the load-dispatch model. Firstly, a coal-fired power plant with six units was investigated; its coal consumption and pollutant-emission characteristics are listed in Table 5 [37]. The total command of these six units was 283.4 MW, and their load-changing speed was limited to 5% of the current output power. To better show the performance of the NSGWO algorithm in the load dispatch, the RCGA and PSO were introduced for a comparison with the optimization results. Then, the simulations were

4.2.1. Optimization of Economy and Environmental Protection

omy and speediness for similar-scale units, respectively.

conducted; the results are shown in Tables 6 and 7.

Population size 200 200 200 100 100 100 100 100 Number of iterations 500 500 500 300 300 300 1000 1000

objective optimization.


**Table 4.** Configurations of NSGWO and other intelligent algorithms used in simulations of multiobjective optimization.

<sup>1</sup> Simulations I–III represent the optimization of economy and environmental protection, the optimization of economy and speediness for various power-scale units, and the optimization of economy and speediness for similar-scale units, respectively.

#### 4.2.1. Optimization of Economy and Environmental Protection

The NSGWO algorithm was employed for the multiobjective optimization of the load dispatch for coal-fired power plants while considering economy and environmental protection. The penalty function was used in the following simulations to deal with the equality constraints in the load-dispatch model. Firstly, a coal-fired power plant with six units was investigated; its coal consumption and pollutant-emission characteristics are listed in Table 5 [37]. The total command of these six units was 283.4 MW, and their load-changing speed was limited to 5% of the current output power. To better show the performance of the NSGWO algorithm in the load dispatch, the RCGA and PSO were introduced for a comparison with the optimization results. Then, the simulations were conducted; the results are shown in Tables 6 and 7.

**Table 5.** Characteristics of dispatched units based on economy and environmental protection.


<sup>1</sup> Variables *P* min *i* and *P* max *i* represent lower limit and upper limit of output power.

**Table 6.** Optimization results of economic load dispatch for coal-fired power plant without valvepoint effect of steam turbines.


The results showed that the performance of the NSGWO algorithm was better than that of the other algorithms in reducing power-plant coal consumption and power-plant emissions, which indicated the advantages of the NSGWO algorithm in the optimal dispatch of multiobjective economic and environmental-protection loads. Tables 6 and 7 also show that the valve-point effect of steam turbines had a great influence on the optimal dispatch results. The optimization results of Unit 1 and Unit 6 in the two models were quite different. When the valve-point effect was considered, the optimal dispatch results of these two units were 21.4 MW and 41.0 MW, respectively. However, the optimal dispatch results changed to 15.7 MW and 35.2 MW without considering the valve-point effect of the steam turbines.


**Table 7.** Optimization results of load dispatch under economy and environmental-protection strategies for coal-fired power plant considering valve-point effect of steam turbines.

In order to further illustrate the advantages of NSGWO, the Pareto fronts were determined for the NSGWO and MOGWO algorithms without considering the valve-point effect, as shown in Figure 4. The solutions of NSGWO were more widely distributed, and the solutions of MOGWO were basically dominated by the solutions of NSGWO. The comparison results show that the NSGWO algorithm had a certain application value in the multiobjective optimization of the load dispatch for coal-fired power plants. *Energies* **2022**, *15*, x FOR PEER REVIEW 15 of 20

**Figure 4.** Pareto fronts of coal-consumption cost and pollutant emissions of load-dispatch optimization under NSGWO and MOGWO algorithms. **Figure 4.** Pareto fronts of coal-consumption cost and pollutant emissions of load-dispatch optimization under NSGWO and MOGWO algorithms.

4.2.2. Optimization of Economy and Speediness for Various Power-Scale Units 4.2.2. Optimization of Economy and Speediness for Various Power-Scale Units

Considering the requirements of rapid load change of coal-fired power plants under a renewable-energy accommodation, simulations were conducted to show the performance of the NSGWO algorithm in the multiobjective optimization of economy and speediness. Here, 10 units of various power scales are selected to be dispatched in the following simulations, the characteristics of which are shown in Table 8 [38]. The total dispatch command of the 10-unit power plant was 1500 MW, and the penalty-factor values of the two-dimensional objective of economy and speediness in the NSGWO algorithm were 0.5 and 2.1, respectively. Table 9 presents the optimization results of the load dispatch under the economy and speediness strategies. To better explain the advantages of nondominated sorting, the optimization of the NSGA-II was also performed, as Considering the requirements of rapid load change of coal-fired power plants under a renewable-energy accommodation, simulations were conducted to show the performance of the NSGWO algorithm in the multiobjective optimization of economy and speediness. Here, 10 units of various power scales are selected to be dispatched in the following simulations, the characteristics of which are shown in Table 8 [38]. The total dispatch command of the 10-unit power plant was 1500 MW, and the penalty-factor values of the two-dimensional objective of economy and speediness in the NSGWO algorithm were 0.5 and 2.1, respectively. Table 9 presents the optimization results of the load dispatch under the economy and speediness strategies. To better explain the advantages of nondominated sorting, the optimization of the NSGA-II was also performed, as shown in Table 9.

**Table 8.** Characteristics of 10 dispatched units with various power scales based on economy and

11 1 1 *fP P P* ( ) 0.00510 2.2034 15 = ++ 15 7 15 60

22 2 2 *fP P P* ( ) 0.00396 1.9101 25 = ++ 20 8 20 80

33 3 3 *fP P P* ( ) 0.00393 1.8518 40 = ++ 30 10 30 100

44 4 4 *fP P P* ( ) 0.00382 1.6966 32 = ++ 35.8 15 25 120

55 5 5 *fP P P* ( ) 0.00212 1.8015 29 = ++ 50.3 8 50 150

66 6 6 *fP P P* ( ) 0.00261 1.5354 72 = ++ 75.4 6 75 280

77 7 7 *fP P P* ( ) 0.00289 1.2643 49 = ++ 124 11 120 320

88 8 8 *fP P P* ( ) 0.00148 1.2130 82 = ++ 251.85 10 125 445

99 9 9 *fP P P* ( ) 0.00127 1.1954 105 = ++ 298.75 12 250 520

10 10 10 10 *fP P P* ( ) 0.00135 1.1285 100 = ++ 298.9 9 250 550

shown in Table 9.

speediness.

1 <sup>2</sup>

2 <sup>2</sup>

3 <sup>2</sup>

4 <sup>2</sup>

5 <sup>2</sup>

6 <sup>2</sup>

7 <sup>2</sup>

8 <sup>2</sup>

9 <sup>2</sup>

10 <sup>2</sup>


**Table 8.** Characteristics of 10 dispatched units with various power scales based on economy and speediness.

**Table 9.** Optimization results of load dispatch under economy and speediness strategies for 10 units of various power scales.


<sup>1</sup> Variables *f<sup>h</sup>* and *T* represent coal consumption per hour and adjustment time, respectively.

The results indicated that the optimal solutions of the NSGWO algorithm had the minimum adjustment times. For the minimum coal consumption of the economy strategy, the optimal solutions of the NSGWO algorithm were better than those of the other algorithms. Moreover, we found that the optimal-compromise solution of the NSGWO algorithm was able to dominate the solutions of the NSGA-II algorithm.

Figure 5 shows the Pareto fronts of the NSGWO algorithm and the MOGWO algorithm under the combined dispatch of economy and speediness. Here, a speediness index was employed to better present the optimal solutions that was defined as the square of the adjustment time. It can be seen that the optimization results of the NSGWO algorithm were more widely distributed than those of the MOGWO algorithm. The solution that dominated the optimization results of the MOGWO algorithm was found in the Pareto fronts of the NSGWO algorithm. In general, this indicated that the performance of the NSGWO algorithm in the load-dispatch optimization of economy and speediness was the best among the simulated algorithms.

was the best among the simulated algorithms.

**Figure 5.** Pareto fronts of coal consumption and speediness index of load-dispatch optimization for 10 units with various power scales under NSGWO and MOGWO algorithms. **Figure 5.** Pareto fronts of coal consumption and speediness index of load-dispatch optimization for 10 units with various power scales under NSGWO and MOGWO algorithms.

**Table 9.** Optimization results of load dispatch under economy and speediness strategies for 10

units of various power scales.

**Item Optimal-Compromise Solution Economy Strategy Speediness Strategy** 

*fh*/(th −1) 1 3118.20 3120.77 3124.15 3115.02 3115.22 3123.25 3130.11 3131.29 3126.27 *T*/min 1 4.91 4.98 4.49 7.48 7.12 5.13 3.52 3.59 3.77 *P1*/MW 15.64 20.96 24.48 15.14 15.00 23.50 34.21 35.94 25.66 *P2*/MW 37.58 37.88 45.04 26.00 27.33 45.24 44.566 45.50 46.30 *P3*/MW 45.16 48.03 48.21 36.20 34.21 45.48 59.17 61.00 52.29 *P4*/MW 65.82 69.80 76.64 58.34 55.86 76.33 80.19 70.07 77.91 *P5*/MW 79.48 76.81 76.43 76.74 82.85 75.24 77.36 78.82 78.27 *P6*/MW 101.45 94.49 97.10 109.45 109.63 95.54 94.91 96.26 97.04 *P7*/MW 159.34 156.95 152.18 146.84 149.06 152.97 157.04 155.41 156.39 *P8*/MW 298.22 294.80 296.78 302.62 306.39 301.44 284.90 286.40 289.48 *P9*/MW 354.25 358.46 345.97 364.46 356.67 339.19 339.36 341.79 344.03 *P10*/MW 343.06 341.81 337.18 364.15 363.01 345.06 328.23 328.81 332.64

**NSGWO NSGA-II MOGWO NSGWO NSGA-II MOGWO NSGWO NSGA-II MOGWO** 

1 Variables *fh* and *T* represent coal consumption per hour and adjustment time, respectively.

algorithm was able to dominate the solutions of the NSGA-II algorithm.

The results indicated that the optimal solutions of the NSGWO algorithm had the minimum adjustment times. For the minimum coal consumption of the economy strategy, the optimal solutions of the NSGWO algorithm were better than those of the other algorithms. Moreover, we found that the optimal-compromise solution of the NSGWO

Figure 5 shows the Pareto fronts of the NSGWO algorithm and the MOGWO algorithm under the combined dispatch of economy and speediness. Here, a speediness index was employed to better present the optimal solutions that was defined as the square of the adjustment time. It can be seen that the optimization results of the NSGWO algorithm were more widely distributed than those of the MOGWO algorithm. The solution that dominated the optimization results of the MOGWO algorithm was found in the Pareto fronts of the NSGWO algorithm. In general, this indicated that the performance of the NSGWO algorithm in the load-dispatch optimization of economy and speediness

#### 4.2.3. Optimization of Economy and Speediness for Similar-Scale Units

Moreover, a real coal-fired power plant with eight 600 MW units and two 660 MW units was investigated under a renewable-energy accommodation, which may further explain the performance of the NSGWO algorithm in the load dispatch. These 10 units had similar power scales, so a higher optimization accuracy was needed in the algorithm. The coal-consumption characteristics, as well as other initial conditions and constraints, are shown in Table 10, in which the unit for coal consumption is grams of coal per kWh. In addition, the penalty-factor values of the two-dimensional targets of economy and speediness in the algorithms were 0.5 and 1.1, respectively, and the total power command of this 10-unit power plant was 4000 MW.

**Table 10.** Characteristics of 10 dispatched units based on economy and speediness with 10 similarscale units.


Table 11 shows the optimization results for the real coal-fired power plant with 10 similar-scale units. It indicates that the minimum coal consumption of NSGWO was 328 g/(kWh), which was better than that of MOGWO. The minimum adjustment time of NSGWO was 1.36 min, which was also shorter than that of MOGWO. Furthermore, the solutions of the MOGWO algorithm for these two minima were dominated by the NSGWO algorithm, which meant that the distribution of the solutions of the NSGWO algorithm was better than that of the MOGWO algorithm.


**Table 11.** Optimization results of load dispatch under economy and speediness strategies for a real coal-fired power plant with 10 similar-scale units.

<sup>1</sup> Variables *f<sup>w</sup>* and *T* represent coal consumption per kWh and adjustment time, respectively.

Figure 6 illustrates the Pareto fronts of coal consumption and adjustment time of the former simulations for NSGWO and MOGWO. It shows that the optimization results of NSGWO were basically at the lower left of MOGWO, indicating that the optimization effect of NSGWO was better. Most of the solutions of the MOGWO algorithm are concentrated in the upper left, while the solutions of the NSGWO algorithm are more uniformly distributed. This indicates that the solutions of the NSGWO algorithm had a better distribution in the combined load dispatch while considering the requirements of both economy and speediness for coal-fired power plants. Figure 6 illustrates the Pareto fronts of coal consumption and adjustment time of the former simulations for NSGWO and MOGWO. It shows that the optimization results of NSGWO were basically at the lower left of MOGWO, indicating that the optimization effect of NSGWO was better. Most of the solutions of the MOGWO algorithm are concentrated in the upper left, while the solutions of the NSGWO algorithm are more uniformly distributed. This indicates that the solutions of the NSGWO algorithm had a better distribution in the combined load dispatch while considering the requirements of both economy and speediness for coal-fired power plants.

**Figure 6.** Pareto fronts of coal consumption and adjustment time of load-dispatch optimization for coal-fired power plant with 10 similar-scale units under NSGWO and MOGWO algorithms. **Figure 6.** Pareto fronts of coal consumption and adjustment time of load-dispatch optimization for coal-fired power plant with 10 similar-scale units under NSGWO and MOGWO algorithms.

The results of this study showed that the proposed nondominated-sorting GWO algorithm achieved high accuracy and had a good distribution of solutions, which may extend its applications in multiobjective optimization. We caution that the presented analyses for the NSGWO and other intelligent algorithms were somewhat simplified, and further investigations are needed. The optimization procedure proposed in this study may help researchers conduct other investigations into the optimization of multiobjective problems and the operation of power-generation systems. The results of this study showed that the proposed nondominated-sorting GWO algorithm achieved high accuracy and had a good distribution of solutions, which may extend its applications in multiobjective optimization. We caution that the presented analyses for the NSGWO and other intelligent algorithms were somewhat simplified, and further investigations are needed. The optimization procedure proposed in this study may help researchers conduct other investigations into the optimization of multiobjective problems and the operation of power-generation systems.

dation were conducted based on the NSGWO algorithm. On the basis of the performed

(1) The proposed NSGWO algorithm employed efficient nondominated sorting, a reference-point selection strategy, and a simulated binary crossover operator in order to avoid falling into the local optimum, enhance the solution diversity, and shorten the convergence time. The optimization results of benchmark functions indicated that the NSGWO algorithm had a better accuracy and a better distribution than the

(2) Regarding the load dispatch of economy and environmental-protection strategies, the performance of the NSGWO algorithm was better than those of the other simulated algorithms in reducing power-plant coal consumption and emissions. In addition, the widely distributed solutions of the NSGWO algorithm could dominate those of the MOGWO algorithm, which elucidated the application value of the NSGWO algorithm in the multiobjective load dispatch of coal-fired power plants.

analyses, the conclusions of this paper can be summarized as follows:

**5. Conclusions** 

MOGWO algorithm.

#### **5. Conclusions**

Multiobjective optimizations of load dispatch under a renewable energy accommodation were conducted based on the NSGWO algorithm. On the basis of the performed analyses, the conclusions of this paper can be summarized as follows:


**Author Contributions:** Conceptualization, Y.C. and F.S.; methodology, Y.C. and T.L.; software, T.L. and T.H.; validation, T.H.; formal analysis, Y.W. and M.L.; investigation, Y.C. and T.L.; writing original draft preparation, Y.C.; writing—review and editing, T.H. and F.S. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded by the Natural Science Foundation of Jiangsu Province (grant number BK20210240).

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** The data presented in this study are available upon request from the corresponding author.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


## *Article* **Reserve-Constrained Unit Commitment Considering Adjustable-Speed Pumped-Storage Hydropower and Its Economic Effect in Korean Power System**

**Woo-Jung Kim <sup>1</sup> , Yu-Seok Lee <sup>1</sup> , Yeong-Han Chun 1,\* and Hae-Seong Jeong <sup>2</sup>**


**Abstract:** The Korean government has declared the goal of net-zero-carbon emissions with a focus on renewable energy expansion. However, a high proportion of baseload generators and an increasing proportion of variable renewable energy (VRE) may cause problems in the power system operation owing to the low cycling capability of baseload generators and variability of VRE. To maintain system reliability, the government is planning to construct pumped-storage hydropower (PSH) plants, which can provide flexibility to the system. This study evaluated the operating cost savings obtained by different types of PSH: the adjustable-speed PSH (AS-PSH) and fixed-speed PSH (FS-PSH), based on the duck-curve phenomenon and the increase in spinning reserve requirement. In this study, the reserve-constrained unit commitment was formulated using a mixed-integer-programming considering the operational characteristics of AS-PSH and conventional generators. To consider the duck-shaped net-load environment, the projected VRE output data were calculated through physical models of wind turbines and photovoltaic modules. The operating costs for the non-PSH, FS-PSH, and AS-PSH construction scenarios were KRW 43,129.38, 40,038.44, and 34,030.46, respectively. The main factor that derived this difference was determined to be the primary reserve of AS-PSH's pumping mode.

**Keywords:** adjustable-speed pumped-storage hydropower; fixed-speed pumped-storage hydropower; variable renewable energy; duck-curve; reserve-constrained unit commitment; spinning reserve requirement; nuclear power generators

## **1. Introduction**

## *1.1. Research Motivation*

The Korean government has declared the goal of achieving net-zero carbon emissions by 2050. Therefore, Korea is planning to significantly increase the proportion of renewable energy resources. In the generation expansion plan, which determines the Korean investment in power generation facilities, 2030 was set as the target year to ensure that 20% of the annual power is generated by renewable energy resources [1]. Among the renewable energy resources, 75% is expected to be generated from wind and solar energy, which are variable renewable energy (VRE) resources. However, VRE resources may cause difficulties in balancing the supply and demand within a power system owing to their output variability and uncertainty [2]. Specifically, solar power generation makes the system net-load have the characteristics of a duck-curve, which is a concept first introduced by the California Independent System Operator [3]. It has a duck-shaped net-load pattern, which is carried by conventional generators. The VRE's long-term and short-term variability require increased cycling capabilities of the generator and operating reserves, respectively. Therefore, it is difficult for the system operator to maintain system reliability while maintaining the operating proportion of baseload generators during the daytime.

**Citation:** Kim, W.-J.; Lee, Y.-S.; Chun, Y.-H.; Jeong, H.-S. Reserve-Constrained Unit Commitment Considering Adjustable-Speed Pumped-Storage Hydropower and Its Economic Effect in Korean Power System. *Energies* **2022**, *15*, 2386. https://doi.org/ 10.3390/en15072386

Academic Editor: Massimiliano Renzi

Received: 3 January 2022 Accepted: 18 March 2022 Published: 24 March 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

In 2020, the first output reduction of nuclear power generators was conducted by the system operator to maintain reliability in the Korean power system. Shinkori nuclear power generators (#3 and #4) have a total capacity of 2800 MW. The power system operator was scheduled to reduce the total output power to 600 MW for 8 h, and the scheduled output power was maintained for 13 h. The system operator predicted that the system demand was lower than usual owing to long special holidays and analyzed that it would be difficult to meet the system frequency maintenance range if a nuclear power generator with the maximum output (1400 MW) is tripped. There are similar issues with implementing net-zero carbon emissions. Although the coal-fired and nuclear power generators are expected to gradually phase out, the proportion of these baseload generators is still expected to be high in 2030. It is especially difficult to expect frequent output adjustments or startup/shutdown operations with nuclear power generators. In addition, existing nuclear power generators in Korea do not provide frequency response capabilities. Accordingly, situations can occur where several nuclear power generators would need to be operated at a minimum generation level or even excluded from scheduling. In those cases, the operating cost of the power system can increase significantly, and the existing baseload generators may even become stranded assets.

Grid-scale energy storage systems, such as battery energy storage systems (BESS) and pumped-storage hydropower (PSH), can provide solutions to operating costs by mitigating the effect of the duck-curve phenomenon of the net-load and increasing the utilization of nuclear power generators. Although BESSs are capable of providing load shifting and fast frequency responses, PSH is technically proven and more desirable in that it has a large energy capacity and the capability of providing operating reserves. The Korean government is planning to build PSH plants [1]. However, there are three types of PSH plants: fixed-speed (FS), adjustable-speed (AS), and ternary (T) types. As a quickstart resource, a conventional FS-PSH can be rapidly deployed to the system in case of a generator's forced outage or insufficient reserve [4]. AS-PSHs and T-PSHs can provide the same operational advantages as FS-PSHs. In addition, AS-PSH can control its rotor speed using power electronic devices, which enables a rapid power adjustment in the generating and pumping modes and is therefore expected to further improve the flexibility of the power system. Given that the pump and turbine of T-PSH can rotate simultaneously on the same shaft in the same direction, it has the advantage of adjusting the pumping power in the hydraulic short-circuit mode and requires less time to transition between the generating and pumping modes [5]. Therefore, we needed to decide which type will be of benefit to our system when they are constructed, which was the motivation of our research. In this paper, we focused on the economic effect obtained by AS-PSH using the unit commitment (UC) optimization approach in a future Korean power system. In the case of T-PSH, there are three modes: generating mode, pumping mode, and short-circuit mode, which each have different operational characteristics. Therefore, future research is needed to model the operational characteristics of T-PSH in UC formulation.

#### *1.2. Related Research*

Typically, PSH charges at night when the system demand is low and discharges during peak-demand hours. However, owing to the large-scale integration of VREs, the economic benefits that can be obtained from the PSH have been studied from new perspectives [6–12]. In [6], the impact of reducing the operating cost achieved by the input of FS-PSH in a power system with high wind energy was analyzed, and FS-PSH was found to reduce the scheduling cost by reducing the startup and shutdown costs of thermal power generators. In [7], the effect of reducing operating costs through FS-PSH in a wind energy environment was analyzed through a genetic algorithm. The paper derived the reduction in energy purchase cost and environmental cost of the power system. In [8], it was found that as the initial upper reservoir volume of PSH increases, the overall operating cost consisting of grid power purchase cost, demand response program purchase cost, and solar power sale income decreases. In [9], the effect of reduction in the wind power curtailment and the

total operating cost consisting of energy cost, ancillary service cost, and startup cost were derived using FS-PSH as a reserve resource. In [10], the energy cost in a standalone hybrid wind/solar/biomass system was lower in the scenario when PSH was the input than when battery storage was the input. In [11], the authors evaluated the potential contribution of PSH to the Greek power system's future scenario. Through UC simulation, it was deduced that the PSH contributes to energy balance by absorbing the over-generation of wind and solar energy. Ref. [12] proposes a stochastic scheduling approach considering price-based demand response and PSH in a microgrid. In [12], the operating cost was reduced by PSH's discharging at the peak time and charging at the off-peak time. However, given the proposed methods in [6–12], it is difficult to verify the economic contribution obtained from the flexibility of AS-PSH.

The reduction in operating costs due to the flexibility of AS-PSH was studied in [13–15]. In [13], the dynamic dispatch optimization problem was solved with the reserve and network constrained in a wind energy environment, and an economic analysis was performed on FS-PSH, T-PSH, and AS-PSH. However, in [13], it was difficult to verify probable changes in operating conditions within the duck-curve phenomenon, given that the influence on solar energy was not considered. In [14], the economic impact of AS-PSH was analyzed under the conditions of minimizing fuel, cycling, and startup cost. It was modeled as having an operating range in the pumping mode and was derived to reduce the operating cost. In [15], the cost benefits obtained with AS-PSH from the viewpoint of daily operation were compared with those of FS-PSH. However, the study focused on the AS-PSH's flexible operating range in pumping. Therefore, it is still necessary to focus on the reserve capability of AS-PSH that can respond to VRE's short-term variations.

In addition to PSH-related studies, UC optimization research is ongoing from various perspectives. Authors in [16–18] developed methods to efficiently find the UC solution at a high time resolution to cope with the variability and intermittent nature of renewable energy. Ref. [16] proposes a thermal unit commitment algorithm through dynamic programming that combines a priority list method to perform UC and ED at a time resolution of less than 60 min. In [17], the authors pointed out that the branch and cut strategy may exceed the allowable value in the sub-hourly UC problem due to large numbers of virtual transactions. In [17], the authors reduced the calculation time by including the concept of ordinal-optimization in surrogate absolute-value Lagrangian relaxation [19]. Ref. [18] proposes an approach to reduce calculation time by implementing soft constraints of reserve and transmission capacity within the surrogate absolute-value Lagrangian relaxation framework. References [20,21] analyzed the impact of greenhouse gas emissions on the UC problem. In [20], the authors formulated the environmental UC problem considering greenhouse gas emission constraints and carbon taxes in the existing UC approach. In [21], the authors adopted the piecewise linearized greenhouse gas function of the generator's loading level to the UC based on the MIP method. The greenhouse gas function was applied in the form of cost by applying a weighting factor.

#### *1.3. Contribution*

Previous studies on PSH verified its effects on the operating cost savings of power systems with integrated VRE and PSH. However, the majority of these studies focused on the traditional FS-PSH's contribution to power systems in terms of energy saving. The other studies [13–15] on AS-PSH did not consider its capability to supply primary and secondary reserves, particularly in the pumping mode. In a system with a considerable amount of solar energy resources, system operators need large operating reserves, and most generation sources need to be off-lined owing to an over-generation; on the other hand, most PSHs are required to operate in pumping mode. Therefore, to analyze the economic contribution of AS-PSH in a power system with a high proportion of renewable energy resources, it is necessary to model the AS-PSH to provide various reserves, including the primary reserve. Based on the above discussion, the purpose of this study is to verify the

economic contribution of AS-PSH's reserve capability compared with that of FS-PSH in the Korean power system.

To achieve the objective of this study, a reserve-constrained UC (RCUC) was formulated using mixed-integer-programming (MIP) to consider the cycling characteristics of conventional thermal generators in the duck-curve environment and reflect the Korean operating reserve services. The primary, secondary, and regulation reserves defined by the Korean power market were modeled in the RCUC formulation. This formulation can be widely used in other power systems with a similar scheme of system frequency restoration by primary and secondary reserves. The time-series data of the VRE were calculated through physical models, including the output model of wind turbines and photovoltaic (PV) modules. The short-term variation of the VRE was extracted from the time-series data. The increment in the spinning reserves was considered under the n-sigma criterion, assuming that the distribution of the VRE's short-term output variation follows a Gaussian distribution. With the operation range set in the generating and pumping modes, the AS-PSH was modeled to provide primary, secondary, and regulation reserves. Simulations were performed on the 2030 generation mix in Korea. Scenarios were set depending on the types of new PSH plants, namely, non-PSH, FS-PSH, and AS-PSH plants. The simulation was implemented using the optimization solver Xpress-MP.

The main contributions in this paper are listed below:


#### **2. Variable Renewable Energy in Future Korean Power System**

Before analyzing the influence of the new PSH on the future power system in Korea, a methodology to generate the time-series data of the 2030 VRE is introduced. Based on the data, the increment in the spinning reserve due to the short-term variation of the VRE was estimated.

#### *2.1. Modeling of Variable Renewable Energy's Output Power*

The 2030 target capacity of the VRE is about 54.2 GW, and wind and solar energy account for 17.7 GW and 36.5 GW, respectively [1]. Based on the capacity of the VRE to be installed, the 2030 output power can be calculated using historic regional meteorological data. From the Korea Meteorological Administration (KMA), the regional wind speed, solar irradiance, and temperature data measured every minute were used [22].

#### 2.1.1. Wind Energy Output Power Model

To calculate the total output power of wind energy in 2030, it is necessary to determine the location and capacity of each wind farm. To solve the problem, it was assumed that the total capacity of wind energy in which the location is not determined would be distributed to areas with high energy potentials. Therefore, the capacity was distributed proportionally to the locations with high average annual wind speed. Finally, a total of 94 locations of the 2030 wind farms and their capacity were determined.

In a wind farm, each wind turbine is installed at a distance. Therefore, input data conversion was performed without applying the same wind speed to all wind turbines in the same wind farm. Each wind farm consists of N turbine units, and the wind speed after (*n* − 1) minutes is applied to calculate the output power of the nth wind turbine at time *t*.

$$P\_{\rm WF}(t) = \sum\_{n=1}^{N} P\_{\rm WT,n}(t) = \sum\_{n=1}^{N} P\_{\rm WT,n}(V\_h(t - n + 1))\tag{1}$$

To calculate the output power of the wind turbine, Equations (2) and (3) were used [23]. For the convenience of calculation, each wind turbine group was modeled with one unit. The data in Table 1 were used for the specifications of onshore and offshore wind turbines [24].

$$P\_{WT,n}(t) = \begin{cases} 0 & \text{for } V\_h(t-n+1) < V\_{\text{in}} \text{ and } V\_h(t-n+1) > V\_{\text{out}}\\ P\_{\text{curv}} \text{for}(V\_h(t-n+1)) \cdot \frac{\text{C}\_{\text{NF}}}{\text{P}\_r \cdot \text{N}} & \text{for } V\_{\text{in}} \le V\_h(t-n+1) \le V\_{\text{r}}\\ \frac{\text{C}\_{\text{NF}}}{\text{N}} & \text{for } V\_r < V\_{\text{ll}}(t-n+1) \le V\_{\text{out}} \end{cases} \tag{2}$$

$$P\_{\rm current}(V\_{\hbar}(t)) = a\_1 V\_{\hbar}(t)^6 + a\_2 V\_{\hbar}(t)^5 + a\_3 V\_{\hbar}(t)^4 + a\_4 V\_{\hbar}(t)^3 + a\_5 V\_{\hbar}(t)^2 + a\_6 V\_{\hbar}(t) + a\_7 \tag{3}$$

**Table 1.** Parameters of wind turbines (onshore and offshore).


Given that wind speed increases with an increase in altitude, the wind speed used as input data for Equations (2) and (3) should be measured at the tower height. However, the wind speed data of the KMA are measured at an altitude of 10 m. Therefore, using Equation (4) based on the wind power law, we estimated wind speed data at the tower height [25].

$$V\_h = V\_m \left(\frac{Z\_h}{Z\_m}\right)^a \tag{4}$$

2.1.2. Solar Energy Output Power Model

By 2030, 36.5 GW of PV plants is expected to be installed. As in the case of wind power, it is necessary to determine the capacity and the location of each PV plant. However, most PV plants are expected to be installed at a small scale and dispersed over the nation. Therefore, it is necessary to estimate insolation data for areas where KMA does not observe the insolation. To address this problem, we expanded the number of observation stations using a two-dimensional interpolation method [26]. The capacity of each PV plant was assigned in proportion to historical accumulated solar insolation.

The solar insolation data were converted to the irradiance data. The PV output power, which is determined by the irradiance and temperature on the PV module surface, can be expressed as Equation (5) [27].

$$P\_{PV,M} = P\_{PV,M}^\* \cdot \frac{G}{G^\*} [1 - \gamma \cdot (T\_{\mathbb{C}} - 25)] \tag{5}$$

Given that each PV plant consists of PV arrays of several PV modules, the output power of each PV plant can be calculated using Equation (6).

$$P\_{PV} = P\_{PV,M} \cdot \mathbb{C}\_{PV} / P\_{PV,M}^\* \tag{6}$$

Given that the temperature on the PV module surface used in Equation (5) is difficult to measure, it can be approximated using Equation (7) based on the irradiance and the air temperature [28]. to measure, it can be approximated using Equation (7) based on the irradiance and the air temperature [28]. , <sup>∙</sup>/<sup>∗</sup>(6)Given that the on the PV module surface in Equation is to measure, it can approximated using Equation on the irradiance and the air temperature [28].

Given that each PV plant consists of PV arrays of several PV modules, the output

∙ ∗ [1 − <sup>∙</sup> <sup>−</sup> 25)] (5)Given that each plant consists of PV arrays of several PV modules, the output

∗

= , ∙ /,

Given that the temperature on the PV module surface used in Equation (5) is difficult

$$T\_{\mathbb{C}} = T\_A + \frac{T\_{\text{NOCT}} - 20}{800}G\tag{7}$$

[1 − ∙ ( − 25)] (5)

(6)

#### *2.2. Characteristics of Variable Renewable Energy in Korea 2.2. Characteristics of Variable Renewable Energy in Korea 2.2. Characteristics of Variable Renewable in Korea*

*Energies* **2022**, *15*, x FOR PEER REVIEW 6 of 24

power of each PV plant can be calculated using Equation (6).

power of each plant can calculated using Equation (6).

, = , ∗ ∙ ∗

= , ∗

REVIEW of

In this subsection, the output power characteristic of the 2030 VRE is described. Figure 1 depicts the calculated time-series output power of the VRE and its duration curve for the year 2030. In this subsection, the output power characteristic of the 2030 VRE is described. Figure 1 depicts the calculated time-series output power of the VRE and its duration curve for the year 2030. In this the output of the 2030 VRE ure 1 depicts the time-series output of the VRE and its curve for the year 2030.

**Figure 1.** Output and duration curve of VRE for 2030. **Figure 1.** Output and duration curve of VRE for 2030. **Figure** Output and curve of

Figure 2 presents the average daily wind and solar power generation of each month. The average daily wind power generation is highest in winter and lowest in summer. The average daily solar power generation is highest in spring and lowest in winter. Based on these results, the average daily VRE power generation is highest in spring and lowest in autumn. Figure 2 presents the average daily wind and solar power generation of each month. The average daily wind power generation is highest in winter and lowest in summer. The average daily solar power generation is highest in spring and lowest in winter. Based on these results, the average daily VRE power generation is highest in spring and lowest in autumn. Figure 2 presents average each month. The average daily power generation highest in and lowest in The average generation is highest in and lowest in Based on these results, average VRE generation is highest spring lowest in autumn.

**Figure 2.** Average daily VRE power generation of each month for 2030. **Figure** daily VRE generation of month for 2030. **Figure 2.** Average daily VRE power generation of each month for 2030.

Figure 3 illustrates the hourly average output power for wind and solar energy throughout the year 2030. Wind energy demonstrates the highest output power between 14:00 and 15:00, and the highest PV output power and both wind and solar output power are between 12:00 and 13:00.

are between 12:00 and 13:00.

**Figure 3***.* Hourly average output (MW) of wind and PV power projected for 2030. **Figure 3.** Hourly average output (MW) of wind and PV power projected for 2030.

*2.3. Influence of Variable Renewable Energy on Reserve Requirements of the Korean Power System*

Figure 3 illustrates the hourly average output power for wind and solar energy throughout the year 2030. Wind energy demonstrates the highest output power between 14:00 and 15:00, and the highest PV output power and both wind and solar output power

*2.3. Influence of Variable Renewable Energy on Reserve Requirements of the Korean Power System* Traditionally, most power system operators have used the N−1 contingency method, which secures a reserve equal to one or more largest units [29]. The system operator of the Korean power system uses the traditional reserve procurement approach to cover the larg-Traditionally, most power system operators have used the N−1 contingency method, which secures a reserve equal to one or more largest units [29]. The system operator of the Korean power system uses the traditional reserve procurement approach to cover the largest generator outage (1400 MW). Table 2 outlines the operating reserve requirements for the Korean power market [30].

est generator outage (1400 MW). Table 2 outlines the operating reserve requirements for

the Korean power market [30]. **Table 2.** Operating reserve requirements for the Korean power system.


Quick-start reserve 2000 MW Standstill Manual Load forecast error, etc.

The primary reserve prevents the system frequency from dropping immediately after an instantaneous event, such as a generator's forced outage by the synchronized generator's governor response, and the secondary reserve is used to restore the system frequency to the nominal frequency (60 Hz). The required primary reserve in Table 2 was calculated based on the value at the time of a quasi-steady state of the system frequency after the outage of the largest unit (1400 MW) in Korea. In this process, an unused capacity of 1400 MW is required as the secondary reserve to restore the system frequency to the nominal frequency (60 Hz). Immediately after restoration to the nominal frequency by the secondary reserve, the primary reserve is automatically resecured. The regulation reserve is estimated as 700 MW to respond to non-instantaneous events, such as short-term load fluctuations. The tertiary reserve is secured by resources in the standstill state to cover the The primary reserve prevents the system frequency from dropping immediately after an instantaneous event, such as a generator's forced outage by the synchronized generator's governor response, and the secondary reserve is used to restore the system frequency to the nominal frequency (60 Hz). The required primary reserve in Table 2 was calculated based on the value at the time of a quasi-steady state of the system frequency after the outage of the largest unit (1400 MW) in Korea. In this process, an unused capacity of 1400 MW is required as the secondary reserve to restore the system frequency to the nominal frequency (60 Hz). Immediately after restoration to the nominal frequency by the secondary reserve, the primary reserve is automatically resecured. The regulation reserve is estimated as 700 MW to respond to non-instantaneous events, such as short-term load fluctuations. The tertiary reserve is secured by resources in the standstill state to cover the case of reserve shortages, and the quick-start reserve responds to load forecasting errors.

case of reserve shortages, and the quick-start reserve responds to load forecasting errors. The unit capacity of VRE resources is lower than that of conventional generators, and the short-term variation of the VRE can influence the system frequency. Therefore, the traditional approach of securing a spinning reserve equal to one or more of the largest The unit capacity of VRE resources is lower than that of conventional generators, and the short-term variation of the VRE can influence the system frequency. Therefore, the traditional approach of securing a spinning reserve equal to one or more of the largest units is not effective, and it is necessary to adopt an approach that can cover the short-term variation of the VRE.

Based on the operating reserve concept presented in [2], we divided the VRE output fluctuation into event and non-event cases. The reserve for an event is assumed necessary for instantaneous events, such as generator outages and short-term output variations of VRE. These events can be covered by the governor response of the generator. The non-event reserve can be provided by the generator ramping capability according to AGC signals or manual operation. In this study, a 1 min variation in the VRE was assigned to the primary reserve and a 5 min variation to the secondary and regulation reserve. The 1 min and 5 min variations were calculated using Equations (8) and (9), respectively, based on the VRE output data calculated at 1 min intervals, as presented in the previous subsection. event reserve can be provided by the generator ramping capability according to AGC signals or manual operation. In this study, a 1 min variation in the VRE was assigned to the primary reserve and a 5 min variation to the secondary and regulation reserve. The 1 min and 5 min variations were calculated using Equations (8) and (9), respectively, based on

units is not effective, and it is necessary to adopt an approach that can cover the short-

Based on the operating reserve concept presented in [2], we divided the VRE output fluctuation into event and non-event cases. The reserve for an event is assumed necessary for instantaneous events, such as generator outages and short-term output variations of VRE. These events can be covered by the governor response of the generator. The non-

$$P(t) - P(t-1) = P\_{1\text{min}}\tag{8}$$

$$P(t) - P(t-5) = P\_{5\text{min}}\tag{9}$$

As outlined in Table 3, the n-sigma criterion was used to calculate the increment in each spinning reserve requirement due to the output fluctuation of the VRE. The standard deviation σ refers to the time series variability of the VRE. Assuming that the output variability of the VRE follows a normal distribution, an additional reserve to cover ±σ of the variability can respond to 68% of the statistical data, a reserve of ±2σ to 95%, and that of ±3σ to 99.7% [31]. Table 3 presents the specifications of the method for allocating additional spinning reserve requirements under the assumption that the current spinning reserve requirement is maintained. As outlined in Table 3, the n-sigma criterion was used to calculate the increment in each spinning reserve requirement due to the output fluctuation of the VRE. The standard deviation σ refers to the time series variability of the VRE. Assuming that the output variability of the VRE follows a normal distribution, an additional reserve to cover ±σ of the variability can respond to 68% of the statistical data, a reserve of ±2σ to 95%, and that of ±3σ to 99.7% [31]. Table 3 presents the specifications of the method for allocating additional spinning reserve requirements under the assumption that the current spinning reserve requirement is maintained.

**Table 3.** Allocation method of the increment amount of reserve requirement. **Table 3.** Allocation method of the increment amount of reserve requirement.


*Energies* **2022**, *15*, x FOR PEER REVIEW 8 of 24

term variation of the VRE.

The simple arithmetic addition of the estimated reserve requirement for VRE to the existing reserve requirement for load variability can result in an overestimation, which can be costly. This problem can be solved by using geometric additions [32]. Figure 4 illustrates the method of calculating the primary reserve requirement secured through the governor response, and Figure 5 presents the calculation method for the requirement of the secondary reserve and regulation reserve activated by AGC signals. The simple arithmetic addition of the estimated reserve requirement for VRE to the existing reserve requirement for load variability can result in an overestimation, which can be costly. This problem can be solved by using geometric additions [32]. Figure 4 illustrates the method of calculating the primary reserve requirement secured through the governor response, and Figure 5 presents the calculation method for the requirement of the secondary reserve and regulation reserve activated by AGC signals.

**Figure 5.** Secondary and regulation reserve requirement for AGC operation. **Figure 5.** Secondary and regulation reserve requirement for AGC operation.

**3. RCUC Considering Adjustable-Speed Pumped-Storage Hydropower**

This section introduces the RCUC formulation. Section 3.1 presents the features of

AS-PSH is a resource that can provide a higher quality of flexibility to power systems than FS-PSH by adjusting the rotor speed and electrical output/input of the generator/motor. With respect to power system operations, AS-PSH has the following advantages when

(1) AS-PSH can be operated over a wider operational range in the generating mode, and

(2) AS-PSH can provide primary, secondary, and regulation reserves in the generating

The critical factors for the advantages of AS-PSH in power system operation are its operation range and spinning reserve capacity. Therefore, it is necessary to determine the operation range and spinning reserve capacity of AS-PSH. In this study, the operation range and spinning reserve capacity were determined based on an investigation of the

The operation range is defined by the difference between the minimum and maximum powers. The operation ranges of FS-PSH and AS-PSH were reviewed in reports [5] and [33], and the comparison is shown in Table 4. However, the operation ranges of practical PSH plants vary. For example, in the Kazunogawa PSH plant [34], Unit #4 is an AStype, and Units #1 and #2 are FS-type. In the generating mode, the operation range of Unit #1 is wider than that of Units #1 and #2 by 32.5%. In the pumping mode, the operation range of Unit #1 is approximately 32%. In this study, the operation range of future AS-PSH plants was set as 30–100% in the generating mode and 60–100% in the pumping

*3.1. Flexibility of Adjustable-Speed Pumped-Storage Hydropower*

the input power can be controlled in the pumping mode.

**Table 4.** Example of pumped-storage hydropower (PSH) operation range.

Generating mode Argonne Report [5] <sup>16</sup>–100% 16~100%

Pumping mode Argonne Report <sup>60</sup>–100% 60~100%

**Operation Range FS-PSH AS-PSH**

JICA Report [33] 30–100% 30~100%

JICA Report 70–100% 70~100%

The primary reserve keeps the system frequency within operation range directly after a disturbance. Therefore, the capacity with which AS-PSH can instantaneously control the

ering AS-PSH's operational characteristics.

compared with FS-PSH.

mode.

and pumping modes.

technical reports and AS-PSH plants in practice.

#### **3. RCUC Considering Adjustable-Speed Pumped-Storage Hydropower**

This section introduces the RCUC formulation. Section 3.1 presents the features of AS-PSH. Section 3.2 presents the objective function for the optimization, which is to minimize the operating costs of the power system. Section 3.3 presents the constraints considering AS-PSH's operational characteristics.

#### *3.1. Flexibility of Adjustable-Speed Pumped-Storage Hydropower*

AS-PSH is a resource that can provide a higher quality of flexibility to power systems than FS-PSH by adjusting the rotor speed and electrical output/input of the generator/motor. With respect to power system operations, AS-PSH has the following advantages when compared with FS-PSH.


The critical factors for the advantages of AS-PSH in power system operation are its operation range and spinning reserve capacity. Therefore, it is necessary to determine the operation range and spinning reserve capacity of AS-PSH. In this study, the operation range and spinning reserve capacity were determined based on an investigation of the technical reports and AS-PSH plants in practice.

The operation range is defined by the difference between the minimum and maximum powers. The operation ranges of FS-PSH and AS-PSH were reviewed in reports [5] and [33], and the comparison is shown in Table 4. However, the operation ranges of practical PSH plants vary. For example, in the Kazunogawa PSH plant [34], Unit #4 is an AS-type, and Units #1 and #2 are FS-type. In the generating mode, the operation range of Unit #1 is wider than that of Units #1 and #2 by 32.5%. In the pumping mode, the operation range of Unit #1 is approximately 32%. In this study, the operation range of future AS-PSH plants was set as 30–100% in the generating mode and 60–100% in the pumping mode.


**Table 4.** Example of pumped-storage hydropower (PSH) operation range.

The primary reserve keeps the system frequency within operation range directly after a disturbance. Therefore, the capacity with which AS-PSH can instantaneously control the power within a few seconds was assumed to be the primary reserve. The range of the primary reserve was set based on the example of the AS-PSH in practical operation. Unit #4 of the Okawachi PSH in Japan is capable of a step response output change of 32 MW (10% of the maximum rated power) within 0.2 s in the generating mode, and 80 MW (20% of the maximum rated power) within 0.2 s in the pumping mode [35]. Accordingly, we set the range of the primary reserve to be secured by AS-PSH to 10% of the maximum rated power in the generating mode and 20% in the pumping mode. The range of secondary/regulation reserves can be set based on the capacity to control the power for several minutes. Given that AS-PSH has a high ramping capability, the maximum capacity of the secondary/regulation reserves was set as the difference between the minimum and maximum power. The operation and reserve range of each PSH type are shown in Figures 6 and 7 and Table 5. The reserves were divided into upward and downward directions.

**Max. Primary Reserve (Up/Downward)**

**Max. Secondary/ Regulation Reserve (Up/Downward)**

power within a few seconds was assumed to be the primary reserve. The range of the primary reserve was set based on the example of the AS-PSH in practical operation. Unit #4 of the Okawachi PSH in Japan is capable of a step response output change of 32 MW (10% of the maximum rated power) within 0.2 s in the generating mode, and 80 MW (20% of the maximum rated power) within 0.2 s in the pumping mode [35]. Accordingly, we set the range of the primary reserve to be secured by AS-PSH to 10% of the maximum rated power in the generating mode and 20% in the pumping mode. The range of secondary/regulation reserves can be set based on the capacity to control the power for several minutes. Given that AS-PSH has a high ramping capability, the maximum capacity of the secondary/regulation reserves was set as the difference between the minimum and maximum power. The operation and reserve range of each PSH type are shown in Figures 6 and 7 and Table 5. The reserves were divided into upward and downward directions.

**Figure 6. Figure 6.** Operation Operation and reserve ranges of PSH (generating mode). and reserve ranges of PSH (generating mode).

**Table 5.** Operation and reserve range of FS-PSH and AS-PSH.

Pumping mode Constant at 100% - -

Pumping mode 60~100% 20% 40%

**Operation Range**

FS-PSH Generating mode 50~100% 10% 50%

**Figure 7.** Operation and reserve ranges of PSH (pumping mode). **Figure 7.** Operation and reserve ranges of PSH (pumping mode).


*3.2. Objective Function to Minimize the Operating Cost of the Power System* **Table 5.** Operation and reserve range of FS-PSH and AS-PSH.

#### : ∑ ∑{(, ) + ∙ , } *3.2. Objective Function to Minimize the Operating Cost of the Power System*

∈ =1 In general, the power generation cost function is expressed as a quadratic function. However, in this study, we modeled the cost function of the quadratic function as a piecewise linearization function [36,37]. *3.3. Constraints Considering AS-PSH* AS-PSH increases the total system load during its pumping mode and can provide To predict the output pattern of the conventional generators of the future power system, it was assumed that the generators produced an economic output under the spinning reserve constraints and generator cycling characteristics. Accordingly, we formulated an objective function to minimize the operating cost of the power system. The system operating cost was divided into the fuel and startup costs of the generator. To implement the constraints of the minimum output, minimum downtime, and minimum uptime of generators, the objective function containing integer variables can be defined as follows:

Constraints (11) and (12) represent the startup and shutdown states of a generator and prevent the simultaneous occurrence of the startup and shutdown states. Constraint (13) prevents the concurrent occurrence of the PSH generating and pumping modes.

$$Min: \sum\_{i \in GII} \sum\_{t=1}^{T} \left\{ FC\_i \left( p\_{i,t}^{\mathcal{S}} \right) + STC\_i \cdot u\_{i,t} \right\} \tag{10}$$

, − , = , − ,−1 (11)

, + , ≤ 1 (12)

, + , ≤ 1 (13)

(10)

the modeling of AS-PSH.

In general, the power generation cost function is expressed as a quadratic function. However, in this study, we modeled the cost function of the quadratic function as a piecewise linearization function [36,37].

#### *3.3. Constraints Considering AS-PSH*

AS-PSH increases the total system load during its pumping mode and can provide primary, secondary, and regulation reserves in the pumping mode, which should be considered in the constraints. This subsection presents the constraints of the RCUC containing the modeling of AS-PSH.

#### 3.3.1. Integer Variable Constraints

Constraints (11) and (12) represent the startup and shutdown states of a generator and prevent the simultaneous occurrence of the startup and shutdown states. Constraint (13) prevents the concurrent occurrence of the PSH generating and pumping modes.

$$u\_{i,t} - d\_{i,t} = i\_{i,t} - i\_{i,t-1} \tag{11}$$

$$u\_{i,t} + d\_{i,t} \le 1\tag{12}$$

$$l\_{i,t} + l\_{i,t} \le 1 \tag{13}$$

#### 3.3.2. Load Balance Constraints

At each hour, the total generated power and the difference between the load and VRE power were set as equal. In addition, considering the pumping input power of the PSH plants, the load balance constraint can be expressed by Constraint (14).

$$\mathbf{P\_t^{Load}} - \mathbf{P\_t^{VRE}} + \sum\_{i \in PP} p\_{i,t}^p = \sum\_{i \in GL} p\_{i,t}^g \tag{14}$$

#### 3.3.3. Power Limit Constraints

The output power of generators and the input power of AS-PSH plants in the pumping mode can range from the minimum to maximum power, as expressed by Constraints (15) and (16).

$$P\_{\mathbf{i}}^{\min} \times \mathbf{i}\_{\mathbf{i},\mathbf{t}} \le p\_{\mathbf{i},\mathbf{t}}^{\mathcal{S}} \le P\_{\mathbf{i}}^{\max} \times \mathbf{i}\_{\mathbf{i},\mathbf{t}} \tag{15}$$

$$PP\_{\mathbf{i}}^{\min} \times \mathbf{l}\_{\mathbf{i},\mathbf{t}} \le p\_{\mathbf{i},\mathbf{t}}^p \le PP\_{\mathbf{i}}^{\max} \times \mathbf{l}\_{\mathbf{i},\mathbf{t}} \tag{16}$$

#### 3.3.4. Minimum Uptime/Downtime Constraints

During the daytime, when the power generated by VRE resources increases, a valley is created in the net-load curve, and generator cycling operations can be required. In the case of a coal-fired power generator in an offline state during the period wherein the net-load is lowered, the offline state should be maintained for a minimum of 8–12 h. Therefore, it is difficult to provide ramping capabilities during the period wherein the net-load increases due to the decrease in the VRE's output. The minimum downtime and minimum up-time constraints were thus considered through Constraints (17) and (18).

$$\sum\_{t=k}^{k+MLI\_l-1} i\_{i,t} \ge \mathcal{M}\mathcal{U}\_l \times \mu\_{i,k} \tag{17}$$

$$\sum\_{t=k}^{k+MD\_i-1} (1 - i\_{i,t}) \ge MD\_i \times d\_{i,k} \tag{18}$$

3.3.5. Generator Ramp-rate Constraints

The difference in output power between hour *t* and *t* − 1 of a generator is limited by the ramp rate and expressed by Constraints (19) and (20).

$$\mathbf{p}\_{\mathbf{i},\mathbf{t}}^{\mathbf{g}} - \mathbf{p}\_{\mathbf{i},\mathbf{t}-1}^{\mathbf{g}} \le 60 \times R\mathcal{U}\_{\mathbf{i}} \tag{19}$$

$$\mathbf{p}\_{\mathrm{i},\mathrm{t}-1}^{\mathrm{g}} - \mathbf{p}\_{\mathrm{i},\mathrm{t}}^{\mathrm{g}} \le 60 \times R D\_{\mathrm{i}} \tag{20}$$

#### 3.3.6. Spinning Reserve Provision Constraints

The spinning reserve requirements should be secured under the constraints of the generator operation range. The target value of a synchronized generator for governor response is determined by the system frequency deviation and its droop characteristic [38]. To maintain the system frequency, the governor response that each unit can provide should be within the operation range of the system frequency. Therefore, the maximum capacity of the primary reserve of each unit can be expressed by Constraint (21) and the primary reserve of a generator at hour *t* by Constraints (22) and (23).

$$\mathbf{GR}\_{\mathbf{i}}^{\mathbf{CG},\mu p,\max} = \mathbf{GR}\_{\mathbf{i}}^{\mathbf{CG},\text{down},\max} = \,^{\mathbf{P}\_{\mathbf{i}}^{\max} \cdot \mathbf{SF}\_{\mathbf{l}}} \Big/ \,^{} \mathbb{R}\_{\mathbf{i}} \cdot \mathbf{SF}\_{\mathbf{s}} \tag{21}$$

$$0 \le \mathcal{g}r\_{i,t}^{\text{CG},\mu p} \le \mathcal{G}R\_i^{\text{CG},\mu p,\text{max}} \times i\_{i,t} \tag{22}$$

$$0 \le \mathcal{g}r\_{i,t}^{\text{CG,down}} \le \mathcal{G}R\_i^{\text{CG,down},\text{max}} \times i\_{i,t} \tag{23}$$

As discussed in the previous subsection, the maximum primary reserve provided in the generating and pumping modes of the AS-PSH is expressed by Constraints (24) and (25). The primary reserve provided by an AS-PSH at hour *t* is expressed by Constraints (26), (27), (28), and (29).

$$PR\_{\rm i}^{g,\mu p,\max} = PR\_{\rm i}^{g,down,\max} = P\_{\rm i}^{\max} \times 10\% \tag{24}$$

$$PR\_{\dot{i}}^{p, \mu p, \max} = PR\_{\dot{i}}^{p, \text{down}, \max} = PP\_{\dot{i}}^{\max} \times \text{20\%} \tag{25}$$

$$0 \le p r\_{i,t}^{g,\mu p} \le P R\_i^{g,\mu p,\max} \times i\_{i,t} \tag{26}$$

$$0 \le \text{pr}\_{i,t}^{\text{g,down}} \le \text{PR}\_i^{\text{g,down,max}} \times i\_{i,t} \tag{27}$$

$$0 \le pr\_{i,t}^{p,up} \le PR\_i^{p,up,\max} \times l\_{i,t} \tag{28}$$

$$0 \le \mathcal{p}r\_{i,t}^{p,down} \le \mathcal{P}\mathcal{R}\_i^{p,down,max} \times l\_{i,t} \tag{29}$$

The secondary/regulation reserves are provided by the ACG signals according to the ramp-rate of each generator. Therefore, a ramp-rate should be reflected in the ACG capacity available to each generator. The UC problem in this study considers the constraints of secondary and regulation reserves at 5 min intervals. The secondary and regulation reserves available to each generator can be calculated using Constraints (30) and (31), and the AGC capacity secured by a generator at hour *t* is expressed by Constraints (32) and (33).

$$\text{AGC}^{\text{CG},up,\text{max}}\_{i} = \mathbf{5} \times \text{RR}^{up}\_{i} \tag{30}$$

$$\text{AGG}^{\mathsf{CG},down,max}\_{\text{i}} = \mathbf{5} \times \text{RR}^{down}\_{\text{i}} \tag{31}$$

$$0 \le \arg\_{i,t}^{\text{CG},\mu p} \le A \text{GC}\_i^{\text{CG},\mu p,\text{max}} \times i\_{i,t} \tag{32}$$

$$0 \le \arg\_{i,t}^{\text{CG,down}} \le \text{AGC}\_{i}^{\text{CG,down,max}} \times \text{i}\_{i,t} \tag{33}$$

The secondary/regulation reserve capacities of the AS-PSH are defined by Constraints (34) and (35). The reserves of the AS-PSH in the generating and pumping modes at hour *t* are expressed by Constraints (36)–(39).

$$\text{AGC}^{g,\upmu p,\text{max}}\_{\text{i}} = \text{AGC}^{g,\text{down},\text{max}}\_{\text{i}} = P\_{\text{i}}^{\text{max}} - P\_{\text{i}}^{\text{min}} \tag{34}$$

$$AGC\_{\text{i}}^{p,\upmu p,\text{max}} = AGC\_{\text{i}}^{p,\text{down},\text{max}} = PP\_{\text{i}}^{\text{max}} - PP\_{\text{i}}^{\text{min}} \tag{35}$$

$$0 \le \arg\_{i,t}^{g,\upmu p} \le A \text{GC}\_i^{g,\upmu p,\text{max}} \times i\_{i,t} \tag{36}$$

$$0 \le \log \mathbf{c}\_{i,t}^{\text{g,down}} \le \text{AGG}\_i^{\text{g,down,max}} \times \mathbf{i}\_{i,t} \tag{37}$$

$$0 \le \arg\_{i,t}^{p,\upmu p} \le \text{AGG}\_i^{p,\upmu p,\upmax} \times l\_{i,t} \tag{38}$$

$$0 \le \arg\_{i,t}^{p,down} \le \text{AGG}\_i^{p,down,max} \times l\_{i,t} \tag{39}$$

#### 3.3.7. Power Limit Constraints Considering Spinning Reserve

The output range of a conventional generator should consider its spinning reserve at hour *t*, as expressed by Constraints (40) and (41).

$$\log\_{i,t}^{g} + \arg\_{i,t}^{\mathsf{CG},\mu p} + \arg\_{i,t}^{\mathsf{CG},\mu p} \le P\_i^{\max} \tag{40}$$

$$\|p\_{i,t}^{\mathcal{S}} - \arg\mathbf{c}\_{i,t}^{\mathbf{C}G,down} - \mathbf{g}r\_{i,t}^{\mathbf{C}G,down} \ge \mathbf{P}\_i^{min} \tag{41}$$

The output and input power range of AS-PSH can be further expressed by Constraints (42), (43), (44), and (45).

$$pr\_{i,t}^{\mathcal{S}} + agc\_{i,t}^{\mathcal{g},\mu p} + pr\_{i,t}^{\mathcal{g},\mu p} \le P\_i^{\max} \times i\_{i,t} \tag{42}$$

$$\|p\_{i,t}^{\mathcal{g}} - ag c\_{i,t}^{\mathcal{g},down} - pr\_{i,t}^{\mathcal{g},down} \ge P\_i^{\min} \times i\_{i,t} \tag{43}$$

$$\|p\_{i,t}^p - \arg\_{i,t}^{p,up} - pr\_{i,t}^{p,down} \le PP\_i^{max} \times l\_{i,t} \tag{44}$$

$$\|p\_{i,t}^p + \arg\_{i,t}^{p,down} + pr\_{i,t}^{p,down} \ge PP\_i^{min} \times l\_{i,t} \tag{45}$$

#### 3.3.8. Spinning Reserve Requirement Constraints

The spinning reserve procured at each hour should satisfy Constraints (46) and (47), which specify the requirements of the primary reserve and secondary/regulation reserves, respectively. In this study, the up-spinning and down-spinning reserve requirements were assumed to be identical. The primary reserve requirement in the Korean power market is secured for the largest unit loss as the change in the generators' output power in a quasi-steady state after a generator outage. The Korean power system has an average governor response of 1000 MW at the quasi-steady state for the largest unit outage [39]. The requirement of a 1400 MW secondary reserve is the capacity required to remove the steady-state error of the system frequency after the largest unit loss. In addition, the standard deviations of VRE are set to vary according to its output power, preventing the overestimation of spinning reserves during the night when the output power of the VRE is low. Based on the reserve requirement determination method in [40], we derived the standard-deviation function of VRE's output.

$$\mathcal{S} \mathcal{R}\_t^{\text{PR},up} = \mathcal{S} \mathcal{R}\_t^{\text{PR},down} = \mathcal{P} \mathcal{R}^{LII} + \boldsymbol{\pi} \cdot \mathcal{sigma}\_{VRE}^{1 \text{min}} \left( \mathcal{P}\_t^{VRE} \right) \tag{46}$$

$$SR\_t^{\text{AGC},\mu p} = SR\_t^{\text{AGC},\text{down}} = SR^{\text{LII}} + \sqrt{\left(n \cdot \sigma\_L^{5\text{min.}}\right)^2 + \left(n \cdot \sigma\_{VRE}^{5\text{min}} \left(P\_t^{VRE}\right)\right)^2} \tag{47}$$

The spinning reserve secured by AS-PSHs and conventional generators should satisfy the reserve requirements at each hour, as expressed by Constraints (48), (49), (50), and (51).

$$\sum\_{i \in AP^{\mathbb{C}}} gr\_{i,t}^{\mathbb{C}G,\mu p} + \sum\_{i \in AP} (pr\_{i,t}^{g,\mu p} + pr\_i^{p,\mu p}) \ge SR\_t^{PR,\mu p} \tag{48}$$

$$\sum\_{i \in AP^{\rm c}} gr\_{i,t}^{\rm CC,down} + \sum\_{i \in AP} \left( pr\_{i,t}^{\rm g,down} + pr\_i^{p,down} \right) \ge SR\_t^{\rm PR,down} \tag{49}$$

$$\sum\_{i \in AP^{\mathbb{C}}} \arg \mathbf{c}\_{i,t}^{\mathbb{C}G,\mu p} + \sum\_{i \in AP} \left( \arg \mathbf{c}\_{i,t}^{\mathbb{S},\mu p} + \arg \mathbf{c}\_{i,t}^{p,\mu p} \right) \ge \mathcal{S} \mathcal{R}\_t^{\mathrm{AGC,\mu p}} \tag{50}$$

$$\sum\_{i \in AP^{\rm c}} \arg \mathbf{c}\_{i,t}^{\rm CG,down} + \sum\_{i \in AP} \left( \arg \mathbf{c}\_{i,t}^{\rm g,down} + \arg \mathbf{c}\_{i,t}^{p,down} \right) \ge \mathcal{S} \mathbf{R}\_t^{\rm AGC,down} \tag{51}$$

#### 3.3.9. Upper Reservoir of PSH Constraint

The PSH has an energy constraint based on its upper reservoir volume. Constraints (52) and (53) consider the efficiency when the PSH charges and discharges. The upper reservoir volume at hour *t* was measured based on electrical energy.

$$
\omega\_{i,t} = \upsilon\_{i,t-1} - p\_{i,t-1}^g + p\_{i,t-1}^p \times EFF\_i \tag{52}
$$

$$0 \le v\_{i,t} \le V\_i^{\max} \tag{53}$$

### **4. Simulation**

This section presents the scenarios, simulation results, and discussion.

#### *4.1. Scenarios*

The construction of three new PSH plants by 2030, with reference to [1], was assumed; and the following scenarios were set. Scenario A denotes no PSH plant construction, Scenario B denotes FS-PSH plant construction, and Scenarios C and D denote AS-PSH plant construction. To verify the contribution of the AS-PSH operating in the pumping mode to primary reserve provision, Scenario C constrained the AS-PSH to provide only secondary/regulation reserves in pumping mode, and the AS-PSH in Scenario D was modeled to provide all the reserves. Table 6 outlines each scenario. Both FS-PSH and AS-PSH were assumed to have the same capacity for the generating and pumping modes.

**Table 6.** Scenario description of RCUC simulation.


Given the low frequency of the high output power of VRE, in addition to the low investment efficiency of transmission and distribution facilities required, we considered curtailing approximately 5% of the target power generation of VRE. If the VRE output is limited to a maximum of 26 GW in 2030, 5% of the total generation can be curtailed, and the VRE output curve in Figure 1 can be depicted as shown in Figure 8.

**Figure 8.** Output and duration curve of 2030 VRE with an output limit of 26 GW. **Figure 8.** Output and duration curve of 2030 VRE with an output limit of 26 GW. *Energies* **2022**, *15*, x FOR PEER REVIEW 16 of 24

*Energies* **2022**, *15*, x FOR PEER REVIEW 16 of 24

3.3.9. Upper Reservoir of PSH Constraint

**Table 6.** Scenario description of RCUC simulation.

**Scenario Existing PSH Plants New PSH Plants in 2030 Total Capacity of PSH** 

A 4700 0 0 4700 B 4700 600 × 3 = 1800 0 6500 C 4700 0 600 × 3 = 1800 6500 D 4700 0 600 × 3 = 1800 6500

**FS-Type, MW FS-Type, MW AS-Type, MW Plants, MW**

the VRE output curve in Figure 1 can be depicted as shown in Figure 8.

**4. Simulation**

*4.1. Scenarios*

The PSH has an energy constraint based on its upper reservoir volume. Constraints (52) and (53) consider the efficiency when the PSH charges and discharges. The upper

0 ≤ , ≤

The construction of three new PSH plants by 2030, with reference to [1], was assumed; and the following scenarios were set. Scenario A denotes no PSH plant construction, Scenario B denotes FS-PSH plant construction, and Scenarios C and D denote AS-PSH plant construction. To verify the contribution of the AS-PSH operating in the pumping mode to primary reserve provision, Scenario C constrained the AS-PSH to provide only secondary/regulation reserves in pumping mode, and the AS-PSH in Scenario D was modeled to provide all the reserves. Table 6 outlines each scenario. Both FS-PSH and AS-PSH were assumed to have the same capacity for the generating and pumping modes.

Given the low frequency of the high output power of VRE, in addition to the low investment efficiency of transmission and distribution facilities required, we considered curtailing approximately 5% of the target power generation of VRE. If the VRE output is limited to a maximum of 26 GW in 2030, 5% of the total generation can be curtailed, and

<sup>+</sup> ,−1

× (52)

(53)

reservoir volume at hour *t* was measured based on electrical energy.

, = ,−1 − ,−1

This section presents the scenarios, simulation results, and discussion.

The calculated output of VRE in Korea differs by month, and the generation of VRE is highest in spring. Generally, the load curve on weekends tends to be lower than that on weekdays. The effects of VRE were assumed to be significant on weekends. Therefore, UC simulation was performed on a weekend daily load curve with high VRE generation in spring 2030. To verify the VRE effect in a typical load curve pattern, special days, such as traditional holidays, were excluded. The calculated output of VRE in Korea differs by month, and the generation of VRE is highest in spring. Generally, the load curve on weekends tends to be lower than that on weekdays. The effects of VRE were assumed to be significant on weekends. Therefore, UC simulation was performed on a weekend daily load curve with high VRE generation in spring 2030. To verify the VRE effect in a typical load curve pattern, special days, such as The calculated output of VRE in Korea differs by month, and the generation of VRE is highest in spring. Generally, the load curve on weekends tends to be lower than that on weekdays. The effects of VRE were assumed to be significant on weekends. Therefore, UC simulation was performed on a weekend daily load curve with high VRE generation in

The daily load, net-load, and output power of the VRE considering the output limit are shown in Figure 9. The requirements of the spinning reserves with respect to Constraints (46) and (47), which adopt the 2σ criterion of the short-term variation of the VRE, are plotted in Figure 10. The current standard adopting the 3σ criterion for the short-term variation of the load was maintained. traditional holidays, were excluded. The daily load, net-load, and output power of the VRE considering the output limit are shown in Figure 9. The requirements of the spinning reserves with respect to Constraints (46) and (47), which adopt the 2σ criterion of the short-term variation of the VRE, are plotted in Figure 10. The current standard adopting the 3σ criterion for the short-term variation of the load was maintained. spring 2030. To verify the VRE effect in a typical load curve pattern, special days, such as traditional holidays, were excluded. The daily load, net-load, and output power of the VRE considering the output limit are shown in Figure 9. The requirements of the spinning reserves with respect to Constraints (46) and (47), which adopt the 2σ criterion of the short-term variation of the VRE, are plotted in Figure 10. The current standard adopting the 3σ criterion for the short-term

variation of the load was maintained.

**Figure 9.** Curves of load, net-load, and VRE output power on spring Sunday, 2030. **Figure 9.** Curves of load, net-load, and VRE output power on spring Sunday, 2030. **Figure 9.** Curves of load, net-load, and VRE output power on spring Sunday, 2030.

In the UC simulation, the following two assumptions were made:

In the UC simulation, the following two assumptions were made:

Assumption 1 implies that there is no startup or shutdown operation of nuclear power generators for 24 h. Nuclear power generators are set to have a significantly limited ramp-rate. Assumption 2 implies that the UC results do not count as a spinning reserve,

Assumption 1 implies that there is no startup or shutdown operation of nuclear power generators for 24 h. Nuclear power generators are set to have a significantly limited

(2) Nuclear power generators do not provide spinning reserves.

(2) Nuclear power generators do not provide spinning reserves.

**Figure 10.** Spinning reserve requirement at each hour. ramp-rate. Assumption 2 implies that the UC results do not count as a spinning reserve, **Figure 10.** Spinning reserve requirement at each hour.

*4.2. Results*

*4.2. Results*

h.

h.

#### *4.2. Results*

In the UC simulation, the following two assumptions were made:


Assumption 1 implies that there is no startup or shutdown operation of nuclear power generators for 24 h. Nuclear power generators are set to have a significantly limited ramp-rate. Assumption 2 implies that the UC results do not count as a spinning reserve, even if nuclear power generators have unused capacity. The total available capacity of nuclear power generators was assumed to be 16.4 GW out of 20.4 GW in 2030, with reference to their historical planned outage rates. Starting with the case where all available capacity of nuclear power generators is operable, the feasible solution was found by excluding the online state nuclear power generators. Table 7 presents the feasible solutions to the optimization problem according to the capacity of the nuclear power generators in the online state for each scenario. *Energies* **2022**, *15*, x FOR PEER REVIEW 17 of 24 even if nuclear power generators have unused capacity. The total available capacity of nuclear power generators was assumed to be 16.4 GW out of 20.4 GW in 2030, with reference to their historical planned outage rates. Starting with the case where all available capacity of nuclear power generators is operable, the feasible solution was found by excluding the online state nuclear power generators. Table 7 presents the feasible solutions to the optimization problem according to the capacity of the nuclear power generators in the online state for each scenario.

> **Table 7.** Feasibility of each scenario. **Table 7.** Feasibility of each scenario.


In all scenarios, the UC problem was found to be infeasible when nuclear power generators were employed at the maximum capacity of 16.4 GW. In Scenarios A and B, the maximum available capacities of the nuclear power generators were derived as 4.2 GW and 7 GW, respectively. Similarly, the maximum available capacities of the nuclear power generators in Scenarios B and D were 7 GW and 12.4 GW, respectively. Figures 11–14 present the diagram of the UC solution for each scenario when the feasible solution is operated as per the maximum capacity of the nuclear power plant, as derived in Table 7. In all scenarios, the UC problem was found to be infeasible when nuclear power generators were employed at the maximum capacity of 16.4 GW. In Scenarios A and B, the maximum available capacities of the nuclear power generators were derived as 4.2 GW and 7 GW, respectively. Similarly, the maximum available capacities of the nuclear power generators in Scenarios B and D were 7 GW and 12.4 GW, respectively. Figures 11–14 present the diagram of the UC solution for each scenario when the feasible solution is operated as per the maximum capacity of the nuclear power plant, as derived in Table 7.

**Figure 11. Figure 11.** The UC solution diagram of Scenario A The UC solution diagram of Scenario A. .

**Figure 12.** The UC solution diagram of Scenario B. **Figure 12.** The UC solution diagram of Scenario B. **Figure 12.** The UC solution diagram of Scenario B. **Figure 12.** The UC solution diagram of Scenario B.

**Figure 13.** The UC solution diagram of Scenario C. **Figure 13.** The UC solution diagram of Scenario C. **Figure 13.** The UC solution diagram of Scenario C. **Figure 13.** The UC solution diagram of Scenario C.

Table 8 presents the operating cost results of each scenario. Given the characteristics of the Korean power market, which is based on fuel cost, the costs of VRE and hydro Table 8 presents the operating cost results of each scenario. Given the characteristics of the Korean power market, which is based on fuel cost, the costs of VRE and hydro Table 8 presents the operating cost results of each scenario. Given the characteristics of the Korean power market, which is based on fuel cost, the costs of VRE and hydro Table 8 presents the operating cost results of each scenario. Given the characteristics of the Korean power market, which is based on fuel cost, the costs of VRE and hydro energy were not considered. In Figures 11–14, the resources named 'others' consists of new energy resources, such as fuel cells and by-product gases. Using the expected power generation of the new energy resources in 2030, the output power was assumed to be uniform throughout the day [1]. Given that the 24 h power generation pattern of the energy sources was applied in all scenarios, their costs were neglected.


**Table 8.** Cost result of each scenario.

The total operating costs decreased in the order of Scenarios A, B, C, and D. The proportion of nuclear power generation was 6.8%, 11.4%, 11.4%, and 19.9% in Scenarios A, B, C, and D, respectively, and that of thermal power generation was calculated as 55.3%, 50.8%, 50.8%, and 42.3%, respectively. Based on the operating cost result of Scenario A, the operating cost savings for scenarios B, C, and D were KRW 3090.94, 3774.46, and 9098.92, respectively.

#### *4.3. Discussion*

The results shown in Table 8 and Figures 11–14 were found to be strongly related to the generator output limit in Constraints (15), (40), and (41); the generator's spinning reserve provision in Constraints (21)–(23) and (30)–(33); and the spinning reserve requirement in Constraints (46)–(51). If nuclear power generators with a total capacity of 16.4 GW are operating, the thermal power generators carry a given net-load, excluding the output power of the nuclear power generators. However, in this case, the spinning reserve requirement is not satisfied. In contrast, if the nuclear power generator is operating at 16.4 GW and the thermal power generators procure the requirement of spinning reserves, the supply and demand balance of Constraint (14) is violated.

Therefore, the AS-PSH operating in pumping mode during low net-load and providing primary reserves was the main factor for the result of Scenario D. The primary reserve from a conventional generator is dominated by its droop and not its ramp-rate characteristics. As mentioned above, the primary reserve that can be secured for each generator is considerably limited when compared with the secondary/regulation reserve. For example, under Constraint (21), if the system-frequency maintenance range is ±0.2 Hz, the primary reserve that can be secured by a 500 MW generator with droop 6% is merely 25 MW (5% of the maximum rated power of the generator). In this case, with a 25 MW increase in the primary reserve, 500 MW of thermal power generation must be committed sequentially.

An increase in the primary reserve requirements requires an increased number of thermal power generators in the online state, but the duck-curve phenomenon limits the number of online generators. The FS-PSH operated in pumping mode, contributing only to mitigating the duck-curve phenomenon, while the AS-PSH operated in pumping mode and contributed to both primary reserve provision and mitigating the duck-curve phenomenon. As shown in Figure 15, the results of the AS-PSH securing all the unused capacity as the primary reserve in the low net-load period were derived. Therefore, the maximum available capacity of nuclear power generators, which are economic resources, was the highest in Scenario D.

**Figure 15.** Reserves of AS-PSHs and conventional generators in Scenario D. **Figure 15.** Reserves of AS-PSHs and conventional generators in Scenario D.

#### **5. Conclusions 5. Conclusions**

In this study, we performed RCUC simulations for the Korean power system projected for 2030 and analyzed the economic effect obtained by the flexibility of AS-PSH. Prior literature focused on the traditional advantages of PSHs, such as energy storage and quick-start capabilities, and provided insufficient analysis on the economic effect from the primary reserve capability of AS-PSH. In this study, we focused on the reserve capability of AS-PSH, considering that AS-PSH in pumping mode can provide primary, secondary, and regulation reserves. To analyze the economic effect obtained by AS-PSH in environments with a high share of VRE, we considered the short-term variations of VRE in the spinning reserve requirement defined by the Korean power market. We then divided the flexibility provided by AS-PSH into primary, secondary, and regulation reserves and modeled them in the RCUC formulation to allow for a range of operations in the pumping mode. In the simulation, it was postulated that the spinning reserve is procured only through generators, including PSHs. We simulated four scenarios: Scenario A denoted no PSH plant construction, Scenario B denoted FS-PSH plant construction. Scenarios C and D denoted AS-PSH plant construction. Scenario D showed the best cost result. In this study, we performed RCUC simulations for the Korean power system projected for 2030 and analyzed the economic effect obtained by the flexibility of AS-PSH. Prior literature focused on the traditional advantages of PSHs, such as energy storage and quick-start capabilities, and provided insufficient analysis on the economic effect from the primary reserve capability of AS-PSH. In this study, we focused on the reserve capability of AS-PSH, considering that AS-PSH in pumping mode can provide primary, secondary, and regulation reserves. To analyze the economic effect obtained by AS-PSH in environments with a high share of VRE, we considered the short-term variations of VRE in the spinning reserve requirement defined by the Korean power market. We then divided the flexibility provided by AS-PSH into primary, secondary, and regulation reserves and modeled them in the RCUC formulation to allow for a range of operations in the pumping mode. In the simulation, it was postulated that the spinning reserve is procured only through generators, including PSHs. We simulated four scenarios: Scenario A denoted no PSH plant construction, Scenario B denoted FS-PSH plant construction. Scenarios C and D denoted AS-PSH plant construction. Scenario D showed the best cost result.

The key findings of this study are as follows. The key findings of this study are as follows.


This study is limited in that the patterns of load curves vary depending on weekdays or weekends and are influenced by the seasons. The characteristics of VRE are also influenced by the seasons. Therefore, it is necessary to evaluate the economic effect over a longer timeframe in a follow-up study. In a long-term evaluation, a more simplified model for long-term economic analysis should be developed because the computational burden of the MIP model increases significantly as the analysis period increases. However, in a simplified model, the results of a short-term analysis model may not be guaranteed. Therefore, in a future long-term economic evaluation model, it will be necessary to develop a novel algorithm capable of considering the operational characteristics derived from the short-term economic evaluation model. This study is limited in that the patterns of load curves vary depending on weekdays or weekends and are influenced by the seasons. The characteristics of VRE are also influenced by the seasons. Therefore, it is necessary to evaluate the economic effect over a longer timeframe in a follow-up study. In a long-term evaluation, a more simplified model for long-term economic analysis should be developed because the computational burden of the MIP model increases significantly as the analysis period increases. However, in a simplified model, the results of a short-term analysis model may not be guaranteed. Therefore, in a future long-term economic evaluation model, it will be necessary to develop a novel algorithm capable of considering the operational characteristics derived from the short-term economic evaluation model.

**Author Contributions:** Conceptualization, W.-J.K.; data curation, W.-J.K.; formal analysis, W.-J.K.; funding acquisition, Y.-H.C.; investigation, W.-J.K. and Y.-S.L.; methodology, W.-J.K. and Y.-S.L.; **Author Contributions:** Conceptualization, W.-J.K.; data curation, W.-J.K.; formal analysis, W.-J.K.; funding acquisition, Y.-H.C.; investigation, W.-J.K. and Y.-S.L.; methodology, W.-J.K. and Y.-S.L.; software, W.-J.K. and H.-S.J.; supervision, Y.-H.C.; validation, W.-J.K. and Y.-S.L.; writing—original

draft, W.-J.K., Y.-S.L., and H.-S.J.; writing—review and editing, W.-J.K. and Y.-H.C. All authors have read and agreed to the published version of the manuscript.

**Funding:** This work was supported by the Korea Institute of Energy Technology Evaluation and Planning (KETEP) and the Ministry of Trade, Industry and Energy (MOTIE) of the Republic of Korea (No. 2019371010006B).

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **Nomenclature**

Sets and indices


Parameters and functions


Variables


Solar and wind power model parameters


## **References**


## *Article* **Techno-Economic Evaluation of Hybrid Energy Systems Using Artificial Ecosystem-Based Optimization with Demand Side Management**

**Hammed Olabisi Omotoso \*, Abdullah M. Al-Shaalan, Hassan M. H. Farh and Abdullrahman A. Al-Shamma'a \***

Electrical Engineering Department, College of Engineering, King Saud University, Riyadh 11421, Saudi Arabia; shaalan@ksu.edu.sa (A.M.A.-S.); hfarh1@ksu.edu.sa (H.M.H.F.)

**\*** Correspondence: homotoso@ksu.edu.sa (H.O.O.); ashammaa@ksu.edu.sa (A.A.A.-S.)

**Abstract:** Electrification of remote rural areas by adopting renewable energy technologies through the advancement of smart micro-grids is indispensable for the achievement of continuous development goals. Satisfying the electricity demand of consumers while adhering to reliability constraints with docile computation analysis is challenging for the optimal sizing of a Hybrid Energy System (HES). This study proposes the new application of an Artificial Ecosystem-based Optimization (AEO) algorithm for the optimal sizing of a HES while satisfying Loss of Power Supply Probability (LPSP) and Renewable Energy Fraction (REF) reliability indices. Furthermore, reduction of surplus energy is achieved by adopting Demand Side Management (DSM), which increases the utilization of renewable energy. By adopting DSM, 28.38%, 43.05%, and 65.37% were achieved for the Cost of Energy (*COE*) saving at 40%, 60%, and 80% REF, respectively. The simulation and optimization results demonstrate the most cost-competitive system configuration that is viable for remote-area utilization. The proposed AEO algorithm is further compared to Harris Hawk Optimization (HHO) and the Future Search Algorithm (FSA) for validation purpose. The obtained results demonstrate the efficacy of AEO to achieve the optimal sizing of HES with the lowest *COE*, the highest consistent level, and minimal standard deviation compared with HHO and FSA. The proposed model was developed and simulated using the MATLAB/code environment.

**Keywords:** Demand Side Management; Artificial Ecosystem-based Optimization; optimal sizing; Hybrid Energy System; Future Search Algorithm; Harris Hawk optimization; Renewable Energy Fraction; surplus energy; Loss of Power Supply Probability

## **1. Introduction**

The reduction of conventional energy sources coupled with increasing global warming have accelerated the growth of renewable energy sources (RES) such as solar and wind [1]. RES can be exploited for both grid connection and for off-grid, especially in rural areas with restricted grid connection [2–4]. This will reduce fossil fuel dependency, harmful emissions, and consumption costs. In spite of their benefits, RES performance is often limited due to the intermittent and unpredictable nature of their output power [5]. These factors affects both energy production and the operational costs of the system. Connecting two or more RES is essential to sustain energy for remote areas by preserving the quality and reliability of power [6]. Likewise, incorporating diesel generators with RES can ensure service quality and reliability and results in less battery maintenance [7,8]. Therefore, it is essential to optimize Hybrid Energy System (HES) size to minimize installation and maintenance cost.

Several studies have adopted analytical, probabilistic, and heuristic techniques in achieving HES optimization. The probabilistic technique is modelled on the random prospect of a particular system [9,10]. However, this method is not robust for obtaining optimal result with dynamic change in RES. The analytical technique is only capable of

**Citation:** Omotoso, H.O.; Al-Shaalan, A.M.; Farh, H.M.H.; Al-Shamma'a, A.A. Techno-Economic Evaluation of Hybrid Energy Systems Using Artificial Ecosystem-Based Optimization with Demand Side Management. *Electronics* **2022**, *11*, 204. https://doi.org/10.3390/ electronics11020204

Academic Editor: Carlos Andrés García-Vázquez

Received: 9 November 2021 Accepted: 4 January 2022 Published: 10 January 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

handling simple and precise methods [11,12]. The potent results of the heuristic methods [13–20] make them dependable for complex optimal sizing of HES with sufficient computational time. In [13], the Harmony Search (HS) algorithm was utilized for sizing PV/diesel energy sources. The obtained result in [13] demonstrated the robustness of HS in reducing pollution and system cost. Masoud et al. [14] demonstrated the effectiveness of Dynamic Multi-Objectives Particle Swarm Optimization (DMOPSO) for sizing the HES. By utilizing the DMOPSO algorithm, the results revealed a decrease in PV panel cost and NPC. In [15], a genetic algorithm combined with particle swarm optimization was used for the optimal design of a hybrid wind–PV–battery system. Loss of Load Expected (LOLE) and Loss of Energy Expected (LOEE) were subjected to constraints to minimize operation cost using multi-objective particle swarm optimization [16]. Eftichios et al. [17] presented a genetic algorithm using the system cost function subject to Loss of Load Hours (LOLH) constraints for HES optimization. Minimization of *COE* is presented in [18] to optimize the size of the HES subject to Renewable Energy Fraction (REF) and LPSP constraints. In [17], the levelized cost of energy was obtained through optimal sizing of PV, wind, battery, and diesel generator. Wang et al. [20] proposed a non-dominated sorting algorithm II, incorporated with re-ranking based genetic operators to determine system reliability, greenhouse gas emission, and lifetime cost for optimal sizing of the HES. Crow Search (CS) and Particle Swarm Optimization (PSO) were introduced in [21,22] for optimal sizing and allocation of renewable distributed generations. Morteza et al. presented a solar-hydrogen model for hybrid energy sizing using Demand Side Management (DSM) [23]. The results revealed the decrease in electricity cost; however, the model did not utilize system constraints. Hassan et al. [24] presented a strategy to allocate renewable distributed generations using CS with PSO.

The majority of the optimal sizing of HES in the literature did not apply DSM concepts. DSM is the future of power systems, incorporating cutting-edge technologies and distributions system for supplying electricity in efficient and smart ways [25]. DSM buttresses smart grid in several ways, such as control, electricity cost reduction, monitoring, and management of energy resources [26]. DSM can help to reduce load peak, which increases the efficiency of grid operation, promotes a decrease in the greenhouse gas effect, and leads to electricity bill reduction. Effective DSM can avoid the upgrading of electrical infrastructures such as transmission lines and distribution networks. Some of the generic method of DSM [27] are valley filling, load conservation, peak clipping, load growth or load establishment, and load shifting. Peak clipping is the reduction of peak loads without shifting it to off-peak hours. Valley filling involves encouraging consumption during the off-peak period. Load conservation involves encouraging consumers to make use of efficient appliances to reduce energy wastage. Load growth is adopted in a situation of surplus energy that might be the result of the integration of renewable energy. Load shifting is most commonly utilized and involves the shifting of load from peak periods to off-peak periods or periods where the energy cost is cheap [27]. Cost saving is the major purpose of the load-shifting process [28]. This method will be utilized in this study for the optimal sizing of the HES. Figure 1a–d shows a pictorial representation of various DSM strategies.

Thus, the foremost objective of this study is to extend the work presented in [29] by incorporating DSM for the optimal sizing of HES for remote communities of Al Sulaymaniyah village, Saudi Arabia, using a new optimization algorithm. The evaluation of three algorithms: Artificial Ecosystem-based Optimization (AEO) [30], Future Search Algorithm (FSA) [31], Harris Hawk Optimization (HHO) [32], were evaluated for optimal sizing. These algorithms were analyzed for optimizing Cost of Energy (*COE*) at zero Loss of Power Supply Probability (LPSP). Additionally, the Renewable Energy factor is proposed to ensure a hybrid energy utilization among the energy sources: diesel, battery storage, PV, and wind. The results demonstrated the efficacy of AEO in obtaining the lowest *COE* with and without DSM.

**Figure 1.** (**a**) Peak Clipping; (**b**) Valley Filling; (**c**) Load Shifting; (**d**) Load establishment.

The novelty and contribution of this paper can be summarized as follows:


The rest of the paper is arranged as follows: Section 2 describes the Al Sulaymaniyah site and load; Section 3 proposes the HES configuration; Section 4 discusses the AEO algorithm; Section 5 describes the power management approach; Section 6 introduces the developed DSM; Section 7 describes the reliability indices; Section 8 illustrates the objective function; Section 9 discusses the results; and Section 10 presents the conclusion.

### **2. Al Sulaymaniyah Site Description and Meteorological Data**

The geographical location of Al Sulaymaniyah is shown in Figure 2 [29]. The village is located in the northern part of Saudi Arabia at an elevation of 1820 ft above sea level. The location has enormous availability of wind and solar energy. Al Sulaymaniyah village comprises mosques, a water pump, a small hospital, and recreation facilities. The village is supplied through diesel generators by the Saudi Electricity Company (SEC). The climatic conditions at the village are continental. The village's temperature ranges between 0 ◦C and 45 ◦C throughout the year. The summer period is usually from May to September, while the winter season is from November to February. The day period varies throughout the year. The lengthiest day occurs in June, with 14 h of sunlight, and the shortest day occurs in December, with 10 h of sunlight.

**Figure 2.** Al Sulaymaniyah village, Saudi Arabia.

The measured average hourly energy consumption at Al Sulaymaniyah village comprises 8760 data points. Currently, the village is powered by three 456 kW diesel generators connected in parallel to a common bus. The generated power is connected to a 1250 kVA step-up transformer. The full description of Al Sulaymaniyah meteorological data and load pattern are described in [29].

### **3. Configuration of the HES**

The proposed structure for the Hybrid Energy System is presented in Figure 3 [33]. This comprises battery storage, the inverter for converting DC energy sources to AC, diesel generators, load, PV, and wind energy sources. In this study, PV and wind are considered as the two major sources of energy. Either the diesel generator or storage energy system compensates the disparity between PV and wind. Al Sulaymaniyah's low PV and wind coefficient of correlation implies that PV and wind might not be able to adequately supply the energy required without the utilization of energy storage and diesel sources. PV and storage system are connected to the DC side of the bus bar, while the diesel generator and wind turbines are linked to the AC side of the bus bar. The diesel generator serves as an alternative source of energy whenever the PV, wind, and storage system are unable to meet the load demand.

**Figure 3.** Overview of the proposed Hybrid Energy System.

The modelling of the PV, wind, storage system, and diesel generator is introduced as follows.

#### *3.1. PV Modelling*

Solar irradiance, PV array manufacturer's data, and temperature are major factors that influence the output power generated from PV systems, as shown in (1) [34].

$$P\_{PV} = \,^p\_{r}f\,pv\left[\frac{\overline{G\_T}}{\overline{G\_{T,ST\mathbb{C}}}}\right](1+\alpha p(T\_c - T\_{c,ST\mathbb{C}}))\tag{1}$$

where *P<sup>r</sup>* , *Tc*,*STC*, *Tc*, *GT*, *GT*,*STC*, *αp*, and *f pv* are the PV rated power, PV temperatures under STC and normal conditions, global irradiance under normal and Standard Test Conditions (STCs), power temperature coefficient, and de-rating factor, respectively. The temperature of the PV steady-state operation is described in (2).

$$T\_c = \frac{T\_a + \left(NOCT - T\_{a, \text{NOCT}}\right)\left(1 - 1.11\_{\eta \text{MPP}}(1 - \alpha p T\_{c, \text{STC}})\right)\left(\frac{\overline{G\_T}}{\overline{G\_{T, \text{NOCT}}}}\right)}{1 + 1.11(\alpha \rho \eta\_{\text{MPP,STC}})\left(NOCT - T\_{a, \text{NOCT}}\right)\left(\frac{\overline{G\_T}}{\overline{G\_{T, \text{NOCT}}}}\right)}\tag{2}$$

where *NOCT* and *T<sup>a</sup>* are operating cell nominal and ambient temperature, respectively. *GT*,*NOCT* and *Ta*,*NOCT* are, respectively, the solar irradiance and ambient temperature with respect to *NOCT*; *η*MPP and *η*MPP,*STC* are the Maximum Power Point (MPP) efficiency of the PV module and efficiency under STCs, respectively.

#### *3.2. Wind Turbine Generator (WTG) Modelling*

The approximated curve of power output of a wind turbine system can be expressed as follows [35]:

$$P\_{WT}(u) = P\_r \times \begin{cases} 0 & u \le u\_c \text{ or } u > u\_f \\ \frac{u^2 - u\_c^2}{u\_r^2 - u\_c^2} & u\_c < u \le u\_r \\ 1 & u\_r \le u \le u\_f \end{cases} \tag{3}$$

where *u*, *uc*, *P<sup>r</sup>* , *u<sup>r</sup>* , and *u<sup>f</sup>* are wind speed, cut-in/starting speed of wind turbine, rated power of wind turbine, rated speed, and cut-out/furling speed, respectively. It is evident from Equation (3) that wind turbine output power depends on five parameters, which are the four wind turbine parameters and the site wind speed (*u*). The four wind turbine parameters are *uc*, *P<sup>r</sup>* , *u<sup>r</sup>* , and *u<sup>f</sup>* . Based on a quadratic model, a typical wind turbine output power characteristic curve is shown in Figure 4 [29].

**Figure 4.** Wind turbine output power characteristics curve.

#### *3.3. Storage System Modelling*

One of the critical aspects of RES is the storage system because of the stochastic nature of RES output power. The depth of discharge and battery bank nominal capacity are important factors to be considered in storage system. The load demand determines the energy flow through the storage system. The State of Charge (*SOC*) of the battery bank is represented as (4) [29].

$$\text{SOC}(t) = \text{SOC}(t-1)(1-\sigma) + \left(P\_{GA}(t) - \frac{P\_L\left(t\right)}{\eta\_{inv}}\right)\eta\_{battery} \tag{4}$$

where *PL*(*t*), *ηinυ*, *ηbattery*, *PGA*(*t*), and *σ* are total load demand, net inverter efficiency, round trip efficiency of battery within [0.5, 0.95], total output power, and self-discharging rate, respectively. The charging and discharging operation of the storage system is described by the positive and negative second term of Equation (4). The constraints for the storage system State of Charge (*SOC*) are described in (5).

$$\text{SOC}(t) = \begin{cases} \text{SOC}\_{\text{min}} & \text{SOC}(t) < \text{SOC}\_{\text{min}} \\ \text{SOC}(t) & \text{SOC}\_{\text{min}} < \text{SOC}(t) < \text{SOC}\_{\text{max}} \\ \text{SOC}\_{\text{max}} & \text{SOC}(t) > \text{SOC}\_{\text{max}} \end{cases} \tag{5}$$

The *SOC* cannot exceed the maximum limit of *SOCmax*, and likewise the minimum *SOC* must be equal to or exceed the minimum allowable limit, *SOCmin*.

#### *3.4. Diesel Generator*

The diesel generator is utilized whenever the storage and RES is insufficient to sustain load demand. The annual fuel consumption and the cost is represented as (6) [35].

$$\mathcal{C}\_{\rm Dsl} \;= \mathcal{C}\_{\rm F} \sum\_{t=1}^{8760} A \times P\_{\rm Dsl}(t) + B \times P\_{\rm R} \tag{6}$$

where *CDsl*, , *PR*, *CF*, and *PDsl*(*t*) are the annual fuel consumption (L/h), nominal power, cost of fuel per liter, and generated power (kW) of diesel, respectively. *A*, *B* are the fuel constants (L/kW). The two main import factor that influences the fuel consumption are both the output power and nominal power. The output power of the diesel generator is recommended by the manufacturers between a preset minimum level and nominal power.

#### **4. Artificial Ecosystem-Based Optimization (AEO)**

AEO is a type of population-based optimization algorithm. It imitates the production, decomposition, and consumption behavior of living organisms. This concept depicts energy flow in the ecosystem of the Earth. In the Earth's ecosystem, producers use water, sunlight, and carbon dioxide to make food energy. Similarly, the AEO production stage is used to improve the balance between exploration and exploitation. The second stage, consumers, just like animals, cannot produce their own food. They acquire their energy and nutrients from fellow consumers or producers. This stage improves the exploration of the algorithm. Decomposition is the last stage. Decomposers feed on both consumers and producers. The exploitation of the algorithm is enriched at this stage [30].

#### *4.1. Producer*

In the search space at this stage, a random individual (*xrandi*) and a best individual (*xq*) are randomly produced. The decomposer (best individual) and the search space's upper and lower boundaries update the producer (worst individual). This update will facilitate other individuals to hunt for separate regions. The mathematical representation for this is illustrated in (7) [30].

$$\mathbf{x}\_1(ti+1) = (1-a)\mathbf{x}\_q(ti) + a\mathbf{x}\_{randi}(ti)\tag{7}$$

where

$$a = \left(1 - \frac{t i}{I t}\right) \* r\_1 \tag{8}$$

$$\mathbf{x}\_{randi} = r \ast (\mathbf{U}p - \mathbf{L}w) + \mathbf{L}w \tag{9}$$

$$\mathbf{x}\_2(ti+1) = \mathbf{x}\_2(ti) + \mathbf{K}[\mathbf{x}\_2(ti) - \mathbf{x}\_1(ti)] \tag{10}$$

where *r*<sup>1</sup> and *r* are random numbers within 0 and 1, *It* is maximum iterations, *a* is linear weighting coefficient, *q* is the population number, and *Lw* and *Up* are the lower and upper boundaries, respectively. The flight levy, *K*, is for enhancement of the exploration level. This is represented in (11) [36]:

$$K = 0.5 \ast v\_1 / |v\_2| \tag{11}$$

where *v*<sup>2</sup> = *N*(0, 1) and *v*<sup>1</sup> =∼ *N*(0, 1). *N*(0, 1) denotes normal distribution.

#### *4.2. Consumption*

This stage enhances the exploration by letting the algorithms upgrade the solution of individuals. The consumers can be classified as herbivore, carnivore, or omnivore. The herbivore feeds on both the consumers and the producers. The second feed on the consumers with an advanced energy level. The last feeds on the producers and/or consumers with an advanced energy level. The mathematical equation for herbivore as a consumer is represented as follows [36]:

$$\mathbf{x}\_{i}(ti+1) = \mathbf{x}\_{i}(ti) + \mathbf{K}[\mathbf{x}\_{i}(ti) - \mathbf{x}\_{i}(ti)], i \in [\mathfrak{Z}, \dots, n] \tag{12}$$

The mathematical equation for consumer as a carnivore is represented as follows:

$$\begin{cases} \mathbf{x}\_{i}(ti+1) = \mathbf{x}\_{i}(ti) + \mathbf{K} \left[ \mathbf{x}\_{i}(ti) - \mathbf{x}\_{j}(ti) \right], \\\ i \in \left[ \mathbf{3}, \dots, n \right], \ j = randi(\left[ 2 \ i - 1 \right]) \end{cases} \tag{13}$$

The mathematical equation for an omnivore consumer is denoted as:

$$\begin{cases} \mathbf{x}\_{i}(ti+1) = \mathbf{x}\_{i}(ti) + \mathbf{K} \left[ r\_{2} (\mathbf{x}\_{i}(ti) - \mathbf{x}\_{1}(ti)) + (1 - r\_{2}) \left( \mathbf{x}\_{i}(ti) - \mathbf{x}\_{j}(ti) \right) \right], \\\quad i \in \left[ 3, \ldots, n \right], \; j = randi(\left[ 2 \; i - 1 \right]) \end{cases} \tag{14}$$

where *r*<sup>2</sup> is within the range of 0 and 1.

#### *4.3. Decomposers*

The decomposition stage is very important because it feeds the producers and encompasses the food chain. In Earth's ecosystem, when a consumer passes away, the decomposers feed on its leftovers. The mathematical Equation (15) for the decomposer model introduces new factors. The factors are weight coefficients (*w<sup>e</sup>* and *he*) and decomposition factor (*De*). The decomposition supports the exploitation of the algorithm by updating individual locations based on the best solution, as described in (16) [30].

$$\mathbf{x}\_{i}(ti+1) = \mathbf{x}\_{n}(ti) + D\_{\varepsilon}(w\_{\varepsilon}\mathbf{x}\_{n}(ti) - h\_{\varepsilon}\mathbf{x}\_{i}(tt)), \ i = 1, \ldots, n \tag{15}$$

The newly introduced factors are defined by Equation (16), shown below.

$$\begin{cases} D\_\varepsilon = \mathfrak{Z}u\_\prime & u \sim N(0, 1) \\ w\_\varepsilon = r\_3 \ast rand([1, 2]) - 1 \\ h\_\varepsilon = 2 \ast r\_3 - 1 \end{cases} \tag{16}$$

where *r*<sup>3</sup> is within the range of 0 and 1.

#### *4.4. Termination*

During this stage, *x<sup>q</sup>* is updated after the fitness is obtained for each individual. Subsequently, the termination condition is evaluated; if it is met, the execution is terminated and *x<sup>q</sup>* is returned, or else the first stage is repeated. Figures 5 and 6 show the typical flow of energy in Earth's ecosystem and the AEO ecosystem [30], and Algorithm 1 summarized the pseudocode AEO algorithm.

**Figure 5.** Energy flow in an ecosystem.

**Figure 6.** AEO ecosystem.

**Algorithm 1:** Pseudocode of Artificial Ecosystem-based Optimization for optimal the sizing of HES

**1. Initialization:** Random initialization of AEO ecosystem, *x*<sup>1</sup> , and evaluation of fitness, *ffi* ; *xq* = best solution established so far. 2. **While** the halt condition is not obtained, perform: **First Stage: Production** Individual *x*<sup>1</sup> , update its position with (7). **Second Stage: Consumption** Individual *x*<sup>1</sup> (i = 2, . . . ,n) Herbivorous act occurs If *rand* < <sup>1</sup> 3 , the individual update is carried out using (12) Omnivorous act occurs Else if 1/3 and *rand* < 2/3 the update to individual is carried out using (14) Carnivorous act occurs Else the individual update is carried out with (13) End if End if **Third stage:** Decomposition Individual update is carried out with (15) Individual fitness is calculated Best position found so far is updated, *xq* End while **Fourth Stage:** Termination Return *xq*

#### **5. Power Management Approach**

The unstable nature of renewable resources creates a multifaceted power management scheme for HES. Due to the fact that the quantity of energy generated from resources is paltry, the capability of the generator cannot be instantaneously amplified to balance the increase in load. Moreover, occasionally, a dump load is required to dispel the surplus when the quantity of electricity generated is more than the demand. Moreover, this will avoid the overcharging of the batteries. To design an effective system, it is essential to adopt a power management methodology. Table 1 shows the parameters used for the simulation [35]. The following cases are considered for the power management methodology.

Case 1: The required energy is supplied by renewable generation and the surplus is utilized for charging the storage facility.

Case 2: As in Case 1, if the excess energy is greater than the storage capacity and load, the surplus is dissipated via a dump load.

Case 3: The storage facility is utilized as a priority when the renewable energy is insufficient to meet the load rather than making use of diesel.

Case 4: The generator is used to charge the battery and to supply the load when the renewable energy is insufficient to sustain the load and the storage is depleted.


**Table 1.** Parameters of the HES system.

#### **6. Demand Side Management (DSM)**

Demand Side Management modifies the consumption patterns of consumer electricity to yield the desired alterations in the load structure of power distributions. The resulted changes are expected to reflect the planned objectives. This study presents a load-shifting technique to maximize the utilization of renewable energy resources and reduce the energy supply from the diesel. The objective load curve is aimed at reducing the load during periods of insufficient renewable energy and likewise increasing the load during periods of surplus renewable energy. The DSM strategy cannot be scheduled beyond a day, and the shifted load is limited to 20% of the total load. If the renewable energy is less than the load and within 24 h, as represented in (17), the load shift is described as (18). If (18) is greater than 20% of the load, the load shift is donated as (19). More so, by considering the future surplus available renewable energy, the load shift is limited to (20) if the available renewable energy is less than the expected shifted load in (19). The load at time, *t*, and future time, *t* + *i*, are represented as (21) and (22), respectively. The methodology also applies when there is surplus renewable energy.

$$P(t) + P(w) < \left(\frac{P1(t)}{univ}\right) \&\; t < 24 \text{ h} \tag{17}$$

$$dt = \left(\frac{P1(t)}{\
u niv}\right) - \left(Pw(t) + Pp(t)\right) \tag{18}$$

$$lt = 0.2 \ast \text{P1}(t) \tag{19}$$

$$lt = P(ren)\tag{20}$$

$$P1(t) = P1(t) - lt \tag{21}$$

$$P1(t) = P1(t) - lt \tag{22}$$

where *P*(*t*), *P*(*w*), *P*1(*t*), *univ*, *P*(*ren*), and *lt* represents PV power, wind power, load, inverter efficiency, surplus renewable energy, and load shift at time, *t*, respectively.

#### **7. Reliability Indices**

#### *7.1. Loss of Power Supply Probability*

The probability of power supply failure can be measured using one of the statistical reliability indices, Loss of Power Supply Probability (LPSP). Loss of power can be as a result of technical failure or low energy supply from the generating source that is not able to meet the demand [30]; 0% and 1% LPSP demonstrate, respectively, that the load will not be supplied and the load will be supplied. This can be calculated as described in (23) [35].

$$LPSP = \frac{\sum \left(P\_{load} - P\_{pv} - P\_{wind} + P\_{soc\_{min}} + P\_{diesel}\right)}{\sum P\_{load}}\tag{23}$$

where *Pload*, *Ppv*, *Pwind*, *Psocmin* , *and Pdiesel*, are load demand, PV power, wind power, minimum battery power, and diesel power, respectively.

#### *7.2. Renewable Energy Fraction (REF)*

This is the ratio of diesel energy generated compared to energy generated from both the wind and the PV system, as shown below. The fraction is an indication of how much renewable energy is served. A fraction of 100% represents an ideal system, with only renewable energy resources. An equal sum of PV and wind power to diesel power represents a fraction of 0%. A fraction between 0% and 100% represent a hybrid energy source. *REF* is represented as (24) [29].

$$REF = \left(1 - \frac{\sum P\_{diesel}}{\sum P\_{wind} + P\_{pv}}\right) \ast 100\tag{24}$$

#### **8. Objective Function**

Appropriate sizing of the energy system with high quality units can lead to resourceful energy management and low-cost electricity for the consumers. The aim of the optimization is to obtain a minimal Cost of Energy (*COE*). *COE* is a widely used indicator for HES financial profitability [31]. This is defined as the ration of cost to unit price of electricity, as described in (25), subject to constraints (26) [29,35].

$$COE\left(\frac{\\$}{kWh}\right) = \begin{array}{c} \text{Annualized cost}(\\$) \\ \stackrel{\text{annual energy delivered by the system} (kWh)}{=} \\ = \frac{\text{Total Net present Cost} \left(\\$\right)}{\text{P}\_{\text{load}}(kW)\left(\frac{8960}{year}\right)} \* \text{CRF} \end{array} \tag{25}$$

$$\text{Constraints} \left\{ \begin{array}{c} \text{LPSP} \le \text{LPSP}\_{\text{desired}} \\ \text{REF} \le \text{REF}\_{\text{desired}} \\ 0 \le P\_{\text{Wind}} \le P\_{\text{wind,max}} \\ 0 \le P\_{\text{pv}} \le P\_{\text{pv},\text{max}} \\ 0 \le P\_{\text{Bst}} \le P\_{\text{bst,max}} \\ 0 \le P\_{\text{DSL}} \le P\_{\text{DSL,max}} \end{array} \right. \tag{26}$$

The total NPC comprises installed capital costs. The installed capital costs includes the replacement cost, operation cost, maintenance cost, and present cost. *Pload* is the power consumed per hour. The current worth of HES components for a specific duration with the interest rate is calculated with the Capital Recovery Factor (*CRF*), which is shown below in (27) [35].

$$\text{CRF} = \frac{i\*(1+i)^n}{\left(1+i\right)^n - 1} \tag{27}$$

where *n* is the system life period, which is represented by PV life duration because of its lengthier life, and *i* represents interest rate. *Pwind*, *Ppv*, *PDSL*, and *Pbat* represent the rated power of wind, solar, diesel, and battery (in kWh), respectively, with their maximum denoted as *Pwind*,*max*, *Ppv*,*max*, *PDSL*,*max*, and *Pbat*,*max*. *REFdesired* and *LPSPdesired* denote the defined *REF* and LPSP, respectively.

#### **9. Simulation Results and Discussions**

The proposed AEO, in addition to HHO and FSA, were employed to determine the optimal sizing of the stand-alone Hybrid Energy System to obtain the minimal Cost of Energy (*COE*). The hourly average load, wind speed, and solar irradiance of Al Sulaymaniyah village, Saudi Arabia, were selected for the study of optimal energy sizing in this study. The variables for the optimization are the capacity of the PV energy source, batteries, diesel generators, and wind turbine energy. The performance of the algorithms was evaluated under four performance indicators, which are Standard Deviation (STD), mean, best solution, and worst solution. The metaheuristics algorithm involves a stochastic model in their computational analysis. As a result, each run of the algorithm will not yield the same result. This creates an enormous challenge to determine the quality of the solution. Hence, it is crucial to investigate the consistency of the solution. By running the algorithms for 30 runs, a unique distinct pattern is obtained for the algorithms, which implies that the parameters always unite at a definite location. To obtain a rational comparison, the algorithms were evaluated under the same population sizes, iterations, boundary conditions, and number of runs. The simulation was developed using the MATLAB environment. Moreover, different levels of REF at 40%, 60%, and 80% at an LPSP of 0% were employed to achieve the *COE* and optimal sizing.

#### *9.1. Optimal Sizing of the HES without DSM*

One substantial standard for evaluating the performance of any optimization method is the assessment of the obtained fitness results. Tables 2–4 show the Standard Deviation (STD), mean, best solution, and worst solution obtained for REF at 40%, 60%, and 80%, respectively. These results were evaluated during 30 runs, 300 iterations, and a population size of 50 without DSM at 0% LPSP. Figures 7–9 show the fitness curve of the three algorithms without DSM at 40%, 60%, and 80% REF, respectively. From the results, it is obvious that AEO obtained the minimum *COE* for all three stages. The algorithm further maintains a low STD value in obtaining the *COE*, as shown in Tables 2–4.


**Table 2.** Optimal, Minimum, Maximum, and STD for AEO, FSA, and HHO at 40% REF and LPSP of 0%.

**Table 3.** Optimal, Minimum, Maximum, and STD for AEO, FSA, and HHO at 60% REF and LPSP of 0%.


**Table 4.** Optimal, Minimum, Maximum, and STD for AEO, FSA, and HHO at 80% REF and LPSP of 0%.



**Figure 7.** Fitness for cost of energy without DSM at 40% REF and 0% LPSP.

**Figure 8.** Fitness for cost of energy without DSM at 60% REF and 0% LPSP.

**Figure 9.** Fitness for cost of energy without DSM at 80% REF and 0% LPSP.

Figure 10 shows the percentage energy utilization for each of the algorithms without DSM. The run with the lowest *COE* for each algorithm was selected for the energy utilization plot. AEO utilizes the right combination of PV and wind energy sources to obtain the minimal cost of energy. Figure 10 shows that FSA utilizes the highest percentage for diesel energy. This influences its *COE* to be highest among the algorithms.

**Figure 10.** Energy utilization percent for AEO, FSA, and HHO without DSM and at 0% LPSP.

#### *9.2. Optimal Sizing of the HES with DSM*

The DSM is performed to optimize the usage of the available renewable energy resources. The loads are shifted in two categories: if there is insufficient renewable energy to meet the load or when there is surplus renewable energy. In the case of the surplus energy, future loads that are less than the available renewable energy are shifted to present time. All these load shifting are limited to a 24 h time frame. Tables 5–7 show the Standard Deviation (STD), mean, best solution, and worst solution obtained for 30 runs under 300 iterations and a population size of 50 for the three stages of Renewable Energy Fraction (REF). Figures 11–13 show the fitness curve (*COE*) for the 30 runs of each algorithm at 40%, 60%, and 80% REF, respectively. It is obvious from Tables 5–7 and Figures 11–13 that AEO achieved the minimal cost of energy at very low STD for all the 30 runs at 0% LPSP. This further demonstrates the ability of AEO to maintain the lowest *COE* with the highest consistent level and minimal STD.


**Table 5.** Optimal, Minimum, Maximum, and STD for AEO, FSA, and HHO with DSM at 40% REF and 0% LPSP.

**Table 6.** Optimal, Minimum, Maximum, and STD for AEO, FSA, and HHO with DSM at 60% REF and 0% LPSP.


**Table 7.** Optimal, Minimum, Maximum, and STD for AEO, FSA, and HHO with DSM at 80% REF and 0% LPSP.



**Figure 11.** Fitness for cost of energy with DSM at 40% REF and 0% LPSP.

**Figure 12.** Fitness for cost of energy with DSM at 60% REF and 0% LPSP.

**Figure 13.** AEO, FSA, and HHO Cost of Energy with DSM at 80% REF and 0% LPSP.

Figure 14 shows the percentage energy utilization for each of the algorithms with DSM. The run with the lowest *COE* for each algorithm was selected for the energy utilization plot. From Tables 5–7 and Figures 11–13, AEO utilizes the right combination of PV and wind energy sources to obtain the minimal cost of energy compared to the other algorithms. In the Figure, FSA utilizes the highest percentage for diesel energy. This influences its *COE* to be highest among the algorithms.

**Figure 14.** Energy utilization percent for AEO, FSA, and HHO with DSM and at 0% LPSP.

*9.3. COE with DSM and without DSM*

The DSM in this study is aimed at reducing the surplus renewable energy. Comparing the several results obtained, it can be observed that reducing the surplus energy decreases the *COE*. Because AEO produces the optimal results among the algorithms, Table 8 shows the percentage energy cost savings and *COE* at different REF when DSM is introduced as obtained from AEO. It can be observed that, with the introduction of DSM, greater energy saving is achieved. Figures 15–17 show a typical one-day load profile for the AEO algorithm compared with and without Demand Side Management. The load profile shows how the surplus energy is utilized when there is low renewable energy and surplus renewable energy. In the figures, during the early hours of the day with low renewable energy supply, the loads are shifted to a later period in the day when there is a surplus renewable energy supply.

**Table 8.** Percentage energy cost saving and *COE* at different REF.


**Figure 15.** Daily load profile of AEO at 40% REF.

**Figure 16.** Daily load profile of AEO at 60% REF.

**Figure 17.** Daily load profile of AEO at 80% REF.

#### **10. Conclusions**

This paper presents a new application of AEO in the optimal sizing of a stand-alone Hybrid Energy System (HES). Al Sulaymaniyah, a rural area in Saudi Arabia, was selected for the optimal sizing of the HES. Cost of Energy (*COE*) was considered as the objective function of this model using both REF and LPSP as the reliability indices. Because of the abundance of renewable energy at the location, the optimal sizing selected both wind and solar energy as the major sources of energy. Furthermore, the optimal sizing was evaluated with and without Demand Side Management (DSM) under different Renewable Energy Fractions. It is obvious for both scenarios that AEO outperformed both HHO and FSA. By performing the DSM, more renewable energy was utilized while reducing the surplus unutilized energy. The DSM strategy achieved 28.38%, 43.05%, and 65.37% in *COE* saving at 40%, 60%, and 80% REF, respectively. The results stressed the importance of utilizing DSM for optimal sizing by further reducing the CEO. While the AEO algorithm introduced in this study has been shown to be efficient, it may not be efficient in other optimization problems. The efficacy of the algorithm can also be extended to the optimal load flow and optimal sitting of distributed generation.

**Author Contributions:** Conceptualization, H.O.O., A.M.A.-S. and A.A.A.-S.; methodology, H.O.O., A.A.A.-S. and H.M.H.F.; software, H.O.O.; validation, H.O.O., H.M.H.F. and A.A.A.-S.; formal analysis, H.O.O.; investigation, H.O.O.; resources, H.O.O. and A.A.A.-S.; data curation, H.O.O.; writing—original draft preparation, H.O.O.; writing—review and editing, H.O.O., A.M.A.-S., H.M.H.F. and A.A.A.-S.; visualization, H.O.O., A.M.A.-S., H.M.H.F. and A.A.A.-S.; supervision, H.O.O., A.M.A.-S., H.M.H.F. and A.A.A.-S.; project administration, H.O.O. and A.M.A.-S.; funding acquisition, A.M.A.-S. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Acknowledgments:** The authors would like to acknowledge the Researchers Supporting Project number (RSP-2021/337), King Saud University, Riyadh, Saudi Arabia.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **Abbreviations**

The following are abbreviations used.



#### **References**


## *Article* **Optimal Power Dispatch in Energy Systems Considering Grid Constraints**

**Alejandro Rubio \*, Frank Schuldt, Peter Klement and Karsten von Maydell**

DLR Institute of Networked Energy Systems, Carl-von-Ossietzky-Str. 15, 26129 Oldenburg, Germany; frank.schuldt@dlr.de (F.S.); peter.klement@dlr.de (P.K.); karsten.maydell@dlr.de (K.v.M.)

**\*** Correspondence: alejandro.rubio@dlr.de; Tel.: +49-441-99906-481

**Abstract:** As a consequence of the increasing share of renewable energies and sector coupling technologies, new approaches are needed for the study, planning, and control of modern energy systems. Such new structures may add extra stress to the electric grid, as is the case with heat pumps and electrical vehicles. Therefore, the optimal performance of the system must be estimated considering the constraints imposed by the different sectors. In this research, an energy system dispatch optimization model is employed. It includes an iterative approach for generating grid constraints, which is decoupled from the linear unit commitment problem. The dispatch of all energy carriers in the system is optimized while considering the physical electrical grid limits. From the considered scenarios, it was found that in a typical German neighborhood with 150 households, a PV penetration of ∼5 kW<sup>p</sup> per household can lead to curtailment of ∼60 MWh per year due to line loading. Furthermore, the proposed method eliminates grid violations due to the addition of new sectors and reduces the energy curtailment up to 45%. With the optimization of the heat pump operation, an increase of 7% of the self-consumption was achieved with similar results for the combination of battery systems and electrical vehicles. In conclusion, a safe and optimal operation of a complex energy system is fulfilled. Efficient control strategies and more accurate plant sizing could be derived from this work.

**Keywords:** sector coupling; optimal power flow; energy system optimization; grid flexibilization; oemof-Solph; PowerFactory

## **1. Introduction**

#### *1.1. Background*

Nowadays, there are several initiatives and international efforts to reduce the CO<sup>2</sup> emissions in all energy sectors as part of the Paris agreement [1]. In Germany, the so-called "Energiewende" establishes the goals for the energy transformation towards a zero-emission national energy system [2]. A fundamental step to achieve such ambitious goals is the electrification of the residential heat and transport sectors that accounted in 2016 for ∼10% and ∼18.2% of the total emissions in Germany, respectively [3]. By 2050, an increase of 50% of the district heating in Europe is expected, with approximately 30% of that demand being covered by heat pumps [4]. However, combined heat and power is expected to serve as a bridge technology coupling electricity and heat sectors [5]. Additionally, a fleet of around six million electrical vehicles is planned by the German government by 2030 [6]. This makes the analysis of modern and future energy systems more challenging, due to the added complexity of the new technologies and energy sectors. Along with a significant deployment of renewable sources in the electricity grid, the integration of these sector coupling technologies may add an extra burden to the existing distribution grids [7,8]. Therefore, new concepts and techniques are needed to properly study and optimize the grid structure for adequate operation of such new energy systems [9]. In this research, an approach that enables the ease evaluation and optimization of energy systems considering the power grid limits is developed. This methodology was based on the energy system for

**Citation:** Rubio, A.; Schuldt, F.; Klement, P.; von Maydell, K. Optimal Power Dispatch in Energy Systems Considering Grid Constraints. *Energies* **2022**, *15*, 192. https:// doi.org/10.3390/en15010192

Academic Editor: Tomonobu Senjyu

Received: 2 December 2021 Accepted: 23 December 2021 Published: 28 December 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

the "Energetisches Nachbarschaftsquartier Fliegerhorst Oldenburg", which will be a living laboratory in the city of Oldenburg, Germany [10].

#### *1.2. Optimization of Energy Systems*

Typically, the aggregation approach has been of common use in the literature when it comes to the analysis and optimization of energy systems [11]. Then, the optimization problem is independent of its actual physical characteristics. Mathematically, the energy system optimization problem can be expressed as:

$$\begin{aligned} \min & \; \sum\_{t \in T} \left( \sum\_{(s,\varepsilon) \in E} \sum\_{i \in I\_1} c\_{(s,\varepsilon)}^i(t) \cdot p\_{s,\varepsilon}^i(t) \cdot \tau \right) \\ & \quad + \sum\_{(s,\varepsilon) \in E} \sum\_{i \in I\_2} c\_{(s,\varepsilon)}^i \cdot p\_{(s,\varepsilon)}^i \\ & \quad + \sum\_{t \in T} \left( \sum\_{n \in N} \sum\_{i \in I\_3} c\_n^i(t) \cdot p\_n^i(t) \cdot \tau \right) \\ & \quad + \sum\_{n \in N} \sum\_{i \in I\_4} c\_n^i \cdot p\_n^i \end{aligned} \tag{1}$$

$$s.t.\sum\_{n\in\mathcal{N}}a\_n^j(t)\cdot p\_n(t) \ge \sum\_{m\in\mathcal{M}}d\_m^j(t), \quad \forall j\in\mathcal{J}, \forall t\in\mathcal{T} \tag{2}$$

$$0 \le p^i\_{(s,e)}(t) \le \overline{p}^i\_{(s,e)}(t), \quad \forall (s,e) \in E, \forall t \in T \tag{3}$$

$$0 \le p\_n^i(t) \le p\_n^i(t), \quad \forall n \in \mathcal{N}, \forall t \in T \tag{4}$$

The Equation (1) minimizes the cost in a period *T* at a resolution *τ*. The set *E* represents the energy flow between nodes, whereas the set *N* represents the flows between nodes and sources. The sets *I*<sup>1</sup> to *I*<sup>4</sup> establish the alternative to have multiple flows with different associated costs. Fixed and time-varying costs are included in *c*. The variable *p* denotes the power that flows between nodes and sources. The set *J* constitutes the types of sectors to consider (e.g., heat and electricity), whereas the set *M* represents the demand associated with the *j th* sector. The inequality constraint (2) ensures that the sum of the *M* demands for each *j th* sector is fulfilled by the sum of the sources multiplied by the coupling factor *a* corresponding to the *nth* source and the *j th* sector. The constraints (3) and (4) represent the boundary conditions for flows and sources.

Many approaches are found in the literature dealing with the optimization of energy systems considering the actual electrical grid topology. For power systems, the implementation of linearized power flows approaches to relax the optimization problem is of common use in the researched literature [12,13]. Whereas the typical DC power flow neglects reactive power, some studies have enhanced the method by adding the consideration of reactive power and with the implementation of data-driven approaches to determine voltage magnitudes and angles [14]. Novoa et al. [15] have applied a decoupled linearized power flow [16] in combination with a mixed-integer linear problem to find the optimal allocation of PV and batteries within an energy system. However, these methods are less accurate than the AC power flow and more complex than the DC power flow. The combination of a commercial tool to solve power flow problems with a linear unit commitment is presented by Nolden et al [17]. In this approach, only an electrical system without storage at a single time step is considered. Similarly, Fortenbacher et al. [11] propose a distributed model predictive control within sub-grids to solve a multi-period dispatch optimization problem.

When an entire energy system with multiple energy carriers is optimized, generally the physical structures are disregarded and the systems are simplified. Some solutions to work around the over-simplification of the models have been developed. Lohmeier et al have proposed specialized tools for each energy sector that can be implemented together in a co-simulation [18]. As the physical constraints of the energy systems are considered, the complexity in solving such problems also increases. For instance, nonconvex systems result when hydraulic equations are taken into consideration in an energy hub with multiple energy carriers [19]. The time-step dependency from energy storage may add extra complexity to the already mentioned systems. The utilization of the Newton– Rapson method is inefficient in time-domain equations. Therefore, Levron et al. [20] have used in combination with a power flow solver, dynamic programming to tackle the timedependent functions. As the share of renewable energies increases at the medium and low voltage of the grid, some power quality issues such as line over-loading and overvoltages may appear [7,9]. For this reason, the potential of CHP and P2G options has been considered to provide grid flexibilization [21,22].

The main contribution of this research is the development of a decoupled approach for the dispatch optimization of a sector coupled energy system. The method takes into consideration the electric grid limits to optimally integrate different energy sectors into an energy system. In order to assess the method, some scenarios comprising an energy system with high share of renewables, storage systems, and sector coupling technologies were considered. As a main result, it was found that high shares of PV without storage lead to a higher energy curtailment. Additionally, the proposed method was capable of reducing energy curtailment and increase self-consumption of the energy system by adding flexibility through the implementation of heat pumps and electric vehicles.

#### **2. Methodology**

In this section, the implemented methodology to consider grid limitations in dispatch optimization problems in energy systems is presented. The idea behind this work is to reduce the complexity of the optimization problem when considering grid constraints. The unit commitment problem and the power flow solutions are decoupled, the solution to the unit commitment is realized considering a mixed-integer linear problem (MILP) using *oemof-Solph* [23], whereas an AC optimal power flow (OPF) calculation is carried out in *PowerFactory* [24] to verify grid quality standards compliance. From this OPF, grid congestion and voltage violations are avoided; the measurements to achieve this are reimplemented in the linear problem as constraints. Figure 1 shows this iterative process to obtain the solution of the optimization problem in the energy system. Even though this approach could be applied to any energy system, regardless of its dimension, this work is primarily intended for its implementation on mid and low voltage systems as described in Section 1.

#### *2.1. Overview: Iterative Process*

Since sector coupling and energy storage technologies will be present in the energy systems of the future, an iterative approach is needed when decoupling the unit dispatch optimization and the grid constraint generation. This is due to the fact that the constraint generation from the OPF may influence the dispatch of the sector coupling technologies and storage such as heat pumps, heat water tank, and batteries.

This section presents an overview of the main steps depicted in Figure 1.

1. Energy system structure: Here, the energy sectors to be considered are defined as well as the relevant technologies and their models. Natural renewable resources and demands time series must also be included. Time steps have to be big enough to make valid the steady-state assumption of the different energy sectors [18]. In this work, only the electric and heat sector are considered and a time step of an hour during a year is evaluated. Although several technologies, markets, and demands could be present in a distributed energy system [25], only the ones presented in Table 1 were considered in this study. Furthermore, the energy system is seen as a system aggregator from the grid perspective. This means that the system can buy and sell electricity in the market.

**Figure 1.** Steps for constraints generation in the optimization process.



2. Linear unit commit: Using a holistic approach, an abstraction of the energy system structure is created in *oemof-Solph*. This abstraction contains all possible energy flows between sources and sinks, and between energy sectors through the coupling technologies as depicted in Figure 2.

**Figure 2.** Simplified example of an energy system structure representation in *oemof-Solph* and the energy flow between sectors. The blue lines represent the electrical part of the system, whereas the red and orange represent the heat and gas sectors, respectively.

Costs and additional constraints can be associated with each of the energy flows [23]. Here, the system will only consider the economical constraints of the energy flows. Therefore, the objective function is the cost minimization. From Equation (1), the following objective equation is derived for the considered energy system:

$$\min \colon \sum\_{t \in T} \left( p\_{\text{elec\\_ext}}(t) \cdot c\_{\text{day\\_ahead}}(t) \right) \tag{5}$$

$$\left( +p\_{\rm in\\_clip}(t) \cdot c\_{\rm gas} + p\_{\rm in\\_boiler}(t) \cdot c\_{\rm gas} \right) \cdot \tau \tag{6}$$

where *pelec*\_*ext* represents the imported or exported electric power from the external grid, *cday*\_*ahead* the day ahead price for a given time, *pin*\_*chp* and *pin*\_*boiler* the gas import for the CHP and boiler, respectively, and *cgas* represents the cost of gas. The power given or provided to the external grid at a given time is given by *pext* = *pelec*\_*loads* − *pelec*\_*gen*. The electric generation *pelec*\_*gen* and electric loads *pelec*\_*loads* can be defined as follows:

$$p\_{\text{elec\\_gen}}(t) = p\_{\text{pv}}(t) + p\_{\text{elec\\_chp}}(t) \tag{7}$$

$$p\_{\text{elec\\_loads}}(t) = p\_{\text{elec\\_demand}}(t) + p\_{\text{battery}}(t) + p\_{\text{ev}}(t) + p\_{\text{elec\\_heat\\_pump}}(t) \tag{8}$$

In Equations (7) and (8), the parameters *ppv* and *pelec*\_*chp* refer to the PV generation and the CHP electric power generation within the system, whereas the parameters *pelec*\_*demand*, *pbattery*, *pev*, and *pelec*\_*heat*\_*pump* refer to the electric demand, the battery power consumption, electrical vehicle charging, and the heat pump demand, respectively. Every storage unit (including EVs) has been considered as a load. That means that a negative power represents a power injection to the system. The same convention has been used for the external grid power flow. As constraints for the optimization problem, the thermal and electric demands must be supplied at any time. Equations (9) and (10) depict such constraints where *pth*\_*demand*, *pelec*\_*demand*, and *ppv* are fixed time series.

$$p\_{\text{th\\_demand}}(t) = p\_{\text{th\\_heat\\_pump}}(t) + p\_{\text{th\\_buffer}}(t) + p\_{\text{th\\_chp}}(t) - p\_{\text{th\\_storage}}(t) \tag{9}$$

$$p\_{\text{elec\\_demand}}(t) = p\_{\text{elec\\_ext}} + p\_{\text{p\v}}(t) + p\_{\text{elec\\_chp}}(t) - p\_{\text{buttery}}(t) - p\_{\text{ev}}(t) - p\_{\text{elec\\_het\\_pump}}(t) \tag{10}$$

As a result, the optimized power dispatch for each non-fixed source to supply the demand at each time step is obtained. However, up to this point, just the total installed capacities and demands have been considered as per the inequality (4). The actual topology of the electrical and thermal grid has been also disregarded as a common practice in the optimization of energy systems [19].

3. Grid topology consideration: Before integrating these flows into the electrical grid in PowerFactory, the flow distribution must be considered. This applies only to the electric sources, sinks, and sector coupling technologies. The distribution of heat technologies is disregarded since only the electrical grid topology is being considered in this study.

To represent a typical low voltage grid, one of the so-called *"Merit Order Netz-Ausbau 2030"* (MONA) reference grids is used. The ONT\_8003 grid model is used as reference [26] and depicted in Figure 3. This topology represents a typical low voltage grid in Germany.

**Figure 3.** Representation of the ONT\_8003 MONA grid with the addition of the technologies mentioned in Table 1.

The energy flows for each node per technology are derived from the optimization results and the capacity installed per building.

4. Power flow calculation: Once the flow distributions at each node have been obtained, these energy flows must be added as time characteristics to the corresponding elements in PowerFactory.

To determine if the optimized dispatch complies with the grid standards, a quasidynamic power flow (QDPF) study is performed for a year with 1 h time steps. From the results of the QDPF, lines exceeding 100% loading and voltage variation outside of the range of ±10% of the nominal voltage are considered as grid violations [8].

The time steps containing such violations will be re-optimized by PowerFactory to avoid line congestion and bus voltage violations. For the system to converge, a dispatchable source is considered as slack, so it has enough power to cover the demand in case that the renewable sources are curtailed due to system violations.

5. Constraints generation: Similar to the constraint generation for the linear optimization [27], the *oemof-Solph* model will be limited by constraints generated from the OPF as denoted in Equation (4). Power time-series are the link between the two tools. Therefore, PowerFactory and *oemof-Solph* will exchange information about the active power flows, being the reactive power flow after the last OPF calculation considered as optimal.

To build the OPF problem, the following objective function and constraints are considered:

$$\begin{aligned} \min & \; \sum\_{t \in T\_{opf}} \left( p\_{elec\\_ext}^{opt}(t) \cdot c\_{day\\_ahead}(t) \\ & \; \left. + p\_{in\\_clap}^{opt}(t) \cdot c\_{\\$^{as}} \right) \cdot \tau \end{aligned} \tag{11}$$

$$\text{s.t.} \mathcal{S}\_{line}^{l}(t) \le \mathcal{S}\_{nom}^{l}(t)\_{\prime} \,\forall l \in L \tag{12}$$

$$\mathcal{S}\_{\text{line}} = \sqrt{\mathcal{P}\_{\text{line}}^2 + \mathcal{Q}\_{\text{line}}^2} \tag{13}$$

$$0.9 \cdot V\_{\text{nom}} \le |V\_{\text{bus}}^b(t)| \le 1.1 \cdot V\_{\text{nom}} \,\forall b \in B \tag{14}$$

$$\frac{-e\_{battery}(t-\tau)}{\tau} \le p\_{battery}^{opt}(t) \le \frac{k\_{battery} - e\_{battery}(t-\tau)}{\tau} \tag{15}$$

$$0 \le p\_{pv}^{opt}(t) \le p\_{pv}(t) \tag{16}$$

$$\mathcal{S}^{\mathfrak{n}} \le \mathcal{S}^{\mathfrak{n}}\_{\text{nom}\prime} \,\forall \mathfrak{n} \in \mathcal{N} \tag{17}$$

In Equation (11), the term for the thermal boiler is not considered. This is due to the fact that only electric units are taken into consideration in the OPF. The set *Top f* is a sub set of *T* comprising only the periods of time that led to grid violations from step 4. The variables *p op f elec*\_*ext* and *p op f elec*\_*chp* denote the optimized power flows to avoid such violations. The constraints (12) and (13) ensure that the nominal capacity *Snom* of any line of the system is not violated by the actual power flow *Sline*. The set of all lines in the energy system is denoted by *L*. The actual power flow in the line is given by the square root of the sum of squares of the real power *Pline* and the reactive power *Qline*. To ensure voltage compliance, the constraint (14) is included. This keeps the voltage magnitude |*Vbus*| of all buses in the system, denoted by the set *B*, within ±10% of the nominal voltage *Vnom*. The constraint (15) limits the power that can be drawn by the batteries. The difference between the battery system capacity *kbattery* and the battery system energy content *ebattery* at the previous time-step *t* − *τ* provides the available energy to be drawn for charging the batteries. The energy content is provided from the linear optimization performed in step 2. The difference is divided by *τ* in order to get the charging power limit. A lower energy bound is given to allow the discharge of the batteries. This lower bound is equal to the negative of the energy content. In the OPF additional flexibility is provided with the PV system. The optimizer can reduce the power output *p op f pv* as per constraint (16). Furthermore, to keep the thermal limits of all technologies within acceptable ranges, constraint (17) is included. Where the apparent power *S <sup>n</sup>* of each technology of the set *N* must not exceed the nominal apparent power *S n nom*. Similarly to the linear optimization, constraint (10) must be complied by the OPF, therefore:

$$\begin{aligned} p\_{\text{elec\\_demand}}(t) &= p\_{\text{elec\\_ext}}^{\text{opf}}(t) + p\_{pv}^{\text{opf}}(t) + \\ p\_{\text{elec\\_chup}}^{\text{opf}}(t) &- p\_{\text{battery}}^{\text{opf}}(t) - p\_{cv}^{\text{opf}}(t) - p\_{\text{elec\\_heat\\_pump}}^{\text{opf}}(t) \end{aligned} \tag{18}$$

In Equation (18), the superscript *op f* denotes the power flow obtained from the OPF from the respective source or sink. Considering that *COPheat*\_*pump* = *pth*\_*heat*\_*pump pelec*\_*heat*\_*pump* and *pth*\_*sources* = *pth*\_*heat*\_*pump* + *pth*\_*boiler* + *pth*\_*chp*, Equations (18) and (9) can be combined to yield:

$$\begin{aligned} \frac{p\_{\text{th\\_demand}} - p\_{\text{th\\_sources}} - p\_{\text{th\\_heat\\_pump}} - p\_{\text{th\\_storage}}}{\text{COP}\_{\text{heat\\_pump}}} &= p\_{\text{elec\\_ext}}^{\text{opf}} + p\_{\text{elec\\_gen}}^{\text{opf}} - p\_{\text{elec\\_hours}}^{\text{opf}}\\ &+ p\_{\text{elec\\_heat\\_pump}}^{\text{opf}} \end{aligned} \tag{19}$$

Equation (19) shows how the changes in the OPF have to be compensated by the heat elements in the energy system. However, as previously indicated, the OPF only takes into consideration the electric components and the sector coupling technologies. Therefore, the changes in the OPF have to be reflected in the linear optimization. Equation (20) is employed to determine the deviation between the OPF and the initial power flow.

$$
\Delta p\_n = |p\_n^{opt} - p\_n|\_\prime \forall n \in N \tag{20}
$$

The power flow per *nth* technology resulting from the OPF is represented by *p op f <sup>n</sup>* , whereas the initial power from the linear optimization is represented by *pn*. Whenever ∆*p<sup>n</sup>* 6= 0, the value of the OPF is passed as an upper bound in (4) for the linear optimization in the next iteration.

#### *2.2. Evaluation Scenarios*

In order to assess the method described in Section 2, three main scenarios are evaluated:


#### **3. Results**

In this section, the main results obtained from the implementation of the electric grid constraints into the optimal operation of an energy system are presented.

#### *3.1. High Generation Scenario*

#### 3.1.1. Line Loading Constraint

As described in Section 2, in this scenario, a significant capacity of PV is considered to be installed in the system. In Figure 4a, the loading on the main feeder during one year is shown. It is observed that the line can be overloaded up to 150% of its capacity. Figure 4b shows the new line loading after the OPF has generated the corresponding grid constraints at each time step. With the consideration of the violated hours shown in Figure 4a as constraints, the optimization reduces the load in the line at around 100% of its nominal capacity.

(**a**) Without grid constraints. (**b**) With grid constraints.

**Figure 4.** Main feeder loading with and without the consideration of the grid constraints into the optimization.

#### 3.1.2. Voltage Constraint

Similarly, Figure 5a,b depict the voltage behavior on the PV bus before and after the grid constraints generation.

Following the same pattern than the over-loading caused by the high in-feed depicted in Figure 4a, voltages above of the nominal voltage occur along with the main feeder over-loading.

**Figure 5.** PV bus voltage at node number 4 with and without the consideration of the grid constraints into the optimization.

Figure 6a,b show the power dispatched by the PV plant before and after the consideration of the voltage and loading constraints of the grid. A noticeable curtailment is needed to maintain the grid quality parameters of loading and voltage within the admissible ranges. The absence of means to store or shift loads during such midday peaks leads to the unavoidable PV curtailment.

#### *3.2. Heat Pump and Heat Storage*

In this section, the proposed methodology is tested by adding some flexibilization technologies such as heat storage and heat pumps.

Even though the installed PV capacity has been decreased to 700 kW<sup>p</sup> for this scenario, the mismatch between the electric demand and the generation will still create energy flows back to the grid. Such energy flows are big enough to overload the main feeder. As the heat pumps and heat storage are considered in the energy system, the optimizer has the possibility to transfer the energy surplus from the PV system into another energy carrier. Figure 7 shows how the line loading constraint due to the high PV feed-in affects the power dispatch of the heat pumps and PV. The flexibilization of the heat pump activation avoids that the locally generated energy leaves the system. This diminishes the energy injected into the grid, avoiding in this manner the line overloading and diminishing the PV curtailment.

**Figure 7.** Flexibilization provided by heat pumps and heat storage. The blue area represents the energy otherwise curtailed without the grid constraints implementation in the dispatch optimization.

Figure 8 shows the variation in energy content in the water storage tank. A noticeable difference is found especially in the months of summer. This is in accordance with the fact that higher values of sun irradiance and less thermal demand are present during such months. Therefore, the heat pumps are continuously being used to tackle PV generation surplus when grid constraint violations occur.

**Figure 8.** Change in energy content of the water heat storage.

#### *3.3. Electromobility Scenario*

As the fleet of electric cars is added to the energy system, the yearly demand is increased by around 300 MWh. This increment in demand causes line overloads throughout the network. In Figure 9, it can be seen how the main feeder is over-loaded while supplying energy to the EVs during the charging phase. It can be noted that the EV batteries support the grid after they have been connected at 6 pm, which is typically a time of the day with high demand. After the implementation of the constraints into the proposed method, the optimizer re-schedules the EV's batteries charging and grid feed-in to avoid the overload of the lines.

**Figure 9.** Cumulative EV charging profile comparison with and without grid constraints.

The proposed method successfully integrates the EV and the distance coverage constraint together with the grid constraints into the dispatch optimization problem of the energy system.

#### **4. Discussion**

In Section 3, it was found that the high share of PV at the low voltage level of the electrical system can lead to over-loads at the point of common coupling. This is caused by the residual energy fed into the grid. Neighborhoods with a PV capacity of 10 kW<sup>p</sup> per household are prompted to lead to grid violations [8]. From the results, it may be inferred that a PV installed capacity of 5 kW<sup>p</sup> per household can be led to feed-in levels exceeding the nominal voltage in a typical low voltage grid [26]. Nevertheless, overvoltages exceeding 10% of the nominal voltage are very unlikely to occur at this level of penetration. In the case of high share of PV without sector coupling or storage possibility, a curtailment up to ∼60 MWh per year is applied by the method to keep grid limits within acceptable ranges.

The integration of different sector coupling technologies may cause some grid events in the low voltage grids [28–30]. However, with a proper control strategy, such technologies can also provide some flexibility to the grid [31,32]. In the scenarios with flexibility options, the method re-schedules the power dispatch from these technologies. Table 2 provides a summary of the influence of the grid constraints in some indicators on each scenario. The summary demonstrates the relevance of the grid constraints in the optimization of an energy system. As it is shown in Table 2, this not only avoids grid limit violations, but it also saves energy from curtailment. From the simulation results, it was found that the methodology here proposed is capable of reducing the energy curtailment up to 45% by adapting to the economic dispatch, the technical grid constraints. Additionally, the self-consumption is increased up to ∼7% per year with the implementation of the method.


**Table 2.** Summary of the influence of grid constraints in the optimization parameters. Values are shown in MWh/year.

Therefore, the most economically and technically feasible dispatch of the energy system is achieved. The contribution of the *oemof-Solph*–PowerFactory toolchain results evident when flexible loads are present in the system. In this regard, the optimization decreases the overall energy that would be otherwise curtailed. Table 3 establishes a comparison between the proposed method and some of the most relevant researches found in the literature.

**Table 3.** Comparison with relevant works found in the literature.


Due to the flexibility of energy systems provided by *oemof-Solph*, the proposed method surpasses the approaches found in the literature in terms of the complexity of the energy system itself. Whereas Martinez et al. [33] consider gas, heat, and power, this study additionally considers, the electromobility sector and the influence on the system by batteries and heat storage. On the other hand, this research only considers the influence of the electric power grid. In some other studies, the grid constraints are considered, but neglecting either storage or other energy sectors [17,34,35].

#### **5. Summary and Outlook**

The proposed method combines a linear optimizer such as *oemof-Solph* and a power analysis tool such as PowerFactory. This combination allows the optimization of an energy system with multiple energy carriers while considering the actual grid limits for sector coupling technologies integration. Such grid limits have an impact on the performance of the energy system:

• For scenarios with PV capacity above 5 kW<sup>p</sup> per household, the voltage and line loading constraints are violated, therefore affecting the optimization results. In the

case study, curtailment of ∼60 MWh/year is needed to keep the grid within its limits. This due to the lack of storage or flexibilization options.


To adapt the method to large-scale grids, different open source tools might provide a faster interface and interaction with *oemof-Solph* compared to the PowerFactory API. Most of these tools are written in python or julia [36]. Another novel approach would be to consider a data-driven constraint generation. This might be especially relevant for online applications or analysis in the transmission system. Due to its high accuracy and run-time, this topic is yet to be exploited [37]. Furthermore, the influence of the heat network is a feature that could be added to this approach in future research.

**Author Contributions:** Conceptualization, A.R., P.K. and F.S.; investigation, A.R.; writing—original draft, A.R.; supervision, P.K.; writing—review and editing, A.R., P.K., F.S. and K.v.M. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** Not applicable.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


## *Article* **Design of Nonlinear Backstepping Double-Integral Sliding Mode Controllers to Stabilize the DC-Bus Voltage for DC–DC Converters Feeding CPLs**

**Subarto Kumar Ghosh <sup>1</sup> , Tushar Kanti Roy 2,\*, Md. Abu Hanif Pramanik <sup>3</sup> and Md. Apel Mahmud <sup>4</sup>**


**Abstract:** This paper proposes a composite nonlinear controller combining backstepping and doubleintegral sliding mode controllers for DC–DC boost converter (DDBC) feeding by constant power loads (CPLs) to improve the DC-bus voltage stability under large disturbances in DC distribution systems. In this regard, an exact feedback linearization approach is first used to transform the nonlinear dynamical model into a simplified linear system with canonical form so that it becomes suitable for designing the proposed controller. Another important feature of applying the exact feedback linearization approach in this work is to utilize its capability to cancel nonlinearities appearing due to the incremental negative-impedance of CPLs and the non-minimum phase problem related to the DDBC. Second, the proposed backstepping double integral-sliding mode controller (BDI-SMC) is employed on the feedback linearized system to determine the control law. Afterwards, the Lyapunov stability theory is used to analyze the closed-loop stability of the overall system. Finally, a simulation study is conducted under various operating conditions of the system to validate the theoretical analysis of the proposed controller. The simulation results are also compared with existing sliding mode controller (ESMC) and proportional-integral (PI) control schemes to demonstrate the superiority of the proposed BDI-SMC.

**Keywords:** backstepping double-integral sliding mode control scheme; constant power load; exact feedback linearization approach; Lyapunov stability theory; negative-resistance characteristics; nonminimum phase; nonlinear dynamical model

## **1. Introduction**

Over the past few decades, the power electronic converters (PECs) have been widely used in vehicular power systems (VPSs) (e.g., space vehicles, sea, land, etc.) and in renewable energy source (RES)-based systems (e.g., DC and AC microgrids) due to their voltage step-up, step-down, or conversion capabilities [1–5]. Recently, the power electronicbased DC distribution networks (DCDNs) are becoming more popular owing to their distinct advantages in terms of efficiency, controllability, flexibility, etc. [6,7]. However, two major challenges need to be addressed for a stable and reliable operation of power electronic-based DCDNs in conjunction of RESs, where the first challenge is the low terminal voltage of RESs while important to maintain a relatively high and stable terminal voltage of the DC-bus as all loads and RESs are either directly or indirectly connected to this bus [8–10]. As a results, DC–DC boost converters (DDBCs) are usually used in DCDNs as the interface between the DC-bus and RESs, which need to be appropriately controlled to achieve the desired DC-bus voltage [11,12]. On the other hand, the maintenance of

**Citation:** Ghosh, S.K.; Roy, T.K.; Pramanik, M.A.H.; Mahmud, M.A. Design of Nonlinear Backstepping Double-Integral Sliding Mode Controllers to Stabilize the DC-Bus Voltage for DC–DC Converters Feeding CPLs. *Energies* **2021**, *14*, 6753. https://doi.org/10.3390/en14206753

Academic Editors: Victor Becerra and Ahmed Rachid

Received: 24 September 2021 Accepted: 14 October 2021 Published: 17 October 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

stable and reliable operations becomes even more challenging when a large number of DC–DC converters are used to interface loads and RESs [13–15]. The second challenge is the instability issue caused by constant power loads (CPLs) in DCDNs with tightly coupled power electronic converters [16–18]. It is well-known that the constant power is drawn by CPLs, and consequently, it exhibits negative-impedance characteristics that destabilize as well as even cause blackouts [18,19]. Thus, it is essential to design an advance controller that not only deals with the destabilization but also provides quick dynamic response while guaranteeing the stability of the system.

Several control strategies have been proposed to resolve the instability issue of DDBCs caused by CPLs. Passive damping methods are proposed in [18,20] to neutralize the destabilization effect by increasing the damping into a system. However, these methods have limited applications due to their high costs and large sizes of passive elements such as capacitors and inductors. To alleviate these drawbacks, active damping methods such as virtual impedance [21], virtual resistance [22], and virtual capacitor [23] are proposed for stabilizing the system by adding extra control loops to reshape impedances. In these methods, the destructive effects of CPLs are mitigated by enforcing the unstable poles into the stable region through the modification in control loops. However, the satisfactory performance of these damping methods is highly dependent on the switching frequency. Thus, a voltage–current (V–I) droop-based dual-loop control scheme is proposed in [24] to stabilize the system by injecting sufficient damping torque into networks that overcomes the aforementioned drawbacks. However, a small variation in the droop gain leads to an inaccurate power sharing in networks [24]. To overcome this issue, an adaptive V–I droop control approach is proposed in [25] in which the virtual resistance is emulated at the output terminal of converters. However, these control approaches so far discussed here can only ensure the stability of the system near nominal operating points as these control schemes are developed by considering small disturbances around actual operating regions. Therefore, the system will be unstable when large disturbances due to variations in RESs and loads appear in the system.

A model predictive controller (MPC) is proposed in [26] to improve the stability of the system and extends the operating point. Though the MPC can handle the operating point issue, it cannot ensure better transient behaviors in the presence of large disturbances as it is designed without considering model uncertainties. This limitation is addressed in [27,28] by designing a hybrid MPC using the Takagi–Sugeno fuzzy-based scheme in conjunction with a traditional MPC. However, an accurate dynamical model is required to achieve the desired performance, which is always practically obstructed due to the presence of external disturbances and model uncertainties. Furthermore, the online computational complexity of MPCs limited their practical application in real-time platforms.

Nonlinear control schemes can be used to stabilize the system by considering the nonlinearity issue of CPLs and non-minimum phase problem of DDBCs. In [29], a passivitybased nonlinear controller is designed to resolve the instability issue due to CPLs while improving the transient and dynamic behaviors of the DC-bus voltage. However, the damping performance is severely affected with variations in the system parameters, which is overcome in [30] by proposing an adaptive passivity-based approach. However, the major drawback of these approaches is that the non-minimum phase problem of DDBCs cannot be avoided and further deteriorates the overall stability margin under large disturbances. Moreover, a time-scale model is required to design these controllers, which are quite hard to deal with in realistic applications.

Nonlinear feedback linearizing controllers (FBLCs) have an uncertainty/noise decoupling capability as discussed in [31–34], and these controllers are used in [35,36] for stabilizing the DC-bus voltage by overwhelming the non-minimum phase problem associated with DDBCs and the instability issue caused by CPLs. Though the approaches in [35,36] tackle the instability issue due to CPLs, the elimination of non-minimum phase problems is not completely achieved. Another FBLC is proposed in [37], where the nonminimum phase problem associated arising from DDBCs is avoided by considering the

inductor current as an output function. However, the zero-dynamic stability is not presented in [37] though it is important and the control law in [37] is derived only for resistive loads though nonlinear dynamics of CPLs lead to unstable operation. Moreover, the performance of FBLC schemes so far discussed here highly rely on the precise parametric information of the system, which is quite impossible as these parameters change due to changes in operating points. On the other hand, the nonlinear sliding mode controller (SMC) is an effective approach for dealing with robustness against parameter uncertainties and external disturbances in both nonlinear and linear systems [38,39]. Hence, the parameter sensitivity problem of a FBLC can be resolved using SMC schemes and a robust pulse width modulation (PWM)-based SMC is used in [40] to achieve the desired transient and steady-state performance of DC microgrids where DDBCs feed CPLs. Similar approaches are presented in [41,42] to achieve the same control objective as that of [40]. Though SMCs effectively handle the effects of parameter variations, the utilization of a discontinuous function and fixed gain for the sliding surface results in unwanted chattering in control laws, which can even damage power electronic converters in practical applications.

The shortcomings of SMCs are overcome in [43–46] by proposing a nonlinear backstepping controller (BSC), and these controllers are explored in [47] for stabilizing DC-bus voltage in the DDBC feeding CPLs. However, the implementation of the BSC still requires knowing the accurate parameters of the system as neither parametric uncertainties nor external disturbances are incorporated during the design. The issues related to parametric uncertainties and external disturbances are alleviated in [48] using an adaptive BSC for ensuring the large signal stability of the DCDN with CPLs. However, an observer is used to estimate external disturbances, which is quite an expensive approach. Moreover, the non-minimum phase problem associated with DDBCs is not dealt with in this work. To avoid this drawback, a composite controller that is a combination of FBLC and adaptive BSC schemes is proposed in [49,50], where the non-minimum phases problem and impacts of the negative-impedance due to CPLs are canceled by transforming the nonlinear model into a linear canonical form. However, these controllers do not ensure better transient response when the system parameters change due to wide variations of operating points. Apart from this, the zero dynamic stability for the remaining state variables that are not transformed through the feedback linearization process is not discussed in these papers. To address the aforementioned problems, an SMC and adaptive BSC are combined in [51]. However, a conventional sliding surface is used in this controller ,which leads to the chattering phenomena in the control effort and makes it impossible to ensure the desired control performance.

Motivated from the limitations of existing literature, a composite scheme based on the backstepping double integral-sliding mode controller (BDI-SMC) is proposed in this work to enhance the stability of DDBC feeding CPLs in DCDNs. In this work, the issues related to the non-minimum phase arising from DDBCs and negative-impedance behaviors of CPLs are tackled by canceling nonlinearities and transforming the nonlinear dynamical model into a feedback linearized model using the exact feedback linearization approach. A double integral-sliding surface is then considered to derive the control law that can enhance the steady-state tracking performance while eliminating unwanted chattering effects. Afterwards, the actual control law is determined following design steps in both BSC and SMC. Finally, the feasibility of the actual control input in terms of maintaining the stability is analyzed using the Lyapunov function. A simulation study is conducted considering different operating scenarios of the system to verify its effectiveness. Additionally, its superiority is demonstrated by comparing the results with those of traditional nonlinear SMC and PI controllers.

#### **2. System Description and Its Dynamical Model**

The overall structure of a DCDN is depicted in Figure 1. It is well-known that the DC source generally consists of renewable energy-based distributed generation systems (DGS) such as solar photovoltaic (PV) system, rectifier in conjunction wind power generation

systems, etc., which works as the main power supply. The DDBCs are commonly used for interfacing the DC source with the main DC-bus so that the voltage requirement of various loads can easily be met. Among these loads, some tightly regulated loads draw constant power from the DC-bus, which are also known as CPLs and exhibit negative-impedance characteristics. The electrical characteristic of CPLs can be described using the following voltage–current relation:

$$\dot{a}\_{\rm CPL} = \frac{P\_{\rm CPL}}{v\_{\rm bus}} \tag{1}$$

where *iCPL* denotes the current of a CPL, *PCPL* represents the power of a CPL, and *vbus* is the DC-bus voltage. The concept of the small disturbance analysis can be applied around an equilibrium point of Equation (1). Based on this analysis, the equivalent impedance of a CPL can be expressed as follows:

$$R\_{\rm CPL} = \frac{\partial v\_{\rm bus}}{\partial i\_{\rm CPL}} = -\frac{P\_{\rm CPL}}{I\_{\rm CPL}^2} \tag{2}$$

where *ICPL* is the CPL steady-state value of the current. From Equation (2), it is obvious that the negative-impedance behavior of a CPL creates a negative impact on the performance, i.e., it decreases the damping of the system, which further destabilizes the system, especially when it is connected with a converter. Since the nonlinearity introduced into system is due to CPLs, traditional linear controllers cannot guarantee the stability against large disturbances. Hence, the employment of a nonlinear controller is essential to neutralize the nonlinear effect of CPLs. However, to achieve such an objective from a controller, it is necessary to have an appropriate dynamical model. Hence, a dynamical model for DDBCs feeding a CPL is developed in the following to meet the controller design requirement.

The simplified structure of a DDBC feeding a CPL in DCDNs is shown in Figure 2. In this configuration, a current controlled source is used to represent a lumped CPL whereas

the lumped resistive load is represented by *RLoad*. It is worth mentioning that the input DC voltage source of the DDBC will be obtained from RESs or another DC–DC converter. Now, according to the simplified structure as shown in Figure 2, the expression of its model while on (*Ton*) and off (*To f f*) can be obtained as follows:

$$\dot{\mathbf{x}} = (A\_{on}\mathbf{x} + b\_{on})\mathbf{u} + (A\_{off}\mathbf{x} + b\_{off})(\mathbf{1} - \mathbf{u})\tag{3}$$

with

$$A\_{0\text{tt}} = \begin{bmatrix} -\frac{r\_b}{L\_b} & 0\\ 0 & -\frac{P\_{\text{CP}\text{t}}}{\upsilon\_{\text{bus}}} \end{bmatrix}, b\_{on} = \begin{bmatrix} \frac{V\_{\text{in}}}{L\_b} \\ 0 \end{bmatrix}, A\_{off} = \begin{bmatrix} -\frac{r\_b}{L\_b} & -\frac{1}{L\_b} \\ \frac{1}{C\_{\text{bus}}} & -\frac{P\_{\text{CP}\text{t}}}{\upsilon\_{\text{bus}}} \end{bmatrix}, b\_{off} = \begin{bmatrix} \frac{V\_{\text{in}}}{L\_b} \\ 0 \end{bmatrix}, \text{and } \mathbf{x} = \begin{bmatrix} i\_b \\ \upsilon\_{\text{bus}} \end{bmatrix}$$

where *i<sup>b</sup>* represents the inductor current, *Vin* is the input voltage of the converter, *vbus* is the main DC-bus voltage, *L* is the inductor, *r<sup>b</sup>* is the parasitic resistance of an inductor, *u* is the duty cycle, and *Cbus* is the DC-bus capacitance. It worth noting that the inductor current, *ib* , and the main DC-bus voltage, *vbus*, are considered state variables. It is well-known that a discontinuous converter model can be approximated by a continuous model by selecting the converter switching frequency as being higher than the natural frequency. Hence, for continuous conduction mode (CCM) operations, the discontinuous control input *u* can be replaced by the duty ratio *µ*, which is a continuous function in the subinterval [0, 1]. Therefore, by considering the CCM operation, the model as represented by Equation (4) can be expressed as follows:

$$\begin{aligned} \frac{di\_b}{dt} &= -\frac{r\_b}{L\_b}i\_b + \frac{V\_{in} - v\_{bus}}{L\_b} + \frac{v\_{bus}}{L\_b}\mu\\ \frac{dv\_{bus}}{dt} &= \frac{1}{\mathcal{C}\_{bus}}(i\_b - \frac{v\_{bus}}{R\_{Load}} - \frac{P\_{CPL}}{v\_{bus}}) - \frac{i\_b}{\mathcal{C}\_{bus}}\mu \end{aligned} \tag{4}$$

The proposed BDI-SMC is designed based on the model as described by Equation (4) which is discussed in the next section.

**Figure 2.** The simplified structure of a DDBC feeding a CPL.

#### **3. Proposed Controller Design Approach**

In this section, the proposed BDI-SMC is designed for DDBCs feeding CPLs. The key control objective is to achieve fast transient and desired steady-state tracking errors of the DC-bus voltage. However, the dynamical model represented by Equation (4) is not suitable for designing the BDI-SMC in its current form. Hence, the following steps need to be followed to make the model suitable for the controller design:


All these steps are discussed in detail through the following subsections.

### *3.1. Transformation of the Model into an Exactly Linearized System Using the Exact Feedback Linearization*

The transformation of the nonlinear model of the DDBC with a CPL in Equation (4) can be represented as the following generalized form:

$$\begin{aligned} \dot{x} &= f(\mathbf{x}) + g(\mathbf{x})u \\ y &= h(\mathbf{x}) \end{aligned} \tag{5}$$

where *x* = [*x*1, *x*2, . . . , *xn*] *T* is the state of the system, *u* is the input, *y* is the output of the system, and *f*(*x*) and *g*(*x*) are nonlinear functions of states and parameters with *g*(*x*) 6= 0. Fitting Equations (4) into (5), it can be written as follows:

$$f(\mathbf{x}) = \begin{bmatrix} -\frac{r\_b}{L\_b}\mathbf{x}\_1 + \left(\frac{V\_{\text{ii}} - \mathbf{x}\_2}{L\_b}\right) \\ \frac{1}{\mathbf{C}\_{\text{bus}}}(\mathbf{x}\_1 - \frac{\mathbf{x}\_2}{R\_{Load}} - \frac{P\_{\text{CPL}}}{\mathbf{x}\_2}) \end{bmatrix}, \text{ and } \mathbf{g}(\mathbf{x}) = \begin{bmatrix} \frac{\mathbf{x}\_2}{L\_{\text{bus}}} \\ -\frac{\mathbf{x}\_1}{\mathbf{C}\_{\text{bus}}} \end{bmatrix}^T$$

where *x* = [*i<sup>b</sup> vbus*] *<sup>T</sup>* and *u* = *µ*. The following steps are necessary to transform the nonlinear system into an exactly linearized system using the exact feedback linearization scheme.

**Step 1:** Selection of the output function

The feedback linearizability depends on the output of the system, which can be selected in a different way. For the DDBC with a CPL, the output function can be chosen as any of the states (i.e., the capacitor voltage or inductor current) and the combination of these states in the form of the total energy of the system. The system becomes partially linearized when the capacitor voltage is selected as the output. Hence, the system will be a non-minimum phase one as the internal dynamic stability cannot satisfy the stability criterion. This issue can be tackled by selecting the inductor current as the output, which also makes the system partially linearized but with the stable internal dynamic. However, the regulation of the output voltage by indirectly controlling the inductor current results in excessive overshoot and slower response. Under this situation, the output needs to be chosen in such a way that satisfies the requirement of the exact linearization approach, i.e., the system is exactly linearized for which the relative degree of the exactly linearized system is equal to the order of the system. Based on this discussion, the total stored energy is considered an output function that can be expressed as follows:

$$y = h(\mathbf{x}) = \frac{1}{2}L\_b i\_b^2 + \frac{1}{2}\mathcal{C}\_{bus}v\_{bus}^2\tag{6}$$

**Step 2:** Relative degree calculation

In this step, the relative degree for the nonlinear system as presented by Equation (5) is calculated using the Lie derivative while considering the output function in Equation (6). To calculate the relative degree, the following condition should be satisfied:

$$L\_{\mathcal{S}}L\_f^{r-1}h(\mathfrak{x}) \neq 0 \tag{7}$$

In Equation (7), *r* is the relative degree while *L* represents the Lie derivative along the vector field denoted by the subscript. To satisfy the condition for the exact linearization approach, the relative degree of the *n*th order system will be *r* = *n*, which satisfies the following expressions:

$$\begin{aligned} L\_{\mathcal{S}}L\_f^{1-1}h(\mathbf{x}) = L\_{\mathcal{S}}L\_f^{2-1}h(\mathbf{x}) = \dots = L\_{\mathcal{S}}L\_f^{n-2}h(\mathbf{x}) = 0\\ L\_{\mathcal{S}}L\_f^{n-1}h(\mathbf{x}) \neq 0 \end{aligned} \tag{8}$$

Based on the model and output function, the following expressions are obtained:

$$r = \mathfrak{n} = \mathfrak{2}$$

$$L\_{\mathfrak{E}}L\_f^{1-1}h(\mathfrak{x}) = \mathfrak{0}$$

and

$$L\_{\mathcal{S}}L\_f^{n-1}h(\mathfrak{x}) \neq 0$$

Hence, the system is exactly linearized.

**Step 3:** Nonlinear coordinate transformation and exact linearization

In this step, the coordinate transformation technique is adopted to convert the original *x* state variables into new *z* state variables, which can be discussed as follows:

$$z\_1 = h(\mathbf{x}) = L\_f^{1-1} h(\mathbf{x}) = \frac{1}{2} L\_b i\_b^2 + \frac{1}{2} \mathbb{C}\_{bus} v\_{bus}^2 \tag{9}$$

in which the dynamic can be written as

$$
\dot{z\_1} = \frac{\partial h(\mathbf{x})}{\partial \mathbf{x}} \dot{\mathbf{x}} \tag{10}
$$

Using Equation (5), Equation (10) can be written as follows:

$$z\_1 = \frac{\partial h(\mathbf{x})}{\partial \mathbf{x}} f(\mathbf{x}) + \frac{\partial h(\mathbf{x})}{\partial \mathbf{x}} \mathbf{g}(\mathbf{x}) \boldsymbol{u} = \mathcal{L}\_f h(\mathbf{x}) + \mathcal{L}\_\mathcal{S} L\_f^{1-1} h(\mathbf{x}) \boldsymbol{u} \tag{11}$$

Since *LgL* 1−1 *f h*(*x*) = 0, Equation (11) can be written as follows:

$$z\_1 = z\_2 = L\_f h(\mathbf{x}) = V\_{in} \mathbf{x}\_1 - \frac{\mathbf{x}\_2^2}{R\_{Load}} - P\_{\rm CPL} - r\_b \mathbf{x}\_1^2 \tag{12}$$

Equation (12) can be written as follows:

$$z\_2^\circ = L\_f^2 h(\mathbf{x}) + L\_\mathcal{g} L\_f^{2-1} h(\mathbf{x}) \mu = \nu \tag{13}$$

Since *LgL* 2−1 *f h*(*x*) 6= 0, the original nonlinear system can be represented as the following exactly linearized form:

$$\begin{aligned} \dot{z}\_1 &= z\_2\\ \dot{z}\_2 &= \nu \end{aligned} \tag{14}$$

where *z*<sup>1</sup> and *z*<sup>2</sup> are the state variables of the exactly linearized system and *ν* is the control variable of the transformed linear systems. The model in Equation (14) can be rewritten as follows:

$$\begin{aligned} \dot{z}\_1 &= z\_2\\ z\_2 &= \nu = a(\mathbf{x}) + b(\mathbf{x})\mu \end{aligned} \tag{15}$$

where

$$\begin{split} a(\mathbf{x}) &= L\_f^2 h(\mathbf{x}) = \frac{(V\_{\text{in}} - 2\mathbf{x}\_1 r\_b)(V\_{\text{in}} - \mathbf{x}\_2 - r\_b \mathbf{x}\_1)}{L\_b} - \frac{2\mathbf{x}\_2}{R\_{Load}\mathbf{C}\_{\text{bus}}} (\mathbf{x}\_1 - \frac{\mathbf{x}\_2}{R\_{Load}} - \frac{P\_{\text{CPL}}}{\mathbf{x}\_2}) \\ b(\mathbf{x}) &= L\_g L\_f h(\mathbf{x}) = \frac{(V\_{\text{in}} - 2\mathbf{x}\_1 r\_b)\mathbf{x}\_2}{L\_b} + \frac{2\mathbf{x}\_1 \mathbf{x}\_2}{R\_{Load}\mathbf{C}\_{\text{bus}}} \end{split}$$

The proposed controller is designed based on this model, which is discussed in the following subsection.

#### *3.2. Controller Design*

In this subsection, the control law is determined using the combination of backstepping and double integral-sliding mode control schemes. Here, the main objectives are to track *z*<sup>1</sup> and *z*2, which ultimately ensures the desired tracking of the DC-bus voltage under large disturbances. The following steps describe the design process for the proposed BDI-SMC.

**Step 1:** For fulfilling the design objective, the first tracking error (*e*1) can be defined as follows:

$$e\_1 = z\_1 - z\_{1(ref)}\tag{16}$$

where *z*1(*re f*) is the reference value of the state *z*1, which can be calculated as follows:

$$z\_{1(ref)} = \frac{1}{2} L\_b r\_{b(ref)}^2 + \frac{1}{2} C\_{bus} v\_{bus(ref)}^2 \tag{17}$$

where *ib*(*re f*)= *PCPL Vin* is the reference value of the inductor current and *vdc*(*re f*) is the reference value of the DC-bus voltage. Now, the dynamic of *e*<sup>1</sup> using Equation (16) can be obtained as follows:

$$
\dot{e}\_1 = z\_2 - \dot{z}\_{1(ref)} \tag{18}
$$

As the actual control input does not appear in Equation (18), *z*<sup>2</sup> is assumed to be the virtual control input and *γ* is assumed to be the corresponding stabilizing function or virtual control law. Hence, the final error (*e*2) can be defined as follows:

$$\mathbf{z}\_2 = \mathbf{z}\_2 - \gamma \tag{19}$$

Using Equation (19), Equation (18) can be rewritten as follows:

$$\dot{e}\_1 = e\_2 + \gamma - \dot{z}\_{1(ref)}\tag{20}$$

At this stage, it is required to check the stability of the tracking error dynamic *e*˙1, and to do this, the following Lyapunov function (LF) is considered:

$$\mathcal{W}\_1 = \frac{1}{2}e\_1^2 \tag{21}$$

in which the time derivative using Equation (20) can be written as follows:

$$
\dot{W}\_1 = e\_1[e\_2 + \gamma - z\_{1(ref)}] \tag{22}
$$

From Equation (22), *γ* can be selected as follows:

$$
\gamma = -k\_1 e\_1 + \dot{z\_1}\_{(ref)} \tag{23}
$$

where *k*<sup>1</sup> is a positive constant. Substituting Equations (23) into (22) yields the following:

$$
\dot{W}\_1 = -k\_1 e\_1^2 + e\_1 e\_2 \tag{24}
$$

From Equation (24), it can be seen that, if *e*<sup>2</sup> = 0, then *W*˙ <sup>1</sup> ≤ 0. To achieve this, the next step is essential.

**Step 2:** The dynamic of *e*<sup>2</sup> can be written as follows:

$$
\dot{\varepsilon}\_2 = \dot{z}\_2 - \dot{\gamma} \tag{25}
$$

Using Equation (15), the dynamic of *e*<sup>2</sup> can be expressed as follows:

$$\epsilon\_2 = a(\mathbf{x}) + b(\mathbf{x})\mu - \dot{\gamma} \tag{26}$$

At this instant, a double-integral sliding surface in term of *e*<sup>2</sup> can be selected as follows:

$$S = e\_2 + a\_1 \int e\_2 dt + a\_2 \int \int e\_2 dt dt\tag{27}$$

where *α*<sup>1</sup> and *α*<sup>2</sup> are constant parameters of the sliding surface. The dynamic of Equation (27), using the value of *e*˙2, can be expressed as follows:

$$\dot{S} = a(\mathbf{x}) + b(\mathbf{x})\mu - \dot{\gamma} + a\_1 e\_2 + a\_2 \int e\_2 dt \tag{28}$$

It is well-known that appropriately selecting a reaching law is very important to mitigate the chattering while improving the convergence time. Hence, a reaching law is selected as follows to meet the above objective:

$$\dot{S} = -\beta\_1 \text{sgn}(S) - \beta\_2 S \tag{29}$$

where *β*<sup>1</sup> and *β*<sup>2</sup> are positive constants and the chattering effect depends on these values. Combining Equations (28) and (29), it can be written as follows:

$$a(\mathbf{x}) + b(\mathbf{x})\mu - \gamma + a\_1\mathbf{e}\_2 + a\_2 \int \mathbf{e}\_2 dt = -\beta\_1 \text{sgn}(\mathbf{S}) - \beta\_2 \mathbf{S} \tag{30}$$

From Equation (30), the actual control law can be determined as follows:

$$\mu = -\frac{1}{b(\mathbf{x})} [a(\mathbf{x}) - \gamma + a\_1 e\_2 + a\_2 \int e\_2 dt + \frac{1}{S} e\_1 e\_2 + \beta\_1 \text{sgn}(S) + \beta\_2 S] \tag{31}$$

Substituting Equation (31) into Equation (28) yields

$$\mathcal{S} = -\beta\_1 \text{sgn}(\mathcal{S}) - \beta\_2 \mathcal{S} - \frac{1}{\mathcal{S}} e\_1 e\_2 \tag{32}$$

The following Lyapunov function is selected to ensure overall stability with the derived control law:

$$
\dot{W}\_2 = \mathcal{W}\_1 + \frac{1}{2}\mathcal{S}^2 \tag{33}
$$

Using Equations (24) and (32), *W*˙ <sup>2</sup> can be written as follows:

$$
\dot{W}\_2 = -k\_1 e\_1^2 - \beta\_1 |\mathbb{S}| - \beta\_2 \mathbb{S}^2 \tag{34}
$$

Since *k*<sup>1</sup> > 0, *β*<sup>1</sup> > 0, and *β*<sup>2</sup> > 0, *W*˙ <sup>2</sup> ≤ 0. Thus, the overall stability of the system is guaranteed with the designed control law as represented by Equation (31). The effectiveness of the designed BDI-SMC is demonstrated in the next section.

#### **4. Simulation Results**

To verify the effectiveness of the designed composite BDI-SMC strategy, a similar simulation model as that shown in Figure 2 is built on the MATLAB/Simulink platform. The nominal voltage of the main-DC bus of the distribution network is considered as 110 V while the rated power of DC loads is considered as 2 kW. In order to capture the highest destabilization effect of the DC load on the DC distribution network, a pure CPL is considered the DC power demand rather than taking into account the composite load demand. The frequency of the power electronic interface boost converter is set as 5 kHz with a sampling frequency of 100 kHz to evaluate the performance of the designed BDI-SMC. It is well-known that, during the implementation of controllers, it is essential to know the system parameters. Hence, the system parameters used for the simulation study are listed in Table 1. The gain parameters of the designed nonlinear controller are listed as follows: *k*<sup>1</sup> = 1000, *α*<sup>1</sup> = 70, *α*<sup>2</sup> = 0.45, *β*<sup>1</sup> = 100, and *β*<sup>2</sup> = 0.01. It is worth mentioning that the gain parameters are selected based on the trial and error method to meet the desired control objective.

**Parameters Description Value** *Vin* Supply voltage 55 V *Vbus* Main DC-bus voltage 110 V *PCPL* Nominal power in constant power load 2 kW *r<sup>b</sup>* Parasitic resistance of an inductor 2 mΩ *Lb* Inductance of the converter 5 mH *Cbus* Capacitance of the main DC-bus 6 mF *RLoad* Resistive Load ∞

**Table 1.** Nominal system parameters of the system.

The performance of the BDI-SMC is verified by widely varying the operating region by considering the variations in the reference power of CPLs, main DC-bus reference voltage, and input power in terms of the input source voltage. Therefore, the following three case studies are considered to demonstrate the performance of the designed controller:


To show the merits of the designed BDI-SMC, the performance is also compared with the existing SMC (ESMC), as proposed in [40], and with a conventional proportionalintegral (PI) controller.

**Case I:** Controller performance investigation with variations in the reference power of the CPL

In this scenario, the variation in the reference power of CPLs is taken into consideration to evaluate the performance of the designed controller and existing controllers. At the beginning of the simulation, the rated power of the CPL is considered as 2 kW whereas it is increased to 4 kW at t = 1 s. On the other hand, other parameters are kept unchanged. Due to the transient power of CPLs, as shown in Figure 3a, the post-disturbance dynamic response of the main DC-bus and inductor current are affected, which can be seen in Figure 3b,c, respectively. From these responses, it can be observed that all three controllers can ensure stability. However, the designed BDI-SMC can provide faster dynamic responses along with less undershoot compared with the ESMC and PI. Again at t = 2 s, the CPL power is decreased from 4 kW to 500 W, which is considered a large disturbance in the system. Due to this large disturbance, the conventional PI controller is unable to provide sufficient damping torque into the networks, consequently leading to instability in both the main DC-bus voltage and inductor current, which can be clearly seen from Figure 3b,c. Meanwhile, both the designed BDI-SMC and the ESMC can ensure stability, which can be clearly seen from Figure 3b,c, respectively. However, though both controllers can maintain zero output voltage tracking errors, the designed BDI-SMC ensures a faster settling time and less overshoot. The corresponding control signal for all three controllers is depicted in

Figure 3d,e, where it is obvious that the designed control signal is more stable compared with the PI and ESMC controllers. Furthermore, it can be observed that the designed controller can effectively eliminate the chattering effect on the PWM whereas the ESMC cannot attenuate the chattering effect.

**Figure 3.** Dynamic response of the system with the variations in the CPL reference power.

**Case II:** Controller performance investigation with the variations in the reference voltage of the DC-bus

In this case study, the main DC-bus voltage increases from its equilibrium state 110 V to 160 V at t = 1.2 s and again increases from 160 V to 220 V at t = 2.2 s, whereas the CPL is set as 2 kW. The corresponding system responses are illustrated in Figure 4. From Figure 4a, it can be observed that the PI and ESMC cannot ensure a main DC-bus voltage tracking performance with a faster settling time and less overshoot/undershoot whilst the BDI-SMC can quickly track the new reference main DC-bus voltage as the settling time overshoot/undershoot are close to zero. Similarly, from Figure 4b, it can be observed that the designed BDI-SMC can provide a faster dynamic performance compared with the ESMC and PI. However, although the overshoot/undershoot in the inductor current is less with the ESMC, the control signal contains higher chattering, as illustrated in Figure 4d. As a result, the switching losses are higher, which reduces the overall system performance. On the other hand, the designed BDI-SMC is able to provide an oscillationfree control signal to the PWM compared with the ESMC, which can be clearly seen from

Figure 4d,e. Throughout this simulation study, the load power is constant due to the electrical characteristics of the CPL, which is shown in Figure 4c.

**Figure 4.** Dynamic response of the system with the variations of the main DC-bus reference voltage.

**Case III:** Controller performance investigation with the variations in the input voltage Practically, the output power of RESs is continuously changeable due to their intermittent characteristics. To show that impact on the distribution network, in this case study, the variation in the input power is considered. For this purpose, the input voltage of the DC voltage increases from 55 V to 70 V and decreases from 70 V to 40 V at t = 1.4 s and t = 2.4 s, respectively. Figure 5 shows the corresponding dynamic responses of the system. From Figure 5b,c, it can be observed that all three controllers can accurately track the main DC-bus voltage and inductor current when input voltage variation occurs. However, the designed BDI-SMC has a faster dynamic response and a smaller overshoot compared with the ESMC and conventional PI control methods. Moreover, it can be seen that the CPL remained unchanged at the equilibrium state 2 kW, which is illustrated in Figure 5d.

From the above simulation results, it can be concluded that the designed BDI-SMC can provide fast dynamic convergence speed, accurate tracking of the DC-bus voltage, and good large disturbances stability for a DC–DC boost converter feeding a CPL. Moreover, it is obvious that the stability of the DC-bus voltage is not destroyed even after the large variations in the CPL power. Hence, the simulation result is consistent with the theoretical analysis.

**Figure 5.** Dynamic response of the system with the variations in the input voltage.

#### **5. Conclusions**

A composite nonlinear controller combining the backstepping control theory and double integral-sliding mode control theory is proposed for a DC–DC boost converter feeding a CPL with distribution networks. In order to design the proposed controller, a canonical form is developed using the exact feedback linearization approach by considering the total stored energy as the output function. Based on the canonical form of the model, a composite nonlinear controller is designed to guarantee DC-bus voltage stability under large disturbances of a DC–DC boost converter feeding CPLs with negligible steady-state tracking error and fast transient responses. The effectiveness and theoretical analysis of the designed controller are verified through simulation results. In the future, a robust adaptive nonlinear composite controller will be designed by considering both parametric uncertainty and external disturbances.

**Author Contributions:** Conceptualization, S.K.G., T.K.R. and M.A.M.; methodology, S.K.G. and T.K.R.; software, S.K.G. and M.A.H.P.; validation, S.K.G. and T.K.R.; formal analysis, S.K.G., T.K.R. and M.A.H.P.; investigation, S.K.G. and T.K.R.; writing—original draft preparation, S.K.G. and T.K.R.; writing—review and editing, T.K.R. and M.A.M.; supervision, T.K.R. and M.A.M.; project administration, T.K.R. and M.A.M. All authors have read and agreed to the current version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** Not applicable.

**Conflicts of Interest:** The authors declare no conflicts of interest.

### **Abbreviations**

The following abbreviations are used in this manuscript:


#### **References**


## *Article* **Reinforcement Learning for Energy-Storage Systems in Grid-Connected Microgrids: An Investigation of Online vs. Offline Implementation**

**Khawaja Haider Ali 1,2,\*, Marvin Sigalo <sup>1</sup> , Saptarshi Das <sup>1</sup> , Enrico Anderlini <sup>3</sup> , Asif Ali Tahir <sup>1</sup> and Mohammad Abusara 1,\***


**Abstract:** Grid-connected microgrids consisting of renewable energy sources, battery storage, and load require an appropriate energy management system that controls the battery operation. Traditionally, the operation of the battery is optimised using 24 h of forecasted data of load demand and renewable energy sources (RES) generation using offline optimisation techniques, where the battery actions (charge/discharge/idle) are determined before the start of the day. Reinforcement Learning (RL) has recently been suggested as an alternative to these traditional techniques due to its ability to learn optimal policy online using real data. Two approaches of RL have been suggested in the literature viz. offline and online. In offline RL, the agent learns the optimum policy using predicted generation and load data. Once convergence is achieved, battery commands are dispatched in real time. This method is similar to traditional methods because it relies on forecasted data. In online RL, on the other hand, the agent learns the optimum policy by interacting with the system in real time using real data. This paper investigates the effectiveness of both the approaches. White Gaussian noise with different standard deviations was added to real data to create synthetic predicted data to validate the method. In the first approach, the predicted data were used by an offline RL algorithm. In the second approach, the online RL algorithm interacted with real streaming data in real time, and the agent was trained using real data. When the energy costs of the two approaches were compared, it was found that the online RL provides better results than the offline approach if the difference between real and predicted data is greater than 1.6%.

**Keywords:** reinforcement learning (RL); microgrid; battery management; offline and online RL; optimisation

## **1. Introduction**

Grid-connected microgrids are becoming the main building blocks of smart grids. They facilitate the vast deployment and better utilisation of RES, reduce stress on the existing power grid, and provide consumers with uninterrupted power supply. The main aim for any Energy Management System (EMS) for grid-connected microgrids is to reduce operational costs by reducing the cost of power imported from the grid. This is achieved by controlling the Battery Energy Storage System (BESS) to store power when RES generation is higher than load demand and release power when it is less than the load demand. However, BESS capacity is finite and hence, depending on the battery size, imported power from the grid is likely to be used. Therefore, grid tariffs, which can vary during the day, must be taken into account when deciding charging and discharging commands. In addition, the feed-in tariff can also be available to enable consumers or prosumers to sell

**Citation:** Ali, K.H.; Sigalo, M.; Das, S.; Anderlini, E.; Tahir, A.A.; Abusara, M. Reinforcement Learning for Energy-Storage Systems in Grid-Connected Microgrids: An Investigation of Online vs. Offline Implementation. *Energies* **2021**, *14*, 5688. https://doi.org/10.3390/ en14185688

Academic Editor: Seon-Ju Ahn

Received: 7 August 2021 Accepted: 7 September 2021 Published: 9 September 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

their excess power to the grid [1]. These factors demonstrate the need for an intelligent optimisation method. However, due to the lack of information regarding future generation and load profiles, this presents a challenge for EMS.

Inspired by the classical economic dispatch of power systems, various studies have suggested different optimisation techniques to plan, schedule, and control the BESS in grid-connected microgrids. These studies have formulated the operation of the BESS as a dispatch optimisation problem and solved it using Linear Programming (LP) [2,3], Mixed Integer Programming (MIP) [4], Mixed-Integer Linear Programming (MILP) [5–7], and Mixed-Integer Nonlinear Programming (MINLP) [8,9]. In Chen et al. [2], a general algebraic modelling system (GAMS) was developed and solved by CPLEX solver and then tested in a physical system based in Taiwan. The study compared two models to determine the impact of energy storage on optimal scheduling. The first model consisted of thermal and electrical loads and a CHP unit. The second model used additional thermal and electrical storage. Both models used in this work relied on the prediction of load profiles. In Luna et al. [5], the model reflects a deterministic problem that promotes self-consumption based on 24 h look-ahead forecast data. The microgrid consists of a supervisory control stage that compensates for any mismatch between the offline scheduling process and the real-time microgrid operation. In Li et al. [7], the microgrid optimisation problem is formalised using a general algebraic modelling system (GAMS) via a discretised step transformation (DST) approach and finally solved using the CPLEX solver. This paper proposes a new optimal scheduling mode by modelling the uncertainty of spinning reserves provided by energy storage with probabilistic constraints. These achievements are highly dependent on the proper estimation of spinning reserves, which is a big challenge while working on a real system. In Mosa and Ali [9], the MINLP algorithm was used to reduce the operational cost of a DC microgrid consisting of a photovoltaic (PV), fuel cell (FC), micro turbine (MT), diesel generator (DE), and battery/BESS. This study uses Egyptian grid load profiles over four seasons of the year based on the prediction.

The above traditional approaches require a detailed and accurate mathematical model of the system, while some of them require the linearisation of the system. In addition, previous knowledge of future RES generation and load demand over a period is required as an input to the optimisation problem. The accuracy of the prediction may affect the accuracy of the BESS operation. Therefore, different forecasting algorithms that can handle the stochastic nature of load demand and RES have been suggested in the literature. These algorithms are designed to forecast short (daily), medium (seasonal), and long (yearly) load demand and availability of RES. Most advanced forecasting algorithms include Artificial Neural Networks (ANN), dynamic programming (DP)-based optimisation, and fuzzy logic by considering the weather conditions. Although forecasting techniques vary within the vast amount of existing literature [10–18], the most common objective of these techniques is to decrease the forecasting error by better modelling the uncertainties in real time. Although forecasting algorithms have improved in recent years, it is still challenging to predict the future load demand and availability of RES with minimum error, especially if the decision making is implemented in real time.

Recent studies [19–26] have introduced reinforcement learning as a potential solution for the optimal operation of BESS due to its ability to develop an optimal policy online. In RL, an agent interacts with the surrounding environment and develops an optimal policy for taking the right action after exploring its state. The agent takes the action to maximise a future accumulative reward. The main advantage for RL over traditional methods is that it does not need any model of the environment and it can learn the optimum policy in real time. Yoldas et al. [27] used the MINLP technique guided by a Q-learning algorithm to decrease the daily energy cost and emission of harmful gases simultaneously. Performance comparisons were made using only conventional Q-learning. The result showed an approximately 1.2% reduction of the daily operational costs associated with the proposed technique over conventional Q-learning approaches.

There are two main types of online RL algorithms: off-policy and on-policy. In off-policy methods (e.g., Q-learning), the action-valued function is approximated independently of the policy being followed. Conversely, in on-policy approaches, e.g., in state–action–reward–state–action (SARSA), the action-valued function is continuously updated according to the developed policy, which makes it harder to converge [28]. Whereas with off-policy RL, the agent does not need to follow any specific policy and in fact could even act randomly, on-policy schemes rely on the policy that is being established. Despite the possibly random behaviour, off-policy methods, including the Q-learning algorithm, can converge onto the optimal policy independent on the policy employed during exploration [29]. On the other hand, offline, or data-driven RL develops the policy on pre-collected data. Once the policy is developed, it is deployed to control the system. The policy is not updated by interacting with the system in real time. Offline learning, such as batch RL and other conventional techniques such as MILP, MINLP, and LP algorithms, work with data in bulk. Therefore, the uncertainty of some unknown variables such as load demand and PV profiles make these offline methods more challenging because the training is done on forecasted and not real data. Conventionally, offline learning algorithms need to be re-run from scratch in order to learn from modified or new data.

In Mbuwir et al. [19], the authors proposed batch reinforcement learning, offline RL, to solve the optimisation of the microgrid problem in order to achieve a cost-efficient solution. The goal was to find or statistically learn the pattern of the best control policy from the training data (previous year's load and PV profiles) in the form of several smaller batches (sets) and then use this policy on the current environment in real time. When the batch RL was compared to the MILP approach, it was shown to be 19% less efficient than MILP. Kuznetsova et al. [30] developed a two-step-ahead RL algorithm to cut down utility bills by learning the stochastic behaviour of the environment using RL and then scheduling the battery two hours ahead from the current time. RL trains the agent and produces the optimal actions of the battery using forecasted wind and load demand power profiles. Liu et al. [20] proposed a cooperative RL algorithm for distributed economic dispatch in microgrids. However, the challenge of using forecasted PV and load data, which can affect function approximation, is not addressed in this paper. Jiang and Fei [21], suggested a Q-learning based, economical smart microgrid with two-level hierarchical agents with flexible demand response and distributed energy resource management. The authors claim that the suggested scheme is very effective while satisfying the user's preference. However, the suggested work requires load demand from the user before scheduling its distributed units and batteries. This can affect the cost optimisation adversely if the user's demand request changes during real-time operations.

In [22,25,31–43], shallow and deep neural networks have been suggested to approximate the Q-value function to achieve better optimisation results with shorter convergence time. Low convergence time is also desirable for online applications. Lu et al. [22] used deep RL to develop an energy-trading scheme according to the predicted future renewable energy generation, estimated future power demand, and battery level. This work also depended on forecasted renewable energy power generation. In Bui et al. [25], a double deep neural network (DDQN) was proposed for function approximation of Q-values. The authors claim that this method trains the model faster as compared with a single deep (having one network) RL algorithm. This work also depends on the estimation of future load demand and PV generation. Zhou et al. [24] suggested an algorithm to train the agent in real time using real data profiles instead of forecasted datasets. A fuzzy Q-learning approach is adopted for a system consisting of household users and a local energy pool in which customers are free to trade with the local energy pool and enjoy low-cost renewable energy while avoiding the installation of new energy generation equipment. Another online approach was proposed in Kim et al. [26] in which real-time pricing is used to reduce the system cost. Both Zhou et al. [24] and Kim et al. [26] do not provide information regarding the efficiency of their algorithms with respect to other offline Q-learning techniques. Another study by Kim and Lim [44] applied Q-learning directly in real time.

Optimal cost was achieved for the whole year rather than for a single day. In contrast to other offline approaches, this direct online approach trains the agent in real time using real data by moving from one day to another. In the beginning, the agent experience is low; however, as days progress, the agent starts exploiting more actions, and an optimal policy is developed. dicted data in order to produce optimal results if the real data is the same as the predicted data, i.e., zero prediction error. The RL online approach, on the other hand, does not rely on predictions and uses real streaming data. However, it is not clear how effective the online RL algorithm is as compared to the offline approaches when the prediction error increases. Motivated by this shortcoming in the existing literature, this paper provides a

regarding the efficiency of their algorithms with respect to other offline Q-learning techniques. Another study by Kim and Lim [44] applied Q-learning directly in real time. Optimal cost was achieved for the whole year rather than for a single day. In contrast to other offline approaches, this direct online approach trains the agent in real time using real data by moving from one day to another. In the beginning, the agent experience is low; however, as days progress, the agent starts exploiting more actions, and an optimal policy is

The literature reviewed here suggests that offline policies (including RL) require pre-

*Energies* **2021**, *14*, x FOR PEER REVIEW 4 of 19

The literature reviewed here suggests that offline policies (including RL) require predicted data in order to produce optimal results if the real data is the same as the predicted data, i.e., zero prediction error. The RL online approach, on the other hand, does not rely on predictions and uses real streaming data. However, it is not clear how effective the online RL algorithm is as compared to the offline approaches when the prediction error increases. Motivated by this shortcoming in the existing literature, this paper provides a comprehensive comparison between the two approaches. Using one year of real PV generation and load data obtained from [45], different profiles for predicted data were created by generating random noise with different standard deviations. The noise was added to the real data to create synthetic predicted data. Then, the 24 h of predicted data were used to train the RL agent, and then, the optimised battery command achieved in this process was applied to the real data (offline RL approach). The online RL, on the other hand, interacted with real data in real time. Then, the energy costs of the two approaches were compared to help users make decisions on the most appropriate approach given the accuracy of the available forecasted data. Finally, the case with zero prediction error was considered in the comparison of MINLP versus RL to establish a benchmark between conventional offline approaches versus the offline RL. comprehensive comparison between the two approaches. Using one year of real PV generation and load data obtained from [45], different profiles for predicted data were created by generating random noise with different standard deviations. The noise was added to the real data to create synthetic predicted data. Then, the 24 h of predicted data were used to train the RL agent, and then, the optimised battery command achieved in this process was applied to the real data (offline RL approach). The online RL, on the other hand, interacted with real data in real time. Then, the energy costs of the two approaches were compared to help users make decisions on the most appropriate approach given the accuracy of the available forecasted data. Finally, the case with zero prediction error was considered in the comparison of MINLP versus RL to establish a benchmark between conventional offline approaches versus the offline RL. **2. Energy Management System**  The aim of the EMS is to reduce the cost of the power imported from the grid as

#### **2. Energy Management System** shown in Figure 1. Thus, the grid supplies energy only if the RES and BESS do not fulfil

developed.

The aim of the EMS is to reduce the cost of the power imported from the grid as shown in Figure 1. Thus, the grid supplies energy only if the RES and BESS do not fulfil the demand, and the time of transfer is chosen in order to minimise the cost. Export of energy to the main grid might also be possible if a feed-in tariff is available. Renewable energy, PV in this study, has a priority to fulfil the load requirement first. If it is not sufficient, then the battery, main grid, or combination of both are used to fulfil the demand. The BESS may charge from the PV directly or charge from or discharge into the main grid if needed. The EMS can make use of different tariff rates within the day by charging the BESS during low-tariff periods and discharging it during high-tariff periods. In this work, a fixed feed-in tariff is assumed, and there are three different tariffs (peak, mid, low) which are assumed to import the energy from the main grid depending on time of the day. the demand, and the time of transfer is chosen in order to minimise the cost. Export of energy to the main grid might also be possible if a feed-in tariff is available. Renewable energy, PV in this study, has a priority to fulfil the load requirement first. If it is not sufficient, then the battery, main grid, or combination of both are used to fulfil the demand. The BESS may charge from the PV directly or charge from or discharge into the main grid if needed. The EMS can make use of different tariff rates within the day by charging the BESS during low-tariff periods and discharging it during high-tariff periods. In this work, a fixed feed-in tariff is assumed, and there are three different tariffs (peak, mid, low) which are assumed to import the energy from the main grid depending on time of the day.

**Figure 1.** Illustration of microgrid model with BESS. **Figure 1.** Illustration of microgrid model with BESS.

In this paper, an RL algorithm is used to interact with the microgrid and take automated decisions to control the BESS, taking into account a dynamically changing environment characterised by the available PV output, load demand, and the level of battery charge (SOC). The decisions are in the form of actions for the battery to charge, discharge, or remain idle. The recommended actions for the BESS are developed through Q-learning.

#### *2.1. System States*

The state space (*S*) is discretised at ∆t = 30 min, which suggests that the learning agent captures the information related to the dynamics of the microgrid after the time interval of 30 min. In Equation (1), *t* represents the time period, which has 48 states in 24 h of a day due to its discretisation every 30 min.

$$s\_t = \left[ \text{SOC}, e\_t^{\text{Net}}, t \right] \in \mathbb{S}, \tag{1}$$

where *SOC*, *e Net t* are the battery state of charge and net power demand, respectively. The *e Net t* is the difference between the load demand and the energy generated by PV such that:

$$e\_t^{Net} = e\_t^{demand} - e\_t^{PV}.\tag{2}$$

The SOC should be bounded by maximum and minimum limits such that:

$$SOC\_{\min} \le SOC(t) \le SOC\_{\max}.\tag{3}$$

We discretise the state space as shown in Equation (4) below in which the *i*, *j*, *k* indices represent the SOC, *e Net t* and *t*, respectively as:

$$\mathcal{S}\_{discrete} = \left\{ \mathcal{S}\_{i,j,k} \right\}\_{\prime} \tag{4}$$

where each index in the state space has the following levels: *i* = 3 levels, *j* = 2 levels, i.e., positive (*e Net <sup>t</sup>* ≥ *e PV t* ) or negative (*e Net <sup>t</sup>* < *e PV t* ), and *k* = 48 levels. Thus, the total number of states is 48 × 3 × 2 = 288.

#### *2.2. Action Space*

The action space consists of the charge, discharge, and idle command of the battery such as:

$$A = \{ a | (\text{Discharge, Ide, Charge}) \}. \tag{5}$$

At each time step *t*, one action is selected from the action space *A*. If the action "Discharge" is chosen, the battery discharges into the main grid, supplies the load, or both. In case of the action "Idle", the load demand is fulfilled by the PV source, main grid, or both. If the "Charge" action is selected, the battery is charged from the PV, the grid, or both.

#### *2.3. Backup Controller*

In this work, we used a backup controller, which acts as a filter for every control action resulting from the policy π to take care of the practical constraint, such as the inability of the battery to charge or discharge beyond its maximum and minimum SOC level, respectively. In addition, there is a certain limit of battery charging or discharging at time *t*. For example, if the "Charge" action is selected by the RL agent at time *t* and one of the discrete states is (e Net <sup>t</sup> > e PV t ), then the battery should charge from the main grid up to a certain limit (∆e) defined in Table 1 even if the capacity of the battery is more than ∆e. Moreover, if at time *t*, one of the discrete states is (e Net <sup>t</sup> < e PV t ) and the RL agent selects the action "Charge", the battery will charge from the extra PV available (after fulfilling the load demand) by respecting the charging rate parameter of the battery. If for example the current PV power is more or less than the charging rate of the battery, the battery is charged up to the maximum charging rate (∆e) or current PV power, respectively.


**Table 1.** Chosen parameters of the microgrid used in this paper.

#### *2.4. Reward*

The reward function *r*(*s<sup>t</sup>* ,*at*) is the immediate incentive gained by taking a specific action *a* at time *t* in state*s*. The reward function is chosen to minimise the running cost of importing power from the grid and maximise the revenue of selling power to the grid. The cost is calculated every 30 min (as ∆*t* = 30) by multiplying the respective tariff rates, as mentioned in Section 2.5. The reward function is the negative of the cost of imported energy or the cost of exported energy. Hence, the reward function can be formulated as follows:

$$\mathbf{r}(\mathbf{s}\_{t\prime}, a\_t) = \left\{ \begin{array}{ll} -\mathbf{P}\_t^{\rm grid} \times \Delta t \times \mathbf{T} \\ \mathbf{P}\_t^{\rm grid} \times \Delta t \times \mathbf{T} \\ \end{array} \right. \\ \left. \begin{array}{ll} \mathbf{P}\_{\rm grid} \ge \mathbf{0} \\ \end{array} \right\}\_{\nu} \tag{6}$$

where *Tari f fimp* and *Tarri f fexp* are the import and export tariffs, respectively. *P grid t* is the grid power and is given by:

$$P\_t^{grid} = e\_t^{Net} + P\_t^{hatt} \,\prime \tag{7}$$

where *P batt t* is the power used to charge the battery.

*2.5. Tariff*

The import tariff has three different values depending on the time of use:

$$Tariff\_{imp} = \left\{ \begin{array}{ll} 0.05 \text{\textquotedblleft}/\text{kWh} \\ 0.08 \text{\textquotedblright}/\text{kWh} \\ 0.171 \text{\textquotedblright}/\text{kWh} \end{array} \text{medium peak, } 9:00 \text{ to } 12:00 \\ \begin{array}{ll} 0.171 \text{\textquotedblleft}/\text{kWh} \\ \text{high peak, } 19:00 \text{ to } 21:00 \end{array} \right\}. \tag{8}$$

The export tariff does not vary and it is *Tari f fexp* = 0.033*£*/kWh.

#### **3. Q-Learning Algorithm**

The backbone of the Q-learning algorithm is based on the two components described in Equation (9) as:

$$\mathcal{R}\_t^{\ \pi} = r(s\_t, a\_t) + \sum\_{i=1}^{\infty} \gamma^i r(s\_{t+i}, a\_{t+i}). \tag{9}$$

The first component shows the impact of the current action on future rewards, and the second component is the total discounted rewards at time step *t* under a given policy *π*. Therefore, *R π t* is defined as the sum of the instant reward at time step *t* plus the future discounted rewards. The parameter *γ* is the discount factor used to determine the importance of future rewards from the next time step (*t* + 1) up to infinity. If *γ* = 0, the algorithm considers the current reward only, while if *γ* = 1, both current and future rewards have equal weight. In *Q*-learning, the policy is learned implicitly without any prior knowledge. This is done by approximating the action-value function by repeatedly updating the *Q*(*s<sup>t</sup>* , *at*) through experience such as:

$$Q(\mathbf{s}\_{l\prime}, a\_{l}) = Q(\mathbf{s}\_{l\prime}, a\_{l}) + \mathfrak{a}[r(\mathbf{s}\_{l\prime}, a\_{l}) + \gamma \max Q(\mathbf{s}\_{l+1\prime}, a\_{l+1}) - Q(\mathbf{s}\_{l\prime} a\_{l})].\tag{10}$$

To approximate the *Q*-table, it is important to estimate all state–action pairs of the table. Parameter α is the learning rate that determines how much the newly obtained value overrides the old value of *Q*(*s<sup>t</sup>* , *at*). If *α* = 0, the newly obtained information is ignored during training, whereas if *α* = 1, only the latest information is used. Therefore, in Q-learning, the selection of *α* (ranging between 0 and 1) is very important to keep a balance between the old and new information. The RL agent takes random actions in the beginning if it follows an ε-greedy policy, which is adopted in this work. The idea behind the purely ε-greedy approach is to try every decision once and then keep picking the one that results in the highest reward as learning progresses. After certain iterations and by performing different actions in each state of the Q-table, the agent learns to maximise the value (state–action) of the Q-table by taking greedy actions. Random and greedy actions correspond to exploration (ε) versus exploitation, respectively. In this work, taking a greedy action, the decision is based on: greedy action, the decision is based on: ε ε ← − *Ms M* ( ) (11) where *M*(*s*) is the number of times a certain action is taken in a specific state. *M*max is the maximum constant value selected after which greedy actions are selected by the Q-learning algorithm. *3.1. Offline RL Implementation* 

*Energies* **2021**, *14*, x FOR PEER REVIEW 7 of 19

dating the ܳ(ݏ௧, ܽ௧) through experience such as:

discounted rewards. The parameter *γ* is the discount factor used to determine the im-

portance of future rewards from the next time step (*t* + 1) up to infinity. If *γ* = 0, the algo-

rithm considers the current reward only, while if *γ* = 1, both current and future rewards

have equal weight. In *Q*-learning, the policy is learned implicitly without any prior

knowledge. This is done by approximating the action-value function by repeatedly up-

*Qs a Qs a rs a Qs a Qs a* ( , ) ( , ) ( , ) max ( , ) ( , ) *tt tt tt t t tt* =+ + −

table. Parameter α is the learning rate that determines how much the newly obtained

value overrides the old value of ܳ(ݏ௧, ܽ௧). If *α* = 0, the newly obtained information is ig-

nored during training, whereas if *α* = 1, only the latest information is used. Therefore, in

Q-learning, the selection of *α* (ranging between 0 and 1) is very important to keep a bal-

ance between the old and new information. The RL agent takes random actions in the

beginning if it follows an ε-greedy policy, which is adopted in this work. The idea behind

the purely ε-greedy approach is to try every decision once and then keep picking the one

that results in the highest reward as learning progresses. After certain iterations and by

performing different actions in each state of the Q-table, the agent learns to maximise the

value (state–action) of the Q-table by taking greedy actions. Random and greedy actions

correspond to exploration (ε) versus exploitation, respectively. In this work, taking a

as charging/discharging/idle commands of the battery. The estimated synthetic data for

max

 γ

To approximate the *Q*-table, it is important to estimate all state–action pairs of the

[ + + 1 1 ]. (10)

α

$$
\varepsilon \leftarrow \varepsilon / \sqrt{M(s) - M\_{\text{max}}} \tag{11}
$$

where *M*(*s*) is the number of times a certain action is taken in a specific state. *M*max is the maximum constant value selected after which greedy actions are selected by the Q-learning algorithm. discussed. At the beginning of each day, the forecasted PV and load data are gathered as inputs to RL. Then, Q-learning is run using the same input data until convergence is

#### *3.1. Offline RL Implementation* achieved. The policy developed at the end of this phase is used to generate the charging,

In this section, the implementation of offline RL to control BESS in microgrids will be discussed. At the beginning of each day, the forecasted PV and load data are gathered as inputs to RL. Then, Q-learning is run using the same input data until convergence is achieved. The policy developed at the end of this phase is used to generate the charging, discharging and idle commands for the next 24 h. This strategy is repeated for each day. The backup controller monitors the control parameters of the battery. After selecting the control actions of the battery from Q-learning, the backup controller ensures that all physical constraints and limitations are met before actually applying the battery actions on the physical system. Figure 2 illustrates the offline implementation of RL-based EMS. discharging and idle commands for the next 24 h. This strategy is repeated for each day. The backup controller monitors the control parameters of the battery. After selecting the control actions of the battery from Q-learning, the backup controller ensures that all physical constraints and limitations are met before actually applying the battery actions on the physical system. Figure 2 illustrates the offline implementation of RL-based EMS.

**Figure 2.** Offline implementation of the RL algorithm.

**Figure 2.** Offline implementation of the RL algorithm. This offline RL implementation is similar to the traditional EMS approaches such as MILP, where estimated data are used by the optimiser to produce decision variables, such This offline RL implementation is similar to the traditional EMS approaches such as MILP, where estimated data are used by the optimiser to produce decision variables, such as charging/discharging/idle commands of the battery. The estimated synthetic data for PV generation and load consumption for the next 24 h are used by RL to schedule the battery command. Each episode of one day consists of 48 steps (30 min time interval). The RL keeps on using the same data until convergence is achieved. A total of 10,000–15,000

iterations were employed for better convergence. The optimised battery commands are dispatched at the start of the following day for the battery to operate in real time. The same process is repeated for every day of the year. As we are interested to find the total cost of a complete year (365 days), for the initialization of offline Q-learning, the Q-table simply initialises the action-value function at time step 0 with the value of 0 or ∞. Online RL is applied directly to real data in real time. Therefore, the agent learns the optimal policy by interacting with the real system. There is no pre-training in this online approach, unlike offline techniques. Figure 3 shows the online RL for EMS. The online RL algorithm updates the actions of the battery and dispatches them every 30 min in real time

*Energies* **2021**, *14*, x FOR PEER REVIEW 8 of 19

PV generation and load consumption for the next 24 h are used by RL to schedule the

battery command. Each episode of one day consists of 48 steps (30 min time interval). The

RL keeps on using the same data until convergence is achieved. A total of 10,000–15,000

iterations were employed for better convergence. The optimised battery commands are

dispatched at the start of the following day for the battery to operate in real time. The

same process is repeated for every day of the year. As we are interested to find the total

cost of a complete year (365 days), for the initialization of offline Q-learning, the Q-table

prediction error. We use an algorithm to add random noise to the real net power demand

tained from [45]. The noise is generated using normally distributed white Gaussian noise

having different standard deviation (*σ*) values. For each *σ*, five noise profiles are gener-

ated, averaged, and then added to the real net power demand data to produce the fore-

casted net power demand, as shown in Figure 4. The increase in the σ value will increase

represents an increasing trend of deviation with respect to the real power demand and

. The real PV and load profiles for a complete year are ob-

simply initialises the action-value function at time step 0 with the value of 0 or ∞.

### *3.2. Online RL Implementation*

*3.2. Online RL Implementation* 

Online RL is applied directly to real data in real time. Therefore, the agent learns the optimal policy by interacting with the real system. There is no pre-training in this online approach, unlike offline techniques. Figure 3 shows the online RL for EMS. The online RL algorithm updates the actions of the battery and dispatches them every 30 min in real time regardless of the status of convergence. Learning can be very slow, especially in first few days. Before convergence, the performance would be suboptimal. With time, the agent develops an optimal policy. The function of the backup controller in the online RL implementation is the same as for offline RL, as described in Section 2.3. regardless of the status of convergence. Learning can be very slow, especially in first few days. Before convergence, the performance would be suboptimal. With time, the agent develops an optimal policy. The function of the backup controller in the online RL implementation is the same as for offline RL, as described in Section 2.3.

**Figure 3.** Online implementation of the RL algorithm.

**Figure 3.** Online implementation of the RL algorithm. There is no separate training stage in online RL; rather, the agent gathers experiences during real-time interaction with the environment. Therefore, in the beginning, the Q-ta-There is no separate training stage in online RL; rather, the agent gathers experiences during real-time interaction with the environment. Therefore, in the beginning, the Q-table is initialised with a shortsighted future reward with the algorithm hyper-parameter *γ* set to 0. Then, the table will be updated in real time by interacting with the real system. This simple initialisation step reduces the convergence time substantially as per [44].

#### *3.3. Prediction Error Generation*

given by ݁௧

vice versa.

ble is initialised with a shortsighted future reward with the algorithm hyper-parameter *γ* set to 0. Then, the table will be updated in real time by interacting with the real system. This simple initialisation step reduces the convergence time substantially as per [44]. *3.3. Prediction Error Generation*  To compare the performance of both offline and online Q-learning, there is a need to create a difference between forecasted data (PV and load) and real data to represent the To compare the performance of both offline and online Q-learning, there is a need to create a difference between forecasted data (PV and load) and real data to represent the prediction error. We use an algorithm to add random noise to the real net power demand given by *e Net <sup>t</sup>* = *e demand <sup>t</sup>* − *e PV t* . The real PV and load profiles for a complete year are obtained from [45]. The noise is generated using normally distributed white Gaussian noise having different standard deviation (*σ*) values. For each *σ*, five noise profiles are generated, averaged, and then added to the real net power demand data to produce the forecasted net power demand, as shown in Figure 4. The increase in the σ value will increase the standard deviation error. Thus, the forecasted net demand with higher sigma values represents an increasing trend of deviation with respect to the real power demand and vice versa.

ௗௗ − ݁௧

ே௧ = ݁௧

**Figure 4.** Real and forecasted net demand per hour for one day after noise addition (at σ =100). **Figure 4.** Real and forecasted net demand per hour for one day after noise addition (at σ =100).

#### **4. Simulation Results 4. Simulation Results**

This section evaluates the performance of the offline and online optimisation techniques. The offline and online RL were both applied on a daily basis with an interval of 30 min. Firstly, we compared the results of offline RL with MILP to analyse the efficiency of both algorithms in terms of cost saving in a grid-tied microgrid system. Then, we compared offline RL with online RL after establishing a benchmark between the offline RL and MILP techniques. In this regard, both the offline and online RL optimisation techniques need to be investigated in terms of cost saving per year. This work compares the behaviour of both the approaches when there is a different percentage of errors present between forecasted and real data profiles (PV and load). This information can be used to decide between offline and online RL when the real data (PV and Load) profiles deviate from the forecasted data. The real data assumed in this work are gathered from the online This section evaluates the performance of the offline and online optimisation techniques. The offline and online RL were both applied on a daily basis with an interval of 30 min. Firstly, we compared the results of offline RL with MILP to analyse the efficiency of both algorithms in terms of cost saving in a grid-tied microgrid system. Then, we compared offline RL with online RL after establishing a benchmark between the offline RL and MILP techniques. In this regard, both the offline and online RL optimisation techniques need to be investigated in terms of cost saving per year. This work compares the behaviour of both the approaches when there is a different percentage of errors present between forecasted and real data profiles (PV and load). This information can be used to decide between offline and online RL when the real data (PV and Load) profiles deviate from the forecasted data. The real data assumed in this work are gathered from the online open-source data platform in reference [45]. The chosen parameters used to simulate the behaviour of the microgrid are provided in Table 1.

open-source data platform in reference [45]. The chosen parameters used to simulate the behaviour of the microgrid are provided in Table 1. Below, Figure 5 shows the average net forecasted ݁௦௧ ௦௨/௬/5 demand per year us-Below, Figure 5 shows the average net forecasted *e sum*/*year f orecast* /5 demand per year using all five samples at each *σ*. Then, Equation (12) is used to find the percentage error between the real and forecasted power demand per year:

$$Net^{error/year} = \frac{1}{n} \sum\_{1}^{n} \left( \frac{e\_{\text{Real}}^{sum/year} - e\_{forecast}^{sum/year}}{e\_{\text{Real}}^{sum/year}} \right). \tag{12}$$

 forecast

Real

1 *sum/ year Real Net n e* = . (12) The error bars in Figure 5 above show the *Neterror*/*year* for each sigma using the highest and lowest sample of *e sum*/*year f orecast* . The arrows indicating 1 and 2 in Figure 5 indicate the constant *e sum*/*year real* and varying *Neterror*/*year* in Equation (12), respectively.

Figure 6 depicts the difference between the generated *e sum*/*year f orecast* /5 and the real power demand. The difference of the total real power demand per year with the highest and lowest sum of the generated forecasted power demand per year out of five samples at each sigma is described using the error bars on the secondary axis of Figure 6.

**Figure 5.** Average sum of the net forecasted demand per year out of five random samples. **Figure 5.** Average sum of the net forecasted demand per year out of five random samples.

**Figure 6.** Average sum of forecasted demand with respect to real demand per year. **Figure 6.** Average sum of forecasted demand with respect to real demand per year.

The two datasets, real (original) and forecasted (synthetic), are related to the net power demand per year and used to compare the performance of offline and online RL. Firstly, we compare offline MILP with offline RL by considering both the forecasted and real data profiles (PV, load demand) to be the same for both approaches. The results (Figure 7) show that both approaches have almost identical performance in terms of cost optimisation of the microgrid, with negligible difference. Over that small difference, MILP The two datasets, real (original) and forecasted (synthetic), are related to the net power demand per year and used to compare the performance of offline and online RL. Firstly, we compare offline MILP with offline RL by considering both the forecasted and real data profiles (PV, load demand) to be the same for both approaches. The results (Figure 7) show that both approaches have almost identical performance in terms of cost optimisation of the microgrid, with negligible difference. Over that small difference, MILP behaves slightly

behaves slightly better due to the convergence requirements of RL. Therefore, the offline

**Figure 7.** Comparison between MILP and RL in terms of cost per hour.

**Figure 6.** Average sum of forecasted demand with respect to real demand per year.

*Energies* **2021**, *14*, x FOR PEER REVIEW 11 of 19

better due to the convergence requirements of RL. Therefore, the offline approaches, such as MILP and RL, which use the same forecasted and real data (having 0% error) for training, are equally good in real time. behaves slightly better due to the convergence requirements of RL. Therefore, the offline approaches, such as MILP and RL, which use the same forecasted and real data (having 0% error) for training, are equally good in real time.

The two datasets, real (original) and forecasted (synthetic), are related to the net power demand per year and used to compare the performance of offline and online RL. Firstly, we compare offline MILP with offline RL by considering both the forecasted and real data profiles (PV, load demand) to be the same for both approaches. The results (Figure 7) show that both approaches have almost identical performance in terms of cost optimisation of the microgrid, with negligible difference. Over that small difference, MILP

**Figure 7. Figure 7.** Comparison between MILP and RL in terms of cost per hour. Comparison between MILP and RL in terms of cost per hour.

Figure 8 shows the convergence in terms of cost using offline RL. In this figure, only real data (PV and load) profiles were employed to analyse the convergence pattern of the offline RL as an ideal case. In the beginning, the cost is high and the curve shows random behaviour. As the number of episodes increases, the learning ability of the agent improves until the Q-table converges to show the optimal cost. Figure 8 shows the convergence in terms of cost using offline RL. In this figure, only real data (PV and load) profiles were employed to analyse the convergence pattern of the offline RL as an ideal case. In the beginning, the cost is high and the curve shows random behaviour. As the number of episodes increases, the learning ability of the agent improves until the Q-table converges to show the optimal cost.

The performance of the two RL approaches is first compared when there is no forecast error; i.e., the forecast data are the same as real data. Of course, this ideal situation does not exist in practice, but it provides an initial benchmark for the results. Both RL approaches are implemented as explained earlier in Sections 3.1 and 3.2. In addition, MILP

when the forecast error is zero, offline RL produces superior results. In Figures 9 and 10, it can be seen that before the convergence of the online RL algorithm, a higher amount of grid energy is imported; therefore, the cost is also higher with respect to offline RL in the initial days. However, as the days progress and online RL learning converges to the optimal policy, the controller follows the same pattern as offline RL in terms of cost saving and reducing the imported energy. In this work, convergence was achieved in between 75

**Figure 8.** Convergence of the cost during the offline RL algorithm training. **Figure 8.** Convergence of the cost during the offline RL algorithm training.

*4.1. Offline vs. Online RL with No Forecast Error* 

and 90 days in the case of online RL.

#### *4.1. Offline vs. Online RL with No Forecast Error*

day.

load and PV (݁௧

The performance of the two RL approaches is first compared when there is no forecast error; i.e., the forecast data are the same as real data. Of course, this ideal situation does not exist in practice, but it provides an initial benchmark for the results. Both RL approaches are implemented as explained earlier in Section 3.1 and Section 3.2. In addition, MILP was also used to optimise the battery operation. Figures 9 and 10 show the energy imported from the grid with respective cost on a daily basis, respectively. It can be seen that when the forecast error is zero, offline RL produces superior results. In Figures 9 and 10, it can be seen that before the convergence of the online RL algorithm, a higher amount of grid energy is imported; therefore, the cost is also higher with respect to offline RL in the initial days. However, as the days progress and online RL learning converges to the optimal policy, the controller follows the same pattern as offline RL in terms of cost saving and reducing the imported energy. In this work, convergence was achieved in between 75 and 90 days in the case of online RL. *Energies* **2021**, *14*, x FOR PEER REVIEW 13 of 19 *Energies* **2021**, *14*, x FOR PEER REVIEW 13 of 19

**Figure 9.** Performance of offline (ideal scenario) vs. online RL in terms of imported energy per **Figure 9.** Performance of offline (ideal scenario) vs. online RL in terms of imported energy per day. day.

**Figure 10.** Performance of offline (ideal scenario) vs. online RL in terms of cost per day. **Figure 10.** Performance of offline (ideal scenario) vs. online RL in terms of cost per day.

**Figure 10.** Performance of offline (ideal scenario) vs. online RL in terms of cost per day. Figure 11 shows the overview of the battery actions generated in an average day after applying the Q-learning algorithm. We used the offline RL for a single day in Figure 11 to show the different states of the battery after convergence during different time intervals of the day with respect to load demand and PV availability. The difference between the Figure 11 shows the overview of the battery actions generated in an average day after applying the Q-learning algorithm. We used the offline RL for a single day in Figure 11 to show the different states of the battery after convergence during different time intervals of the day with respect to load demand and PV availability. The difference between the load and PV (݁௧ ே௧) is shown in the graph below at each time step. The ݁௧ ே௧ can be nega-Figure 11 shows the overview of the battery actions generated in an average day after applying the Q-learning algorithm. We used the offline RL for a single day in Figure 11 to show the different states of the battery after convergence during different time intervals of the day with respect to load demand and PV availability. The difference between the load and PV (*e Net t* ) is shown in the graph below at each time step. The *e Net t* can be negative if

ே௧ can be nega-

 > ݁௧ ே௧

 > ݁௧ ே௧

ே௧) is shown in the graph below at each time step. The ݁௧

physical constraints, as described in Section 2.3. As shown in Figure 11, when ݁௧

tive if the PV power generated is greater than the load demand at time *t*. The suggested

tive if the PV power generated is greater than the load demand at time *t*. The suggested battery actions by the RL agent pass through the backup controller to accommodate all

at time *t*, the battery charges from the current PV power up to the maximum level of ∆*e* after fulfilling the load demand. The outstanding PV power is sold to the main grid. During discharging, the battery discharges up to maximum level of ∆*e* either to fulfil the load

at time *t*, the battery charges from the current PV power up to the maximum level of ∆*e* after fulfilling the load demand. The outstanding PV power is sold to the main grid. During discharging, the battery discharges up to maximum level of ∆*e* either to fulfil the load

demand or sell power to the utility grid.

demand or sell power to the utility grid.

the PV power generated is greater than the load demand at time *t*. The suggested battery actions by the RL agent pass through the backup controller to accommodate all physical constraints, as described in Section 2.3. As shown in Figure 11, when *e PV <sup>t</sup>* > *e Net t* at time *t*, the battery charges from the current PV power up to the maximum level of ∆*e* after fulfilling the load demand. The outstanding PV power is sold to the main grid. During discharging, the battery discharges up to maximum level of ∆*e* either to fulfil the load demand or sell power to the utility grid. *Energies* **2021**, *14*, x FOR PEER REVIEW 14 of 19

**Figure 11.** Comparison of offline vs. online RL after convergence. **Figure 11.** Comparison of offline vs. online RL after convergence.

Figure 11 is the randomly selected day of the year when both online and offline RL converge. The plot shows the same battery actions for both offline and online RL. Therefore, the cost achieved in both approaches are also same. Figure 11 is the randomly selected day of the year when both online and offline RL converge. The plot shows the same battery actions for both offline and online RL. Therefore, the cost achieved in both approaches are also same.

#### *4.2. Offline vs. Online with Varying Forecast Error 4.2. Offline vs. Online with Varying Forecast Error*

The forecasted profiles generated in Section 3.3 are used by the offline RL to create the battery charge/discharge/idle commands, which are then applied to the real data as was explained in Section 3.1 and Figure 2. For online RL, real data are used to generate the battery commands that are directly applied to the physical microgrid system. Figure 12 shows the overview of the battery actions (kW) generated in an average day after introducing 1.6% forecasted error with respect to the real net demand. Figure 12 also showed the daily cost of offline and online RL. The offline RL cost is higher than that of online RL at the time steps 31 to 35. Therefore, the overall average cost of offline RL in a The forecasted profiles generated in Section 3.3 are used by the offline RL to create the battery charge/discharge/idle commands, which are then applied to the real data as was explained in Section 3.1 and Figure 2. For online RL, real data are used to generate the battery commands that are directly applied to the physical microgrid system. Figure 12 shows the overview of the battery actions (kW) generated in an average day after introducing 1.6% forecasted error with respect to the real net demand. Figure 12 also showed the daily cost of offline and online RL. The offline RL cost is higher than that of online RL at the time steps 31 to 35. Therefore, the overall average cost of offline RL in a day is higher than that of online RL (after convergence).

day is higher than that of online RL (after convergence). Figure 13 shows the optimal average cost achieved per year for both offline and online RL.

The results show that the average cost and imported energy of the offline Q-learning increase as the relative error between the forecasted and real power demand grows or vice versa. The error between real and synthetic predicted profiles are calculated using Equation (12). A rise in the value of the standard deviation reflects the increase in the error (in percent), as shown in Figure 5. Therefore, the noise level (*σ*) and the relative error are proportional to each other. At the start, when the error of the forecasted demand with respect to the real demand is low, the offline Q-learning performs better in comparison with the online RL in terms of cost optimisation per year. However, as the error increases, for example at 1.61% (between forecasted and real net demand), the online Q-learning begins to perform better and results in a lower cost than the offline Q-leaning.

**Figure 12.** Comparison of offline vs. online RL (with forecast error) after convergence.

**Figure 11.** Comparison of offline vs. online RL after convergence.

fore, the cost achieved in both approaches are also same.

day is higher than that of online RL (after convergence).

*4.2. Offline vs. Online with Varying Forecast Error* 

Figure 11 is the randomly selected day of the year when both online and offline RL converge. The plot shows the same battery actions for both offline and online RL. There-

The forecasted profiles generated in Section 3.3 are used by the offline RL to create the battery charge/discharge/idle commands, which are then applied to the real data as was explained in Section 3.1 and Figure 2. For online RL, real data are used to generate the battery commands that are directly applied to the physical microgrid system. Figure 12 shows the overview of the battery actions (kW) generated in an average day after introducing 1.6% forecasted error with respect to the real net demand. Figure 12 also showed the daily cost of offline and online RL. The offline RL cost is higher than that of online RL at the time steps 31 to 35. Therefore, the overall average cost of offline RL in a

**Figure 12.** Comparison of offline vs. online RL (with forecast error) after convergence. **Figure 12.** Comparison of offline vs. online RL (with forecast error) after convergence. Figure 13 shows the optimal average cost achieved per year for both offline and

online RL.

**Figure 13.** Performance of online vs. offline RL in terms of average total cost per year. **Figure 13.** Performance of online vs. offline RL in terms of average total cost per year.

#### **5. Discussion**

increase as the relative error between the forecasted and real power demand grows or vice versa. The error between real and synthetic predicted profiles are calculated using Equation (12). A rise in the value of the standard deviation reflects the increase in the error (in percent), as shown in Figure 5. Therefore, the noise level (*σ*) and the relative error are proportional to each other. At the start, when the error of the forecasted demand with respect to the real demand is low, the offline Q-learning performs better in comparison with the online RL in terms of cost optimisation per year. However, as the error increases, for example at 1.61% (between forecasted and real net demand), the online Q-learning begins to perform better and results in a lower cost than the offline Q-leaning. **5. Discussion** In this study, we report a comparison between the offline and online optimisation approaches to reduce the operational cost of a grid-connected microgrid by optimally managing the BESS in the presence of forecasting error. A variation between the forecasted and real-time demand may occur due to a change in the weather, ultimately affecting the suggested real-time optimal battery actions obtained through offline training. Hence, the results may not be optimal in terms of reducing cost. To avoid the In this study, we report a comparison between the offline and online optimisation approaches to reduce the operational cost of a grid-connected microgrid by optimally managing the BESS in the presence of forecasting error. A variation between the forecasted and real-time demand may occur due to a change in the weather, ultimately affecting the suggested real-time optimal battery actions obtained through offline training. Hence, the results may not be optimal in terms of reducing cost. To avoid the complication of using forecasted data profiles (PV, load) offline, the literature suggested the online Q-learning [44]. However, online Q-learning needs some days to converge, as its training happens online after exploring real-time data. This knowledge gap in the existing research indicated a need for thorough analysis and comparison of both offline and online approaches. Therefore, we proposed the comparison between offline versus online algorithms on an annual basis. Conventional MILP and offline RL show approximately similar behaviour in terms of saving cost in microgrid operation. Then, the average annual costs of offline and online RL approaches were analysed using different forecasted data profiles. The synthetic forecasted data with respect to real data were produced by adding random white Gaussian noise with specified standard deviation.

The results show that the average cost and imported energy of the offline Q-learning

complication of using forecasted data profiles (PV, load) offline, the literature suggested

existing research indicated a need for thorough analysis and comparison of both offline and online approaches. Therefore, we proposed the comparison between offline versus online algorithms on an annual basis. Conventional MILP and offline RL show approximately similar behaviour in terms of saving cost in microgrid operation. Then, the average annual costs of offline and online RL approaches were analysed using different forecasted data profiles. The synthetic forecasted data with respect to real data were produced by adding random white Gaussian noise with specified standard deviation.

Table 2 suggested the possible implementation of offline or online RL on modern research to optimise the economy of power systems. For example, different type of optimisation problems were solved by using different methods in the past, as mentioned in Section 1. These solutions faced a range of challenges, including convergence,

Table 2 suggested the possible implementation of offline or online RL on modern research to optimise the economy of power systems. For example, different type of optimisation problems were solved by using different methods in the past, as mentioned in Section 1. These solutions faced a range of challenges, including convergence, inefficient optimisation (cost, Co2 Emission), and computational time under different or varied conditions. The scope of this article suggests that reinforcement learning (offline/online) has full potential to deal with the different types of optimisation problems and challenges.

**Table 2.** Proposed offline and online RL on the current application scenario.


#### **6. Conclusions**

The following are the key findings of this paper:


Therefore, a higher difference between forecasted and real PV and load profiles suggests adopting an online Q-learning approach. For example, there are certain countries and areas where the forecasting of PV and load demand are not certain due to abrupt changes in the weather condition or the user behaviour. The actual energy demand at run times may change a lot in contrast to the predictions. The online Q-learning for cost optimisation provides a better solution in these regions. While operating in favourable prediction conditions, the offline Q-learning performs better.

In the future, there may be other quantification methods such as root mean square error (RMSE) that can be used to introduce errors in forecasted data with respect to real data to compare these two approaches for other type of noise distributions. In the future, both offline and online Q-learning approaches may be employed as a two-layer structure. This can provide a better and more efficient solution for the cost optimisation in a real microgrid.

**Author Contributions:** Conceptualisation, M.A. and K.H.A.; methodology, K.H.A.; validation, M.A. and S.D.; formal analysis, E.A.; investigation, K.H.A. and M.S.; resources, A.A.T.; writingoriginal draft preparation, K.H.A.; writing—review and editing, M.A., E.A., S.D., A.A.T. and K.H.A.; supervision, M.A. and A.A.T. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded by the Engineering and Physical Sciences Research Council (EPSRC) grant number EP/T025875/1 and The APC was partially funded by the editor allowance.

**Data Availability Statement:** The data is available from the lead or the corresponding author upon reasonable requests.

**Acknowledgments:** The financial support was provided by the Engineering and Physical Sciences Research Council (EPSRC), U.K. under research Grant No. EP/T025875/1. K.H.A. would also like to thank Sukkur Institute of Business Administration University, Pakistan for fully funded Ph D studentship.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **Nomenclature**


#### **References**


## *Article* **Modified Beetle Annealing Search (BAS) Optimization Strategy for Maxing Wind Farm Power through an Adaptive Wake Digraph Clustering Approach**

**Yanfang Chen 1,2, Young-Hoon Joo 2,\* and Dongran Song <sup>3</sup>**


**Abstract:** Owing to scale-up and complex wake effects, the centralized control that processes the command from turbines may be unsuitable, as it incurs high communication overhead and computational complexity for a large offshore wind farm (OWF). This paper proposes a novel decentralized non-convex optimization strategy for maxing power conversion of a large OWF based on a modified beetle antennae search (BAS) algorithm. First, an adaptive threshold algorithm which to establish a pruned wake direction graph while preserving the most critical wake propagation relationship among wind turbines are presented. The adaptive graph constraints were used to create wake sub-digraphs that split the wind farm into nearly uncoupled clustering communication subsets. On this basis, a Monte Carlo-based beetle annealing search (MC-BAS) nonlinear optimization strategy was secondly designed to adjust the yaw angles and axial factors for the maximum power conversion of each turbine subgroup. Finally, the simulation results demonstrated that a similar gain could be achieved as a centralized control method at power conversion and reduces the computational cost, allowing it to solve the nonlinear problem and real-time operations of the OWF.

**Keywords:** beetle antennae search optimization; wake propagation; direct graph; offshore wind farm; clustering subset; graph adaptive pruning

## **1. Introduction**

With increasing environmental problems, such as the greenhouse effect, clean energy has become a critical issue that needs to be solved worldwide. Wind energy has become very competitive in comparison with other green energy production technologies for the mature control technology, such as the control approach to improve the performance of wind turbines in different conditions [1], model predictive control [2], etc. Moreover, the exploitation of wind energy is mainly fulfilled by wind turbines in arrays or groups to reduce the cost of energy [3,4]. Additionally, energy storage is also a very important branch [5].

To increase the total output power, the optimization methodology [6] plays a vital role by considering the topography, prevailing wind direction environment [7], and the space of turbines. For a designed wind farm, there are also some other methodologies for improving the total output power [8–10], decreasing the thrust load [11], improving the lifetime of turbines [12], and tracking power reference signals to improve wind integration into the power grid [13–16]. Researchers have recently proposed some control methods for optimizing power conversion in large wind farms without considering wake infection [17,18]. Some kinds of literature [19–21] propose novel graph theories. The production of generating an interaction matrix is treated in the same way. However, the difference is the kinetic energy deficits with considering the wind speed and direction probabilities, which is explained in detail in Equation (7) in paper [18]. Additionally, in this paper, we

**Citation:** Chen, Y.; Joo, Y.-H.; Song, D. Modified Beetle Annealing Search (BAS) Optimization Strategy for Maxing Wind Farm Power through an Adaptive Wake Digraph Clustering Approach. *Energies* **2021**, *14*, 7326. https://doi.org/10.3390/ en14217326

Academic Editor: Frede Blaabjerg

Received: 21 September 2021 Accepted: 30 October 2021 Published: 4 November 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

focus on the variations of wind velocity between the upstream and downstream turbines. However, the constraints of the offshore land limit the space of turbines, not infinitely long. Therefore, the wake effects in a wind farm are inevitable, and the closely spaced turbines will produce little output power. The reason is that wake affects the downstream wind speed. Moreover, the relationship between the upstream and downstream turbines is inseparable from the basic turbine model, and the existing turbine models currently contain Jensen's model [22], Frandsen's model [23], CFD-wake model [24], etc. The Jensen and Frandsen models have assumed "top-hat" shape distributions for the wake velocity deficit. In fact, the actual distribution is Gaussian as in [21,25], which is based on self-similarity theory and is often used in free shear flows. It has become a hot topic that the optimization algorithm performs the search of the values of optimal axial and yaw angle to maximize the output power and mitigate the wake interference.

For large-scale wind farms, the burden of communication with the central unit is significantly large, and the speed of centralized implementation is no longer suited for real-time control [15,16]. More recent studies have investigated decentralized control, distributed control [26], and cooperate control [27] which are proposed to decrease the communication burden and improve control speed by dividing whole turbines into several decoupled subsets. Centralized control only suits a small-scale wind farm because there is much large information sharing between turbines and the central controller, and every turbine needs no other turbines' information. Distributed control method was proposed in [26]; there are several controllers in local groups instead of the central controller, and some key information of shared turbines will be transmitted between them. Moreover, every turbine considers the power information of itself and the neighbors to set the control actions. Additionally, a cooperative optimization problem is designed in [27], the information of the whole wind farm output power is transmitted to every turbine, and the control variable is axial induction factors.

For example, an equivalent model for wind farm reduction was proposed in [28] to cluster the same-feature wind turbines into a group. However, for a complex wake, the same feature presented a difficult problem to find the k-means clustering algorithm [29,30], k-median clustering algorithm [31], and k-nearest physical neighbor [32], which were proposed to uncover the clustered index. Moreover, the disadvantage of [33,34] was the clustering index with one-dimensional data, which only focuses on wind speed without considering the wake effect. However, for a large-scale offshore wind farms, one-dimensional data cannot explain the wake relationship matrix of the wake effect, and two-dimensional data, considering the wind speed and the wake effect are more suitable. Therefore, based on a clustering index with two-dimensional data, another possible solution was to propose a novel clustering approach that can construct decouple communication architecture. In [35], the turbines were clustered into some groups through the singular value decomposition (SVD) clustering algorithm. However, those clustering algorithms were assumed to behave similarly and were only valid for limited wind directions without considering the differences in the incoming wind.

As mentioned previously, there is a lack of efficient cluster methods to cluster the subset of communication neighbors of the turbines considering complex wake effects. To solve this problem, in this paper, we propose an adaptive pruning wake digraph to divide the large-scale offshore wind farm (OWF) into decoupled groups, then cluster the neighbors into the same subset by setting the k-median of two turbines' wake imperfect weights as the clustering index. The wind farm control and the optimization problem in every subset are non-convex, so it is critical to devise an efficient optimization algorithm. By combining the wake model and the single objective function, there are some optimization methods proposed in the literature [26,36–39]: in [38], a gradient-based method on wind farm layout optimization is presented; however, a gradient-based optimization algorithm is based on an analytical wind farm function which is simplified, so it cannot precisely reflect the wind farm control. Therefore, without the wind farm function, only some necessary measurement data for data-driven optimization algorithms are proposed, for example, in [26,37]. However, the high model complexity of the measurable data will become significantly large. Moreover, the performance may not be good with noisy measurement data unavoidable in real applications. The authors of [36] explored a cooperative greedy algorithm optimizing energy production of wind farms; because of the wake effects, the downstream turbine could not generate much energy, so the total output power was not maximum.

Due to the large amount of large-scale wind farm data and nonlinear and non-convex characteristics, the above-mentioned control methods become unsuitable for obtaining the global optimum. Evolutionary algorithms (EAs) are one class of novel nature-inspired global optimization algorithms that are proposed in the literature [40,41]. For maximizing the total output power of wind farms, other algorithms such as modified grey wolf (GW) [42,43], particle swarm optimization (PSO) [44,45], and genetic algorithm (GA) [46] have been proposed. The authors of [44] present PSO intelligent algorithms for power conversion maximization using a nonlinear wake model. Importantly, no one algorithm can fit all systems. The above-mentioned algorithms are prone to premature convergence leading to a local optimum, not a global optimum, because of some unsuitable parameters. Therefore, we use a novel intelligent algorithm dubbed the beetle antennae search (BAS) [47] to improve calculation performance and searching ability to maximize wind farm power conversion. The BAS may generally converge early and fall into the optimal local solution for the unsuitable step size. In addition, the Monte Carlo (MC) method can be used to prevent the evolutionary algorithm from stagnating at a local optimum.

To summarize, in this paper, we first define an adaptive wake digraph to cluster a subset of communication neighbors of turbines. Secondly, we propose an MC-BAS optimization algorithm based on adaptive communication network topology to solve the non-convex power optimization problem. Finally, the yaw angles and axial factors are optimized to increase the power output by simulating a wind farm with 2 × 2 turbines, 3 × 3 turbines, and 5 × 5 turbines. The main contributions are summarized as follows:


This study is organized as follows: Section 2 introduces the Gaussian-based wake model considering yaw angle. Section 3 introduces the algorithm to cluster turbine via adaptive pruning wake digraph through setting the suitable global threshold ε**p**. In Section 4, the new MC-BAS control strategy in OWF is proposed to optimize the axial vale and yaw angle in every subset. Section 5 presents the simulated result of the proposed algorithm, the efficiency of power optimization and minimizing the calculating time is verified, and some important look-up tables are constructed. Finally, some important conclusions and summaries are presented in Section 6.

#### **2. Gaussian-Based Wake Model Considering Yaw Angle**

This section describes the wind turbine wake model for total output power optimization through axial induction and yaw angle control. The three-dimensional wind velocity deficit behind the upstream turbine *i* is defined as Gaussian shape, which was derived through Navier–Stokes equation [25]:

$$\frac{V(x,y,z)}{V\_{\infty}} = 1 - \mathcal{C}e^{-\left(y-\delta\right)^{2}/2\sigma\_{y}^{2}}e^{-\left(z-z\_{h}\right)^{2}/2\sigma\_{z}^{2}}\tag{1}$$

where *V* denotes the velocity in the wake, *V*<sup>∞</sup> denotes the free-stream inflow wind velocity of the wind farm, *x*, *y*, *z* is the direction of streamwise, horizontal spanwise, and vertical spanwise, a decoupled topology, *δ* is the wake centerline, *z<sup>h</sup>* is the hub height, σ*y*, σ*<sup>z</sup>* is the wake expansion in *y*, *z* direction, and *C* is the velocity deficit at the wake center. The main parameters are shown in Appendix A.

The relationship of *α*, and *γ*, can be defined as [21,25]:

$$\alpha \approx \frac{0.3\gamma}{\cos \gamma} \left( 1 - \sqrt{1 - \mathcal{C}\_T \cos \gamma} \right) \tag{2}$$

where *C<sup>T</sup>* denotes the thrust coefficient.

The relationship between the initial lateral deflection of wake deflection *δ*<sup>0</sup> as denied as in [25], *x*<sup>0</sup> is the length of the near wake as defined in [25], and the wake deflection angle *α* can be defined as the following equation [21]:

$$
\delta\_0 = \delta\_0 \tan \alpha \tag{3}
$$

$$\delta = \delta\_0 + \frac{\theta E\_0}{5.2} \sqrt{\frac{\sigma\_{y0} \sigma\_{z0}}{k\_y k\_z M\_0}} \ln \left[ \frac{(1.6 + \sqrt{M\_0}) \left(1.6 \sqrt{\frac{\sigma\_y \sigma\_z}{\sigma\_{y0} \sigma\_{z0}}} - \sqrt{M\_0} \right)}{(1.6 - \sqrt{M\_0}) \left(1.6 \sqrt{\frac{\sigma\_y \sigma\_z}{\sigma\_{y0} \sigma\_{z0}}} + \sqrt{M\_0} \right)} \right] \tag{4}$$

After describing the atmospheric wake model of the wind field, the turbine model used in this paper is introduced. The turbine model consists of a power coefficient *C<sup>p</sup>* and thrust coefficient *C<sup>T</sup>* which are all based on wind speed, tip speed ratio, and a blade pitch angle. In this paper, the *C<sup>p</sup>* and *C<sup>T</sup>* curves are used to form the fitting data of FAST and the National Renewable Energy Laboratory's (NREL's) 5 MW turbine [48].

To calculate the output power *P* of a wind turbine, the formula can be shown as follows [30]:

$$P\_{\dot{\jmath}}(a\_{\dot{\jmath}\prime}\gamma\_{\dot{\jmath}\prime}V\_{\dot{\jmath}}) = \frac{1}{2}\eta\rho A\_{\dot{\jmath}}\mathbb{C}\_{P}\cos\left(a\_{\dot{\jmath}\prime}\gamma\_{\dot{\jmath}}\right)^{1.88}V\_{\dot{\jmath}}^{3} \tag{5}$$

where *η* denotes generator efficiency; *ρ* is the air density; *A<sup>j</sup>* is the rotor swept area; cos *αj* , *γ<sup>j</sup>* 1.88 represents the correction factor of axial factors *<sup>α</sup><sup>j</sup>* and yaw misalignment angle *γ<sup>j</sup>* ; wind velocity *V<sup>j</sup>* can be calculated from Equation (1).

As depicted in Equations (1)–(5), the power conversion can be optimized by adjusting the axial factors *α<sup>j</sup>* and yaw angle *γ<sup>j</sup>* . The interested reader can read a more detailed description of wake deflection in [19].

#### **3. Clustering Turbine via Pruning Wake Digraph**

This section will partition the large-scale OWF into several decoupled subsets based on the weights calculated from the k-Median clustering algorithm [31].

The adaptive pruning wake digraph process can be summarized in Figure 1, including the wake farm model, original digraph generation, digraph pruning, and turbines clustering. The decision variable is the magnitude of the strength between turbines *wij* as shown in Figure 1c,e,g, then the large-scale OWF can be defined into several decoupled subsets. The section is focused on how to obtain adaptive wake digraphs of the large-scale OWF with wake interaction.

digraphs of the large-scale OWF with wake interaction.

turbines clustering. The decision variable is the magnitude of the strength between turbines as shown in Figure 1c,e,g, then the large-scale OWF can be defined into several decoupled subsets. The section is focused on how to obtain adaptive wake

**Figure 1.** The proposed whole decentralized control scheme on a 3 × 3 matrix OWF with ஶ = 8m/s, = 15: (**a**) Wind Farm; (**b**) wake field; (**c**) original wake digraph; (**d**) turbine original matrix; (**e**) pruned wake digraph with *k* = 3.2; (**f**) turbine pruned matrix with *k* = 3.2; (**g**) pruned wake digraph with *k* = 5.1; (**h**) turbine pruned matrix with *k* = 5.1. **Figure 1.** The proposed whole decentralized control scheme on a 3 × 3 matrix OWF with *V*<sup>∞</sup> = 8 m/s, *ϕ* = 15◦ : (**a**) Wind Farm; (**b**) wake field; (**c**) original wake digraph; (**d**) turbine original matrix; (**e**) pruned wake digraph with *k* = 3.2; (**f**) turbine pruned matrix with *k* = 3.2; (**g**) pruned wake digraph with *k* = 5.1; (**h**) turbine pruned matrix with *k* = 5.1.

#### *3.1. The Original Wake Graph of Wind Farm 3.1. The Original Wake Graph of Wind Farm*

where *wake* <sup>=</sup> ಮିೕ

ಮ

*Energies* **2021**, *14*, x FOR PEER REVIEW 6 of 24

An autonomous wind farm can be modeled as a weighted directed graph with wake impact from upstream turbines i to downstream turbines j [49]. Under this paradigm, the digraph is determined by the wind speed direction and wind farm layout. Assuming that there are N turbines in the OWF, we will explain how to construct the original wake digraph G in detail. An autonomous wind farm can be modeled as a weighted directed graph with wake impact from upstream turbines i to downstream turbines j [49]. Under this paradigm, the digraph is determined by the wind speed direction and wind farm layout. Assuming that there are N turbines in the OWF, we will explain how to construct the original wake digraph in detail.

To demonstrate the advanced approach, some definitions as follows are necessary: To demonstrate the advanced approach, some definitions as follows are necessary:

**Definition 1.** *The original wake digraph* G = (V, E) *where vertices* V*,* V = {*v<sup>i</sup> i* = 1, 2, . . . , *N*} *denote the turbines. Edges* E*,* E ⊂ V × V *represents the wake distribution between every pair of upstream turbine and downstream turbine.* **Definition 1.** *The original wake digraph*  = (, ℰ) *where vertices*  , = { ∣ = 1,2, … , } *denote the turbines. Edges* ℰ*,* ℰ ⊂ ×  *represents the wake distribution between every pair of upstream turbine and downstream turbine,* ℰ *can also be used the weight to indicate the strength of interactions between two nodes [30]:*

E can also be used the weight to indicate the strength of interactions between two nodes [30]: ℰ = ൛: , ∈ ൟ (6)

$$\mathcal{E} = \{w\_{ij} : i, j \in \mathcal{V}\} \tag{6}$$

(7)

where *wij* is a non-negative value. When the upstream turbine *v<sup>i</sup>* exerts a wake effect on the downstream turbine *v<sup>j</sup>* , the following equation can describe it: the downstream turbine , the following equation can describe it: = ቐ*overlap* , ∗ *wake*  / , shadowing,

$$w\_{ij} = \begin{cases} \frac{A\_{\text{overlap}\_{\
i,j}} \* V\_{\text{value}}}{\text{x}/D}, & \text{shadowing},\\ 0, & \text{no shadowing}. \end{cases} \tag{7}$$

where *Vwake* = *V*∞−*V<sup>j</sup> V*∞ , the wake overlap effect area *Aoverlap <sup>i</sup>*,*<sup>j</sup>* is described in Figure 2, *x* represents the physical distance between the upstream turbine *V<sup>i</sup>* and downstream turbine *Vj* ; *D* represents the turbine rotor diameter of all the turbines. It is critical to note that the wake distribution should remain constant during the control period so that the control speed is high enough to counteract the changing wake distribution. turbine ; represents the turbine rotor diameter of all the turbines. It is critical to note that the wake distribution should remain constant during the control period so that the control speed is high enough to counteract the changing wake distribution. • The communication neighbors of vertex (turbine) are denoted by = ቄ ∣ <sup>∣</sup> ቀ൫, ൯ ∈ ℰቁ ቅ. • The set of shared turbine *i* in communication neighbors between the subset


**Figure 2.** An example of a two-turbine wake redirection control through setting the yaw misalignment angle. is the upstream turbine 's yaw angle, is the deflection angle, and denotes the wake deflection. The black dashed lines represent the wake of the upstream turbine with no yaw control, and the red lines indicate the wake of the upstream turbine with yaw **Figure 2.** An example of a two-turbine wake redirection control through setting the yaw misalignment angle. *γ* is the upstream turbine *i*'s yaw angle, *α* is the deflection angle, and *δ* denotes the wake deflection. The black dashed lines represent the wake of the upstream turbine *i* with no yaw control, and the red lines indicate the wake of the upstream turbine *i* with yaw control. *A overlap i*→*j* represents the area overlap ratio that the wake effect area *A*(*Ti*|*Tj*) of the upstream turbine *i* to the downstream turbine j and the rotor area *A*(*Tj*) of downstream turbine *j*.

### *3.2. Decoupled Communication Scheme of Wind Farm Based on Adaptive Pruned Algorithm*

For most turbines, the coupling degree of one turbine with other turbines may be high or low. Therefore, the pruned wake digraph G*<sup>p</sup>* can be pruned from the original wake digraph G.

**Definition 2.** *The pruned wake digraph* G*<sup>p</sup>* = V, E*p*, *, pruning edges* E*<sup>p</sup> set* E*<sup>p</sup>* ⊂ V × V, E*<sup>p</sup>* = *<sup>w</sup>ijp* : *<sup>i</sup>*, *<sup>j</sup>* ∈ V *where*

$$w\_{ijp} = \begin{cases} w\_{ij} & w\_{ij} \ge \varepsilon\_{k\prime} \\ 0, & w\_{ij} < \varepsilon\_k \end{cases} \tag{8}$$


$$f(\mathbf{x}) \stackrel{\text{def}}{=} \sum\_{i=1}^{N} \sum\_{j=1}^{N} ||\mathbf{x} - \mathbf{W}\_{ij}||\_2 \tag{9}$$

where *εe argmin*(*f*(*x*)).

A decoupled topology in this section will be achieved by using the adaptive pruned wake digraph algorithm. However, determining the degree of pruning digraph has not been deeply studied, and there is little literature discussing it. This paper proposes an adaptive pruned algorithm to find a suitable threshold *ε<sup>k</sup>* to solve the grouped problem by obtaining more reasonable decoupled subsets.

Based on the pruned wake digraph G*<sup>p</sup>* = V, E*<sup>p</sup>* , for each angle *ϕ* ∈ {*ϕ* <sup>1</sup>, . . . , *ϕ <sup>w</sup>*}, there are clustering subsets *N<sup>l</sup>* , *l* ≥ 1. A given direction *ϕ* has a corresponding cluster subset *N<sup>i</sup>* ∈ {*N*1, . . . , *NM*}, *M* is the number of subsets. Then, according to the wind farm layout, we build the original wake digraph G and calculate the weight coefficient matrix *wij* to find the communication neighborhood of turbines. The algorithm for turbine clustering via adaptive pruned wake digraph is shown as follows (Algorithm 1):

**Algorithm 1:** The method of clustering turbine via pruning wake digraph (Adaptive pruned wake digraph algorithm)

**Step 1:** Based on the layout of the position of the wind farm (*X*, *Y*), collect all relevant parameters, including wind direction *Φ*, wind speed *V***∞**. Additionally, the parameters of the wind turbines, for example, the rotor diameter *D*, the physical distance *x* between WTs, and the overlap wake area *A* **overlap** *i*→*j* , etc.

**Step 2:** Calculate the threshold E, and set the initial hyper-parameter *k*, step hyper-parameter ∆*k*.

**Step 3:** Obtain the pruned digraph G*<sup>p</sup>* from the original wake digraph G according to the global threshold *ε<sup>k</sup>* according to the global threshold *εk*.

**Step 4:** Digraph clustering. Firstly, define the leading turbines for each subset that is experiencing free-stream velocity *V*∞.

Secondly, each leading turbine decides the communication neighbors through the connectivity information of the digraph G*<sup>p</sup>* by a depth-first tree search (BFS) algorithm into the same subset *N<sup>i</sup>* .

**Step 5:** If there is a set of shared turbines *S<sup>i</sup>* , we need to continue to tune the value of k by set *k* = *k* + ∆*k*, go back to Step 3. If not, go directly to step 6.

**Step 6:** Calculate the output power and calculating time with the *k* value from step 5, save the parameters.

**Step 7:** If the coefficients of G*<sup>p</sup>* are not all 0, continue to tune the *k* value by setting *k* = *k* + ∆*k*, go back to Step 2. If the coefficients of G*<sup>p</sup>* are all 0, go to step 8.

**Step 8:** Based on the adaptive pruned wake digraph G*p*, we can establish turbine clustering subsets *N<sup>i</sup>* and analyze all the parameters with different *k*, and select the suitable value *k*2.

> Overall, the adaptive pruned digraph and the contribution can be simplified as follows: Firstly: for one, considered wind speed and wind direction, to satisfy the pruned wake digraph are decoupled (no shared wind turbines between all the subsets), the range of *k* can be found out to be *k* ∈ [*k*1, *k*3], *k*<sup>1</sup> is the minimum value, and *k*<sup>3</sup> is the maximum value

of *k*. In most literature, the *k* is one random value between the range so that the result may be suboptimal.

Secondly: by comparing the performance of output power and calculating time, deduce a suitable value *k*2.

Thirdly: change the wind speed and direction by the above method and obtain another corresponding suitably *k*2.

Lastly: create a line-off query look-up table using the obtained result, which can be used for quick reference with input winds velocity *V*<sup>∞</sup> and wind direction *ϕ*, the output is *k*2.

The contributions are mainly to select suitable parameter *k*<sup>2</sup> with the proposed algorithm. Based on this condition, the control optimization can obtain the best result. The procedure will be demonstrated in Section 5 by one simulation example.

#### **4. Wind Farm Control Strategy**

Upon establishing the pruned wake digraph and clustering subsets, the overall OWF is controlled by multiple individual independents rather than by a single controller. The decentralized optimization process is described in this section. In order to achieve the control objectives of real-time output power optimization, a decentralized control scheme is proposed for the large-scale OWF, as shown in Figure 3. This control scheme is divided into two steps. First, wind farms are decoupled into several independent wind turbine clusters, and their communication neighbors are determined by the network topology of the adaptive pruned wake digraph. Therefore, a decentralized control strategy for OWF is proposed to realize the power control of the host computer. Second, we use the beetle antennae search (BAS) algorithm approach to optimize the yaw setting and the axial induction factor setting in the OWF to maximize the total wind plant power conversion. *Energies* **2021**, *14*, x FOR PEER REVIEW 9 of 24

**Figure 3.** The proposed whole decentralized control scheme on a 3 × 3 turbines offshore wind farm. **Figure 3.** The proposed whole decentralized control scheme on a 3 × 3 turbines offshore wind farm.

#### *4.1. The Output Power Optimization Problem 4.1. The Output Power Optimization Problem*

For a given decoupled clustering subsets N୧, the corresponding decentralized power control is based on the optimization objective: minimizing the power output of wind farms in Equation (4). Each wind turbine cluster is composed of several neighbor wind turbines that are decentralized on different communication network topologies of adaptive pruned wake digraphs. The proposed decentralized control scheme is shown in Figure 3. The corresponding decentralized power control for a given clustering subset is based on the following single-objective optimization problem. The control variables for For a given decoupled clustering subsets N<sup>i</sup> , the corresponding decentralized power control is based on the optimization objective: minimizing the power output of wind farms in Equation (4). Each wind turbine cluster is composed of several neighbor wind turbines that are decentralized on different communication network topologies of adaptive pruned wake digraphs. The proposed decentralized control scheme is shown in Figure 3. The corresponding decentralized power control for a given clustering subset *N<sup>i</sup>* is based on the following single-objective optimization problem. The control variables for the optimization

the optimization are the axial factor and yaw angle of the wind turbines. The whole

ୀଵ

where indicates subsets, j indicates turbines in subsets, M indicates the number of uncoupled subsets on the wind farm, and K indicates the number of turbines in every subset. The yaw angle , is between the upper bound ௫ and the lower bound −; α and α௫ represent the lower bound and upper bound range of axial factor , and the power , is between the minimum power *low* and the rated power *rate* .

The OWF cost function is given in Equation (9), which is a nonlinear and non-convex optimization problem. Many methods do not guarantee to find the optimal global solution. Herein, we adopt the BAS algorithm to solve optimization problems. Like genetic algorithms (GA), particle swarm optimization (PSO), and other intelligent algorithms, BAS does not require prior knowledge of the specific shape of the function or gradient information to optimize efficiency. The main two steps are the searching process and the result detection, tuning the adaptive step size until the optimization value is reached. The advantage of BAS is simple and fast speed to get the optimization object than other intelligence algorithms. Moreover, the high-speed advantage of BAS over the particle swarm optimization algorithm is that it requires only one individual, a longicorn

The Monte Carlo (MC) law of the annealing algorithm (SA) is introduced to improve the repeatability and stability of the algorithm. The improved algorithm is then applied to wake steering control so as to maximize power conversion on the wind farm. The

− ≤ , ≤ ௫ ≤ , ≤ ௫

ቀ ,൫,, ,; ,൯ቁ

(10)

OWF decentralized optimization function () can be expressed as follows:

ெ

ୀଵ

௫ () = − 

s. t.൜

*4.2. Monte Carlo Law with the BAS(MC-BAS) Controller for Wind Turbines* 

beetle.

are the axial factor *α* and yaw angle *γ* of the wind turbines. The whole OWF decentralized optimization function *f*(*x*) can be expressed as follows:

$$\begin{aligned} \min\_{\mathbf{x}} f(\mathbf{x}) &= -\sum\_{i=1}^{M} \sum\_{j=1}^{K} \left( P\_{i,j} (\gamma\_{i,j}, \alpha\_{i,j}; V\_{i,j}) \right) \\ \text{s.t.} & \begin{cases} -\gamma\_{\min} \le \gamma\_{i,j} \le \gamma\_{\max} \\ \alpha\_{\min} \le \alpha\_{i,j} \le \alpha\_{\max} \end{cases} \end{aligned} \tag{10}$$

where *i* indicates subsets, j indicates turbines in subsets, M indicates the number of uncoupled subsets on the wind farm, and K indicates the number of turbines in every subset. The yaw angle *γi*,*<sup>j</sup>* is between the upper bound *γmax* and the lower bound −*γmin*; *αmin* and *αmax* represent the lower bound and upper bound range of axial factor *αi*,*<sup>j</sup>* and the power *Pi*,*<sup>j</sup>* is between the minimum power *Plow* and the rated power *Prate* .

#### *4.2. Monte Carlo Law with the BAS(MC-BAS) Controller for Wind Turbines*

The OWF cost function is given in Equation (9), which is a nonlinear and non-convex optimization problem. Many methods do not guarantee to find the optimal global solution. Herein, we adopt the BAS algorithm to solve optimization problems. Like genetic algorithms (GA), particle swarm optimization (PSO), and other intelligent algorithms, BAS does not require prior knowledge of the specific shape of the function or gradient information to optimize efficiency. The main two steps are the searching process and the result detection, tuning the adaptive step size until the optimization value is reached. The advantage of BAS is simple and fast speed to get the optimization object than other intelligence algorithms. Moreover, the high-speed advantage of BAS over the particle swarm optimization algorithm is that it requires only one individual, a longicorn beetle.

The Monte Carlo (MC) law of the annealing algorithm (SA) is introduced to improve the repeatability and stability of the algorithm. The improved algorithm is then applied to wake steering control so as to maximize power conversion on the wind farm. The optimal target value of the object is determined by simulating the annealing process, and the lowest energy of the target and simulated annealing incorporates random variables during the search process. For example, it embraces a worse solution than the current solution with a certain probability, increasing the possibility of exiting local optimization. The Modified MC-BAS algorithm is shown below (Algorithm 2).

1. Random direction vector

To simulate the search behavior of longicorn, its direction vector is defined as [47]:

$$\overrightarrow{b} = \frac{\text{rand}(k, 1)}{\|\text{rand}(k, 1)\|} \tag{11}$$

where rand(*k*, 1) denotes a random function, and *k* represents position dimensions.

2. The coordinate of both right-hand and left-hand sides of the antennae of beetles are presented as [47]:

$$\begin{aligned} \mathbf{x}\_{l} &= \mathbf{x}^{t} + \boldsymbol{d}^{t} \stackrel{\rightarrow}{\mathbf{b}} \\ \mathbf{x}\_{l} &= \mathbf{x}^{t} - \boldsymbol{d}^{t} \stackrel{\rightarrow}{\mathbf{b}} \end{aligned} \tag{12}$$

where *t* represents the number of iterations; *x<sup>r</sup>* and *x<sup>l</sup>* denote the spatial position of the right and left beetles of longicorn beetles in the *t* iteration, respectively; *d t* represents the exploitability of antennae sensing length in the *t* iteration.

3. Fitness value:

$$\begin{cases} f\_{right} = f(\mathbf{x\_r}) \\ f\_{left} = f(\mathbf{x\_l}) \end{cases} \tag{13}$$

where *fright* and *f le f t* denote the fitness value of the right beetle and the left beard in the current spatial position; *f*(·) is the fitness function as Equation (9). during the search process. For example, it embraces a worse solution than the current solution with a certain probability, increasing the possibility of exiting local optimization. The Modified MC-BAS algorithm is shown below (Algorithm 2).

optimal target value of the object is determined by simulating the annealing process, and the lowest energy of the target and simulated annealing incorporates random variables

#### **Algorithm 2:** The grouped MC-BAS methodology for wind farm power production (MC-BAS Algorithm) **Algorithm 2**: The grouped MC-BAS methodology for wind farm power production (MC-BAS Algorithm)

*Energies* **2021**, *14*, x FOR PEER REVIEW 10 of 24

**Result:** The best yaw angles and the best axial factors *x bst <sup>n</sup>* and the best output power *f bst n* . **Input:** Establish output objective function *f x t n* , where variable *x t <sup>n</sup>* = - *x t* 1 , *x t* 2 , . . . , *x t N* and initialize the parameters *x* 0 , *d* 0 , *δ* 0 , *η<sup>d</sup>* , *η<sup>δ</sup>* , *MT*,*N*, *α*, *δ*criterion , *tmax*. **While** (*n* < *N*) **do Result:** The best yaw angles and the best axial factors ݔ ௦௧ ܽnd the best output power ݂ ௦௧ . **Input:** Establish output objective function ݂(ݔ ௧ ), where variable ݔ <sup>௧</sup> = [ݔ<sup>ଵ</sup> ௧ <sup>ଶ</sup>ݔ , ௧ ேݔ , ... , ௧ ] and initialize the parameters , ݀ , δ .௫ݐ, criterionߜ ,ߙ ்,ܰ,ܯ ,ఋߟ ,ௗߟ , **While** (݊ < ܰ) **do While** (ݐ > ݐ௫) or (ߜ > ߜcriterion )**do** Generate the direction vector ܾሬ⃗ via Equation (9); Search in variable space with two kinds of antennae ݔ and ݔ via Equation (12); Update fitness value ݂(ݔ(, ݂)ݔ (via Equation (13); Update the state variable ݔ <sup>௧</sup> via Equation (14); ݔ**)**݂ **if** <sup>௧</sup> **)** < ݂ ௦௧ then ݔ ݔ = ௦௧ ᇲ ௧ ݂ ݔ)݂ = ௦௧ ௧ ), else ݔ)݂)=delta <sup>௧</sup> )− ݂ ௦௧)/݂ ௦௧; ;(்ܯ / delta− (exp= ܮ if **r**and (1) <= ܮ,then ݔ ݔ)݈݁ܽ݉ݏ = ௦௧ ௧ ), ݂ ௦௧ = (݂൫ݔ௦ ௧ ൯), ்ܯ ∗ ߙ = ்ܯ End End Update sensing diameter ݀,step ߜ via Equation (16) End +1 ݐ = ݐ :ݐ Update Update ݊: ݊ = ݊ +1 End End

> 1. Random direction vector To simulate the search behavior of longicorn, its direction vector is defined as [47]: 4. Pre-update position:

$$\mathbf{x}^{t} = \mathbf{x}^{t-1} + \delta^{t} \overrightarrow{\mathbf{b}} \operatorname{sign} \left( f\_{\text{right}} - f\_{\text{left}} \right) \tag{14}$$

where rand (݇, 1) denotes a random function, and k represents position dimensions. 2. The coordinate of both right-hand and left-hand sides of the antennae of beetles are Pre-update the position of the beetles based on the iteration, and the *sign*(·) is a symbol function; *δ t* is the step size factor of the algorithm in the *t* iteration.

presented as [47]: = <sup>௧</sup> +݀ ௧ሬ⃗ = <sup>௧</sup> −݀ ௧ሬ⃗ (12) 5. Accepted solution using the Monte Carlo law The Monte Carlo law of the SA algorithm is embedding into BAS. In the iterative process, the probability *P* is used to accept the inferior solution to improve the global optimization ability of BAS:

$$L\_p = \begin{cases} 1, & f(\mathbf{x}^t) < f(\mathbf{x}^{t-1}) \\ \exp\left(-\frac{f(\mathbf{x}^t) - f(\mathbf{x}^{t-1})}{M\_T}\right), & f(\mathbf{x}^t) \ge f(\mathbf{x}^{t-1}) \end{cases} \tag{15}$$

where *f x t* denotes a pre-update position, *f x t*−1 denotes the best position in the last iteration; exp (.) represents the exponential function; *M<sup>T</sup>* is the higher temperature. 6. Step size:

$$\begin{cases} d^t = \eta\_d d^{t-1} + d^0\\ \delta^t = \eta\_\delta \delta^{t-1} + \delta^0 \end{cases} \tag{16}$$

where *d* and *δ* denote antennae length and step size, *d* <sup>0</sup> and *δ* 0 is the initial value, where *d <sup>t</sup>* and *δ t* is the step size factor of the algorithm in the *t* iteration, and the two parameters *η<sup>d</sup>* and *η<sup>δ</sup>* are set by the user.

The Monte Carlo law is mixed with the BAS algorithm. The basic steps of the grouped MC-BAS algorithm can be summarized as the pseudo-code shown in MC-BAS algorithm.

#### **5. Validation and Discussion**

Due to the randomness and intermittence, there are no constant wind speed and constant wind direction. For simplification, based on the probability of the known wind rose, we can calculate the average value of them in 10 min, then obtain an approximately constant value to describe them. In this experiment, assume that the average wind speed *V*<sup>∞</sup> = 8 m/s, the range of wind direction is *ϕ* = {0 ◦ , 15◦ , 30◦ , . . . , 180◦}, with which the baseline direction is the *x*-axis direction and under the assumption that it is constant within one control cycle.

The reason is that the wake is affected not much by the wind speed *V*<sup>∞</sup> but by the wind direction *ϕ*, so we only study the one wind speed *V*<sup>∞</sup> = 8 m/s, however, the whole wind direction is *ϕ* ∈ [0 ◦ , 360◦ ], with considering the symmetricity in the square wind farm, we only need to study the wind direction *ϕ* ∈ [0 ◦ , 180◦ ], and in this Simulink, we choose 5◦ as the step size in wind direction.

The wake digraph is the basic digraph from the wake field, as shown in Figure 4. The performance of proposed optimization approaches will be verified in this section by simulation results with the same NREL-5 MW Type III WT [48] turbine. The main parameters are given in Table A1 (Appendix A). The layout structure of OWF with lateral distance X = 500 m, longitudinal distance Y = 200 m, and rotor diameter D = 126 m, and nominal power P = 5 MW. The test is conducted for a 10-minute average of free wind speed and the direction range belongs to *ϕ* ∈ [0 ◦ , 180◦ ] at 15◦ increment. Furthermore, to verify the scalability and the feasibility of the proposed algorithm, in this paper, we study three different scales of wind farms with the different numbers of turbines N = 4, N = 9, and N = 25, respectively. The initial yaw angles *γ* are set to 0 with a range of *γ* ∈ [−30◦ , 30◦ ], and the initial factors are set to 1/3 with a range of *α* ∈ [0, 1/3]. It was observed that the numerical results showed that the proposed control method could reach an improved increase rate with a larger wind farm by comparing the result of 2 × 2, 3 × 3, 5 × 5 matrix turbine wind farms. In other words, verification of the proposed method means that it is suited for a large-scale wind farm.

### *5.1. Processing the Adaptive Pruning Wake Redirect Digraph*

In this section, the cluster method splits the large OWF into several independent subsets using the proposed pruned wake digraph clustering approach (see Section 3). The 5 × 5 wind farm location under different wind directions is shown in the proposed wake digraph. In this case, we consider a wind direction of *Φ* = 45◦ , 90◦ , the wake original digraph G, pruned digraph G*<sup>p</sup>* as illustrated in Figures 5 and 6, respectively. In addition, the decoupled communication topology comes from the suitably pruned wake digraph.

nominal power P = 5 MW. The test is conducted for a 10-minute average of free wind

to verify the scalability and the feasibility of the proposed algorithm, in this paper, we study three different scales of wind farms with the different numbers of turbines N = 4, N = 9, and N = 25, respectively. The initial yaw angles are set to 0 with a range of ∈

that the numerical results showed that the proposed control method could reach an improved increase rate with a larger wind farm by comparing the result of 2 × 2, 3 × 3, 5 × 5 matrix turbine wind farms. In other words, verification of the proposed method means

, 30∘], and the initial factors are set to 1/3 with a range of α ∈ [0,1/3]. It was observed

, 180∘] at 15 <sup>∘</sup>

increment. Furthermore,

speed and the direction range belongs to ∈ [0∘

that it is suited for a large-scale wind farm.

[−30∘

**Figure 4.** The original wake digraph under different wind turbines = 90: (**a**) 2 × 2 turbines wake field; (**b**) 2 × 2 turbines original wake digraph; (**c**) 3 × 3 turbines wake field; (**d**) 3 × 3 turbines original wake digraph; (**e**) 5 × 5 turbines wake field; (**f**) 5 × 5 turbines original wake digraph. **Figure 4.** The original wake digraph G under different wind turbines *ϕ* = 90◦ : (**a**) 2 × 2 turbines wake field; (**b**) 2 × 2 turbines original wake digraph; (**c**) 3 × 3 turbines wake field; (**d**) 3 × 3 turbines original wake digraph; (**e**) 5 × 5 turbines wake field; (**f**) 5 × 5 turbines original wake digraph. digraph , pruned digraph ୮ as illustrated in Figures 5 and 6, respectively. In addition, the decoupled communication topology comes from the suitably pruned wake digraph.

Figure 5 shows that the original wake digraph is the same as the pruned wake

As shown in Figure 6, the original wake digraph differs in the adaptive pruned wake digraph . The pruned wake digraph is different with varying . From Figure 6b to Figure 6c, the threshold =∗ε, ∈ [0.5,2.6], so the edges which are smaller than are cut off, such as the edges ℰଵଵ,ଶହ , etc. From Figure 6c to Figure 6d, the global threshold become bigger , with the range of ∈ [2.7,4.6], then the edges which are smaller than are cut off, for example, the edge ℰଵ,ଵହ, ℰଵ,ଶହ, ℰଶଶ,ଶହ, etc. Lastly, from Figure 6d to Figure 6e, get bigger at the range of ∈ [4.7,7.2], the more edges are cut off, for example, the edges ℰଵଷ,ଶ, ℰଵ଼,ଶହ, etc. and then the wake field is divided into 13

Figure 6 illustrates that the wake topology is parameter-dependent, as the external relevant variables (, ) influence the wake effect. For a given , the suitable is vital for pruning the wake digraph to obtain the optimization decoupled subsets. The method regarding tuning the hyperparameters will be discussed in the next step, which is

direction changes to = 45°, the power conversion of the OWF is increased since the

**Figure 5.** The wake digraph ( = 0.157481474) with wind direction = 90°: (**a**) wake field; (**b**) original wake digraph ( =0.1); (**c**) pruned wake digraph ( = 3.2). **Figure 5.** The wake digraph (*ε* = 0.157481474) with wind direction *ϕ* = 90◦ : (**a**) wake field; (**b**) original wake digraph (*k* = 0.1); (**c**) pruned wake digraph (*k* = 3.2).

influence of wake interaction becomes low.

subsets with no shared turbine.

shown in Tables 1 and 2 as follows.

*Energies* **2021**, *14*, x FOR PEER REVIEW 14 of 24

**Figure 6.** The process of adaptive pruning wake digraph method with varying for the wake digraph ( = 0.0122) with wind direction = 45: (**a**) wake field; (**b**–**e**) pruned wake digraph with different . **Figure 6.** The process of adaptive pruning wake digraph method with varying *k* for the wake digraph (*ε* = 0.0122) with wind direction *ϕ* = 45◦ : (**a**) wake field; (**b**–**e**) pruned wake digraph G*<sup>p</sup>* with different *k*.

**Table 1.** The relationship of subsets and shared turbines (ST) with different .  **No of Subsets With ST or Not**  0–0.4 9 Yes 0.5–2.6 13 Yes 2.7–4.6 13 No Figure 5 shows that the original wake digraph is the same as the pruned wake digraph. This is because when the wind direction is *ϕ* = 90◦ , the entire wake effect is concentrated on the downstream turbines without any diffused. However, when the wind direction changes to *ϕ* = 45◦ , the power conversion of the OWF is increased since the influence of wake interaction becomes low.

4.7–7.2 16 No 7.3 21 NO **Table 2.** Comparing the results of different pruning wakes under different . **Baseline Power Total Power (W) Groups T(s)** ∆**P**  2.7 3.23E+07 3.62 × 107 13 2.84 × 102 12.19% 2.8 3.23E+07 3.62 × 107 13 2.44 × 102 12.19% As shown in Figure 6, the original wake digraph G differs in the adaptive pruned wake digraph G*p*. The pruned wake digraph G*<sup>p</sup>* is different with varying *k*. From Figure 6b to Figure 6c, the threshold *ε <sup>p</sup>* = *k*∗ε, *k* ∈ [0.5, 2.6], so the edges which are smaller than *ε <sup>p</sup>* are cut off, such as the edges E11,25, etc. From Figure 6c to Figure 6d, the global threshold *ε <sup>p</sup>* become bigger *k*, with the range of *k* ∈ [2.7, 4.6], then the edges which are smaller than *ε <sup>p</sup>* are cut off, for example, the edge E1,15, E16,25, E22,25, etc. Lastly, from Figure 6d to Figure 6e, get bigger *k* at the range of *k* ∈ [4.7, 7.2], the more edges are cut off, for example, the edges E13,20, E18,25, etc. and then the wake field is divided into 13 subsets with no shared turbine.

3.3 3.23E+07 3.62 × 107 13 2.42 × 102 12.19% 3.8 3.23E+07 3.62 × 107 13 2.36 × 102 12.19% 4.3 3.23E+07 3.58 × 107 13 2.35 × 102 10.95% 4.8 3.23E+07 3.58 × 107 13 2.32 × 102 10.95% 5.3 3.23E+07 3.51 × 107 13 2.30 × 102 8.78% Figure 6 illustrates that the wake topology is parameter-dependent, as the external relevant variables (*ϕ*, *k*) influence the wake effect. For a given *ϕ*, the suitable *k* is vital for pruning the wake digraph to obtain the optimization decoupled subsets. The method regarding tuning the hyperparameters *k* will be discussed in the next step, which is shown in Tables 1 and 2 as follows.

5.7 3.23E+07 3.43 × 107 13 2.26 × 102 6.30% **Table 1.** The relationship of subsets and shared turbines (ST) with different *k*.


turbine. In other words, the subsets are all decoupled. Then, there is another problem of


**Table 2.** Comparing the results of different pruning wakes under different *k*.

Table 1 shows that there are shared turbines in the subsets depending on the value of *k*. In this paper, we focus on the range of 2.7 ≤ *k* ≤ 7.3 because of no shared wind turbine. In other words, the subsets are all decoupled. Then, there is another problem of how to set the suitable value *k*. In this paper, the proposed adaptive pruning algorithm can solve this problem. An adaptive threshold ε<sup>p</sup> can be proposed by comparing the output power and calculating time, and the comparison results are displayed in Table 2. It is essential to note that in Table 2, considering the objective of the real-time control, we choose the suitable value *k*<sup>2</sup> that focuses more on computational efficiency and an increased power rate by more than 4%. Moreover, when *k* is set as 5.3 as in Table 2, the control time is 226 s which is smaller than others. The high control speed is a very vital parameter during the control process. Therefore, we can find the suit *k*<sup>2</sup> = 5.3.

From Table 2, we chose *k* = 5.7 as a suitable value. The reason is that when *k* > 5.7, the output power is smaller than the baseline value, which is not permitted in this paper. Moreover, the calculation time is the smallest at the range of 0 ≤ *k* ≤ 5.7. The controller speed is important for the objection of real-time control. In this condition, the pruned wake digraph will be divided into 13 decoupled subsets, and the clustered turbines' neighbor turbines of every subset are N1 = {T1, T8, T15}, N2 = {T2, T9}, N3 = {T3, T10}, N4 = {T4}, N5 = {T5}, N6 = {T6, T13, T20}, N7 = {T7, T14}, N8 = {T11, T18, T25}, N9 = {T12, T19}, N10 = {T16, T23}, N11 = {T17, T24}, N12 = {T21}, N13 = {T22}.

Using the above-mentioned method, the range of wind direction extends to 0 ◦ ≤ *ϕ* ≤ 90◦ with an increment of 15◦ . Under different wind directions, to obtain decoupled communication topology by pruning the wake digraph, the experimental range of *k* is *k*<sup>1</sup> ≤ *k* ≤ *k*<sup>3</sup> and *k*<sup>2</sup> is the suitable value, which can be obtained from the proposed adaptive pruning algorithm.

In this way, wind speed keeps *V*<sup>∞</sup> = 8 m/s, we can also obtain the suitable value *k*<sup>2</sup> when wind direction *ϕ* varies from the range of [ 0 ◦ , 90◦ ] which is shown in Table 3 as follows. When wind speed *V*<sup>∞</sup> and wind direction *ϕ* changed, a look-up table of *k*<sup>2</sup> can be obtained by the proposed adaptive algorithm, which is shown in Appendix B—Table A2.

#### *5.2. The Combined Evaluation of the Decentralized MC-BAS Algorithm*

When *V*<sup>∞</sup> = 8 m/s, *ϕ* = 45◦ , and *k* = 5.7, as shown in Tables 1 and 2, the OWF wake digraph can be divided into 13 decoupled subsets. Taking the subset N2, for example, it concludes two neighbor turbines in cluster N2, the upstream turbine WT2 and the downstream turbine WT9. In this paper, the control actions and wake infection only work in the same subset. To maximize the output power of OWF, the yaw angles *γ* and the axial factors *α* are activated in an optimally decentralized manner. We will explain the sensitive relationship between the control actions *α*, *γ*, the output power *P*, and the consequent wind speed direction *ϕ* of the neighbor wind turbine in one subset as shown in Figure 7.

Taking subset N2, for example, in the range of wind direction *ϕ* ∈ [90◦ , 180◦ ], the upstream turbine WT2, the output power of MC-BAS is larger than that with the greedy method as shown in Figure 7b,c. Moreover, for the downstream WT9, the output power of MC-BAS is larger than that of the greedy method. However, the reason is not because of controlling the axial factors *α* and the yaw angle *γ* of WT2 but the decreasing wake effect of WT2, which is clearly shown in Figure 7e,f. For WT9 having no downstream turbines, the control parameters do not need to change the value significantly. This method is also applicable to other wind direction ranges and some other subsets. For brevity, we will not repeat the description in this paper.

When the wind direction is in the range of wind direction *ϕ* ∈ [20◦ , 50◦ ], WT2 and WT9 are in the same subset. In other words, WT2 and WT9 are neighbor turbines. Essentially, when wind direction *ϕ* changes significantly, the communication topology will also vary. The wind turbine will infect each other for the same subset, and for different subsets, the wind turbines are all independent. In this way, every wind turbine in all subsets with varying wind direction is optimized, allowing the total output power to reach the maximum value. The result is shown in Figure 8 and Table 4.


**Table 3.** The different adaptive *k* with the varying wind direction *ϕ*.

in Figure 7.

of WT9.

= 90

*k*1 = 0.6 2.5473 × 107 9.53% 174.9643 *k*2 = 16.5 2.4385 × 107 4.85% 136.9856 *k*3 = 27.3 2.1072 × 107 −9.39% 181.6532

Baseline 1.8731 × 107 0% 0.1133 *k*1 = 0.0 2.2795 × 107 21.70% 112.5742 s *k*2 = 31.6 2.1596 × 107 15.30% 98.7756 s *k*3 = 83.7 1.6765 × 107 −12.00% 117.329 s

*5.2. The Combined Evaluation of the Decentralized MC-BAS Algorithm* 

When ஶ = 8 m/s, = 45°, and = 5.7, as shown in Tables 1 and 2, the OWF wake digraph can be divided into 13 decoupled subsets. Taking the subset N2, for example, it concludes two neighbor turbines in cluster N2, the upstream turbine WT2 and the downstream turbine WT9. In this paper, the control actions and wake infection only work in the same subset. To maximize the output power of OWF, the yaw angles and the axial factors are activated in an optimally decentralized manner. We will explain the

consequent wind speed direction of the neighbor wind turbine in one subset as shown

**Figure 7.** Comparison of control parameters of subset N2 between MC-BAS, PSO, and Greedy control with different wind direction:(**a**) The output power of WT2; (**b**) The axial factors of WT2; (**c**) The yaw angle of WT2; (**d**) The output power of WT9; (**e**) The axial factors of WT9; (**f**) The yaw angle **Figure 7.** Comparison of control parameters of subset N2 between MC-BAS, PSO, and Greedy control with different wind direction: (**a**) The output power of WT2; (**b**) The axial factors of WT2; (**c**) The yaw angle of WT2; (**d**) The output power of WT9; (**e**) The axial factors of WT9; (**f**) The yaw angle of WT9. will also vary. The wind turbine will infect each other for the same subset, and for different subsets, the wind turbines are all independent. In this way, every wind turbine in all subsets with varying wind direction is optimized, allowing the total output power to reach the maximum value. The result is shown in Figure 8 and Table 4.

Figure 8 presents the results of the respective power output, while Table 4 shows the comparison of total power output. Generally, when the MC-BAS control method is used

The calculated time differs between the decentralized and centralized methods. The control speed of decentralized control is higher than the centralized control because there

wind direction = 45 can be obtained from Table 4. For example, one cluster subset N1 includes turbines T1, T8, and T15, symbolled as blue hexagon lines. Figure 8 shows that, with the greedy control algorithm, the lead wind turbine T1 produces the maximum power output, while the communication neighbors T8 and T15 only produce minimal power output without regulating the wake effect. Furthermore, the upstream wind turbine can cause significant wake disruption, reducing wind speed and lowering power conversion of the downstream wind turbine [17]. The wake effect is taken into account in the MC-BAS decentralized and centralized control scheme for optimizing overall power output. The majority of upstream turbines reduce output power, whereas downstream turbines increase power conversion, thereby increasing the entire power conversion. Furthermore, subsets N4, N5, N12, N13 have only one turbine, which is symbolized as a little red diamond on WT4, WT5, WT21, and WT22, and, respectively, the output power has no significant difference in the three different control methods since they are unconcerned about the downstream turbine. Other wind directions can be analyzed in the same way. However, the methodologies were not described in this paper to maintain

**Figure 8.** The respective power output of wind turbine: = 45. **Figure 8.** The respective power output of wind turbine: *ϕ* = 45◦ .

brevity.


**Table 4.** Comparison of power output and calculating time on OWF using centralized and decentralized algorithms on 5 × 5 matrix wind farm.

> Figure 8 presents the results of the respective power output, while Table 4 shows the comparison of total power output. Generally, when the MC-BAS control method is used to implement a centralized and decentralized approach, the total produced power increases compared to the greedy control method. The decoupled cluster subset with wind direction *ϕ* = 45◦ can be obtained from Table 4. For example, one cluster subset N1 includes turbines T1, T8, and T15, symbolled as blue hexagon lines. Figure 8 shows that, with the greedy control algorithm, the lead wind turbine T1 produces the maximum power output, while the communication neighbors T8 and T15 only produce minimal power output without regulating the wake effect. Furthermore, the upstream wind turbine can cause significant wake disruption, reducing wind speed and lowering power conversion of the downstream wind turbine [17]. The wake effect is taken into account in the MC-BAS decentralized and centralized control scheme for optimizing overall power output. The majority of upstream turbines reduce output power, whereas downstream turbines increase power conversion, thereby increasing the entire power conversion. Furthermore, subsets N4, N5, N12, N13 have only one turbine, which is symbolized as a little red diamond on WT4, WT5, WT21, and WT22, and, respectively, the output power has no significant difference in the three different control methods since they are unconcerned about the downstream turbine. Other wind directions can be analyzed in the same way. However, the methodologies were not described in this paper to maintain brevity.

> The calculated time differs between the decentralized and centralized methods. The control speed of decentralized control is higher than the centralized control because there are fewer turbines to solve, as shown in Table 4. The rate of power (∆P\_total) is the increased power at the baseline of *P*\_total of the greedy centralized algorithm. Table 4 shows that ∆P\_total increases at varying degrees in the decentralized MC-BAS control and centralized MC-BAS method under different wind directions. Moreover, the decentralized MC-BAS computation time (T\_total) is reduced to less than 1/3 times of the centralized approach. The mean total power generated by the centralized MC-BAS algorithm and decentralized MC-BAS algorithm improves by 14.4% and 11.3676%, respectively, compared to the baseline. This indicates about 3.0324% power loss in the decentralized MC-BAS compared to the centralized MC-BAS method. Thus, the proposed control strategy is practical for increasing power output and improving calculation speed from the perspective of real-time control and the profit of the large-scale OWF. For different wind directions,

we set the appropriate value *k*<sup>2</sup> so as to improve the calculation efficiency. Moreover, the importance of the adaptive pruned algorithm is also verified in Table 4.

#### *5.3. The Advantage of MC-BAS over Other Algorithms*

Generally, the higher the number of iterations, the more accurate the computation. This section aims to take the least number of iterations possible to reach the optimum control actions, resulting in improved total power conversion and communication burden.

According to Figure 9, the calculating time will increase significantly as the number of iterations increases by 100 to 300 in 20 increments, verifying the statements made in Section 3.1. The number of iterations plays a crucial role in reducing the calculation time, therefore reducing the communication burden. Consequently, the exact iteration value is a significant tuning value for optimization algorithms. The MC-BAS algorithm takes far less calculating time than the other three control algorithms and is about 1/9 time of the PSO method and 1/4 time of the GA method. Thus, the proposed centralized MC-BAS method outperforms other intelligent methods (GA and PSO) in terms of calculating speed.

Figure 10 shows the power conversion depending on wind directions and iterations with four different control algorithms. The PSO method can produce more power than others in most iterations. However, when the number of iterations exceeds 140, the total production in the PSO algorithm is equal to that of the MC-BAS algorithm. Consequently, we set the number of iterations to 140 to obtain better total power conversions with the proposed MC-BAS. The main drawback of the GA algorithm is the unstable output power, which varies at different iterations, as shown in Figure 10b,d. Therefore, the GA algorithm is not a suitable choice for OWF. The convergence of the algorithm can be measured by error variation and the number of iterations. In this paper, in order to test the influence of the number of iterations on the results, the number of iterations is used as the condition for the end of the simulation. *Energies* **2021**, *14*, x FOR PEER REVIEW 19 of 24

**Figure 9.** The computational time versus iteration of different algorithms in 3 × 3 matrix OWF: (**a**) = 0; (**b**) = 25; (**c**) = 45; (**d**) = 90. **Figure 9.** The computational time versus iteration of different algorithms in 3 × 3 matrix OWF: (**a**) *ϕ* = 0 ◦ ; (**b**) *ϕ* = 25◦ ; (**c**) *ϕ* = 45◦ ; (**d**) *ϕ* = 90◦ .

Figure 10 shows the power conversion depending on wind directions and iterations with four different control algorithms. The PSO method can produce more power than others in most iterations. However, when the number of iterations exceeds 140, the total

we set the number of iterations to 140 to obtain better total power conversions with the proposed MC-BAS. The main drawback of the GA algorithm is the unstable output power, which varies at different iterations, as shown in Figure 10b,d. Therefore, the GA algorithm is not a suitable choice for OWF. The convergence of the algorithm can be measured by error variation and the number of iterations. In this paper, in order to test the influence of the number of iterations on the results, the number of iterations is used as the condition

for the end of the simulation.

turbines.

*Energies* **2021**, *14*, x FOR PEER REVIEW 20 of 24

*Energies* **2021**, *14*, x FOR PEER REVIEW 20 of 24

**Figure 10.** The total power output of 3 × 3 matrix OWF on MC-BAS algorithm compared to the iteration algorithms: (**a**) = 0; (**b**) = 25; (**c**) = 45; (**d**) = 90. **Figure 10.** The total power output of 3 × 3 matrix OWF on MC-BAS algorithm compared to the iteration algorithms: (**a**) *ϕ* = 0 ◦ ; (**b**) *ϕ* = 25◦ ; (**c**) *ϕ* = 45◦ ; (**d**) *ϕ* = 90◦ . **Figure 10.** The total power output of 3 × 3 matrix OWF on MC-BAS algorithm compared to the iteration algorithms: (**a**) = 0; (**b**) = 25; (**c**) = 45; (**d**) = 90.

Moreover, we demonstrate the effectiveness and scalability of the proposed methodology. The range of wind direction is ∈ [0, 10, … ,180] and the number of turbines is a 2 × 2 matrix and 3 × 3 matrix as shown in Figure 11. Moreover, we demonstrate the effectiveness and scalability of the proposed methodology. The range of wind direction is *ϕ* ∈ [0 ◦ , 10◦ , . . . , 180◦ ] and the number of turbines is a 2 × 2 matrix and 3 × 3 matrix as shown in Figure 11. Moreover, we demonstrate the effectiveness and scalability of the proposed methodology. The range of wind direction is ∈ [0, 10, … ,180] and the number of turbines is a 2 × 2 matrix and 3 × 3 matrix as shown in Figure 11.

that of PSO, which is better than the value produced by the greedy algorithm in most

(**c**) the total power conversion with 3 × 3 matrix turbines; (**d**) the calculating time with 3 × 3 matrix turbines. Figure 11a,c shows that the total power conversion of MC-BAS is exactly the same as **Figure 11.** Comparison of the centralized MC-BAS algorithms with other iteration methods: (**a**) the total power conversion with 2 × 2 matrix turbines; (**b**) the calculating time with 2 × 2 matrix turbines; (**c**) the total power conversion with 3 × 3 matrix turbines; (**d**) the calculating time with 3 × 3 matrix **Figure 11.** Comparison of the centralized MC-BAS algorithms with other iteration methods: (**a**) the total power conversion with 2 × 2 matrix turbines; (**b**) the calculating time with 2 × 2 matrix turbines; (**c**) the total power conversion with 3 × 3 matrix turbines; (**d**) the calculating time with 3 × 3 matrix turbines.

Figure 11a,c shows that the total power conversion of MC-BAS is exactly the same as that of PSO, which is better than the value produced by the greedy algorithm in most wind directions. In addition, the MC-BAS algorithm takes less calculation time than the PSO algorithm on the 2 × 2 matrix and 3 × 3 matrix wind farms (Figure 11b,d). Notably, the ordinate scale of Figure 11c is 10 times that of Figure 11a,d and is 10 times that of Figure 11b. A more detailed analysis with the increasing output is displayed in Tables 5 and 6.

**Table 5.** The total power conversion improvement rate compared to the baseline with the 3 × 3 matrix OWF.


**Table 6.** The total power conversion improvement rate compared to the baseline in 2 × 2 matrix WF.


Tables 5 and 6 show the total production improvement rate is relative to the baseline of greedy control with 2 × 2 matrix turbines and 3 × 3 matrix turbines. It can be observed in some wind directions that the increased power rate is zero, such as *ϕ* = 20◦ , 25◦ , etc., in Table 6 and *ϕ* = 135◦ in Table 5. The advantage of the proposed MC-BAS algorithm was verified by comparing it with several other algorithms, especially in increasing the output power and decreasing the calculating time. Additionally, in a large-scale OWF, an adaptive pruned wake digraph is proposed to divide it into several decoupled subsets. Then, the same controller works on every subset to ensure real-time control. By analyzing the data in Tables 3–6, we can conclude that the ∆P with the MC-BAS algorithm increases with a greater number of wind turbines on the wind farm. For example, when *ϕ* = 15◦ , ∆P = 2%, 4%, 12.73% on the 2 × 2, 3 × 3, and 5 × 5 matrix wind farms, respectively. We anticipate that the proposed algorithm will demonstrate a good performance of large-scale wind farms.

#### **6. Conclusions**

This paper proposed a decentralized real-time power optimization for large-scale OWFs using an adaptive pruned wake digraph approach. The results of this paper can be summarized as follows:


Future work will focus on increasing the control parameters and control objectives of the large-scale OWF, considering the infection of nonlinear turbulent flow [50,51]. Moreover, optimizing the wind farm layout with irregularly shaped wind farms will be studied by decreasing the wake effect.

**Author Contributions:** Conceptualization, Y.-H.J.; methodology, Y.C.; software, Y.C.; validation, Y.-H.J.; investigation, D.S.; data curation, Y.C.; writing—original draft preparation, Y.C.; writing review and editing, D.S.; visualization, D.S.; supervision, Y.-H.J. All authors have read and agreed to the published version of the manuscript.

**Funding:** This work was partially supported by the Basic Science Research Program through the National Research Foundation of Korea (NRF), funded by the Ministry of Education (NRF-2016R1A6A1A03013567, NRF-2021R1A2B5B01001484) and the Innovation-Driven Project of Central South University (2020CX031).

**Institutional Review Board Statement:** Not applicable.

**Conflicts of Interest:** The authors declare no conflict of interest.

### **Appendix A**

**Table A1.** The main parameters of the 5 MW wind turbine for an offshore wind farm.


#### **Appendix B**


**Table A2.** The look-up table of different *k*<sup>2</sup> with the varying wind.

#### **References**


## *Article* **Home Energy Management Considering Renewable Resources, Energy Storage, and an Electric Vehicle as a Backup**

**Oussama Ouramdane, Elhoussin Elbouchikhi \*, Yassine Amirat, Franck Le Gall and Ehsan Sedgh Gooya**

ISEN Yncréa Ouest Brest, LABISEN, 20, Rue Cuirassé Bretagne, 29200 Brest, France; oussama.ouramdane@isen-ouest.yncrea.fr (O.O.); yassine.amirat@isen-ouest.yncrea.fr (Y.A.); franck.le-gall@isen-ouest.yncrea.fr (F.L.G.); ehsan.sedgh-gooya@isen-ouest.yncrea.fr (E.S.G.) **\*** Correspondence: elhoussin.elbouchikhi@isen-ouest.yncrea.fr; Tel.: +33-06-44-25-24-08

**Abstract:** The vehicle-to-grid concept emerged very quickly after the integration of renewable energy resources because of their intermittency and to support the grid during on-peak periods, consequently preventing congestion and any subsequent grid instability. Renewable energies offer a large source of clean energy, but they are not controllable, as they depend on weather conditions. This problem is solved by adding energy storage elements, implementing a demand response through shiftable loads, and the vehicle-to-grid/vehicle-to-home technologies. Indeed, an electric vehicle is equipped with a high-capacity battery, which can be used to store a certain amount of energy and give it back again later when required to fulfill the electricity demand and prevent an energy shortage when the main-grid power is limited for security reasons. In this context, this paper presents a comparative study between two home microgrids, in one of which the concept of vehicle-to-home is integrated to provide a case study to demonstrate the interest of this technology at the home level. The considered microgrid is composed of renewable energy resources, battery energy storage, and is connected to the main grid. As the vehicle is not available all day, in order to have consistent results, its intervention is considered in the evening, night, and early morning hours. Two case studies are carried out. In the first one, the vehicle-to-home concept is not taken into account. In this case, the system depends only on renewable resources and the energy storage system. Subsequently, the electric vehicle is considered as an additional energy storage device over a few hours. Electric vehicle integration brings an economic contribution by reducing the cost, supporting the other MG components, and relieving the main grid. Simulation results using real weather data for two cities in France, namely Brest and Toulon, show the effectiveness of the vehicle-to-home concept in terms of cost, energy self-sufficiency, and continuity of electrical service.

**Keywords:** vehicle-to-grid; vehicle-to-home; microgrids; distributed energy generation; energy storage; optimal sizing; energy management systems; optimization; energy dispatching

## **1. Introduction**

Vehicle-to-grid (*V*2*G*) could be considered as the biggest technological advancement since renewable energy resources (RER) became commercially viable [1]. With more than 3 million EVs worldwide [2], these EVs are used only about 5% of the time; the rest of the time they are parked when the owners are working or at home [3]. EVs can be used for a secondary role during the 95% of available time. Indeed, electric vehicles (EV) with a large battery can recover energy produced by RER, which would otherwise be lost. In this case, the EV functions as a secondary storage system. In the opposite case, where production is low, the storage systems and the main grid intervene by sending the missing energy, to achieve the production consumption balance. This demand for energy can overload the grid and cause blackouts or voltage and frequency drops. To relieve the grid, the energy it supplies can be limited to a certain amount, the rest being supplied by the EV as it discharges. In some cases, for same precedents reasons, it can be possible to sell the energy

**Citation:** Ouramdane, O.; Elbouchikhi, E.; Amirat, Y.; Le Gall, F.; Sedgh Gooya, E. Home Energy Management Considering Renewable Resources, Energy Storage, and an Electric Vehicle as a Backup. *Energies* **2022**, *15*, 2830. https://doi.org/ 10.3390/en15082830

Academic Editor: Angelo Zarrella

Received: 14 February 2022 Accepted: 4 April 2022 Published: 13 April 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

stored in the EV to the grid. In this case, the EV functions as a secondary energy source; hence, the concept of *V*2*G* [4,5].

The microgrid (MG) concept has emerged for the optimal integration of renewable energy resources, energy storage systems (ESS), and shedding loads, which may include EVs. The integration of EVs must not change the roles of the MG elements, both in gridconnected or islanded modes, i.e., to supply a given load at a given time. RER must be exploited to the maximum, by implementing maximum power point tracking (MPPT) and excess energy is stored in ESS. In the *V*2*G* concept, EVs represent a secondary production and storage system in the MG. One of the advantages of bi-directional energy in the *V*2*G* concept is the exchange of energy between EVs [6]. However, it is mandatory to consider the degradation of the EVs batteries while implementing *V*2*G* technology. Optimal MG sizing is one of the studies to be conducted for *V*2*G* concept integration. It involves many aspects that need to be addressed, including the appropriate RER technologies with the optimal siting and the environment considered. Moreover, other considerations should be taken into account, such as improving conversion efficiency at charging stations, transmission efficiency and loss minimization, optimal control of operation, the need for adequate loads that better match the generation elements, and, finally, the type, capacity, and number of EVs needed for safe and reliable operation [7]. As a result, interest in EV technologies has increased considerably in recent years, resulting in several scientific publications on the subject [8,9]. Moreover, the market share of EVs is expected to grow exponentially. Most of the research deals with very important topics such as energy efficiency [10], EV charger topologies and the impact of charging on the network [11], communication architecture [12], degradation of the vehicle battery [13], frequency regulation [14], and the support provided to the integration of RERs [15], mode of payment [16], etc.

Several technical aspects have been investigated in the literature. Indeed, an EV can be considered as a storage medium for RERs depending on the availability and states of charge of the EVs. In [17], Hasan and Elyas aim to optimise the operation of EVs in order to mitigate the intermittency of RERs and to reduce energy costs by minimising the charge/discharge cycles of vehicle batteries to avoid their degradation. In [18], Abdul et al. proposed a new formulation for the design and management of grid-connected EV charging stations with integrated solar panels. In [19], Mehdi et al. examined the value of the *V*2*H* concept at the building level, and whether it can operate in an islanded mode by introducing PV as an energy source to relieve the grid. One of the cited advantages of *V*2*H* is the support it provides to the grid. In fact, an EV parked at home during peak hours can power appliances. In [20], Philipp et al. assessed the ability of the expanded German transmission network to cope with the additional demand due to the integration of electric vehicles using a transmission problem formulation. In [21], authors presented an EMS of an energy system composed of main grid, load, and a fleet of EVs. The scheduling of EV charging/discharging can be done 24 h in advance, which allows for voltage and frequency regulation in the grid. The EV can sometimes be used as a back-up when the house is disconnected from the grid or when a blackout has occurred. Critical loads will be connected directly to the vehicle, to ensure the continuity of supply for the user's comfort. In [22], Vítor et al. designed a *V*2*H*, such that the EV intervenes during power outages. In [23], Hasan presented a model that significantly improves the energy resilience of a building based on PV, a battery for storage, and an EV for *V*2*H* operating in connected and islanded mode over a few hours. The coordination of the EV and the battery allows one to respond to the load after failures in the grid and PV. Furthermore, an EV can be added to increase battery lifetime. Since the vehicle's storage capacity is greater than the battery's, the vehicle will experience fewer charge/discharge cycles, and, therefore, reduce the charge/discharge cycles of the battery without degrading its battery too much [24]. Finally, an EV can allow for an efficient management of load shedding [25].

The economic perspective has been investigated in some studies. In [26], Sangyoon and Dae-Hyun presented a hierarchical deep reinforcement learning method for energy management of household appliances considering RER, ESS, and EVs. They consider

four study cases: two during the week with different weather data, and two during the weekend, with different vehicle states of charge. They find that the energy cost decreases significantly in the cases where the RERs are productive. The presence of the vehicle at home during the weekend is very cost effective due to its high storage capacity. In [27], Harun and Seddik developed seven optimal strategies studied on 1000 use cases and with four different daily energy price profiles in order to reduce the vehicle charging cost. A *V*2*G* algorithm named Optimal Logic Control based on series of logic commands has been developed. The results show the advantage of this algorithm over the traditional optimal charging strategies available in the literature. The algorithm is mainly made to be very efficient when the selling price/purchase ratio is greater than one, allowing the user to earn money. In [28], Sausen et al. proposed a model for coupling an EV to a smart home containing PV and batteries, having the objective of achieving cost minimization that was conditional on maximizing vehicle efficiency. Four scenarios are considered: in the first one, uncontrolled EV charging was adopted; in the second one, economic charging was applied; in the third one, economic EV charging was adopted, including distributed generation; and in the fourth one, economic EV charging was considered, with distributed generation and *V*2*H* operation. The results show that the peak consumption has been reduced thanks to the intervention of the EV and the bill has been decreased. In [29], Xuan et al. proposed a strategy to extend the lifetime of the batteries, such that the batteries operate when there is a surplus of RER production and when electricity prices increase during peak hours. The results show the effectiveness of this strategy, as when the battery discharge cost is higher than the market energy price, no power is supplied by the batteries, and conversely, when the discharge cost is lower than the market energy price, the power demanded is first supplied by the batteries, then, if the load is very high, the grid is allowed to supply the remaining load.

There is a lot of literature on *V*2*G*, but the *V*2*H* concept is not thoroughly addressed, as most research studies focus on the interaction of electric vehicles with the electric grid. Additional research is needed to address the cost of implementing the *V*2*H* concept and its profitability. In this context, this paper presents a sizing and optimization study of a MG operating in a grid-connected mode at the home level. The MG is composed of PVs and WT as RER, a battery as ESS, and a domestic load. The EV is used as a secondary energy storage device, which allows for the enhancement of the overall MG performance. This study concerns two aspects that are related to MG optimal sizing and smart day-ahead energy management. Two case studies are considered in this paper: a first grid-connected MG with only RERS and ESS is investigated; next, a second grid-connected MG with the integration of RERs, ESS, and an EV is implemented and the results are compared. An economic study is performed to investigate the advantages offered by the use of the EV and the interest of *V*2*H* technology in terms of energy supply reliability, cost reduction, and grid support during congestion periods. Simulation results are presented using actual weather data from two cities, which are Brest (cold climate) and Toulon (hot climate) in France. Generally speaking, the contributions targeted in this work are threefold:


This article is organised as follows: Section 2 presents the proposed MG architecture, presenting the used renewable energy resources and energy storage devices. Section 3 describes the MG optimal sizing and energy management considering two aspects, which are optimal energy dispatching and economic viability. Section 4 discusses the simulation results for the two considered case studies for two different cities in France. Finally, Section 5 concludes this paper and gives some perspectives for future work.

#### **2. Microgrid Architecture**

A home microgrid, which implements the *V*2*H* concept, is designed for the relief of main grid constraints. The MG is operated in grid-connected mode and consists of PV, WT, battery energy storage, and an EV. The architecture of the MG is presented in Figure 1. All components are linked to the DC bus through *DC*/*DC* converters (PV, batteries, and EV) or *AC*/*DC* converters (WT and grid). The PV and WT produce a quantity of energy depending on weather conditions, which does not always match the load demand. There are two possible scenarios: energy production is insufficient or, conversely, there is surplus energy generation. Energy storage and the main grid allow for the buffering of insufficient or excess power generation. Indeed, the battery and grid can restore the production/consumption balance by supplying the lack of energy or, conversely, by absorbing excess energy. Since the energy absorbed/fed by the battery is limited by its SOC and its maximum power, and since grid power can also limited, an EV can be used as an additional backup energy storage. Indeed, an EV can be charged through public charging stations or when parked at home. In order to support the grid and ensure the continuity of electricity supply, it can have a bidirectional energy exchange with the home MG.

**Figure 1.** Structure of the home MG considering an EV as backup energy storage.

#### *2.1. Home MG Components*

The choice of MG components is based on the maturity of the distributed energy resources (DER) and the efficiency of the ESS. Currently, the most mature RER is PV, which is the most mastered and used by researchers in projects aiming to achieve autonomous systems, relieve the grid, or produce clean energy. The PV used in this project is monocrystalline panels, which are suitable for cold climates, while polycrystalline panels are more suitable for warm climates [30]. A domestic wind turbine is the second mature renewable technology, known as an individual wind turbine [30]. WT is mainly installed at small-scale residential customers. It can also be used to power larger loads, such as small businesses and agricultural fields. The interest of this installation is to satisfy a part of the load, reducing the dependence on the power grid. Two classes of these wind turbines can be distinguished: vertical-axis wind turbines and horizontal-axis wind turbines. In the considered MG, the installed wind turbine is a horizontal-axis configuration with three blades, a drive train consisting of a gearbox and a generator, and a tower to support the rotor.

The considered battery in this study is a lithium-ion battery. It is lighter than a leadacid battery (five times lighter), has a long lifespan (1500 cycles), and is more resistant to deep discharges. New trends in the use of batteries for energy systems focus on integration with multiple RERs (PV, WT, etc.) and also on the integration with other energy storage

systems that complement them [31]. This trend fits perfectly with the proposed MG, which includes PV and WT as well as an EV as an additional energy storage device.

The installed PV and WT maximum power and battery capacity are unknown and are computed by the optimisation program based on the load data, weather data, grid energy cost, renewables and battery investment cost, battery power limits, grid power limits, and battery capacity of the EV. Subsequently, an optimal energy dispatching is computed, which determines the power flows between all components within the battery power and capacity limits, grid limits, RER availability, and EV displacement, considering the economic aspects.

#### *2.2. Modeling of the Renewable Resources and Energy Storage Devices*

PV panels and wind turbines are considered as energy resources in this study. The following subsections describe these renewable resources modeling for optimal sizing and smart energy management.

#### 2.2.1. PV Panels

The power generated by the PV panels depends on the installed maximum power and weather conditions. The PV output power depends on the irradiance, the efficiency of the generation, the area of the panels and the optimal orientation depending on the location. The chosen PV technology has an efficiency *ηpv* = 15%. The power produced by PV over a day is given as follows:

$$P\_{PV}(t) = \eta\_{PV} \cdot \mathbf{S}\_{PV} \cdot P\_{Irr}(t); \quad 1 < t < 24 \text{ h} \tag{1}$$

where:


#### 2.2.2. Wind Turbine

A wind turbine has an output power that mainly depends on its radius and the wind speed in the considered area. The other variables are constants, such as air density (*ρ* = 1.225 kg/m<sup>3</sup> ), or can be considered as constants by setting them to a value given by the control algorithm by implementing maximum power point algorithm (MPPT), such as the power coefficient (*C<sup>p</sup>* = 0.4). The electrical power extracted by a wind turbine is as follows:

$$P\_{\rm WT}(t) = \frac{1}{2} \cdot \rho \cdot \mathbb{C}\_p \cdot \pi \cdot \mathbb{R}\_{\rm WT}^2 \cdot V\_{\rm wind}(t)^3; \quad 1 < t < 24 \text{ h} \tag{2}$$

where:


#### 2.2.3. Energy Storage Devices: Battery and EV

The energy supplied by the battery and the EV depend on their capacity and their state of charge (SOC). Moreover, the output power is limited to enhance the lifespan of energy storage devices. The SOC represents the ratio between the energy contained in the battery (respectively, EV battery) and its maximum capacity (respectively, EV battery capacity). The SOC of the battery (respectively, EV battery) depends on the power supplied or recovered by the battery (respectively, EV battery), the time of application of this power (*δt* = 1 h), and its maximum capacity (respectively, EV battery capacity). The SOC varies from one hour to another and can increase during charging and decrease during discharging. The SOC for both the battery and the EV battery is given by the following formula:

$$\text{SOC}\_{bat}(t) = \text{SOC}\_{bat}(t-1) - \frac{P\_{bat}(t) \cdot dt}{E\_{bat}^{max}}; \quad 1 < t < 24 \text{ h} \tag{3}$$

$$\text{SOC}\_{EV}(t) = \text{SOC}\_{EV}(t-1) - \frac{P\_{EV}(t) \cdot dt}{E\_{EV}^{\text{max}}}; \quad 1 < t < 24 \text{ h} \tag{4}$$

where:


#### **3. Optimal Sizing and Planning of a Home MG with** *V***2***H* **Technology**

A domestic MG with *V*2*H* concept optimization is carried out in two steps, which are the optimal sizing and the energy management system (EMS), as shown in Figure 2. MG sizing is performed based on the weather data, the market energy cost, and the investment cost, all while fulfilling the technical constraints. Next, an optimal energy management algorithm is proposed considering that the home MG is optimally designed. In this section, the objective of the proposed energy management system is presented, which is mainly related to the sizing and the minimisation of the energy cost over 20 years of operation of the MG while integrating the *V*2*H* technology.

**Figure 2.** Scheme describing the proposed MG sizing and energy management system.

#### *3.1. Energy Management Strategy*

The objective in this work is to achieve a *V*2*H* system with maximum autonomy and minimum energy cost. Figure 3 shows the scheme of the proposed energy management algorithm. Five different layers can be distinguished in this energy management strategy. Indeed, in order to ensure maximum autonomy and favour the use of renewable energy, the load has to be fed by RERs at any time. On-maximum power point tracking (On-MPPT) mode is prioritized. Depending on the value of net power *Pnet* = *PLoad* − (*PPV* + *PWT*), there are three possible scenarios as follows:


• *Pnet* < 0: Battery free capacity is used to store the overproduction. If the battery is fully charged or the required power is higher than the maximum power, the excess power is supplied to the grid. However, if the EV is available or the grid power is not within its power limitations, the EV battery is used as a backup energy supply.

**Figure 3.** Energy management strategy (*Pnet* is the net power, *Pres* stands for residual power, and *Pback* corresponds to backup power).

In the renewables underproduction situation, the first element that will intervene is the battery. This step is shown in purple on the algorithm diagram. On one hand, if the battery is sufficiently charged and the power demand does not exceed the maximum discharge power specified by the manufacturer, the battery provides the power required to ensure the generation/consumption power balance. On the other hand, if the discharge power exceeds the maximum acceptable battery discharge power, battery provides its maximum power while the remaining power is provided by the grid, as shown by the yellow stage in the EMS scheme. Even though the grid can be considered as a large energy resource, its power can be limited for security and reliability reasons (to prevent overload, for example). If the power demand does not exceed grid power limitations, the load is supplied by the grid. Otherwise, the EV is used as a backup to ensure the electricity continuity of service, as shown in the pink section of the algorithm. A final situation can occur for which the EV maximum power is exceeded or the EV state of charge does not allow the use of the EV battery. In this case, a load shedding is required as depicted by the grey part of the EMS scheme.

Conversely, in an overproduction situation, the battery is first used to store the excess energy generation, as shown by the purple section of the algorithm. If the battery is fully charged or the required power exceeds the maximum charge power given by the manufacturer, the main grid intervenes to guarantee the generation/consumption balance (yellow part of the diagram). As previously specified, the grid power is limited to enhance the system reliability and improve autonomy, which is the reason why the EV is used to store a part of the produced energy if the grid maximum power is achieved, as described in the pink part of the energy management strategy. Finally, if the EV is not available or the EV battery is fully charged, renewables are operated in off-MPPT mode in order to reduce energy production (grey part).

#### *3.2. Objective Function*

The objective of this work is to design an optimal home MG, which minimizes the investment and energy costs while ensuring the integration of *V*2*H* technology. This is done by maximizing home autonomy and reducing the energy issued from the grid. The

optimization is performed over 20 years, taking into account weather data, grid energy price, and investment costs. The investment cost includes all equipment, namely PV panels, a wind turbine, batteries, power electronics, and cables, purchase and installation, and the operating and maintenances costs. The energy cost, however, mainly consists of the energy bills from the grid operator. The objective function is to ensure the optimal operation of the microgrid by minimizing its operating cost. Indeed, the area of the PV panels and the wind turbine power should be minimized to satisfy the load while minimizing the energy cost. Moreover, battery installed capacity should be optimized by taking into consideration its degradation cost. Consequently, the cost function to be optimized can be expressed as follows:

$$\text{Cost} = \mathbb{C}\_{\text{inv}} + \mathbb{C}\_{\text{ene}} \tag{5}$$

where:

• *Cinv* corresponds to the investment cost, which is given by:

$$\mathbf{C}\_{\rm inv} = \mathbf{C}\_{PV} \cdot \mathbf{P}\_{PV}^{\rm ins} + \mathbf{C}\_{WT} \cdot \mathbf{P}\_{WT}^{\rm ins} + \mathbf{C}\_{Bat} \cdot \mathbf{E}\_{bat}^{\rm max} \tag{6}$$

with:


$$\mathsf{C}\_{\text{ene}} = \mathsf{C}\_{\text{g}} \cdot \sum\_{i=1}^{20\_{\text{yurs}}} \left( \sum\_{t=1}^{24\_{\text{h}}} (P\_{\text{g}}^{i}(t)) \right) \tag{7}$$

with:


#### *3.3. System Constraints*

#### 3.3.1. Generation/Consumption Balance and Power Limitations

Minimizing the previously presented cost function is performed while respecting constraints related to each component of the home MG. The production/consumption balance is the main constraint of the system. To ensure a continuous and reliable power supply, the energy supplied by the RERs, the batteries, the grid, and the EV must be equal to the power required by the load at any time. This constraint is provided in Equation (8). Energy conversion efficiency is considered to account for the losses that occur when charging or discharging batteries or the EV battery. To express these losses in the power balance, the charging power is divided by the charging energy conversion efficiency (*η cha bat* = *η cha EV* = 0.9) and the discharging power is multiplied by the discharging energy conversion efficiency (*η disch bat* = *η disch EV* = 0.9). Equation (9) represents the new power balance constraint. Where *P* + *bat*(*t*), *P* + *g* (*t*), and *P* + *EV*(*t*) correspond to power supplied to the load and *P* − *bat*(*t*), *P* − *g* (*t*), and *P* − *EV*(*t*) stand for power fed to battery, power grid, and EV, respectively. Equations (10)–(12) describe the fact that these energy resources cannot charge and discharge at the same time. Moreover, to ensure that the battery, main grid, and EV do not exchange energy with each other, the constraint in Equation (13) is added. This is mainly set to minimize losses introduced by the energy conversion between these elements.

Equations (14) and (15) are set to ensure that the battery and EV battery recover its initial state of charge at the end of each day. To ensure this, the battery and EV provide a quantity of energy during underproduction periods, which is recovered during overproduction periods. Since the EV is not available all the day, it can be used only for a certain period of the day as specified by Equation (16). Finally, the power supplied (absorbed) by the battery is limited because of the characteristics of the conductors and switches, as shown by Equation (17). In addition of that, grid power should be limited to increase autonomy and avoid overloading. Its power is limited between minimum and maximum values as depicted by Equation (18).

$$P\_{Load}(t) = P\_{PV}(t) + P\_{WT}(t) + P\_{bat}(t) + P\_{\mathcal{g}}(t) + P\_{EV}(t) \quad ; \quad 1 < t < 24 \text{ h} \tag{8}$$

$$\begin{aligned} P\_{\text{Load}}(t) &= P\_{\text{PV}}(t) + P\_{\text{WT}}(t) + \eta\_{\text{bat}}^{\text{disch}} \cdot P\_{\text{bat}}^{+}(t) - \frac{1}{\eta\_{\text{bat}}^{\text{ch}}} \cdot P\_{\text{bat}}^{-}(t) + P\_{\mathcal{S}}^{+}(t) - \\ P\_{\text{S}}^{-}(t) &+ \eta\_{\text{EV}}^{\text{disch}} \cdot P\_{\text{EV}}^{+}(t) - \frac{1}{\eta\_{\text{EV}}^{\text{ch}}} \cdot P\_{\text{EV}}^{-}(t) \quad ; \quad 1 < t < 24 \text{ h} \end{aligned} \tag{9}$$

$$\eta\_{EV}$$

$$P\_{bat}^+(t) \cdot P\_{bat}^-(t) = 0 \quad ; \quad 1 < t < 24 \text{ h} \tag{10}$$

$$P\_{\varrho}^{+}(t) \cdot P\_{\varrho}^{-}(t) = 0 \quad ; \quad 1 < t < 24 \text{ h} \tag{11}$$

$$P\_{EV}^{+}(t) \cdot P\_{SV}^{-}(t) = 0 \quad ; \quad 1 < t < 24 \text{ h} \tag{11}$$

$$P\_{EV}^{+}(t) \cdot P\_{EV}^{-}(t) = 0 \quad ; \quad 1 < t < 24 \text{ h} \tag{12}$$

$$|P\_{EV}(t)| + |P\_{bat}(t)| + |P\_{\mathcal{S}}(t)| = |P\_{EV}(t) + P\_{bat}(t) + P\_{\mathcal{S}}(t)| \; \text{ : } 1 < t < 24 \text{ h} \tag{13}$$

$$\sum\_{t=1}^{24\_h} P\_{bat}(t) = 0\tag{14}$$

$$\sum\_{t=1}^{24} P\_{EV}(t) = 0\tag{15}$$

$$P\_{EV}^{+}(t) = P\_{EV}^{-}(t) = 0 \quad ; \quad 8 < t < 18 \text{ h} \tag{16}$$

$$P\_{bat}^{min} < P\_{bat}(t) < P\_{bat}^{max} \quad ; \quad 1 < t < 24 \text{ h} \tag{17}$$

$$P\_{\mathcal{S}}^{\text{min}} < P\_{\mathcal{S}}(t) < P\_{\mathcal{S}}^{\text{max}} \quad ; \quad 1 < t < 24 \text{ h} \tag{18}$$

#### 3.3.2. Renewable Resources and ESS Constraints

The installed PV power is determined by the optimisation algorithm. This power depends on the peak irradiation and is limited by the available area on the roof of the house. Equation (19) ensures that the area is within the tolerated limits. The installed wind power depends on the peak value of the wind speed. The radius of the turbine is also limited by a maximum value. This maximum value should not exceed the value tolerated for domestic installations in urban areas and it cannot be negative, which is expressed by the constraint in Equation (20).

$$0 < S\_{PV} < S\_{limit} \tag{19}$$

$$0 < R\_{WT} < R\_{limit} \tag{20}$$

To ensure maximum lifespan (maximum cycles) for the battery and the EV battery, minimum SOC should be limited. On one hand, to ensure a safe operation of batteries (respectively EV battery), the SOC is limited by an upper limit, equal to 1. On the other hand, batteries (respectively, EV battery) usually have a maximum DOD set by the designer, indicating that by respecting this DOD the battery (respectively, EV battery) will be able to be operated at the maximum of charge/discharge cycles. SOCs for both the battery ESS and EV battery are then limited by a lower limit equal to the DOD for the whole day. These constraints are presented in Equations (21) and (22). Maximum battery capacity to be installed is optimized and is consequently limited to a reasonable capacity, denoted *C max bat* , as expressed by Equation (23).

$$DOD\_{\text{bat}} < SOC\_{\text{bat}}(t) < 1; \quad 1 < t < 24 \text{ h} \tag{21}$$

$$\text{DOD}\_{EV} < \text{SOC}\_{EV}(t) < 1; \quad 1 < t < 24 \text{ h} \tag{22}$$

$$0 < E\_{bat}^{\max} < \mathcal{C}\_{bat}^{\max} \tag{23}$$

with:


#### **4. Simulation Results**

Numerical studies have been carried out to investigate the benefits of integrating EVs in terms of financial benefits, optimal sizing, and guaranteed energy autonomy by limiting power issued from main grid. From Equations (6) and (9), it is possible to distinguish decision variables, which must have the optimal values to minimize the total cost while respecting constraints; mainly the power balance constraint. The decision variables are:


The dealt with problem is a constrained non-linear optimization problem, which is described by Figure 4. The model is characterised by non-linear equality constraints and non-linear inequality constraints. In addition to these constraints, it is mandatory to define a set of lower and upper bounds on the design variables so that the solution is always in the specified range, such as the limits of the PV area and the radius of the wind turbine.

The sizing and energy management system is formulated as an optimization problem and is solved using the following method based on the Interior-point algorithm [32] using MATLAB software:

$$\min\_{\mathbf{x}} f(\mathbf{x}), \text{s.t.:} \begin{cases} \mathbf{C}(\mathbf{x}) \le \mathbf{0} \\ \mathbf{C}\_{eq}(\mathbf{x}) = \mathbf{0} \\ A \times \mathbf{x} \le b \\ A\_{eq} \times \mathbf{x} = b\_{eq} \\ l\_b \le \mathbf{x} \le u\_b \end{cases} \tag{24}$$

With:


This method fits the problem as it minimises a multivariate function; some variables are scalars and some are vectors. It allows one to compute the minimum of the objective function under non-linear equality and non-linear inequality constraints, with upper and lower bounds of the decision variables.

**Figure 4.** Scheme of the optimization problem.

In the following, two case studies are presented and the results are discussed as follows:


#### *4.1. Optimization Input Data*

The investment, maintenance, and operating costs of PV, WT, battery, and grid energy costs are taken from [33], considering a lifetime of 20 years for PV and wind turbine, and 5 years for batteries. These data are provided in Table 1. Variable energy consumption for 24 h is considered in this study. Indeed, the used load profile is equivalent to the energy consumption of a small house with variable consumed power. Figure 5b shows the hourly load pattern of 24 h. Three load peaks can be noticed in the provided load profile: in the morning, at noon, and in the evening. The same consumer profile is considered for the two case studies and for the two cities for a fair and reliable comparison.

The weather data used for home MG optimal sizing and energy management system design are actual weather data issued from [34,35]. Numerical studies have been performed for two different cities in France considering a cold and hot climate for the sake of comparison and to prove the effectiveness of the proposed approaches for different climates and weather conditions. First, a cold city, namely Brest, is considered, which is characterised by high speed wind during a significant period of the year and low sunshine with high solar irradiance variability during the year. Second, Toulon, which is a city in the south of France, is considered. Toulon is characterised by higher constant irradiance compared to Brest, but it has less wind due to its geographical location. Weather conditions of these two cities are different, which is of great interest to show differences in terms of MG sizing, autonomy of the installation, and the economical benefits. Since the designed MG is connected to the main grid, the worst case is not presented hereafter. Indeed, yearly mean values are considered for both PV panels and wind turbine optimal sizing. The irradiance and wind curves for the considered cities are shown in Figure 5b and Figure 5c, respectively.

**Figure 5.** Load power and weather data for typical day in Brest and Toulon.

**Table 1.** Investment, maintenance, and lifetime costs of the used energy resources, with a lifetime of 20 years for PV and wind turbine, and 5 years for batteries.


In the simulations carried out, all parameters to be optimized are bounded in order to get more realistic installation corresponding to urban usage. All decision variables' boundaries are provided in Table 2, as described in the optimization problem constraints.

**Table 2.** Decision variables boundaries.


#### *4.2. Simulation Results for Home MG without EV Integration*

Simulations have been performed first for a home MG without considering EV usage. Hot and cold weather climate regions in France are selected to assess the renewable resources-based MG optimal design and, consequently, to study the output power and the economic benefits. The design results give the optimal sizing of the energy generation system and is given in Table 3.


**Table 3.** Case study 1: PV, wind turbine, and battery optimal sizing.

These results are coherent since the solar energy potential in Toulon is higher than in Brest and wind power generation is more available in Brest. On one hand, the area of PV panels is larger in Toulon than in Brest, which is due to Toulon's high solar energy potential compared to Brest. The installed power in Toulon is 3.84 kW, whereas in Brest it is equal to 2.58 kW. On the other hand, due to high wind speed in Brest, the installed wind power is higher than in Toulon. Indeed, wind power reaches 2.46 kW against 0.89 kW in Toulon. Battery capacity in Toulon is much higher than in Brest. This is mainly due to the fact that the net power, which represents the difference between energy consumption and the actual PV and WT energy generation is higher in Toulon. Indeed, the battery is the first component to intervene in the energy management algorithm in order to maintain the consumption/generation balance.

Once the optimal sizing is performed, a one-day simulation is performed in order to show the behaviour of the MG components. Hence, PV and WT energy generation in Brest and Toulon are provided in Figure 6. Figure 7 shows the total RER energy production and net power (difference between RER production and load consumption). Net power can be provided using battery ESS if the SOC and the required power are within the specified limits or using the main grid, as shown by Figure 8. The sum of the battery and grid powers is equal to net power at any time, which shows that the main constraint related to power balance is achieved. The constraint related to the prohibited exchange of power between the battery and the grid (in both directions) is also fulfilled during the day.

**Figure 6.** Case study 1: Renewable energy resources energy generation in Brest and Toulon.

The variation of battery SOC is provided in Figure 8b. It can be noticed that the battery charges and discharges during the day while respecting the the specified DOD. The battery is safely operated, its SOC varies between the limits indicated in the constraints, which makes it possible to consider a lifetime of 5 years. It is also shown that the battery SOC at the end of the day is equal to the initial value.

The total energy from the grid before the integration of the RERs was 377.24 MWh during 20 years. This energy consumption issued from the grid is reduced to 92.2 MWh in Brest and 126.95 MWh in Toulon after the integration of the RERs, allowing an autonomy of 75.56% in Brest and 66.35% in Toulon. These results show the profitability of the system after 20 years of operation. Indeed, the total energy cost before the integration of the RER was 70,703 €. This energy cost is reduced to 40,449 € in Brest and 46,105 € in Toulon. The investment cost, the grid energy cost, and the autonomy in the considered cities are given in Table 4.

**Figure 7.** Case study 1: Net power and battery power for Brest and Toulon. (**a**) Net power highlighting production/consumption unbalance during the day. (**b**) Battery power for 24 h.

**Figure 8.** Case study 1: Grid power and state of charge of the battery for Brest and Toulon.

**Table 4.** Case study 1: Energy cost and autonomy achieved without EV.


#### *4.3. Simulation Results for a Home MG with EV Integration*

A second case study has been investigated to assess the interest of integrating the *V*2*H* concept in a home MG in terms of energy self-sufficiency and total cost comprising the investment, operating and maintenance costs of the MG, and the energy bill from the main grid. Two operating conditions may arise during operation, which are the vehicle is parked at home and can be used as an additional ESS or the vehicle is in normal every-day use. In the last operating situation, the home MG is operated as in case study 1 operating conditions. Numerical studies have been conducted for the same domestic MG, taking into account the use of EV batteries and limiting the available main grid power. Indeed, another constraint is considered in this second case study in order to relieve the grid by limiting the power exchange with the main grid to ±1000 W. The same cities are considered for the sake of comparison. This allows for the studying of the power outputs of renewable resources, battery ESS, EV, and grid, as well as the economic benefits of the integration of EVs at domestic MGs.

Domestic MG optimal sizing results with the integration of the EV are provided in Table 5 for Brest and Toulon. The same conclusions can be drawn from these results as the results discussed in case study 1. Indeed, the solar energy potential of Toulon remains higher than that of Brest and vice versa for the wind energy generation. The battery ESS to be installed has increased as the grid power is severely limited to enhance the use of renewables and EV batteries, and, consequently, to limit the stress on the grid, especially during on-peak periods.


**Table 5.** Case study 2: PV, battery, and wind turbine sizing results.

Renewable resources generated powers for 24 h are shown in Figure 9. It can be noticed that the integration of the EV as an additional ESS does not significantly change the sizing and the installed power of the RERs. It can be considered that renewable energy production mainly depends on meteorological data and these results allow for the extraction of the maximum energy from the RERs. The imbalance between renewable energy production and load consumption is shown in Figure 10. This imbalance is compensated for using battery-based ESS, the main grid, and the EV when the EV is parked at home. Otherwise, during EV daily use, only the battery-based ESS and the grid intervene to guarantee the production/consumption balance. System constraints are respected and power balance is achieved all the time. Moreover, there is no energy exchange between the battery, the grid, and the EV, as can be seen from Figure 11. Finally, the EV power exchange with a domestic MG and state of charge of the EV batteries are given by Figure 12. It can be seen that the EV is mainly used during periods where renewables energy generation is lower and load consumption is higher. This usage relieves the grid and decreases the energy bill.

(**a**) PV energy generation for 24 h. (**b**) WT energy generation for 24 h.

**Figure 9.** Case study 2: Renewable energy resources generation for Brest and Toulon.

**Figure 10.** Case study 2: Net power representing the difference between renewables energy generation and load consumption and batteries power for Brest and Toulon.

**Figure 11.** Case study 2: Grid power and batteries state of charge for Brest and Toulon.

(**a**) EV power during 24 h. (**b**) State of charge of EV batteries.

**Figure 12.** Case study 2: EV batteries power and state of charge for Brest and Toulon.

EV usage in a home MG, even if limited to few hours, is still profitable because the home MG self-sufficiency is considerably increased in Toulon, which raises to 84.61%, and slightly increases in Brest, being raised to 76.87%, reducing the high electric network utilisation. The total cost for 20 years operation is 35,063 € in Brest and 38,827 € in Toulon. This allows financial savings of 5000 € and 8000 € in both cities, respectively compared to case study 1. The investment cost, battery energy, main grid energy, total energy cost, and achieved energy self-efficiency in the two considered cities are given in Table 6. It can be seen that the total cost is divided by almost 2 for the two cities. Moreover, the grid is relieved due to the use of the *V*2*H* concept and the energy bill is significantly decreased while achieving a very good autonomy.


Brest 377.24 70,703 257.520 21,880 87.238 13,182 57.642 47.289 35,063 76.87 Toulon 377.24 70,703 291.170 29,476 58.073 9350 95.126 53.494 38,827 84.61

**Table 6.** Case study 2: Energy cost and achieved autonomy with EV.

#### *4.4. Discussion*

Numerical studies have proved that the integration of an EV in home microgrids has brought several benefits, starting with the economic gains, which is of interest for householders. Such financial savings encourage customers to switch to the *V*2*H* concept, given the reduction in the electricity bill and in the capacity of the installed battery ESS. The major disadvantage is the investment cost, which is still a bit expensive, but considering the duration of the project, this cost is quickly paid back in few years.

The second advantage to mention is the reduction of the energy provided by the main grid. Simulation results showed that the integration of an EV reduces the grid power and, consequently, lowers the stress on the grid, especially during congestion periods. The energy decreases from 92.2 MWh to 87.238 MWh in Brest, and it decreases by almost 50% in Toulon as the energy issued from the main grid decreases from 126.95 MWh to 58.073 MWh. The EV battery provides a certain amount of energy when it is connected to the home

MG, and, therefore, allows grid power limitation. If the *V*2*H* and *V*2*G* concept is further expanded, there will be a decrease in grid power and therefore fewer outages, fewer voltage and frequency regulation problems, and also fewer CO<sup>2</sup> emissions thanks to the massive use of local renewable resources.

In a home MG, battery power decreases after the integration of an EV as an additional ESS. This leads to the reduction of the number of cycles, and, therefore, the lifetime of the battery increases. The increase in lifetime induces financial savings as the operating and maintenance costs will straightforwardly decrease. Moreover, the installed battery capacity decreases, which allows for a decrease in the initial investment cost. Indeed, a part of the required energy storage system to handle the intermittency of RER is ensured by using EV batteries.

Despite the large amount of energy supplied by the EV, its SOC does not significantly vary, which can be explained by the large storage capacity of the EV to ensure certain autonomy. Indeed, the EV will have enough energy to support the home MG while limiting main grid power. In this work, the EV SOC is limited to a DOD of 85% to avoid its battery being significantly discharged, but in reality, the EV can be discharged even more. Indeed, the Lithium-ion batteries used in EVs can be discharged even more than 50% of their maximum capacity to support the home MG.

Table 7 summarizes the two conducted case studies. It is clear from this table and the previously discussed results that the integration of the EV brings great benefits to a home MG. Indeed, the energy self-sufficiency has increased in both cities, with a large improvement for Toulon (hot climate with huge PV potential). In terms of financial benefits, it is obvious from the given results that the use of renewable resources and the EV energy storage capacity lowers the energy bill for 20 years of operation, even if the investment cost is quite high, mainly due the price of battery energy storage devices.

The use of the electric vehicle as a RER energy storage device leads to the degradation of its battery over time [13], as the lifespan of the battery depends on the number of charge/discharge cycles. It would be more interesting to integrate an EV battery degradation cost in future work. Moreover, more aspects related to home MG EMS can be investigated, such as demand response and energy trading in the community integrating an EV fleet for enhancing the overall operation of the community MG [36].


**Table 7.** Summary of the achieved benefits for 20 years of operation of the home MG in Brest and Toulon.

#### **5. Conclusions and Perspectives**

This paper has presented an optimal sizing algorithm and energy management scheme for a home MG integrating the *V*2*H* technology for both hot and cold climates. The presented case studies show the advantage of the *V*2*H* concept, especially with the high integration of renewable energy resources. The electric vehicle presents a support for renewable energies by offering a larger storage capacity, thus efficiently increasing renewable energy generation by increasing on-MPPT mode usage. This increase in renewable production reduces the dependence on the grid, as shown in the simulation results. Therefore, the energy bill is reduced and the use of clean energy is empowered for sustainable development. It is mainly demonstrated that the use of *V*2*H* presents a financial interest for homeowners, supports the integration of RER for sustainable development, and relieves the main grid during on-peak periods.

The economic advantage of *V*2*H* and its cost-effectiveness may encourage homeowners to switch from thermal to electric vehicles, thus reducing the pollution rate, knowing that the total amount of CO<sup>2</sup> emitted by the design of the electric vehicle is half that emitted by the manufacturing and operation of the combustion vehicle. This greatly reduces the level of CO<sup>2</sup> emissions over a period of 20 years. Moreover, in a house MG, the electric vehicle reduces the size of the storage system and avoids its degradation by storing surplus production, which lowers the total investment cost and reduces the rate of manufacturing and changing of storage systems, in this case the battery ESS.

In the near future, it is interesting to scale up to a district, using a fleet of electric vehicles, to show the benefits of the *V*2*G* concept. It is foreseeable that the integration of electric vehicles will significantly reduce the pollution rate. However, the economic profitability needs to be demonstrated through further investigations. The move towards a larger scale will lead to more constraints, as vehicles will not have the same displacement and availability over a day, and this constraint will further complicate the optimisation of the objective function. It is not certain that, by increasing the number of electric vehicles, the network will be relieved, it may happen the other way round, where the charging stations require a huge amount of energy, and if the number of stations increases, the energy required increases accordingly. One solution to this problem is to power the charging stations with renewable energy.

Other aspects linked to artificial intelligence must be used to forecast weather data for optimal energy production and also to predict the displacement and consumption of vehicles, as well as their availability. Other parameters need to be considered in the modelling of the battery degradation cost, solar panels, and wind turbines to achieve more accuracy, such as the temperature, the change of the energy cost in the energy market, and load shedding.

**Author Contributions:** Conceptualization, O.O. and E.E.; methodology, O.O. and E.E.; validation, E.E.; formal analysis, O.O., E.E.; investigation, O.O. and E.E.; resources, O.O. and E.E.; writing original draft preparation, O.O.; writing—review and editing, E.E., Y.A., F.L.G. and E.S.G.; visualization, O.O.; supervision, E.E.; project administration, E.E.; funding acquisition, E.E. and Y.A. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded by Brest Métropole and ISEN Yncréa Ouest, France.

**Conflicts of Interest:** The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

#### **Nomenclature**

The following variables are used in this manuscript:



### **References**


## *Article* **Solar-Based DG Allocation Using Harris Hawks Optimization While Considering Practical Aspects**

**Suprava Chakraborty <sup>1</sup> , Sumit Verma 2,\*, Aprajita Salgotra <sup>2</sup> , Rajvikram Madurai Elavarasan 3,\*, Devaraj Elangovan <sup>1</sup> and Lucian Mihet-Popa 4,\***


**Abstract:** The restructuring of power systems and the ever-increasing demand for electricity have given rise to congestion in power networks. The use of distributed generators (DGs) may play a significant role in tackling such issues. DGs may be integrated with electrical power networks to regulate the drift of power in the transmission lines, thereby increasing the power transfer capabilities of lines and improving the overall performance of electrical networks. In this article, an effective method based on the Harris hawks optimization (HHO) algorithm is used to select the optimum capacity, number, and site of solar-based DGs to reduce real power losses and voltage deviation. The proposed HHO has been tested with a complex benchmark function then applied to the IEEE 33 and IEEE 69 bus radial distribution systems. The single and multiple solar-based DGs are optimized for the optimum size and site with a unity power factor. It is observed that the overall performance of the systems is enhanced when additional DGs are installed. Moreover, considering the stochastic and sporadic nature of solar irradiance, the practical size of DG has been suggested based on analysis that may be adopted while designing the actual photovoltaic (PV) plant for usage. The obtained simulation outcomes are compared with the latest state-of-the-art literature and suggest that the proposed HHO is capable of processing complex high dimensional benchmark functions and has capability to handle problems pertaining to electrical distribution in an effective manner.

**Keywords:** RDS; Harris hawks; optimal power flow; optimization; solar PV; transmission loss

## **1. Introduction**

Installing distributed generation (DG) sources in the distribution network system has been standard practice in recent years to minimize overall power losses and enhance the power quality [1,2]. The optimum sizing and positioning of DGs in power system networks are essential to maximize the benefits from those installations. The incorrect allocation and unreasonable sizing of DG units in the power system networks may increase voltage sags, voltage flickering, harmonic distortion, fault current, and power losses. With the application of DG units, the power system losses may be reduced by 13% [3,4]. In the functioning of power systems, economic damage and voltage collapse may be avoided through the reduction in power loss and voltage stability enhancement, respectively [5]. Thus, the investigation in optimal location selection and sizing of DG units in the distribution network is a step towards a profitable electricity supply [6,7]. Among all DG systems, solar photovoltaic DG systems seek attention worldwide for their abundant availability, easy installation, maintenance, and environment-friendly features.

The major goals of most techniques to determine the best location and size of DG units are to reduce power loss and improve voltage profile. The various techniques such

**Citation:** Chakraborty, S.; Verma, S.; Salgotra, A.; Elavarasan, R.M.; Elangovan, D.; Mihet-Popa, L. Solar-Based DG Allocation Using Harris Hawks Optimization While Considering Practical Aspects. *Energies* **2021**, *14*, 5206. https:// doi.org/10.3390/en14165206

Academic Editor: Carlo Renno

Received: 22 July 2021 Accepted: 17 August 2021 Published: 23 August 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

as analytical methods, ant bee colony (ABC), genetic algorithm (GA), tabu search (TS), particle swarm optimization (PSO), fuzzy system, evolutionary programming, dynamic programming, etc., have been utilized to achieve the aforesaid objectives in a distribution network through the proper allocation and sizing of DG units. In the literature, GA is used to estimate the placement and size of DG units to improve the voltage profile and reduce power loss. Once DG units are appropriately placed in the distribution system network, voltage stability and loss reduction are improved significantly. The GA is utilized as the most applied optimization technique in resolving the problem of DGs allocation and sizing [8,9]. The multiobjective genetic optimization method is used in radial distribution systems to determine the best position and size for renewable-based DG units [10]. For site determination of DGs planning and performance index-based size, a GA-based multiobjective optimization is utilized to minimize the actual power loss in distribution systems with constant power, current, and impedance models [11]. Almabsout et al. [12] suggest an improved GA to determine the best placement and capacity of the simultaneous allocation of DGs/SCs in radial systems by combining the benefits of genetic algorithms and local search [12]. To minimize system losses, a mix of analytical and genetic algorithm approaches is utilized to optimize the allocation of numerous DGs in a distribution network [13].

To reduce real power losses and improve the voltage profile, Madhusudhan et al. [14] proposed the GA to identify the optimum location, as well as the size of the distribution network's DG units. Ayodele et al. [15] used GA to find the best DG technology for optimal power system functioning, as well as the best position and size of the DG to reduce network power loss. GA is applied to reduce the cost of system expansion and improves system stability [16,17]. However, GA convergence time is high, especially, when applied in the solution of complex problems, and may suggest inaccurate solutions. When compared to GA and TS techniques, Hassan et al. and Fan et al. [18,19] employed simulated annealing (SA) to find and specify the capacity of DGs while lowering computation time. However, the SA method has disadvantages such as termination at a local minimum, significant computational time, no information regarding the divergence of the local minimum from the global minimum, and no upper constraint for the calculation time. Using the TS approach, Liu et al. and Azam et al. [20,21] concentrated on DG optimum planning with the goal of minimizing both losses and line loadings. The TS technique, on the other hand, has the drawback of requiring a large number of iterations and parameter calculations. PSO was used to determine the best scale and distribution of DG units in the power system, together with its benefits [22].

One of the most effective and widely used optimization strategies is the PSO [23–25]. Barik et al. [26] presented a multiobjective PSO method for determining the best location and size of DG units while taking economic and technical factors into account. The advanced versions of PSO methods, such as improved PSO [27], binary PSO [28], social learning PSO [29], PSO with inertia weight, and PSO with constriction factor [30], are also applied in the DG allocation and sizing problems. However, the PSO technique has some disadvantages, such as difficulty in initializing the design parameters and inapplicability to scattering problems. Tolabi et al. and Oloulade et al. [31,32] introduced the ant colony optimization (ACO) technique to tackle the allocation and size problem of renewable energy source-based DGs in radial distribution networks with the goal of minimizing overall system losses. Their analysis showed that ACO gives a better solution, and computational time is less than GA. However, ACO takes more time to converge due to the complex nature of the problem but is still shorter than analytical methods. The major disadvantage of the ACO technique lies with its uncertainty in time to convergence. Das et al. [33] and Seker and Hocaoglu [34] used the artificial bee colony (ABC) method to compare results to the PSO technique and discovered that ABC provides a higher-quality solution with a faster convergence rate. The cuckoo search algorithm was used by Yuvaraj and Ravi [35] to improve the voltage profile and reduce power losses in biomass and solar–thermal DG

units. To optimize the system voltage profile and decrease line losses, Arya and Koshti [36] used a shuffled frog leaping algorithm.

Rajaram et al. [37] used a plant growth simulation algorithm with objectives such as decreasing the losses and improving the voltage profile. To reduce energy losses in a distribution network system, Othman et al. [38] used the big bang–big crunch approach to find appropriate DG units. The bat algorithm was suggested by Sudabattula and Kowsalya [39] for the efficient allocation of solar-based DGs in the distribution network. To decrease power losses while preserving voltage profile, Duong et al. [40] developed an efficient biogeography-based optimization for optimal location and size of solar photovoltaic distributed generating units.

Harris hawks optimization (HHO) is a new metaheuristic optimization algorithm used in various applications, as tabulated in Table 1.

After a thorough search in credible academic publications, as shown in Table 1, to-date, the efficient newly invented HHO method has not been utilized to optimal solar-based DG allocation in a radial distribution system. As a result, this study compares and contrasts the suggested work with well-known optimization techniques. Suitable DG unit placement may bring significant benefits, including cost saving through a reduction in power loss and increasing the purchasing power capacity.


**Table 1.** Application of HHO in different literatures.


**Table 1.** *Cont.*


**Table 1.** *Cont.*


#### **Table 1.** *Cont.*

#### *Motivation and Contributions*

The primary motivation behind this work is to design a novel technique for appropriate allocation and sizing of solar photovoltaic DGs to reduce power losses and enhance the voltage profile. Worldwide sustainable development is possible through the generation of electricity from renewable energy resources. The Indian government has taken a number of steps to stimulate the use of renewable energy (RE) resources, including setting state-specific RE objectives in the form of solar purchase obligations (SPO) and renewable purchase obligations (RPO). Every state has set a goal to fulfill a significant part of its overall energy demand from renewable resources under the provisions of RPO and SPO. The solar photovoltaic DGs (PV DG) considered in this paper are among all the renewable energy resources; solar energy has received major importance due to its abundant availability worldwide. Although researchers have previously used a variety of

approaches to tackle the problem of DG allocation and size in the power system network, the authors have not considered the actual field installation capacity of PV DG; instead, they have considered only the actual power to be injected. Whereas the output of the solar PV DG is a meteorological parameter and PV module parameter-dependent system, thus, it is imperative to calculate the actual size of the PV DG to be installed to inject the targeted power into the grid. The contribution of this work is presented below:


The remainder of this article is structured in the following order. The mathematical formulation of the problem with various constraints is detailed in Section 2. The detail of the proposed HHO and the solution approach for the considered problem is presented in Section 3. In Section 4, the problem is tested with a benchmark function and with standard test systems. Section 5 deals with the practical calculation of solar PV DG. Section 6 concludes with some final observations together the breadth of future development.

#### **2. Formulation of the Mathematical Problem**

#### *2.1. Loss Minimization*

The objective of the present work is to relax the congestion in power lines along with determining the proper size and optimal location of DGs while keeping the losses (202.67 kW and 224.9 kW for IEEE 33 and IEEE 69 bus RDS, respectively) to the minimum. The major objective function (OF) is framed in the form of total system losses. Therefore, the OF may be stated by Equation (1).

$$\text{OF} = \text{Minimize}(P\_{\text{Loss}}) \tag{1}$$

where

$$P\_{\rm Loss} = \sum\_{k=1}^{n} g\_k \left( V\_i^2 + V\_j^2 - 2V\_i \times V\_j \times \cos\left(\delta\_i - \delta\_j\right) \right) \tag{2}$$

The various constraints of the proposed optimization problem are as stated in Equations (3)–(7).

$$V\_i^{\min} \le V\_i \le V\_i^{\max} \tag{3}$$

$$P\_{DG}^{\min} \le P\_{DG} \le P\_{DG}^{\max} \tag{4}$$

$$\mathbf{Q}\_{\rm DG}^{\rm min} \le \mathbf{Q}\_{\rm DG} \le \mathbf{Q}\_{\rm DG}^{\rm max} \tag{5}$$

$$\mathbf{Q}\_{\mathbf{G}\_i}^{\min} \le \mathbf{Q}\_{\mathbf{G}\_i} \le \mathbf{Q}\_{\mathbf{G}\_i}^{\max} \tag{6}$$

$$P\_{G\_i}^{\min} \le P\_{G\_i} \le P\_{G\_i}^{\max} \tag{7}$$

where *g<sup>k</sup>* is the conductance of branch *k*; *V<sup>i</sup>* and *V<sup>j</sup>* are the magnitude of voltages at sending and receiving bus, respectively; *PDG* and *QDG* represent active and reactive power generation by DG; *δ<sup>i</sup>* is the phase angle at *i*th and *j*th bus, respectively; and *PG<sup>i</sup>* represents active power generation at *i*th bus.

In Equations (3)–(7), the superscripts max and min represent the upper and the lower limits of the respective variables. The major objective here is to reduce congestion in lines and minimize losses.

#### *2.2. Practical Sizing of PV DG*

The power output of the PV module depends on meteorological parameters (such as ambient temperature and solar irradiance at the particular location) and on the parameters of the PV modules. To address the dependence on solar irradiation, the beta probability density function was used to model the uncertain nature of solar irradiance. The distribution of solar irradiance may be written as Equation (8) [61].

$$f\_{\varnothing}(s) = \begin{cases} \frac{\Gamma(a+\beta)}{\Gamma(a)\Gamma(\beta)} s^{(a-1)} & (1-s)^{(\beta-1)} \\ 0 & \end{cases} \qquad 0 \le s \le 1 \quad \& \ a, \beta \ge 0 \tag{8}$$

$$\beta = (1 - \mu) \left( \frac{\mu(1 - \mu)}{\sigma^2} - 1 \right) \tag{9}$$

$$\alpha = \frac{\mu \beta}{(1 - \mu)}\tag{10}$$

where Γ( ◦ ) is defined as the gamma function, *s* is defined as the random variable of solar irradiance, *fb*(*s*) is defined as the beta distribution function of *s*, *α* and *β* are defined as the parameters of the beta distribution function, *µ* and *σ* are defined as the mean and standard deviation of *s*.

Equations (11)–(13) have been used to address the effect of ambient temperature on the output of the PV module. The temperature of the PV module is influenced by the nominal operating module temperature (NOMT), solar irradiance, and ambient temperature, as shown by Equation (11) [62].

$$T\_M = T\_A + s \left(\frac{\text{NOMT} - 20}{0.8}\right) \tag{11}$$

The output current of the PV module is a function of the solar irradiance, short-circuit current, temperature coefficient of current, and temperature of PV module, shown by Equation (12) [63].

$$I\_M = s \left[ I\_{\mathcal{SC}} + \varepsilon\_i (T\_M - \mathfrak{T} \mathfrak{F}) \right] \tag{12}$$

The voltage of the PV module is a function of the open-circuit voltage, voltage temperature coefficient of the module, and its temperature, as shown by Equation (13).

$$V\_M = \left[V\_{OC} - \varepsilon\_v(T\_M - 25)\right] \tag{13}$$

$$FF\_M = \frac{I\_{mpp} \times V\_{mpp}}{I\_{SC} \times V\_{OC}} \tag{14}$$

Considering aforesaid environmental and PV module parameters correction factors, Equation (14) will be modified to Equation (15).

$$FF\_M = \frac{I\_{mpp} \times V\_{mpp}}{I\_M \times V\_M} \tag{15}$$

The output power of PV module, operating at maximum power point at solar irradiance *s*, may be estimated using Equation (16).

$$P\_o(\text{s}) = I\_{mpp} \times V\_{mpp} = F \text{F}\_M \times I\_M \times V\_M \tag{16}$$

The output power of the PV plant, operating at maximum power point at solar irradiance s may be estimated using Equation (17) [64].

$$P\_o(s) = \mathcal{N}\_M \times F\mathcal{F}\_M \times I\_M \times V\_M \tag{17}$$

The power output from the PV module considering maximum power point may be obtained by Equations (14) and (15). The variables used are defined as follows: *T<sup>M</sup>* is the temperature of the PV module; *T<sup>A</sup>* is the ambient temperature; NOMT is the nominal operating module temperature; *I<sup>M</sup>* is the current of the PV module; *V<sup>M</sup>* is the voltage of the PV module; *ISC* is the short-circuit current of PV module; *VOC* is the open-circuit voltage of PV module; *ε<sup>i</sup>* is the temperature coefficient of current; *ε<sup>v</sup>* is the temperature coefficient of voltage; *Impp* is the current at maximum power point at standard test condition (STC); *Vmpp* is the voltage at maximum power point at STC; *FF<sup>M</sup>* is defined as the fill factor; *N<sup>M</sup>* denotes the number of PV modules used in the PV plant; and *Po*(*s*) is the power output from the PV module (*N<sup>M</sup>* = 1)/plant at solar irradiance *s*.

The expected output power from the PV module considering the effect of solar irradiance s and ambient temperature *T<sup>A</sup>* may be calculated using Equation (18), and the expected total output power for a specific time period may be calculated using Equation (19) [64].

$$EOP(\mathbf{s}) = P\_o(\mathbf{s}) \times f\_b(\mathbf{s})\tag{18}$$

$$ETOP = \int\_0^1 EOP(s) \, ds\tag{19}$$

Monocrystalline silicon PERC PV module of the following specifications as presented in Table 2, was used for calculation [65].

$$
\varepsilon\_i = T\_i \times I\_{mpp} \tag{20}
$$

$$
\varepsilon\_{\upsilon} = T\_{\upsilon} \times V\_{mpp} \tag{21}
$$

**Table 2.** PV module parameters.


To convert the temperature coefficient of voltage and current from %/◦C (*T<sup>i</sup>* and *Tv*) to A/◦C (*ε<sup>i</sup>* ) and V/◦C (*εv*), Equations (20) and (21) are used, respectively. Impact of NOMT and irradiance on the temperature, voltage, and current of the PV module are depicted in Figure 1.

**Figure 1.** Different PV module parameters considering the NOMT and solar irradiance. **Figure 1.** Different PV module parameters considering the NOMT and solar irradiance.

The ambient temperature, mean, and standard derivation of solar irradiance during a specified time period are considered as 30.76°, 0.52 kW/m<sup>2</sup> , and 0.21 kW/m<sup>2</sup> , respectively [61]. The expected output power from the PV module considering the effect of solar irradiance and ambient temperature, associated environmental parameters, PV mod-Considering the parameters effecting the output of PV module, it is observed that the voltage and the current of the PV module varies from 45.85 to 42.62 V and 0.49 to 9.43 A, respectively, with the variation of solar irradiance. The temperature of the PV module varies from 32.3 to 60 ◦C as irradiance changes, as shown in Figure 1.

ule parameters and modeling parameters are tabulated in Table 3. **Table 3.** The expected output power from the PV module considering correction factors. **Environmental Parameters PV Module Parameters Modeling Parameters Output** The ambient temperature, mean, and standard derivation of solar irradiance during a specified time period are considered as 30.76◦ , 0.52 kW/m<sup>2</sup> , and 0.21 kW/m<sup>2</sup> , respectively [61]. The expected output power from the PV module considering the effect of solar irradiance and ambient temperature, associated environmental parameters, PV module parameters and modeling parameters are tabulated in Table 3.

() ()


*s* () **Table 3.** The expected output power from the PV module considering correction factors.

**Considering Correction Factors**

diance s is presented in Figure 2.

The power output from the PV module *Po*(*s*) and *fb*(*s*) with respect to solar irradiance s is presented in Figure 2.

*Energies* **2021**, *14*, x FOR PEER REVIEW 12 of 29

The expected total power obtained from a single PV module is the average of (), which is shown in Table 3 and depicted in Figure 3, i.e., 175.78 W. The expected total power obtained from a single PV module is the average of *EOP*(*s*), which is shown in Table 3 and depicted in Figure 3, i.e., 175.78 W. The expected total power obtained from a single PV module is the average of (), which is shown in Table 3 and depicted in Figure 3, i.e., 175.78 W.

**Figure 3.** Expected output power from the PV module. **Figure 3.** Expected output power from the PV module. **Figure 3.** Expected output power from the PV module.

#### **3. Proposed HHO and Solution Approach 3. Proposed HHO and Solution Approach**

#### **3. Proposed HHO and Solution Approach** *3.1. HHO: Features*

*3.1. HHO: Features* The Harris hawks is a recent population-based and gradient-free metaheuristic [66], hence, equally applicable to all optimization models or problems. The different phases of *3.1. HHO: Features* The Harris hawks is a recent population-based and gradient-free metaheuristic [66], hence, equally applicable to all optimization models or problems. The different phases of The Harris hawks is a recent population-based and gradient-free metaheuristic [66], hence, equally applicable to all optimization models or problems. The different phases of Harris hawks formulation are described in the next subsections.

#### Harris hawks formulation are described in the next subsections. Harris hawks formulation are described in the next subsections. *3.2. Exploration Phase*

*3.2. Exploration Phase 3.2. Exploration Phase* In this phase, Harris hawks randomly search on locations and adopt a wait and In this phase, Harris hawks randomly search on locations and adopt a wait and watch strategy to catch the prey, as per Equation (22) [66].

(() − ()) − 3( + <sup>4</sup>

$$X(t+1) = \begin{cases} X\_{rand}(t) - r\_1 |X\_{rand}(t) - 2r\_2 X(t)| & q \ge 0.5\\ \left(X\_{rabbit}(t) - X\_m(t)\right) - r\_3 (LB + r\_4(UB - LB)) & q < 0.5 \end{cases} \tag{22}$$

( − )) < 0.5

In this phase, Harris hawks randomly search on locations and adopt a wait and

where Xrabbit is the rabbit position and *X (t+1)* is the hawks' position for next iteration; *x* (*t*) shows the current position of the hawks. The *LB* and *UB* are maximum and minimum of decision variables. *Xrand*(*t*) is a randomly selected hawk from the current position. The random number in the range (0,1) is shown by the *r*(1–4).

$$X\_m(t) = \frac{1}{N} \sum\_{i=1}^{N} X\_i(t) \tag{23}$$

where *X<sup>m</sup>* is the mean of the current population of hawks while *N* indicates the hawks' total population.

Assuming the energy of the rabbit is given by

$$E = 2E\_o \left( 1 - \frac{t}{T} \right) \tag{24}$$

*E*, *Eo*, and *T* represent the escaping energy of prey, primary energy, and the maximum number of iterations taken, respectively.

#### *3.3. Exploitation Phase*

3.3.1. Soft Besiege

This behavior is demonstrated by Equation (25) [66].

$$X(t+1) = \Delta X(t) - E|f X\_{rabbit}(t) - X(t)|\tag{25}$$

$$
\Delta X(t) = X\_{rabbit}(t) - X(t) \tag{26}
$$

where *X*(*t*) and *J* represent the difference between the position vector of the rabbit and the current location in iteration and the random jump strength of the rabbit, respectively.

#### 3.3.2. Hard Besiege

This behavior is showcased by (27).

$$\mathbf{x}(t+1) = \mathbf{X}\_{ratio}(t) - E|\Delta \mathbf{X}(t)|\tag{27}$$

3.3.3. Soft Besiege along with Rapid Drives

In this behavior, it is assumed that hawks may choose their next step provided by the rule given in Equation (28) [66].

$$\mathcal{Y} = X\_{rabbit}(t) - E|fX\_{rabbit}(t) - X(t)|\tag{28}$$

$$Z = \mathcal{Y} + \mathcal{S} \times LF(D) \tag{29}$$

*D*, *S*, and *LF* are problem dimensions, a random number of order (1 × *D*), and levy flight function, respectively. In addition, u and v are random numbers (0 to 1 range), while beta is the default constant value (assuming 1.5).

$$LF(\mathbf{x}) = 0.01 \times \frac{\mathcal{U} \times \sigma}{|\boldsymbol{\theta}|^{\frac{1}{\beta}}}, \ \sigma = \left(\frac{\Gamma(1+\beta) \times \sin\left(\frac{\pi\beta}{2}\right)}{\Gamma\left(\frac{1+\beta}{2}\right) \times \mathfrak{F} \times 2^{\left(\frac{\beta-1}{2}\right)}}\right)^{\frac{1}{\beta}}\tag{30}$$

Soft besiege updates the position of the hawks by

$$X(t+1) = \begin{cases} Y\_{\prime} & \text{if } F(Y) < F(X(t)) \\ Z\_{\prime} & \text{if } F(Z) < F(X(t)) \end{cases} \tag{31}$$

#### 3.3.4. Hard Besiege along with Rapid Drives

Figure 4 [66].

Hard besiege condition given by the following rule: *Energies* **2021**, *14*, x FOR PEER REVIEW 14 of 29

$$\mathbf{X}(t+1) = \begin{cases} \mathbf{Y}\_{\prime} & \text{if } \mathbf{F}(\mathbf{Y}) < \mathbf{F}(\mathbf{X}(t)) \\ \mathbf{Z}\_{\prime} & \text{if } \mathbf{F}(\mathbf{Z}) < \mathbf{F}(\mathbf{X}(t)) \end{cases} \tag{32}$$

$$\mathbf{Y} = \mathbf{X}\_{rabit}(t) - \mathbb{E}|f\mathbf{X}\_{rabit}(t) - \mathbf{X}\_m(t)|\tag{33}$$

$$Z = Y + S \times LF(D) \tag{34}$$

The step-by-step procedure of HHO is summarized to the pseudocode, as shown in Figure 4 [66]. = + × () (34) The step-by-step procedure of HHO is summarized to the pseudocode, as shown in

**Figure 4. Figure 4.** Flow chart of HHO. Flow chart of HHO. Reproduced from [ Reproduced from [66], Elsevier: 2019 66], Elsevier: 2019. .

### *3.4. Solution Approch*

3.4.1. HHO for PV DG Placement and Location

The major goal of this research is to determine the best placement and size for numerous PV DGs with the least amount of network power loss and a better voltage profile. In this work, the inequality constraints are converted to the penalty functions (*PFs*), and these *PFs* are added to the *OF* to construct the fitness function (*FF*) defined in Equation (35).

$$\text{Minimum FF} = \text{OF} + \text{PF} \times \sum\_{j=1}^{VB} \left(\Delta V\_j\right)^2 \tag{35}$$

Here, *FF* is essential to be minimized in order to get minimum loss value, *VB* represents the set of overloaded lines and voltage violated load buses, and *PF* represents the penalty factor. The violation in inequality constraints such as load bus voltage and line power flows was handled using the penalty function approach. *PF* that represents penalty factor was taken as 10,000 throughout the simulation process.

#### 3.4.2. Computational Practice of HHO for DG Location and Values

**Step 1** Read the input data of the system, such as the maximum number of iterations, number of PV DG units, and population size.

**Step 2** Generate the value of the size of PV DG within their upper (*DGmax*) and lower limits (*DGmin*). The same is shown in Equation (36).

$$DG\_{\bar{i}} = DG\_{\bar{i}}^{\min} + rand \times \left(DG\_{\bar{i}}^{\max} - DG\_{\bar{i}}^{\min} \right) \tag{36}$$

Here, *DG<sup>i</sup>* represents the size of *i*th *DG* unit. Now, constitute a vector *X<sup>j</sup>* , that contains the possible locations (*LOC*) and size of *DG*s as mentioned in Equation (37).

$$X\_{\sf j} = \left[ DG\_{\sf j,1}, DG\_{\sf j,2}, \dots, \dots, DG\_{\sf j,n}, LOC\_{\sf j,1}, LOC\_{\sf j,2}, \dots, \dots, \dots, DC\_{\sf j,n} \right] \tag{37}$$

The *LOC* is generated randomly. Initial solution set *X* is then formulated as shown in Equation (38).

$$X = [X\_1, X\_2, \dots, \dots, \dots, X\_N] \tag{38}$$

**Step 3** Evaluation of the fitness function is processed using Equation (35) for individual Harris hawks, and the best hawk location is acknowledged.

**Step 4** Calculate *E* using Equation (24).

**Step 5** Exploration phase: Update the location of Harris hawks using Equation (22).

**Step 6** Exploitation phase: Update the position using Equation (25), (27), (31), and (32). **Step 7** Once the number of iterations reaches the maximum value, then terminate. Else, go back to Step 3.

#### **4. Simulation Results and Discussions**

*4.1. Testing Strategies*

The simulations were run on a MATLAB 9.9 computer with an Intel i3 CPU running at 2.4 GHz and 4 GB of RAM. The software utilized is MATPOWER 7.2, which is a well-known power modeling tool.

#### *4.2. Case 1*

In order to establish an algorithm, the proposed HHO was tested with selected extremely complex benchmark functions taken from CEC-2014 (see Table 4). The results obtained are tabulated in Table 5. The HHO seems to provide very competitive results as compared to other recent metaheuristic optimization techniques.


**Table 4.** Summary of the CEC-2014 benchmark functions considered.

**Table 5.** Comparative experimental outcomes on selected benchmark functions.


*4.3. Case 2*

The proposed HHO-based approach is applied to find the suitable location and capacity of the DGs in the IEEE 33-bus RDS test system where the network and load data may be obtained from [67]. The single line diagram of the IEEE 33-bus RDS is shown in Figure 5. IEEE 33-bus RDS has a total of 33 buses, among which 32 are load buses and 1 is a generator bus. It can be visualized from Figure 5 that at bus no. 1 generator is connected; the other buses may have any type of load connected, as per the requirement. The total active power demand is 3.72 MW while reactive is 2.3 MVAR. Total power loss of the system is 202.67 kW.

*Energies* **2021**, *14*, x FOR PEER REVIEW 17 of 29

**Figure 5.** Single line diagram of IEEE 33-bus RDS. **Figure 5.** Single line diagram of IEEE 33-bus RDS.

In order to find candidate buses for locating a PV DG using this approach for each individual bus, it is assumed that there is a PV DG at that bus at a time. For optimal sizing of a PV DG at this stage, it is assumed that the PV DG may produce electric power in all possible ranges (e.g., 0–1 MW). The proposed HHO algorithm is applied for the minimization of overall loss as the objective function of the problem. First, only one PV DG is used to relax the congestion in lines, and the results obtained are tabulated in Table 6. With the application of proposed HHO on distribution problem, the losses are reduced to 129.2 from 202.67 kW with only one DG in installation of size 0.95 MW. In order to find candidate buses for locating a PV DG using this approach for each individual bus, it is assumed that there is a PV DG at that bus at a time. For optimal sizing of a PV DG at this stage, it is assumed that the PV DG may produce electric power in all possible ranges (e.g., 0–1 MW). The proposed HHO algorithm is applied for the minimization of overall loss as the objective function of the problem. First, only one PV DG is used to relax the congestion in lines, and the results obtained are tabulated in Table 6. With the application of proposed HHO on distribution problem, the losses are reduced to 129.2 from 202.67 kW with only one DG in installation of size 0.95 MW.


**Table 6.** Variation in power loss with change in an optimal allocation of PV DGs. **Table 6.** Variation in power loss with change in an optimal allocation of PV DGs.

in the power network. The results obtained are presented in Table 6. The overall active power losses decreased to 86.9 and 72.10 kW with the application of two and three PV DGs, respectively, using HHO. The comparative results are portrayed in Table 7 in terms of the best location and size of PV DGs. The locations suggested by HHO to install PV plants in IEEE 33 bus are depicted in Figure 6. For further improvement, the problem is tested by installing two and three PV DGs in the power network. The results obtained are presented in Table 6. The overall active power losses decreased to 86.9 and 72.10 kW with the application of two and three PV DGs, respectively, using HHO. The comparative results are portrayed in Table 7 in terms of the best location and size of PV DGs. The locations suggested by HHO to install PV plants in IEEE 33 bus are depicted in Figure 6.

For further improvement, the problem is tested by installing two and three PV DGs

**Table 7.** Comparative results for optimal location and values of PV DGs corresponding to case 2. **Table 7.** Comparative results for optimal location and values of PV DGs corresponding to case 2.


32 0.8300


30 0.9500

**Table 7.** *Cont.* 11 0.9250

**Figure 6.** IEEE 33 bus line diagram with PV plants at locations suggested by HHO. **Figure 6.** IEEE 33 bus line diagram with PV plants at locations suggested by HHO.

The size and location suggested by HHO (refer to Table 7) provide maximum reduction in losses as compared to TLBO [68], GA [69], PSO [69], GA/PSO [69], QOTLBO [68], CTLBO ε-method [70], IMOEHO [71], I-DBEA [72], CTLBO [70], and BA [39]. In addition, the voltage graph of all the buses obtained after utilization of PV DGs is The size and location suggested by HHO (refer to Table 7) provide maximum reduction in losses as compared to TLBO [68], GA [69], PSO [69], GA/PSO [69], QOTLBO [68], CTLBO ε-method [70], IMOEHO [71], I-DBEA [72], CTLBO [70], and BA [39]. In addition, the voltage graph of all the buses obtained after utilization of PV DGs is showcased in Figure 7.

The bus voltages are obtained from the load flow analysis. The bus voltage profile improves significantly under the application of three PV DGs at their respective optimal locations. The variation of fitness function against the number of iterations for installation with three PV DGs using HHO is showcased in Figure 8. The iterative graph shows that the HHO converges to an optimal solution value with very few iterations. showcased in Figure 7. The bus voltages are obtained from the load flow analysis. The bus voltage profile improves significantly under the application of three PV DGs at their respective optimal locations. The variation of fitness function against the number of iterations for installation with three PV DGs using HHO is showcased in Figure 8. The iterative graph shows that the HHO converges to an optimal solution value with very few iterations.

**Figure 7. Figure 7.** Voltage graph for IEEE 33 bus system without and with PV DGs. Voltage graph for IEEE 33 bus system without and with PV DGs.

**Figure 8.** Convergence characteristics of fitness function pertaining to case 2 (with three PV DG). **Figure 8.** Convergence characteristics of fitness function pertaining to case 2 (with three PV DG).

*4.4. Case 3*

224.9 kW.

To test the effectiveness of the HHO on a larger system, the proposed approach is

line diagram of the IEEE 69-bus RDS is shown in Figure 9. IEEE 69-bus RDS consists of 69 buses, including 68 load buses and 1 generator bus. The generator is connected at bus no. 1 and a load of the required amount can be connected to the other buses. The total active power demand is 3.80 MW while reactive is 2.69 MVAR. Total power loss of the system is

**Optimization** 

GA [69] 3

PSO [69] 3

#### *4.4. Case 3*

To test the effectiveness of the HHO on a larger system, the proposed approach is tested to find the suitable location and capacity (size) of the DGs in the IEEE 69 bus RDS test system where the load and branch data values may be obtained from [73]. The single line diagram of the IEEE 69-bus RDS is shown in Figure 9. IEEE 69-bus RDS consists of 69 buses, including 68 load buses and 1 generator bus. The generator is connected at bus no. 1 and a load of the required amount can be connected to the other buses. The total active power demand is 3.80 MW while reactive is 2.69 MVAR. Total power loss of the system is 224.9 kW. *Energies* **2021**, *14*, x FOR PEER REVIEW 21 of 29

**Figure 9.** Single line diagram of IEEE 69-bus RDS. **Figure 9.** Single line diagram of IEEE 69-bus RDS.

For addressing the most suitable candidate buses for locating a PV DG using this approach for each individual bus, it is assumed that there is a PV DG at that bus at a time. For optimal sizing of a PV DG at this stage, it is assumed that the PV DG may produce electric power in all possible ranges (e.g., 0–1 MW). The proposed HHO algorithm is applied for the reduction/minimization of overall loss as the objective function of the problem. First, only one PV DG is used to relax the congestion and reduce losses in lines, and the results obtained are tabulated in Table 8. The installation of one optimum PV DG the line losses reduced by 48.86% with DG size of 0.95 MW. For addressing the most suitable candidate buses for locating a PV DG using this approach for each individual bus, it is assumed that there is a PV DG at that bus at a time. For optimal sizing of a PV DG at this stage, it is assumed that the PV DG may produce electric power in all possible ranges (e.g., 0–1 MW). The proposed HHO algorithm is applied for the reduction/minimization of overall loss as the objective function of the problem. First, only one PV DG is used to relax the congestion and reduce losses in lines, and the results obtained are tabulated in Table 8. The installation of one optimum PV DG the line losses reduced by 48.86% with DG size of 0.95 MW.

**Table 8.** Variation in power loss with change in an optimal allocation of PV DGs. **Table 8.** Variation in power loss with change in an optimal allocation of PV DGs.


in the power network. The results obtained are presented in Table 8. The overall active power losses decreased to 71.80 kW with the application of three PV DGs using HHO. The comparative results are portrayed in Table 9 in terms of the best location and size of PV DGs. The locations suggested by HHO to install PV plants in the IEEE 69 bus are depicted in Figure 10. For further improvement, the problem is tested by installing two and three PV DGs in the power network. The results obtained are presented in Table 8. The overall active power losses decreased to 71.80 kW with the application of three PV DGs using HHO. The comparative results are portrayed in Table 9 in terms of the best location and size of PV DGs. The locations suggested by HHO to install PV plants in the IEEE 69 bus are depicted in Figure 10.

62 1.0752 2.9897 89 60.43

<sup>61</sup> 1.1998 2.9879 83.2 60.43 <sup>63</sup> 0.7956

For further improvement, the problem is tested by installing two and three PV DGs

**(MW) <sup>P</sup>loss (kW) Loss Reduction**

**(%)**

Base case - - - - 224.9 0.00

21 0.9297

64 0.9848

**Table 9.** Comparative results for optimal location and values of PV DGs corresponding to case 3.


**Table 9.** Comparative results for optimal location and values of PV DGs corresponding to case 3.

The size and location suggested by HHO (refer to Table 9) provides maximum reduction in losses as compared to GA [69], PSO [69], TLBO [68], GA/PSO [69], QOTLBO [68], CTLBO ε-method [70], I-DBEA [72], SA [74], CTLBO [70], IWO [75], BFOA [76], and MFO [77]. In addition, the voltage graph of all the buses obtained after utilization of PV DGs is showcased in Figure 11. The bus voltages are obtained from the load flow analysis. The bus voltage profile improves significantly under the application of three PV DGs at their respective optimal locations. The variation of fitness function against the number of iterations for installation with three PV DGs using HHO is showcased in Figure 12. The iterative graph shows that the HHO converges to an optimal solution value with very few iterations.

0.14

0.142

0.144

0.146

0.148

**Fitness Function**

0.15

0.152

0.154

**Figure 10.** IEEE 69 bus line diagram with PV plants at locations suggested by HHO. **Figure 10.** IEEE 69 bus line diagram with PV plants at locations suggested by HHO. *Energies* **2021**, *14*, x FOR PEER REVIEW 24 of 29

The size and location suggested by HHO (refer to Table 9) provides maximum re-

**Figure 11.** Voltage graph for IEEE 69 bus system without and with PV DGs. **Figure 11.** Voltage graph for IEEE 69 bus system without and with PV DGs.

**Figure 12.** Convergence characteristics of fitness function pertaining to case 3 (with three PV DGs).

0 10 20 30 40 50 60 70 80 90 100

**Number of Iterations**

**Figure 11.** Voltage graph for IEEE 69 bus system without and with PV DGs.

**Figure 12.** Convergence characteristics of fitness function pertaining to case 3 (with three PV DGs). **Figure 12.** Convergence characteristics of fitness function pertaining to case 3 (with three PV DGs).

#### **5. Practical PV DG Size Analysis**

The optimum allocation of PV DGs using HHO indicates (refer to Table 7) that PV DGs need to be installed at bus number 13, 24, and 30 in case 2 and at bus number 17, 61 and 62 in case 3 (refer to Table 9) for optimum results. The analysis based on calculation from Section 2.2 suggests that in case 2, power to be injected at bus number 13, 24, and 30 are 831, 950, and 950 kW, respectively. In case 3, power to be injected at bus number 17, 61, and 62 are 532.9, 950, and 822 kW, respectively. Practically, to inject the targeted power as calculated through HHO, DC overload needs to be considered as described in Section 2.2. The actual field size of the PV DG plant and the number of PV modules required for case 2 and case 3 are tabulated in Table 10.

**Table 10.** Size of practical PV DG corresponding to case 2 and case 3 (with three PV DGs).


In case 2, the number of PV modules used in bus number 13 is 4728 to form a PV array of capacity 1.655 MWP. The PV array of capacity 1.891 MW<sup>P</sup> is proposed to be installed at bus number 24 and 30 using 5404 numbers of PV modules at each bus. In case 3, the practical size of the PV plant to be installed at bus number 17, 61, and 62 are of size 1.061, 1.892, and 1.637 MWP, respectively. The analysis shows that in practical conditions, approximately 50% DC overload exists on PV DGs installation. The allocation of PV DGs in-network without considering the practical size of the PV DG may lead to underperformance.

#### **6. Conclusions**

The proposed HHO-based approach is significantly effective in finding the optimum number, optimal locations, and optimal sizes of DGs. On conventional IEEE 33 and IEEE 69 bus test systems, the efficacy of the suggested technique is evaluated. With the employment of optimally sized DGs at their optimum location, voltage profiles of load buses are improved and the losses reduced noticeably. When more PV DGs are installed, the system's performance improves. The comparison with the TLBO, GA, PSO, QOTLBO, CTLBO, CTLBO ε-method, IMOEHO, I-DBEA, SA, IWO, BFOA, MFO, and BA methods shows that the proposed method performs comparatively better among all. The active power loss is reduced by 64.42% and 68.07% using this approach with the installation of three PV DGs in IEEE 33 and IEEE 69 bus RDS, respectively. The collected findings demonstrate that the proposed technique reduces power loss by a greater proportion with a smaller DG size when compared to other algorithms, and it offers better convergence properties. According to the analysis, there is roughly a 50% DC overload on PV DGs installations under real-world situations. The in-network allocation of PV DGs without consideration of the PV DG's realistic size may result in underperformance.

The HHO is established as a reliable optimization technique for tackling the congestion problem of power systems with the application of DGs. The proposed method provides an alternative way for both system operators and energy producers to tackle complex problems such as voltage instability, transmission congestion, and huge system losses in an impressive way. In addition, current work suggests the practical size of PV DGs that may be employed for producing effective outcomes.

**Author Contributions:** Writing—original draft, formal analysis, and validation, S.C.; writing original draft, conceptualization, investigation, methodology, and software, S.V.; writing—original draft, resources, supervision, and visualization, A.S.; data curation, validation and investigation, R.M.E.; writing—review and editing, D.E.; writing—review and editing, L.M.-P.; All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** Not applicable.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


## *Article* **An Optimized Framework for Energy Management of Multi-Microgrid Systems**

**Komal Naz <sup>1</sup> , Fasiha Zainab <sup>1</sup> , Khawaja Khalid Mehmood <sup>2</sup> , Syed Basit Ali Bukhari <sup>2</sup> , Hassan Abdullah Khalid <sup>1</sup> and Chul-Hwan Kim 3,\***


**Abstract:** Regarding different challenges, such as integration of green energy and autonomy of microgrid (MG) in the multi-microgrid (MMG) system, this paper presents an optimized and coordinated strategy for energy management of MMG systems that consider multiple scenarios of MGs. The proposed strategy operates at two optimization levels: local and global. At an MG level, each energy management system satisfies its local demand by utilizing all available resources via local optimization, and only sends surplus/deficit energy data signals to MMG level, which enhances customer privacy. Thereafter, at an MMG level, a central energy management system performs global optimization and selects optimized options from the available resources, which include charging/discharging energy to/from the community battery energy storage system, selling/buying power to/from other MGs, and trading with the grid. Two types of loads are considered in this model: sensitive and non-sensitive. The algorithm tries to make the system reliable by avoiding utmost load curtailment and prefers to shed non-sensitive loads over sensitive loads in the case of load shedding. To verify the robustness of the proposed scheme, several test cases are generated by Monte Carlo Simulations and simulated on the IEEE 33-bus distribution system. The results show the effectiveness of the proposed model.

**Keywords:** cost optimization; differential evolution; energy management system; multi-microgrids system; renewable generation

## **1. Introduction**

Over the past decade, energy demand has considerably increased because of energydependent lifestyle of humans, industries, and electric vehicles (EVs) [1]. Conventional power systems are traditionally designed and operate on key reliability principles of security and adequacy [2]; they cannot meet these enormous rising demands adequately. The up-gradation of power systems to smart grids has been considered to address the energy shortage problem, improve reliability, and facilitate the integration of renewable energy sources (RESs). One of the significant ideas in smart grid is microgrids (MGs). An MG is comprised of a segment of distribution framework, including distributed energy resources (DERs), and diverse end clients. An MG can operate in two modes: grid-tied and islanded modes. The MG generally works in grid-tied mode. In grid-tied mode, there is bidirectional flow of energy and information between an MG and different MGs or the power grid, i.e., an MG can absorb or supply power. In stand-alone mode, it only acts as a source, i.e., only supplies power to the connected load. DERs include distributed generator (DG) and distributed storage (DS) units with various limits. The DG can be controllable DG (CDG) or renewable DG (RDG): PVs and WTs. Integration of RESs (PVs and WTs)

**Citation:** Naz, K.; Zainab, F.; Mehmood, K.K.; Bukhari, S.B.A.; Khalid, H.A.; Kim, C.-H. An Optimized Framework for Energy Management of Multi-Microgrid Systems. *Energies* **2021**, *14*, 6012. https://doi.org/10.3390/en14196012

Academic Editors: Abu-Siada Ahmed and Victor Becerra

Received: 20 July 2021 Accepted: 17 September 2021 Published: 22 September 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

participates in cost minimization and reduction in load shedding. In an MG, energy is generated near the point of demand which reduces transmission and distribution losses, as well as grid expansion deferral. The whole operation of the MG is supervised by the MG energy management system (EMS) which incorporates power generation, energy storage, and load management programs. A fast energy management strategy is required for sound functioning of the MG with various DERs especially in autonomous mode [3–5]. The EMS performs optimal scheduling of all available energy resources and energy storage systems (ESSs) to meet load demands [6]. The optimal operation of smart grid is a great challenge due to uncertainties in the DGs' generation, load requirement, real-time prices, and penetration of EVs. The MMGs system, the combination of different MGs, can deal with these uncertainties, to a large extent. The main objectives of considering MMGs are cost optimization and the reliability of the system.

Significant research has been conducted to put forward different algorithms and optimization models for MMG-EMS. However, these studies have some limitations in comparison to the vast literature available on the energy management of a single MG. A multi-step hierarchical optimization algorithm based on a multi-agent system considering adjustable power and demand response (DR) was proposed in Reference [6]. In Reference [7], authors proposed energy management of islanded DC MG with dual active bridge converter-based power management units and control interface. The optimized and coordinated strategies for energy management in MMGs were proposed, and a 69-bus system was used for case study in Reference [8]. According to References [9,10], the MGs could transfer energy and coordinate with each other to make the entire system reliable and stable. In Reference [11], the impact of DR, which was synchronous with the MMG-based operation of active distribution networks, was investigated, and the optimization problem was solved with non-dominated genetic algorithm-II (NSGA-II) and tested on an IEEE 69-bus distribution system. A multi-period optimal dispatch model was suggested for a distribution network with clustered MGs in Reference [12]; a modified hierarchical genetic algorithm was used to solve bi-level optimization problem and tested on IEEE 14-bus distribution system. A hierarchical decentralized system of systems has been studied for coordinating multiple autonomous microgrid systems with grid connection in Reference [13]. In Reference [14], a hierarchical multi-agent EMS was proposed to increase the utilization of RES based on the DR programs (DRP) and BESSs within the MG. Interconnected MMGs (IMMGs) have numerous benefits over the single MGs. First, IMMGs have good economic characteristics both in grid-tied [15,16] and in islanded modes [1,17] because energy sharing fulfills load demands of the MGs with their own cheap RESs, thus reducing the cost of generation based on fossil fuels and the power losses occurred in distant transference by trading energy among the MGs which are close to each other. Second, the IMMGs are more reliable than the single MG in reducing stress on the main grid.

In all of the aforementioned papers, the main focus of the research was to maximize the utilization of the cheapest controllable DG (CDG) by considering its generation price. Few papers considered the price of RESs to ensure maximum utilization of renewable sources. Most of the papers have sufficiently discussed the DRP or demand-side management (DSM) schemes to meet load demands. Moreover, some researchers have considered islanded individual MG, islanded MMG system and grid-connected MMG system for their studies. However, they overlooked the different combinations of MGs. The MMG system is the most optimized network and can operate in two ways: either as a cooperative or as an autonomous system. In the cooperative MMG system, the benefits of overall system will be considered in that it is not necessary that each individual MG is getting profit, whereas, in the autonomous MMG, each MG will consider and prefer its own benefit in that it is not necessary that the overall system is in the most optimized form. Therefore, it is possible that an MG is getting more benefit individually than to be a part of the cooperative MMG system, and vice versa. Furthermore, the types of loads and the load shedding of sensitive load have not been considered so far in the literature.

In this paper, a hierarchical EMS for the optimal operation of MMGs is proposed; the MGs are in islanded or MMGs connected mode. A two-level optimization, local and global, has been formulated. In local optimization, each MG acts autonomously and maximizes the utilization of RESs to meet load demand. In global optimization, there is a cooperative operation between the MGs which optimizes the overall system. This optimization strategy maximizes the utilization of cheap renewable energy, minimizes total cost, and reduces the load shedding. The total loads have been divided into two sets: sensitive and nonsensitive loads. The penalty prices of these loads are considered much higher than all other prices to avoid load shedding, whereas the penalty prices of sensitive loads are considered higher than that of non-sensitive loads to provide uninterrupted supply to sensitive load during load shedding. The proposed model is tested on the IEEE 33-bus test system, and the presented optimization problem is solved by differential evolution (DE) algorithm. Moreover, we have studied the proposed methodology for selected cases in order to check the robustness of the proposed scheme. For this purpose, we have used Monte Carlo Simulations (MCS) to generate the random cases of all total possible cases and run the numerical simulation of the proposed algorithm. Some of the major contributions of this study are summarized as:


The rest of the paper is organized as follows. Section 2 focuses on the proposed MMG model and mathematical modeling of wind turbine and photovoltaic DGs. Section 3 explains the optimization formulation of the strategy: local and global. Section 4 refers to the simulations and case study of the modified IEEE 33-bus distribution system. Section 5 discusses the results, including the proposed scenarios and the cases generated by MCS. The conclusion is summarized in Section 6.

#### **2. System Model**

#### *2.1. Configuration of Proposed MMG System*

Figure 1 shows the proposed hierarchical two-step energy management system model. Each MG includes CDG, PV, and WT units which have sufficient power generation to fulfill its MG demands, BESS to reserve surplus energy in off-peak hours and utilize it in on-peak hours to avoid load shedding, and an EMS to make all this management possible. The EMS at the MG level ensures optimized energy management by utilizing all energy and ancillary services. In this paper, two sets of loads are taken into consideration: sensitive loads (SLs) and non-sensitive loads (NSLs). If a customer, in the event of low power generations, requires to shed loads due to shortage of power, the NSL is curtailed, and the SL gets uninterrupted supply. The CEMS is connected to each MG-EMS, central BESS (CBESS), and to the grid to coordinate the overall system to make it reliable and minimize emergency load shedding, which lowers the chance of system-collapse. The CEMS at the MMG level ensures interconnection of different autonomous MGs. The EMS of every single MG is also connected to the EMS of every other neighboring MG via power lines for power flow when

needed. The MMG system, presented in Figure 1, has the following benefits: (i) they share reserves in on-peak hours or in crucial condition (e.g., faults on generation side or sudden increase in the load demand); (ii) economic dispatch in the whole system, either islanded or grid-connected; and (iii) a CBESS can serve critical loads after utilizing all available options and increases the resilience of the system.

**Figure 1.** Proposed energy management model.

For the purpose of simulations, wind speed and solar irradiance are modeled by implementing the Weibull [18] and beta probability distributions, respectively. Each MG has the freedom to take part in the MMG system. Otherwise, it can operate as an autonomous single islanded entity.

#### *2.2. Wind Turbine DGs*

#### 2.2.1. Wind Speed Modeling

Different probability distributions are used to model variations in the wind speed. Weibull probability distribution is often used [19] and is also used in this paper. The probability density function of this distribution is given in (1).

$$f(s\_w|a,b) = \begin{cases} \frac{b}{a^b} s\_w^{b-1} e^{-\left(\frac{s\_W}{a}\right)^b}, \text{ for } \ge 0\\ 0, \text{ for } \ge 0 \end{cases},\tag{1}$$

where *a* and *b* are the scale and shape parameters of the Weibull distribution, respectively. Maximum likelihood estimation (MLE) method is used to find these parameters.

#### 2.2.2. Wind Power Model

The wind power DG-model used for obtaining electrical energy from the generated wind speed data is given in (2) [19].

$$P\_w = \begin{cases} P\_r \times \frac{s\_{w} - s\_{ci}}{s\_r - s\_{ci}}, & \text{for } s\_{ci} \le s\_w \le s\_r\\ P\_r, & \text{for } s\_r \le s\_w \le s\_{co} \\ 0, & \text{otherwise} \end{cases} \tag{2}$$

where *P<sup>w</sup>* and *P<sup>r</sup>* are power generated from a WT and rated capacity of a WT, respectively; *sw*, *sci*, *sco* and *s<sup>r</sup>* are current, cut-in, cut-out and rated wind speeds, respectively.

### *2.3. Photovoltaic DGs*

2.3.1. Solar Irradiance Modeling

The beta distribution is usually used for modeling of the variations in the solar irradiance [20]. For *α*, *β* > 0, the pdf of Beta distribution is given as (3).

$$f(s\_{\bar{r}rd}|\alpha, \beta) = \begin{cases} \frac{\gamma(a+\beta)}{\gamma(a)\gamma(\beta)} \times s\_{\bar{r}rd}^{(a-1)} \\ \times (1 - s\_{\bar{r}rd})^{(\beta - 1)} \text{, for } 0 \le s\_{\bar{r}rd} \le 1 \\ 0, \text{ for } s\_{\bar{r}rd} < 0 \end{cases} \tag{3}$$

where *γ* is the gamma function; *α* and *β* are known as the shape parameters of the beta distribution. Both parameters are estimated by the MLE method.

#### 2.3.2. Solar Power Model

The model of a PV panel given in (4)–(8) provides electric power obtained from the generated solar irradiance samples [20].

$$T\_{cell} = T\_{amb} + \left( s\_{id} \times \frac{T\_{tot} - 20}{0.8} \right) \tag{4}$$

$$I = s\_{\rm ind} \times \left( I\_{\rm sc} + \mathbf{K}\_{\rm i} \times \left( T\_{\rm cell} - \mathbf{25} \right) \right), \tag{5}$$

$$V = V\_{oc} - K\_{v} \times T\_{cell} \tag{6}$$

$$FF = \left(\frac{V\_{\text{max}} \times I\_{\text{max}}}{V\_{\text{oc}} \times I\_{\text{sc}}}\right) \tag{7}$$

$$P\_s = N\_{\text{total}} \times FF \times V \times I. \tag{8}$$

where *sird*, *FF*, *Ntotal*, *P<sup>s</sup>* , *Tcell*, *Tamb* and *Tnot* are solar irradiance, fill factor, total number of PV modules, power generated from a PV panel, temperature of a cell, ambient temperature and nominal operating temperature of a cell, respectively; *Isc*, *Imaxp*, *K<sup>i</sup>* , *Voc*, *Vmaxp* and *K<sup>v</sup>* are short circuit current, current at maximum power point, temperature coefficient for current, open circuit voltage, voltage at maximum power point and temperature coefficient for voltage, respectively.

#### **3. Optimization Formulation**

In this section, a day-ahead energy management problem for proposed MMGs model is formulated based on the DE algorithm. The objective function is to minimize the total operational cost of the MG network whether it is operating as a single islanded or grid-tied MG system; the MMGs islanded or grid-connected network makes the system reliable by giving uninterrupted supply of energy and minimizes curtailment of loads. The elements of bi-level optimization model are shown in Figure 2 including decision input variables, constraints, objective function and optimal output variables. A constraints-handling technique is used in the presented DE algorithm to get the best fitness value of the decision variables, and penalty factor technique is used to attain the most feasible and optimized value of objective function. The proposed model is figured for a twenty-four-hour scheduling horizon with a time period of *'t'*; *'t'* can be any uniform interval of time. However, in the suggested model, *'t'* has been considered to be one hour. As mentioned earlier, it is a two-step optimization strategy: local and global. Step-wise detailed mathematical models are developed in the following subsections.

**Figure 2.** Elements of the optimization model.

#### *3.1. Local Optimization*

At local level, the EMS of each MG generates energy management strategy based on the proposed model, load/generation data, prices, and constraints. The objective function is given by (9) and includes CDG generation cost, RDG generation cost, prices of buying/selling power from/to the grid, and penalty prices of SL/NSL curtailments. The penalty price of curtailment of the SL is considered higher than that of the NSL. Therefore, in case of load shedding, only a part of NSL will be shed, and the SL will get continuous supply.

$$\min \quad \sum\_{i=1}^{I} \sum\_{t=1}^{T} (\mathbf{C}\_{i,t}^{\text{CDG}} \cdot \mathbf{P}\_{i,t}^{\text{CDG}} + \mathbf{C}\_{i,t}^{\text{BDG}} \cdot \mathbf{P}\_{i,t}^{\text{BDG}}) + \sum\_{t=1}^{T} (\mathbf{PR}\_{t}^{\text{Buy}} \cdot \mathbf{P}\_{t}^{\text{Def}} + \mathbf{PR}\_{t}^{\text{Sell}} \cdot \mathbf{P}\_{t}^{\text{Sir}}) + \sum\_{t=1}^{T} (\mathbf{C}\_{t}^{\text{PenSL}} \cdot \mathbf{P}\_{t}^{\text{SheSL}} + \mathbf{C}\_{t}^{\text{PenNSL}} \cdot \mathbf{P}\_{t}^{\text{SkelNSL}}), \quad \text{(9)}\\\min \quad \sum\_{i=1}^{I} \sum\_{t=1}^{T} (\mathbf{P}\_{i,t}^{\text{Buy}} \cdot \mathbf{P}\_{i,t}^{\text{D}} + \mathbf{P}\_{i,t}^{\text{D}} \cdot \mathbf{P}\_{i,t}^{\text{D}}) + \sum\_{t=1}^{T} (\mathbf{P}\_{i,t}^{\text{D}} \cdot \mathbf{P}\_{t}^{\text{D}}) + \sum\_{t=1}^{T} (\mathbf{P}\_{i,t}^{\text{D}} \cdot \mathbf{P}\_{t}^{\text{Sph}}) + \sum\_{t=1}^{T} (\mathbf{P}\_{i,t}^{\text{D}} \cdot \mathbf{P}\_{t}^{\text{Sph}}) + \sum\_{t=1}^{T} (\mathbf{P}\_{i,t}^{\text{D}} \cdot \mathbf{P}\_{t}^{\text{Sph}}) + \sum\_{t=1}^{T} (\mathbf{P}\_{i,t}^{\text{D}} \cdot \mathbf{P}\_{t}^{\text{Sph}})$$

where *P* is respective power value being multiplied with corresponding cost/price (i.e., *C* = cost and *PR* = price) for time from *t* = 1 to *t* = *T* and for DGs (CDGs and RDGs) from *i* = 1 to *i* = *I* in an MG.

Subject to:

$$\mathbf{c}\_{i,t} \cdot \mathbf{P}\_{\mathrm{i}}^{\mathrm{CDG}\_{\mathrm{min}}} \le P\_{i,t}^{\mathrm{CDG}} \le \mathbf{c}\_{i,t} \cdot \mathbf{P}\_{\mathrm{i}}^{\mathrm{CDG}\_{\mathrm{max}}} \; \forall \; i \in \mathbf{I}, t \in \mathbf{T}\_{\prime} \tag{10}$$

$$P\_t^{PV} + P\_t^{WT} + \sum\_{i=1}^{I} P\_{i,t}^{CDG} + P\_t^{Def} + P\_t^{B-} = P\_t^L + P\_t^{Sur} + P\_t^{B+} \quad \forall \ t \in \mathcal{T}\_\prime \tag{11}$$

$$0 \le P\_t^{B+} \le P\_B^{Cap} \cdot \frac{1 - SOC\_{t-1}^B}{\eta\_B} \; \forall \; t \in T\_\prime \tag{12}$$

$$0 \le P\_t^{B-} \le P\_B^{Cap} \cdot \text{SOC}\_{t-1}^B \cdot \eta\_B \; \forall \; t \in T,\tag{13}$$

$$\text{SOC}\_{t}^{B} = \text{SOC}\_{t-1}^{B} - \frac{1}{P\_{B}^{Cap}} \cdot \left(\frac{P\_{t}^{B-}}{\eta\_{B}} - P\_{t}^{B+} \cdot \eta\_{B}\right)\_{t} \tag{14}$$

$$0 \le \text{SOC}\_t^B \le 1 \; \forall \; t \in T. \tag{15}$$

In (10), the inequality constraint related to the generation of CDG unit *'i'* is given, which shows that the output of a CDG is between its minimum and maximum generating capacities. The power generated by all DGs units, deficit power, and power from battery must be adjusted with demand, surplus power, and power to the battery at each interval and is represented by (11). During charging, the storage system is considered as a load, and it acts as a source during the discharging of power. The constraints of the BESS of an MG, which include the charging, discharging, and SOC, are given by (12)–(15). The SOC is updated in each interval.

In single grid-tied mode, the MG adjusts surplus/deficit power by trading with the main grid, whereas, in the case of single islanded mode, the surplus power is balanced by ramping down the dispatchable generating units. Likewise, a load curtailment mechanism is employed for balancing the deficient amount of power. If the MG network is islanded or grid-tied MMGs system, after finalizing local optimization by every single MG-EMS at the first step, every MG-EMS delivers the information to the CEMS agent about the surplus and deficient amount of powers calculated from the local optimization algorithm.

#### *3.2. Global Optimization*

After being informed by each MG-EMS about surplus/deficit powers and connection/disconnection of that MG with the grid in the MMGs network, the CEMS carries out the global optimization in this step. The objective function of the MMGs network is given by (16). It contains buying-selling energy prices of all the MGs and the grid. In this paper, the CBESS is considered in subservient mode in that it will act under the command of the CEMS, instead of acting in autonomous mode. Therefore, the costs of charging and discharging of CBESS are not added in the objective function.

$$\min \sum\_{k=1}^{K} \sum\_{t=1}^{T} (PR\_{k,t}^{Rec} \cdot P\_{k,t}^{Rec} + PR\_{k,t}^{Send} \cdot P\_{k,t}^{Send}) + \sum\_{k=1}^{K} \sum\_{t=1}^{T} (PR\_{t}^{Buy} \cdot P\_{k,t}^{Buy} + PR\_{t}^{Sell} \cdot P\_{k,t}^{Sell}), \tag{16}$$

where *P* is respective power value being multiplied with corresponding price (i.e., *PR* = price) for time from *t* = 1 to *t* = *T* and for the MGs from *k* = 1 to *k* = *K* participating in the MMG system at that time.

Subject to:

$$\sum\_{k=1}^{K} P\_{k,t}^{\text{Rec}} + \sum\_{k=1}^{K} P\_{k,t}^{\text{Buy}} + \sum\_{k=1}^{K} P\_{k,t}^{\text{Sur}} + P\_t^{\text{CB}-} = \sum\_{k=1}^{K} P\_{k,t}^{\text{Send}} + \sum\_{k=1}^{K} P\_{k,t}^{\text{Sell}} + \sum\_{k=1}^{K} P\_{k,t}^{\text{Def}} + P\_t^{\text{Ce}} + \forall \ t \in \mathsf{T}, \quad \text{(17)}$$
 
$$\therefore \quad 1 - \mathsf{S} \mathsf{O} \mathsf{C}^{\text{CB}}$$

$$0 \le P\_t^{CB+} \le P\_{CB}^{Cap} \cdot \frac{1 - \text{SOC}\_{t-1}^{CB}}{\eta\_{CB}} \; \forall \; t \in T,\tag{18}$$

$$0 \le P\_t^{CB-} \le P\_{CB}^{Cap} \cdot \text{SOC}\_{t-1}^{CB} \cdot \eta\_{CB} \; \forall \; t \in \mathcal{T}\_\prime \tag{19}$$

$$\text{SOC}\_{t}^{\text{CB}} = \text{SOC}\_{t-1}^{\text{CB}} - \frac{1}{P\_{\text{CB}}^{\text{Cap}}} \left( \frac{P\_{t}^{\text{CB}-}}{\eta\_{\text{CB}}} - P\_{t}^{\text{CB}+} \cdot \eta\_{\text{CB}} \right)\_{\text{'}} \tag{20}$$

$$0 \le SOC\_t^{\mathcal{C}B} \le 1 \; \forall \; t \in \mathcal{T}. \tag{21}$$

The power bought from the neighboring MGs and the main grid, surplus power from each MG, and the CBESS discharging must be adjusted with the amount of sold power to the neighboring MG and the grid, deficit from each MG, and the CBESS charging demand as represented in (17). The limitations related to the CBESS charging, discharging, and the SOC are given by (18)–(21). In this cooperative model, selling/buying power to/from other MGs of the MMGs system is preferred. If further power is required, trading with the CBESS will be performed. If it is grid-tied MMGs system, the last option will be to trade with grid if required. Similarly, an MG with surplus energy after fulfilling its own demand and charging its BESS will sell power to other deficit MGs, charge the CBESS, and will sell power to the grid in the end. To make this strategy possible, the prices are introduced interval-wise, which will help the algorithm to make a quick, and optimized decision.

$$\min \sum\_{i=1}^{I} \sum\_{t=1}^{T} (\mathbf{C}\_{i,t}^{\text{CDG}} \cdot \mathbf{P}\_{i,t}^{\text{CDG}} + \mathbf{C}\_{i,t}^{\text{BDG}} \cdot \mathbf{P}\_{i,t}^{\text{RDG}}) + \sum\_{t=1}^{T} (\mathbf{PR}\_{t}^{\text{Rec}} \cdot \mathbf{P}\_{t}^{\text{Rec}} + \mathbf{PR}\_{t}^{\text{Smd}} \cdot \mathbf{P}\_{t}^{\text{Smd}}) + \sum\_{t=1}^{T} (\mathbf{PR}\_{t}^{\text{Buy}} \cdot \mathbf{P}\_{t}^{\text{Buy}} + \mathbf{PR}\_{t}^{\text{Sfd}} \cdot \mathbf{P}\_{t}^{\text{Sfd}}), \tag{22}$$

subject to:

$$\mathbf{P}\_{l}^{PV} + \mathbf{P}\_{l}^{WT} + \sum\_{i=1}^{I} \mathbf{P}\_{i,l}^{CDG} + \mathbf{P}\_{l}^{Buy} + \mathbf{P}\_{l}^{Rx} + \mathbf{P}\_{l}^{B-} + \mathbf{P}\_{l}^{CB-} = \mathbf{P}\_{l}^{L} + \mathbf{P}\_{l}^{Sell} + \mathbf{P}\_{l}^{Snd} + \mathbf{P}\_{l}^{B+} + \mathbf{P}\_{l}^{CB+} \ \forall \, t \in \mathbf{T}. \tag{23}$$

In the proposed MMGs model, the CEMS notifies all MG-EMSs about their commitments and schedulings. Each MG-EMS reschedules its generation after global optimization via local optimization again (rescheduling). Now, the objective function of local optimization of an MG in MMGs network is (22). It includes the CDG generation cost, RDG generation cost, and prices of buying/selling power from/to other MGs and grid. The load balancing constraint is presented in (23). In addition to this constraint, objective function constraints (10)–(15) are also considered. The complete optimization procedure for the proposed MMGs network is illustrated in Figure 3.

**Figure 3.** Flowchart of the proposed methodology.

#### **4. Case Study**

The proposed scheme has been verified on the IEEE 33-bus distribution system. The detailed specifications of the test system can be found in Reference [21]. We have modified the test system and constructed four autonomous MGs, as shown in Figure 4. For MCS, the wind speed and solar irradiance data of four seasonal data of six years (2007–2012) and seventeen years (1998–2014) have been taken [22,23].

**Figure 4.** Modified IEEE 33-bus distribution system with four MGs.

Four different load curves, presented in Figure 5a, are used as daily load demands of MGs of the modified test system. The load curves of NYISO-CAPITAL zone, NYISO-N.Y.C. zone, ERCOT, and ISO-NE are used as demands of MG1, MG2, MG3, and MG4, respectively [24]. The parameters of a PV panel and a WT are same as considered in Reference [25]. Each MG has its own CDG, PV, WT, BESS, and EMS. Figure 5b shows the daily forecasted curves of output percentage of solar and wind DGs. The following three scenarios have been studied for the simulations. In Scenario 1, all four MGs are participating in the MMGs network; MG2 and MG3 are grid-tied. Only MG2 and MG3 can trade energy with the grid after sending or receiving power from other MGs. In Scenario 2, three MGs are participating in the MMGs grid-tied network, while the MG1 is in single islanded MG mode. In Scenario 3, three MGs are participating in MMGs grid-tied network, while MG3 is in single islanded MG mode.

**Figure 5.** (**a**) Fixed load profile of the different MGs and (**b**) daily forecasted curve of output percentage of solar and wind DG.

It can be observed that there can be many possible combinations of the MGs. In the proposed model, an MG can be in State 1: islanded single MG entity; State 2: grid-tied single MG; State 3: islanded MMGs system or State 4: grid-tied MMGs system, as shown in Figure 6a. Depending on these four possible states of an MG, there are total 256 cases for our test system. In order to verify the robustness of the scheme, 10% of the total cases are generated by MCS, and their details are provided in the next section. To understand the connection of MGs in different states, Cases 07 and 11 from Table 1 are illustrated in Figure 6b,c, respectively. State 1 refers to a single islanded entity, i.e., it has to fully depend on its resources, and no external source is available as MG4 in Figure 6b. In State 2, the MG is a single autonomous entity but has the grid as an external resource, as is MG1, in Figure 6c. In State 3, the MG is connected to other MGs that are also taking part in the MMG system. Other MGs may be connected with the grid depending on their respective states, but this MG is not connected to the grid, as depicted MG2 in Figure 6c, which is

connected with other MGs but has no connection with the grid. In State 4, the MG is also connected to the grid besides the connection with other MGs in the MMG system. The connections of other MGs with the grid depend on their respective states. For instance, MG3 in Figure 6b is in the MMG system with MG2, and both are also grid-connected, whereas MG3 in Figure 6c is in MMG system with MG2 and MG4; MG3 and MG4 are connected to the grid, but MG2 is not depending on their states.

**Figure 6.** (**a**) States of an MG, (**b**) States in case 07, and (**c**) States in case 11.



Note: PS = Power sold; PB = Power bought; LS = Load shed; C = Total optimal cost.

#### **5. Results Analysis**

#### *5.1. Proposed Scenarios*

Figures 7–9 show total generation, total demand, charged/discharged powers of the BESS, trading with grid and other MGs of the MMG system, and load curtailment of each MG. In the first scenario, as it can be seen in Figure 7a, MG1 has sold the surplus power from 5:00 a.m. to 7:00 a.m. to others deficit MGs. From 5:00 p.m. to 9:00 p.m., MG1 needed to discharge the BESS after utilizing the maximum capacity of its resources. In Figure 7b,c,

MG2 and MG3 were selling their surplus energy to the grid from 12:00 a.m. to 4:00 p.m. and utilizing the stored-BESS-energy from 5:00 p.m. to 9:00 p.m. MG2 was charging the BESS from 10:00 p.m. to 11:00 p.m., but MG3 was still deficient in energy, and its BESS was completely discharged already, and no neighboring MG had energy to sell the power. Therefore, it was receiving the discharged power from the CBESS for these two hours. MG4 was considered with the demand more than the generation to study the behavior and impact of the proposed scheme for worst-case scenario of any practical MG. In Figure 7d, from 12:00 a.m. to 4:00 a.m., the demand was being fulfilled by the storage system, which was completely discharged at 5:00 a.m. Therefore, it had to buy the energy from other MGs from 5:00 a.m. to 7:00 a.m. Later, it started charging the storage system from 8:00 a.m. to 12:00 p.m.; at 12:00 p.m., it was completely charged. From 4:00 p.m. to 7:00 p.m., the local energy storage was completely utilized, and no other MG had extra energy to sell. As a result, from 7:00 p.m. to 11:00 p.m., MG4 was using the discharged energy of the CBESS, and there was no load shedding in the whole day. This is the benefit of the proposed MMG system.

**Figure 7.** Scenario 1: (**a**) MG1, (**b**) MG2, (**c**) MG3, (**d**) MG4.

In the Scenario 2, MG1 was the MG which had capacity to meet the demand by its own without buying any energy. This behavior can be seen in Figure 8a. Even though MG1 was in single MG islanded mode, there was no load shedding during the whole day, and it was in the most optimized form. The behavior of MG2 was same as in Scenario 1; the only difference was: it was selling the energy to MG4 as in Figure 8b. The behavior of MG3 was same in Figure 8c as in the previous scenario. In this scenario, MG4 was grid-connected, Therefore, its behavior was slightly different from Scenario 1, i.e., it was selling surplus energy to the grid from 12:00 p.m. to 3:00 p.m. in Figure 8d.

**Figure 8.** Scenario 2: (**a**) MG1, (**b**) MG2, (**c**) MG3, (**d**) MG4.

In Scenario 3, MG1 was grid-tied similar to MG2 and MG4, it was selling the surplus power to the grid in Figure 9a. Figure 9b,d show behaviors of MG2 and MG4, which were identical to Scenarios 1 and 2, respectively. MG3 was successfully meeting its demand from 12:00 a.m. to 9:00 p.m. by its own resources in Figure 9c, and there was a little amount of load curtailment from 10:00 p.m. to 11:00 p.m. of the NSL, which shows that the MG is reliable and providing uninterruptible energy to the SL.

**Figure 9.** Scenario 3: (**a**) MG1, (**b**) MG2, (**c**) MG3, (**d**) MG4.

Figure 10a compares these three scenarios for the sum of load shedding in four MGs hour-wise. In Scenarios 1 and 2, there was no load shedding, whereas, in Scenario 3, there was a small load curtailment from 10:00 p.m. to 11:00 p.m. Figure 10b compares all scenarios for the sum of total optimal cost of all MGs hour-wise. It can be seen that the costs in all scenarios were nearly the same because, in each scenario, the proposed algorithm took the decision considering minimum cost. Therefore, all three scenarios were having the most optimized costs at each hour.

**Figure 10.** (**a**) Total Load shedding in all MGs and (**b**) total optimal cost of all MGs.

#### *5.2. Cases Generated by MCS*

The 25 cases were generated by MCS, and their respective results per day for each MG are presented in Table 1.

MG1 and MG2 were modeled in such a way that, most of the time, both MGs had surplus energy and never depended on external resources, whereas MG3 and MG4 were modeled in such a way that they were frequently independent of external resources, but, for a few hours, they were deficient in energy. This type of modeling revealed the effectiveness of the proposed strategy and the stability of the presented system model. In Table 1, it can be seen that the MGs in State 4, the MMG grid-connected state, were the most optimized and reliable with the minimum total cost and maximum profit. After meeting their demands, the MGs were selling/buying energy to/from other MGs, charging or discharging the CBESS, and later selling/buying energy to/from the grid. As MG1 and MG2 were microgrids with surplus energy most of the time, they were earning maximum profit in this state by selling the energy to other deficit MGs, charging the CBESS, and selling the energy to the grid. Their buying energy and load shedding were zero for all cases. In State 4, MG3 and MG4 were buying the deficit energy from MG1 and MG2 for few hours if they were in the MMG system, discharging the CBESS, and buying from

the grid. In State 4, MG3 and MG4 were selling power to the MGs, charging the CBESS, or selling energy to the grid for rest of the hours, thus also earning the profit, and there was again no load shedding in these two MGs for State 4. In State 3, all MGs could only sell/buy energy to/from other MGs and charge/discharge the CBESS, and there was no trading with the grid. Again, the MG with maximum sold energy earned maximum profit, which is ultimately making its total cost minimum. The MG with buying energy is also in profit because there is no curtailment of the load, which would increase total cost with its penalty price. Thus, it proved that the proposed MMGs scheme is beneficial for each MG, either it has more or less generation capacity than the total demand, making it reliable and curtailment-free MG. In State 2, the MGs were only connected to the grid. They could neither sell/buy energy to/from other MGs nor charge/discharge the CBESS. The MGs could sell power to the grid in surplus energy hours and buy in deficit energy hours. We can see in Table 1 that, for respective states, the total cost of the MGs was nearly equal to or slightly more than that in previous states. In State 1, the MGs had only to depend on their own resources, i.e., they were single islanded entities. MG1 and MG2 in this state were fulfilling their respective demands, whereas MG3 and MG4 needed energy for few hours. No option was available to buy the power, thus having to shed the load. Now, the algorithm preferred to shed the NSL and had provided continuous supply to the SL. In this state, the MGs had more total cost than that in all three previous states. They were still reliable because there was no curtailment in the SL which would maximize the total cost, and there would be no use of energy management scheme. It can be clearly observed from Table 1 that the participation of a large number of MGs in the MMG system will be profitable for each MG, and there will be minimum local and global total costs as in Case 1. If the lesser number of MGs will participate in the MMG system, the lesser the profit will be, and the local and global total cost will increase, as in Case 25.

#### **6. Conclusions**

This paper proposed a novel energy management strategy for the day-ahead scheduling of the MMG systems. Four MGs were considered and optimized locally and globally. Each MG was fully connected with all other MGs in the MMG system. The MG-EMS performed local optimization and, after satisfying its local demand, updated the CEMS about its surplus and deficient amount of energy, thus preserving its privacy from other MGs. Only the CEMS took decisions about the power exchange between the MGs or with the grid by considering the state of the MG, generation and trading prices. As it was a two-level management strategy, thus, it increased privacy and resilience of the network. Various test cases were generated by MCS for all possible connections of the MGs. The study was performed on the IEEE 33-bus distribution system, and it has been proven that the proposed strategy and the algorithm are able to satisfy the demands of all MGs optimally. The numerical outcomes confirmed that the proposed methodology is compatible and can be executed effectively for practical microgrid applications. In summary, the proposed method can be utilized to make the management of the MMGs easy for real-world power systems, which will eventually transform the conventional power systems.

**Author Contributions:** Conceptualization, software, methodology, literature analysis, modeling, simulations, and writing—original draft, K.N.; writing—editing and literature analysis, F.Z.; supervision, editing and resources, K.K.M.; review, S.B.A.B.; supervision and review, H.A.K.; funding acquisition and review, C.-H.K. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was financed by the National Research Foundation of Korea (NRF) grant funded by the Korean Government (MSIP), Grant No. 2021R1A2B5B03086257.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** Data sharing not applicable.

**Conflicts of Interest:** The authors declare no conflict of interest.

### **Nomenclature**


#### **References**


## *Article* **A Novel Mean Field Game-Based Strategy for Charging Electric Vehicles in Solar Powered Parking Lots**

**Samuel M. Muhindo 1,2,3,**<sup>∗</sup> **, Roland P. Malhamé 1,2,3 and Geza Joos 3,4**


<sup>3</sup> Réseau Québécois sur l'Énergie Intelligente (RQEI), Trois-Rivières, QC G9A 5H7, Canada; geza.joos@mcgill.ca

<sup>4</sup> Department of Electrical and Computer Engineering, McGill University, Montreal, QC H3A 0E9, Canada

**\*** Correspondence: samuel.muhindo-mugisho@polymtl.ca

**Abstract:** We develop a strategy, with concepts from Mean Field Games (MFG), to coordinate the charging of a large population of battery electric vehicles (BEVs) in a parking lot powered by solar energy and managed by an aggregator. A yearly parking fee is charged for each BEV irrespective of the amount of energy extracted. The goal is to share the energy available so as to minimize the standard deviation (STD) of the state of charge (SOC) of batteries when the BEVs are leaving the parking lot, while maintaining some fairness and decentralization criteria. The MFG charging laws correspond to the Nash equilibrium induced by quadratic cost functions based on an inverse Nash equilibrium concept and designed to favor the batteries with the lower SOCs upon arrival. While the MFG charging laws are strictly decentralized, they guarantee that a mean of instantaneous charging powers to the BEVs follows a trajectory based on the solar energy forecast for the day. That day ahead forecast is broadcasted to the BEVs which then gauge the necessary SOC upon leaving their home. We illustrate the advantages of the MFG strategy for the case of a typical sunny day and a typical cloudy day when compared to more straightforward strategies: first come first full/serve and equal sharing. The behavior of the charging strategies is contrasted under conditions of random arrivals and random departures of the BEVs in the parking lot.

**Keywords:** battery electric vehicle; mean field games; nash equilibrium; parking lot; solar energy

## **1. Introduction**

The massive introduction of battery electric vehicles (BEVs) [1–3] in modern power systems is bound to have important impacts, positive or negative, depending on the way this novel situation is managed [4]. There will be a great pressure to introduce numerous charging stations where the need is anticipated, but if too many high speed charging BEVs are connected at any one time (for example upon departure from work towards residence place), that may create both local transformer and eventually system wide overloads [5]. Many works in the literature [6–10] present algorithms for an optimal scheduling of vehicleto-grid (V2G). The authors in [6] propose a centralized algorithm based on reinforcement learning which reduces the total power grid load variance by 65% in a test scenario of 300 consecutive days by charging 50 homogeneous BEVs in each hourly time slot in a neighborhood of 250 households. The authors in [7] propose a centralized weighted fair queuing (WFQ) algorithm with a 5 min time slot control switch in each smart charger to charge 300 homogeneous BEVs, by favoring those arriving with less charge. The algorithm selects a subset of BEVs to charge in each interval during peak demand when there is not enough energy. They compare the results with a first come first serve (FCFS) algorithm. They show that when the supply demand ratio (SDR) is equal to 1, there is 5% of BEVs

**Citation:** Muhindo, S.M.; Malhamé R.P.; Joos, G. A Novel Mean Field Game-Based Strategy for Charging Electric Vehicles in Solar Powered Parking Lots. *Energies* **2021**, *14*, 8517. https://doi.org/10.3390/en14248517

Academic Editors: Victor Becerra and Ahmed Rachid

Received: 19 November 2021 Accepted: 11 December 2021 Published: 17 December 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

which cannot leave their homes on time while it is 7% for FCFS. Recently, the authors in [10] study the case where the maximum charging power depends on the state of charge of the BEV's battery. They propose a centralized mixed integer linear programming (MILP) algorithm to charge a fluctuating heterogeneous population of BEVs at a single station considering availability of each BEV in order to minimize time-dependent electricity costs.

On the other hand, adequate management of the battery storage associated with an aggregate of BEVs can turn such an aggregate into a virtual power plant. Thus, in a context of integration with clean sources of energy, such as photovoltaics, despite other energy harvesting and storage techniques in the literature [11,12], BEVs' batteries could be storing the solar energy available during the day when BEVs are parked [13–16]. As is well known in the photovoltaics rich state of California for example, a so-called power demand *duck curve* [17] is observed: the peak demand occurs at the end of the day, upon return of working people to their homes. At that point, available solar radiation has all but disappeared and while solar energy may have been used by consumers during the day, there is a need for a high electric power ramp at dusk followed by several hours of sustained high power consumption. The latter power demand will be most likely met by fossil based energy sources, unless some other mitigating actions are taken. In [18,19], the authors show in their geographic context, that if the electric energy storage contained in a large number of BEVs is properly utilized, this could help significantly reduce the power needed from fossil sources during the evening peak.

The authors in [20], whose objective is close to ours in this paper, propose a centralized linear programming (LP) algorithm, in a solar powered parking lot of a car-share service to fairly distribute the available solar energy amongst 97 heterogeneous BEVs by favoring those arriving with less charge. They study the case where the SDR is strictly inferior to 1, and that all BEVs are available during the daily charging session of 5 h in the parking lot. They demonstrate, by charging a subset of 5 BEVs during each time slot, a reduction of 60% of yearly average standard deviation in the battery charge levels at the end of recharging compared to the equal sharing (ES) approach. The authors in [6,7,10,20] do not propose a decentralized algorithm. A decentralized algorithm scheme allows individual BEVs to determine their own charging pattern. Their decisions could, for example, be made on the basis of time-of-day, electricity price or battery state of health [21,22].

Table 1 below places our work in the BEVs charging optimization when the aggregators are parking lot operators (PLOs) or distribution system operators (DSOs).


**Table 1.** Classification of related work.

Our objective in this paper is to propose an algorithm for sharing solar photovoltaic (PV) power amongst homogeneous BEVs parked in a parking lot, or a collection of federated parking lots. The BEVs belong to commuters working in the neighborhoods of these parking lots and could recharge at least partially depending on sunshine availability, their batteries at the parking lot charging stations. One particular business model is that the parking lots aggregator would charge a yearly fee for use of a parking space and the associated charging station. In a potential extension of the business model, if the BEVs' owners wish to recuperate part of their parking costs, they could choose to participate in a financially compensated grid support operation coordinated by the aggregator of the parking lots. In that case, *it would be in the interest of the aggregator to equalize charging of BEVs to maximize the probability of vehicle-to-grid charging participation*.

We suggest relying on an adequately tailored variation of a Mean Field Game-based algorithm scheme [23,24] which, while it fills all BEVs simultaneously, tends to provide more instantaneous charging to the BEVs with the lowest current fill levels. The problem is

formulated as large population game on a finite charging interval. In [21], the authors study the existence, uniqueness and optimality of the Nash equilibrium of the charging problems to minimize local electricity costs and to fully charge. In a decentralized computational mechanism, they show in a deterministic case that the large population charging games will converge to a unique Nash equilibrium which is globally optimal for a homogeneous population.

In what follows we shall present a novel Mean Field Game-based charging algorithm to calculate the operator broadcasted decentralized algorithm laws according to the potential solar energy available. Subsequently, the performance of these laws will be compared to that of two common algorithms used in the literature. The first algorithm, *first come first full* (which, in a dynamic context, will be upgraded to *first come first serve*), consists of recharging *maximally* (or *up to an adequately updated SOC in real time*) the BEVs in order of arrivals at the parking lot. The second algorithm, *equal sharing*, consists of sharing equally at all times the available solar power amongst battery BEVs still not fully charged. All algorithms make full use of the available daily energy. Furthermore, for the purpose of meaningfully comparing the performance of the different algorithms in our case studies, we assume that the SDR is less than one.

In order to implement the charging algorithms, we make the following assumptions:


The rest of this paper is organized as follows. In Section 2, we present the theoretical underpinnings and details of the MFG-based algorithm in the case of *homogeneous* BEVs. In Section 3, we present the numerical results *assuming a fixed population of BEVs which are charged in the parking lot simultaneously* with common characteristics of batteries. In Section 4, we present the algorithmic modifications and the numerical results in a more realistic situation *where BEVs arrive and depart randomly in the parking lot*. Finally, in Section 5, we conclude and give an outlook on future research.

#### **2. Mean Field Game-Based Control of a Large Population of BEVs**

#### *2.1. Battery Model*

We consider a population of *n* homogeneous BEVs in a parking lot. The assumption of a *large* population is needed only if, as we do in (1) below, we assume randomness in the dynamics of battery charging and later on in our analysis we will assimilate the empirical mean of SOCs with its mathematical expectation (a predictable deterministic quantity) by virtue of *the law of large numbers*. Because of the linearity of the model, the analysis will be perfectly exact for arbitrarily small numbers of BEVs if the battery charging processes remain deterministic. Each BEV *i*, *i* = 1, . . . , *n*, has a SOC *xi*,*arr* upon arrival which results of a daily traffic pattern from home to parking lot. We can then write the SOC stochastic dynamics for BEV *i* as follows [21] :

$$d\mathbf{x}\_{i,t} = bu\_{i,t}dt + \nu d\omega\_{i\nu} \tag{1}$$

where *t* ∈ [*t*0, *T*] is time in hours (*h*), *xi*,*<sup>t</sup>* is the SOC in per unit (pu) of capacity, *b* = *<sup>α</sup> β* , *<sup>α</sup>* <sup>∈</sup> (0, 1] is the charger efficiency in pu/*h*, *<sup>β</sup>* is the battery capacity in *kWh*, *<sup>u</sup>i*,*<sup>t</sup>* <sup>∈</sup> <sup>R</sup><sup>+</sup> is the charging rate in *kW*, *ω<sup>i</sup>* is a normalized Brownian process, *ν* is the intensity of that Brownian noise and *ω<sup>i</sup>* is assumed independent of *ω<sup>j</sup>* for *i* 6= *j*. The term *νdω<sup>i</sup>* defines the *stochasticity* of the SOC which can result physically from fluctuations in the charging and losses of the battery. We first present the algorithm in the simple case where all BEVs are started charging at the same time and depart at the end of control horizon, i.e., *t*<sup>0</sup> = *tbegin* and *T* = *tend*. Then the algorithm will be updated in a dynamic context, where [*t*0, *T*] will

be made up of successive fixed short control horizons representing random arrivals and random departures of BEVs between *tbegin* and *tend*.

#### *2.2. Considerations*

The charging algorithms tested in this paper will be compared with respect to the following requirements:

#### 2.2.1. Fairness

We develop here our notion of fairness. While it is very hard to realize in practice, we wish that BEVs have near identical SOCs at the time they leave the parking lot irrespective of the time they arrived or their SOC at arrival. Furthermore, for BEVs that are simultaneously present in the parking lot, at no time should the BEV that arrived with a lower SOC be allowed to have its SOC exceed that of the other BEV. Such a criterion will lead us to the definition of a fairness coefficient by means of which we shall contrast the performance of various charging algorithms. We aim at:


This leads us to the definition of a fairness coefficient (FC) that will help compare the fairness of the various charging strategies.


We suggest using the following expression:

$$\text{FC} = \frac{\sigma\_{\text{X}\_{i, \text{arr}}} - \sigma\_{\text{X}\_{i, \text{dep}}}}{\sigma\_{\text{X}\_{i, \text{arr}}}} \ e^{-\eta} \text{, where } \eta = \frac{N\_{\text{O}}}{\binom{N}{2}} = \frac{2 \, N\_{\text{O}} \, (N-2)!}{N!}. \tag{2}$$


#### 2.2.2. Decentralization

From the point of view of the parking lot operator, decentralized charging algorithm laws are quite desirable because they minimize the need to observe the state of charge of individual batteries, a process which is both complex and invasive. Furthermore, a local algorithm allows a user to interrupt the process at any time, particularly if the parking operator has designed a charging scheme based on a poor model of the battery.

We wish to address the decentralized control of battery recharging of a set of BEVs as part of a so-called Mean Field Game (MFG). The control will be of linear-quadratic (LQ) type [25,26]. The parking lot operator broadcasts an average SOC target trajectory (*x target t* ) based on the solar energy forecast for the current day. The goal is that the BEVs store up as much of the solar energy available as possible and yet share the energy in a manner which will be deemed as fair. The proposed algorithm requires that the parking lot operator know the average SOC of the BEVs upon arrival (*x*0). This can be achieved by recording initial SOCs as BEVs enter the parking lot.

The key point of the approach is the prescription of a daily quadratic cost *J<sup>i</sup>* for each BEV *i* to ensure that the BEVs that are initially fuller recharge less quickly than those that are less full, so that final SOC standard deviation of departing BEVs is reduced; while still maintaining the goal of using up the available solar power at all instants. More precisely the cost functions are designed so that by optimizing the individual BEV costs *J<sup>i</sup>* , one achieves the aggregator's goals (fairness and decentralization) while using all available solar energy in the parking lot.

#### *2.3. Establishment of Individual Battery Cost Function*

The battery cost function is a mathematical expectation. It is designed by the aggregator and defined for a BEV as follows:

$$J\_l(\mathbf{x}\_{i,0}, u\_{i,l}) = \mathbb{E}\left[\int\_{t\_0}^T e^{-\delta t} \left[\frac{q\_t^y}{2}(\mathbf{x}\_{i,t} - y)^2 + \frac{r}{2}u\_{i,t}^2 + \frac{q\_{\mathbf{x}\_0}}{2}(\mathbf{x}\_{i,t} - \mathbf{x}\_{i,0})^2\right] dt \mid \mathbf{x}\_{i,0}\right],\tag{3}$$

where E is the expectation operator, *t*<sup>0</sup> is the charging starting time, *T* is the charging stopping time, *δ* is a discount coefficient to ensure convergence of the cost, *y* is the collective direction of the BEVs' SOCs which is equal to 1 in our case (it serves as a direction signal to all BEVs, such that all BEVs should move toward *y* but not beyond), *r* is a coefficient which penalizes the level for charging rate, *qx*<sup>0</sup> is a pressure coefficient aimed at limiting the distance from the SOC *xi*,0 (for the state of health of the user's battery and fairness to others) and *q y t* is the pressure field trajectory. The latter is common to all BEVs and is numerically obtained as the solution of a system of differential equations. It is the key quantity which will drive all SOCs towards a full state of charge while sharing instantaneously available solar energy in a way that reduces the standard deviation of SOCs. Its computation is further detailed below and it is at the heart of our inverse Nash equilibrium procedure. Note that the class of quadratic cost functions has been frequently used in the MFG literature [23,25,27].

The BEVs will collectively recharge their batteries with a time dependent coefficient (*q y t* ) penalizing the gap between the current SOC (*xi*,*<sup>t</sup>* ) and the ultimate destination direction of SOCs defined by the value of *y*. Once the target for steady-state mean SOC of BEVs (*x target T* ) is reached, *q y <sup>t</sup>* will settle to a constant value, allowing the reaching of a mean SOC steady-state that meets the constraints set by the parking lot operator.

#### *2.4. Optimal Control Problem and Solutions*

We begin by making the following variable changes to simplify the expression of cost *Ji* in Equation (3):

$$\begin{aligned} X\_{i,t} &= (\mathbf{x}\_{i,t} - y)e^{-\delta t/2}, \mathbf{U}\_{i,t} = u\_{i,t}e^{-\delta t/2}, \\ V\_t &= \nu e^{-\delta t/2} \text{ and } Z\_{i,t} = (\mathbf{x}\_{i,t} - \mathbf{x}\_{i,0})e^{-\delta t/2}. \end{aligned} \tag{4}$$

Then:

$$\begin{split}dX\_{i,t} &= dx\_{i,t}e^{-\delta t/2} - \frac{\delta}{2} \mathbf{x}\_{i,t}e^{-\delta t/2}dt + \frac{\delta}{2}ye^{-\delta t/2}dt \\ &= (bu\_{i,t}dt + \nu d\omega\_{i})e^{-\delta t/2} - \frac{\delta}{2}(\mathbf{x}\_{i,t} - \mathbf{y})e^{-\delta t/2}dt \\ &= -\frac{\delta}{2}\mathbf{X}\_{i,t}dt + b\mathbf{U}\_{i,t}dt + V\_{t}d\omega\_{i}.\end{split} \tag{5}$$

The solution approach is based on assuming a quadratic form of the optimal cost function [26] (with coefficients *π*,*s* and *γ*):

$$J\_{i}^{\*}(X\_{i,t}) = \frac{1}{2}\pi\_{i,t}X\_{i,t}^{2} + s\_{i,t}X\_{i,t}e^{-\delta t/2} + \gamma\_{i,t} \tag{6}$$

We then write the dynamic programming equation corresponding to this guess [28].

$$\frac{\partial f\_{i}^{\*} }{\partial t} + \min\_{\mathcal{U}} \left\{ \frac{q\_{t}^{y}}{2} X\_{i,t}^{2} + \frac{q\_{x0}}{2} Z\_{i,t}^{2} + \frac{r}{2} \mathcal{U}\_{i,t}^{2} + \frac{\partial f\_{i}^{\*}}{\partial X} \left( -\frac{\delta}{2} X\_{i,t} + b \mathcal{U}\_{i,t} \right) + \frac{\partial^{2} f\_{i}^{\*}}{\partial X^{2}} \frac{V\_{t}^{2}}{2} \right\} = 0. \tag{7}$$

Differentiating with respect to *Ui*,*<sup>t</sup>* yields:

$$\begin{split} \nabla\_{\mathcal{U}} \{ \frac{q\_{\boldsymbol{t}}^{\mathcal{Y}}}{2} \mathbf{X}\_{\boldsymbol{i},\boldsymbol{t}}^{2} + \frac{q\_{\mathbf{x}\_{0}}}{2} \mathbf{Z}\_{\boldsymbol{i},\boldsymbol{t}}^{2} + \frac{r}{2} \mathbf{U}\_{\boldsymbol{i},\boldsymbol{t}}^{2} \} + \nabla\_{\mathcal{U}} \{ \frac{\partial \mathbf{J}\_{\boldsymbol{i}}^{\*}}{\partial \mathbf{X}} \left( -\frac{\delta}{2} \mathbf{X}\_{\boldsymbol{i},\boldsymbol{t}} + b \mathbf{U}\_{\boldsymbol{i},\boldsymbol{t}} \right) + \frac{\partial^{2} \mathbf{J}\_{\boldsymbol{i}}^{\*}}{\partial \mathbf{X}^{2}} \frac{\mathbf{V}\_{\boldsymbol{i}}^{2}}{2} \} &= \mathbf{0} \\ \longrightarrow \mathbf{U}\_{\boldsymbol{i},\boldsymbol{t}}^{\*} = -\frac{b}{r} \frac{\partial \mathbf{J}\_{\boldsymbol{i}}^{\*}}{\partial \mathbf{X}} .\end{split} \tag{8}$$

The second derivative with respect to *Ui*,*<sup>t</sup>* is *r* > 0, so the value found will indeed correspond to a minimum:

$$
\delta L\_{i,t}^\* = -\frac{b}{r} \left[ \pi\_{i,t} X\_{i,t} + s\_{i,t} e^{-\delta t/2} \right]. \tag{9}
$$

The optimal control therefore depends on the values *πi*,*<sup>t</sup>* and *si*,*<sup>t</sup>* . The expressions of *πi*,*<sup>t</sup>* and *si*,*<sup>t</sup>* are then determined by identification:

$$\begin{split} &\frac{1}{2}\mathbf{X}\_{i,t}^{2}\frac{d\pi\_{i,t}}{dt} + \mathbf{X}\_{i,t}e^{-\delta t/2}\left(\frac{ds\_{i,t}}{dt} - \frac{\delta}{2}s\_{i,t}\right) + \frac{d\gamma\_{i,t}}{dt} \\ &= -\frac{q\_{i}^{y}}{2}\mathbf{X}\_{i,t}^{2} - \frac{q\_{x\_{0}}}{2}\mathbf{Z}\_{i,t}^{2} + \frac{b^{2}}{2r}(\pi\_{i,t}^{2}\mathbf{X}\_{i,t}^{2} + 2\mathbf{X}\_{i,t}\pi\_{i,t}s\_{i,t}e^{-\delta t/2} + s\_{i,t}^{2}e^{-\delta t}) \\ &+ \frac{\delta}{2}\mathbf{X}\_{i,t}(\pi\_{i,t}\mathbf{X}\_{i,t} + s\_{i,t}e^{-\delta t/2}) - \frac{V\_{t}^{2}}{2}\pi\_{i,t} \\ &= \mathbf{X}\_{i,t}^{2}(\frac{b^{2}}{2r}\pi\_{i,t}^{2} + \frac{\delta}{2}\pi\_{i,t} - \frac{q\_{t}^{y}}{2} - \frac{q\_{x\_{0}}}{2}) + \mathbf{X}\_{i,t}e^{-\delta t/2}(\frac{b^{2}}{r}\pi\_{i,t}s\_{i,t} + \frac{\delta}{2}s\_{i,t} - yq\_{x\_{0}} + \mathbf{x}\_{i,0}q\_{x\_{0}}) + \dots \end{split} \tag{10}$$

The analysis results in the following system of differential equations that must be solved backwards:

$$\begin{aligned} \frac{d\pi\_{i,t}}{dt} &= \frac{b^2}{r}\pi\_{i,t}^2 + \delta\pi\_{i,t} - q\_t^y - q\_{\ge\_{0'}}\\ \frac{ds\_{i,t}}{dt} &= (\delta + \frac{b^2}{r}\pi\_{i,t})s\_{i,t} + q\_{\ge\_0}(\pi\_{i,0} - y), \end{aligned} \tag{11}$$

and the optimal control law is given by:

$$u\_{i,t}^{\*} = -\frac{b}{r} \left[ \pi\_{i,t}(\mathbf{x}\_{i,t} - y) + s\_{i,t} \right]. \tag{12}$$

The coefficient *q y t* appearing in the differential equation of *π* is unknown at this stage. Nonetheless, it must respect the fact that when the BEVs use the optimal control *u* ∗ *i*,*t* , their empirical average trajectory (*xt*), assimilated thanks to the law of large numbers to the mathematical expectation of the SOC of a generic BEV E[*xi*,*<sup>t</sup>* ] , will correspond to the average target trajectory (*x target t* ) imposed by the parking lot operator. The charging strategy that is developed, relies on knowing the anticipated solar energy during the day. Based on the dynamics of the SOC *xi*,*<sup>t</sup>* in Equation (1), the average target *x target t* needed is determined by integrating the curve of the forecast solar power (*uW<sup>t</sup>* ) over the time [*t*0, *T*] of interest and dividing, for a case of homogeneous battery capacities, by the total number of BEVs present in the parking lot for recharging. The total forecast solar energy *W* available in the parking lot is assumed less than the total energy that all BEVs would need to fully recharge their batteries.

#### *2.5. Calculation of q<sup>y</sup> t by Nash Equilibrium Inversion*

We calculate the pressure field *q y t* directly by numerical resolution of differential equations. The system of differential equations to be solved is obtained by imposing that under the action of *q y t* , and the associated optimal control law in Equation (12), the average trajectory *x<sup>t</sup>* of the BEVs follows the average target trajectory *x target t* broadcasted by the parking lot operator. This restriction allows us to write the Mean Field equations based on taking the mathematical expectation of the SOC of a generic battery in the population subject to decentralized control law (Equation (12)).

$$\begin{aligned} \frac{d\overline{\boldsymbol{x}}\_{t}^{\text{target}}}{dt} &= b\overline{\boldsymbol{u}}\_{t}^{\*} = -\frac{b^{2}}{r} [\boldsymbol{\pi}\_{t}(\overline{\boldsymbol{x}}\_{t}^{\text{target}} - \boldsymbol{y}) + \overline{\boldsymbol{s}}\_{t}], \\ \frac{d\boldsymbol{\pi}\_{t}}{dt} &= \frac{b^{2}}{r} \boldsymbol{\pi}\_{t}^{2} + \delta \boldsymbol{\pi}\_{t} - q\_{t}^{y} - q\_{\mathbf{x}\_{0}}, \\ \frac{d\overline{\boldsymbol{s}}\_{t}}{dt} &= (\delta + \frac{b^{2}}{r} \boldsymbol{\pi}\_{t}) \overline{\boldsymbol{s}}\_{t} + q\_{\mathbf{x}\_{0}}(\overline{\boldsymbol{x}}\_{0} - \boldsymbol{y}), \end{aligned} \tag{13}$$

where *π<sup>t</sup>* = *πi*,*<sup>t</sup>* and *s<sup>t</sup>* = ∑ *n i*=1 *si*,*t n* . With this approach, the goal is to obtain a mathematical relationship between *dx target t dt* and *q y t* . This is the so-called *inverse Nash algorithm*, its first steps were developed in the control of electric space heaters [27].

With the first equation of (13), we can write the relation between *π<sup>t</sup>* and *s<sup>t</sup>* :

$$\pi\_{t} = -\frac{\overline{s}\_{t}}{\overline{x}\_{t}^{target} - y} - \frac{r \frac{d\overline{x}\_{t}^{target}}{dt}}{b^{2}(\overline{x}\_{t}^{target} - y)}.\tag{14}$$

The differential equation governing the dynamics of *s<sup>t</sup>* is then written as follows:

$$\frac{d\overline{\mathbf{s}}\_{t}}{dt} = -\frac{\overline{\mathbf{s}}\_{t}^{2}b^{2} + r\overline{\mathbf{s}}\_{t}\frac{d\overline{\overline{\mathbf{x}}\_{t}^{target}}}{dt}}{r(\overline{\overline{\mathbf{x}}\_{t}^{target}} - y)} + (\overline{\mathbf{x}}\_{0} - y)\left[q\_{\mathbf{x}\_{0}} + \frac{\frac{d\overline{\mathbf{x}}\_{t}^{target}}{dt}}{b^{2}(\overline{\overline{\mathbf{x}}\_{t}^{target}} - y)}\right].\tag{15}$$

To solve this differential equation numerically in the interval of time [*t*0, *T*], we need to specify a terminal condition at *T*. Since this equation is solved backwards in time, this is equivalent to determining *sT*. By choosing a time horizon *T* such that the solar power curve has already fallen to zero at *T*, *x target <sup>t</sup>* will settle at *x target T* . Thus, *dx target t dt* = 0, *t* ≥ *T*, and this is consistent with imposing *<sup>d</sup>π<sup>t</sup> dt* = *dst dt* = 0, *t* ≥ *T*. We will then assimilate *q y T* , *πT*,*s<sup>T</sup>* to their steady-state values. This allows us to write:

$$\begin{aligned} 0 &= -\frac{b^2}{r} [\pi\_T(\overline{\mathbf{x}}\_T^{target} - \mathbf{y}) + \overline{\mathbf{s}}\_T], \\ 0 &= \frac{b^2}{r} \pi\_T^2 - q\_T^y - q\_{\mathbf{x}\_0}, \\ 0 &= \frac{b^2}{r} \pi\_T \overline{\mathbf{s}}\_T + q\_{\mathbf{x}\_0}(\overline{\mathbf{x}}\_0 - \mathbf{y}), \end{aligned} \tag{16}$$

and yields:

$$\begin{split} q\_T^y &= q\_{\mathbf{x}\_0} \left( \frac{\overline{\mathbf{x}}\_0 - \overline{\mathbf{x}}\_T^{\text{target}}}{\overline{\mathbf{x}}\_T^{\text{target}} - y} \right), \\ \pi\_T &= \sqrt{\frac{r}{b^2} (q\_{\mathbf{x}\_0} + q\_T^y)}, \\ \overline{s}\_T &= \pi\_T (y - \overline{\mathbf{x}}\_T^{\text{target}}) \text{ or } \overline{s}\_T = \frac{r q\_{\mathbf{x}\_0} (y - \overline{\mathbf{x}}\_0)}{\pi\_T b^2}. \end{split} \tag{17}$$

Thereafter, we solve numerically and backwards the differential equation of *ds<sup>t</sup> dt* , which yields the trajectory of *s<sup>t</sup>* . The latter is re-injected into the equation of *π<sup>t</sup>* . Finally, we have all the necessary ingredients to calculate *q y t* from:

$$q\_t^y = \frac{b^2}{r}\pi\_t^2 + \delta\pi\_t - \frac{d\pi\_t}{dt} - q\_{x\_0} \tag{18}$$

#### **3. Numerical Results in the Case of a Fixed Population of BEVs**

#### *3.1. Required Data*


### *3.2. MFG Inverse Nash Algorithm of Charging BEVs*

Figure 1 shows the outline of the operation of the algorithm.

**Figure 1.** Block diagram of the MFG inverse Nash algorithm of BEV *i* at time *t*.

Below is the detailed Algorithm 1.

#### **Algorithm 1** MFG inverse Nash

**Require:** *t*<sup>0</sup> (charging *starting* time), *T* (charging *stopping* time), *uW<sup>t</sup>* (forecast solar power at time *t*), *n* (number of BEVs present in the parking lot, i.e., at time *t*0), *xi*,0 (BEVs' SOCs at time *t*0), *x*<sup>0</sup> = ∑ *n i*=1 *xi*,0 *n* , *α*, *β*, *b* = *<sup>α</sup> β* , *dt*, *ν*, *qx*<sup>0</sup> , *r*, *y*, *δ*.

**Ensure:** *The parking lot operator computes the pressure field (q<sup>y</sup> t ) of n BEVs using the steps: target*

 $\textbf{1. Solve } \frac{d\textbf{T}\_{l}^{\textbf{ur}^{\textbf{st}}}}{dt} = \frac{1}{n} bu\_{Wt} \text{ and note the mean target SOC ( $\textbf{T}\_{T}^{\textbf{target}}$ ) of } n \text{ BEVs.}$ 

$$\begin{array}{ll} \textbf{2. Calculate } q\_T^y = q\_{\mathrm{x}\_0} \left( \frac{\overline{\mathbf{x}}\_0 - \overline{\mathbf{x}}\_T^{\mathrm{target}}}{\overline{\mathbf{x}}\_T^{\mathrm{target}} - y} \right), \pi\_T = \sqrt{\frac{r}{b^2} (q\_{\mathrm{x}\_0} + q\_T^y)} \text{ and } \overline{\mathbf{s}}\_T = \pi\_T (y - \overline{\mathbf{x}}\_T^{\mathrm{target}}). \text{ } \\\ \textbf{2. } \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad \pi\_t^{\mathrm{target}} \text{ } \end{array}$$

$$\textbf{3. Solve } \frac{d\overline{\textbf{s}}\_{t}}{dt} = -\frac{\overline{\textbf{s}}\_{t}^{2}b^{2} + r\overline{\textbf{s}}\_{t}\frac{u\overline{\textbf{s}}\_{t}}{dt}}{r(\overline{\textbf{x}}\_{t}^{\text{dragst}} - y)} + (\overline{\textbf{x}}\_{0} - y)\left[q\_{\text{x}\_{0}} + \frac{\frac{u\overline{\textbf{x}}\_{t}}{dt}}{b^{2}(\overline{\textbf{x}}\_{t}^{\text{dragst}} - y)}\right]\textbf{backwards}.$$

**4**. Calculate *π<sup>t</sup>* = − *st x target <sup>t</sup>* −*y* − *r t dt b* <sup>2</sup>(*x target <sup>t</sup>* −*y*) and determine *<sup>d</sup>π<sup>t</sup> dt* (by *mean value theorem*). *y* 2 2 *dπt*

**5**. Calculate *q <sup>t</sup>* = *<sup>b</sup> r π <sup>t</sup>* + *δπ<sup>t</sup>* − *dt* − *qx*<sup>0</sup> . **Ensure:** *Each BEV i*, *i* = 1, 2, 3, · · · , *n, computes its local feedback strategy using the steps:* **1**. Solve *dsi*,*<sup>t</sup> dt* = (*<sup>δ</sup>* <sup>+</sup> *<sup>b</sup>* 2 *r πt*)*si*,*<sup>t</sup>* + *qx*<sup>0</sup> (*xi*,0 − *y*) backwards with *si*,*<sup>T</sup>* = *rqx*<sup>0</sup> (*y*−*xi*,0) *πTb* 2 . **2**. Solve *dxi*,*<sup>t</sup>* = *bu*<sup>∗</sup> *i*,*t dt* <sup>+</sup> *<sup>ν</sup>dω<sup>i</sup>* <sup>=</sup> <sup>−</sup>*<sup>b</sup>* 2 *r* - *πt*(*xi*,*<sup>t</sup>* − *y*) + *si*,*<sup>t</sup> dt* + *νdω<sup>i</sup>* .

*3.3. Obtaining the Average Target SOC by Using Daily Solar Energy in the Parking Lot*

Realistic generation curves based on historical meteorological data are used (Figure 2), assuming that similar generation curves can be predicted using for example, a machine

learning based model. The meteorological data is obtained from photovoltaic geographical information System (PVGIS) made available by the European Commission. A typical meteorological year in the city of Montreal (45.50 North, 73.58 West) is used with a data resolution of one hour. The very same data can also be found in Canadian Weather Energy and Engineering Datasets (CWEEDS). The solar photovoltaic (PV) power output is then modeled with the simulation software TRNSYS using type 103 appropriate for modeling the electrical performance of mono and polycrystaline PV panels.

**Figure 2.** Solar energy production in the parking lot with 100 PV panels in Montreal (Canada).

We determine two different real solar power curves (sunny day and cloudy day cases) in order to compare the influence of the difference in generation on the behaviour of the BEVs charging. Starting with the sunny day case, and 400 BEVs, we get an average target curve that saturates at the end of the horizon (*x target <sup>T</sup>* = 0.90). Looking at the cloudy day case and as a result of that a lower energy output at the end of the horizon, charging the same number of BEVs with the same distribution SOCs upon arrival (*x*<sup>0</sup> = 0.15) would result in an average target that is low (*x target <sup>T</sup>* = 0.50). The parking lot operator would then announce the situation the day before, so that the BEVs arrive next day more full in the parking lot. However here, for comparison purposes, we shall work with the same distribution SOCs upon arrival as for the sunny day case.

#### *3.4. Pressure Field, Empirical per BEV Average SOC and Individual SOCs of BEVs Using MFG Inverse Nash Algorithm*

In Figures 3–5, first we see that the trajectories of the empirical per BEV average SOC and the mean target SOC broadcasted by the parking lot operator are quite the same in both days, *thanks to the law of large numbers*. And as expected in steady-state we have constant values, the proposed MFG algorithm makes full use of the available daily energy between 6 a.m. and 6 p.m. The results of individual SOCs of BEVs in the sunny day case, in Figure 6, show a *strong reduction in standard deviation σxi*,*<sup>T</sup>* while in the cloudy day case, in Figure 7, we have a *slight reduction in standard deviation*. Additionally, all the curves' behaviour in the cloudy day case well reflects the characteristic of solar fluctuations, and more importantly the results are *less desirable* in the cloudy day case relative to the sunny day case because the charging rate needed to achieve full solar utilization is lower in the cloudy day case. Furthermore, we confirm the main features of the proposed MFG algorithm, that of *filling more batteries that were emptier to start with while bringing all batteries close to a predefined mean target*.

**Figure 3.** Daily solar power curves (*uW*) for charging 400 BEVs and mean target SOC trajectories (*x target t* ) broadcasted by the parking lot operator.

**Figure 4.** Pressure fields (*q y t* ) of a fixed population of 400 BEVs.

**Figure 5.** Empirical per BEV average SOCs (*xt*) of a fixed population of 400 BEVs.

**Figure 6.** Individual SOCs of a fixed population of 400 BEVs in the sunny day.

**Figure 7.** Individual SOCs of a fixed population of 400 BEVs in the cloudy day.


In the following, we present the results for three charging strategies in Figures 8 and 9.

**Figure 8.** SOC levels at time of departure for a fixed population of 400 BEVs in the sunny day.

**Figure 9.** SOC levels at time of departure for a fixed population of 400 BEVs in the cloudy day.

#### 3.6.1. *FCFF Strategy*

It is the easiest strategy to implement. A centralized signal is sent to fill maximally the BEVs in the parking lot regardless of their SOCs upon arrival, and only depending on the ordering of their arrival times. This approach is not suitable at all for our charging objective. It does not meet any of the requirements. First, it is a centralized charging strategy since the parking lot operator must not only note the order of arrivals of BEVs in the parking lot, but also when they finish filling up. Secondly, at the end of the day, there will generally be users who have not recharged their batteries at all (12% of BEVs in the sunny day case and 59% of BEVs in the cloudy day case). Thirdly, we have the highest standard deviations of SOCs upon departure: *σxi*,*<sup>T</sup>* = 0.280 (an increase of 175%) in the sunny day case and *σxi*,*<sup>T</sup>* = 0.426 (an increase of 318%) in the cloudy day case.

#### 3.6.2. *ES Strategy*

This is a straightforward charging scheme. Using the forecast solar power available throughout the day, the parking lot operator estimates at every instant an average level per BEV available for charging and thus, the average charging rate. As BEVs get charged, the number of BEVs still in demand must be updated. Thus, this scheme, although superior to the previous one, is not decentralized. By giving the same amount to everyone, one does not significantly reduce *σxi*,*<sup>T</sup>* relative to *σxi*,0 . We have *σxi*,*<sup>T</sup>* = 0.082 (a reduction of 20%) in the sunny day case and *σxi*,*<sup>T</sup>* = 0.102 in the cloudy day case. Indeed, the SOCs' standard deviation remains close to what it was at the beginning, except for a slight reduction due to some BEVs completely filling up in the sunny day case.

#### 3.6.3. *MFG Strategy*

This is our novel charging strategy scheme based on the idea of inverse Nash control. The parking lot operator, after broadcasting *x target T* , prescribes a decentralized strategy via an inverse Nash algorithm in the smart charger in the parking lot. Each user applies their optimal control locally so that, the average trajectory (*xt*) of all users corresponds to the average target trajectory (*x target t* ) broadcasted by the parking lot operator. The standard deviation at the end of recharging has been significantly reduced. We have *σxi*,*<sup>T</sup>* = 0.013 (a reduction of 87%) in the sunny day case and *σxi*,*<sup>T</sup>* = 0.061 (a reduction of 40%) in the cloudy day case. Furthermore, except for the broadcasting of some initialization data by the parking lot operator, the charging scheme is decentralized. This strategy meets all the requirements of our charging objective.

In Table 2 the fairness coefficient (FC), as defined previously in Section 2.2.1, is computed for each charging strategy.


**Table 2.** Comparison of fairness coefficient and departing SOCs (min *xi*,0 = 0 and max *xi*,0 = 0.45) for a fixed population of 400 BEVs.

The results in both days show that the MFG strategy is the fairest strategy while the FCFF strategy is rejected.

#### **4. Numerical Results in the Case of a fluctuating Population of BEVs**

#### *4.1. Considerations*

• In the parking lot, the BEVs start arriving before 6 a.m., stop arriving at 12 p.m., start departing after 3 p.m. until after 6 p.m. (with *tbegin* = 6 h, *tend* = 18 h).


**Figure 10.** Daily solar power curves (*uW*) for charging 400 BEVs and arrivals/departures of a fluctuating population of 400 BEVs in the parking lot.

#### *4.2. Updated MFG Inverse Nash Algorithm*

Each charging time interval *ϕ<sup>c</sup>* (considering both arrival time interval *ϕ<sup>a</sup>* and departing time interval *ϕ<sup>d</sup>* ), i.e., *c* = 1, 2, · · · , 25, 26, 27, · · · , 38 (where *ϕ*<sup>1</sup> = [6, 6.25[ *h*, *ϕ*<sup>2</sup> = [6.25, 6.5[ *h*, · · · , *ϕ*<sup>25</sup> = [12, 15] *h*, *ϕ*<sup>26</sup> =]15, 15.25] *h*, *ϕ*<sup>27</sup> =]15.25, 15.5] *h*, · · · , *ϕ*<sup>38</sup> = ]17.75, 18] *h*), the parking lot operator calculates the mean target SOC trajectory of all BEVs present in the parking lot. Unlike in the case of a fixed population of BEVs in the parking lot where we worked with a solar curve over the entire control horizon between *tbegin* and *tend*, here we work with successive fixed short control horizons at the start of which the number of BEVs present in the parking lot is recorded. In order to enforce the Riccati steady-state conditions (illustrated in Section 2), in each charging time interval *ϕ<sup>c</sup>* we add to the solar power curve *uWϕ<sup>c</sup>* a fictitious extension *u*˜*Wϕ<sup>c</sup>* into *ϕc*+<sup>1</sup> with the solar power falling smoothly to zero at the end of *ϕc*+1. Below, in Figure 11, is the example of a piece of solar power curve between 8 a.m. and 10 a.m. in the sunny day case with fictitious added parts.

**Figure 11.** Section of solar power curve in the sunny day case with *fictitious added parts*.

The MFG inverse Nash algorithm (see Algorithm 1 in Section 3.2) will be applied here in each charging time interval *ϕ<sup>c</sup>* on short control horizons (i.e., *here t*<sup>0</sup> *is the beginning of ϕ<sup>c</sup> and T is the end of ϕc*) with the resulting mean target SOC trajectory *x target t* , *t* ∈ {*ϕ<sup>c</sup>* ∪ *ϕc*+1}, knowing that resulting solar power curve is equal to {*uWϕ<sup>c</sup>* ∪ *u*˜*Wϕ<sup>c</sup>* } (Figure 11).

#### *4.3. Adjustment of Different Charging Strategies*


$$\frac{\sum\_{i=1}^{n} \mathfrak{x}\_{i,0}}{n} + \frac{1}{n} \int\_{t\_0}^{T} \mathfrak{u}\_{\mathcal{W}t} dt.$$


### *4.4. Comparison of Charging Strategies Considering SOCs' BEVs at Departure Times*

First, we present in Figure 12 the results of the SOCs' averages and standard deviations for the remaining BEVs in the parking lot knowing that the SoCs' averages before the departures (i.e., before 3 p.m.) are all the same for all charging strategies in both days, as they use all solar energy in the parking lot. As expected, the FCFS strategy comes first with the highest SOCs' averages and the lowest SOCs' standard deviations for the remaining BEVs as its perfectly equalizes the BEVs' SOCs regardless of their SOCs at the beginning of recharging, while the MFG strategy comes second as its also equalizes the BEVs' SOCs by maintaining some fairness criterion (with regard to their SOCs at the beginning of recharging). The ES strategy, considered as the base case, comes third, while the FCFF is clearly the worst strategy with the lowest SOCs' averages and the highest SOCs' standard deviations as its fills up to maximum capacity the BEVs' SOCs.

**Figure 12.** SOCs' averages and standard deviations of the remaining BEVs in the parking lot.

Then, we present in Figures 13 and 14 the results of *SOCs for the departing BEVs* knowing that the average parking time is 8 h.

In both days, we note that the MFG strategy remains the best between the four options insofar as the resulting *standard deviation* for the departed BEVs are concerned.

In the sunny day case, the MFG strategy reduces significantly the standard deviation (*σxi*,*dep* = 0.026, a reduction of 75%). The FCFF strategy gives the worst standard deviation (*σxi*,*dep* = 0.296, an increase of 190%) with 12% of users who have not recharged their batteries at all. The FCFS strategy is the improved version of the FCFF, as we can see that the BEVs are recharged with a better standard deviation (*σxi*,*dep* = 0.062, a reduction of 39%). The ES strategy results in a slight increase of 16% of the standard deviation (*σxi*,*dep* = 0.118) due to the fluctuating population of BEVs and to some BEVs completely filling up.

In the cloudy day case, the MFG strategy also reduces significantly the standard deviation (*σxi*,*dep* = 0.024, a reduction of 76%) due to the fluctuating population of BEVs. The FCFF strategy gives the worst standard deviation (*σxi*,*dep* = 0.457, an increase of 348%) with 54% of users who have not recharged their batteries at all. The FCFS strategy is the improved version of the FCFF, as we can see that the BEVs are recharged with a

better standard deviation (*σxi*,*dep* = 0.053, a reduction of 48%). The ES strategy results in a slight reduction of 21% of the standard deviation (*σxi*,*dep* = 0.081) due to the fluctuating population of BEVs.

**Figure 13.** SOC levels at time of departure for a fluctuating population of 400 BEVs in the sunny day.

**Figure 14.** SOC levels at time of departure for of a fluctuating population of 400 BEVs in the cloudy day.

In Table 3 the fairness coefficient (FC), as defined previously in Section 2.2.1, is evaluated for each charging strategy.


**Table 3.** Comparison of fairness coefficient and departing SOCs (min *xi*,*arr* = 0, max *xi*,*arr* = 0.45) for a fluctuating population of 400 BEVs.

The results show that the MFG strategy remains the fairest strategy in both days while the FCFF strategy is rejected in both days and the ES strategy is rejected in the sunny day case.

#### **5. Conclusions and Future Research**

We have considered the situation of a large daytime work parking lot with homogeneous battery electric vehicles (BEVs) for simplicity, and solar sources based electricity charging. We have used realistic data to implement deterministic daily solar power curves with photovoltaic panels in a parking lot for a typical sunny day and a typical cloudy day. One should note that a large heterogeneous population of BEVs can be analyzed by assuming that it is possible to group the BEVs into classes considered homogeneous. Thus, all the BEVs of a class share the same physical parameters and, in order to better redistribute energy according to individual BEV needs, the forecast solar energy can be distributed by favoring a class with more BEVs, a larger-size battery and a lower charger efficiency.

In Section 3, a fair, and decentralized MFG strategy, for recharging a large fixed population of BEVs, has been developed. The goal was to reduce significantly the SOCs' standard deviation while elevating the SOCs of BEVs to a satisfactory level regardless of their SOCs upon arrival. A comparison was carried out with an equal sharing (ES) strategy and a first come first full (FCFF) strategy which we saw could result in some unsatisfied individual users with little SOCs at the end of recharging. In Section 4, we considered a large fluctuating population of homogeneous BEVs. This new situation allowed us to improve the FCFF strategy into first come first serve (FCFS) strategy. The results showed that the MFG strategy remains the most desirable charging strategy with regards to the standard deviation of SOCs upon departure and fairness criterion. Finally, we did much better than the literature [20] (60% as maximum reduction of SOCs' standard deviation in the case of a fluctuating population of BEVs) as we illustrated in the summary Table 4 below when we compared our results to the base case which is here the ES strategy.


**Table 4.** Standard deviation increase/reduction of SOCs' BEVs.

In future research, we shall extend this work by considering stochastic solar acquisition in the parking lot and explore MFG based algorithms this time for potential partial restitution of battery energy from solar charged BEVs to the grid during evening peak hours.

**Author Contributions:** Conceptualization, S.M.M., R.P.M. and G.J.; Formal Analysis, S.M.M. and R.P.M.; Modeling and Simulation, S.M.M.; Experimental Results, S.M.M.; Investigation, S.M.M., R.P.M. and G.J.; Writing—Original Draft Preparation, S.M.M.; Writing—Review and Editing, S.M.M., R.P.M. and G.J.; Supervision, R.P.M. and G.J.; Funding Acquisition, R.P.M. and G.J. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded by the Réseau Québécois sur l'Énergie Intelligente and the NSERC/Hydro-Québec Industrial Research Chair on the Integration of Renewable Energies and Distributed Generation into the Electric Distribution Grid.

**Acknowledgments:** The authors would like to thank the Réseau Québécois sur l'Énergie Intelligente and the NSERC/Hydro-Québec Industrial Research Chair on the Integration of Renewable Energies and Distributed Generation into the Electric Distribution Grid for the financial support of this research. The first author would also like to thank Kato Vanroy from the department of Mechanical Engineering at Polytechnique Montreal for practical utilization of the TRNSYS software and Quentin Lénet for the first steps in inverse Nash algorithm.

**Conflicts of Interest:** The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

## **Nomenclature**


### **References**


## *Article* **Optimal Reactive Power Dispatch under Transmission and Distribution Coordination Based on an Accelerated Augmented Lagrangian Algorithm**

**Bin Zhou <sup>1</sup> , Xiaodong Shen 1,\*, Caimei Pan <sup>2</sup> , Yuanbao Bai <sup>2</sup> and Tian Wu <sup>2</sup>**

> <sup>1</sup> College of Electrical Engineering, Sichuan University, Chengdu 610065, China; 2019223030002@stu.scu.edu.cn <sup>2</sup> Sichuan Energy Investment Integrated Energy Co., Ltd., Chengdu 611130, China; 015740@ntsd789.com (C.P.);

baiyuanbao0827@yeah.net (Y.B.); oversky2012@163.com (T.W.)

**Abstract:** As many distributed power sources flood into the distribution network, the relationship between transmission and distribution grids in reactive power and voltage is becoming closer and closer. The traditional way of independent reactive power optimization in transmission and distribution grids is no longer appropriate. In this study, a collaborative and distributed reactive power optimization method for transmission and distribution grids based on the accelerated augmented Lagrangian (AAL) algorithm is proposed to adapt to the independence of the transmission and distribution grids in operation and management. The global reactive power optimization problem is decomposed into the transmission network subproblem and several distribution network subproblems. According to AAL, subproblems are solved in a distributed manner until the optimal global solution is finally reached after several iterations, and coordination between transmission and distribution grids is achieved only through the interaction of information on coordinating variables. For better convergence, a linearized and convergence-guaranteed optimal power flow model (OPF) with reactive power and voltage magnitude was applied to model the transmission grid optimization subproblem, while the second-order cone programming (SOCP) technique is used in the distribution network subproblems. The simulation results confirm that the method in this paper can effectively reduce network losses and achieve better economic performance, and converges better when compared to other algorithms.

**Keywords:** transmission and distribution coordination; linear approximation; reactive power optimization; accelerated augmented Lagrangian algorithm; distributed generator

## **1. Introduction**

With the rapid development of distributed energy, distribution networks are gradually evolving into active distribution networks, and the reactive power characteristic has also changed [1]. Traditionally, the transmission and distribution network0 s optimal reactive power dispatch (ORPD) is carried out separately. The lower distribution networks remain unchanged when optimizing the transmission network, and the voltage of the boundary buses are used as the reference when optimizing the distribution networks, making it prone to overvoltage problems and cascading blackouts of distributed generators when there is a large-scale influx of distributed generators into distribution networks [2]. Therefore, it is necessary to give full play to the voltage support role of the transmission network and make full use of the reactive power regulation capability of the distribution network, which helps active distribution networks to eliminate overvoltage problems and reduce network losses [3,4].

Theoretically, the centralized optimization method could be used to establish a global optimal dispatch model to realize the coordinated optimal dispatch of transmission and

**Citation:** Zhou, B.; Shen, X.; Pan, C.; Bai, Y.; Wu, T. Optimal Reactive Power Dispatch under Transmission and Distribution Coordination Based on an Accelerated Augmented Lagrangian Algorithm. *Energies* **2022**, *15*, 3867. https://doi.org/10.3390/ en15113867

Academic Editor: Abu-Siada Ahmed

Received: 28 April 2022 Accepted: 23 May 2022 Published: 24 May 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

**<sup>\*</sup>** Correspondence: shengxd@scu.edu.cn; Tel.: +86-166-0282-7868

distribution networks [5]. However, for the existing power system, transmission and distribution networks are under the jurisdiction of different dispatching agencies, which makes it hard for centralized optimization to deal with the confidentiality of information among various stakeholders and the independence of transmission and distribution networks in the process of operation and management. On the other hand, the centralized optimization method must concentrate the data of the global system to the dispatch center for calculation. However, aggregating data from multiple distribution networks will dramatically increase the complexity and computational cost of optimization problems. Therefore, it is reasonable to use a distributed optimization method when carrying out the optimal dispatch under transmission and distribution network coordination [6].

Recently, several researches have been conducted on distributed optimization methods for coordinated transmission and distribution networks. In [7], a reactive power optimization model considering coordinated transmission and distribution networks was established and solved by the generalized Benders decomposition method. However, this approach requires the subproblems to be convex to obtain the efficient Benders cut. However, shunt capacitors (SCs) and on-load transformers (OLTCs) in distribution networks are discrete control variables, which poses a challenge to the Benders decomposition method. In [8,9], the heterogeneous decomposition (HGD) was applied to the economic scheduling problem of coordinated transmission and distribution networks. However, the HGD may lead to local optima and is unsuitable for topologically complex networks, especially when there are connections between different distribution networks [10]. As a commonly used distributed optimization algorithm in optimization problems, the alternating direction multiplier (ADMM) has been widely used in multi-region network operation problems, including distributed optimal power flow and fully distributed reactive power optimization [11,12]. In [13], ADMM was applied to the reactive power optimization problem of multiple partition coordination in active distribution networks, and the results proved that the ADMM is an effective method for dealing with collaborative optimization problems. By improving the ADMM algorithm, the accelerated augmented Lagrangian algorithm (AAL) can obtain a faster convergence speed when dealing with collaborative optimization problems [14]. In [15], the AAL was applied to the distributed restoration problem of an integrated power transmission and distribution system with distributed energy sources. In [16], a framework was established to deal with the reactive power optimization of transmission and distribution networks coordination through the method of curve fitting, but the method cannot take into account the role of DGs. In addition, methods such as multiparameter planning and the principle of auxiliary problems were also used to optimize transmission and distribution networks coordination problems [17,18].

The optimization problem of transmission and distribution networks can be formulated by nonlinear AC-OPF, DC-OPF, or other forms, such as mixed-integer linear programming, second-order cone programming (SOCP), and semidefinite programming. In [19], the DC-OPF was used for the transmission network optimization problem, and the nonlinear AC-OPF was used for the distribution network optimization problem, thereby improving the calculation speed. However, since the variables related to reactive power are not modeled in the traditional DC-OPF, it cannot be used to deal with the optimal reactive power dispatch problem. For that issue, a novel linearized OPF model with reactive power (Q) and voltage magnitude (V) was established to achieve the better performance of the DC-OPF models [20]. By applying the novel linear approximation method to the transmission network subproblem model, the complexity of the transmission network subproblem and the global optimization problem can be significantly reduced. In addition, the effectiveness of SOCP for radial networks has been demonstrated in the related literature, and it is often used in the optimization of distribution networks [21].

To give full play to the synergy between the transmission network and the distribution network, a coordinated transmission and distribution optimal reactive power dispatch (CTD-ORPD) framework is established in this paper. The main contributions are listed as follows:


The rest of this paper is organized as follows: In Section 2, the framework of the CTD-ORPD is introduced, including the separate modeling of transmission network and distribution network subproblems, as well as the novel linearized OPF and SOCP applied to the modeling of the transmission network and distribution networks, respectively. In Section 3, the distributed solving algorithm of the CTD-ORPD is detailed, and the algorithm flowchart is given. Section 3 establishes three test cases to verify the validity of the proposed CTD-ORPD and compares the performance of the proposed method in this paper with other methods. Finally, Section 4 presents our conclusions.

#### **2. Framework of the CTD-ORPD**

In this section, the CTD-ORPD framework based on AAL is introduced. In the proposed CTD-ORPD framework, the global reactive power optimization problem is decomposed into the transmission network subproblem and several distribution network subproblems. Based on AAL, the subproblems are solved in a distributed manner until the optimal global solution is finally reached after several iterations, and coordination between transmission and distribution grids is achieved through the interaction of information on coordinating variables. Coordinating variables are the variables of the points of common coupling (PCCs) between the transmission network and the distribution networks, including bus voltage magnitude and phase angle, active power output and reactive power output.

In the transmission network subproblem, control variables such as SCs, OLTCs, and reactive power output of generators need to be optimized. A novel linearized approximation method is applied to handle the nonlinearity of the power flow equations of the transmission network for better convergence [20]. In the distribution network subproblem, control variables such as SCs, OLTCs, and reactive power output of various distributed power sources need to be optimized. Additionally, SOCP method is applied to the convex modeling of the distribution network subproblems. The framework of the established CTD-ORPD is illustrated in Figure 1.

**Figure 1.** The framework of the proposed CTD-ORPD.

#### **Figure 1.** The framework of the proposed CTD-ORPD. *2.1. Linear Approximation of Power Flow Equations*

*2.1. Linear Approximation of Power Flow Equations* A novel linear approximation of power flow equations to construct the linearized OPF model of the transmission subproblem is detailed in this part, which is the foundation A novel linear approximation of power flow equations to construct the linearized OPF model of the transmission subproblem is detailed in this part, which is the foundation for formulating the proposed CTD-ORPD model.

for formulating the proposed CTD-ORPD model. Taking advantage of the power flow equations in the polar coordination, the expres-Taking advantage of the power flow equations in the polar coordination, the expression for power flows on branch *i-j* are as follows:

$$P\_{\vec{i}\vec{j}} = \left(v\_{\vec{i}}^2 - v\_{\vec{i}}v\_{\vec{j}}\cos\theta\_{\vec{i}\vec{j}}\right)g\_{\vec{i}\vec{j}} - v\_{\vec{i}}v\_{\vec{j}}h\_{\vec{i}\vec{j}}\sin\theta\_{\vec{i}\vec{j}}\tag{1}$$

$$Q\_{i\dot{j}} = -\left(v\_i^2 - v\_i v\_{\dot{j}} \cos\theta\_{i\dot{j}}\right) b\_{i\dot{j}} - v\_i v\_{\dot{j}} g\_{i\dot{j}} \sin\theta\_{i\dot{j}} \tag{2}$$

where *Pij* and *Qij* are the active and reactive power passing on branch *i-j*, respectively; *i v* and *j v* are the voltage magnitudes of bus *i* and bus *j*, respectively; and *gij* and *ij b* are conductance and susceptance of branch *i-j*. Given the initial values of voltage magnitudes and phase angles 0 0 ( , ) *v θ* , the trigonometric function in the above formula can be where *Pij* and *Qij* are the active and reactive power passing on branch *i-j*, respectively; *v<sup>i</sup>* and *v<sup>j</sup>* are the voltage magnitudes of bus *i* and bus *j*, respectively; and *gij* and *bij* are conductance and susceptance of branch *i-j*. Given the initial values of voltage magnitudes and phase angles (*v*0, θ0), the trigonometric function in the above formula can be linearized and approximated by the first-order Taylor series as follows:

$$\sin\theta\_{i\dot{j}} \approx s\_{i\dot{j}}^1 \theta\_{i\dot{j}} + s\_{i\dot{j}}^0 \tag{3}$$

$$\cos\theta\_{i\dot{j}} \approx c\_{i\dot{j}}^1 \theta\_{i\dot{j}} + c\_{i\dot{j}}^0 \tag{4}$$

$$s\_{ij}^1 = \cos\theta\_{ij,0}, s\_{ij}^0 = \sin\theta\_{ij,0} - \theta\_{ij,0}\cos\theta\_{ij,0} \tag{5}$$

$$\begin{array}{ccccccccc} 1 & \cdot & 0 & \cdot & \cdot & \cdot & \cdot & \cdot \end{array}$$

$$c\_{ij}^1 = -\sin\theta\_{ij,0\prime}c\_{ij}^0 = \cos\theta\_{ij,0} + \theta\_{ij,0}\sin\theta\_{ij,0} \tag{6}$$
 
$$\text{the voltage phase angle is very close to the final value after iteration}$$

 (6) The initial value of the voltage phase angle is very close to the final value after iteration due to the quasi-linear relationship between the active power and the voltage phase angle in the power system [23]. Therefore, Equations (3) and (4) have a good accuracy. The initial value of the voltage phase angle is very close to the final value after iteration due to the quasi-linear relationship between the active power and the voltage phase angle in the power system [23]. Therefore, Equations (3) and (4) have a good accuracy. Substituting Equations (3) and (4) into Equations (1) and (2), we can obtain:

$$\mathbf{P}\_{\text{ij}} = \mathbf{v}\_i^2 \mathbf{g}\_{\text{ij}} - \mathbf{v}\_i \mathbf{v}\_j \left( \mathbf{g}\_{\text{ij}} \mathbf{c}\_{\text{ij}}^0 + \mathbf{b}\_{\text{ij}} \mathbf{s}\_{\text{ij}}^0 \right) - \mathbf{v}\_i \mathbf{v}\_j \theta\_{\text{ij}} \left( \mathbf{g}\_{\text{ij}} \mathbf{c}\_{\text{ij}}^1 + \mathbf{b}\_{\text{ij}} \mathbf{s}\_{\text{ij}}^1 \right) \tag{7}$$
 
$$\mathbf{g}\_{\text{ij}} = \mathbf{v}\_i^2 \mathbf{g}\_{\text{ij}} - \mathbf{v}\_i \mathbf{v}\_j \left( \mathbf{g}\_{\text{ij}} \mathbf{c}\_{\text{ij}}^0 + \mathbf{b}\_{\text{ij}} \mathbf{s}\_{\text{ij}}^1 \right) \tag{8}$$

$$\mathbf{r}\_{\text{ij}} = \mathbf{v}\_i \mathbf{g}\_{\text{ij}} - \mathbf{v}\_l \mathbf{v}\_j \left( \mathbf{g}\_{\text{ij}} \mathbf{c}\_{\text{ij}} + \mathbf{v}\_l \mathbf{g}\_{\text{ij}} \right) - \mathbf{v}\_l \mathbf{v}\_j \mathbf{v}\_{\text{ij}} \left( \mathbf{g}\_{\text{ij}} \mathbf{c}\_{\text{ij}} + \mathbf{v}\_l \mathbf{g}\_{\text{ij}} \right) \tag{1}$$
 
$$\mathbf{Q}\_{\text{ij}} = -\mathbf{v}\_i^2 \mathbf{b}\_{\text{ij}} + \mathbf{v}\_i \mathbf{v}\_j \left( -\mathbf{g}\_{\text{ij}} \mathbf{s}\_{\text{ij}}^0 + \mathbf{b}\_{\text{ij}} \mathbf{c}\_{\text{ij}}^0 \right) - \mathbf{v}\_l \mathbf{v}\_j \theta\_{\text{ij}} \left( \mathbf{g}\_{\text{ij}} \mathbf{s}\_{\text{ij}}^1 - \mathbf{b}\_{\text{ij}} \mathbf{c}\_{\text{ij}}^1 \right) \tag{8}$$

Considering *viv<sup>j</sup>* as a whole, use the first-order Taylor series expansion to decouple the variable *viv<sup>j</sup>* from the variable *θij* in the expression *vivjθij*:

$$
v\_{i}v\_{j}\theta\_{ij} \approx v\_{i,0}v\_{j,0}\theta\_{ij} + \left(v\_{i}v\_{j} - v\_{i,0}v\_{j,0}\right)\theta\_{ij,0} \tag{9}$$

By substituting Equation (9) into Equations (7) and (8), the following expressions can be obtained:

$$P\_{\rm ij} = g\_{\rm ij} v\_i^2 - g\_{\rm ij}^{\rm P} v\_i v\_j - b\_{\rm ij}^{\rm P} (\theta\_{\rm ij} - \theta\_{\rm ij,0}) \tag{10}$$

$$Q\_{i\rangle} = -b\_{i\rangle}v\_i^2 + b\_{i\bar{j}}^{\mathcal{Q}}v\_i v\_{\bar{j}} - g\_{i\bar{j}}^{\mathcal{Q}}(\theta\_{i\bar{j}} - \theta\_{i\bar{j},0}) \tag{11}$$

where *g* P *ij*, *b* P *ij*, *g* Q *ij* , and *b* Q *ij* are the active equivalent conductance, active equivalent susceptance, reactive equivalent conductance, and reactive equivalent susceptance of line *i*-*j*, respectively, and their expressions are as follows:

$$\mathbf{g}\_{ij}^{\mathbf{P}} = \left( g\_{i\bar{j}} c\_{i\bar{j}}^{0} + b\_{i\bar{j}} s\_{i\bar{j}}^{0} \right) + \left( g\_{i\bar{j}} c\_{i\bar{j}}^{1} + b\_{i\bar{j}} s\_{i\bar{j}}^{1} \right) \theta\_{i\bar{j},0} \tag{12}$$

$$b\_{i\rangle}^{\mathbb{P}} = \left(\mathbf{g}\_{i\rangle}\mathbf{c}\_{i\rangle}^{1} + b\_{i\rangle}\mathbf{s}\_{i\rangle}^{1}\right)\mathbf{v}\_{i,0}\mathbf{v}\_{j,0} \tag{13}$$

$$\mathbf{g}\_{i\dot{j}}^{\mathbf{Q}} = \left(\mathbf{g}\_{i\dot{j}}\mathbf{s}\_{i\dot{j}}^{1} - b\_{i\dot{j}}\mathbf{c}\_{i\dot{j}}^{1}\right)\upsilon\_{i,0}\upsilon\_{j,0} \tag{14}$$

$$b\_{ij}^{\mathbb{Q}} = \left(-g\_{ij}s\_{ij}^{0} + b\_{ij}c\_{ij}^{0}\right) - \left(g\_{i\bar{j}}s\_{i\bar{j}}^{1} - b\_{i\bar{j}}c\_{i\bar{j}}^{1}\right)\theta\_{\bar{i}\bar{j},0} \tag{15}$$

There is still a nonlinear variable *viv<sup>j</sup>* in Equations (10) and (11), respectively. In order to eliminate the nonlinearity of the model, the following mathematical transformations are performed on *viv<sup>j</sup>* :

$$v\_i v\_j = \frac{1}{2} \left[ v\_i^2 + v\_j^2 - \left( v\_i - v\_j \right)^2 \right] = \frac{v\_i^2 + v\_j^2}{2} - \frac{v\_{ij}^2}{2} \tag{16}$$

The expression *v* 2 *<sup>i</sup>* + *v* 2 *j* /2 is linear when considering *v* <sup>2</sup> as an independent variable. In practical power systems, the values of *v* 2 *ij* are very small and often negligible. Thus, *v* 2 *<sup>i</sup>* + *v* 2 *j* /2 can be used as an approximation of *viv<sup>j</sup>* , the approximation error would be small and not affected by the initial value of the voltage magnitude [24]. For keeping the error as low as possible, the following approximation is made to *v* 2 *ij* in Equation (16) based on a Taylor series expansion:

$$v\_{ij}^2 \approx 2v\_{ij,0}v\_{ij} - v\_{ij,0}^2 \approx 2v\_{ij,0}v\_{ij}\frac{v\_i + v\_j}{v\_{i,0} + v\_{j,0}} - v\_{ij,0}^2 = 2\frac{v\_{i,0} - v\_{j,0}}{v\_{i,0} + v\_{j,0}} \left(v\_i^2 - v\_j^2\right) - v\_{ij,0}^2 = v\_{ij,L}^s \tag{17}$$

where *v* s *ij*,L is an approximation of *v* 2 *ij*. Equation (17) are linear when considered *v* <sup>2</sup> as an independent variable. Since *v* 2 *ij* is non-negativity, the following non-negativity constraint should be applied:

$$
v\_{\rm ij,L}^s + \varepsilon\_{\rm ij} \ge 0, \varepsilon\_{\rm ij} \ge 0 \tag{18}$$

By substituting Equations (16) and (17) into Equations (10) and (11), the linearized approximation of the power flow equation is obtained:

$$P\_{ij}^{\rm L} = g\_{ij}v\_i^2 - g\_{ij}^{\rm P}\frac{v\_i^2 + v\_j^2}{2} - b\_{ij}^{\rm P}(\theta\_{ij} - \theta\_{ij,0}) + g\_{ij}^{\rm P}\frac{v\_{ij,L}^8}{2} \tag{19}$$

$$Q\_{ij}^{\rm L} = -b\_{ij}v\_i^2 + b\_{ij}^{\rm Q}\frac{v\_i^2 + v\_j^2}{2} - g\_{ij}^{\rm Q}(\theta\_{ij} - \theta\_{ij,0}) - b\_{ij}^{\rm Q}\frac{v\_{ij,L}^s}{2} \tag{20}$$

In Equations (19) and (20), the active and reactive components of the power flow are both linear functions when considering *v* <sup>2</sup> as an independent variable.

The following are the expressions of the nodal injection power equations:

$$P\_i = \sum\_{j=1}^{N} \left( v\_i v\_j G\_{i\bar{j}} \cos \theta\_{i\bar{j}} + v\_i v\_{\bar{j}} B\_{i\bar{j}} \sin \theta\_{i\bar{j}} \right) \tag{21}$$

$$Q\_i = -\sum\_{j=1}^{N} \left( v\_i v\_j B\_{ij} \cos \theta\_{ij} - v\_i v\_j G\_{lj} \sin \theta\_{ij} \right) \tag{22}$$

where *Gij* and *Bij* are the real and imaginary parts of the node admittance matrix, respectively. Equations (21) and (22) can be converted into the following forms:

$$P\_i = \sum\_{(i,j)} P\_{ij} + \left(\sum\_{j=1}^{N} G\_{ij}\right) v\_i^2 \tag{23}$$

$$Q\_i = \sum\_{(i,j)} Q\_{ij} + \left(\sum\_{j=1}^{N} -B\_{ij}\right) v\_i^2 \tag{24}$$

Finally, the linearized power flow equation constraints are obtained, which are composed of Equations (19), (20), (23), and (24). They are all linear functions when *v* 2 is considered as an independent variable.

#### *2.2. Transmission Network Subproblem Model*

In this part, the linearized OPF model of the transmission network subproblem is established based on the above linear approximation of power flow equations.

In traditional optimal reactive power dispatch, minimizing network losses is often applied as the objective function, where network losses are functions of the voltage magnitude and the voltage phase angle [25]. However, minimizing network losses without considering generator costs may be in conflict with economic principles. Thus, minimizing active power injections at the root bus is also usually used as the objective function of the optimal reactive power dispatch problem [26]. In this paper, minimizing the operating cost was adopted as the objective function of ORPD of the transmission network subproblem, and its expression is as follows:

$$\min F\_{\mathbb{T}} = \sum\_{\mathcal{g} \in G} \mathbb{C}(P\_{\mathcal{g}}) \tag{25}$$

#### 2.2.1. The Cost of Coal Consumption

According to the characteristics of coal consumption, a convex quadratic function is used as the generation cost of generator *g* [27]. By default, the unit commitment is already achieved, so the binary variables indicating the on/off status of the unit are not included in the equation.

$$\mathcal{C}(P\_{\mathcal{S}}) = \left(c\_{\mathcal{S},2}^{\mathcal{P}} P\_{\mathcal{S}}^2 + c\_{\mathcal{S},1}^{\mathcal{P}} P\_{\mathcal{S}} + c\_{\mathcal{S},0}^{\mathcal{P}}\right) \tag{26}$$

where *c* P *<sup>g</sup>*,2, *c* P *<sup>g</sup>*,1, and *c* P *<sup>g</sup>*,0 are the quadratic term coefficient, primary term coefficient, and constant term of the quadratic cost curve of the generator *g*, respectively. *P<sup>g</sup>* denotes the active power output of the generator *g*.

For better convergence, a segmented linearization technique is applied to the linearization of the generator cost function. Figure 2 shows the linearized cost curve of generator *g*. We equally divided [*P* min *g* , *P* max *g* ], the power output range of generator *g*, into *m<sup>g</sup>* segments, and the length of each segment is denoted as *PIg*,1, *PIg*,2, . . . , *PIg*,*m<sup>g</sup>* ; the slope is denoted as *K Ig*,1, *K Ig*,2, . . . , *K Ig*,*m<sup>g</sup>* , respectively. Since *C*(*Pg*) is a convex function, then we have *K Ig*,1 < *K Ig*,2 < . . . < *K Ig*,*m<sup>g</sup>* and the power of the later segments can only be used after the

power of the previous segments has been used to the upper limit. Denoting the power at the segmented points as *P* min *g* , *<sup>P</sup>g*,1, *<sup>P</sup>g*,2, . . . , *<sup>P</sup>g*,*mg*−<sup>1</sup> , *P* max *g* , then the linearized operating cost and power output can be expressed as follows: after the power of the previous segments has been used to the upper limit. Denoting the power at the segmented points as min *P<sup>g</sup>* , *P<sup>g</sup>* ,1 , *Pg*,2 , …, , 1 *<sup>g</sup> P g m* − , max *P<sup>g</sup>* , then the linearized operating cost and power output can be expressed as follows:

$$\mathcal{C}(P\_{\mathcal{S}}) = \mathcal{C}(P\_{\mathcal{S}}^{\min}) + \sum\_{k=1}^{m\_{\mathcal{S}}} P I\_{\mathcal{S},k} \times K I\_{\mathcal{S},k} \tag{27}$$

$$P\_{\mathcal{S}} = P\_{\mathcal{S}}^{\min} + \sum\_{k=1}^{m\_{\mathcal{S}}} P I\_{\mathcal{S},k} \tag{28}$$

$$1 - P\_{\sigma}^{\min}, 0 \le P I\_{\sigma \mathcal{I}} \le P\_{\sigma \mathcal{I}} - P\_{\sigma \mathcal{I} \iota}, \dots, 0 \le P I\_{\mathcal{S}, \mathcal{W} \iota} \le P\_{\sigma}^{\max} - P\_{\sigma, m\_{\mathcal{S}} - 1} \tag{29}$$

$$0 \ll PI\_{\mathfrak{g},1} \lesssim P\_{\mathfrak{g},1} - P\_{\mathfrak{g}}^{\min},\\0 \ll PI\_{\mathfrak{g},2} \lesssim P\_{\mathfrak{g},2} - P\_{\mathfrak{g},1}, \dots, 0 \leqslant PI\_{\mathfrak{g},m\_{\mathfrak{g}}} \lesssim P\_{\mathfrak{g}}^{\max} - P\_{\mathfrak{g},m\_{\mathfrak{g}}-1} \tag{29}$$

*mg*

**Figure 2.** The linearized cost curve of generator *g*. **Figure 2.** The linearized cost curve of generator *g*.

When the initial values of the voltage magnitude and phase angle 0 0 ( , ) *v θ* are given, according to the description in Section 2.1, the constraints of the transmission network subproblem are constructed as follows: When the initial values of the voltage magnitude and phase angle (*v*0, θ0) are given, according to the description in Section 2.1, the constraints of the transmission network subproblem are constructed as follows:

#### 2.2.2. Nodal Injection Power Balancing Constraints 2.2.2. Nodal Injection Power Balancing Constraints

For all the nodes, the following constraints need to be satisfied: For all the nodes, the following constraints need to be satisfied:

$$P\_i^k = \sum\_{\mathcal{J} \in i} P\_{\mathcal{J}} - P\_{i,d} = \sum\_{(i,j)} P\_{ij}^k + \left(\sum\_{j=1}^N G\_{ij}\right) V\_i \tag{30}$$

$$\mathcal{S}^{\in 1} \qquad \qquad (i, j) \qquad \mathbb{U}^{=1} \quad \text{/}$$

$$\mathcal{Q}\_i^k = \sum\_{\mathcal{S}^{\in}} \mathcal{Q}\_{\mathcal{S}} - \mathcal{Q}\_{i, d} = \sum\_{(i, j)} \mathcal{Q}\_{ij}^k + \left(\sum\_{j=1}^N -B\_{ij}\right) V\_i + Q\_{i, \text{sc}} \tag{31}$$

*N*

*Pid*, and imaginary parts of the node admittance matrix, respectively; *Qi*,sc is the reactive power output of SCs at bus *i*.; <sup>2</sup> *V v i i* = represents the square of voltage magnitude at node *i*, and it is used as an independent variable in this paper. where *Pi*,*<sup>d</sup>* and *Qi*,*<sup>d</sup>* are the active and reactive loads at bus *i*; *Gij* and *Bij* are the real and imaginary parts of the node admittance matrix, respectively; *Qi*,sc is the reactive power output of SCs at bus *i*.; *V<sup>i</sup>* = *v* 2 *i* represents the square of voltage magnitude at node *i*, and it is used as an independent variable in this paper.

#### 2.2.3. Branch Power Balancing Constraints 2.2.3. Branch Power Balancing Constraints

and

where

Equations (19) and (20) are the branch power balancing constraints for those branches without OLTCs. For those branches with OLTCs, the branch power balancing constraints are as follows: Equations (19) and (20) are the branch power balancing constraints for those branches without OLTCs. For those branches with OLTCs, the branch power balancing constraints are as follows:

$$\text{S. I. II } \text{uose Dranuues winn OILCS, une bonant pwower panantung consunaus}$$

$$P\_{ij}^{k} = g\_{ij} V\_{i, \text{OLTC}} - g\_{ij}^{\text{P}} \frac{V\_{i, \text{OLTC}} + V\_j}{2} - b\_{ij}^{\text{P}} (\theta\_{ij} - \theta\_{ij, 0}) + g\_{ij}^{\text{P}} \frac{V\_{i \text{ILTC}}^{\text{OLTC}}}{2} \tag{32}$$

$$b\_{ij}^{k} = -b\_{ij} V\_{i, \text{OLTC}} + b\_{ij}^{\text{Q}} \frac{V\_{i, \text{OLTC}} + V\_j}{2} - g\_{ij}^{\text{Q}} (\theta\_{ij} - \theta\_{ij, 0}) - b\_{ij}^{\text{Q}} \frac{V\_{i j, \text{L}}^{\text{OILTC}}}{2} \tag{33}$$

$$\mathbf{Q}\_{ij}^{k} = -b\_{ij}\mathbf{V}\_{i\text{OLTC}} + b\_{ij}^{\text{Q}}\frac{V\_{i\text{OLTC}} + V\_{j}}{2} - g\_{ij}^{\text{Q}}(\theta\_{ij} - \theta\_{ij,0}) - b\_{ij}^{\text{Q}}\frac{V\_{i\text{OLTC}}^{\text{Q}}}{2} \tag{33}$$

$$\mathbf{V}\_{ij\text{OLTC}}^{\text{Q}\text{LTC}} = 2\frac{v\_{i\text{OLTC},0} - v\_{j,0}}{2}\left(V\_{\text{OLTC},0} - V\_{j}^{2}\right) - \left(V\_{\text{OLTC},0} - V\_{\text{OL}}\right)^{2} \tag{34}$$

$$\mathbf{Q}\_{ij}^{k} = -b\_{ij} V\_{i\text{OLTC}} + b\_{ij}^{\text{Q}} \frac{\mathbf{v}\_{i\text{OLTC}} - \mathbf{v}\_{j}}{2} - g\_{ij}^{\text{Q}} (\theta\_{ij} - \theta\_{ij,0}) - b\_{ij}^{\text{Q}} \frac{\mathbf{v}\_{ij,\text{L}}}{2} \tag{33}$$

$$V\_{ij,\text{L}}^{\text{OLTC}} = 2 \frac{v\_{i\text{OLTC},0} - v\_{j,0}}{v\_{i\text{OLTC},0} + v\_{j,0}} \left( V\_{i\text{OLTC}} - V\_{j}^{2} \right) - \left( V\_{i\text{OLTC},0} - V\_{j,0} \right)^{2} \tag{34}$$

2.2.4. Constraints of OLTCs

Denote *K* ∈ {*K*0, *K*1, *K*2, . . . , *Kn*} as the optional status of OLTCs. When the operation status of OLTCs is *Km*, the following constraint should be satisfied:

$$V\_i = K^2 V\_{i, \text{OLTC}} = K\_0^2 V\_{i, \text{OLTC}} + \sum\_{m=1}^t \left( K\_m^2 - K\_{m-1}^2 \right) V\_{i, \text{OLTC}} \tag{35}$$

However, *K* 2 *<sup>t</sup> Vi*,OLTC in the above equation contains a nonlinear term, which is detrimental to the solution of the model. A set of 0–1 variables {*z*1, *z*2, . . . , *z*n} is introduced to linearize it.

$$V\_i = K\_0^2 V\_{i, \text{OLTC}} + \sum\_{m=1}^{n} \Delta V\_m \tag{36}$$

$$0 \le \Delta V\_m \le z\_m V\_{\text{max}} \left( K\_m^2 - K\_{m-1}^2 \right) \tag{37}$$

$$
\Delta V\_m \le \left( K\_m^2 - K\_{m-1}^2 \right) V\_{i, \text{OL.TC}} \tag{38}
$$

$$
\Delta V\_m \ge \left( K\_m^2 - K\_{m-1}^2 \right) V\_{i, \text{OLTC}} - (1 - z\_m) V\_{\text{max}} \left( K\_m^2 - K\_{m-1}^2 \right) \tag{39}
$$

$$z\_m \ge z\_{m+1}, m = 1, 2, \dots, n-1\tag{40}$$

where ∆*V<sup>m</sup>* stands for the term (*K* 2 *<sup>m</sup>* − *K* 2 *m*−1 )*Vi*,OLTC, replacing *K* as the decision variable about OLTCS in the optimization process. When *z<sup>m</sup>* = 0, then ∆*V<sup>m</sup>* = 0 according to Equation (37); when *z<sup>m</sup>* = 1, then ∆*V<sup>m</sup>* = *K* 2 *<sup>m</sup>* − *K* 2 *m*−1 *Vi*,OLTC according to Equations (38) and (39). According to Equation (40), when *z<sup>t</sup>* − *z<sup>t</sup>* <sup>−</sup> <sup>1</sup> = 1, then the values of the 1st to the tth term of the set {*z*1, *z*2, . . . , *z*n}, then the values of the 1st to the *t*th term of the set are 1, while all the remaining values are 0. Substituting all the values into Equation (36), we can see that the set of constraints Equations (36)–(40) are equivalent to the original constraints Equation (35).

#### 2.2.5. Constraints of SCs

The set {*Bi*,1, *Bi*,2, . . . , *Bi*,n} is used to represent the conductance of SCs available at bus *i*. Assuming that the capacitors are put in sequentially, i.e., *Bi*,*<sup>t</sup>* is put in before *Bi*,*t*+<sup>1</sup> , then the reactive power output of SCs can be expressed as follows:

$$Q\_{i, \text{sc}} = V\_i \sum\_{m=1}^{t} B\_{i, m} = \sum\_{m=1}^{t} \left( V\_i B\_{i, m} \right) \tag{41}$$

Similar to the modeling process of OLTCs, a set of 0–1 variables {*x*1, *x*2, . . . , *x*n} is introduced to represent the operational state of SCs, then the constraint Equation (41) can be equivalently represented by the following set of linear constraints:

$$Q\_{i, \text{sc}} = \sum\_{m=1}^{n} Q\_{i, m}^{\text{sc}} \tag{42}$$

$$0 \le Q\_{i,m}^{\rm sc} \le \propto\_m V\_{\rm max} B\_{i,m} \tag{43}$$

$$Q\_{i,m}^{\rm sc} \le V\_i B\_{i,m} \tag{44}$$

$$Q\_{i,m}^{\rm sc} \ge V\_i B\_{i,m} - (1 - \chi\_m) V\_{i,\max} B\_{i,m} \tag{45}$$

$$\mathbf{x}\_{t} \ge \mathbf{x}\_{t+1}, \mathbf{t} = \mathbf{1}\_{\prime}\mathbf{2}\_{\prime}\dots\mathbf{n}-1\tag{46}$$

where *Q*sc *i*,*m* stands for the term *ViBi*,*m*.

#### 2.2.6. Operational Constraints

For the maintenance of the system in a normal working condition, the following constraints need to be satisfied:

$$P\_{\mathcal{g}}^{\min} \le P\_{\mathcal{g}} \le P\_{\mathcal{g}}^{\max} \tag{47}$$

$$Q\_{\mathcal{S}}^{\min} \le Q\_{\mathcal{S}} \le Q\_{\mathcal{S}}^{\max} \tag{48}$$

$$v\_{i, \min}^2 \le V\_i \le v\_{i, \max}^2 \tag{49}$$

In addition, the linearized form of line capacity constraint can be referred to [20] and will not be repeated in this paper.

In the above transmission network ORPD subproblem, all constraints are linear when considering *V* = *v* <sup>2</sup> as an independent variable. Since discrete variables are included, the above-mentioned ORPD subproblem is a mixed-integer programming problem with linear constraints.

#### *2.3. Distribution Network Subproblem Model*

In this part, the distribution network subproblem model is established based on the SOCP method. In the ORPD of distribution network subproblems, in addition to the traditional OLTCs and SCs, the variables that need to be optimized also include Static Var Compensators (SVCs) and various controllable DGs. Since there are generally no conventional generators in the distribution network, using the minimum network loss as the objective function of the distribution network subproblems is in line with the economic principle. The objective function of the ORPD of the distribution network *DSO<sup>k</sup>* is as follows:

$$\min F\_{\rm D,k} = \sum\_{(i,j)} r\_{ij,k} L\_{ij,k} \tag{50}$$

where *L*ij,k represents the square of the current on branch *i-j*. The constraints of the DSO*<sup>k</sup>* subproblem are as follows:

#### *2.4. Power Flow Equation Constraints*

For a typical radial distribution network, the power flow constraint can be expressed as follows:

$$\left(P\_{ij,k}\right)^2 + \left(Q\_{ij,k}\right)^2 = L\_{ij,k}u\_{i,k} \tag{51}$$

$$\sum\_{i \in u(j)} \left( P\_{i\bar{j},k} - L\_{i\bar{j},k} r\_{i\bar{j},k} \right) + P\_{\mathsf{G}j,k} = \sum\_{t \in v(j)} \left( P\_{jt,k} \right) + P\_{\mathsf{D}j,k} \tag{52}$$

$$\sum\_{i \in \mathfrak{u}(j)} \left( Q\_{\mathrm{ij},k} - L\_{\mathrm{ij},k} \mathfrak{x}\_{\mathrm{ij},k} \right) + Q\_{\mathrm{G}\mathrm{j},k} = \sum\_{t \in \mathbb{D}(j)} \left( Q\_{\mathrm{j}t,k} \right) + Q\_{\mathrm{D}\mathrm{j},k} \tag{53}$$

$$u\_{j,k} = u\_{i,k} - 2\left(r\_{ij,k}P\_{ij,k} + x\_{ij,k}Q\_{ij,k}\right) + \left(\left(r\_{ij,k}\right)^2 + \left(x\_{ij,k}\right)^2\right)L\_{ijk} \tag{54}$$

In the above distribution network subproblem, Equation (51) contains quadratic terms, and SOCP is usually applied to transform it into the following form:

$$\left\| \begin{array}{cc} 2P\_{ijk} \\ 2Q\_{ij,k} \\ L\_{ij,k} - u\_{i,k} \end{array} \right\|\_2 \le L\_{ij,k} + u\_{i,k} \tag{55}$$

### *2.5. Security Constraints*

For a typical radial distribution network, the power flow constraint can be expressed as follows:

$$L\_{ij,k,\text{min}} \le L\_{ij,k} \le L\_{ij,k,\text{max}} \tag{56}$$

$$
u\_{i,k,\min} \le u\_{i,k} \le u\_{i,k,\max} \tag{57}$$

#### *2.6. Constraints of DGs*

There are a large number of wind turbines, photovoltaics (PVs), and micro gas turbines (GTs) in distribution networks. They are generally connected to the grid through inverters, have good reactive power potentials, and participate in the distribution network's reactive power optimization process. Depending on the types of DGs and their inverter control modes, they have different reactive power regulation characteristics.

The inverter of the micro GTs generally adopts the PQ decoupling control method, and the reactive power regulation capacity is determined by the active power output and the total capacity of the inverter:

$$Q\_{\rm DG,max} = \sqrt{S\_{\rm max}^2 - P\_{\rm DG}^2} \tag{58}$$

$$Q\_{\rm DG,min} = -Q\_{\rm DG,max} \tag{59}$$

where *Q*DG,max and *Q*DG,min represent the upper and lower limits of the reactive power regulation capability of the micro gas turbine; *S*max is the maximum apparent power of the inverter; and *P*DG is the active power output of the micro gas turbine.

For Doubly fed Induction Generators (DFIGs), the reactive power regulation capability is mainly determined by the reactive power capacity on the stator side and the total capacity of the inverter:

$$Q\_{\rm DG,max} = Q\_{\rm s,max} - Q\_{\rm c,min} \tag{60}$$

$$Q\_{\rm DG,min} = Q\_{\rm s,min} - Q\_{\rm c,max} \tag{61}$$

For PVs, the grid-connected inverters mainly have two control modes: voltage control and current control. Due to the limitations of the voltage control mood, the photovoltaic inverter usually adopts the current control mood. In the reactive power optimization process, the reactive power output of photovoltaics is jointly determined by the nodal voltage, active power output, and current constraints:

$$Q\_{\rm DG,max} = \sqrt{V\_{\rm DG}^2 I\_{\rm DG,max}^2 - P\_{\rm DG}^2} \tag{62}$$

$$Q\_{\rm DG,min} = 0\tag{63}$$

#### *2.7. Other Constraints*

The modeling of the OLTCs and SCs in the distribution networks is similar to that in the transmission network and will not be repeated here.

#### **3. Solving Algorithm of the CTD-ORPD**

This section describes the details of the algorithm AAL for solving the proposed CTD-ORPD framework.

The AAL algorithm is a novel distributed algorithm that combines the advantages of the Diagonal Quadratic Approximation (DQA) method and the ADMM method. It uses a local augmented Lagrangian function similar to that of DQA, while discarding its inner loop step. Compared to ADMM, the updated rules for original variables and dual variables are improved. The original variables are updated one more time compared to ADMM on the basis of the solution of the local augmented Lagrangian function, which is then used for the update of the dual variables. In addition, a fully distributed rule is used for the update of dual variables in the AAL algorithm [14].

Based on AAL, the global ORPD problem is decomposed into the TSO subproblem and several DSO*<sup>k</sup>* subproblems, and the subproblems are solved in a distributed manner until the optimal global solution is finally reached after several iterations. Coordination between the TSO subproblem and DSO*<sup>k</sup>* subproblems is achieved only through the interaction of information on coordinating variables, which consists of the variables of PCCs, including bus voltage magnitude and phase angle, active power output and reactive power output, and denoted as *x*CT and *x*CD in this paper. The following is the definition of the coordinating variables:

$$\mathbf{x\_{CT}} = \bigcup\_{k=1}^{d} \mathbf{x\_{CT,k'}} \cdot \mathbf{x\_{CT,k}} = \begin{bmatrix} P\_{\mathbf{CT},k} & Q\_{\mathbf{CT},k} & V\_{\mathbf{CT},k} & \delta\_{\mathbf{CT},k} \end{bmatrix} \tag{64}$$

$$\mathbf{x\_{CD}} = \bigcup\_{k=1}^{d} \mathbf{x\_{CD}}\_{\mathbf{CD}k'} \mathbf{x\_{CD}} = \begin{bmatrix} P\_{\mathbf{CD}k} & Q\_{\mathbf{CD}k} & V\_{\mathbf{CD}k} & \delta\_{\mathbf{CD}k} \end{bmatrix} \tag{65}$$

The CTD-ORPD problem is an optimization problem defined by the objective functions and related constraints of the TSO and DSO*<sup>k</sup>* subproblems, which can be expressed as follows:

$$\min\_{\mathbf{x}} \text{F}\_{\text{T}}(\mathbf{x}\_{\text{T}}, \mathbf{x}\_{\text{CT}}) + \sum\_{k=1}^{d} F\_{\text{D},k}(\mathbf{x}\_{\text{D},k}, \mathbf{x}\_{\text{CD},k}) \tag{66}$$

$$s.t.\,g\_{\mathsf{T}}(\mathsf{x\_{\mathsf{T}}},\mathsf{x\_{\mathsf{CT}}}) \le 0, h\_{\mathsf{T}}(\mathsf{x\_{\mathsf{T}}},\mathsf{x\_{\mathsf{CT}}}) = 0\tag{67}$$

$$\log \mathsf{p}\_{\mathsf{D},k} \left( \mathbf{x}\_{\mathsf{CD},k'} \mathbf{x}\_{\mathsf{D},k} \right) \le 0,\\ h\_{\mathsf{D},k} \left( \mathbf{x}\_{\mathsf{CD},k'} \mathbf{x}\_{\mathsf{D},k} \right) = \mathbf{0} \, k = 1, 2, \ldots, \mathbf{d} \tag{68}$$

$$\mathbf{x\_{CT}} - \mathbf{x\_{CD}} = \mathbf{0} \tag{69}$$

In Equation (66), *F*<sup>T</sup> and *F*D,*<sup>k</sup>* are the objective functions of the TSO and DSO*<sup>k</sup>* subproblems established in the previous section, respectively. Equations (67) and (68) represent the inequality constraints and equality constraints of TSO and DSO*<sup>k</sup>* (*k* = 1,2, . . . ,d), respectively. *x*<sup>T</sup> and *x*D,*<sup>k</sup>* are local variables in the respective systems of TSO and DSO*<sup>k</sup>* (*k* = 1,2, . . . ,d). Equation (69) demonstrates the consistency constraint of the coordinating variables.

In the proposed distributed method, the CTD-ORPD problem Equation (66) is decomposed into (d + 1) subproblems and coordinated optimization is performed. According to AAL, the decomposed subproblems have the following expressions:

$$\min \Lambda\_{\text{TSO}}(\mathbf{x}\_{\text{T}}, \mathbf{x}\_{\text{CT}}, \boldsymbol{\lambda}) = \min \mathbb{F}\_{\text{T}}(\mathbf{x}\_{\text{T}}, \mathbf{x}\_{\text{CT}}) + \sum\_{k=1}^{d} \left[ \lambda\_{k} \left( \mathbf{x}\_{\text{CT},k} - \tilde{\mathbf{x}}\_{\text{CD},k} \right) + \frac{\rho}{2} \left\| \mathbf{x}\_{\text{CT},k} - \tilde{\mathbf{x}}\_{\text{CD},k} \right\|^{2} \right] \tag{70}$$

$$\begin{aligned} \min & \Lambda\_{\text{DSO},k} \left( \mathbf{x}\_{\text{D},k'} \mathbf{x}\_{\text{CD},k'} \lambda\_k \right) = \min \mathbf{F}\_{\text{D},k} \left( \mathbf{x}\_{\text{D},k'} \mathbf{x}\_{\text{CD},k} \right) + \\ & \left[ \lambda\_k \left( \tilde{\mathbf{x}}\_{\text{CT},k} - \mathbf{x}\_{\text{CD},k} \right) + \frac{\rho}{2} \left\| \tilde{\mathbf{x}}\_{\text{CT},k} - \mathbf{x}\_{\text{CD},k} \right\|^2 \right], k = 1, 2, \dots, d \end{aligned} \tag{71}$$

ΛTSO/ΛDSO,*<sup>k</sup>* are the local augmented Lagrangian functions of the TSO/DSO*<sup>k</sup>* subproblems, respectively. They are, respectively, composed of the corresponding objective function, the penalty term of the consistency constraint, and a quadratic term that improves the convergence speed of the Lagrangian function. *λ* is the Lagrange multiplier associated with coordinating variables, and *ρ* > 0 is the penalty term coefficient. The expression of *λ* is as follows:

$$
\lambda = \bigcup\_{k=1}^{d} \lambda\_{k\prime} \lambda\_k = \left[ \begin{array}{cccc} \lambda\_{P\cdot k} & \lambda\_{Q\cdot k} & \lambda\_{V\cdot k} & \lambda\_{\delta\cdot k} \end{array} \right] \tag{72}
$$

where *<sup>~</sup> <sup>x</sup>*CT,*<sup>k</sup>* and *<sup>~</sup> x*CD,*<sup>k</sup>* denote the reference values of the DSO*<sup>k</sup>* and TSO coordination variables, respectively, which are derived from the last optimization results of the TSO and DSO*<sup>k</sup>* subproblems. When optimizing the TSO subproblem, the coordinating variables of the DSO*<sup>k</sup>* subproblem *<sup>~</sup> x*CD,*<sup>k</sup>* serve as the benchmark values of *x*CT,*<sup>k</sup>* in the TSO subproblem. Similarly, *<sup>~</sup> x*CT,*<sup>k</sup>* would be served as the benchmark values of *x*CD,*<sup>k</sup>* when optimizing the DSO*<sup>k</sup>* subproblem. Figure 3 illustrates the overall pictorial view of the interactions between TSO and DSOs in the proposed distributed framework, where the solid line connections between TSOs and DSOs represent connections through transformers and the dotted line connections indicate information interactions.

and DSO*<sup>k</sup>* subproblems. When optimizing the TSO subproblem, the coordinating varia-

optimizing the DSO*<sup>k</sup>* subproblem. Figure 3 illustrates the overall pictorial view of the interactions between TSO and DSOs in the proposed distributed framework, where the solid line connections between TSOs and DSOs represent connections through transformers

serve as the benchmark values of

would be served as the benchmark values of

CT,*<sup>k</sup> x*

in the TSO

when

CD,*<sup>k</sup> x*

CD,*<sup>k</sup> x*

and the dotted line connections indicate information interactions.

CT,*<sup>k</sup> x*

bles of the DSO*<sup>k</sup>* subproblem

subproblem. Similarly,

**Figure 3.** Interactions between TSO and DSOs in the proposed distributed framework.

**Figure 3.** Interactions between TSO and DSOs in the proposed distributed framework. In each iteration, the transmission network subproblem is solved first, and then the coordinating variables are transferred to each distribution network subproblem. After the distribution network receives the coordinating variables, the subproblems are solved in parallel, and then the updated coordinating variables are transmitted to the transmission network. After the transmission grid receives data from all distribution grids, feasibility constraints and optimality constraints are checked. The optimal solution is obtained, and the optimization process ends if these two constraints are satisfied. Otherwise, the next iteration begins. Due to the parallelism of the subproblems, the time consumed by each iteration process depends on the transmission network optimization time and the distri-In each iteration, the transmission network subproblem is solved first, and then the coordinating variables are transferred to each distribution network subproblem. After the distribution network receives the coordinating variables, the subproblems are solved in parallel, and then the updated coordinating variables are transmitted to the transmission network. After the transmission grid receives data from all distribution grids, feasibility constraints and optimality constraints are checked. The optimal solution is obtained, and the optimization process ends if these two constraints are satisfied. Otherwise, the next iteration begins. Due to the parallelism of the subproblems, the time consumed by each iteration process depends on the transmission network optimization time and the distribution network with the longest optimization time. The detailed iterative steps of the algorithm are as follows.


$$\mathbf{x}\_{\rm T}^{t} = \mathbf{x}\_{\rm T}^{t-1} + \tau(\mathbf{x}\_{\rm T}^{\rm t} - \mathbf{x}\_{\rm T}^{t-1}) \tag{73}$$

$$
\tilde{\mathbf{x}}\_{\rm CT} = \mathbf{x}\_{\rm CT}^t = \mathbf{x}\_{\rm CT}^{t-1} + \tau(\hat{\mathbf{x}}\_{\rm CT}^t - \mathbf{x}\_{\rm CT}^{t-1}) \tag{74}
$$

3. Optimization of the DSO*<sup>k</sup>* subproblem: Optimize the problem Equation (71) at fixed *<sup>λ</sup>* and *<sup>~</sup> x*CT,*<sup>k</sup>* , and obtain the local/coordinating variables *^ x t* CD/ *^ x t* <sup>D</sup>. After the following update steps are completed, then transfer *<sup>~</sup> x*CD,*<sup>k</sup>* to TSO.

$$\mathbf{x}\_{\mathbf{D},k}^{t} = \mathbf{x}\_{\mathbf{D},k}^{t-1} + \tau(\mathbf{x}\_{\mathbf{D}}^{\star t} - \mathbf{x}\_{\mathbf{D},k}^{t-1}) \tag{75}$$

$$\tilde{\mathbf{x}}\_{\text{CD},k} = \mathbf{x}\_{\text{CD},k}^{t} = \mathbf{x}\_{\text{CD},k}^{t-1} + \tau(\hat{\mathbf{x}}\_{\text{CD}}^{t} - \mathbf{x}\_{\text{CD},k}^{t-1}) \tag{76}$$


$$\mathbf{x}\_{\rm CT}^{t} - \tilde{\mathbf{x}}\_{\rm CD} < \boldsymbol{\Psi}\_{\prime} \,\, \mathbf{x}\_{\varepsilon}^{t} - \overset{\star}{\mathbf{x}}\_{\varepsilon}^{t} < \gamma\_{\varepsilon\prime} \,\, \mathbf{F}\_{\varepsilon}^{t} - \overset{\star}{\mathbf{F}}\_{\varepsilon}^{t-1} < \mathfrak{f}\_{\varepsilon\prime} \,\, \boldsymbol{e} \in \{\text{TSO}, \text{DSO}\_{1}, \dots, \text{DSO}\_{\mathbf{d}}\} \tag{77}$$

$$
\lambda^{t+1} = \lambda^t + \rho \tau (\mathbf{x}\_{\rm CT}^t - \tilde{\mathbf{x}}\_{\rm CD}) \tag{78}
$$

In Equation (77), *ψ*, *γ<sup>e</sup>* , and *ξ<sup>e</sup>* together determine the accuracy of the algorithm, all of which took the value of 10−<sup>2</sup> in this paper.

Based on the concept of independent system operators, the program only needs to exchange a small amount of boundary information, which protects the information privacy and independent decision-making of system operators at all levels. Each distribution network subproblem can be optimized in parallel, which is beneficial to improving computational efficiency.

#### **4. Analysis of the Simulation Case**

### *4.1. Introduction of the Simulation Case*

In order to prove the effectiveness of the proposed distributed CTD-ORPD framework, three test cases were constructed based on the IEEE-30 node system (transmission network) and the IEEE-33 node system (distribution network) for simulation calculation.


In the three test cases, the transmission network and the distribution network were connected through 132/12.66 kv transformers. The high voltage side bus of the transformer was treated as the PCC. The active and reactive power exchanged between transmission and distribution networks through PCC, as well as the magnitude and phase angle of PCC, were used as the coordinating variables in the process of transmission and distribution co-optimization. The power reference value of both the distribution network and the transmission network was set to 100 MVA.

In the IEEE-30 node system, the OLTCs were set to be adjustable within 0.95~1.05, and the adjustment step was 0.01.

In the IEEE-33 node systems, nodes 2, 4, 6, 10, 13, 24, and 28 were connected to DGs, and the detailed parameters of the DGs are shown in Table 1.


**Table 1.** Buses and parameters of the DGs.

Since the accuracy of the linearization of the power flow equation in the transmission network is related to the quality of the initial value (*v*0, θ0), the latest result of the integrated power flow calculation was used as the value of (*v*0, *θ*0).

Other parameters in the transmission and distribution grids, such as load data, impedance data, and unit cost coefficients, were derived from MATPOWER (v.7.1, Cornell University's Charles H. Dyson School, U.S.).

The program in this paper was established in MATLAB R2019b (v.9.7, MathWorks, U.S.) and ran on a computer with Intel(R) Core(TM) i7-10850H CPU @ 2.70GHz CPU and 32 GB memory.

#### *4.2. Comparison with Other Methods*

To verify the superiority of the transmission and distribution coordination method proposed in the paper over the traditional independent method and centralized method, the centralized method, the independent method and the method proposed in the paper were used for simulation calculations for three test cases. "Centralized method" means that the distribution network is regarded as a new branch connected to the PCC node in the transmission network, and the centralized optimization calculation is performed by the control center. "Independent method" means that there is no information interaction between the transmission network and the distribution network, and independent optimization is carried out according to the boundary power flow information.

Tables 2–4 shows the results of solving the three test cases by different methods, and the boundary voltage magnitudes are in the form of per-unit values. It can be seen from the tables that in the three calculation cases, the network losses and system operating costs of the independent optimization method are significantly higher than those of the centralized optimization method. However, the results obtained by the method proposed in this paper are very close to that of the centralized optimization method. This is due to the fact that the initial value (*v*0, *θ*0) used in the linearization of the power flow equations of the transmission network adopts the result of the last integrated power flow calculation. This proves that our proposed method can obtain a solution with high precision when the initial value (*v*0, *θ*0) has high accuracy. This also demonstrates that it is feasible and effective to realize the ORPD problem of the global system through the cooperative calculation between the transmission and distribution networks.


**Table 2.** Comparison of the results of solving test case 1 by different methods.

In test case 1, the method proposed in the paper converges after three iterations, the total duration is 4.3 s, and the total duration of centralized optimization is 1.890 s. The total duration of the algorithm in this paper is 2.3 times that of centralized optimization. In test case 2, the method proposed in this paper converges to the optimal solution after four iterations. The whole process takes 5.1 s, and the total duration of centralized optimization is 7.269 s. At this time, the total duration of the method proposed in this paper is only 0.7 times that of centralized optimization. As for the situation in test case 3, the method proposed in this paper converges to the optimal solution after four iterations. The whole process takes 5.8 s, and the total duration for centralized optimization is 21.326 s. Eventually, the method proposed in this paper is obviously less time-consuming than centralized optimization, and the total duration of the algorithm in this paper is only 0.23 times that for

centralized optimization. It can be inferred that, when the number of distribution networks connected to the transmission grid is further increased, the time consumption of centralized optimization increases sharply, while the total time consumption of the method proposed in this paper would be basically stable. This is because the distribution network optimization subproblems are solved in parallel in the proposed distributed computing method. In this case, the total duration is only related to the number of iterations, the time consumption of the transmission network subproblem, and the most time-consuming distribution network subproblem, but has little to do with the number of connected distribution networks. At the same time, the linearization approximation on the transmission subproblem greatly reduces the complexity and time consumption of the transmission subproblem, so the total time consumption of the method proposed in this paper is relatively low.

**Table 3.** Comparison of the results of solving test case 2 by different methods.


**Table 4.** Comparison of the results of solving test case 3 by different methods.


Figures 4 and 5 show the voltage distribution of TSO and D26 using different methods to solve test case 1, respectively. (The results of test case 2 and test case 3 are similar.) Tables 5 and 6 present the optimization results of controllable resources in TSO and D26, respectively, while Table 7 shows the power exchange between TSO and D26. The result indicates that the proposed method makes fuller use of the reactive power regulation potential of the DGs in the distribution network. On the one hand, it improves the voltage

distribution and reduces the network losses in D26. On the other hand, the demand of D26 for remote reactive resources in TSO is reduced, thus further improving the voltage distribution and reducing the network losses in TSO. As can be seen from the figures, the nodal voltage distributions obtained by the proposed method are very close to that of the centralized optimization method. For the transmission grid, the local voltage distribution is improved by making full use of the reactive power regulation capability of the distribution grid in the transmission and distribution coordination optimization process. In either the proposed method or the centralized optimization method, the voltage magnitudes of the PCC node of the transmission network, i.e., node 26 and its nearby nodes, are significantly improved when compared to that of the independent optimization case. In addition, Appendix A shows the detailed branch power flow of TSO, which demonstrates the influence of coordinated transmission and distribution ORPD on the branch power flow of TSO. distribution and reduces the network losses in D26. On the other hand, the demand of D26 for remote reactive resources in TSO is reduced, thus further improving the voltage distribution and reducing the network losses in TSO. As can be seen from the figures, the nodal voltage distributions obtained by the proposed method are very close to that of the centralized optimization method. For the transmission grid, the local voltage distribution is improved by making full use of the reactive power regulation capability of the distribution grid in the transmission and distribution coordination optimization process. In either the proposed method or the centralized optimization method, the voltage magnitudes of the PCC node of the transmission network, i.e., node 26 and its nearby nodes, are significantly improved when compared to that of the independent optimization case. In addition, Appendix A shows the detailed branch power flow of TSO, which demonstrates the influence of coordinated transmission and distribution ORPD on the branch power flow of TSO.

consumption of centralized optimization increases sharply, while the total time consumption of the method proposed in this paper would be basically stable. This is because the distribution network optimization subproblems are solved in parallel in the proposed distributed computing method. In this case, the total duration is only related to the number of iterations, the time consumption of the transmission network subproblem, and the most time-consuming distribution network subproblem, but has little to do with the number of connected distribution networks. At the same time, the linearization approximation on the transmission subproblem greatly reduces the complexity and time consumption of the transmission subproblem, so the total time consumption of the method proposed in this

Figures 4 and 5 show the voltage distribution of TSO and D26 using different methods to solve test case 1, respectively. (The results of test case 2 and test case 3 are similar.) Tables 5 and 6 present the optimization results of controllable resources in TSO and D26, respectively, while Table 7 shows the power exchange between TSO and D26. The result indicates that the proposed method makes fuller use of the reactive power regulation potential of the DGs in the distribution network. On the one hand, it improves the voltage

*Energies* **2022**, *15*, 3867 16 of 23

paper is relatively low.

**Figure 4.** Voltage distribution of TSO with different methods on test case 1. **Figure 4.** Voltage distribution of TSO with different methods on test case 1.

*Energies* **2022**, *15*, 3867 17 of 23

**Figure 5.** Voltage distribution of D26 with different methods on test case 1. **Figure 5.** Voltage distribution of D26 with different methods on test case 1.

**Table 5.** Power output of generators in TSO with different methods on test case 1.

**Independent Method Proposed Method Centralized Method P (MW) Q (MVAr) P (MW) Q (MVAr) P (MW) Q (MVAr)**

**Table 6.** Reactive power output of DGs in D26 with different methods on test case 1.

**Table 7.** Power transmitted from TSO to D26 with different methods on test case 1.

Active Power/MW 2.9310 2.9266 2.9253 Reactive Power/MVar 1.4407 0.8632 0.8628

In order to test the superiority of the proposed method in relation to the other distributed methods, ADMM [13], G-MSS [9], ATC [28], AAL (without the linearized approximation method to model the TSO subproblem), and the proposed method in this paper

It can be seen from Table 8 that the operating costs of the systems optimized by various methods are not much different, and all the results are fairly close to those of the centralized optimization method, which demonstrates the effectiveness of various distributed methods for solving transmission and distribution synergy problems. However, compared with the other methods, the iterations and the total duration of the proposed method are significantly reduced. This means that the accuracies of various distributed

DFIG-2/MVar 0.1264 0.3712 0.3717 GT-4/MVar 0.2012 0.3873 0.3869 PV-6/MVar 0.2367 0.3426 0.3430 PV-13/MVar 0.3246 0.3651 0.3654 DFIG-28/MVar 0.4000 0.4000 0.4000

**Independent Method Proposed Method Centralized Method**

**Independent Method Proposed Method Centralized Method**

*4.3. Comparison of Different Distributed Algorithms*

were applied to test case 1, and the results are shown in Table 8.

G1 41.52 −5.13 41.31 −5.14 41.31 −5.14 G2 55.37 2.53 55.09 2.49 55.09 2.49 G3 16.13 35.21 16.05 35.22 16.05 35.22 G4 22.7 33.59 22.58 33.35 22.58 33.35 G5 16.13 6.87 16.05 6.63 16.05 6.63 G6 39.62 31.95 39.44 30.85 39.44 30.85


**Table 5.** Power output of generators in TSO with different methods on test case 1.

**Table 6.** Reactive power output of DGs in D26 with different methods on test case 1.


**Table 7.** Power transmitted from TSO to D26 with different methods on test case 1.


#### *4.3. Comparison of Different Distributed Algorithms*

In order to test the superiority of the proposed method in relation to the other distributed methods, ADMM [13], G-MSS [9], ATC [28], AAL (without the linearized approximation method to model the TSO subproblem), and the proposed method in this paper were applied to test case 1, and the results are shown in Table 8.



It can be seen from Table 8 that the operating costs of the systems optimized by various methods are not much different, and all the results are fairly close to those of the centralized optimization method, which demonstrates the effectiveness of various distributed methods for solving transmission and distribution synergy problems. However, compared with the other methods, the iterations and the total duration of the proposed method are significantly reduced. This means that the accuracies of various distributed methods are almost the same, but the method proposed in this paper has a higher efficiency. On the one hand, the AAL algorithm, which is the basis of the proposed method in the paper, combines the advantages of other algorithms to obtain improved convergence performance and solution efficiency. On the other hand, the linear approximation is applied to the power flow equations of the TSO subproblem, which greatly reduces the complexity of the optimization model, thereby improving the calculation speed and convergence performance. With the simultaneous application of the novel linearized approximation

method and AAL algorithms, the proposed method achieves a significant improvement in convergence performance and solution efficiency compared to other methods.

The above results are obtained with a convergence accuracy of 10−<sup>2</sup> . Figures 6 and 7 illustrate the system operating costs and iterations of the various methods for different convergence accuracies. As can be seen, the proposed method consistently achieves better optimization results as well as convergence performance compared to other distributed algorithms when the convergence accuracy varies. At the same time, as the convergence accuracy increases, the optimization results of various methods gradually converge and approach the results of centralized optimization, but the number of required iterations also increases. As the convergence accuracy increases from 10−<sup>2</sup> to 10−<sup>3</sup> , the quality of the optimization results of various algorithms improves significantly, while the number of iterations increases only slightly. When the convergence accuracy is increased from 10−<sup>4</sup> to 10−<sup>5</sup> , the quality of the optimization results of various methods improves just a little, while the number of required iterations increases a lot. We can thus see that the distributed methods can obtain the same results as the centralized optimization when the convergence accuracy is high enough, however, at the expense of the convergence performance. In practice, a reasonable compromise between optimization accuracy and convergence performance should be made before the computation process. It can also be seen that, when compared to the case of using only the AAL algorithm for distributed computation without using the novel linear approximation method to model the TSO subproblem, the proposed method in the paper has a better convergence performance at all convergence accuracies, which indicates that using both methods can effectively improve the convergence performance of the transmission and distribution collaborative framework. Furthermore, the inconsistency in the total operating cost between the two is caused when the nonlinear network loss term is linearized according to the linearized approximation method. *Energies* **2022**, *15*, 3867 19 of 23 *Energies* **2022**, *15*, 3867 19 of 23

**Figure 6.** System operating cost with various algorithms under different accuracy. **Figure 6.** System operating cost with various algorithms under different accuracy. **Figure 6.** System operating cost with various algorithms under different accuracy.

*4.4. Effect of the Linearized Approximation on the TSO Subproblem* **Figure 7.** Iterations of various algorithms under different accuracy. **Figure 7.** Iterations of various algorithms under different accuracy.

As can be seen from the analysis of the above arithmetic cases, the nonlinear optimization solver used by MATPOWER converges in all arithmetic scenarios. However, the linearized approximation method used in the paper for the TSO subproblem has the fol-

As can be seen from the analysis of the above arithmetic cases, the nonlinear optimization solver used by MATPOWER converges in all arithmetic scenarios. However, the

1. All formulas are convex functions, which theoretically guarantee convergence;

1. All formulas are convex functions, which theoretically guarantee convergence;

3. The linearized OPF model of the transmission grid subproblem leads to a significant reduction in the complexity of the global optimization model, especially considering

3. The linearized OPF model of the transmission grid subproblem leads to a significant reduction in the complexity of the global optimization model, especially considering

However, the accuracy of the linearized approximation of the flow equations de-

However, the accuracy of the linearized approximation of the flow equations de-

, and in the previous analysis, the results

, and in the previous analysis, the results

 P and Q

 P and Q

> 

 

2 N 2 N <sup>1</sup> , <sup>1</sup>

2 N 2 N <sup>1</sup> , <sup>1</sup>

0 0 ( , ) *v θ*

0 0 ( , ) *v θ*

. The above

. The above

(79)

(79)

are introduced to

are introduced to

0 0 ( , ) *v θ*

0 0 ( , ) *v θ*

analysis shows that this approach causes little error in the optimization process when the loads do not vary much. To further verify the feasibility of the linearized approximation

analysis shows that this approach causes little error in the optimization process when the loads do not vary much. To further verify the feasibility of the linearized approximation

analyze the errors caused by the linearization of the power flow equations when the loads

analyze the errors caused by the linearization of the power flow equations when the loads

N N *i i i i i i P P*

 *Q Q* − − = + = + 

N N *i i i i i i P P*

 *Q Q* − − = + = + 

,d ,d <sup>P</sup> ,d ,d <sup>Q</sup>

,d ,d <sup>P</sup> ,d ,d <sup>Q</sup>

the weight of the TSO time consumption in the global optimization process.

the weight of the TSO time consumption in the global optimization process.

**Figure 7.** Iterations of various algorithms under different accuracy.

*4.4. Effect of the Linearized Approximation on the TSO Subproblem*

2. The local optimum corresponds to the global optimum;

2. The local optimum corresponds to the global optimum;

method adopted in the paper, load variation coefficients

method adopted in the paper, load variation coefficients

vary. The node loads after changes are as follows:

vary. The node loads after changes are as follows:

lowing unique advantages:

lowing unique advantages:

pends on the accuracy of the initial value

pends on the accuracy of the initial value

of the latest integrated flow calculation are used as the initial value

of the latest integrated flow calculation are used as the initial value

#### *4.4. Effect of the Linearized Approximation on the TSO Subproblem*

As can be seen from the analysis of the above arithmetic cases, the nonlinear optimization solver used by MATPOWER converges in all arithmetic scenarios. However, the linearized approximation method used in the paper for the TSO subproblem has the following unique advantages:


However, the accuracy of the linearized approximation of the flow equations depends on the accuracy of the initial value (*v*0, *θ*0), and in the previous analysis, the results of the latest integrated flow calculation are used as the initial value (*v*0, *θ*0). The above analysis shows that this approach causes little error in the optimization process when the loads do not vary much. To further verify the feasibility of the linearized approximation method adopted in the paper, load variation coefficients *α*<sup>P</sup> and *α*<sup>Q</sup> are introduced to analyze the errors caused by the linearization of the power flow equations when the loads vary. The node loads after changes are as follows:

$$P\_{i, \mathbf{d}}' = P\_{i, \mathbf{d}} \times \left(1 + \mathbf{a}\_{\mathbf{P}} \times \frac{2i - \mathbf{N}}{\mathbf{N}}\right), Q\_{i, \mathbf{d}}' = Q\_{i, \mathbf{d}} \times \left(1 + \mathbf{a}\_{\mathbf{Q}} \times \frac{2i - \mathbf{N}}{\mathbf{N}}\right) \tag{79}$$

where *P* 0 *i*,d/*Q*<sup>0</sup> *<sup>i</sup>*,d represents the changed load at bus *i*, and N is the number of transmission grid buses. When calculating the errors, the calculated results after the load change are used as the base values, and the errors at different load variation coefficients are shown in Table 9. As can be seen from the table, the errors in operating costs as the objective function of the optimization problem increase when the absolute values of *α*<sup>P</sup> and *α*<sup>Q</sup> increase, because the errors due to the linearization of the network losses become larger in that case. It can also be seen that the operating costs errors of the system and the modeling accuracy of *Q* are satisfactory for different load variation coefficients. This shows that the method in the paper of directly adopting the results of the latest integrated power flow calculation as the initial values (*v*0, *θ*0) and linearizing the transmission grid power flow equations is feasible and effective when the accuracy requirement is not particularly high. Alternatively, when a higher accuracy is required, the accuracy of the linearized approximation method can be further promoted by using the warm-start iterative method described in [20], where only one more iteration is required to obtain a solution significantly close to the exact modeling case.


**Table 9.** Errors of the linearized approximation method at different load variation coefficients.

#### **5. Conclusions**

This paper established a distributed framework of the coordinated transmission and distribution networks optimal reactive power dispatch (CTD-ORPD) based on the AAL method. The global reactive power optimization problem of transmission and distribution grids was decomposed into a TSO subproblem and several DSO*<sup>k</sup>* subproblems. Each of the subproblems ran in a distributed manner and iterated until the optimal global solution is finally reached. The transmission grid and distribution grid subproblems achieved fully distributed cooperative optimal reactive power control only through the information interaction of the coordinating variables. The main conclusions are as follows:


The AAL algorithm and the novel linearized approximation method improve the convergence performance and solving efficiency of the cooperative transmission and distribution reactive power optimization problem in terms of the distributed computation method and the model building method, respectively, to achieve more stable convergence and faster solution. Considering the demand for regulating the voltage problems in the distribution networks in the background of the influx of DERs, the proposed CTD-ORPD framework in this paper is expected to play a good role in the short-term or real-time cooperative reactive power optimal control process of transmission and distribution, thus promoting further DER consumption.

**Author Contributions:** Conceptualization, B.Z. and X.S.; Data curation, B.Z.; Formal analysis, B.Z. and C.P.; Funding acquisition, X.S. and C.P.; Investigation, B.Z. and Y.B.; Methodology, B.Z.; Project administration, X.S.; Resources, T.W.; Software, B.Z. and T.W.; Supervision, X.S.; Validation, B.Z. and X.S.; Visualization, B.Z.; Writing—original draft preparation, B.Z.; Writing—review and editing, X.S. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded by Natural Science Foundation of China (U2066209), by technology project of China Southern Power Grid Co., Ltd. (060400KK52190017), by Deep mining of the demand characteristics of the sales side and application and industrialization of load data template technology (2020ZHCG0022).

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** Not applicable.

**Conflicts of Interest:** The authors declare no conflict of interest.

### **Appendix A**


**Table A1.** Branch power flow in TSO with different methods evaluated on test case 1.

### **References**


## *Article* **Optimal Power Sharing in Microgrids Using the Artificial Bee Colony Algorithm**

**Kalim Ullah, Quanyuan Jiang \*, Guangchao Geng, Sahar Rahim and Rehan Ali Khan**

College of Electrical Engineering, Zhejiang University, Hangzhou 310027, China; kalim125@zju.edu.cn (K.U.); ggc@zju.edu.cn (G.G.); sahar@ciit.edu.pk (S.R.); rehan@zju.edu.cn (R.A.K.) **\*** Correspondence: jqy@zju.edu.cn

**Abstract:** In smart grids, a hybrid renewable energy system that combines multiple renewable energy sources (RESs) with storage and backup systems can provide the most cost-effective and stable energy supply. However, one of the most pressing issues addressed by recent research is how best to design the components of hybrid renewable energy systems to meet all load requirements at the lowest possible cost and with the best level of reliability. Due to the difficulty of optimizing hybrid renewable energy systems, it is critical to find an efficient optimization method that provides a reliable solution. Therefore, in this study, power transmission between microgrids is optimized to minimize the cost for the overall system and for each microgrid. For this purpose, artificial bee colony (ABC) is used as an optimization algorithm that aims to minimize the cost and power transmission from outside the microgrid. The ABC algorithm outperforms other population-based algorithms, with the added advantage of requiring fewer control parameters. The ABC algorithm also features good resilience, fast convergence, and great versatility. In this study, several experiments were conducted to show the productivity of the proposed ABC-based approach. The simulation results show that the proposed method is an effective optimization approach because it can achieve the global optimum in a very simple and computationally efficient way.

**Keywords:** microgrid; ABC; power-sharing; cost optimization; renewable energy

**1. Introduction**

Today, the world's greatest challenges are the rapidly growing demand for electrical energy [1], rising electricity prices, the increasing use of non-renewable energy, the limits of conventional energy for power generation, global warming, global climate change, and related environmental problems [2]. All these challenges have resulted in the world being in a catastrophic economic and political crisis. Because of this, every government in the world is eager to expand renewable energy generation [3]. Renewable energy generation can help nations achieve their long-term development goal of providing safe, cheap, clean, environmentally friendly, and sustainable energy [4]. Despite the many advantages offered by renewable energy compared to conventional energy, they all have weaknesses in common, such as high weather vulnerability, low stability, and high unpredictability [5], all of which lead to low reliability and efficiency of energy generation [6]. Consequently, the hybrid renewable energy system can solve important problems and constraints in efficiency, reliability, and economy [7], which makes it an effective choice to meet the load requirements and support and improve the system [8]. The use of a central control unit to control the integrated power management of microgrids improves the efficiency, flexibility, and response time to fluctuations. The optimal power flow in the grid, the voltage level in each bus, and the transmission losses are all essential variables to be considered in efficient power systems [9]. The actual problem is an MILP problem, and the constraints on the optimal power flow are linear. For this purpose, a stochastic two-stage model of the interactions between the distribution system operator and many microgrids on the distribution line is used in [10].

**Citation:** Ullah, K.; Jiang, Q.; Geng, G.; Rahim, S.; Khan, R.A. Optimal Power Sharing in Microgrids Using the Artificial Bee Colony Algorithm. *Energies* **2022**, *15*, 1067. https://doi.org/10.3390/en15031067

Academic Editor: Adel Merabet

Received: 20 December 2021 Accepted: 19 January 2022 Published: 31 January 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

The Stackelberg game concept is used in [11] for the interconnections between generators and microgrids. This model considers all electrical power flow constraints, voltage constraints, and line losses. The main objective of each is to maximize the profit. erators and microgrids. This model considers all electrical power flow constraints, voltage constraints, and line losses. The main objective of each is to maximize the profit. MGs are classified in terms of power, control, mode, phase, and applications, as

The Stackelberg game concept is used in [11] for the interconnections between gen-

stage model of the interactions between the distribution system operator and many mi-

*Energies* **2022**, *15*, x FOR PEER REVIEW 2 of 21

crogrids on the distribution line is used in [10].

MGs are classified in terms of power, control, mode, phase, and applications, as shown in Figure 1. shown in Figure 1.

**Figure 1.** Classification of microgrids. **Figure 1.** Classification of microgrids.

A microgrid (MG), as shown in Figure 2, consists of various distributed energy sources (DERs), responsive loads, and critical loads. A common connection point (PCC) connects the MG to the main grid [12]. Each DER is connected to the power electronic interface (PEI) in both grid-connected and islanded modes to perform control, measurement, protection, and plug-and-play functions. An MG in the grid-connected mode benefits from sharing power with the main grid. However, in the event of a fault or failure of the main grid, MG switches to the islanded mode to ensure system stability. In this mode, critical loads are continuously supplied with power by efficiently integrating DERs, demand response (DR), and load shedding (LS). The central microgrid controller (MGCC) and local controllers (LCs) manage and coordinate the entire MG operation [13]. Efficient DER management and coordination in MG lead to higher system performance and longterm development [14]. Due to increased environmental awareness, socioeconomic growth, and the need to reduce greenhouse gas emissions, MGs are mainly composed of sustainable energy systems, such as renewable energy sources and energy-efficient systems that use local heat waste [15]. Various energy storage technologies that have potential for high penetration and integration in microgrids are mentioned in [16]. Different energy trading systems are examined for interactive energy trading, multienergy management, and resilient operations in [17]. The DSM technique is used in [18] to reduce the operating cost of the grid-connected microgrid. An optimization-based energy management system is used to reduce the generation and curtailment costs [19]. In [20], an opti-A microgrid (MG), as shown in Figure 2, consists of various distributed energy sources (DERs), responsive loads, and critical loads. A common connection point (PCC) connects the MG to the main grid [12]. Each DER is connected to the power electronic interface (PEI) in both grid-connected and islanded modes to perform control, measurement, protection, and plug-and-play functions. An MG in the grid-connected mode benefits from sharing power with the main grid. However, in the event of a fault or failure of the main grid, MG switches to the islanded mode to ensure system stability. In this mode, critical loads are continuously supplied with power by efficiently integrating DERs, demand response (DR), and load shedding (LS). The central microgrid controller (MGCC) and local controllers (LCs) manage and coordinate the entire MG operation [13]. Efficient DER management and coordination in MG lead to higher system performance and long-term development [14]. Due to increased environmental awareness, socioeconomic growth, and the need to reduce greenhouse gas emissions, MGs are mainly composed of sustainable energy systems, such as renewable energy sources and energy-efficient systems that use local heat waste [15]. Various energy storage technologies that have potential for high penetration and integration in microgrids are mentioned in [16]. Different energy trading systems are examined for interactive energy trading, multienergy management, and resilient operations in [17]. The DSM technique is used in [18] to reduce the operating cost of the grid-connected microgrid. An optimization-based energy management system is used to reduce the generation and curtailment costs [19]. In [20], an optimization controller is developed to control the energy management system for distributed energy sources in microgrids.

mization controller is developed to control the energy management system for distributed energy sources in microgrids. Table 1 shows a few examples of sustainable energy being used to run the energy management system from the literature review.

**Figure 2.** Microgrid setup**. Figure 2.** Microgrid setup.

**Table 1.** Sustainable energy (SE) system in microgrids.


[26] ✓ [27] ✓ ✓ ✓ [28] ✓ ✓ ✓ ✓ The microgrid EMS is a decision-making technique. For sustainable development, these techniques improve system efficiency, boost system reliability, decrease energy consumption, reduce DER operating costs, decrease system losses, and eliminate GHG emissions.

[29–31] ✓ The microgrid EMS is a decision-making technique. For sustainable development, these techniques improve system efficiency, boost system reliability, decrease energy consumption, reduce DER operating costs, decrease system losses, and eliminate GHG emissions. This paper presents a model for a hybrid renewable energy system integrated with a smart grid. The hybrid system includes wind turbines (WTs), photovoltaic (PV) systems, an electricity distribution company (Disco), gas turbines (GTs), and battery storage. Each component of the generation and load side is represented by a model. The hourly data of This paper presents a model for a hybrid renewable energy system integrated with a smart grid. The hybrid system includes wind turbines (WTs), photovoltaic (PV) systems, an electricity distribution company (Disco), gas turbines (GTs), and battery storage. Each component of the generation and load side is represented by a model. The hourly data of the wind speed and solar radiation on a daily basis are used as a case study in this paper. The proposed approach is used to optimize the power-sharing in the MGs to minimize the total cost of the system and the cost of each integrated microgrid. An artificial bee colony (ABC), as the optimization algorithm, is designed here to deal with the proposed model, with the objective of minimizing the cost and power imported from outside the MG. The idea of this paper is represented in Figure 3.

the wind speed and solar radiation on a daily basis are used as a case study in this paper. The proposed approach is used to optimize the power-sharing in the MGs to minimize the total cost of the system and the cost of each integrated microgrid. An artificial bee colony (ABC), as the optimization algorithm, is designed here to deal with the proposed model, with the objective of minimizing the cost and power imported from outside the

MG. The idea of this paper is represented in Figure 3.

**Figure 3.** The proposed idea. **Figure 3.** The proposed idea.

Figure 3 shows a microgrid (MG) connected to a distribution network. This MG contains multiple small MGs, which are connected with each other by electric lines. The electric line can be allowed to export and impart energy from small MGs. The small MG contains a mixture of components that consume and produce electricity (solar and wind plants) in addition to electricity storage units. The operational status of the small MG is divided into three sections: first, self-sufficiency in the event of equal production and consumption, so the MG will neither import nor export energy; second, import, which occurs when consumption is greater than production, as the MG needs energy; and third, export, which occurs when there is a surplus of energy within the MG and it needs to be exported. Optimization is a mathematical problem that may be found in all engineering do-Figure 3 shows a microgrid (MG) connected to a distribution network. This MG contains multiple small MGs, which are connected with each other by electric lines. The electric line can be allowed to export and impart energy from small MGs. The small MG contains a mixture of components that consume and produce electricity (solar and wind plants) in addition to electricity storage units. The operational status of the small MG is divided into three sections: first, self-sufficiency in the event of equal production and consumption, so the MG will neither import nor export energy; second, import, which occurs when consumption is greater than production, as the MG needs energy; and third, export, which occurs when there is a surplus of energy within the MG and it needs to be exported.

mains. This term's literal definition is "best possible or desirable". Because optimization problems are so broad and varied, it is a significant academic field. Optimization algorithms are classified into two types: deterministic and stochastic. Optimization is a mathematical problem that may be found in all engineering domains. This term's literal definition is "best possible or desirable". Because optimization problems are so broad and varied, it is a significant academic field.

Previously, tackling optimization issues required tremendous computational effort, which frequently failed as the problem size grew larger. This is why bio-inspired stochastic optimization algorithms are being used as computationally efficient alternatives to deterministic approaches. Metaheuristics are based on the iterative improvement of either a population of solutions (evolutionary algorithms or swarm-based algorithms) or a single solution (Tabu Search) to solve a given optimization problem, and they primarily use randomization and local search. The literature on bio-inspired algorithms (BIAs) for solving a wide variety of issues Optimization algorithms are classified into two types: deterministic and stochastic. Previously, tackling optimization issues required tremendous computational effort, which frequently failed as the problem size grew larger. This is why bio-inspired stochastic optimization algorithms are being used as computationally efficient alternatives to deterministic approaches. Metaheuristics are based on the iterative improvement of either a population of solutions (evolutionary algorithms or swarm-based algorithms) or a single solution (Tabu Search) to solve a given optimization problem, and they primarily use randomization and local search.

is vast, and various studies have reported on the usefulness of such tactics for handling difficult problems in the main disciplines of engineering in recent years. The two most prevalent and successful BIA classes or routes are evolutionary algorithms and swarmbased algorithms, both inspired by animals' collective behavior and natural development. In order to obtain a broader view on the subject, the algorithms were divided into regions based on where the inspiration for them came from in nature. The literature on bio-inspired algorithms (BIAs) for solving a wide variety of issues is vast, and various studies have reported on the usefulness of such tactics for handling difficult problems in the main disciplines of engineering in recent years. The two most prevalent and successful BIA classes or routes are evolutionary algorithms and swarmbased algorithms, both inspired by animals' collective behavior and natural development. In order to obtain a broader view on the subject, the algorithms were divided into regions based on where the inspiration for them came from in nature.

Swarm intelligence is a novel and rising paradigm used for creating adaptive systems in bio-inspired computing. In this sense, evolutionary computation (EC) is an extension of this. Swarm intelligence is based on organisms' collective social behaviors, whereas Swarm intelligence is a novel and rising paradigm used for creating adaptive systems in bio-inspired computing. In this sense, evolutionary computation (EC) is an extension

of this. Swarm intelligence is based on organisms' collective social behaviors, whereas evolutionary algorithms are based on species' genetic adaptability. Swarm intelligence, as defined in the literature, is the use of the collective intelligence of groups of simple organisms to solve problems, based on the behavior of actual insect swarms. The term "swarm" refers to the chaotic movement of particles in the affected region. Some important SI algorithms are particle swarm optimization (PSO), the ant colony optimization algorithm (ACO), the fish swarm algorithm (FSA), the firefly algorithm, and the artificial bee colony (ABC) methods, which have all been employed as optimization methodologies. In the foraging process, these algorithms, which were inspired by animals' collective behavior, exhibit decentralized self-organized patterns. However, the artificial bee colony (ABC) approach was used in this article [32].

The ABC algorithm outperforms or is similar to other population-based algorithms, with the added advantage of requiring fewer control parameters. The ABC algorithm also features good resilience, fast convergence, and great versatility.

The ABC algorithm, developed by Karaboga and Basturk, replicates the intelligent foraging behavior of a honeybee swarm. The ABC algorithm's artificial bee colony is made up of three categories of bees: hired bees, bystanders, and scouts. An employed bee is a spectator who does not participate in the dance but instead travels to the food source being frequented by the observer. The scout bee, on the other hand, performs random searches for fresh sources. The quality (or fitness) of a solution may be measured by comparing the location of a food source to the amount of nectar it generates. Beehives are built and then released into the two-dimensional search space. Bees build social relationships with one another while foraging for nectar. Intense bee–bee interactions are essential to the discovery of a solution.

#### **2. Problem Formulation**

#### *2.1. ABC Algorithm*

In 2005, Karaboga discovered the ABC algorithm, influenced by honey bee behavior. The algorithm of a honey bee colony has the ability to find the best quality food sources in nature with ease. Therefore, the concept of ABC was derived from the clever foraging behavior of honey bees to find suitable solutions to optimization problems. Generally, bee colonies are classified into three types according to their foraging ability: employed bees, onlooker bees, and scout bees. The employed bees are responsible for collecting nectar (food). They investigate the location of the food supply in advance and alert the scout bees about the quality of the food. Based on the information relayed by the employed bees, the scout bees wait in the swarm and decide whether to take advantage of a food source. The scout bees randomly search the environment for a new nectar supply, either from internal motivation or from likely external cues [33]. The quality (fitness) of the feasible solution to the optimization questions is related to the profitability of a nectar source. The presence of a nectar source indicates a feasible solution to the optimization issues. Each nectar source is visited by only one honey bee. In other words, the number of employed or onlooker bees is proportional to the number of nectar sources [34]. Employed bees maintain an excellent solution, onlooker bees accelerate convergence, and scout bees improve the ability to eliminate local optimums [35,36].

#### ABC Algorithm Iteration Steps

ABC algorithm's main steps are listed as follows [37,38]:

1. Initialization. Generate N random solutions (food sources) *Xi* (*i* = 1, 2, 3, . . . , *N*) in a dimensional searching space D, where N represents the number of food sources, which is half the size of the colony *Xi* (*i* = 1, 2, 3, . . . , *D*) is a D-dimensional solution vector. For *i* = 1, 2, 3, . . . , *N*, the *i*th food source in the original population, and the number of optimization population parameters is denoted by D.

2. During the honey collection stage, each employed bee creates a new nectar source in the food source's vicinity. When a new nectar source is compared to the previous one, the high probability will be memorized. Each onlooker bee assesses the attractiveness of nectar sources received from all employed bees and selects a food source with a high probability. Like the employed bees, she changes the source location in her memory and maintains a higher nectar supply. In these two phases, the following formula is employed to regenerate nectar sources:

$$l\_{\rm ij} = h\_{\rm ij} + \theta \cdot \left(h\_{\rm ij} - h\_{\rm kj}\right) \tag{1}$$

where *i* (*k* = 1, 2, 3, . . . , *N*), (*j* = 1, 2, 3, . . . , *D*), and *θ*[0, 1] is a random number that determines the generation range of *hij*0 *s* neighborhoods. As the search comes closer to an optimal solution, the number of neighborhoods available will decrease.

3. Food source selection. In the next step, the onlooker bees compare the probability calculated by the fitness value to select a food source. Nectar sources with a high probability are chosen with a high degree of certainty. The chance of being chosen for food sources is computed using the equation below:

$$P\_i = \frac{Fit\_i}{\sum\_{i}^{N} Fit\_i} \tag{2}$$

The fitness value of the *i*th solution, *Fit<sup>i</sup> i*(*i* = 1, 2, 3, . . . , *N*) may be determined using following equation:

$$Fit\_i = \left\{ \begin{array}{c} 1 \\ \hline 1 + f\_i \\ 1 + |f\_i| \text{ } if \, f\_i \le 0 \end{array} \right. \\ \tag{3}$$

If the quality of the new food source location is the same as or better than the previous one, the old one is updated with a new one, where *f<sup>i</sup>* is the value of the objective function for *i* = 1, 2, 3, . . . , *N*, which is unique to the optimization problem. Otherwise, the old one will be kept the same as the stage of employed bees.

4. Population elimination. A solution is considered to have fallen into a local optimum solution if it has not improved significantly after a specified number of trials, known as "max iteration", and the starting position is abandoned. As a consequence, the matching employed bees will become scout bees, and a new solution, which may be described as follows, will be generated at random in place of the discarded solution:

$$h\_i = \theta \* (h\_{-max} - h\_{-minj}) + h\_{-minj} \tag{4}$$

where *h*\_*maxj* and *h*\_*minj* are the *j*th and *i*th individual maximum and lowest values, respectively, and *i* and *j* are the same (1).

The ABC algorithm may be expressed in the following stages based on the preceding phases [39].Flowchart of ABC algorithm is shown in Figure 4.

**Figure 4.** ABC flow chart. **Figure 4.** ABC flow chart. **Figure 4.** ABC flow chart.

#### **3. Mathematical Modeling 3. Mathematical Modeling 3. Mathematical Modeling**

The microgrid architecture studied in this paper is represented in Figure 5. It includes wind turbines (WTs), gas turbines (GTs), photovoltaic (PV), batteries (BT), additional storage components, general loads, and essential loads with different characteristics. The connection point, also known as the point of common coupling (PCC), is the interface between this architecture's utility and microgrid systems. As a result, there are two types of modes. The microgrid architecture studied in this paper is represented in Figure 5. It includes wind turbines (WTs), gas turbines (GTs), photovoltaic (PV), batteries (BT), additional storage components, general loads, and essential loads with different characteristics. The connection point, also known as the point of common coupling (PCC), is the interface between this architecture's utility and microgrid systems. As a result, there are two types of modes. The microgrid architecture studied in this paper is represented in Figure 5. It includes wind turbines (WTs), gas turbines (GTs), photovoltaic (PV), batteries (BT), additional storage components, general loads, and essential loads with different characteristics. The connection point, also known as the point of common coupling (PCC), is the interface between this architecture's utility and microgrid systems. As a result, there are two types of modes.

**Figure 5.** Typical architecture of the microgrid system. **Figure 5.** Typical architecture of the microgrid system. **Figure 5.** Typical architecture of the microgrid system.

When the MG is connected to the main grid via PCC, it is in the grid-connected mode, and when it does not connect with the main grid, it is in the islanded mode. When the MG is connected to the main grid via PCC, it is in the grid-connected mode, and when it does not connect with the main grid, it is in the islanded mode. Renewable energy sources (RESs) can be connected via a DC, AC, or a hybrid DC/AC

Renewable energy sources (RESs) can be connected via a DC, AC, or a hybrid DC/AC bus. For most generators and loads, the appropriate configuration is determined by the type of output power. Therefore, DC bus coupling is preferred when both loads and generators are DC [40]. When the loads and generators are AC, then AC bus coupling is preferred [41] when the generation and load are mixed, such as AC and DC hybrid renewable energy sources (HRESs). A hybrid AC, DC bus coupling system is used [42], as shown in Figure 6. bus. For most generators and loads, the appropriate configuration is determined by the type of output power. Therefore, DC bus coupling is preferred when both loads and generators are DC [40]. When the loads and generators are AC, then AC bus coupling is preferred [41] when the generation and load are mixed, such as AC and DC hybrid renewable energy sources (HRESs). A hybrid AC, DC bus coupling system is used [42], as shown in Figure 6.

**Figure 6.** Schematic diagram of hybrid wind/PV/GT/battery storage. **Figure 6.** Schematic diagram of hybrid wind/PV/GT/battery storage.

#### *3.1. Hybrid Wind/PV/Battery Storage/Gas Turbine 3.1. Hybrid Wind/PV/Battery Storage/Gas Turbine*

The configuration shown in Figure 6 consists of wind energy, a PV energy system, battery storage, a gas turbine, and the main load. The response of this configuration is simple and easy to understand. Due to the bidirectional converter, the WT and the PV system are mainly responsible for supplying the main load. The excess power generated by wind and/or PV is stored in a battery storage system until the battery is fully charged to ௫ . Excess power generated above ௫ is supplied to dedicated loads, i.e., dummy loads, such as loads for cooling, home appliances and heating, and charging the batteries of emergency lights when the battery storage is full. When the load power exceeds the generated power, the batteries are used to make up the difference until they reach the minimum (). Suppose the battery is fully discharged by and the hybrid renewable energy sources cannot meet the microgrid's load demand. In this case, it imports energy from another microgrid. Moreover, when it is unable to purchase energy from another microgrid, the microgrid purchases power from the main grid to balance the load demand [43]. The configuration shown in Figure 6 consists of wind energy, a PV energy system, battery storage, a gas turbine, and the main load. The response of this configuration is simple and easy to understand. Due to the bidirectional converter, the WT and the PV system are mainly responsible for supplying the main load. The excess power generated by wind and/or PV is stored in a battery storage system until the battery is fully charged to *SOCmax* . Excess power generated above *SOCmax* is supplied to dedicated loads, i.e., dummy loads, such as loads for cooling, home appliances and heating, and charging the batteries of emergency lights when the battery storage is full. When the load power exceeds the generated power, the batteries are used to make up the difference until they reach the minimum *SOC* (*SOCmin*). Suppose the battery is fully discharged by *SOCmin* and the hybrid renewable energy sources cannot meet the microgrid's load demand. In this case, it imports energy from another microgrid. Moreover, when it is unable to purchase energy from another microgrid, the microgrid purchases power from the main grid to balance the load demand [43].

#### 3.1.1. Wind Energy System 3.1.1. Wind Energy System

Wind generation depends on both the wind speed and the height of the hub at a given location. The power–law equation [44] is used to calculate the wind speed at the hub height of WT using data collected at the anemometer height: (ℎ) and (ℎ) are the wind speeds at the hub height (ℎ) and the anemometer height (ℎ), respectively, and is the roughness factor. The value changes from location to location and over time at the same location: Wind generation depends on both the wind speed and the height of the hub at a given location. The power–law equation [44] is used to calculate the wind speed at the hub height of WT using data collected at the anemometer height: *u*(*h*) and *u*(*hg*) are the wind speeds at the hub height (*h*) and the anemometer height (*hg*), respectively, and *α* is the roughness factor. The value changes from location to location and over time at the same location:

$$V(h) = V\left(h\_{\mathcal{S}}\right) \left(\frac{h}{h\_{\mathcal{S}}}\right)^{\alpha} \tag{5}$$

where *V*(*h*) and *V hg* are the wind speeds at hub height *h* and anemometer height *hg*. α is a roughness factor and varies from location to location and time to time.

The output power of WT from the typical WT curve is as follows [45]:

$$p\_{wt} = \begin{cases} 0, \ v\_{wt} < v\_{ci}, v\_{wt} > v\_{co} \\\ p\_r \* \frac{v\_{wt}^2 - v\_{co}^2}{v\_r^2 - v\_{ci}^2}, \ v\_{ci} < v\_{wt} < v\_r \\\ p\_{r\prime} \quad v\_{ci} < v\_{wt} < v\_{co} \end{cases} \tag{6}$$

where *pwt* is the output power of the wind turbine, *p<sup>r</sup>* is the rated output power of the wind turbine, *vci* is the cut in wind speed of WT, *v<sup>r</sup>* is the rated wind speed of WT, and *vco* is the cut off wind speed of the wind turbine.

#### 3.1.2. PV Energy System

The solar radiation on a tilted surface (*Ht*) can be calculated using solar insolation, the ambient temperature, data from the PV panel manufacturer, the PV panel slope, and the site latitude and longitude [46,47]. The following equation [48] is used to compute the PV system's output power:

$$p\_{pv}(t) = h\_l(t) \* \text{PVA} \* \mu\_c(t) \tag{7}$$

where *µc*(*t*) is the PV system's hourly generating efficiency, which may be calculated in terms of the cell temperature using Equation (8) [49]:

$$
\mu\_{\mathcal{C}}(t) = \mu\_{\mathcal{C}} [1 - \beta\_{\mathcal{I}} \* (T\_{\mathcal{C}}(t) - T\_{\mathcal{C}})] \tag{8}
$$

where *β<sup>t</sup>* is the temperature coefficient, *µcr* and *Tcr* are the solar cell efficiency and temperature at maximum radiation solar flux *Tc*(*t*), It is an hourly solar cell temperature at ambient temperature (*Ta*):

$$T\_{\mathcal{C}}(t) = T\_a + \lambda h\_l(t) \tag{9}$$

where *λ* (Ross coefficient) is a coefficient that represents how the temperature increases above ambient as solar flux increases. The overall output of the PV array is:

$$p\_s = \left(\frac{G\_{pv}}{G\_0}\right) \* p\_r$$

where *p<sup>s</sup>* is the PV array output power, *Gpv* is the irradiation of solar, *G<sup>o</sup>* is the solar irradiance under standard test conditions (1000 W/m<sup>2</sup> ). *p<sup>r</sup>* is the rated power of solar [49]. Table 2 represents the PV parameter values.


NOCT 44 ◦C ΠPV 0 Cent/kWh

**Table 2.** PV panel parameters. Reproduced from [49], the (Journal of Energy storage): 2021.

#### 3.1.3. Battery Storage

The state of charge (*SOC*) of a battery is determined based on the energy balance between the wind, PV energy systems, and the load as given by the following equations after a particular time (*t*):

$$E\_b(t) = E\_b(t-1)(1 - \sigma) + P\_s \* \eta\_{bc} \tag{11}$$

where (11) is the battery charging mode equation:

$$E\_b(t) = E\_b(t-1)(1 - \sigma) - \frac{p\_d}{\eta\_{bd}} \tag{12}$$

where (12) is the battery discharging mode equation. *E<sup>b</sup>* is the energy of the battery bank, and *ηbc*, *ηbd* is the charging and discharging efficiency of the battery storage system. It is considered to be 90% and 85%, respectively, in [50], where *σ* is the battery self-discharge rate and is assumed to be 0.2% per day for most batteries [51]. *Ps* is the surplus power, and *pd* is the deficit power.

The battery bank should always follow the following limitations:

$$E\_{b,min} \le E\_b(t) \le E\_{b,max} \tag{13}$$

$$E\_b(t) = E\_b(t-1)(1-\sigma) \tag{14}$$

where *Eb*,*max* and *Eb*,*min* are the battery bank's maximum and minimum storage capacity, respectively. The following equation can be used to calculate *Eb*,*min*:

$$E\_{b,\text{min}} = DOD \ast E\_b \tag{15}$$

where *E<sup>b</sup>* is the battery nominal storage capacity, and *DOD* is the depth of discharge of a battery opposite to the *SOC* of a battery.

#### **4. Mathematical Modeling of the Proposed Approach**

The proposed approach of this article is to optimize the power transfer between microgrids to minimize the overall cost of the system and each microgrid. For this purpose, a mathematical model is designed for a net load for each microgrid first. The proposed idea is represented in Figure 3, and the proposed work is represented by the flowchart shown in Figure 7.

The proposed approach has two main tasks: storage; the other is energy sharing in the first part, which is energy storage. If the net load is greater than zero, storage is used to discharge energy to meet the load requirements. If the net load is less than zero, the extra system energy is either shared with other microgrids or stored in the storage system to reduce the total cost of electricity. Data about the load, PV, and wind in each MG is calculated. After obtaining data of the load, PV, and wind in each MG, the sharing parameters are initialized, such as the *SOC* of the battery, the size of the battery, and the minimum and maximum generation in each MG, the net load is calculated using the generation and load balance equations.

The proposed work shown in Figure 7 is about energy management in small MGs and is an attempt to reduce imports from energy distribution networks as much as possible, by linking several small microgrids together. These MGs contain different mixtures of energy production and consumption, which helps to increase the reliability of these small MGs and gradually dispense with power distribution networks. The novelty is that the difference in the energy mix between these small MGs and linking them together will help dispense with distribution networks, which will reduce energy losses and the price of electricity and provides the possibility for small consumers to benefit from the production and sale of energy.

Suppose a microgrid has a battery energy storage system. In this case, it will have two possibilities if the net load exceeds zero. The *SOC* of a battery is checked to observe whether it is above 20%, and the battery will be discharged to meet the load requirements. Still, if the net load is not greater than zero, there is some extra energy in the system, which may be used to charge the battery if the *SOC* of a battery is less than the maximum storage capacity.

**Figure 7.** Flowchart of the proposed idea. **Figure 7.** Flowchart of the proposed idea.

In the second task of the proposed approach, whether the net load is greater than or less than zero is checked. If it is greater than zero, the microgrid will import energy from other microgrids or the main grid. If the net load is less than zero, the microgrid will export energy to other microgrids or the main grid. In the second task of the proposed approach, whether the net load is greater than or less than zero is checked. If it is greater than zero, the microgrid will import energy from other microgrids or the main grid. If the net load is less than zero, the microgrid will export energy to other microgrids or the main grid.

The above process is iterated until optimal power-sharing among the microgrids is achieved. The above process is iterated until optimal power-sharing among the microgrids is achieved.

#### *4.1. Microgrids' Net Load 4.1. Microgrids' Net Load*

24 h PV profile :

The mathematical equation for Disco is: The mathematical equation for *Disco* is:

$$\text{Disco Net Load} = \text{L}\_{\text{D}} - \text{G}\_{\text{D}} \tag{16}$$

where is the load of the distribution company (Disco) and is the conventional generation in Disco: where *L<sup>D</sup>* is the load of the distribution company (*Disco*) and *G<sup>D</sup>* is the conventional generation in *Disco*:

where ெீଵ is the load of 1, ெீଵ is the rated power of PV in 1, and is the

$$\text{MG1 Net Load} = \text{L}\_{\text{MG1}} - \text{PV}\_{\text{MG1}} \ast \text{PF}\_{\text{PV}} \tag{17}$$

where *LMG*<sup>1</sup> is the load of *MG*1, *PVMG*<sup>1</sup> is the rated power of PV in *MG*1, and *PFPV* is the 24 h PV profile:

$$\text{MG2 Net Load} = \text{L}\_{\text{MG2}} - \text{G}\_{\text{D1}} - \text{G}\_{\text{D2}} - \text{WT}\_{\text{MG2}} \ast \text{PF}\_{\text{WT}} \tag{18}$$

where *LMG*<sup>2</sup> is the load of *MG*2, *GD*1, *GD*<sup>2</sup> are the generation of conventional generators 1 and 2, *WTMG*<sup>2</sup> is the wind turbine generation in *MG*2, and *PFWT* is the wind profile of 24 h

$$\text{MG3 Net Load} = \text{L}\_{\text{MG3}} - \text{PV}\_{\text{MG3}} \ast \text{PF}\_{\text{PV}} \tag{19}$$

*LMG*<sup>3</sup> is the load of *MG*3, *PVMG*<sup>3</sup> is the rated PV power, and *PFPV* is the PV profile for 24 h.

$$\text{MG4 Net Load} = \text{L}\_{\text{MG4}} - \text{WT}\_{\text{MG4}} \ast \text{PF}\_{\text{WT}} \tag{20}$$

*LMG*<sup>4</sup> is the load of *MG*4, *WTMG*<sup>4</sup> is the wind turbine generation in *MG*4, *PFWT* is the 24 h wind profile:

$$\text{MG5 Net Load} = L\_{\text{MG5}} - G\_D - \mathcal{W}T\_{\text{MG5}} \* PF\_{\text{WT}} \tag{21}$$

where *LMG*<sup>5</sup> is the load of *MG*5, *G<sup>D</sup>* is the generation of a conventional generator, *WTMG*<sup>5</sup> is the wind generation in *MG*5, and *PFWT* is the 24 h wind profile.

The net load is the difference between the load and generation inside the MG itself. It is used to determine whether the microgrid has a shortage or excess of energy to import or export to other microgrids with a shortage of energy, store it in battery energy storage, or sell it to the main grid.

#### *4.2. Energy Management Strategy*

The proposed HRES management algorithm is described in the following strategy. If the amount of power generated by RES surpasses the amount needed to meet the load requirements, the excess power will be used to charge the batteries until they reach their maximum capacity, *Eb*,*max*. The extra power in the batteries will be used to power the dummy load, *Pdummy*. The reason behind this can be described as follows:

$$\begin{aligned} \text{if } P\_{\mathcal{S}}(t) > P\_{\mathcal{L}}(t) \text{ and } \text{SOC} < E\_{b,\text{max}} \text{ then:}\\ P\_{bc} = [P\_{\mathcal{W}}(t) - P\_{\mathcal{L}}(t) + P\_{\mathcal{PV}}(t)] \eta\_{bc} \text{ Changing Process} \end{aligned} \tag{22}$$

$$\begin{aligned} \text{if } P\_{\mathcal{S}} > P\_L(t) \text{ and } \text{SOC} \ge E\_{b, \text{max}} \text{ then} \\ P\_{bc}(t) = 0 \\ P\_d(t) = \left[ P\_w(t) - P\_L(t) + P\_{PV}(t) \right] \end{aligned} \tag{23}$$

where *P<sup>d</sup>* is the power to the dummy load:

$$\begin{aligned} \text{if } P\_{\overline{w}}(t) < P\_{\mathcal{L}}(t) \text{ , } \left[P\_{\overline{w}}(t) + P\_{PV}(t)\right] > P\_{\mathcal{L}}(t) \text{ and} \\ \text{SOC} < E\_{b,\text{max}} \text{ then} \\ P\_{\text{bc}}(t) = \left[P\_{PV}(t) - \left(P\_{\mathcal{L}}(t) - P\_{\overline{w}}(t)\right)\right] \eta\_{\text{bc}} \end{aligned} \tag{24}$$

Equation (24) shows the charging process:

$$\begin{aligned} \text{if } P\_w(t) < P\_L(t) \text{ and } (P\_w(t) + P\_{PV}(t)) > P\_L(t) \text{ and} \\ \text{SOC} > E\_{b,\text{max}} \text{ then;} \\ P\_{bc}(t) = 0 \\ P\_d(t) = [P\_w(t) - P\_L(t) + P\_{PV}(t)] \end{aligned} \tag{25}$$

If the load demand requires more power than RES can provide, the battery will meet the demand until it is reduced to its minimal level *Eb*,*min*. If there is still a power shortage, the DG will make up for the deficit load demand. The equations that describe this logic are as follows:

$$\begin{aligned} \text{if } [P\_w(t) + P\_{PV}(t)] < P\_L(t) \text{ and } \text{SOC} > E\_{b,\text{min}} \text{ then;}\\ P\_{bd}(t) = \frac{[P\_L(t) - P\_w(t) - P\_{PV}(t)]}{\eta\_{bd}} \end{aligned} \tag{26}$$

where *Pbd*(*t*) is discharging power, and this is a discharging process:

$$\begin{aligned} \left[ if \left[ P\_{wt}(t) + P\_{PV}(t) \right] < P\_L(t) \text{ and } \text{SOC} < E\_{b, \text{min}} \text{ then:} \right] \\ P\_{bd}(t) = 0 \end{aligned}$$

All the above equations and details of the generation, load, and storage indicate how the energy management strategy is effectively working in this case. When the generation of HRES in the microgrid is higher than the load, the surplus energy is stored in the battery energy storage system. When the generation of the HRES is less than the load of the microgrid, then the shortage of energy is met by the battery storage system. If the generation of HRES is less than the load and the battery energy is less than the minimum level of stored energy, then the microgrid will import energy from another microgrid to meet the demand. In the above equations, *SOC* is the state of the charge of the battery, *Pbc* is the power used to charge the battery, and *Pbd* is the discharging power of the battery. *ηbc*, *ηbd* is the battery charging and discharging efficiencies, respectively.

The power flow constraints of the HRES and battery are as follows:

$$0 \le P\_{\mathcal{G}} \le P\_{\text{max}} \text{ conventional Generalration} \tag{27}$$

$$0 \le P\_{PV} \le P\_{\text{max}} \text{ PVGeneration} \tag{28}$$

$$0 \le P\_{wt} \le P\_{\max} \text{ WTGeneration} \tag{29}$$

$$P\_{\min} \le P\_b \le P\_{\max} \text{ Buttery power} \tag{30}$$

$$\text{SOC}\_{\text{min}} \le \text{SOC} \le \text{SOC}\_{\text{max}}\\\text{ Butterity State of Charge} \tag{31}$$

#### *4.3. Microgrid Energy Sharing Problem*

The microgrid has an excess or shortage of energy. The microgrid with excess energy will export the energy to other microgrids, which need a power through-line between them (*S*1, *S*2, *S*3, . . . , *S*15). The microgrid with a shortage of energy will import the energy from other microgrids, which have an excess power through-line between them (*S*1, *S*2, *S*3, . . . , *S*15). If all microgrids do not have enough energy to cover the shortage in the system, the system will import energy from the main grid. If we consider a sequence of 1 : *Disco*, 2 : *MG*1, 3 : *MG*2, 4 : *MG*3, 5 : *MG*4, 6 : *MG*5, if sending end *SN* = 1 and receiving end *RN* = 2, it means the line is connected from Disco to *MG*1 and so on.

#### **5. Power Balance**

The power balance in the generation and load is represented as:

$$Systems\,\,Load - P\_{PV} - P\_w - P\_{con} - P\_{Grid} = 0\tag{32}$$

$$\text{MG Load} + \text{Export Energy} - \text{MGGeneration} - \text{Import Energy} = 0 \tag{33}$$

#### *Objective Function*

The objective function is a cost reduction of the generated power, transfer power, and import and export power:

$$\mathbf{C}\_{T} = \sum\_{1}^{G} \mathbf{C}\_{G} + \mathbf{C}\_{D\text{is}} + \mathbf{C}\_{MG} + \mathbf{C}\_{B} + \mathbf{C}\_{P} \tag{34}$$

where *C<sup>G</sup>* is the total conventional generation cost, *CDis* is the total transfer energy cost from Disco to *MG*, *CMG* is the total transfer energy cost from *MG to MG*, *C<sup>B</sup>* is the total energy cost from the battery, and *C<sup>P</sup>* is the penalty cost for the unsupplied energy.

Mathematically, all of the above costs are represented as:

$$\mathbf{C}\_{G} = \left(\mathbf{G}\_{\text{Dis}} + \mathbf{G}1\_{\text{MG}2} + \mathbf{G}2\_{\text{MG}2} + \mathbf{G}5\_{\text{MG}5}\right) \ast P\_{\text{gas}}\tag{35}$$

where *GDis* is conventional generation in Disco, *G*1*MG*<sup>2</sup> is conventional generation in *MG*2, *G*2*MG*<sup>2</sup> is conventional generation in *MG*2, *G*5*MG*<sup>5</sup> is conventional generation in *MG*5, and *Pgas* is the gas price:

$$\mathcal{C}\_{Dis} = \left(\mathbf{S1} + \mathbf{S2} + \mathbf{S3} + \mathbf{S4} + \mathbf{S5}\right) \ast P\_{\rm Dis} \tag{36}$$

where *S*1, *S*2, *S*3, *S*4, *S*5 transfer power from Disco to *MG*1, *MG*2, *MG*3, *MG*4, *MG*5, respectively, and *PDis* is the energy cost of Disco:

$$\mathcal{C}\_{MG} = (\mathbf{S}\mathbf{6} + \mathbf{S}\mathbf{7} + \mathbf{S}\mathbf{8} + \dots + \mathbf{S}\mathbf{15}) \ast P\_{MG} \tag{37}$$

where (*S*6 + *S*7 + *S*8 + · · · + *S*15) is the power transfer from *MG to MG*, and *PMG* is the cost of energy from *MG to MG*:

$$C\_B = (BES\_{Disco} + BES\_{MG2} + BES\_{MG4} + BES\_{MG5}) \ast P\_B \tag{38}$$

In Equation (38), *BESDisco* is the energy of the battery in Disco, *BESMG*<sup>2</sup> is the energy of the battery in *MG*2, *BESMG*<sup>4</sup> is the energy of the battery *in MG*4, *BESMG*<sup>5</sup> is the energy of the battery in *MG*5, and *P<sup>B</sup>* is the cost of energy from the battery:

$$\mathcal{C}\_P = E\_{\text{ns}} \ast P\_P \tag{39}$$

where *Ens* is the unsupplied energy and *P<sup>P</sup>* is the penalty price that is fixed [49].

Table 3 represents the cost data of different parameters (also taken from [49]).


**Table 3.** Cost data. Reproduced from [49], the (Journal of Energy Storage): 2021.

#### **6. Simulations and Results Analysis**

As discussed above, in all the mathematical formulations regarding the objective functions and microgrid components, an ABC optimization technique reduces the system's total cost and minimal sharing cost of all the microgrids. Microgrids consist of conventional generation and intermittent energy sources, such as PV, wind, and battery energy storage systems. Among all the sources, conventional generators produce 24 h electricity, and wind generation is wind speed dependent and PV dependent on solar irradiations. Conventional generation occurs in Disco, *MG*2, and *MG*5. If their capacity fails to meet the desired load demands, they will import energy from the main grid if there is an unavailability of energy from other microgrids.

There are different power-through lines between Disco and microgrids and from microgrids to microgrids. The lines connecting Disco and microgrids are (*S*1, *S*2, *S*3, *S*4, *S*5), i.e., from *MG*1 *to MG*5. From *MG*1 *to* (*MG*2, *MG*3, *MG*4, *MG*5), the lines are (*S*6, *S*7, *S*8, *S*9); from *MG*2 *to* (*MG*3, *MG*4, *MG*5), the lines are (*S*10, *S*11, *S*12); from *MG*3 *to*(*MG*4, *MG*5), *the lines are* (*S*13, *S*14); and the last connection between *MG*4 *and MG*5 is *S*15. These lines from *S*1, *S*2, . . . , *S*15 represent power transfer from Disco to *MG* and MGs to MGs. If the

value of *S*1 *to S*15 is negative, power is transferred from the receiving to the sending end and vice versa if it is positive.

A cost convergence curve is shown below in Figure 8, which shows how the best solution deals with each iteration. We can see the best solution decreases when the iteration increases until it reaches the best one. The maximum number of iterations considered in the proposed idea is 150, upon which it converges to an optimal value. A cost convergence curve is shown below in Figure 8, which shows how the best solution deals with each iteration. We can see the best solution decreases when the iteration increases until it reaches the best one. The maximum number of iterations considered in the proposed idea is 150, upon which it converges to an optimal value.

*Energies* **2022**, *15*, x FOR PEER REVIEW 15 of 21

**Figure 8.** Cost convergence graph. **Figure 8.** Cost convergence graph.

Regarding the MG with only PV generation source, Figure 9 shows that it can only generate power according to the solar irradiance data in the PVWT profile. For the rest of the hours, it imports energy from outside, and from 11 to 13 h, it sells energy as it exceeds the load demand. MG1 has only one PV generation source, and as per Equation (7), PV generation is dependent on thee tilted surface, PV array, and its efficiency. Maximum generation is achieved when maximum sunlight is available for the tilted surface, and it is generally from 7 to 19 h that power is generated. Moreover, the intensity of light is very high, from 11 to 13 h, during which the generated power reaches th maximum and is available to sell to other MGs or the main grid. In the absence of sunlight, MG1 will not generate electricity. Hence, it will purchase electricity from other MGs. Regarding the MG with only PV generation source, Figure 9 shows that it can only generate power according to the solar irradiance data in the PVWT profile. For the rest of the hours, it imports energy from outside, and from 11 to 13 h, it sells energy as it exceeds the load demand. MG1 has only one PV generation source, and as per Equation (7), PV generation is dependent on thee tilted surface, PV array, and its efficiency. Maximum generation is achieved when maximum sunlight is available for the tilted surface, and it is generally from 7 to 19 h that power is generated. Moreover, the intensity of light is very high, from 11 to 13 h, during which the generated power reaches th maximum and is available to sell to other MGs or the main grid. In the absence of sunlight, MG1 will not generate electricity. Hence, it will purchase electricity from other MGs.

Microgrid 2 contains wind generation, battery storage, and 2 thermal units. The wind is almost always the available generation source, as shown in Figure 10. From 1–6 h, the load demand can easily be met by the wind generation source and thermal unit generation. During this time, extra power is used for charging the battery. From 7–8 h, EES is used to meet the load demand along with the wind and thermal generation sources. From 9–23 h, wind generation, thermal generation, and EES are not able to meet the load demand; hence, electricity is purchased from outside.

**Power (kW)**  Microgrid 3 has one source of PV and load demand, as shown in Figure 11. To meet the load demand with just one PV source, it will purchase electricity from other microgrids in the absence of sunlight. When sunlight becomes available, it will gradually overcome the load demand and at high intensity, sunlight from 11–13 h MG3 will be able to meet the entire demand. The extra amount of electricity will be sold to other MGs.

**Figure 9.** Microgrid 1.

generate electricity. Hence, it will purchase electricity from other MGs.

A cost convergence curve is shown below in Figure 8, which shows how the best solution deals with each iteration. We can see the best solution decreases when the iteration increases until it reaches the best one. The maximum number of iterations considered

Regarding the MG with only PV generation source, Figure 9 shows that it can only generate power according to the solar irradiance data in the PVWT profile. For the rest of the hours, it imports energy from outside, and from 11 to 13 h, it sells energy as it exceeds the load demand. MG1 has only one PV generation source, and as per Equation (7), PV generation is dependent on thee tilted surface, PV array, and its efficiency. Maximum generation is achieved when maximum sunlight is available for the tilted surface, and it is generally from 7 to 19 h that power is generated. Moreover, the intensity of light is very high, from 11 to 13 h, during which the generated power reaches th maximum and is available to sell to other MGs or the main grid. In the absence of sunlight, MG1 will not

in the proposed idea is 150, upon which it converges to an optimal value.

**Figure 9.** Microgrid 1. **Figure 9.** Microgrid 1. mand; hence, electricity is purchased from outside.

**Figure 8.** Cost convergence graph.

**Figure 10.** Microgrid 2 response. **Figure 10.** Microgrid 2 response.

**Figure 11.** Microgrid 3 response.

**Power (kW)** 

Microgrid 3 has one source of PV and load demand, as shown in Figure 11. To meet the load demand with just one PV source, it will purchase electricity from other microgrids in the absence of sunlight. When sunlight becomes available, it will gradually overcome the load demand and at high intensity, sunlight from 11–13 h MG3 will be able to meet the entire demand. The extra amount of electricity will be sold to other MGs. MG4 contains wind generation and battery storage, and the load demand is shown in the Figure 12. The result in Figure 12 shows that from 1–6 h, wind generation is able to meet the required load demand and the extra amount of electricity will be used to charge the battery. Due to the intermittent nature of wind, it will purchase electricity if the wind generation is less than the load demand. As shown in Figure 12, from 7–10 h, the load demand exceeds wind generation; hence, EES is used to meet the load demand along with wind generation, and some energy is still required to meet the required demand, it will be purchased from other microgrids. From 11–24 h, wind generation cannot meet the required load demand; hence, the extra amount of electricity needed will be purchased from other microgrids.

Microgrid 2 contains wind generation, battery storage, and 2 thermal units. The wind is almost always the available generation source, as shown in Figure 10. From 1–6 h, the load demand can easily be met by the wind generation source and thermal unit generation. During this time, extra power is used for charging the battery. From 7–8 h, EES is used to meet the load demand along with the wind and thermal generation sources. From 9–23 h, wind generation, thermal generation, and EES are not able to meet the load de-

Microgrid 3 has one source of PV and load demand, as shown in Figure 11. To meet the load demand with just one PV source, it will purchase electricity from other microgrids in the absence of sunlight. When sunlight becomes available, it will gradually overcome the load demand and at high intensity, sunlight from 11–13 h MG3 will be able to meet the entire demand. The extra amount of electricity will be sold to other MGs.

mand; hence, electricity is purchased from outside.

**Figure 11.** Microgrid 3 response. **Figure 11.** Microgrid 3 response. other microgrids.

**Figure 10.** Microgrid 2 response.

**Figure 12.** Load profile of MG4. **Figure 12.** Load profile of MG4.

**Figure 13.** Load profile of MG5.

**Power (kW)** 

MG5 contains wind generation, battery storage, and one thermal unit. The load demand is shown in Figure 13. Wind generation is available 24 h as shown in the figure. From 1–6 h, the load demand is met by wind and thermal generation and the extra amount of electricity is used to charge the battery. From 7–8 h, the load demand exceeds the wind and thermal generation; hence, EES is used to meet the required demand. From 9–24 h, as the load demand is higher than the inside generation, the extra energy required to meet the demand is purchased from other microgrids. MG5 contains wind generation, battery storage, and one thermal unit. The load demand is shown in Figure 13. Wind generation is available 24 h as shown in the figure. From 1–6 h, the load demand is met by wind and thermal generation and the extra amount of electricity is used to charge the battery. From 7–8 h, the load demand exceeds the wind and thermal generation; hence, EES is used to meet the required demand. From 9–24 h, as the load demand is higher than the inside generation, the extra energy required to meet the demand is purchased from other microgrids.

the demand is purchased from other microgrids.

**Figure 13.** Load profile of MG5. **Figure 13.** Load profile of MG5.

**Figure 12.** Load profile of MG4.

other microgrids.

Disco contains battery storage and one thermal unit. The thermal generation is a constant source of generation as shown in Figure 14, and the load demand of Disco is shown in the figure. From 1–6 h, thermal generation alone cannot meet the load demand. Therefore, energy is purchased from other microgrids at the optimal rate and the battery is charged during this time. From 7–8 h, thermal generation and EES are both used, and the extra amount needed to meet the load demand is purchased from outside. From 9–24 h, thermal generation is used, but the generation is not sufficient to meet the required demand. Therefore, the extra amount is purchased from outside. Disco contains battery storage and one thermal unit. The thermal generation is a constant source of generation as shown in Figure 14, and the load demand of Disco is shown in the figure. From 1–6 h, thermal generation alone cannot meet the load demand. Therefore, energy is purchased from other microgrids at the optimal rate and the battery is charged during this time. From 7–8 h, thermal generation and EES are both used, and the extra amount needed to meet the load demand is purchased from outside. From 9–24 h, thermal generation is used, but the generation is not sufficient to meet the required demand. Therefore, the extra amount is purchased from outside.

MG4 contains wind generation and battery storage, and the load demand is shown in the Figure 12. The result in Figure 12 shows that from 1–6 h, wind generation is able to meet the required load demand and the extra amount of electricity will be used to charge the battery. Due to the intermittent nature of wind, it will purchase electricity if the wind generation is less than the load demand. As shown in Figure 12, from 7–10 h, the load demand exceeds wind generation; hence, EES is used to meet the load demand along with wind generation, and some energy is still required to meet the required demand, it will be purchased from other microgrids. From 11–24 h, wind generation cannot meet the required load demand; hence, the extra amount of electricity needed will be purchased from

MG5 contains wind generation, battery storage, and one thermal unit. The load demand is shown in Figure 13. Wind generation is available 24 h as shown in the figure. From 1–6 h, the load demand is met by wind and thermal generation and the extra amount of electricity is used to charge the battery. From 7–8 h, the load demand exceeds the wind and thermal generation; hence, EES is used to meet the required demand. From 9–24 h, as the load demand is higher than the inside generation, the extra energy required to meet

**Figure 15.** Cost optimization results.

**Cost(\$)**

The cost optimization results of the microgrids and total systems are shown in Figure 15. It is clearly shown in the figure that the total cost of MG2, MG4, and MG5 is zero, which means that they do not import energy from other microgrids or the amount of electricity that is bought or sold is the same; hence, they compensate for their cost effects, which is why their cost is zero. The cost optimization results of the microgrids and total systems are shown in Figure 15. It is clearly shown in the figure that the total cost of MG2, MG4, and MG5 is zero, which means that they do not import energy from other microgrids or the amount of electricity that is bought or sold is the same; hence, they compensate for their cost effects, which is why their cost is zero.

Cost Optimization Results

**Microgrids**

The cost optimization results of the microgrids and total systems are shown in Figure 15. It is clearly shown in the figure that the total cost of MG2, MG4, and MG5 is zero, which means that they do not import energy from other microgrids or the amount of electricity that is bought or sold is the same; hence, they compensate for their cost effects,

Disco contains battery storage and one thermal unit. The thermal generation is a constant source of generation as shown in Figure 14, and the load demand of Disco is shown in the figure. From 1–6 h, thermal generation alone cannot meet the load demand. Therefore, energy is purchased from other microgrids at the optimal rate and the battery is charged during this time. From 7–8 h, thermal generation and EES are both used, and the extra amount needed to meet the load demand is purchased from outside. From 9–24 h, thermal generation is used, but the generation is not sufficient to meet the required de-

mand. Therefore, the extra amount is purchased from outside.

**Figure 15.** Cost optimization results. **Figure 15.** Cost optimization results.

**Figure 14.** Load profile of Disco.

which is why their cost is zero.

#### **7. Conclusions and Future Work**

The microgrid has emerged as a new paradigm shift for global energy systems to replace remote centralized power plants with more efficient, localized, and distributed generation systems, especially in various cities and towns around the globe. It provides more stable, flexible, and energy-efficient solutions for the power grid so that an increasing number of loads can be handled without new infrastructure needing to be built. This work provides a new iterative ABC optimization method that addresses HRESs, such as solar, wind, GT, and battery. In particular, this work optimizes the power transmission between different microgrids to minimize the cost of each microgrid and the whole system while maintaining the load requirements and stability. The optimization problem was solved by the new iterative method ABC. This study also conducted several experiments with historical data to test the proposed method for the HRES model. The simulation results show that the proposed method is efficient in finding the best solution and adjusting the optimization parameters and constraints. This method can also be applied to any site with meteorological data and any WT with technical information.

Future research will focus on the use of hybrid swarm intelligence systems, such as the particle swarm algorithm and the artificial bee colony algorithm or the differential evolution algorithm, for the economic dispatch of microgrids. In addition, flexible switching techniques for real-time scheduling in microgrids that consider more constraint situations is an interesting future research direction.

**Author Contributions:** Conceptualization, K.U. and Q.J.; methodology, software and validation, K.U. formal analysis, investigation and resources, R.A.K.; writing—original draft preparation, K.U.; writing—review and editing, G.G.; visualization, S.R.; supervision, Q.J. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** Not available.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** The data that support the study's findings, such as numerical simulation, model, or code generated or used during the study, are available upon request from the journal and the corresponding author.

**Acknowledgments:** Thanks to the China Scholarship Council, Zhejiang University, Hangzhou, China (www.zju.edu.cn, accessed on 18 January 2022) and the University of Science & Technology Bannu, Pakistan (www.ustb.edu.pk, accessed on 18 January 2022) for providing us great research environment during this work.

**Conflicts of Interest:** The corresponding author declares that there is no contradiction on behalf of all authors.

#### **References**


## *Article* **Multi-Objective Immune-Commensal-Evolutionary Programming for Total Production Cost and Total System Loss Minimization via Integrated Economic Dispatch and Distributed Generation Installation**

**Mohd Helmi Mansor 1,\*, Ismail Musirin <sup>2</sup> and Muhammad Murtadha Othman <sup>2</sup>**


**Abstract:** Economic Dispatch (ED) problems have been solved using single-objective optimization for so long, as Grid System Operators (GSOs) previously only focused on minimizing the total production cost. In modern power systems, GSOs require not only optimizing the total production cost but also, at the same time, optimizing other important objectives, such as the total emissions of the greenhouse gasses, total system loss and voltage stability. This requires a suitable multi-objective optimization approach in ensuring the ED solution produced is satisfying all the objectives. This paper presents a new multi-objective optimization technique termed Multi-Objective Immune-Commensal-Evolutionary Programming (MOICEP) for minimizing the total production cost and total system loss via integrated Economic Dispatch and Distributed Generation installation (ED-DG). This involved the application of a weighted-sum multi-objective approach that combined with an optimization technique called Immune-Commensal-Evolutionary Programming (ICEP). The proposed MOICEP has been compared with other multi-objective techniques, which are Multi-Objective-Evolutionary Programming (MOEP) and Multi-Objective-Artificial Immune System (MOAIS). It was found that MOICEP performs very well in producing better optimization results for all the three types of Economic Dispatch (ED) problems compared to MOEP and MOAIS in terms of cheap total production costs and low total system loss.

**Keywords:** multi-objective optimization; economic dispatch; distributed generation installation; evolutionary programming; artificial immune system; symbiotic organisms search; commensalism

## **1. Introduction**

In the early years after the introduction of Economic Dispatch (ED), most of the ED problems were solved based on single optimization [1]. This means that only the total production cost was optimized to get the minimum value. Nowadays, researchers and engineers are interested in solving ED problems with multiple objectives [2]. Besides the total production cost, the total greenhouse gasses emissions and total system loss can also be optimized at the same time. The reason for this multi-objective optimization introduced to solve ED problems is that greenhouse gasses emissions from fossil-fueled generating units and total system loss are also grid system operators' main concerns in modern electrical power systems [3]. There are two popular approaches to solve multi-objective problems, which are the weighted-sum and Pareto optimal [4]. The weighted-sum approach has been widely used to solve multi-objective problems due to its simplicity, while Pareto optimal solution is more complicated but could give a better nondominated solution for certain optimization problems [5].

**Citation:** Mansor, M.H.; Musirin, I.; Othman, M.M. Multi-Objective Immune-Commensal-Evolutionary Programming for Total Production Cost and Total System Loss Minimization via Integrated Economic Dispatch and Distributed Generation Installation. *Energies* **2021**, *14*, 7733. https://doi.org/10.3390/ en14227733

Academic Editors: Victor Becerra and Ahmed F. Zobaa

Received: 1 October 2021 Accepted: 8 November 2021 Published: 18 November 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

Numerous multi-objective ED problems have been solved by researchers and engineers for the past ten years. Thenmalar et al. [3] solved Economic Dispatch and Economic Emission Dispatch using multi-objective optimization. Two objective functions are optimized by them, which are minimizing the total production cost and minimizing the emission of nitrogen oxides (NOx). Quadratic programming by Wolfe's method was chosen to solve the multi-objective ED problem. M. Musau et al. [6] proposed to solve the multi-objective ED problem by considering the presence of renewable energy. They optimized both the renewable cost and total production cost functions by combining them to be solved using one single-objective function. Roy et al. [7] optimized the total production cost and total emissions simultaneously. In this study, a Normal Boundary Intersection (NBI)-based decomposition scheme was utilized to implement multi-objective optimization. In Reference [8], Dash and Mohanty applied the weighted-sum approach to solve a multi-objective ED problem. There three objectives were solved simultaneously using a simulated annealing optimization technique. The objectives were to minimize the total production cost, to minimize the nitrous oxide (NOx) emissions and to improve the security of the transmission lines. Therefore, there were three weights that were associated with the three fitness function equations. Meiqin et al. [9] also solved three objectives of the ED problem, which were the total production cost, consumer outage cost and total emissions. They used weighted-sum integrated with a membership degree of the fuzzy set theory to solve the multi-objective ED problem. Man-im et al. [10] applied the Pareto solution instead of the weighted-sum approach to solve a multi-objective ED problem that simultaneously minimized the total production cost and power system security risk. The Pareto optimal solution approach was associated with the optimization technique and termed Non-dominated Sorting Particle Swarm Optimization (NSPSO). In Reference [11], minimizing the total system loss was also considered as one of the ED problem objectives, together with minimizing the total production cost and minimizing the NO<sup>x</sup> emissions. Minimizing the total production cost and total system loss are more crucial than minimizing the total emissions in a modern power system that consists of substantial amounts of renewable energy sources. The emissions and fuel costs were optimized simultaneously using the Pareto optimal approach in Reference [12]. In this study, the multi-objective results of ED were compared with single-objective results. It was found that the results of the single-objective optimization were better than the multi-objective optimization in terms of producing the lowest fuel costs. However, in the multi-objective optimization, two objectives were achieved, which were to minimize the fuel costs and emissions. This is far better compared to only one objective being achieved. In a later study, Huang et al. [13] optimized three objective functions while solving for ED. The objective functions were the total fuel consumption function, oxynitride emission function and electricity purchase cost function. The ED problem was solved using a hybrid optimization technique termed the Hybrid Intelligent Algorithm. The technique is based on Particle Swarm Optimization (PSO) and Artificial Fish Swarm Algorithm (AFSA). From this review, it was discovered that multi-objective optimization is more powerful than single-objective optimization in solving ED problems, as more objectives can be optimized using multi-objective optimization at one time.

Single-objective optimization has its limitation in terms of satisfying more than one ED problem's objectives. For instance, some GSOs prefer to minimize both the total system loss and total production cost or to minimize the total production cost and emissions at the same time. Furthermore, GSOs will have flexibility in solving the ED problem of their power systems. Instead of solving the ED problem with only one objective, they can add other important objectives to be optimized concurrently. In previous practices, a power system problem was solved independently without integrating with other power system problems. For instance, the ED problem was solved independently to find the optimal production cost, and the Distributed Generation (DG) installation problem was solved to find the optimal location and sizing of DG units in a power system so that the system loss

is the minimum. This practice is not efficient for GSOs to solve various problems of their power systems, especially in modern power systems.

This paper presents the implementation of a multi-objective optimization technique termed Multi-Objective Immune-Commensal-Evolutionary Programming (MOICEP) for solving integrated ED-DG problems. Two power system problems (ED problem and DG installation problem) are solved concurrently using MOICEP. Furthermore, two important objectives in power system optimization, which are the total production cost and total system loss, are chosen to be optimized simultaneously using the weighted-sum multiobjective approach with the help of ICEP as the optimizer. The weight coefficients of the two objectives are varied to observe and analyze the multi-objective optimization solution produced by MOICEP. There are three ED problems solved in this study, which are the basic ED problem (smooth cost function), ED problem with prohibited operating zones and ED problem with valve-point loading effect. Each of these ED problems is solved with the DG installation problem of three DG units. It was decided to install three DG units in this study based on the sizes of the test systems. It was assumed that the more DG units in the system, the lower the total production cost of the power system. The solutions for these integrated problems are the optimal output of the conventional generating units and the optimal location and sizing of the DG units. The proposed MOICEP technique is validated by implementing it on the IEEE 30-Bus RTS and IEEE 57- Bus RTS. Besides that, the solution of the integrated ED-DG problems produced by MOICEP is compared with the solution produced by the existing multi-objective techniques of Multi-Objective-Evolutionary Programming (MOEP) and Multi-Objective-Artificial Immune System (MOAIS) for comparative studies purposes.

#### **2. Proposed Weighted-Sum Multi-Objective Immune-Commensal-Evolutionary Programming for Total Production Cost and Total System Loss Minimization**

In this study, the total production cost and total system loss are optimized simultaneously using the weighted-sum Multi-Objective Immune-Commensal-Evolutionary Programming (MOICEP) technique. The total production is calculated using Equation (1) for the basic ED problems and ED problems with prohibited operating zones, while Equation (2) is used to calculate the total production cost for the ED problems with a valve-point loading effect. The total system loss is calculated using Equation (3).

$$\mathbb{C}\_{total} = \sum\_{i=1}^{n} \mathbb{C}\_{i}(P\_{i}) = \sum\_{i=1}^{n} a\_{i}P\_{i}^{2} + b\_{i}P\_{i} + c\_{i} \tag{1}$$

where

*Ci* is the production cost of *i*th generating unit, and *Pi* is the real power output of the *i*th generating unit. *ai* , *b<sup>i</sup>* and *c<sup>i</sup>* are the cost coefficients of the *i*th generating unit and *n* is the number of dispatchable generating units:

$$\mathcal{C}\_{total} = \sum\_{i=1}^{n} \mathcal{C}\_{i}(P\_{i}) = \sum\_{i=1}^{n} a\_{i}P\_{i}^{2} + b\_{i}P\_{i} + c\_{i} + \left| e\_{i}\sin\left(f\_{i}\left(P\_{i}^{\min} - P\_{i}\right)\right) \right| \tag{2}$$

where *e<sup>i</sup>* and *f<sup>i</sup>* are the valve-point loading effect coefficients of the *i*th generating unit.

$$P\_{\rm loss} = \sum\_{k=1}^{l} g\_k [V\_i^2 + V\_j^2 - 2V\_i V\_j \cos\left(\delta\_i - \delta\_j\right)], \ k \in \{1, 2, \dots, l\} \tag{3}$$

where

*gk* is the conductance of *kth* line,

*V<sup>i</sup>* and *δ<sup>i</sup>* are the voltage magnitude and angle of bus *i*, respectively,

*V<sup>j</sup>* and *δ<sup>j</sup>* are the voltage magnitude and angle of bus *j*, respectively, and

*l* is the number of lines in the system.

The implementation of this proposed technique is illustrated in Figure 1. The procedure of minimizing the total production costs and total system loss using MOICEP shown in Figure 1 is explained in the following steps. The implementation of this proposed technique is illustrated in Figure 1. The procedure of minimizing the total production costs and total system loss using MOICEP shown in Figure 1 is explained in the following steps.

**Figure 1.** Overall MOICEP processes for the total production costs and total system loss minimiza-**Figure 1.** Overall MOICEP processes for the total production costs and total system loss minimization.

Step 1: Generate decision variable

is the number of lines in the system.

Step 1: Generate decision variable MOICEP starts with the randomization of the decision variable values based on their constraints. There are three types of decision variables for this problem: real power output of generating units, location of DG units and size of DG units. This randomization is a part of the initialization process of MOICEP. Twenty individuals of each decision variable are stored in the initialization pool that satisfied the constraint violation test. The group of individuals in the pool is called as the initial population. MOICEP starts with the randomization of the decision variable values based on their constraints. There are three types of decision variables for this problem: real power output of generating units, location of DG units and size of DG units. This randomization is a part of the initialization process of MOICEP. Twenty individuals of each decision variable are stored in the initialization pool that satisfied the constraint violation test. The group of individuals in the pool is called as the initial population.

#### Step 2: Cloning process Step 2: Cloning process

Subsequently, the initial population is cloned to increase its size to two hundred. The fitness of this multi-objective is then calculated using Equation (4). The weight coefficients are varied depending on the problem's desired output. For this study, there were five settings of weight coefficients used to minimize the total production costs and total system loss, which are ଵ = 0.9, ଶ = 0.1, ଵ = 0.5, ଶ = 0.5, and ଵ = 0.1, ଶ = 0.9. Subsequently, the initial population is cloned to increase its size to two hundred. The fitness of this multi-objective is then calculated using Equation (4). The weight coefficients are varied depending on the problem's desired output. For this study, there were five settings of weight coefficients used to minimize the total production costs and total system loss, which are *w*<sup>1</sup> = 0.9, *w*<sup>2</sup> = 0.1, *w*<sup>1</sup> = 0.5, *w*<sup>2</sup> = 0.5, and *w*<sup>1</sup> = 0.1, *w*<sup>2</sup> = 0.9.

$$F(\mathbf{x}) = w\_1 f\_1(\mathbf{x}) + w\_2 f\_2(\mathbf{x}) \tag{4}$$

where where

tion.

() is weighted objective function, ଵ is weight coefficient for the first objective function, ଶ is weight coefficient for the second objective function ଵ() is the first objective function (total production cost minimization) and ଶ() is the second objective function (total system loss minimization). Step 3: Mutation process *F*(*x*) is weighted objective function, *w*<sup>1</sup> is weight coefficient for the first objective function, *w*<sup>2</sup> is weight coefficient for the second objective function *f*1(*x*) is the first objective function (total production cost minimization) and *f*2(*x*) is the second objective function (total system loss minimization).

Step 3: Mutation process

Then, the best two hundred cloned individuals that produce the best value of fitness are mutated using the commensalism equation of the Symbiotic Organisms Search (SOS), Then, the best two hundred cloned individuals that produce the best value of fitness are mutated using the commensalism equation of the Symbiotic Organisms Search (SOS),

as shown in Equation (5). This mutation process is conducted to produce offspring of the mutated population. There are two hundred individual offspring after this process.

$$X\_{i,new} = X\_i + rand(-1, 1) \times \left(X\_{best} - X\_j\right) \tag{5}$$

where

*Xi*,*new* is the offspring of the *i*th individual, *Xi* is the parent of the *i*th individual, *Xbest* is the fittest individual and *Xj* is another individual besides the *i*th individual.

#### Step 4: Combination process

The cloned population and the mutated populated are then combined to become a large population with the size of four hundred.

Step 5: Ranking and selection process

From these four hundred individuals, the twenty best individuals will be selected to undergo the convergence process. However, prior to that, the combined individuals are ranked based on their fitness. Those who give the best value of fitness will be included into the best twenty group; otherwise, they will be excluded from undergoing the next process.

#### Step 6: Convergence test

Subsequently, the best twenty individuals will undergo the convergence process. In this convergence process, a stopping criterion is set to indicate that the algorithm has found the global optima of the multi-objective optimization problem. The stopping criterion used is as shown in Equation (6). If the stopping criterion is not met, the process will proceed to the cloning process and repeat the previous whole steps until the solution converges.

$$fitness\_{max} - fitness\_{min} \le 0.0001\tag{6}$$

where

*f itnessmax* is the maximum fitness value and *f itnessmin* is the minimum fitness value.

#### **3. Results and Discussion**

This section presents the results and discussion on Multi-Objective Immune-Commensal-Evolutionary Programming (MOICEP) for the total production costs and total system loss minimization via integrated Economic Dispatch and Distributed Generation installation (ED-DG). As mentioned previously, the weight coefficients of the objective functions can be varied from 0 to 1.0 between the two objective functions (total production cost minimization and total system loss minimization) to find the trade-off between the two objective functions. The set of weight coefficients for the two fitness functions are as shown in Table 1. The objective functions for this multi-objective problem are the total production cost and total system loss minimization. It can be seen from the table that the summation of *w*<sup>1</sup> and *w*<sup>2</sup> is equal to one. If the weight of one of the fitness functions is set to one, then the weight for the other fitness function will be zero. This means that only the objective function with the value of one is optimized, and the other fitness function is not optimized. If the weight value is more than zero, it means that the fitness function is optimized with the amount of the weight. The difference between the fitness and objective functions needs to be understood. Fitness is the equation for the problem formulation, while the objective function is the minimization or the maximization of the fitness function. Let us say, our fitness function *f*<sup>1</sup> = *f*1(*x*); thus, the objective function will be *max* (*f*1(*x*)) or *min* (*f*1(*x*)).


**Table 1.** Weight coefficients values for two objective functions (total production cost and total system loss). **Table 1.** Weight coefficients values for two objective functions (total production cost and total system loss).

As suggested by N. A. Rahmat [14], the best settings of the weight coefficients (tradeoff) for optimizing the two objective functions were settings 2, 6 and 10, as shown in Table 1. This was because the other settings produced the same nondominated values as settings 2, 6 and 11. Therefore, in this study, the weighted-sum Multi-Objective Immune-Commensal-Evolutionary Programming (MOICEP) technique applied settings 2, 6 and 10 for minimizing the total production costs and total system loss minimization simultaneously. As suggested by N. A. Rahmat [14], the best settings of the weight coefficients (tradeoff) for optimizing the two objective functions were settings 2, 6 and 10, as shown in Table 1. This was because the other settings produced the same nondominated values as settings 2, 6 and 11. Therefore, in this study, the weighted-sum Multi-Objective Immune-Commensal-Evolutionary Programming (MOICEP) technique applied settings 2, 6 and 10 for minimizing the total production costs and total system loss minimization simultaneously.

The overall implementation of the MOICEP is illustrated in Figure 2. The proposed technique has been used to minimize the total production costs and total system loss for three different ED problems. The problems are a basic ED problem, ED problem with prohibited operating zones and ED problem with a valve-point loading effect. Furthermore, for each ED problem, two test systems are used for implementing the MOICEP, which are IEEE 30-Bus RTS and IEEE 57-Bus RTS. The overall implementation of the MOICEP is illustrated in Figure 2. The proposed technique has been used to minimize the total production costs and total system loss for three different ED problems. The problems are a basic ED problem, ED problem with prohibited operating zones and ED problem with a valve-point loading effect. Furthermore, for each ED problem, two test systems are used for implementing the MOICEP, which are IEEE 30-Bus RTS and IEEE 57-Bus RTS.

**Figure 2.** Overall implementation of the MOICEP for total production cost and total system loss minimization. **Figure 2.** Overall implementation of the MOICEP for total production cost and total system loss minimization.

#### *3.1. MOICEP-Based Technique for Basic Economic Dispatch Problem*  For this basic ED problem (smooth cost function), Equations (1) and (3) are used to *3.1. MOICEP-Based Technique for Basic Economic Dispatch Problem*

calculate the total production cost and total system loss, respectively. The MOICEP was implemented on the IEEE 30-Bus RTS and IEEE 57-Bus RTS. For each test system, there are five settings of weight coefficients of the two objectives (total production cost minimi-For this basic ED problem (smooth cost function), Equations (1) and (3) are used to calculate the total production cost and total system loss, respectively. The MOICEP was implemented on the IEEE 30-Bus RTS and IEEE 57-Bus RTS. For each test system, there are

zation and total system loss minimization). The settings are ଵ = 0.9, ଶ = 0.1, ଵ = 0.5,

five settings of weight coefficients of the two objectives (total production cost minimization and total system loss minimization). The settings are *w*<sup>1</sup> = 0.9, *w*<sup>2</sup> = 0.1, *w*<sup>1</sup> = 0.5, *w*<sup>2</sup> = 0.5 and *w*<sup>1</sup> = 0.1, *w*<sup>2</sup> = 0.9. *w*<sup>1</sup> is the weight coefficient of objective 1 (total production cost minimization), while *w*<sup>2</sup> is weight coefficient of objective 2 (total system loss minimization).

### 3.1.1. For *w*<sup>1</sup> = 0.9 and *w*<sup>2</sup> = 0.1

The results of the integrated Basic ED and DG installation of the IEEE 30-Bus and 57-Bus RTSs for the second setting of the weight coefficients (*w*<sup>1</sup> = 0.9 and *w*<sup>2</sup> = 0.1) are tabulated in Tables 2 and 3, respectively.

**Table 2.** Results of the integrated basic ED and DG installation of the IEEE 30-Bus RTS (*w*<sup>1</sup> = 0.9 and *w*<sup>2</sup> = 0.1).


**Table 3.** Results of the integrated basic ED and DG installation of the IEEE 57-Bus RTS (*w*<sup>1</sup> = 0.9 and *w*<sup>2</sup> = 0.1).


From Table 2, it can be seen that MOICEP continued to give the best solution of the basic ED and DG installation of the IEEE 30-Bus RTS by producing the lowest total production costs and total system loss of 355.91 \$/h and 1.44 MW, respectively. The locations of the three DG units found via MOICEP were bus 29, bus 9 and bus 5 with the sizes of 5.00, 50.00 and 100.00 MW, respectively. While the locations and sizes for the three DG units found via MOEP were bus 19 with 4.62 MW, bus 28 with 27.70 MW and bus 5 with 75.81 MW. MOAIS identified bus 19 with 4.62 MW, bus 28 with 27.70 MW and

bus 25 with 101.63 MW, respectively. MOEP and MOAIS produced total production costs and total system losses of 463.64 \$/h and 2.60 MW and 402.50 \$/h and 2.28 MW, respectively. These values were higher than the MOICEP. The proposed MOICEP performed much better than MOEP and MOAIS.

For the results of the IEEE 57-Bus RTS shown in Table 3, similar pattern of results as for the IEEE 30-Bus RTS could be seen. MOICEP maintained producing the lowest total production cost and total system loss of 33285.97 \$/h and 9.90 MW, respectively, whereas MOEP and MOAIS found the same total production cost and total system loss of 35,369.69 \$/h and 18.95 MW. The optimal location determined by the MOICEP to install the three DG units were bus 25, bus 22 and bus 13 with the sizes of 4.84 MW, 49.99 MW and 150.00 MW, respectively, while MOEP and MOAIS identified bus 33, bus 10 and bus 40 to install the three DG units (*Lo*1, *Lo*<sup>2</sup> and *Lo*3) with the sizes of 2.66, 36.55 and 141.03 MW, respectively. MOICEP forced generating unit 8, *PG*8, to produce the highest power to the system, while generating unit 12, *PG*12, produced the highest power in MOEP and MOAIS.

## 3.1.2. For *w*<sup>1</sup> = 0.5 and *w*<sup>2</sup> = 0.5

The results value for *w*<sup>1</sup> was set to 0.5, and *w*<sup>2</sup> was 0.5 as the third setting. For the third setting of the weight coefficients, the results of the IEEE 30-Bus RTS and IEEE 57-Bus RTS are tabulated in Tables 4 and 5, respectively.



From Table 4, the MOICEP achieved 402.50 \$/h as the total production cost and 1.40 MW for the total system loss. These values were the lowest compared to MOEP (557.01 \$/h and 2.25 MW) and MOAIS (429.30 \$/h and 2.28 MW). The best total production cost and total system loss in MOICEP were contributed through the placement of *DG*1, *DG*<sup>2</sup> and *DG*<sup>3</sup> at bus 29, bus 21 and bus 5 with the sizes of 5.00 MW, 31.87 MW and 101.63 MW, respectively. Generating unit 5, *PG*5, produced the least power in MOICEP, while generating unit 8, *PG*8, and generating unit 13, *PG*13, in MOEP and MOAIS, respectively. Apparently, MOICEP consistently outperformed MOEP and MOAIS.


**Table 5.** Results of the integrated basic ED and DG installation of the IEEE 57-Bus RTS (*w*<sup>1</sup> = 0.5 and *w*<sup>2</sup> = 0.5).

From the results of the IEEE 57-Bus shown in Table 5, it can be seen that MOICEP continued to produce better total production costs and total system loss as compared to MOEP and MOAIS. MOICEP produced 37,513.91 \$/h and 9.90 MW, while MOEP and MOAIS produced 38,111.41 \$/h and 13.26 MW and 37,752.82 \$/h and 14.61 MW, respectively. Bus 35, bus 38 and bus 3 were determined as the optimal locations to install *DG*1, *DG*<sup>2</sup> and *DG*3. The corresponding sizes for the three DG units were 3.65 MW, 44.23 MW and 68.97 MW. Generating unit 2, *PG*2, produced the least power output compared to the other generating units in MOICEP and MOEP, while generating unit 1, *PG*1, in MOAIS. From the results when *w*<sup>1</sup> = 0.5 and *w*<sup>2</sup> = 0.5, it is observed that the total production cost was slightly expensive, and the total system loss was lower than the previous two settings of weight coefficients.

### 3.1.3. For *w*<sup>1</sup> = 0.1 and *w*<sup>2</sup> = 0.9

The fourth setting was *w*<sup>1</sup> = 0.1 and *w*<sup>2</sup> = 0.9. For this fourth setting of the weight coefficients, the results of integrated Basic ED and DG installation of the IEEE 30-Bus RTS and IEEE 57-Bus RTS are tabulated in Tables 6 and 7, respectively. From the comparison of the results between the three techniques (MOICEP, MOEP and MOAIS), it was observed that the total production cost produced by MOAIS was the lowest, while the total system loss produced by MOICEP was the lowest. The total production cost in MOAIS was lower than in MOICEP and MOEP, because the total DG size for the three DG units was the highest, worth 133.95 MW compared to MOICEP (121.26 MW) and MOEP (98.08 MW). However, MOICEP still exhibited the lowest total system loss of 1.39 MW as compared to MOEP (2.25 MW) and MOAIS (2.28 MW). Bus 29, bus 21 and bus 5 were the optimal locations to install *DG*1, *DG*<sup>2</sup> and *DG*3. The corresponding sizes for the three DG units were 5.00, 29.77 and 86.49 MW. Generating unit 1, *PG*1, in MOICEP produced the lowest power output, while, in MOEP, generating unit 8, *PG*8, and, in MOAIS, generating unit 13, *PG*13, were the ones with the lowest power outputs.


**Table 6.** Results of the integrated basic ED and DG installation of the IEEE 30-Bus RTS (*w*<sup>1</sup> = 0.1 and *w*<sup>2</sup> = 0.9).

**Table 7.** Results of the integrated basic ED and DG installation of the IEEE 57-Bus RTS (*w*<sup>1</sup> = 0.1 and *w*<sup>2</sup> = 0.9).


For the results of IEEE 57-Bus RTS shown in Table 7, both the total production cost and total system loss for this setting of weight coefficients (*w*<sup>1</sup> = 0.1 and *w*<sup>2</sup> = 0.9) were found to be the lowest in MOICEP, which were 36,159.62 \$/h and 6.52 MW as compared to MOEP with 38,111.41 \$/h and 13.26 MW and MOAIS with 37,752.82 \$/h and 14.61 MW. The optimal locations to install the three DG units identified by MOICEP were bus 25, bus 23 and bus 13 with the corresponding sizes of 4.99, 44.63 and 123.36 MW, respectively. It could be observed that MOICEP forced the DG units to operate near to their maximum capacity. This was the reason why the total production cost reduced significantly as compared to MOEP and MOAIS.

#### *3.2. MOICEP-Based Technique for Economic Dispatch Problem with Prohibited Operating Zones*

The proposed MOICEP technique was also used to solve the ED problem with prohibited operating zones. In this study, the same equations of the total production cost (Equation (1)) and total system loss (Equation (3)) in the first ED problem were reutilized. However, there was a difference in the constraint of the operating limits of the generating units. The generating units would only be allowed to operate in the allowable zones to

make sure the stability of their operation. The allowable and prohibited zones for the IEEE 30-Bus RTS and IEEE 57-Bus RTS are shown in Tables 8 and 9, respectively.

**Table 8.** Cost coefficients and generating unit limits for the ED problem with the prohibited operating zones (IEEE 30-Bus RTS).


**Table 9.** Cost coefficients and generating unit limits for the ED problem with the prohibited operating zones (IEEE 57-Bus RTS).


## 3.2.1. For *w*<sup>1</sup> = 0.9 and *w*<sup>2</sup> = 0.1

The integrated ED problem with prohibited operating zones and DG installation continued to be solved using MOICEP, MOEP and MOAIS for the second setting of weight coefficients (*w*<sup>1</sup> = 0.9 and *w*<sup>2</sup> = 0.1). The results for this setting are shown in Table 10 (IEEE 30-Bus RTS) and Table 11 (IEEE 57-Bus RTS). Similar to the first setting, the solutions produced by MOEP and MOAIS were the same for both systems. From Table 10 of the IEEE 30-Bus RTS, the total production cost and total system loss produced by MOICEP were 380.05 \$/h and 1.71 MW, respectively. While MOEP and MOAIS produced a total production cost of 503.05 \$/h and total system loss of 3.81 MW. MOICEP significantly outperformed MOEP and MOAIS.

For the DG installation, it was found that bus 29 with 5.00 MW, bus 21 with 50.00 MW and bus 5 with 100.00 MW were the optimal locations and sizes for the three DG units in the IEEE 30-Bus RTS, while, for the IEEE 57-Bus RTS, as shown in Table 11, bus 25, bus 14 and bus 13 were found to be the suitable locations to install the three DG units with the sizes of 3.87, 50.00 and 150.00 MW, respectively, via MOICEP. This resulted in a total production cost and total system loss worth 33,315.31 \$/h and 11.29 MW, respectively.

Generating unit 6, *PG*6, was found to be the least generating unit supplying its real power output to the IEEE 57-Bus RTS in MOICEP, while generating unit 2, *PG*2, in MOEP and MOAIS. Comparing the total production cost produced in this second setting with the first setting, it can be observed that the total production cost in the second setting was higher for both systems. This was due to the value of *w*<sup>1</sup> decreasing to 0.1 and the value of *w*<sup>2</sup> increasing to 0.1 in the second setting.


**Table 10.** Results of the integrated ED problem with prohibited operating zones and DG installation of the IEEE 30-Bus RTS (*w*<sup>1</sup> = 0.9 and *w*<sup>2</sup> = 0.1).

**Table 11.** Results of the integrated ED problem with prohibited operating zones and DG installation of the IEEE 57-Bus RTS (*w*<sup>1</sup> = 0.9 and *w*<sup>2</sup> = 0.1).


3.2.2. For *w*<sup>1</sup> = 0.5 and *w*<sup>2</sup> = 0.5

Tables 12 and 13 show the results of the integrated ED problem with the prohibited operating zones and DG installation of the IEEE 30-Bus RTS and IEEE 57-Bus RTS, respectively, with weight coefficients *w*<sup>1</sup> = 0.5 and *w*<sup>2</sup> = 0.5.

Again, MOEP and MOAIS produced the same solution for both systems. As referred to in Table 12, MOICEP produced a total production cost of 492.48 \$/h and total system loss of 1.84 MW, while MOEP and MOAIS produced a similar total production cost of 559.24 \$/h and a similar total system loss of 3.34 MW. MOICEP managed to save about 66.76 \$/h in the total production cost. Besides that, the total system loss found via MOICEP was almost half of the total system loss produced via MOEP and MOAIS.

According to Table 13 for the IEEE 57-Bus RTS, MOICEP managed to produce the lowest total production cost of 35,926.32 \$/h and total system loss of 11.83 MW. MOEP and MOAIS produced a similar total production cost of 36,146.27 \$/h and a similar total system loss of 12.75 MW. MOICEP identified the optimal locations of *DG*1, *DG*<sup>2</sup> and *DG*<sup>3</sup> in the IEEE 57-Bus RTS at bus 57, bus 36 and bus 43, respectively. The sizes for the three DG units were 4.52, 44.13 and 101.28 MW. MOEP and MOAIS identified bus 50, bus 33 and bus 10 as the optimal locations to install *DG*1, *DG*<sup>2</sup> and *DG*<sup>3</sup> with the sizes of 3.62, 23.72 and 120.12 MW, respectively. Generating unit 8, *PG*8, was found to be the highest generating unit to supply a real power output to the IEEE 57-Bus RTS in MOICEP. In MOEP and MOAIS, generating unit 12, *PG*12, was the highest contributor of real power output to the system.

**Table 12.** Results of the integrated ED problem with prohibited operating zones and DG installation of the IEEE 30-Bus RTS (*w*<sup>1</sup> = 0.5 and *w*<sup>2</sup> = 0.5).


**Table 13.** Results of the integrated ED problem with prohibited operating zones and DG installation of the IEEE 57-Bus RTS (*w*<sup>1</sup> = 0.5 and *w*<sup>2</sup> = 0.5).


3.2.3. For *w*<sup>1</sup> = 0.1 and *w*<sup>2</sup> = 0.9

The results of the fourth setting of the weight coefficients (*w*<sup>1</sup> = 0.1 and *w*<sup>2</sup> = 0.9) for the IEEE 30-Bus RTS and IEEE 57-Bus RTS are tabulated in Tables 14 and 15, respectively. As referred to in the two tables, it can be seen that MOEP and MOAIS continued to produce the same solution of the integrated ED problem with prohibited operating zones and DG installation. This could be due to the prohibited operating zones of the generating units that make the searching space of the MOEP and MOAIS algorithms smaller and lead to the same global optima. MOICEP achieved a total production cost of 517.76 \$/h and total system loss of 1.49 MW for the IEEE 30-Bus RTS, while, in the IEEE 57-Bus RTS, 35,184.12 \$/h and 6.94 MW. As referred to in Table 14, the optimal locations to install *DG*1,

*DG*<sup>2</sup> and *DG*<sup>3</sup> in the IEEE 30-Bus RTS were bus 29, bus 21 and bus 5 with the sizes of 5.00, 27.69 and 85.03 MW, respectively.

**Table 14.** Results of the integrated ED problem with prohibited operating zones and DG installation of the IEEE 30-Bus RTS (*w*<sup>1</sup> = 0.1 and *w*<sup>2</sup> = 0.9).


**Table 15.** Results of the integrated ED problem with prohibited operating zones and DG installation of the IEEE 57-Bus RTS (*w*<sup>1</sup> = 0.1 and *w*<sup>2</sup> = 0.9).


On the other hand, in MOEP and MOAIS, the locations and sizes were bus 23 with 3.84 MW, bus 10 with 42.29 MW and bus 27 with 54.73 MW. The six generating units (*PG*1, *PG*2, *PG*5, *PG*8, *PG*<sup>11</sup> and *PG*13) in MOICEP produced 13.55, 38.62, 27.96, 34.17, 16.25 and 36.61 MW, respectively. In MOEP and MOAIS, they produced 57.48, 20.05, 42.07, 17.98, 18.35 and 30.00 MW, respectively. Generating unit 1, *PG*1, in MOICEP produced the lowest real power output compared to the other units. Generating unit 8, *PG*8, produced the lowest real power output in MOEP and MOAIS.

According to Table 15 of the IEEE 57-Bus RTS, bus 21, bus 26 and bus 13 were found to be the optimal locations to install *DG*1, *DG*<sup>2</sup> and *DG*3, respectively. Their sizes were 4.99 MW, 34.73 and 150.00 MW, respectively.

Meanwhile, in MOEP and MOAIS, bus 3, bus 53 and bus 16 were the optimal locations to install *DG*1, *DG*<sup>2</sup> and *DG*<sup>3</sup> with the sizes of 4.87, 28.54 and 77.88 MW, respectively. It was observed that generating unit 2, *PG*2, in MOICEP produced a very little real power output of 1.04 MW compared to the other units. The lowest real power output produced by a generating unit in MOEP and MOAIS was 10.46 MW, which was generating unit 2, *PG*2.

#### *3.3. MOICEP-Based Technique for Economic Dispatch Problem with Valve-Point Loading Effect*

The total production cost and total system loss for this type of ED problem were calculated using Equations (2) and (3), respectively. The cost function of this ED was not smooth because of the valve-point loading effect of the generating unit considered in the cost function. The results of the integrated ED problem with a valve-point loading effect and DG installation are observed and discussed in this section.

### 3.3.1. For *w*<sup>1</sup> = 0.9 and *w*<sup>2</sup> = 0.1

Tables 16 and 17 show the results of the integrated ED problem with a valve-point loading effect and DG installation of the IEEE 30-Bus RTS and 57-Bus RTS with *w*<sup>1</sup> = 0.9 and *w*<sup>2</sup> = 0.1, respectively. It can be seen from Table 16 that MOICEP maintained producing the best solution of an ED with a valve-point loading effect and DG installation for the second setting of the weight coefficients by giving the lowest total production cost of 11,817.81 \$/h and the lowest total system loss of 2.04 MW. MOEP and MOAIS produced 13,331.25 \$/h and 3.61 MW and 11,627.07 \$/h and 13.32 MW, respectively. The best solution from MOICEP was backed by the placement of *DG*1, *DG*<sup>2</sup> and *DG*<sup>3</sup> at bus 26, bus 5 and bus 9 with the sizes of 5.00, 50.00 and 100.00 MW, respectively. Generating unit 1, *PG*1, and generating unit 8, *PG*8, produced the lowest and the highest real power outputs in MOICEP, respectively. The most expensive total production cost was obtained in MOEP, which was 13,331.25 \$/h. However, MOAIS produced the highest total system loss of 13.32 MW compared to the other two techniques. It was five times more than the total system loss produced via MOICEP.

As referred to in Table 17 of the IEEE 57-BUS RTS, it can be seen that MOICEP produced the cheapest total production cost of 32,560.84 \$/h and total system loss of 9.78 MW compared to MOEP and MOAIS. MOEP and MOAIS produced the same solution, with the total production cost of 35,369.69 \$/h and total system loss of 18.95 MW.

MOICEP found bus 26, bus 10 and bus 14 as the optimal locations to install *DG*1, *DG*<sup>2</sup> and *DG*<sup>3</sup> with the sizes of 5.00, 49.96 and 149.98 MW, respectively. MOEP and MOAIS identified bus 33, bus 10 and bus 40 as the optimal locations to install *DG*1, *DG*<sup>2</sup> and *DG*<sup>3</sup> with the sizes of 2.66, 36.55 and 141.03 MW, respectively. It was observed that the real power output of the three DG units was higher in MOICEP compared to the real power output in MOEP and MOAIS.


**Table 16.** Results of the integrated ED problem with a valve-point loading effect and DG installation of the IEEE 30-Bus RTS (*w*<sup>1</sup> = 0.9 and *w*<sup>2</sup> = 0.1).


**Table 17.** Results of the integrated ED problem with a valve-point loading effect and DG installation of the IEEE 57-Bus RTS (*w*<sup>1</sup> = 0.9 and *w*<sup>2</sup> = 0.1).

## 3.3.2. For *w*<sup>1</sup> = 0.5 and *w*<sup>2</sup> = 0.5

The third setting of the weight coefficients for solving the integrated ED problem with a valve-point loading effect and DG installation for the IEEE 30-Bus RTS and IEEE 57-Bus RTS are tabulated in Tables 18 and 19. Referring to Table 18, it can be seen that MOICEP managed to provide a better solution of ED with a valve-point loading effect and DG installation by producing a total production cost of 11,758.98 \$/h and total system loss of 1.50 MW compared to MOEP (13,340.00 \$/h and 3.31 MW) and MOAIS (13,178.04 \$/h and 2.28 MW). This was caused by the placement of *DG*1, *DG*<sup>2</sup> and *DG*<sup>3</sup> at bus 18, bus 9 and bus 5 with the sizes of 5.00 MW, 50.00 MW and 100.00 MW, respectively. The total DG size was the highest in MOICEP (155.00 MW), while, for MOEP, it was 125.91 MW and, for MOAIS, it was 133.95 MW. Generating unit 1, *PG*1, produced the lowest real power output of 7.05 MW compared to the other generating units in MOICEP.

**Table 18.** Results of the integrated ED problem with a valve-point loading effect and DG installation of the IEEE 30-Bus RTS (*w*<sup>1</sup> = 0.5 and *w*<sup>2</sup> = 0.5).



**Table 19.** Results of the integrated ED problem with a valve-point loading effect and DG installation of the IEEE 57-Bus RTS (*w*<sup>1</sup> = 0.5 and *w*<sup>2</sup> = 0.5).

The results of the IEEE 57-Bus are shown in Table 19. It can be observed that MOICEP continued to produce a better solution of the integrated ED problem with a valve-point loading effect and DG installation. This was based on the total production cost and total system loss produced, which were the lowest at 33,447.51 \$/h and 7.10 MW, respectively. MOEP and MOAIS produced the same results, with a total production cost of 38,013.28 \$/h and total system loss worth 13.26 MW.

In MOICEP, bus 25, bus 22 and bus 13 were identified as the best locations to install *DG*1, *DG*<sup>2</sup> and *DG*<sup>3</sup> with the sizes of 4.97, 42.96 and 150.00 MW, respectively. The generating units 1, 2, 5, 6, 8, 9 and 12 produced real power outputs of 131.64, 36.11, 65.13, 72.78, 273.04, 100.00 and 381.24 MW, respectively.

MOEP and MOAIS identified bus 45, bus 30 and bus 55 as the optimal locations to place *DG*1, *DG*<sup>2</sup> and *DG*<sup>3</sup> with the sizes of 3.53, 24.77 and 132.47 MW, respectively. The total DG size was higher in MOICEP, which was 197.93 MW compared to MOEP and MOAIS, which was 160.77 MW. As a result, MOICEP achieved a lower total production cost than MOEP and MOAIS.

### 3.3.3. For *w*<sup>1</sup> = 0.1 and *w*<sup>2</sup> = 0.9

The breadth of this study was expanded with the next weight coefficient setting, i.e., *w*<sup>1</sup> = 0.1 and *w*<sup>2</sup> = 0.9. Tables 20 and 21 show the results of the integrated ED problem with a valve-point loading effect and DG installation of the IEEE 30-Bus RTS and IEEE 57-Bus RTS, respectively.

From Table 20, it can be seen that MOICEP achieved the lowest total system loss of 1.47 MW compared to MOEP (2.34 MW) and MOAIS (2.28 MW). However, the total production cost produced in MOAIS was the lowest compared to MOICEP and MOEP. MOAIS produced a total production cost of 13,178.04 \$/h, while MOICEP and MOEP produced 13,472.72 \$/h and 15,179.41 \$/h, respectively. The highest total production cost was found in MOEP. The optimal locations and sizes found via MOICEP for *DG*1, *DG*<sup>2</sup> and *DG*<sup>3</sup> were bus 29 with 5.00 MW, bus 22 with 28.71 MW and bus 5 with 86.75 MW. It was observed that, to make sure the total system loss was the lowest, MOICEP avoided the three DG units operating near their maximum capacity.

A similar observation was experienced by the IEEE 57-Bus RTS, as shown in Table 21; the total system loss produced via MOICEP was the lowest. However, the total production cost was the highest compared to MOEP and MOAIS. MOICEP produced a total production cost of 38,352.96 \$/h and total system loss of 8.35 MW. MOEP and MOAIS produced the same results, with a total production cost of 38,013.28 \$/h and total system loss of 13.26 MW. The optimal locations to install *DG*1, *DG*<sup>2</sup> and *DG*<sup>3</sup> in the IEEE 57-Bus RTS via MOICEP were bus 29, bus 22 and bus 13 with the sizes of 1.03, 44.02 and 59.53 MW, respectively. The total system loss produced via MOICEP was 37.03% lower than the total system loss produced via MOEP and MOAIS. However, in terms of the total production cost, MOICEP did not force the three DG units to operate near their maximum capacity. This resulted in a high total production cost produced via MOICEP. It must be kept in mind that the weight coefficients also play an important role that leads to the results. As the value of *w*<sup>1</sup> decreases and the value of *w*<sup>2</sup> increases, the algorithm tends to focus on optimizing objective function 2 (total system loss), as expected.

**Table 20.** Results of the integrated ED problem with a valve-point loading effect and DG installation of the IEEE 30-Bus RTS (*w*<sup>1</sup> = 0.1 and *w*<sup>2</sup> = 0.9).


**Table 21.** Results of the integrated ED problem with a valve-point loading effect and DG installation of the IEEE 57-Bus RTS (*w*<sup>1</sup> = 0.1 and *w*<sup>2</sup> = 0.9).


#### **4. Conclusions**

The new Multi-Objective Immune-Commensal-Evolutionary Programming (MOICEP) technique for the total production cost and total system loss minimization via integrated Economic Dispatch and Distributed Generation installation (ED-DG) was presented in this paper. It was employed for the total production cost and total system loss minimization in an integrated weighted-sum multi-objective optimization technique for solving the ED problem with DG installation consideration. This scheme is uniquely termed as Economic

Dispatch and Distributed Generation installation (ED-DG). The proposed weighted-sum multi-objective function that integrated both the total production cost and total system loss was highlighted in this study. The introduction of weight coefficients in the proposed fitness exhibited convincing results in addressing both components. The weight coefficients were varied between 0.1 and 0.9 to find the optimal solution that satisfied both the total production cost and total system loss. From the variations, it could be seen that, as the value of *w*<sup>1</sup> was greater than *w*2, the solution produced favored total production cost minimization, while, if the value of *w*<sup>2</sup> was greater than *w*1, the solution favored total system loss minimization. GSOs can decide which solution is the best for their system requirements and their desired output. The proposed MOICEP technique managed to achieve better results over benchmarked techniques, i.e., MOEP and MOAIS, for the minimization of the total production costs and total system loss for three types of ED problems (basic ED, ED problem with prohibited operating zones and an ED problem with a valve-point loading effect). This phenomenon highlights the superiority of MOICEP over MOEP and MOAIS. This phenomenon has been identified as cutting edge in ED problems as the original independent scheme, i.e., ED and DG installation. The proposed multi-objective technique could be feasible for solving other optimization problems with considerable fine-tuning. It is also beneficial for power system utilities in solving their ED problems. It can also be concluded that, while optimizing the two objectives, total production cost and total system loss, the integrated Economic Dispatch and Distributed Generation installation ED-DG problems solved using the proposed MOICEP technique simultaneously solved two power system problems, ED and DG installation, which have been previously solved separately by many researchers. The two objectives are inextricably linked to the two problems.

The number of objectives for the multi-objective ED problems can be increased in the future. This can be done by adding more objectives like voltage stability improvement, emission minimization and DG installation cost minimization into the integrated ED-DG problems.

**Author Contributions:** All the authors listed conceived and designed the project. Conceptualization, M.H.M. and I.M.; acquired the funding, M.H.M.; methodology, M.H.M., I.M. and M.M.O.; validation, M.H.M. and I.M.; analysis, M.H.M. and M.M.O.; writing—original draft preparation, M.H.M. and I.M. and writing—review and editing, M.M.O. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded by the Ministry of Higher Education Malaysia (MOHE), grant number FRGS/1/2020/TK0/UNITEN/03/4.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


## *Article* **DSO Strategies Proposal for the LV Grid of the Future**

**Bartłomiej Mroczek 1,2,\* and Paweł Pijarski <sup>1</sup>**


**Abstract:** A significant challenge for the DSO (Distribution System Operator) will be to choose the optimum strategy for flexibility service in the LV area with high RES (renewable energy sources) penetration. To this end, a representative LV grid operated in Poland was selected for analysis. Three research scenarios with RES generation were presented in the range of 1–8 kW for the power factor from 0.9 to 1. The grid PV capacity was determined for four load profiles. Based on this factor, optimum RES volume management service types were determined. Under the flexibility service, the proposed power conversion services and active RES operations for DSO were proposed. The research was conducted using the Matlab and PowerWorld Simulator environment. Optimum active power values were obtained for the RES generation function for single and dual operation systems of the power conversion system. In future, the knowledge in the field of grid capacity will enable the DSO to increase the operating efficiency of the LV grid. It will enable the optimum use of the RES generation maximisation function and proper strategy selection. It will improve the energy efficiency of the power input through the MV/LV node.

**Keywords:** grid feeding; energy storage; PV; flexibility service; optimisation model

## **1. Introduction**

The European Union strategy presented in the European Green Deal aims at reducing greenhouse gas emissions by at least 55% by 2030 [1]. The set target will be attained among other things by installing renewable energy sources, while improving the energy efficiency. The direction to de-carbonise the power generation sector has already been accepted by the member states, including Poland. One of the main premises of the transformation process is action intended to protect the planet and human health. The validity of the assumed transformation path is confirmed by section VI of the IPCC report (AR6) [2], which indicates that global warming of 1.5–2 deg C/F can be stopped by quick greenhouse gas reduction.

The commonly observed mega-trend towards the development of prosumer power generation in Poland is confirmed by the statistics in the reports presented by the President of Energy Regulatory Office [3], who point to the significant dynamics of the year-to-year increase in new installations. The dynamics of the increase in prosumer microinstallations was approximately 191% in the period of 2018/2019 and approximately 202% in the period of 2019/2020. Due to the availability of subsidies and financial support for such projects, territorial units work with the residents to de-carbonise the urban and suburban sections of the LV grid. The electricity supply and consumption balance for end consumers changes locally.

At the national level, Poland's energy policy until 2040 predicts a fivefold increase in the number of prosumers by 2030, relative to the current condition. In addition, the number of energy-sustainable areas will increase at the local level [4]. It is predicted that microgrids will be formed based on separated sections of the LV grid. The power and electrical energy settlement for end consumers will be shifted to local balance areas as the local dispersed generation capacity increases.

**Citation:** Mroczek, B.; Pijarski, P. DSO Strategies Proposal for the LV Grid of the Future. *Energies* **2021**, *14*, 6327. https://doi.org/10.3390/ en14196327

Academic Editors: Victor Becerra and Ahmed Rachid

Received: 30 August 2021 Accepted: 29 September 2021 Published: 3 October 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

It should be noted that the photovoltaic panel technology changes at the same time. At present, manufacturers are introducing to the market a new type of panel in bifacial technology [5–7], with higher capacity levels (500–670 kWp) than previous types, defined as ultra-high power modules. This will provide the prosumers with higher generation capacities, often exceeding their momentary power requirements.

The process of power generation sector transformation through dispersed power generation changes the operating characteristics of the LV grid [8,9], to which these sources are connected. This phenomenon has at least two new contexts. The first context is the technical adjustment of the LV grid and the second context is the current balancing of the demand and the supply according to the established quality of service parameters. The European Commission has identified the need to manage these contexts. It defined the mitigation tool as energy grid flexibility and addressed it in art. 32 of chapter IV of Directive 944/2019 [10]. This enabled the DSO to apply a freely selected strategy (demand size response, storage system, etc.), depending on the context of the LV grid operation.

Considering the above-listed factors changing the existent LV grid operation, this research was motivated by the need to develop and describe the possible strategies for the DSO in the area. At present, the authors of the paper point to two possible technical solutions. The first solution is centralised and consists in using power converter systems as energy storage systems (ESS) in the grid operation system on the DSO side [11–15], as part of the infrastructure. In this solution, there is no description of a particular location for grid connection (beginning, middle, end of the LV grid) and the number of such systems as ESS. These considerations are exclusively connected with a single specific test case. The second solution is decentralised and consists in using power conversion systems with energy storage systems at the consumer (prosumers) [16–20]. The use of dispersed ESS systems depends on the times of response to the service of the ESS user [21] and/or the availability of the ESS systems at the given adjustment moment [22]. At the given moment, each ESS features different parameters: Capacity, power, ramp, rate, cycle time [23]. The flexibility of this service is negligible, including the control/adjustment capacity on the side of the DSO. Therefore, the authors of this paper identified the research problem in the form of the selection of the strategy for the LB grid based on a measurable factor. The grid PV capacity indicator was selected [24–26] for the LV grid operating in Poland. The power flow throughput limits of the LV grid were the subject of this research and the resulting evaluation, positive or negative, is a proposal for strategy development by the DSO.

#### **2. Materials and Methods**

#### *2.1. Research Problem*

The basic assumption of the research is the absorption of the maximum output of electrical energy generation of all PV sources in the LV grid, without applying restrictions to micro-producers. The role of the DSO is to use the available grid flexibility utilities to:


The authors of the paper assumed the research hypothesis that flexibility service rating exists, connected with the prosumer and generation load profile. The rating was built on the basis of the cost index [33,34] (unit power price of the converter system, number of ESS in relation to the existing PV inverters) and effective grid regulation. The obtained results are presented in Section 4. The ultimate purpose was to provide the DSO with a proposal including effective service flexibility strategies in accordance with the process presented in Figure 1.

**Figure 1.** Launch the grid LV flexibility service. **Figure 1.** Launch the grid LV flexibility service.

The light beige colour was used to mark the tasks that the DSO currently performed in the distribution service provision process. The dark brown colour was used to mark the tasks that the DSO would perform in the future [10]. The light beige colour was used to mark the tasks that the DSO currently performed in the distribution service provision process. The dark brown colour was used to mark the tasks that the DSO would perform in the future [10].

on the basis of the cost index [33,34] (unit power price of the converter system, number of ESS in relation to the existing PV inverters) and effective grid regulation. The obtained results are presented in Section 4. The ultimate purpose was to provide the DSO with a proposal including effective service flexibility strategies in accordance with the process

#### *2.2. Research Object 2.2. Research Object*

presented in Figure 1.

The DSO infrastructure of the LV grid selected for research purposes was typical of suburban areas of agglomerations, to which new dispersed energy sources were connected systematically. The LV grid is radial and overhead, with two feeders (F1 and F2). The bus is made using a wire with a section of 70 mm2; this results directly from the DSO standard in Poland [35]. The LV grid diagram is presented in Figure 2. The LV grid is connected with 23 consumer/prosumers with the connection power of 11 kW. Figure 2 shows the distances of the individual sections of the F1 and F2 bus (unit: Meter, e.g., −45 between the power poles Nr S1-1 and Nr S1-2). The DSO infrastructure of the LV grid selected for research purposes was typical of suburban areas of agglomerations, to which new dispersed energy sources were connected systematically. The LV grid is radial and overhead, with two feeders (F1 and F2). The bus is made using a wire with a section of 70 mm<sup>2</sup> ; this results directly from the DSO standard in Poland [35]. The LV grid diagram is presented in Figure 2. The LV grid is connected with 23 consumer/prosumers with the connection power of 11 kW. Figure 2 shows the distances of the individual sections of the F1 and F2 bus (unit: Meter, e.g., −45—between the power poles Nr S1-1 and Nr S1-2).

**Figure 2.** Diagram of the research LV grid. **Figure 2.** Diagram of the research LV grid.

The technical specifications of the analysed grid were specified in Tables 1–3. The technical specifications were extracted from the LV line design and supplemented with catalogue specifications. The technical specifications of the analysed grid were specified in Tables 1–3. The technical specifications were extracted from the LV line design and supplemented with catalogue specifications.

**Table 1.** Technical specifications of the overhead line [36]. **Table 1.** Technical specifications of the overhead line [36].


**Table 2.** Technical specifications of the transformer. **Table 2.** Technical specifications of the transformer.


**Table 3.** Distances between the MV/LV transformer and the connection points (only data per consumer/connection point). **Table 3.** Distances between the MV/LV transformer and the connection points (only data per consumer/connection point).


9 0.261 (1.09) 0.716 (2.09)


**Table 3.** *Cont.*

The total length of LV line for feeder 1 = 0.427 km, for feeder 2 = 0.732 km.

#### *2.3. Boundary Conditions for Simulations*

Research works were conducted based on specifications of the actual LV grid in Central Poland (Tables 1–3). To this end, the Matlab environment was used, integrated through dedicated scripts with the PowerWorld Simulator environment (Figure 3). Power distribution for the given condition, for each connection point, was calculated in accordance with the following formula (Equations (1) and (2)) [22,37].

$$P\_k(V\_{k\prime}, V\_{j\prime}\theta\_k, \theta\_j) = \sum\_{j=1}^{N} |V\_k| \left| V\_j \right| \left( G\_{kj} \cos(\theta\_{kj}) + B\_{kj} \sin(\theta\_{kj}) \right) \tag{1}$$

$$Q\_k(V\_{k'}V\_j, \theta\_{k'}\theta\_j) = \sum\_{j=1}^{N} |V\_k| \left| V\_j \right| (G\_{kj}\sin(\theta\_{kj}) - B\_{kj}\cos(\theta\_{kj})) \tag{2}$$

where:

*Pk*—active power at bus *k*;

*Qk*—reactive power at bus *k*;

*Vk*—Voltage magnitude at bus *k*;

*Vj*—Voltage magnitude at bus *j*;

*Gkj*—Mutual conductance between buses *k* and *j*;

*Bkj*—Mutual susceptance between buses *k* and *j*;

*θkj*—Voltage angle difference between buses *k* and *j*;

One of the most important components for the development of proper evaluation of LV grid operation with a large number of PV generators is proper selection of the *P<sup>l</sup>* load model by the consumer/prosumers. The consumer profiles used in simulation are selected to include the broadest possible range of cases possible under the given conditions, the most extreme for the LV grid operation [38]. In addition, *P<sup>l</sup>* has a direct impact on the LV grid operation, balancing or not the current/present generation profile—*Pgc.* The four selected profiles *P<sup>l</sup>* are presented in Table 4.



**Figure 3. Figure 3.**  Section of the LV grid (Figure Section of the LV grid (Figure 1) impl 1) implemented in the PowerWorld environment. emented in the PowerWorld environment**.**  4. Random data, Figure 4b RD <0.5–2.5> kW <0.5–2.5> kW

The generation PV profiles *Pgc* for Central Poland are the production hours from 9 am to 6 pm within prosumer load demand *P<sup>l</sup>* , in the same time [39]. The value *P<sup>l</sup>* on the side of prosumers is different for period: (A) Saturday–Sunday/holiday, (B) Monday–Friday. Generally, *P<sup>l</sup>* in the LV grid is stochastic. Therefore, an attempt was made to define the boundary conditions so that they contain the largest number of solutions corresponding to the real *P<sup>l</sup>* profile. The first three profiles—SF, RS, LS—correspond to period A. With this end in view, defined load with a constant value (SF) was calculated in accordance with the standard SEP-E-002 (power connection 11 kW). The other two profiles, RS and LS, are unequal loads. There are differences at the beginning and end of the circuit F1 and F2. This has been described mathematically (Table 4). The fourth profile (Table 4.4) is typical loads for period B, where the profile is developed based on small household appliances (refrigerator) and electronics in stand-by mode. The generation PV profiles *Pgc* for Central Poland are the production hours from 9 am to 6 pm within prosumer load demand *Pl*, in the same time [39]. The value *Pl* on the side of prosumers is different for period: (A) Saturday–Sunday/holiday, (B) Monday–Friday. Generally, *Pl* in the LV grid is stochastic. Therefore, an attempt was made to define the boundary conditions so that they contain the largest number of solutions corresponding to the real *Pl* profile. The first three profiles—SF, RS, LS—correspond to period A. With this end in view, defined load with a constant value (SF) was calculated in accordance with the standard SEP-E-002 (power connection 11 kW). The other two profiles, RS and LS, are unequal loads. There are differences at the beginning and end of the circuit F1 and F2. This has been described mathematically (Table 4). The fourth profile (Table 4.4) is typical loads for period B, where the profile is developed based on small household appliances (refrigerator) and electronics in stand-by mode.

10 0.262 (1.10) 0.768 (2.10) 11 0.354 (1.11) - 12 0.402 (1.12) - 13 0.453 (1.13) -

Research works were conducted based on specifications of the actual LV grid in Central Poland (Tables 1–3). To this end, the Matlab environment was used, integrated through dedicated scripts with the PowerWorld Simulator environment (Figure 3). Power distribution for the given condition, for each connection point, was calculated in accord-

( ( ) + () ) (1)

( sin()− cos( )) (2)

The total length of LV line for feeder 1 = 0.427 km, for feeder 2 = 0.732 km.

ance with the following formula (Equations (1) and (2)) [22,37].

ே ୀଵ

> ே ୀଵ

൫, , , ൯= ||หห

(, , , )= ||หห

*Gkj*—Mutual conductance between buses k and *j*; *Bkj*—Mutual susceptance between buses k and *j*; *ϴkj*—Voltage angle difference between buses k and *j*;

*Energies* **2021**, *14*, x FOR PEER REVIEW 3 of 21

*2.3. Boundary Conditions for Simulations* 

*Pk*—active power at bus k; *Qk*—reactive power at bus k; *Vk*—Voltage magnitude at bus k; *Vj*—Voltage magnitude at bus *j*;

where:

**Figure 4.** *Cont*.

**Figure 4.** Distribution of load profiles: (**a**) SR; (**b**) RS; (**c**) LS F1; (**d**) RD F1; (**e**) LS F2; (**f**) RD F2. **Figure 4.** Distribution of load profiles: (**a**) SR; (**b**) RS; (**c**) LS F1; (**d**) RD F1; (**e**) LS F2; (**f**) RD F2.

The apparent power distribution in the LV grid for each of the described profiles *Pl,*  were simulated for *Pgc* in the range of 1 kW to 8 kW and tracking of 0.25 kW. Since the LV grid, despite the high R/X ratio [37], does not operate in pure resistance mode (PV power electronic system operation) [40], PV generation is simulated for the power factor (p.f.) from 0.9 to 1. The encountered voltage asymmetry in the LV grid was eliminated by the four-wire power conversion system, controlled independently for each phase [41,42]. In the research, this fact was omitted as negligible; phases F1 and F2 can be considered as technically separate and controlled independently. The apparent power distribution in the LV grid for each of the described profiles *Pl,* were simulated for *Pgc* in the range of 1 kW to 8 kW and tracking of 0.25 kW. Since the LV grid, despite the high R/X ratio [37], does not operate in pure resistance mode (PV power electronic system operation) [40], PV generation is simulated for the power factor (p.f.) from 0.9 to 1. The encountered voltage asymmetry in the LV grid was eliminated by the four-wire power conversion system, controlled independently for each phase [41,42]. In the research, this fact was omitted as negligible; phases F1 and F2 can be considered as technically separate and controlled independently.

#### *2.4. Scenarios 2.4. Scenarios*

The boundary conditions described in the previous section as components of data input for the research were framed in three research scenarios. In addition, the operating system of the LV grid (Figure 1) is supplemented with ESS operating in the grid feeding mode [43] (Figure 5). The boundary conditions described in the previous section as components of data input for the research were framed in three research scenarios. In addition, the operating system of the LV grid (Figure 1) is supplemented with ESS operating in the grid feeding mode [43] (Figure 5).

The operation of the LV grid with ESS was controlled by the inputs *P* (active power) and *Q* (reactive power) of the power converter system (Figure 5) to achieve the objectives described in Section 2.1 (Numbers 1–3).

*Energies* **2021**, *14*, x FOR PEER REVIEW 5 of 21

**Figure 5.** Control system—Grid Feeding. **Figure 5.** Control system—Grid Feeding. **Figure 5.** Control system—Grid Feeding.

The operation of the LV grid with ESS was controlled by the inputs *P* (active power) and *Q* (reactive power) of the power converter system (Figure 5) to achieve the objectives described in Section 2.1 (Numbers 1–3). The first research scenario is the evaluation of the grid PV capacity for LV, the "AS The first research scenario is the evaluation of the grid PV capacity for LV, the "AS IS" condition analysis. The input data values *Pgc* and *P<sup>l</sup>* were searched for the LV grid, where the voltage requirement of 1.1 *Un* is not met. In addition, power flows towards MV through the transformer were checked. The operation of the LV grid with ESS was controlled by the inputs *P* (active power) and *Q* (reactive power) of the power converter system (Figure 5) to achieve the objectives described in Section 2.1 (Numbers 1–3). The first research scenario is the evaluation of the grid PV capacity for LV, the "AS

IS" condition analysis. The input data values *Pgc* and *P<sup>l</sup>* were searched for the LV grid, where the voltage requirement of 1.1 *Un* is not met. In addition, power flows towards MV through the transformer were checked. The second research scenario was implemented using ESS in the operation system, Figure 6a. The analysis covered the LV grid and its operating characteristics with the boundary conditions described in Section 2.3 to define the values of *P* and *Q* for ESS. Location at one point of the grid—the MV/LV station (Figure 6a). IS" condition analysis. The input data values *Pgc* and *P<sup>l</sup>* were searched for the LV grid, where the voltage requirement of 1.1 *Un* is not met. In addition, power flows towards MV through the transformer were checked.

The third research scenario was implemented using two ESS systems: *S1* and *S2* in the operation system, Figure 6b. The analysis covered the LV grid and its operating characteristics with the boundary conditions described in Section 2.3 to define the values of *P1*, *P2* and *Q1*, *Q2* for *S1* and *S2* (Figure 6b). The location of the ESS in the depth of the LV grid is based on the data of scenario no. 1.

#### **3. Results**

By iterative power flow analysis in the dedicated LV grid, within the scope of three scenarios and four load profiles, the answer was obtained in the field of grid PV capacity. Developed results presented in Table 5. Based on them, a non-linear function was developed, described in three dimensions *f(x, y, z)* where: *x*—distance from the MV/LV station (distance); *y*—values of PV current generation (*Pgc*); *z*—voltage at the given node (*U p.u*.). It should be emphasized that the results in Table 5 are the maximum values of PV power generation for which the *Un* = 1.1 condition is met for each node in the LV grid.

**Table 5.** Capacity Grid for four load profiles.


The values presented in Table 5 are the starting point for the regulation of power converter systems (power tracking) in accordance with the diagram in Figure 5. The obtained grid PV capacity results will also be the basis for the evaluation of the strategy selection on the DSO side. Green: Grid PV capacity values that do not require adjustment for a given load profile; red: Those cases of LV network operation that require a response from the DSO; yellow: Result values are shown for little regulation adjustment.

Analytical research was carried out in two ways. The first track of works was carried out in the field of searching for a grid PV capacity for one value of the maximum RES generation capacity per every node. The second course was carried out by the optimal grid PV capacity values for all nodes (different values of RES per node). Below is a mathematical model for the second track.

The purpose of the research is to maximise RES generation. Therefore, the objective function is grid PV capacity measured by the maximum generation value of *P* at cos(ϕ) = const., for which the voltage requirements for the electrical energy supply are met. For the purposes of optimisation of the objective function *f(x, y, z),* the equations of Karush–Kuhn–Tucker [44,45] for non-linear programming algorithms were used, following the formulae (Equations (3)–(6)) [46]:

$$\max\_{\mathbf{x}\in\mathbb{R}^n} \quad = f(\mathbf{x})\tag{3}$$

$$L(\mathbf{x}, \boldsymbol{\mu}, \boldsymbol{v}) = f(\mathbf{x}) + \sum\_{i=1}^{m} \mu\_{i} g\_{i}(\mathbf{x}) + \sum\_{j=1}^{r} \lambda\_{j} h\_{j}(\mathbf{x}),\tag{4}$$

$$-\nabla f(\mathbf{x}) + \sum\_{i=1}^{m} \mu\_i \nabla \mathbf{g}\_i(\mathbf{x}) + \sum\_{j=1}^{r} \lambda\_i \nabla h\_j(\mathbf{x}) = \mathbf{0},\tag{5}$$

$$\begin{array}{l} \text{g}\_{i}(\mathbf{x}) \le \mathbf{0}, \text{ for } i = 1, \dots, m\\ \text{h}\_{i} = \mathbf{0}, \text{ for } j = 1, \dots, r\\ \mu\_{i} \mathbf{g}\_{i}(\mathbf{x}) = \mathbf{0}, \text{ for } i = 1, \dots, m. \end{array} \tag{6}$$

The objective function has two limits: *gi*—in the RES power range (*Pgc*); *hi*—in the 1.1 *p.U.* value range for each node (*U per unit*).

#### *3.1. The First Scenario*

The first scenario is dedicated to the evaluation of grid PV capacity without flexibility service. It clearly indicates the limits, at which the given feeder (F1 and F2) achieves the

(**f**) F2, p.f = 0.9;.

maximum operating values. In addition, Figure 6 presents the results of optimisation of RES operation. However, Figure 5 presents the operation profile of the MV/LV transformer in this type of grid. **Capacity Grid SF RS LS RD** Feeder 1, cosf(φ) = 1.00 for PV <8 kW <8 kW <8 kW 7.5 kW

In Table 5, the max *Pgc* values obtained for each *P<sup>l</sup>* profile were listed. At the same

Feeder 2, cosf(φ) = 1.00 for PV 7.75 kW <8 kW 7.0 kW 5.75 kW

#### 3.1.1. Grid PV Capacity Feeder 1, cosf(φ) = 0.95 for PV 6.5 kW 7.25 kW 5.75 kW 5.5 kW

3.1.1. Grid PV Capacity

In Table 5, the max *Pgc* values obtained for each *P<sup>l</sup>* profile were listed. At the same time, reactive power from the RES side was input at connection points. Feeder 2, cosf(φ) = 0.95 for PV 6.0 kW 6.75 kW 5.25 kW 4.25 kW Feeder 1, cosf(φ) = 0.90 for PV 5.75 kW 6.25 kW 5.0 kW 4.5 kW

#### 3.1.2. Optimisation of the Non-Linear Function Feeder 2, cosf(φ) = 0.90 for PV 5.25 kW 6.0 kW 4.75 kW 4.0 kW The values presented in Table 5 are the starting point for the regulation of power converter sys-

**Table 5.** Capacity Grid for four load profiles.

*Energies* **2021**, *14*, x FOR PEER REVIEW 7 of 21

time, reactive power from the RES side was input at connection points.

In Figure 7, only for *P<sup>l</sup>* = SR (limited due to the number of possible diagrams), optimum RES generation values were presented for each connection point as a function of the distance. These are the optimum values of grid PV capacity per node. The analysis points to the limits for the individual feeders and individual nodes. This points to the potential ESS connection point in the depth of the grid, the minimum value of the function in the given distance range. tems (power tracking) in accordance with the diagram in Figure 5. The obtained grid PV capacity results will also be the basis for the evaluation of the strategy selection on the DSO side. Green: Grid PV capacity values that do not require adjustment for a given load profile; red: Those cases of LV network operation that require a response from the DSO; yellow: Result values are shown for little regulation adjustment. 3.1.2. Optimisation of the Non-Linear Function

#### 3.1.3. Operation Balance for the MV/LV Transformer In Figure 7, only for *P<sup>l</sup>* = SR (limited due to the number of possible diagrams), opti-

For the load profile of *P<sup>l</sup>* = SR, as in the previous sub-section, power inputs through the MV/LV station were determined as a function of RES generation increase. Figure 8 shows how the power input from the MV grid to the LV grid changes the vector to LV to MV. The process is completed regardless of the type of load profile and p.f., in connection with the increase of the RES generation value (*Pgc*). mum RES generation values were presented for each connection point as a function of the distance. These are the optimum values of grid PV capacity per node. The analysis points to the limits for the individual feeders and individual nodes. This points to the potential ESS connection point in the depth of the grid, the minimum value of the function in the given distance range.

**Figure 7.** *Cont*.

(**e**) (**f**) **Figure 7.** Optimisation results for SR: (**a**) F1, p.f = 1; (**b**) F2, p.f = 1; (**c**) F1, p.f = 0.95; (**d**) Figure 2. p.f = 0.95; (**e**) F1, p.f = 0.9;

> For the load profile of *P<sup>l</sup>* = SR, as in the previous sub-section, power inputs through the MV/LV station were determined as a function of RES generation increase. Figure 8 shows how the power input from the MV grid to the LV grid changes the vector to LV to MV. The process is completed regardless of the type of load profile and p.f., in connection

3.1.3. Operation Balance for the MV/LV Transformer

with the increase of the RES generation value *(Pgc).*

(**c**) (**d**)

**Figure 7.** Optimisation results for SR: (**a**) F1, p.f = 1; (**b**) F2, p.f = 1; (**c**) F1, p.f = 0.95; (**d**) Figure 2. p.f = 0.95; (**e**) F1, p.f = 0.9; (**f**) F2, p.f = 0.9;. **Figure 7.** Optimisation results for SR: (**a**) F1, p.f. = 1; (**b**) F2, p.f. = 1; (**c**) F1, p.f. = 0.95; (**d**) Figure 2. p.f. = 0.95; (**e**) F1, p.f. = 0.9; (**f**) F2, p.f. = 0.9.

**Figure 8.** Operation profile of the MV/LV transformer as a function of RES generation increase (P*gc*). **Figure 8.** Operation profile of the MV/LV transformer as a function of RES generation increase (*Pgc*).

Therefore, the selection of transformer power at high RES penetration is important. Nominal service operation with PV requires re-selection of the transformer power or in addition requires the use of flexibility service. Therefore, the selection of transformer power at high RES penetration is important. Nominal service operation with PV requires re-selection of the transformer power or in addition requires the use of flexibility service.

#### 3.1.4. Reactive Power during PV Generation 3.1.4. Reactive Power during PV Generation

The requirements for PV inverters on the side of the DSO are p.f. = 1. Due to the quality of the power electronic systems used by the prosumers, deviations from this value exist. Notwithstanding, this type of operation in case of small *Pl* power values does not serve its purpose in terms of grid PV capacity (Table 5). The requirements for PV inverters on the side of the DSO are p.f. = 1. Due to the quality of the power electronic systems used by the prosumers, deviations from this value exist. Notwithstanding, this type of operation in case of small *P<sup>l</sup>* power values does not serve its purpose in terms of grid PV capacity (Table 5).

The second method is active collaboration with the prosumers for reactive power consumption from the LV grid. Reactive power could be managed by PV inverters in the p.f. range of 0.9–1. This will reduce the RES production, but at the same time it will enable the prosumer to provide service to the DSO. Below is presented the most demanding case: *Pl* = RD and p.f. = 0.9, the results of the described prosumer service. Figure 9 features the RMS value of the nodal voltage for the two feeders—blue, while red states restriction level for node 1.1 *Un*. The second method is active collaboration with the prosumers for reactive power consumption from the LV grid. Reactive power could be managed by PV inverters in the p.f. range of 0.9–1. This will reduce the RES production, but at the same time it will enable the prosumer to provide service to the DSO. Below is presented the most demanding case: *P<sup>l</sup>* = RD and p.f. = 0.9, the results of the described prosumer service. Figure 9 features the RMS value of the nodal voltage for the two feeders—blue, while red states restriction level for node 1.1 *Un*.

The conducted research proved that in this case, regardless of the load profile, and for RES generation in the range of 〈1 ∶ 8〉 kW, there is no voltage excess at node points.

(**a**) (**b**) **Figure 9.** The voltage value distribution at connection points, with reactive power consumption by RES: (**a**) RD, p.f = 0.9

*3.2. The Second Scenario* 

This is the most effective method; however, it depends on a third party.

for F1; (**b**) RD, p.f = 0.9 for F2.

serve its purpose in terms of grid PV capacity (Table 5).

addition requires the use of flexibility service.

3.1.4. Reactive Power during PV Generation

**Figure 9.** The voltage value distribution at connection points, with reactive power consumption by RES: (**a**) RD, p.f = 0.9 for F1; (**b**) RD, p.f = 0.9 for F2. **Figure 9.** The voltage value distribution at connection points, with reactive power consumption by RES: (**a**) RD, p.f. = 0.9 for F1; (**b**) RD, p.f. = 0.9 for F2.

The conducted research proved that in this case, regardless of the load profile, and for RES generation in the range of 〈1 ∶ 8〉 kW, there is no voltage excess at node points. This is the most effective method; however, it depends on a third party. The conducted research proved that in this case, regardless of the load profile, and for RES generation in the range of h1 : 8i kW, there is no voltage excess at node points. This is the most effective method; however, it depends on a third party. *Energies* **2021**, *14*, x FOR PEER REVIEW 10 of 21

**Figure 8.** Operation profile of the MV/LV transformer as a function of RES generation increase (P*gc*).

Therefore, the selection of transformer power at high RES penetration is important. Nominal service operation with PV requires re-selection of the transformer power or in

The requirements for PV inverters on the side of the DSO are p.f. = 1. Due to the quality of the power electronic systems used by the prosumers, deviations from this value exist. Notwithstanding, this type of operation in case of small *Pl* power values does not

The second method is active collaboration with the prosumers for reactive power consumption from the LV grid. Reactive power could be managed by PV inverters in the p.f. range of 0.9–1. This will reduce the RES production, but at the same time it will enable the prosumer to provide service to the DSO. Below is presented the most demanding case: *Pl* = RD and p.f. = 0.9, the results of the described prosumer service. Figure 9 features the RMS value of the nodal voltage for the two feeders—blue, while red states restriction level

#### *3.2. The Second Scenario 3.2. The Second Scenario*

for node 1.1 *Un*.

The second scenario is dedicated to the evaluation of the required capacity of the power conversion system for voltage condition adjustment at nodes. The operation system is presented in Figure 6a. The volume of data collected for *S1* at the given *P<sup>l</sup>* depends on the grid PV capacity (Table 5). Figures 10–12 present the active power values *P* for *S1*, with identification of the function *S1 = f*(*Pgc*). The second scenario is dedicated to the evaluation of the required capacity of the power conversion system for voltage condition adjustment at nodes. The operation system is presented in Figure 6a. The volume of data collected for *S1* at the given *P<sup>l</sup>* depends on the grid PV capacity (Table 5). Figures 10–12 present the active power values *P* for *S1*, with identification of the function *S1= f(Pgc).*

**Figure 10.** *S1*, p.f. = 1: (**a**) All figures for *S1*; (**b**) SR; (**c**) LS; (**d**) RD. **Figure 10.** *S1*, p.f. = 1: (**a**) All figures for *S1*; (**b**) SR; (**c**) LS; (**d**) RD.

In accordance with the obtained results (Table 5), only three load profiles are subject to power input adjustment. Adjusted R-square (R2) for SR = 1 (small data volume), R<sup>2</sup> for

(**a**) (**b**)

LS = 0.9997, R<sup>2</sup> for RD = 0.999.

In accordance with the obtained results (Table 5), only three load profiles are subject to power input adjustment. Adjusted R-square (R<sup>2</sup> ) for SR = 1 (small data volume), R<sup>2</sup> for LS = 0.9997, R<sup>2</sup> for RD = 0.999. (**c**) (**d**) **Figure 10.** *S1*, p.f. = 1: (**a**) All figures for *S1*; (**b**) SR; (**c**) LS; (**d**) RD.

*Energies* **2021**, *14*, x FOR PEER REVIEW 10 of 21

(**a**) (**b**)

with identification of the function *S1= f(Pgc).*

The second scenario is dedicated to the evaluation of the required capacity of the power conversion system for voltage condition adjustment at nodes. The operation system is presented in Figure 6a. The volume of data collected for *S1* at the given *P<sup>l</sup>* depends on the grid PV capacity (Table 5). Figures 10–12 present the active power values *P* for *S1*,

In accordance with the obtained results (Table 5), four load profiles are subject to power input adjustment. R<sup>2</sup> for SR = 0.9993, R<sup>2</sup> for PS = 1, R<sup>2</sup> for LS = 0.9998, R<sup>2</sup> for RD = 0.9996. In accordance with the obtained results (Table 5), only three load profiles are subject to power input adjustment. Adjusted R-square (R<sup>2</sup> ) for SR = 1 (small data volume), R<sup>2</sup> for LS = 0.9997, R<sup>2</sup> for RD = 0.999.

**Figure 11.** *S1*, p.f. = 0.95: (**a**) All figures for *S1*; (**b**) SR; (**c**) PS; (**d**) LS; (**e**) RD. **Figure 11.** *S1*, p.f. = 0.95: (**a**) All figures for *S1*; (**b**) SR; (**c**) PS; (**d**) LS; (**e**) RD.

In accordance with the obtained results (Table 5), four load profiles are subject to

for PS = 1, R<sup>2</sup>

for LS = 0.9998, R<sup>2</sup>

for RD =

for SR = 0.9993, R<sup>2</sup>

(**a**) (**b**)

power input adjustment. R<sup>2</sup>

0.9996.

(**e**)

0.9996.

power input adjustment. R<sup>2</sup>

(**c**) (**d**)

**Figure 11.** *S1*, p.f. = 0.95: (**a**) All figures for *S1*; (**b**) SR; (**c**) PS; (**d**) LS; (**e**) RD.

for SR = 0.9993, R<sup>2</sup>

In accordance with the obtained results (Table 5), four load profiles are subject to

for PS = 1, R<sup>2</sup>

for LS = 0.9998, R<sup>2</sup>

for RD =

**Figure 12.** *S1*, p.f. = 0.9: (**a**) All figures for *S1*; (**b**) SR; (**c**) PS; (**d**) LS; (**e**) RD. **Figure 12.** *S1*, p.f. = 0.9: (**a**) All figures for *S1*; (**b**) SR; (**c**) PS; (**d**) LS; (**e**) RD.

In accordance with the obtained results (Table 5), four load profiles are subject to power input adjustment. R<sup>2</sup> for SR = 0.998, R<sup>2</sup> for PS = 0.9996, R<sup>2</sup> for LS = 0.9997, R<sup>2</sup> for RD = 0.9997. The application of only *S1* for operation adjustment only is to maintain the voltage conditions with active power from 2.5 kW to 201.5 kW. In accordance with the obtained results (Table 5), four load profiles are subject to power input adjustment. R<sup>2</sup> for SR = 0.998, R<sup>2</sup> for PS = 0.9996, R<sup>2</sup> for LS = 0.9997, R<sup>2</sup> for RD = 0.9997. The application of only *S1* for operation adjustment only is to maintain the voltage conditions with active power from 2.5 kW to 201.5 kW.

#### *3.3. The Third Scenario 3.3. The Third Scenario*

The third scenario is dedicated to the evaluation of the required capacity for two power conversion systems for voltage condition adjustment at nodes, independent of *F1* and *F2*. The operation system is presented in Figure 6b. Two ESS systems: *S1* and *S2* are The third scenario is dedicated to the evaluation of the required capacity for two power conversion systems for voltage condition adjustment at nodes, independent of *F1* and *F2*. The operation system is presented in Figure 6b. Two ESS systems: *S1* and *S2* are

connected in the depth of the LV grid. For *F1*, it is the node No. *S2*\_10\_2 (Figure *2*); for *F2* it is the node No. *S1*\_7 (Figure 2). The volume of data collected for *S1* at the given load

> - -

**5.75 6.0 6.25 6.5 6.75 7.0 7.25 7.5 7.75 8.0**


0 0.5

0 2.5



results are presented in tables (Tables 6–8).

**Table 6.** Results for *S1* and *S2*, p.f. = 1.

**Load profile/Pgc (kW)**

> S1 (kW) S2 (kW)

SR

LS

connected in the depth of the LV grid. For *F1*, it is the node No. *S2*\_10\_2 (Figure 2); for *F2* it is the node No. *S1*\_7 (Figure 2). The volume of data collected for *S1* at the given load profile depends on the grid PV capacity (Table 5). Due to the collected data volume, the results are presented in tables (Tables 6–8).

**Table 6.** Results for *S1* and *S2*, p.f. = 1.


R 2 for SR S2 = 1 (small data volume), R<sup>2</sup> for LS S2 = 0.9982; R<sup>2</sup> for RD S1 = 0.9997; RD S2 = 0.9994. On average, approximately 40% smaller active power is required to control the LV grid if two, not one, ESS systems are used. In extreme cases (*Pgc* = 8 kW), even approximately 90% smaller.

**Table 7.** Results for S1 and S2, p.f. = 0.95.


R 2 for SR S1 = 0.9991, S2 = 0.9967; R<sup>2</sup> for LS S1 = 0.9995, S2= 0.9994; R<sup>2</sup> for PS S1 = 0.9968, S2 = 0.9762; R<sup>2</sup> for RD S1 = 0.9995, RD S2 = 0.9972.



R 2 for SR S1 = 0.9989, S2 = 0.986; R<sup>2</sup> for LS S1 = 0.9994, S2= 0.9988; R<sup>2</sup> for PS S1 = 0.9986, S2 = 0.9989; R<sup>2</sup> for RD S1 = 0.9985, RD S2 = 0.9985.

#### **4. Discussion**

The collected simulation data clearly indicate that the consumer load profile with simultaneous RES generation has a direct impact on the grid PV capacity (Table 5). Based

on the completed mathematical power flow simulation in the LV grid, in different research scenarios, it should be ascertained that the control (maintenance of the voltage value of 1.1 *Un*) and efficient balance optimisation (Figure 5) of the grid are possible. The research indicates that there is no single method of flexibility service. Proper selection of the method depends on the effective grid regulation and profitability (capex and/or opex) and the day of the week (confirmed hypothesis—Section 2.1). These above-mentioned items will build a method rating for a given area of the grid.

Table 9 lists flexibility service proposals according to the load profile type (Table 4). It also represents a method rating using a colour and mark (+, −, +/−). Green is used to mark the types of service that provides technical efficiency at the lowest cost per service. Yellow is used to mark the services that do not provide the full range of LV grid adjustment and/or that may have a high cost per service. Red is used to mark the services that do not provide LV grid adjustment and/or are too expensive [47,48].


**Table 9.** Proposed flexibility service rating.

In addition, the research indicates a quasi-linear (laboratory confirmation required) relationship between the P and jQ of RES generation, and the P and jQ of the power conversion system *S1* and/or *S2*. There is an error connected with non-linearity, measured by the indicator of function alignment with measurement data, adjusted R-square statistic, which is close to 1 (descriptions under Tables 6–8). The error results only from the precision of ESS control by power increase in 0.25 kW increments [49]. The simplest and cheapest service method is the maintenance of the p.f. close to 1 by RES or reactive power consumption by the inverter [50,51]. The developed results (data) will be used in the future to build machine learning for the purposes of power and voltage regulation in the LV grid [52–54].

#### **5. Conclusions**

There is still need for research works on bridging the gap between power generation based on end consumer supplied from central generation units and power generation based on local RES. The assumed research hypothesis (section research problem) was confirmed by the results obtained in dedicated scenarios (Tables 5–8, Figures 7–12). The basic task of the future DSO is to prepare the bus LV grid—the minimum section of 70 mm2—for the mega-process that is RES growth in the grid. In addition, the accumulation of knowledge of the load profile and generation is necessary for the development of grid PV capacity, as well as future strategies of dedicated services.

Proper selection of the MV/LV transformer with ESS remains an extremely important issue. Even constant PV generation at the level of 3.5 kW in the investigated LV grid resulted in the change of the power flow direction. The services proposed in Table 9 directly contribute to the transformer load balance, but full coverage of the power flow towards MV will only be possible in connection with ESS.

**Author Contributions:** Oprogramowanie Naukowo-Techniczne sp. z o.o. sp. k, Software Vendor of Matlab in Poland in the scope of technical support in the optimisation of non-linear functions. Conceptualization, B.M. and P.P.; methodology, B.M.; software, B.M. and P.P.; validation, B.M.; formal analysis, B.M.; investigation, B.M. and P.P.; resources, B.M.; data curation, B.M.; writing—original draft preparation, B.M.; writing—review and editing, B.M. and P.P.; visualization, B.M.; supervision, P.P.; project administration, B.M.; funding acquisition, B.M. and P.P. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** Not applicable.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


## *Article* **Distributed Generation Based Virtual STATCOM Configuration and Control Method**

**Su-Han Pyo <sup>1</sup> , Tae-Hun Kim <sup>1</sup> , Byeong-Hyeon An <sup>1</sup> , Jae-Deok Park <sup>1</sup> , Jang-Hyun Park <sup>1</sup> , Myoung-Jin Lee <sup>2</sup> and Tae-Sik Park 1,\***


**Abstract:** Recently, because of the increase in the number of connections to Distributed Generation (DG), the problem of lowering voltage stability in the distribution system has become an issue. Reactive power compensators, such as Static Synchronous Compensators (STATCOM), may be used to solve the problem of voltage stability degradation. However, because of the complexity of the distribution system, it is very difficult to select the installation location for STATCOM. Furthermore, when installed in the wrong location, economical efficiency and availability problems may occur. This paper proposes a Virtual STATCOM Configuration and Control method that would operate like a single STATCOM based on multiple DGs connected to the system. The proposed Virtual STATCOM has the merit of being economical by using existing facilities without adding new power facilities, and it solves the problem of the difficulty of selecting the installation location because of the complexity of the distribution system. In addition, while the conventional STATCOM uses an independent control method in consideration of the power quality of the access point, the Virtual STATCOM performs the Point of Common Coupling (PCC) power quality compensation using the integrated control of multiple DGs connected to the system. In the proposed method, the Virtual STATCOM integrated control algorithm is configured by adopting linear programming, and the compensation is performed while considering the distance between DG and PCC, the inverter's rated capacity, and the power generation. The performance of the Virtual STATCOM power quality compensation was verified using MATLAB/SIMULINK and Real Time Simulator (OPAL-RT).

**Keywords:** STATCOM; reactive power compensation; power quality

## **1. Introduction**

In the existing power system, electricity generated from large-scale power sources such as nuclear power generation and thermal power generation is supplied to loads tens or hundreds of kilometers away through long-distance high-voltage transmission lines. For about 50 to 60 years, this has been operating well without changing the configuration of the power system, providing inexpensive and stable electric power to industrial, commercial, and residential loads. However, existing large-scale power generation sources emit a lot of carbon, causing problems such as global warming. To solve these problems, the configuration of the power system is changing for de-carbonization worldwide [1]. As small-scale DGs such as wind power and solar power are rapidly increasing on the distribution system level, the configuration of the distribution system is changing to become very different from the existing ones [2–5]. However, there may be problems in that the power supply is not constant because of the increase in the connection of intermittent power sources to the distribution system, and the degradation of voltage stability caused by the Ferranti Effect occurs as the voltage of the distribution system increases [6].

**Citation:** Pyo, S.-H.; Kim, T.-H.; An, B.-H.; Park, J.-D.; Park, J.-H.; Lee, M.-J.; Park, T.-S. Distributed Generation Based Virtual STATCOM Configuration and Control Method. *Energies* **2022**, *15*, 1762. https:// doi.org/10.3390/en15051762

Academic Editor: Mario Marchesoni

Received: 24 January 2022 Accepted: 24 February 2022 Published: 26 February 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

Many studies have been conducted on the Flexible AC Transmission System (FACTS) to solve the following problems [7]. STATCOM, as one kind of FACTS, is a facility that is connected to the power system in parallel to compensate for the power quality of the connected point. STATCOM is mainly installed in transmission systems and substations with large capacities to compensate for the power quality using the reactive power compensation for the power system [8–11]. Depending on the STATCOM installation location, the power quality compensation performance has large deviations. When the optimal installation location is selected, the active power loss may be minimized, and the voltage stability of the power system may be improved [12,13]. The DG inverter is composed of a smart inverter, so it is possible to control active and reactive power. This means that the DG connected to the system can be controlled in an integrated way [14–17]. When a PV solar farm is used as a STATCOM, it has been proven to increase the grid interconnection of the wind power plants connected to the surrounding area and to improve the power transmission capacity [18–20]. Power oscillation damping (POD) can be performed using a PV power plant as a STATCOM. As soon as power oscillations caused by a system disturbance are detected, the solar farm discontinues its real power generation function very briefly and makes its entire inverter capacity available to operate as a STATCOM for POD [20–24].

STATCOM installed in the transmission system cannot compensate for the power quality considering the distribution system. When installing STATCOM to compensate for the power quality of the distribution system, it is difficult to select the installation location because of the complexity of the distribution system. Furthermore, when installing a large number of STATCOMs, the investment cost is high, resulting in economic degradation. The installation location can be selected using an optimization algorithm [25]. However, as the number of DGs increases, the complexity of the distribution system increases and the optimal installation location may be changed. To solve the above-mentioned problems, this paper proposes a solution to the problem of selecting the optimal installation location by configuring a Virtual STATCOM that operates like a single STATCOM based on DGs.

#### **2. Overview of STATCOM Power Quality Compensation**

The active power of the power system affects frequency fluctuations, and the reactive power affects the voltage fluctuations [26]. To improve the power quality, voltage, frequency, and waveform should be kept constant. It is very important to keep the voltage constant. When the supply and demand balance of reactive power is not achieved, voltage fluctuations occur. A generator is used to supply the active power to the power system. However, facilities such as a synchronous generator, a passive filter, and an active filter may be used to supply the reactive power.

The frequency and voltage of the power system are kept constant using the active power—frequency (*P* − *f*(δ)) control by the generator and the reactive power—voltage (*Q* − *V*) control by the reactive power compensator. When the load voltage *V<sup>L</sup>* and the line reactance component *X* are constant, the voltage fluctuation ∆*V* has a proportional relationship with the reactive power *Q* [27]. Where *V<sup>s</sup>* = source voltage:

$$
\Delta V = V\_s - V\_L = \frac{QX}{V\_L} \left( R \ll X \right) \tag{1}
$$

Equation (1) shows the relationship between the voltage fluctuation ∆*V* and the reactive power *Q*. It is found that the voltage fluctuation can be controlled by controlling the reactive power from STATCOM.

#### *Principle of STATCOM Voltage Stability Improvement*

Figure 1 is an equivalent circuit diagram of a system in which STATCOM is installed to explain the principle of voltage stability improvement. *V<sup>S</sup>* is the equivalent of the bus voltage, *Y<sup>L</sup>* is the equivalent of the admittance, and *Z<sup>S</sup>* is the equivalent of the line impedance generated between the bus voltage *V<sup>S</sup>* and the load. STATCOM is connected to the grid in parallel to improve the voltage stability by the amount of the reactive power

compensated for when the load-end voltage stability is improved. The voltage drop between the load terminal and the bus voltage is caused by the line impedance *ZS*. When the voltage drops because of load admittance *Y<sup>L</sup>* and the line impedance *Z<sup>S</sup>* is ∆*V*, it can be expressed by the following equation [28]: grid in parallel to improve the voltage stability by the amount of the reactive power compensated for when the load-end voltage stability is improved. The voltage drop between the load terminal and the bus voltage is caused by the line impedance . When the voltage drops because of load admittance and the line impedance is ∆, it can be expressed by the following equation [28]:

Figure 1 is an equivalent circuit diagram of a system in which STATCOM is installed to explain the principle of voltage stability improvement. is the equivalent of the bus voltage, is the equivalent of the admittance, and is the equivalent of the line impedance generated between the bus voltage and the load. STATCOM is connected to the

*Energies* **2022**, *15*, 1762 3 of 17

*Principle of STATCOM Voltage Stability Improvement*

$$
\Delta V = V\_S - V\_L = Z\_S I\_L \tag{2}
$$

$$\begin{split} \Delta V &= (R\_S + jX\_S) \frac{(P\_L - jQ\_L)}{V\_L} \\ &= \frac{(R\_S P\_L + X\_S Q\_L)}{V\_L} + j \frac{(X\_S P\_L - R\_S Q\_L)}{V\_L} \\ &= \Delta V\_R + \Delta V\_X = I\_S R\_S + jI\_S X\_S \end{split} \tag{3}$$

**Figure 1.** Principle of STATCOM voltage stability improvement. **Figure 1.** Principle of STATCOM voltage stability improvement.

Figure 2 is a vector diagram of voltage drop ∆ caused by the line impedance . Depending on the power system situation, STATCOM operates in the form of supplying reactive power (Capacitive Reactive Power) or absorbing the reactive power (Inductive Reactive Power), or in the form of supplying the active power or absorbing the active power. When a voltage drop occurs at the receiving end because of the inductance of the line impedance , the reactive power compensation current ℎ is supplied for compensating the voltage drop of the system. A voltage drop is generated by resistance component and reactance component , and before compensation the load current angle is . and can be kept the same through ℎ current compensation, and the load current angle is compensated to . The reactive power compensation current ℎ can be expressed as Equation (4) [29]: Figure 2 is a vector diagram of voltage drop ∆*V* caused by the line impedance *ZS*. Depending on the power system situation, STATCOM operates in the form of supplying reactive power (Capacitive Reactive Power) or absorbing the reactive power (Inductive Reactive Power), or in the form of supplying the active power or absorbing the active power. When a voltage drop occurs at the receiving end because of the inductance of the line impedance *ZS*, the reactive power compensation current *Ish* is supplied for compensating the voltage drop of the system. A voltage drop is generated by resistance component *IsR<sup>s</sup>* and reactance component *jIsX<sup>s</sup>* , and before compensation the load current angle is *θL*. *V<sup>S</sup>* and *V<sup>L</sup>* can be kept the same through *Ish* current compensation, and the load current angle is compensated to *θS*. The reactive power compensation current *Ish* can be expressed as Equation (4) [29]:

$$I\_{\rm S} = I\_{\rm L} + I\_{\rm sh} \tag{4}$$

 = + ℎ (4) When the loads is an inductive load, STATCOM can make a capacitive current and scale it to change the phase angle of . By adjusting ∆ according to the compensation current of STATCOM, the voltage of the load stage and the magnitude of the bus voltage When the loads is an inductive load, STATCOM can make a capacitive current and scale it to change the phase angle of *IS*. By adjusting ∆*V* according to the compensation current of STATCOM, the voltage of the load stage and the magnitude of the bus voltage can be kept constant.

can be kept constant.

**ISh**

*Energies* **2022**, *15*, 1762 4 of 17

**Figure 2.** Vector Diagram of STATCOM Power Quality Compensation. **Figure 2.** Vector Diagram of STATCOM Power Quality Compensation. **3. Proposed Virtual STATCOM System**

#### **3. Proposed Virtual STATCOM System 3. Proposed Virtual STATCOM System** DGs, such as solar power and wind power ESS, are connected to the power system

DGs, such as solar power and wind power ESS, are connected to the power system via inverters, and a number of inverters connected to the DG are capable of outputting active power and reactive power. However, distributed power sources currently connected to the grid operate as the power sources and supply only active power. The reactive power generated in the power system is not compensated for. Since each inverter is individually and independently controlled, it is difficult to effectively compensate for the reactive power of the power system. The proposed Virtual STATCOM system can effectively control the reactive power of the power system with an integrated control rather than the independent control of multiple inverters of distributed power sources connected to the power system. Since no new facilities are installed for compensating for the reactive power, the installation cost can be economically feasible. Furthermore, compensating for the reactive power of the PCC stage to 0 can be expected to improve voltage stability, to remove harmonics, to increase transmission capacity, and to enhance the lifespan of facilities. Figure 3 shows the configuration schematic of Virtual STATCOM. DGs, such as solar power and wind power ESS, are connected to the power system via inverters, and a number of inverters connected to the DG are capable of outputting active power and reactive power. However, distributed power sources currently connected to the grid operate as the power sources and supply only active power. The reactive power generated in the power system is not compensated for. Since each inverter is individually and independently controlled, it is difficult to effectively compensate for the reactive power of the power system. The proposed Virtual STATCOM system can effectively control the reactive power of the power system with an integrated control rather than the independent control of multiple inverters of distributed power sources connected to the power system. Since no new facilities are installed for compensating for the reactive power, the installation cost can be economically feasible. Furthermore, compensating for the reactive power of the PCC stage to 0 can be expected to improve voltage stability, to remove harmonics, to increase transmission capacity, and to enhance the lifespan of facilities. Figure 3 shows the configuration schematic of Virtual STATCOM. via inverters, and a number of inverters connected to the DG are capable of outputting active power and reactive power. However, distributed power sources currently connected to the grid operate as the power sources and supply only active power. The reactive power generated in the power system is not compensated for. Since each inverter is individually and independently controlled, it is difficult to effectively compensate for the reactive power of the power system. The proposed Virtual STATCOM system can effectively control the reactive power of the power system with an integrated control rather than the independent control of multiple inverters of distributed power sources connected to the power system. Since no new facilities are installed for compensating for the reactive power, the installation cost can be economically feasible. Furthermore, compensating for the reactive power of the PCC stage to 0 can be expected to improve voltage stability, to remove harmonics, to increase transmission capacity, and to enhance the lifespan of facilities. Figure 3 shows the configuration schematic of Virtual STATCOM.

**Figure 3.** Virtual STATCOM configuration schematic. **Figure 3.** Virtual STATCOM configuration schematic.

#### *3.1. Configuration of Virtual STATCOM*

*Energies* **2022**, *15*, 1762 5 of 17

In general, STATCOM is a facility that improves the stability of the power system by controlling the reactive power of the connection point. When multiple DG inverters connected to the system independently compensate for reactive power at the connection point, like in the case of a general STATCOM, the system stability may deteriorate because of the hunting that occurs during the control operation. The purpose of this paper is to compensate for or control the reactive power at a designated location in the system by controlling the inverters of DGs connected to multiple connection points. That is, it is expected that the proposed Virtual STATCOM can improve the stability of the entire power system by controlling reactive power at a desired point, such as a weak or important location of the connection or distribution end of the transmission and distribution system, like the PCC of the power system. In this paper, PCC was designated as the control reference point that is the connection point of transmission and distribution, and the Virtual STATCOM integrated control algorithm was configured by the linear programming. To improve the voltage stability of the PCC, the reactive power of the PCC should be controlled to 0. The reactive power of the PCC stage is divided into a component generated by the line impedance and a component generated by the load. To compensate for the reactive power of the PCC stage to 0, it is necessary to compensate for the reactive power while considering both causes of the reactive power. *3.1. Configuration of Virtual STATCOM* In general, STATCOM is a facility that improves the stability of the power system by controlling the reactive power of the connection point. When multiple DG inverters connected to the system independently compensate for reactive power at the connection point, like in the case of a general STATCOM, the system stability may deteriorate because of the hunting that occurs during the control operation. The purpose of this paper is to compensate for or control the reactive power at a designated location in the system by controlling the inverters of DGs connected to multiple connection points. That is, it is expected that the proposed Virtual STATCOM can improve the stability of the entire power system by controlling reactive power at a desired point, such as a weak or important location of the connection or distribution end of the transmission and distribution system, like the PCC of the power system. In this paper, PCC was designated as the control reference point that is the connection point of transmission and distribution, and the Virtual STATCOM integrated control algorithm was configured by the linear programming. To improve the voltage stability of the PCC, the reactive power of the PCC should be controlled to 0. The reactive power of the PCC stage is divided into a component generated by the line impedance and a component generated by the load. To compensate for the reactive power of the PCC stage to 0, it is necessary to compensate for the reactive power while considering both causes of the reactive power.

Figure 4 shows the configuration of the Virtual STATCOM system. Three solar power plants are connected between the voltage source and load. It is assumed that each solar power plant has a different distance from the bus of the PCC stage, and the amounts of connected load, solar power plant capacities, and power generation amounts are different from each other. The inverter of the solar power plant can output the active power and the reactive power as much as the rated capacity *S*. When there is room in the inverter output rated capacity after supplying the active power *P* generated by the PV panel to the grid, the reactive power can be supplied. The inverter rated capacity *S* of a solar power plant can be expressed as Equation (5): Figure 4 shows the configuration of the Virtual STATCOM system. Three solar power plants are connected between the voltage source and load. It is assumed that each solar power plant has a different distance from the bus of the PCC stage, and the amounts of connected load, solar power plant capacities, and power generation amounts are different from each other. The inverter of the solar power plant can output the active power and the reactive power as much as the rated capacity *S*. When there is room in the inverter output rated capacity after supplying the active power *P* generated by the PV panel to the grid, the reactive power can be supplied. The inverter rated capacity *S* of a solar power plant can be expressed as Equation (5):

$$\mathcal{S}\_{\text{inv\\_n}} = \sqrt{\mathcal{P}\_{\text{inv\\_n}}^2 + \mathcal{Q}\_{\text{inv\\_n}}^2} \tag{5}$$

**Figure 4.** Configuration Scheme of the Virtual STATCOM System. **Figure 4.** Configuration Scheme of the Virtual STATCOM System.

The Virtual STATCOM integrated control algorithm selects the inverter closest to the PCC to give it priority, and when the amount of the reactive power that can be output is insufficient, the compensation will be performed from the inverter as the next priority. The Virtual STATCOM integrated control algorithm selects the inverter closest to the PCC to give it priority, and when the amount of the reactive power that can be output is insufficient, the compensation will be performed from the inverter as the next priority.

Figure 5 shows the configuration of the integrated control algorithm of Virtual STAT-COM. By combining Solar Station, PMU, and Distance Information, the Virtual STATCOM Control algorithm is configured based on LP, and the control reference of the inverter is derived. A plurality of inverters improves the voltage stability by the reactive power compensation of the PCC stage. Figure 5 shows the configuration of the integrated control algorithm of Virtual STAT-COM. By combining Solar Station, PMU, and Distance Information, the Virtual STAT-COM Control algorithm is configured based on LP, and the control reference of the inverter is derived. A plurality of inverters improves the voltage stability by the reactive power compensation of the PCC stage.

**Figure 5.** Configuration of Virtual STATCOM Integrated Control Algorithm. **Figure 5.** Configuration of Virtual STATCOM Integrated Control Algorithm.

#### *3.2. Scheme of Virtual STATCOM Control 3.2. Scheme of Virtual STATCOM Control*

To configure the integrated control algorithm of Virtual STATCOM based on LP, the decision variables, the objective functions, and the constraints must be selected according to the control purpose [30]. To configure the integrated control algorithm of Virtual STATCOM based on LP, the decision variables, the objective functions, and the constraints must be selected according to the control purpose [30].

#### 3.2.1. Decision Variable 3.2.1. Decision Variable

The decision variable is for setting the variable that is the control criterion of the LP. Since the purpose of the Virtual STATCOM is to output the inverter reactive power compensation reference, the inverter reactive power output amount was selected as a decision variable. The decision variable is for setting the variable that is the control criterion of theLP. Since the purpose of the Virtual STATCOM is to output the inverter reactive power compensation reference, the inverter reactive power output amount was selected as a decision variable.

> Decision Variable = *Qinv*:Inverter Reactive Power Output Decision Variable = *Qinv*: Inverter Reactive Power Output

#### 3.2.2. Objective Function 3.2.2. Objective Function

The objective function is for setting the goal of the decision variable. As the distance from the PCC terminal increases, the loss caused by the line impedance increases, so it is designated as the goal to use an inverter with a shorter distance. To select a minimum distance, a point where the product of the decision variable and the weight is minimized is set as the optimal point. The objective function is for setting the goal of the decision variable. As the distance from the PCC terminal increases, the loss caused by the line impedance increases, so it is designated as the goal to use an inverter with a shorter distance. To select a minimum distance, a point where the product of the decision variable and the weight *w<sup>n</sup>* is minimized is set as the optimal point.

$$\text{Objective Function} = \min(w\_1 \ast Q\_{inv1} + w\_2 \ast Q\_{inv2} + \dots + w\_n \ast Q\_{inv\_n})$$

where, refers to the distance weight of inverter, and refers to the line distance of inverter from the PCC end point. is decided by the following equation as a consideration of the distance between the inverter and the PCC end point. where, *w<sup>n</sup>* refers to the distance weight of inverter, and *l<sup>n</sup>* refers to the line distance of inverter from the PCC end point. *w<sup>n</sup>* is decided by the following equation as a consideration of the distance between the inverter and the PCC end point.

$$w\_n = \frac{l\_n}{l\_1 + l\_2 + l\_3 + \dots + l\_n} \tag{6}$$

#### 3.2.3. Constraints

The constraints determine the minimum compensation value and the maximum compensation value for the decision variable of the inverter to determine the reactive power that is to be compensated for by the inverter. The constrains for the minimum and maximum amounts of the inverter reactive power output can be expressed as Equation (7):

$$Q\_{inv\\_n\\_min} \le Q\_{inv\\_n} \le Q\_{inv\\_n\\_max} \tag{7}$$

where *Qinv*\_*n*\_*min* = Minimum of Inverter Reactive Power Compensation (var), and *Qinv*\_*n*\_*max* = Maximum of Inverter Reactive Power Compensation (var).

When the output of the inverter is maximized, the life of the inverter rapidly decreases, so the maximum output of the inverter is limited to 80%. To determine the amount of the reactive power compensation of the inverter, the minimum and maximum amounts of reactive power compensation of the inverter should be set as the constraints. Constraints are determined in consideration of the effective load amount of the inverter terminal, the reactive load amount of the inverter terminal, the rated capacity, the inverter active power generation amount, and the inverter generation rated limit.

*Qinv*\_*n*\_*max* means the maximum reactive power compensation amount that can be compensated for by the inverter. The maximum amount of inverter reactive power compensation is determined by considering the rated capacity of the inverter, the amount of active power currently being generated by the inverter, and the inverter power generation rating limit.

$$(S\_{inv\_n} \* 0.8) = \sqrt{P\_{inv\_n n}^2 + Q\_{inv\_n}^2} \tag{8}$$

$$Q\_{inv\\_n\\_max} = \sqrt{(S\_{inv\\_\*} \* 0.8)^2 - P\_{inv\\_\*}^2} \tag{9}$$

where *Qinv*\_*n*\_*min* means the minimum reactive power compensation amount that can be compensated by the inverter. The reactive power can be expressed in negative or positive number according to inductivity and capacity, so the negative number of the maximum amount of compensable reactive power is determined as the minimum compensable amount of the inverter.

$$Q\_{\text{inv\\_n\\_min}} = -Q\_{\text{inv\\_n\\_max}} \tag{10}$$

Considering the reactive power *QPCC* generated in the PCC stage, the reference value of the reactive power compensation for each inverter is calculated. The reactive power of PCC stage consists of the reactive power *QLine* generated by line impedance and the reactive power *QLoad* generated by load. When the total amount of reactive power generated in the PCC stage is equal to the total amount of reactive power compensated for by the inverter, the reactive power in the PCC stage is compensated to 0.

$$Q\_{\rm PCC} = Q\_{\rm Line} + Q\_{\rm load} \tag{11}$$

$$Q\_{\rm PCC} - \left(Q\_{\rm inv1} + Q\_{\rm inv2} + Q\_{\rm inv3} + \dots + Q\_{\rm inv\_n}\right) = 0\tag{12}$$

Table 1 shows the parameters for configuring the Virtual STATCOM system.


Figure 6 shows the Virtual STATCOM Integrated Control Algorithm Diagram. The compensation algorithm is configured through five steps. Step 1 measures the parameter values required for Virtual STATCOM configuration. Step 2 determines the reactive power compensation's possible range based on the measured parameter value. Step 3 determines the optimal reference *QLP* for reactive power compensation using LP. Step 4 performs reactive power compensation through an inverter and measures the reactive power of PCC. If the reactive power of PCC converges to 0, proceed to Step 5. If it does not converge, return to Step 1 and perform the algorithm again. Step 5 is the step of storing the reactive power compensation value determined by Steps 1–4. Equation (13) shows the reactive power compensation value of the algorithm diagram, where *QPCC*\_*n*\_*pre* is the stored PCC reactive power value of Step 5. Figure 6 shows the Virtual STATCOM Integrated Control Algorithm Diagram. The compensation algorithm is configured through five steps. Step 1 measures the parameter values required for Virtual STATCOM configuration. Step 2 determines the reactive power compensation's possible range based on the measured parameter value. Step 3 determines the optimal reference for reactive power compensation using LP. Step 4 performs reactive power compensation through an inverter and measures the reactive power of PCC. If the reactive power of PCC converges to 0, proceed to Step 5. If it does not converge, return to Step 1 and perform the algorithm again. Step 5 is the step of storing the reactive power compensation value determined by Steps 1–4. Equation (13) shows the reactive power compensation value of the algorithm diagram, where \_ \_ is the stored PCC reactive power value of Step 5.

*Energies* **2022**, *15*, 1762 8 of 17

Inverter Output Rating Limit (%) 80%

Inverter Compensation min (var) \_ = −\_

Inverter Compensation max (var) \_ <sup>=</sup> �� <sup>∗</sup> 0.8�

PCC Distance (km)

$$Q\_{\rm PCC} = Q\_{\rm PCC\\_n\\_pre} + \Delta Q\_{\rm PCC} \tag{13}$$

<sup>2</sup> <sup>−</sup> \_ <sup>2</sup>

If the reactive power compensation value is not saved and the algorithm is executed, only the value of ∆*QPCC* is compensated for when a load change occurs, and PCC reactive power is generated to as much as the *QPCC*\_*n*\_*pre* value. If the reactive power compensation value is not saved and the algorithm is executed, only the value of ∆ is compensated for when a load change occurs, and PCC reactive power is generated to as much as the \_ \_ value.

**Figure 6. Figure 6.**  Virtual STATCOM Integrated Control Algorithm Diagram. Virtual STATCOM Integrated Control Algorithm Diagram.

#### **4. Virtual STATCOM Simulation**

To verify the Virtual STATCOM's performance, simulation was conducted using Matlab Simulink and the Real Time Simulator (OPAL-RT) OP5700.

The Real Time Simulator is a simulation technology that proceeds by setting the simulation time to be same as the real time. By running the simulation under the same conditions as in reality, problems that do not appear in the off-line simulation can be

derived, and the reliability of the simulation can be improved. By establishing a virtual controller and a virtual plant in the target PC, a Model In the Loop Simulation (MILS) was performed. The plant of the simulation model was designated as Master, and the controller was designated as Slave, and tow cores were used. The result data of the simulation were output via the target PC. Figure 7 shows the Simulation Model of Virtual STATCOM. and the reliability of the simulation can be improved. By establishing a virtual controller and a virtual plant in the target PC, a Model In the Loop Simulation (MILS) was performed. The plant of the simulation model was designated as Master, and the controller was designated as Slave, and tow cores were used. The result data of the simulation were output via the target PC. Figure 7 shows the Simulation Model of Virtual STATCOM.

To verify the Virtual STATCOM's performance, simulation was conducted using

The Real Time Simulator is a simulation technology that proceeds by setting the simulation time to be same as the real time. By running the simulation under the same conditions as in reality, problems that do not appear in the off-line simulation can be derived,

*Energies* **2022**, *15*, 1762 9 of 17

Matlab Simulink and the Real Time Simulator (OPAL-RT) OP5700.

**4. Virtual STATCOM Simulation**

**Figure 7.** Virtual STATCOM Simulation Model. **Figure 7.** Virtual STATCOM Simulation Model.

The system model used in the simulation was that of Anseong, in South Korea. The distributed resources connected to the Anseong area, and the system diagram of the entire systems was made equivalent by integrating the system load into sections as shown in Figure 8. Table 2 shows the DG inverter and load data of the Anseong area system. For the simulation system model, the simulation was performed with equivalence to The system model used in the simulation was that of Anseong, in South Korea. The distributed resources connected to the Anseong area, and the system diagram of the entire systems was made equivalent by integrating the system load into sections as shown in Figure 8. Table 2 shows the DG inverter and load data of the Anseong area system. *Energies* **2022**, *15*, 1762 10 of 17

> For the simulation sample time, the simulation was conducted by dividing it into three categories: the power system, the inverter controller, and the algorithm according to

> > **Load Active Power (MW)**

**Load Reactive Power (Mvar)**

was set to 100 (s). The Virtual STATCOM integrated control algorithm sample time was

set to 0.1 (s) in consideration of the data measurement period of the PMU.

**Distance (km)**

 0.5 2 1 0.988 0.33 1.5 6 2 1.25 0.541 3 9 3 1.75 0.707 1 17 4 3.12 1.3 5 18 5 0.294 0.104 3 18 6 0.159 76.5 1 2 7 0.178 74.2 0.5 4 8 1.68 0.743 1.5 5 9 3.32 1.32 2 7 10 1.48 0.588 2 10 11 0.211 88.6

Table 3 shows the Virtual STATCOM simulation scenario. This table indicates whether the Virtual STATCOM Control Algorithm operates according to the simulation time, and also indicates the load change time. The data for the load change are shown in Table 4. For the simulation time of 0~1 (s), a situation, in which the Virtual STATCOM Control Algorithm does not operate and the load fluctuation does not occur, is simulated.

**Figure 8.** Virtual STATCOM Simulation Grid Diagram. **Figure 8.** Virtual STATCOM Simulation Grid Diagram.

**Table 2.** Virtual STATCOM Simulation Model Parameters.

**Inverter Rating (MVA)**


**Table 2.** Virtual STATCOM Simulation Model Parameters.

For the simulation system model, the simulation was performed with equivalence to a system including one voltage power source, 11 DGs, and 11 loads. The power source output AC of 22.9kV, 60Hz via the Three Phase Source. The load was simulated by dividing the section of the system using the three-phase parallel RLC load and making the load within the section equivalent to one load. The load between PCC and DG1 was set as *Load*1, and the subsequent loads were designated as *Load<sup>n</sup>* by grouping the loads between DGs. All loads used in the simulation were divided into 11 sections. To perform compensation considering the distance between the PCC terminal and the DGs, the line impedance according to the distance was made equivalent using the PI Section Line.

For the simulation sample time, the simulation was conducted by dividing it into three categories: the power system, the inverter controller, and the algorithm according to the purpose. The sample time of the power system was set to 50 (µs) for simulating the instantaneous system change situation, and the sample time for the inverter controller was set to 100 (µs). The Virtual STATCOM integrated control algorithm sample time was set to 0.1 (s) in consideration of the data measurement period of the PMU.

Table 3 shows the Virtual STATCOM simulation scenario. This table indicates whether the Virtual STATCOM Control Algorithm operates according to the simulation time, and also indicates the load change time. The data for the load change are shown in Table 4. For the simulation time of 0~1 (s), a situation, in which the Virtual STATCOM Control Algorithm does not operate and the load fluctuation does not occur, is simulated.

For the simulation time of 1 (s), the load variation situation was simulated. *Load*<sup>1</sup> generated a variation of +0.7 (Mvar) in 1 (s), *Load*<sup>3</sup> generated +0.3 (Mvar), *Load*<sup>4</sup> generated −0.25 (Mvar), *Load*<sup>5</sup> generated +1.1 (Mvar), *Load*<sup>8</sup> generated +2.1 (Mvar), *Load*<sup>9</sup> generated +0.2 (Mvar), and *Load*<sup>10</sup> generated the variation of −0.15 (Mvar), so the reactive power of PCC generated the variation of +4 (Mvar). At the simulation time of 2 (s), *Load*<sup>1</sup> had the variation of −0.4 (Mvar), *Load*<sup>2</sup> had −0.2 (Mvar), *Load*<sup>3</sup> had +0.5 (Mvar), *Load*<sup>4</sup> had −0.3 (Mvar), *Load*<sup>5</sup> had −0.5 (Mvar), *Load*<sup>6</sup> had +1 (Mvar), *Load*<sup>7</sup> had +0.2 (Mvar), *Load*<sup>8</sup> had −2 (Mvar), and *Load*<sup>10</sup> had the variation of −0.3 (Mvar), so the reactive power of PCC having the variation of −2 (Mvar) was simulated.

**Table 3.** Virtual STATCOM Simulation Scenario.



**Table 4.** Virtual STATCOM Simulation Model Parameters (Load Variation).

Figure 9 shows the simulation results, assuming that power quality compensation using the Virtual STATCOM control algorithm is not conducted. During the simulation run time of 0~1 (s), the distributed power supply was connected to the system and the steady state operation was in progress. The voltage of PCC has a voltage drop of 22.87 kV caused by the inductance L and resistance R. In PCC reactive power, the ground reactive power of 5.8 (Mvar) is generated by line impedance components and non-linear loads. In 1~2 (s), it is found that the reactive power of +4 (Mvar) is increased by the load variation, and the voltage of PCC is 22.85 kV, so that the voltage drop is increased. In 2~3 (s), the occurrence of voltage transients and reactive power of −2 (Mvar) were reduced because of the increase of the capacitive load, and reactive power of 7.8 (Mvar) is generated at the PCC point, and a voltage drop of 22.86 kV occurs.

Figure 10 shows the power quality compensation simulation results using the Virtual STATCOM control algorithm. Virtual STATCOM was configured using inverters of multiple DGs connected to the system, and the optimum compensation command value of inverters was controlled using LP. Each inverter was controlled for the purpose of improving voltage stability by compensating for the reactive power of PCC to 0. Figure 10a,b show the PCC voltage and reactive power, and Figure 10c shows the amount of reactive power compensation of Virtual STATCOM inverters.

As the simulation time of 0~0.5 (s) is the status in which the Virtual STATCOM algorithm is not applied, the power quality is not compensated for, so the reactive power of the PCC stage is about 5.8 (Mvar). Furthermore, the voltage at the PCC terminal is 2.287 kV due to the line and load impedance components, and a voltage drop has occurs. It was found that when the Virtual STATCOM control algorithm operated at 0.5 (s), the reactive power of PCC stage converged to 0 by the reactive power compensation, and the voltage of the PCC stage was compensated to 22.9 kV. At 1 (s), as the reactive power of PCC increased with +4 (Mvar) by Load Variation, a voltage drop of 22.88 kV occurred, but the reactive power and the voltage of the PCC were compensated for to the reference values by the Virtual STATCOM integrated algorithm. For 1~2 (s), there was no change. At 2 (s), Load Variation occurred, and the capacitance reactive power of −2 (Mvar) occurred in PCC, so the voltage rose to 22.93 kV. However, it was found that the reactive power of the PCC stage was compensated to 0 and the voltage was sustained to 22.9 kV by the integrated control algorithm.

**Figure 9.** PCC Variation according to the Load Variation of Virtual STATCOM (Before Compensation by Algorithm) (**a**) PCC Voltage, (**b**) PCC Reactive Power. **Figure 9.** PCC Variation according to the Load Variation of Virtual STATCOM (Before Compensation by Algorithm) (**a**) PCC Voltage, (**b**) PCC Reactive Power.

Figure 10 shows the power quality compensation simulation results using the Virtual STATCOM control algorithm. Virtual STATCOM was configured using inverters of multiple DGs connected to the system, and the optimum compensation command value of inverters was controlled using LP. Each inverter was controlled for the purpose of improving voltage stability by compensating for the reactive power of PCC to 0. Figure 10a,b show the PCC voltage and reactive power, and Figure 10c shows the amount of reactive power compensation of Virtual STATCOM inverters. As the simulation time of 0~0.5 (s) is the status in which the Virtual STATCOM algorithm is not applied, the power quality is not compensated for, so the reactive power of the PCC stage is about 5.8 (Mvar). Furthermore, the voltage at the PCC terminal is 2.287kV due to the line and load impedance components, and a voltage drop has occurs. It was found that when the Virtual STATCOM control algorithm operated at 0.5 (s), the reactive Figure 11 shows the amount of reactive power compensation of Virtual STATCOM inverters 1–11. To control the reactive power of the PCC to 0 using the inverter integrated control, the reactive power compensation amount of the inverters was determined in consideration of the available reactive power amount of the inverter and the distance between the PCC and the inverter. Since inverters have different distances from the PCC, the loss caused by the line impedance may increase when the long-distance inverter compensates for a large amount. Therefore, compensation was carried out by selecting an inverter that is relatively close to the PCC as the compensation priority. In addition, the rated capacity is different for each connected inverter, and the amount of the reactive power that can be output is different because the amount of power currently being output is different. Therefore, by considering the rating and remaining capacity of each inverter, the optimum compensation command is generated with priority according to the maximum amount of reactive power and distance.

power of PCC stage converged to 0 by the reactive power compensation, and the voltage of the PCC stage was compensated to 22.9 kV. At 1 (s), as the reactive power of PCC increased with +4 (Mvar) by Load Variation, a voltage drop of 22.88 kV occurred, but the reactive power and the voltage of the PCC were compensated for to the reference values by the Virtual STATCOM integrated algorithm. For 1~2 (s), there was no change. At 2 (s), Load Variation occurred, and the capacitance reactive power of −2 (Mvar) occurred in PCC, so the voltage rose to 22.93 kV. However, it was found that the reactive power of the

grated control algorithm.

(**a**)

(**b**)

**Figure 10.** Virtual STATCOM Power Quality Compensation (**a**) PCC Voltage, (**b**) PCC Reactive Power, (**c**) Compensation Amount for Inverter Reactive Power. **Figure 10.** Virtual STATCOM Power Quality Compensation (**a**) PCC Voltage, (**b**) PCC Reactive Power, (**c**) Compensation Amount for Inverter Reactive Power.

Figure 11 shows the amount of reactive power compensation of Virtual STATCOM inverters 1–11. To control the reactive power of the PCC to 0 using the inverter integrated

active power and distance.

control, the reactive power compensation amount of the inverters was determined in consideration of the available reactive power amount of the inverter and the distance between the PCC and the inverter. Since inverters have different distances from the PCC, the loss caused by the line impedance may increase when the long-distance inverter compensates for a large amount. Therefore, compensation was carried out by selecting an inverter that is relatively close to the PCC as the compensation priority. In addition, the rated capacity is different for each connected inverter, and the amount of the reactive power that can be output is different because the amount of power currently being output is different. Therefore, by considering the rating and remaining capacity of each inverter, the optimum compensation command is generated with priority according to the maximum amount of re-

**Figure 11.** *Cont.*

**Figure 11.** 11 Reactive Power Compensation Amount of Virtual STATCOM Inverter 1~: (**a**) Inverter 1, (**b**) Inverter 2, (**c**) Inverter 3, (**d**) Inverter 4, (**e**) Inverter 5, (**f**) Inverter 6, (**g**) Inverter 7, (**h**) Inverter 8, (**i**) Inverter 9, (**j**) Inverter 10, (**k**) Inverter 11. **Figure 11.** 11 Reactive Power Compensation Amount of Virtual STATCOM Inverter 1: (**a**) Inverter 1, (**b**) Inverter 2, (**c**) Inverter 3, (**d**) Inverter 4, (**e**) Inverter 5, (**f**) Inverter 6, (**g**) Inverter 7, (**h**) Inverter 8, (**i**) Inverter 9, (**j**) Inverter 10, (**k**) Inverter 11.

In the time of 0~0.5 (s), the Virtual STATCOM control algorithm does not operate, so the Virtual STATCOM inverters are not outputting the reactive power. The control algorithm operates at 0.5 (s), and the reactive power compensation is carried out in consideration of the distance of the inverter to the maximum reactive power output. The reactive power compensation is carried out by dividing reactive power 5.8 (Mvar) generated in PCC by inverters 1, 2, 7, 8, 9, and 10. Inverter 1 compensated 0.4 (Mvar) reactive power, Inverter 2 compensated 1.35 (Mvar), Inverter 7 compensated 0.9 (Mvar), Inverter 8 compensated 0.45 (Mvar), Inverter 9 compensated 1.4 (Mvar), and Inverter 10 compensated 1.2 (Mvar). In Load Variation 1 (s), inverters 1, 2, 3, 7, 8, 9, 10 and 11 divided PCC reactive In the time of 0~0.5 (s), the Virtual STATCOM control algorithm does not operate, so the Virtual STATCOM inverters are not outputting the reactive power. The control algorithm operates at 0.5 (s), and the reactive power compensation is carried out in consideration of the distance of the inverter to the maximum reactive power output. The reactive power compensation is carried out by dividing reactive power 5.8 (Mvar) generated in PCC by inverters 1, 2, 7, 8, 9, and 10. Inverter 1 compensated 0.4 (Mvar) reactive power, Inverter 2 compensated 1.35 (Mvar), Inverter 7 compensated 0.9 (Mvar), Inverter 8 compensated 0.45 (Mvar), Inverter 9 compensated 1.4 (Mvar), and Inverter 10 compensated 1.2 (Mvar).

In Load Variation 1 (s), inverters 1, 2, 3, 7, 8, 9, 10 and 11 divided PCC reactive power of 9.8 (Mvar), so Inverter 1 compensated 0.4 (Mvar), Inverter 2 compensated 1.35 (Mvar), Inverter 3 compensated 2.5 (Mvar), Inverter 7 compensated 0.9 (Mvar), Inverter 8 compensated 0.45 (Mvar), Inverter 9 compensated 1.4 (Mvar), Inverter 10 compensated 1.8 (Mvar), and Inverter 11 compensated 1 (Mvar). In the case of a load variation of 2 (s), inverters 1, 2, 3, 7, 8, 9 and 10 divided PCC reactive power of 7.8 (Mvar), so Inverter 1 compensated 0.4 (Mvar), Inverter 2 compensated 1.35 (Mvar), Inverter 3 compensated 1.5 (Mvar), Inverter 7 compensated 0.9 (Mvar), Inverter 8 compensated 0.45 (Mvar), Inverter 9 compensated 1.4 (Mvar), and Inverter 10 compensated 1.8 (Mvar).

#### **5. Conclusions**

This paper proposed a Virtual STATCOM Configuration and Control method that operates like a single STATCOM based on multiple DGs connected to the system. The conventional STATCOM has the following demerits: as it is installed in the power transmission/transformation system, it is hard to operate while considering the power quality of the distribution system; when installing STATCOM in the distribution system, it is hard to select the installation location because of the complexity of the distribution system; and when proper positioning is not considered, the economic feasibility of STATCOM will be reduced. The Virtual STATCOM solves the problem of installation location selection with the configuration that includes a DG inverter connecting to the grid system, and enhances the economic feasibility by using the equipment connected to the existing system.

Virtual STATCOM is controlled using the operating principle of the existing STAT-COM. It improves voltage stability through reactive power compensation. Multiple DGs connected to the grid were integrated and controlled through an LP-based algorithm. As a result of the simulation, we confirmed that the voltage drop occurred because of the line impedance and load impedance components before compensation through the Virtual STATCOM integrated control algorithm. As a result of the compensation using the Virtual STATCOM algorithm, we confirmed that the voltage stability was improved through the reactive power compensation of the PCC. To verify the performance, a simulation was conducted based on a real-time simulator. The off-line simulation results and the real-time simulation results are the same, so we verified that there is no problem in an environment similar to the real one.

**Author Contributions:** Conceptualization, T.-S.P. and S.-H.P.; methodology, B.-H.A. and J.-D.P.; software, S.-H.P. and T.-H.K.; validation, J.-H.P. and M.-J.L.; formal analysis, S.-H.P.; writing—original draft preparation, S.-H.P.; review and editing, T.-S.P. and S.-H.P. All authors have read and agreed to the published version of the manuscript.

**Funding:** This work was supported by the Gwangju Jeonnam Local Energy Cluster Manpower Training of the Korea Institute of Energy Technology Evaluation and Planning (KETEP) grant funded by the Korean government's Ministry of Knowledge Economy (20214000000560).

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


## *Article* **Co-Optimizing Battery Storage for Energy Arbitrage and Frequency Regulation in Real-Time Markets Using Deep Reinforcement Learning**

**Yushen Miao <sup>1</sup> , Tianyi Chen <sup>1</sup> , Shengrong Bu 2,\*, Hao Liang <sup>3</sup> and Zhu Han <sup>4</sup>**


**Abstract:** Battery energy storage systems (BESSs) play a critical role in eliminating uncertainties associated with renewable energy generation, to maintain stability and improve flexibility of power networks. In this paper, a BESS is used to provide energy arbitrage (EA) and frequency regulation (FR) services simultaneously to maximize its total revenue within the physical constraints. The EA and FR actions are taken at different timescales. The multitimescale problem is formulated as two nested Markov decision process (MDP) submodels. The problem is a complex decision-making problem with enormous high-dimensional data and uncertainty (e.g., the price of the electricity). Therefore, a novel co-optimization scheme is proposed to handle the multitimescale problem, and also coordinate EA and FR services. A triplet deep deterministic policy gradient with exploration noise decay (TDD–ND) approach is used to obtain the optimal policy at each timescale. Simulations are conducted with real-time electricity prices and regulation signals data from the American PJM regulation market. The simulation results show that the proposed approach performs better than other studied policies in literature.

**Keywords:** battery energy storage; energy arbitrage; frequency regulation; real-time market; deep reinforcement learning

#### **1. Introduction**

With wider integration of renewable resources, energy storage has become a significant technology to help eliminate uncertainties associated with renewable energy generation, in order to maintain stability and improve flexibility of power networks. Among different kinds of energy storage technologies, battery energy storage systems (BESSs) have played an irreplaceable role in energy storage, grid synchronization, and other operation-assistance services [1,2] due to the following advantages: (1) BESSs can be flexibly configured depending on the power and energy requirements of system applications [3]; (2) BESSs have an instantaneous response nature [4,5]; (3) BESSs are not limited by external conditions such as geographical resources. Various research has been carried out related to battery energy storage systems planning and design for different applications. Optimized planning was proposed for a battery energy storage system considering battery degradation to reduce the operational costs of the nanogrid and microgrid [6]. In [7], the authors identified the optimal conditions for wireless charging of electric vehicles when they were in motion to reduce energy consumption. A model was presented for a residential energy management system to dispatch battery energy storage in a market-based setting [8]. A privacy-aware

**Citation:** Miao, Y.; Chen, T.; Bu, S.; Liang, H.; Han, Z. Co-Optimizing Battery Storage for Energy Arbitrage and Frequency Regulation in Real-Time Markets Using Deep Reinforcement Learning. *Energies* **2021**, *14*, 8365. https://doi.org/ 10.3390/en14248365

Academic Editors: Victor Becerra and Ahmed Rachid

Received: 29 October 2021 Accepted: 7 December 2021 Published: 11 December 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

framework was presented for utility-driven demand-side management with a realistic energy storage system model [9]. However, the economic viability of using BESSs to provide various services with a large scale is questionable due to their high investment costs [10].

One of the most discussed revenue sources for BESSs is to provide energy arbitrage (EA) services in a real-time electricity market by deliberately charging at low prices and discharging at higher prices to gain profit [11,12]. EA using BESSs was studied in [13], where the electricity price was assumed to be known before making storage decisions. More recent research took electricity price uncertainty into consideration, and thus many forecast methods were proposed to improve the quality of electricity price prediction, and a reinforcement learning method was proposed to maximize the profit of EA based on historical prices [14]. A stochastic dynamic programming method was used to optimize the BESS based on the forecast electricity price [15]. Neural networks were used to address the price prediction uncertainty by introducing a scenario-based optimal control framework [16]. Different models were presented in [17] to process the various price signals to optimize the price forecast.

In order to further increase the revenue of BESSs, some research work has considered a battery to provide EA and frequency regulation (FR) services simultaneously [18], since FR is a significant income source for energy storage [19–23]. For FR, BESSs are used to regulate the frequency of the power grid by charging or discharging based on the regulation signals sent by the power grid operator [5,19,24,25]. A comprehensive evaluation for stacked revenue by using the grid-connected BESS was introduced to provide EA and FR services [26]. A linear programming method was used to maximize the potential revenue of electrical energy storage from participation in EA and FR in the day-ahead market [27]. Co-optimizing EA and FR services simultaneously is considered a multitimescale problem, and a dynamic programming approach was proposed to solve the co-optimization problem [19,20]. These two existing works on co-optimizing EA and FR services assumed that the electricity prices, regulation signals, or their distributions were known in advance. However, the distributions or the values are hard to attain in the real-time market. Furthermore, these two works did not consider the degradation cost of the BESS, a key factor in energy operational planning, without which there might be aggressive charging or discharging of the BESS [4].

Deep reinforcement learning (DRL), combined with deep neural networks (DNNs) and reinforcement learning (RL) techniques, can be powerful tools for addressing BESSrelated decision-making problems using the trial-and-error mechanism [28,29]. Compared to model-based methods, such as MILP methods, DRL approaches have the following advantages: the ability to learn from historical data, to be self-adaptable, and to learn a good control policy even under a very complex environment [12]. A novel continuous DRL algorithm was used for energy management of the hybrid electric vehicles [30]. An expert-assistance deep deterministic policy gradient (DDPG) strategy was introduced to minimize the energy consumption and optimize the power allocation of the hybrid electric buses [31]. A multiphysics-constrained fast-charging strategy was proposed for lithium-ion batteries in [32] based on an environmental perceptive DDPG. However, DDPG is not effective in avoiding overestimation in the actor–critic setting [33,34].

To address the above issues, a novel co-optimization scheme considering the degradation of the battery cell in the BESS is proposed for the multitimescale problem of cooptimizing EA and FR services. A novel deep reinforcement learning (DRL) approach, a triplet deep deterministic policy gradient with exploration noise decay (TDD–ND), is proposed to handle the uncertainty of the real-time electricity prices and frequency regulation signals in the multitimescale co-optimization problem due to the following reasons: (1) TDD–ND does not rely on the knowledge of probability distributions; (2) TDD–ND can be used to solve the problem with continuous action space directly by using deterministic policy in an actor–critic algorithm [34–36]; (3) The TDD–ND algorithm takes the weighted action value of triplet critics, which overcomes estimation bias in the deep deterministic policy gradient (DDPG) algorithm and the twin delayed deep deterministic policy gradient

(TD3) algorithm [34]; (4) The TDD–ND algorithm adopts the exploration ND policy, which improves the exploration at the beginning of the training compared to DDPG and TD3. The main contributions of this paper are as follows:

1. A novel co-optimization scheme is proposed to handle the multitimescale problem. The BESS decides an optimal EA action every five minutes to maximize its revenue due to the total amount of energy change, and every two seconds the BESS decides an optimal FR action to maximize the total reward including the revenue due to energy change and FR settlement reward. Based on the FR action, the EA action has to be adjusted based on the power constraints of the BESS to maximize the total revenue of the day on the two-second level.


The rest of this paper is organized as follows. Section 2 explains the Pennsylvania New Jersey Maryland (PJM)'s frequency regulation market. Section 3 presents the nested system model used to formulate the co-optimizing problem. Our proposed TDD–ND approach is described in Section 4. The simulation results are discussed in Section 5. The conclusion is made in Section 6.

#### **2. PJM Frequency Regulation Market**

In the PJM frequency regulation market, generators and other devices (e.g., energy storage) can provide grid ancillary services in exchange for regulation credits [37]. PJM sends the regulation (RegD) signal to the resources wishing to provide regulation services every two seconds. Afterwards, PJM tracks the response from each resource and computes a performance score for each resource every two seconds based on the RegD signal and regulation response. For every five minutes, the market also calculates the average performance score within the five-minute period. The performance score is a weighted sum of correlation, delay, and precision [38,39]. A BESS typically has the nature of the instantaneous response and hence the scores of correlation and delay are close to 1. Therefore, the average performance score SC of a BESS within a five-minute period can be calculated based on the precision score as follows [19]:

$$\text{SC} = \frac{\sum\_{t=0}^{150\Delta t} \text{SC}\_t}{150},\tag{1}$$

$$\mathcal{SC}\_t = |1 - \left| \lambda\_t^F + rd\_t \right| / ar |\_{\prime} \tag{2}$$

where *λ F t* , *rd<sup>t</sup>* , *ar* are denoted as the regulation response power taken by BESS response to the RegD signal at time *t*, the RegD signal at time *t*, and the maximum power capacity assigned for FR, respectively. ∆*t* is set to two seconds because the RegD signal *rd<sup>t</sup>* is sent every two seconds. *SC<sup>t</sup>* denotes the two-second performance score. When the BESS is 100% following the RegD signal, *λ F <sup>t</sup>* + *rd<sup>t</sup>*  = 0 and *SC<sup>t</sup>* = 1.

Every five minutes, the PJM market determines the eligibility of the resource for regulation based on its average performance score *SC*, and calculates the amount of the regulation credit settlement received by the eligible resource. If the average performance score is less than 40%, the resource will lose its regulation qualification and regulation credits during that time period [37]. The five-minute regulation credit settlement *R <sup>C</sup>* can be calculated as follows [37]:

$$R^{\mathbb{C}} = \begin{cases} \mathbb{S}\mathbb{C} \cdot ar \cdot P^{\mathbb{C}}, & \mathbb{S}\mathbb{C} \ge 0.4, \\ 0, & \mathbb{S}\mathbb{C} < 0.4, \end{cases} \tag{3}$$

where *P <sup>C</sup>*, in \$/MW·5min, is the five-minute regulation clearance price for 1 MW regulation capacity.

#### **3. Nested System Model and Problem Formulation**

The system model, illustrated in Figure 1, consists of two main parts, i.e., the power grid and the BESS including a battery and an energy management system (EMS). The BESS participates in the energy and regulation market. The power grid sends the real-time electricity locational marginal price (LMP), FR signal, and FR market clearance price to the EMS in the BESS. The EMS then generates the operation signal to the battery to take action. At the same time, the battery sends feedback with its real-time status to the EMS. Based on the real-time status of the battery and the information from the power grid, the EMS generates a new operation signal to the battery.

**Figure 1.** The configuration of the system model.

The BESS co-optimizes EA and FR services to maximize its total reward within a oneday time horizon in a real-time PJM market: EA acts every five minutes, and FR responds every two seconds [19]. Due to the nature of the problem, the timescale is divided into two dimensions: a large timescale with five-minute intervals and a small timescale with two-second intervals, where two-second intervals are nested in the five-minute timescales. The two optimization problems are formulated as two nested MDP submodels in the two following subsections, respectively.

#### *3.1. The Five-Minute MDP Submodel Formulation*

The one-day horizon of five-minute submodel *T <sup>A</sup>*, decomposed into 288 five-minute increments (i.e., ∆*T* = 5 min) illustrated in Figure 2, is denoted as *T <sup>A</sup>* <sup>=</sup> {0, <sup>∆</sup>*T*, 2∆*T*, 3∆*T*, . . . <sup>287</sup>∆*T*}. The BESS takes a charging or discharging action every five minutes based on its current state to maximize the cumulative reward within the one-day horizon. The state, action, and reward are defined as follows.

**Figure 2.** The nested timescales in a day.

#### 3.1.1. State

The state of the BESS at time *T* can be defined as *S A <sup>T</sup>* = *ET*, *P A T* , where *E<sup>T</sup>* is the BESS energy level, and *P A T* is the real-time electricity locational marginal price (LMP) at time *T*.

#### 3.1.2. Action

The action in the five-minute submodel, denoted as *λT*, is the total amount of power change due to EA and FR at time *T* within the five-minute interval. *λ<sup>T</sup>* > 0 represents that the BESS is charging, while *λ<sup>T</sup>* < 0 implies that the BESS is discharging. The optimal action at time *T* is denoted as *λ* ∗ *T* . The action space should not exceed the maximum power capacity of BESS *B*:

$$|\lambda\_T| \le B.\tag{4}$$

The total amount of energy stored in the BESS at time *T* should be within its maximum energy capacity *E* max:

$$0 \le E\_T + \lambda\_T \cdot \Delta T \le E^{\max}.\tag{5}$$

After taking action *λT*, state *S A T* is converted to state *S A T*+∆*T* at time *T* + ∆*T*. The real-time price *P A T* is updated to *P A T*+∆*T* , and the energy level *E<sup>T</sup>* evolves to *ET*+∆*T*, which can be calculated as follows:

$$E\_{T+\Delta T} = \begin{cases} E\_T + \Delta T \cdot \lambda\_T \cdot \eta^c\_{\prime \prime} & \lambda\_T \ge 0, \\\ E\_T + \Delta T \cdot \lambda\_T / \eta^d\_{\prime \prime} & \lambda\_T < 0, \end{cases} \tag{6}$$

where *η <sup>c</sup>* and *η <sup>d</sup>* denote the charging and discharging efficiency, respectively.

#### 3.1.3. Degradation Cost and Reward

The degradation cost of the BESS is a key factor in energy operational planning [4] as the battery cells degrade for repeated charge/discharge cycles. The degradation cost of the BESS can be calculated as follows [4]:

$$f\_T(b) = c\_b |\lambda\_T| \cdot \Delta T\_\prime \tag{7}$$

where *c<sup>b</sup>* is the degradation cost coefficient, and can be calculated as follows [4]:

$$\mathcal{L}\_b = \frac{P\_{\text{cell}}}{2\text{N} \cdot (\text{SOC}\_{\text{max}} - \text{SOC}\_{\text{min}})} \prime \tag{8}$$

where *Pcell* is the price of the battery cell in the BESS and *N* is the number of cycles that the BESS could be operated within the state of charge (SoC) constraint [*SOC*min, *SOC*max].

After taking the action, the BESS will receive a reward. In order to avoid conservative actions caused by the negative reward in the learning process, an average electricity price *P*¯*<sup>A</sup>* is introduced in the reward *R A T S A T* , *λ<sup>T</sup>* for performing *λ<sup>T</sup>* action in state *S A T* based on the basic principle of EA [14] as follows:

$$R\_T^A \left( S\_{T'}^A \lambda\_T \right) = \left( \tilde{P}^A - P\_T^A \right) \cdot \lambda\_T \cdot \Delta T - f\_T(b). \tag{9}$$

#### *3.2. The Two-Second MDP Submodel Formulation*

The BESS needs to respond to the updated RegD signal every two seconds. Because twosecond intervals are nested in the five-minute horizon, the time horizon of one day within every two-second increment is denoted as *T <sup>F</sup>* <sup>=</sup> {0, <sup>∆</sup>*t*, 2∆*t*, 3∆*t*, . . .(<sup>150</sup> · <sup>288</sup> <sup>−</sup> <sup>1</sup>)∆*t*}, where ∆*t* = 2 s, shown in Figure 2. The BESS takes a charging or discharging action every two seconds based on its current state to maximize the cumulative reward within the one-day horizon. The state, action, and reward are defined as follows.

#### 3.2.1. State

The state at time *t* is denoted as *S F <sup>t</sup>* = (*E<sup>t</sup>* ,*rdt*), where *E<sup>t</sup>* is the energy level of the BESS and *rd<sup>t</sup>* is the received RegD signal at that time.

#### 3.2.2. Action

The action is the regulation response power, denoted as *λ F t* at time *t*, which is constrained by Equation (4). The action space also should not go beyond the maximum power capacity of BESS *B*. After performing an action *λ F t* at time *t*, state *S F <sup>t</sup>* will transfer to state *S F t*+∆*t* at time *t* + ∆*t*, and the energy level *E<sup>t</sup>* will be updated to *Et*+∆*<sup>t</sup>* based on *λ<sup>t</sup>* denoted as the total amount of power change at time *t* due to EA and FR:

$$E\_{t+\Delta t} = \begin{cases} E\_t + \Delta t \cdot \lambda\_t \cdot \eta^c\_{\prime \prime} & \lambda\_t \ge 0, \\\ E\_t + \Delta t \cdot \lambda\_t / \eta^d\_{\prime \prime} & \lambda\_t < 0. \end{cases} \tag{10}$$

#### 3.2.3. Reward

Based on the PJM market regulation policy, the reward *R<sup>t</sup> S F t* , *λ F t* by performing action *λ F t* at state *S F t* can be calculated as

$$R\_t\left(\mathcal{S}\_t^F, \lambda\_t^F\right) = R\_t^A + R\_t^F - f\_t(b),\tag{11}$$

where *ft*(*b*) = *c<sup>b</sup>* |*λt*| · ∆*t* according to Equation (7) and *R A t* is the reward due to the total amount of energy change caused by both EA and FR within the two-second interval:

$$R\_t^A = \left(\tilde{P}^A - P\_T^A\right) \cdot \lambda\_t \cdot \Delta t. \tag{12}$$

Instead of calculating the FR settlement at the end of every five minutes, in the twosecond submodel, we need to evaluate the FR reward every two seconds once choosing an action *λ<sup>t</sup>* . Based on Equation (3), *R F t* is the equivalent real-time regulation settlement reward within the two-second interval:

$$R\_t^F = \begin{cases} \text{SC}\_t \cdot B \cdot \left(P^C / 150\right), & \text{SC}\_t \ge 0.4, \\ 0, & \text{SC}\_t < 0.4, \end{cases} \tag{13}$$

assuming that the maximum power capacity *ar* assigned for FR is the power capacity of the BESS *B*.

#### *3.3. Proposed Co-Optimization Scheme*

Solving the co-optimizing problem for EA and FR is to find the optimal action selection policy for the BESS to obtain the maximum expected reward within a day. A co-optimization scheme is proposed to handle the multitimescale problem and coordinate

the EA and FR services, which is illustrated in Figure 3. Once *λ F t* is derived, *λ<sup>t</sup>* can be calculated as follows:

$$\lambda\_{t} = \begin{cases} \begin{array}{c} B + \lambda\_{t}^{F} \\ \lambda\_{T'}^{\*} \end{array} \Big| \begin{array}{c} \lambda\_{T}^{\*} - \lambda\_{t}^{F} > B, \\ \left| \lambda\_{T}^{\*} - \lambda\_{t}^{F} \right| \leq B, \\ -B + \lambda\_{t'}^{F} \quad \lambda\_{T}^{\*} - \lambda\_{t}^{F} < -B, \end{array} \end{cases} \tag{14}$$

where *λ<sup>t</sup>* is not always equal to *λ* ∗ *T* , due to the power constraint (Equation (4)) of the BESS. The first case shows that when the optimal action for FR *λ F t* is discharging (i.e., *λ F <sup>t</sup>* < 0) and the best action for EA is charging, the action for EA will be charging with the highest power capacity *B*. In this case, the charging value of *λ* ∗ *<sup>T</sup>* was set too high, and *λ<sup>t</sup>* is less than *λ* ∗ *T* . For the second case, the *λ<sup>t</sup>* is set to *λ* ∗ *T* . For the third case, when the optimal action for FR *λ F t* is charging and the best action for EA is discharging, the action for EA is discharging with the highest power capacity −*B*. In this case, the discharging value of *λ* ∗ *<sup>T</sup>* was set too low, and *λ<sup>t</sup>* is greater than *λ* ∗ *T* .

**Figure 3.** The TDD–ND approach for the proposed co-optimization scheme.

#### **4. Proposed Triplet Deep Deterministic Policy Gradient with Exploration Noise Decay Approach**

A novel DRL approach, combining TDD [34] and ND, is proposed to address the co-optimization problem. TDD–ND is a model-free, off-policy actor–critic algorithm, in which the triplet critics are used to limit estimation bias, and the exploration ND policy is used to improve the exploration in the algorithm.

#### *4.1. Triplet Deep Deterministic Policy Gradient Algorithm*

The TDD algorithm [34] is an off-line RL algorithm which can be applied to solve the optimization problem with continuous state space as well as continuous actions [35,36]. TDD includes a single actor network (i.e., a deterministic policy network) *π<sup>φ</sup>* and its actor target network *πφ*<sup>0</sup> . In addition, TDD adopts three critic networks *Qθ*<sup>1</sup> , *Qθ*<sup>2</sup> , and *Qθ*<sup>3</sup> for Q-value estimation. *Q<sup>θ</sup>* 0 1 , *Q<sup>θ</sup>* 0 2 and *Q<sup>θ</sup>* 0 3 represent three target networks, corresponding to critic networks *Qθ*<sup>1</sup> , *Qθ*<sup>2</sup> , and *Qθ*<sup>3</sup> , respectively. The target value *y<sup>t</sup>* can be updated using the weighted minimum Q-value of target Q-networks *Q<sup>θ</sup>* 0 1 and *Q<sup>θ</sup>* 0 2 , combined with the weighted value of *Q<sup>θ</sup>* 0 3 as follows [34]:

$$y\_t = r\_t + \gamma \left[ \beta \min\_{j \in \{1, 2\}} Q\_{\theta\_j^\prime}(s\_{t+1}, \tilde{a}\_{t+1}) + (1 - \beta) Q\_{\theta\_3^\prime}(s\_{t+1}, \tilde{a}\_{t+1}) \right],\tag{15}$$

where *β* ∈ (0, 1) is the weight of the pair of critics, *γ* ∈ [0, 1] is a discount factor, and *a*˜*t*+<sup>1</sup> is the clipped target action, calculated as follows:

$$\mathfrak{a}\_{t+1} \leftarrow \pi\_{\phi'}(s\_{t+1}) + \mathfrak{e}\_{\prime} \quad \mathfrak{e} \sim \text{clip}(\mathcal{N}(0, \mathfrak{d}), -\mathfrak{e}, \mathfrak{e}),\tag{16}$$

where *e* is the clipped Gaussian noise with standard deviation of *σ*˜, and *c* is the edge value. The parameters of the critic networks will be updated by minimizing the following loss:

$$\mathcal{L}(\theta) = \mathbb{E}\_{\left(s\_t, a\_t, r\_t, s\_{t+1}\right) \sim \mathcal{R}} \left[ \left( Q\_{\theta}(s\_t, a\_t) - y\_t \right)^2 \right],\tag{17}$$

where R is a replay buffer to store and relay experience transactions (*s<sup>t</sup>* , *a<sup>t</sup>* ,*rt* ,*st*+1) including states, actions, rewards, and next states. The deterministic policy network in actor is updated using sampled policy gradient which is shown as follows:

$$\nabla\_{\Phi} I(\boldsymbol{\phi}) = \boldsymbol{N}^{-1} \sum \nabla\_{a} Q\_{\theta\_{l}}(\boldsymbol{s}\_{l}, \boldsymbol{a}\_{l}) \Big|\_{\boldsymbol{a} = \pi\_{\Phi}(\boldsymbol{s}\_{l})} \nabla\_{\Phi} \pi\_{\Phi}(\boldsymbol{s}\_{l}) \,. \tag{18}$$

#### *4.2. Proposed TDD–ND Co-Optimization Approach*

The ND policy is combined with the TDD algorithm to address the co-optimization problem. For the ND policy, the standard deviation of the exploration noise *e* is set to the maximum value *σmax* at the beginning of the training, gradually reduced to the minimum value *σmin* with a decay of *σdecay* with the increase of the number of the training episodes, and kept at the minimum value *σmin* for the rest of the training. A TDD–ND algorithm for five-minute submodel optimization is presented in Algorithm 1.


Initialize the actor network *πφ*, the actor target network *πφ*<sup>0</sup> ← *πφ*, the size *R* of replay buffer R, and the mini-batch size *m*.

Initialize the three critic networks *Qθ*<sup>1</sup> , *Qθ*<sup>2</sup> and *Qθ*<sup>3</sup> , and three critic target networks *Q<sup>θ</sup>* 0 1 ← *Qθ*<sup>1</sup> , *Q<sup>θ</sup>* 0 2 ← *Qθ*<sup>2</sup> and *Q<sup>θ</sup>* 0 3 ← *Qθ*<sup>3</sup> .

1: **for** episode *i* ← 0 to I **do**

$$\text{2: } \quad \mathbf{for} \stackrel{\cdot}{t} t \in \underset{\cdot}{T^A} \mathbf{do}$$

	- *θ* 0 *<sup>i</sup>* ← *τθ<sup>i</sup>* + (1 − *τ*)*θ* 0 *i* , *i* = 1, 2, 3 every 2 iterations, where *τ* 1 is the target update parameter.
	- *φ*<sup>0</sup> ← *τφ* + (1 − *τ*)*φ*<sup>0</sup> every 2 iterations.

The flow chart of the proposed TDD–ND co-optimization approach is illustrated in Figure 3. The TDD–ND algorithm is used to train the neural networks for five-minute submodel optimization. The best actions of the five-minute submodel *λ* ∗ *T* are then input into the two-second submodel environment. The TDD–ND algorithm is then used to train neural networks for two-second submodel optimization. For each training iteration, after action *λ F t* is chosen, *λ<sup>t</sup>* is calculated based on Equation (14), and the reward *Rt*(*S F t* , *λ F t* ) will be calculated using Equation (11) to maximize the accumulated reward within the one-day horizon. After each time step, a mini-batch of *m* transitions is sampled uniformly from a replay buffer R.

#### **5. Experimental Results**

The performance of the proposed co-optimization approach is evaluated in a realworld scenario. The values of the parameters used in the simulations are listed in Table 1. Some of the parameters are varied in the simulation and will be noted accordingly. The parameter settings for the TDD–ND algorithm are listed in Table 2.


**Table 1.** The value of the parameters used in the simulation.

**Table 2.** TDD–ND parameter settings.


#### *5.1. Performance Evaluation of the Proposed TDD–ND Algorithm*

Based on the principle of EA, the BESS charges at low electricity prices and discharges at high electricity prices. The average price works as a simple indicator to determine whether the price *P A T* is low or high compared to the historical values. The operations of the BESS in a day are illustrated in Figure 4. The figure shows that when the *P A T* is lower than the average price, the BESS actions are mainly larger than 0, which means the BESS is charging. However, when the *P A T* is higher than the average price, the BESS operations are discharging to gain profits. The figure matches well with the principle of EA.

**Figure 4.** The BESS operation in a one-day period after five-minute submodel optimization.

The performance of the TDD–ND algorithm for co-optimizing EA and FR is studied by comparing it with another widely-used DRL algorithm, the deep Q-learning (DQL) algorithm. TDD–ND and DQL algorithms were used to train the five-minute and twosecond submodels for 500 times (500 episodes). During the training using TDD–ND, the total revenue of a day was validated after every 10 episodes without adding exploration noise *e* to see whether the results were close to the training results. The learning curves of the TDD–ND algorithm and the DQL algorithm are illustrated in Figures 5 and 6, respectively. These two figures show that the TDD–ND algorithm has a much better performance than the DQL algorithm in terms of the average performance score and the total reward. The reason is that the TDD–ND algorithm can choose more accurate continuous actions rather than using discretized actions in DQL, and can thus obtain a higher average performance score and total reward.

**Figure 5.** The learning curve of the average performance score of the day trained by TDD–ND and DQL.

**Figure 6.** The learning curves of TDD–ND and DQL of the total revenue within a day.

The impact of different levels of power capacity *B* and energy capacity *E* max on the performance of the TDD–ND algorithm and the DQL algorithm are studied. After training, the TDD–ND test results are slightly higher than their training values without the exploration noise. Figure 7 shows that the proposed TDD–ND algorithm always performs better than the DQL algorithm. The reason is that the DQL algorithm chooses discretized actions rather than continuous actions to take, and thus negatively impacts the total revenue. The figure also shows that the total revenue using both algorithms increases with power capacity *B* in the similar trend. For both algorithms, the total revenue increases sharply with *B* when *B* is between 0.5 and 1.0. The reason is that when *B* is 0.5, the *SC<sup>t</sup>* is smaller than 0.4 in most time slots, and thus the regulation settlement reward *R F t* becomes 0. When *B* increases to 1, the *SC<sup>t</sup>* is greater than 0.4 in many more time slots. Therefore, the total revenue is significantly increased. Between *B* = 1 and *B* = 2.5, the improvement in total revenue approximately follows the increase of *B*, since *R F t* is the dominant factor in the total revenue, and is a linear function of *B*.

How energy capacity *E* max impacts the total revenue using the proposed TDD–ND algorithm and the DQL algorithm is shown in Figure 8. The figure shows that the TDD–ND algorithm generates more total revenue than the DQL algorithm under each of the *E* max settings. Compared to the impact of power capacity *B*, the increase of energy capacity *E* max only makes a slight change to the total revenue. For both algorithms, the total revenue rises slowly with the increase of *E* max between *E* max = 2.5 and 12.5, as the energy capacity increasing only improves *R A t* but *R F <sup>t</sup>* dominates the total revenue in Equation (11) when *B* = 1 MW. Compared to the TDD–ND algorithm, the DQL algorithm has a slightly higher improvement rate of the total reward with the increase of *E* max, since higher *E* max allows the DQL algorithm to choose better discretized actions for EA, and thus a higher improvement rate of *R A t* compared to the TDD–ND algorithm with continuous actions.

**Figure 7.** The total revenue of the day with different levels of power capacity *B*.

**Figure 8.** The total revenue of the day with different levels of energy capacity *E* max .

#### *5.2. Performance Comparison of Various Schemes*

To demonstrate the effectiveness of our proposed TDD–ND co-optimization scheme, the following methods are compared: (1) Pure-EA scheme, in which the BESS only provides the EA service; (2) Pure-FR scheme, in which the BESS only provides the FR service; (3) Rule-based co-optimization scheme, in which the BESS provides the EA and FR services. The rule is as follows: The action *λ<sup>t</sup>* is set to *λ* ∗ *T* to maximize the reward due to the total amount of energy change caused by both EA and FR; (4) TDD–ND co-optimization scheme is our proposed TDD–ND algorithm and co-optimization scheme.

The total revenue using each scheme with different settings of *B* and *E* max is illustrated in Figures 9 and 10, respectively. Figure 9 shows that the TDD–ND co-optimization scheme generates much more total revenue than the other three schemes at every setting of *B*, as the TDD–ND co-optimization scheme tries to maximize the total accumulated reward. The total revenue using the pure-EA scheme is very small, since FR is much more profitable than EA. For the TDD–ND scheme, the rule-based scheme and the pure-FR scheme, the

regulation settlement reward *R F t* increases with *B*, and thus the total revenue increases with *B*. For the pure-EA scheme, the higher *B* allows the BESS to charge more when *P A T* is low and discharge more when *P A T* is high. The increasing rates of the total revenue using the TDD–ND co-optimization scheme and the rule-based co-optimization scheme from *B* = 0.5 to *B* = 1 are higher than those from *B* = 1 to *B* = 2.5. The reason is that when *B* = 0.5, the performance score is smaller than 0.4 in most time slots, and thus the regulation settlement reward becomes 0. Therefore, the total revenue of the rule-based co-optimization scheme is close to that of the pure-FR scheme. When B increases to 1, the performance score is greater than 0.4 in many more time slots, and with the coordination of EA, the total revenue is significantly increased. When *B* is between 1 and 1.5, the total revenue of the pure-FR scheme is much lower than that of the rule-based co-optimization scheme and that of the TDD–ND cop-optimization scheme. The reason is that the pure-FR scheme cannot follow *rd<sup>t</sup>* signals closely due to the limitations of the energy capacity, while the rule-based scheme can coordinate the energy capacity for EA and FR. When *B* reaches 2 or higher, the rule-based scheme has similar total revenue to the pure-FR scheme, as the setting of *B* allows the rule-based scheme to follow *rd<sup>t</sup>* signal closely.

**Figure 9.** The comparison of the total revenues between using our proposed TDD–ND cooptimization scheme, rule-based co-optimization scheme, pure-FR scheme, and pure-EA scheme under different levels of power capacity *B*.

The total revenue of each of the four schemes under different settings of energy capacities *E* max is presented in Figure 10. The total revenue of the proposed TDD–ND co-optimization scheme is much higher than those of the other three schemes. FR is much more profitable than EA under all of the *E* max settings. The total revenue of the TDD–ND scheme and the rule-based scheme increases slightly with *E* max, because the increase of *E* max only improves the value of *R A t* , which is a small portion of the total revenue *R<sup>t</sup>* . For the pure-EA scheme, the total revenue increases with energy capacity *E* max, as higher energy capacity allows the BESS to charge more when *P A T* is low and discharge more when *P A T* is high.

**Figure 10.** The comparison of the revenues between following the proposed TDD–ND cooptimization, rule-based co-optimization, and pure-FR and pure-EA schemes under different settings of energy capacity *E* max .

#### **6. Conclusions**

A battery energy storage system (BESS) providing both energy arbitrage (EA) and frequency regulation (FR) services simultaneously to maximize its total revenue within a day was considered. The BESS takes an EA action every five minutes and an FR action every two seconds. The multitimescale co-optimization problem was formulated as two nested Markov decision process (MDP) submodels. A novel co-optimization scheme was proposed to handle the multitimescale problem and to coordinate the EA and FR services to maximize the total revenue. The novel deep reinforcement learning (DRL) algorithm, triplet deep deterministic policy gradient with exploration noise decay (TDD–ND), was proposed to determine the best actions to take to maximize the accumulated reward within the one-day horizon. The proposed TDD–ND algorithm achieved 22.8% to 32.9% higher total revenue than the deep Q-learning (DQL) algorithm under various power capacity settings of the BESS when its energy capacity was 5 MWh, and achieved 16.7% to 26.6% higher total revenue under various energy capacity settings when the power capacity was 1 MW. Additionally, our proposed TDD–ND co-optimization scheme achieved 37.7% to 148.8%, 41.8% to 156.3%, and 3507.8% to 15,583.2% higher total revenues compared to the rule-based co-optimization scheme, the pure-FR scheme, and the pure-EA scheme, respectively, under various power capacity settings when the energy capacity of the BESS was 5 MWh. When the power capacity was set to 1 MW, the proposed TDD–ND cooptimization scheme achieved total revenues 49.6% to 56.2%, 51.0% to 198.4%, and 7156.2% to 12,777.0% higher than the rule-based co-optimization scheme, the pure-FR scheme, and the pure-EA scheme, respectively, under the various energy capacity settings. In the future, investigation can be carried out on the use of the co-optimization methods in multivector energy systems considering different timescales.

**Author Contributions:** Conceptualization, Y.M.; methodology, Y.M., T.C. and S.B.; software, Y.M., and T.C.; simulation, Y.M.; validation, Y.M.; formal analysis, Y.M.; investigation, Y.M., T.C. and S.B.; resources, Y.M. and S. B.; data curation, Y.M.; writing—original draft preparation, Y.M.; writing—review and editing, Y.M., S.B., H.L. and Z.H.; visualization, Y.M.; supervision, S.B.; project administration, Y.M., T.C. and S.B. All authors have read and agreed to the published version of the manuscript.

**Funding:** This work was supported by start-up funds provided by Brock University, NSF CNS-2128368, CNS-2107216, Toyota and Amazon.

**Conflicts of Interest:** The authors declare no conflicts of interest.

### **Nomenclature**


### **References**


## *Article* **Multi-Objective Optimal Integration of Solar Heating and Heat Storage into Existing Fossil Fuel-Based Heat and Power Production Systems**

**Guangxuan Wang 1,2,\* and Julien Blondeau 1,2**


**Abstract:** Increasing the share of Renewable energy sources in District Heating (DH) systems is of great importance to mitigate their CO<sup>2</sup> emissions. The combined integration of Solar Thermal Collectors (STC) and Thermal Energy Storage (TES) into existing Combined Heat and Power (CHP) systems can be a very cost-effective way to do so. This paper aims at finding the optimal design of STC and TES systems integrated in existing CHP's considering two distinct objectives: economic profitability and environmental impact. To do so, we developed a three-stage framework based on Pareto-optimal solutions generated by multi-objective optimization, a Technique for Order Preference by Similarity to Ideal Solution (TOPSIS)-entropy method to select the optimal solution, followed by the definition of final Operation strategy. We proposed relevant improvement of the state-of-theart models used in similar analysis. We also applied the proposed methodology to the case of a representative, 12 MWth CHP plant. Our results show that, while the addition of TES or STC alone results in limited performances and/or higher costs, both the cost and the CO<sup>2</sup> emissions can be reduced by integrating the optimal combination of STC and TES. For the selected, optimal solution, carbon emissions are reduced by 10%, while the Annual Total Cost (*ATC*) is reduced by 3%. It also improves the operational flexibility and the efficiency by peak load shaving, load valley filling and thus by decreasing the peak load boiler operation. Compared to the addition of STC alone, the use of TES results in an increased efficiency, from 88% to 92%. The optimal share of STC is then increased from 7% to 10%.

**Keywords:** combined heat and power; thermal storage; solar heating; multi-objective optimization; decision making

## **1. Introduction**

The share of Renewable Energy Sources (RES) in energy systems is growing rapidly to accelerate the energy transition and tackle climate change. However, their penetration in the heating and cooling sector, which accounts for more than 50% of the final energy demand in EU [1], is only about 22% in Europe [2]. One of the key solutions to increase this share on the short term and in a cost-effective way is to integrate renewable energy in existing District Heating (DH) systems. Many initiatives are currently taken to do so. According to IRENA [3], Denmark has the ambition to increase the share of RES in their DH systems up to 73% in 2030 (vs. 42% in 2014). China targets a 24% share by 2030.

In terms of renewable resources, biomass, solar heating and geothermal energy are the options with largest potential to reach higher shares of RES in DH's [3], also at large scale [4–6]. Among these options, solar collectors present the additional challenge of being an intermittent source of energy, therefore potentially requiring additional heat storage, which represents an interesting optimization problem.

**Citation:** Wang, G.; Blondeau, J. Multi-Objective Optimal Integration of Solar Heating and Heat Storage into Existing Fossil Fuel-Based Heat and Power Production Systems. *Energies* **2022**, *15*, 1942. https:// doi.org/10.3390/en15051942

Academic Editors: Francesco Calise and Hee-Je Kim

Received: 13 December 2021 Accepted: 1 March 2022 Published: 7 March 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

Hot water Thermal Energy Storage (TES) can provide load shifting and is 100 times cheaper than electricity storage for the same energy capacity [7]. Thanks to TES, up to 25% RES would be integrated in energy systems without significantly affecting its efficiency [8]. Hybrid systems composed of Combined Heat and Power (CHP) units integrated with RES and TES is therefore considered as a first step towards the 4th generation district heating systems [8].

TES alone can also be added to existing CHP's. It contributes to peak load shaving, it can store energy when the demand is low and deliver it when the demand is high. With an optimal operation, it can also increase the energy efficiency, which in turn results in lower CO<sup>2</sup> emissions [9]. In many studies, the optimization of a combined CHP-TES system often focuses on economic aspects, taking into account the cost of CO<sup>2</sup> emissions. Benalcazar [10] proposed an optimization method based on the economic performance for the optimal sizing of how water TES, integrated in an existing CHP plant considering specific investment cost and different carbon prices. His analysis showed that the integration of the TES units can save operational cost and decrease the use of the heat-only boiler, which reduces fuel consumption and decreases CO<sup>2</sup> emissions. Mugnini et al. [11] assessed possible energy flexibility strategies to improve the performance of such system. Their results revealed that a hot water tank can increase the CHP working hours and primary energy savings. Lai et al. [12] developed an operation optimization model based on Particle swarm optimization method, to investigate the flexibility and thermodynamic performance of a CHP unit integrated with an integrated heat storage tank. Their results show that such an integration led to an increased range of operational conditions of CHP units.

Although the addition of Solar Thermal Collectors (STC) can lead to larger CO<sup>2</sup> emission savings than TES alone, they generally increase the production costs of existing systems [3]. The question of the optimal combination of STC and TES in terms of both economic and environmental impacts may therefore be raised. This corresponds to a multiobjective optimization problem with two design variables, i.e., the sizes of the STC and the TES systems.

Single economic objective optimization does not provide alternative solutions to deal with conflicting objectives [13]. Therefore, recent research efforts focused on multi-objective optimization of energy systems. Multi-objective optimization is used to find a trade-off between two or more conflicting objectives to support decision making. Ren et al. [14] proposed a multi-objective linear programming method for operational strategy of a Distributed Energy System (DES). Their model was based on trade-off analysis of economic and environmental optimization. Fazlollahi et al. [15] developed a multi-objective, multi-period optimization for sizing and operating a DH system with the objectives of maximizing the system efficiency and minimizing the CO<sup>2</sup> emissions and the Annual Total Cost (*ATC*), the annualized value of the total cost over the lifetime of the project. Luo et al. [16] developed a framework for the optimization of DES integrated with Genetic Algorithm for multi-objective optimization, and multi-criteria evaluated by Technique for Order Performance by Similarity to an Ideal Solution (TOPSIS) method. Karmellos et al. [17] presented a multi-objective Mixed-integer linear programming (MILP) model for the optimal design and operation of DES by ε-constraint method, with minimizing the *ATC*, and the total carbon emission as objective function. Franco and Versace [18] carried out a multi-objective strategy considering energetic and economic objectives to investigate design and operation strategy of a CHP-TES to DH network.

As discussed above, various approaches have therefore been applied to determine the economically and/or environmentally optimal design and operation of RES and TES integrated into existing CHP systems. However, the proposed methodologies can be further improved in the following respects. First, the modelling of the CHP systems could be more accurate. An increased accuracy of the techno-economic models for the following aspects could lead to more accurate results [19]: Piece-wise Linear Investment functions allowing for a non-linear evolution of the investment costs, and account for part-load efficiencies, start-up costs, CHP acceptable operation ranges and maximum ramp rates, which significantly affect technical and economic performances. Secondly, few recent works [15,16] integrated multi-objective optimization models with decision making methods to optimize the capacity and the operation strategy of TES and RES integrated to existing CHP systems. Moreover, limited research was carried out on the effect of fluctuating investment cost on the sizing and the operation of the system.

In this work, we therefore aim at integrating multi-objective optimization and decisionmaking methods featuring advanced techno-economic models, and to apply them to 1the STC and TES systems.

Our main objectives are the development a comprehensive methodology to allow decision-makers to determine the optimal design of hybrid heat and power production systems and to assess the economic and environmental impact of the optimal integration of STC and TES systems into existing, conventional CHP systems.

The main innovative features of this work can be summarized as follows:


The paper is organized as follows. Section 2 describes the proposed methodologies. The case of a hybrid energy system is defined in Section 3, including the input data and the investigated scenarios. Section 4 presents and discusses the results of the case study. Lastly, conclusions are drawn in Section 5.

#### **2. Materials and Methods**

This Section describes the proposed methodology for the optimization of the studied hybrid systems. Their general structure is defined in Section 2.1. The multi-objective optimization model is then described in Section 2.2. Finally, the Decision-making method is presented in Section 2.3.

Figure 1 illustrates the flow chart of the optimization framework used in this study. The framework consists of three stages:


#### *2.1. Hybrid Heat and Power Production System*

The components of the existing CHP system considered in this study are a 12 MWth coal-fired CHP with Extraction Condensing (EC) steam turbine, and two gas-fired Heat Only Boilers (HOB) with a thermal power of 5 MWth, respectively, also connected to the system. The sizes of the STC and of the TES that will be added to the existing system are the design variables. A schematic of the system is given in Figure 2. Electricity is produced by the CHP and sold to the local market. The generated heat can be sent to both the heat clients and the TES system. HOBs are only in operation when the heat production is insufficient. The constraints related to each unit described in Section 2.2.2, and the input data described in Section 3.1.

**Figure 1.** Optimization Framework of hybrid energy system.

**Figure 2.** Schematic of Hybrid energy system.

#### *2.2. Optimization Model*

#### 2.2.1. Objective Function

In this study, the weighting method [20] will be used to solve the multi-objective optimization problem through MILP method. The two objective functions to be minimized are the *ATC* and the total carbon emissions. A general objective function is then defined as shown in Equation (1):

$$\min \left\{ \mathbf{F}\_{obj} = \mathbf{a} \cdot \frac{\text{ATC}}{\text{ATC}\_{\text{min}}} + (1 - \mathbf{a}) \cdot \frac{\text{Carbon}}{\text{Carbon}\_{\text{min}}} \right\} \tag{1}$$

where α is a weighting factor and where the objectives functions are normalized using the corresponding single optimization value *ATCmin* and *Carbonmin*.

The *ATC* of the studied hybrid systems is defined in Equation (2) as the sum of the annualized investment cost *Cinv*, the maintenance cost *C<sup>M</sup>* and the operating cost *CO*, minus the revenues R<sup>S</sup> from the electricity production [14]. The total CO<sup>2</sup> emissions are calculated for all heat and power generation systems by multiplying the fuel consumption with the corresponding CO<sup>2</sup> emission factor, as shown in Equation (3).

$$\text{ATC} = \text{C}\_{\text{inv}} + \text{C}\_{M} + \text{C}\_{O} - \text{R}\_{S} \tag{2}$$

$$\text{Carbon} = \sum\_{\boldsymbol{\mu}} \sum\_{\boldsymbol{t}} \text{CF}\_{\boldsymbol{\mu}} \* \text{FC}\_{\boldsymbol{\mu}, \boldsymbol{t}} \tag{3}$$

The total investment cost *Cinv* is calculated following Equation (4). The annualized investment cost *c inv u*,*t* , *c invTES t* and *c invSTC t* (Equations (5)–(7)) are calculated using the annuity factor defined in Equation (8), which considers the discount rate *i* and the lifetime *n<sup>u</sup>* for each unit, respectively. The maintenance costs are defined as a fixed proportion of the investment costs and are calculated following the same structures, see Equation (9). Operational costs consist of fuel consumption and unit startup costs, see Equation (10).

$$\mathbb{C}\_{\text{inv}} = \sum\_{\boldsymbol{\mu}} \sum\_{\boldsymbol{t}} (c\_{\boldsymbol{u},\boldsymbol{t}}^{\text{inv}} \ast \mathbb{C}ap\_{\boldsymbol{\mu}} + c\_{\boldsymbol{t}}^{\text{inv}TES} \ast \boldsymbol{V} + c\_{\boldsymbol{t}}^{\text{inv}STC} \ast \boldsymbol{A}) \tag{4}$$

$$c\_{\iota,t}^{inv} = \frac{a\_{\iota t} \* I\_{\iota}^{inv}}{8760} \tag{5}$$

$$c\_t^{invTES} = \frac{a\_{\mathcal{U}} \* I^{invTES}}{8760} \tag{6}$$

$$c\_t^{invSTC} = \frac{a\_{\mu} \* I^{invSTC}}{8760} \tag{7}$$

$$a\_{\mathfrak{u}} = \frac{i\*(1+i)^{\mathfrak{u}\_{\mathfrak{u}}}}{(1+i)^{\mathfrak{u}\_{\mathfrak{u}}}-1}, \forall \text{ } \mathfrak{u} \in \text{units}, \text{ } \mathfrak{t} \in \text{periods} \tag{8}$$

$$\mathbf{C}\_{M} = \sum\_{\boldsymbol{\mu}} \sum\_{t} \left( c\_{\boldsymbol{\mu},t}^{M} \* \mathbf{C}ap\_{\boldsymbol{\mu}} + c\_{t}^{MTES} \* V + c\_{t}^{MSTC} \* A \right) \tag{9}$$

$$\mathcal{C}\_{\mathcal{O}} = \sum\_{\mathfrak{u}} \sum\_{t} (c\_{\mathfrak{u},t}^{fuel} \* \mathcal{F}\mathcal{C}\_{\mathfrak{u},t} + \delta\_{\mathfrak{u},t} \* \mathcal{S}\mathcal{C}\_{\mathfrak{u}}) \tag{10}$$

The revenues *R<sup>S</sup>* consist in selling electricity to the grid, see Equation (11).

$$R\_S = \sum\_t P \ast El\_t \tag{11}$$

#### 2.2.2. Constraints for System Design and Operation

All units are subject to some constraints. The following constraints are considered in this study for the design and the operation of each sub-system.

### Common Types of Constraints

Common types of constraints are applied to all units:

• Minimum and maximum loads

For all units, the production must be within the minimum and maximum loads, see Equation (12):

$$PLR\_{\mathfrak{u},l}^{lb} \* \mathbb{Q}\_{\mathfrak{u}}^{norm} \le \mathbb{Q}\_{\mathfrak{u},l} \le PLR\_{\mathfrak{u},l}^{ub} \* \mathbb{Q}\_{\mathfrak{u}}^{norm} \,\forall \, \mathbf{u} \in \text{units}, \, \mathbf{t} \in \text{periods} \tag{12}$$

• Ramping rate limits

Maximum ramping rates (up and down) of the thermal units should be defined for stability, integrity and safety reasons, see Equation (13) and (14):

$$Q\_{\mathbf{u},t+1} - Q\_{\mathbf{u},t} \le \operatorname{ramp}\_{\operatorname{up}\_{\mathbf{u},t}} \* Q\_{\mathbf{u}}^{\operatorname{norm}} \,\forall \,\mathbf{u} \in \operatorname{units}, \,\mathbf{t} \in \operatorname{periods} \tag{13}$$

$$Q\_{\mathbf{u},t+1} - Q\_{\mathbf{u},t} \le \operatorname{ramp}\_{\text{down}\_{\mathbf{u},t}} \* Q\_{\mathbf{u}}^{\text{norm}} \,\forall \,\mathbf{u} \in \text{units}, \,\mathbf{t} \in \text{periods} \tag{14}$$

• State of units

In the framework of MILP optimization, the binary variable *I use u*,*t* is used to illustrate the state of the units. It is constrained by the variables *T ON u*,*t* and *T OFF <sup>u</sup>*,*<sup>t</sup>* defining whether the unit is on or off at time t, see Equations (15) and (16).

$$I\_{\boldsymbol{\mu},t+1}^{\mathrm{use}} - I\_{\boldsymbol{\mu},t}^{\mathrm{use}} \le T\_{\boldsymbol{\mu},t+1}^{\mathrm{ON}} \,\forall \,\mathbf{u} \in \text{units}, \,\mathbf{t} \in \text{periods} \tag{15}$$

$$I\_{\mathbf{u},t}^{\rm us\varepsilon} - I\_{\mathbf{u},t+1}^{\rm us\varepsilon} + T\_{\mathbf{u},t+1}^{\rm ON} \le T\_{\mathbf{u},t+1}^{\rm OFF} \,\forall \,\mathbf{u} \in \mathbf{u} \text{mits} \,\,\mathbf{t} \in \text{peroids} \tag{16}$$

• Minimum uptime and downtime

Minimum uptime and downtime constraints are considered for safety and reliability reasons. If a unit is in service, the duration should be at least equal to the defined minimum uptime, as indicated in Equation (17). Similarly, if a unit is not in running, the period should be at least equal to the defined minimum downtime as expressed in Equation (18).

$$\sum\_{t=\min\\_uptime}^{t} T\_{u,t}^{ON} \le I\_{u,t}^{\text{use}} \; \forall \; \mathbf{u} \in \text{units}, \; \mathbf{t} \in (\min\\_uptime, \; \text{periods}), \tag{17}$$

$$\sum\_{t-\min\_{\text{down}}}^t T\_{\mathbf{u},t}^{\text{OFF}} \le 1 - I\_{\mathbf{u},t}^{\text{use}} \,\forall\,\mathbf{u}\in\text{units},\,\mathbf{t}\in(\text{min\\_down};\,\text{periods}),\tag{18}$$

Extraction Condensation CHP Unit

The CHP plant considered in this study is of the Extraction Condensation type (EC): heat is provided to the heat clients through the condensation of steam extracted from the turbine. EC-based CHP can be operated in a flexible way, since steam can usually be extracted from more than one pressure stage, and power may vary with same heat production. In this study, the total energy efficiency of the CHP plant is considered as constant, as suggested in [17,19].

For EC-based CHP's, the power loss coefficient β, defined as the power generation reduction caused by heat extraction, is a key characteristic [21]. It can be calculated based on the ambient and DH temperatures [22], see Equations (19) and (20):

$$\beta = 1 - \frac{T\_0}{T\_M} \tag{19}$$

$$T\_M = \frac{T^{supply} - T^{return}}{\ln\left(\frac{T^{supply}}{T^{return}}\right)}\tag{20}$$

The acceptable operation zone of the CHP plant in terms of heat extraction and electrical power for a given supply and return temperature is illustrated in Figure 3. It is elaborated as suggested by [22]. The blue line BC is the backpressure line, exhibiting a slope σ. It represents the operation points with the designed power to heat ratio [21]. The maximum load line CD is parallel to the minimum load line AB (both with a slope β). Any line with the same slope corresponds to a constant fuel consumption.

**Figure 3.** Cogeneration of Power and heat extraction for EC based CHP plant.

Equation (21) and (22) express the corresponding constraints on the operational loads. The fuel consumption can be calculated using Equation (23).

$$P\_t^{\text{EC}} = PLR\_t^{\text{EC}} \ast I\_{\text{EC},t}^{\text{use}} \ast \left(\sigma^{\text{EC}} + \beta\_l\right) \ast Q\_{\text{EC}}^{\text{norm}} - Q\_t^{\text{EC}} \ast \beta\_l \,\forall \, t \in \text{peroids} \tag{21}$$

$$P\_t^{EC} \ge \sigma^{EC} \* Q\_t^{EC} \,\forall \,\mathbf{t} \in \text{peroids} \tag{22}$$

$$F\mathbb{C}\_t^{EC} = (Q\_t^{EC} + P\_t^{EC}) / \eta\_{EC} \,\forall \, t \in \text{periods} \tag{23}$$

Heat Only Boiler

The part load efficiency of the gas-fired Heat Only Boiler (HOB) is considered according to the model proposed by [19]. Equation (24) provides an expression for the consumption of the HOB boiler, which leads to the non-linear evolution of its efficiency, as shown in Figure 4.

**Figure 4.** Normalized efficiency for HOB, according to [19].

$$\rm{FC}\_{l}^{HOB} = 0.4576 \ast \,\rm{Q}\_{HOB}^{norm} \ast \,\rm{I}\_{HOB,t}^{us} + 0.6599 \ast \,\rm{Q}\_{l}^{HOB} \,\forall \,\rm{t} \in \text{periods} \tag{24}$$

Equations (25) and (26) state that the two HOB (HOB1 and HOB2) are only in operation when the heat demand cannot be satisfied by the main steam boiler (peak load), during maintenance or when the demand exceeds the CHP output.

$$\mathbf{Q}\_{t}^{\rm EC} \ge \mathbf{I}\_{HOB1,t}^{\rm use} \ast \mathbf{Q}\_{\rm CHP}^{\rm norm} \tag{25}$$

$$Q\_t^{EC} + Q\_t^{HOB} \ge \ I\_{HOB2,t}^{use} \ast (Q\_{CHP}^{norm} + Q\_{HOB}^{norm}) \tag{26}$$

Thermal Energy Storage Unit

The Thermal Energy Storage (TES) system is modeled based on the formulation proposed by Wang et al. [23]. Upper and lower boundary conditions are defined for the charging and discharging rates, see Equations (27) and (28). To ensure the availability of the TES, the stored thermal energy at the end of the schedule period is assumed to be equal to its initial value, see Equation (29). The energy balance of the system, taking into account the storage efficiency and the charging and discharging efficiencies, is expressed in Equation (30). The binary variable *z*(*t*) is used to illustrate the TES charging and discharging operation status, see Equation (31). Equations (32) and (33) express the constraints that the stored thermal energy must be lower than the maximum storage capacity of the system, which depends on TES volume and DH supply and return temperatures.

$$0 \le HS\_t^{\text{chr}} \le \left. a\_t^{\text{chr}} \cdot \text{Cap}\_t^{\text{TES}} \,\forall \,\mathbf{t} \in \text{periords} \tag{27}$$

$$0 \le HS\_t^{\text{dis}} \le a\_t^{\text{dis}} \cdot \text{Cap}\_t^{\text{TES}} \forall \text{ t} \in \text{peroids} \tag{28}$$

$$Q\_1^{TES} = Q\_{end}^{TES} \vee \mathbf{t} \in \text{peroids} \tag{29}$$

$$\mathbf{Q}\_{t}^{TES} = \eta\_{s}^{TES}\mathbf{Q}\_{t-1}^{TES} + \eta\_{clr}^{TES}HS\_{t}^{clr} - HS\_{t}^{dis}/\eta\_{dis}^{TES} \,\forall \mathbf{t} \in \text{periords} \tag{30}$$

$$z(t) = \begin{cases} 1, & HS\_t^{dis} = 0 \\ 0, & HS\_t^{chr} = 0 \end{cases} \forall \text{ t } \in \text{periods} \tag{31}$$

$$\text{Cap}\_{t}^{\text{TES}} = \frac{\rho Vc}{3600} \{ \min((T\_{t}^{\text{supply}} - 5), 98) - (T\_{t}^{\text{return}} + 5) \} \,\forall \, t \in \text{periods} \tag{32}$$

$$\mathbf{Q}\_t^{\text{TES}} \le \mathbf{C}ap\_t^{\text{TES}} \; \forall \; \mathbf{t} \in \text{periods} \tag{33}$$

The investment costs for the TES system, including the auxiliary equipment, can be estimated following [10,24]. Due to limitation of linear programming method, the exponential decay function of specific investment cost should be replaced with piecewise linear approximation for further use [10]. The specific investment costs curve according to [24], a piecewise linear function is used to make it compatible with MILP, see in Figure 5.

**Figure 5.** Investment costs for TES systems: (**a**) Specific investment cost according to [24]; (**b**) Cumulative investment cost.

#### Solar Thermal Collectors Unit

Equations (34) shows how the heating performance of STC can be calculated based on the collector area, solar irradiance and DHN temperature according to the formulation from [25].

$$\begin{aligned} Q\_t^{\rm STC} &= A \ast \left( (0.839 \ast Q\_t^{\rm SR} - 2.46 \ast \left( T\_t^{\rm aer} - T\_t^{\rm ambient} \right) \right) \\ &- 0.0197 \ast \left( T\_t^{\rm aer} - T\_t^{\rm ambient} \right)^2 ) \,\forall \,\mathbf{t} \in \text{periods} \end{aligned} \tag{34}$$

The specific investment costs of ground mounted solar collector field (shown in Figure 6a) are estimated based on [25]. PLI is also applied, cumulative investment cost see Figure 6b.

#### Energy Balance

Equation (35) expresses that the heat demand from the local distribution network must always be fulfilled for each time interval.

$$Q\_t^{CHP} + Q\_t^{HOB} + Q\_t^{STC} + \eta\_{dis}^{TES} HS\_t^{dis} - HS\_t^{drr} \ge Q\_t^{demand} \; \forall \; t \in \text{peroids} \tag{35}$$

**Figure 6.** Investment costs for STC: (**a**) Specific investment cost according to [25]; (**b**) Cumulative investment cost.

#### *2.3. Decision-Making Method*

Figure 7 illustrates the flow chart of the decision-making process used in this study. The decision-making process is to provide a quantitative evaluation on the system with several conflicting objectives, to support the Decision makers the most potential solutions by considering the important criteria [13]. To evaluate performance of the multi-objective optimization model, a decision-making method combining Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) with Entropy method (EM) is investigated. TOPSIS is a classic method to find optimal solution within finite objectives [26], while EM aims at determining the weights of each objective in an automatic way, in order to minimize man-made error [27]. As illustrated in Figure 7, The TOPSIS-entropy method selects the solution with the maximal relative quality as the optimal solution.

**Figure 7.** Flowchart of Decision-making process.

The different steps followed to the TOPSIS-entropy method are as follows [28]:

• Weights calculation:

The first step is matrix normalization, see Equation (36). This ensures that all indicators are positive and present comparable ranges. Then, standardized value *pij* and entropy value of objective *e<sup>j</sup>* are calculated using Equations (37) and (38). Finally, weighting values of the objectives *ω<sup>j</sup>* are defined, see Equation (39).

$$y\_{i\bar{j}} = \frac{\max(\mathbf{x}\_{i\bar{j}}) - \mathbf{x}\_{i\bar{j}}}{\max(\mathbf{x}\_{i\bar{j}}) - \min(\mathbf{x}\_{i\bar{j}})} \; \forall \; i \in [1 \ldots m], \; \mathbf{i} \in (1 \ldots n) \tag{36}$$

$$p\_{i\bar{j}} = \frac{y\_{i\bar{j}}}{\sqrt{\sum\_{i=1}^{m} y\_{i\bar{j}}}} \forall \text{ i } \in [1 \dots m] \text{ } \text{ i } \in (1 \dots n) \tag{37}$$

$$e\_{\vec{j}} = -\frac{1}{\ln(m)} \sum\_{i=1}^{m} p\_{\vec{ij}} \ln(p\_{\vec{ij}}) \,\forall \,\vec{j} \in \,\,(1\,\ldots\,\text{n})\tag{38}$$

$$
\omega\_{\dot{j}} = \frac{1 - e\_{\dot{j}}}{\sum\_{\dot{j}=1}^{n} \left(1 - e\_{\dot{j}}\right)} \tag{39}
$$

## • TOPSIS method

The weighted matrix used as a basis for the TOPSIS method is described in. Equation (40) The positive ideal solution *V* <sup>+</sup> and negative ideal solution *V* − are then determined using Equations (41) and (42). The distances *D*<sup>+</sup> and *D*<sup>−</sup> between any evaluated result and the two positive and negative solution *V* <sup>+</sup> and *V* − are calculated using Equations (43) and (44). Finally, the relative quality *C<sup>i</sup>* is defined to compare the distance between any result and the two ideal solutions, see Equation (45). The optimal solution corresponds to the highest value of *C<sup>i</sup>* .

$$
\omega\_{i\bar{j}} = \omega\_{i\bar{j}} \* y\_{i\bar{j}} \forall \text{ i } \in [1 \dots m], \text{ i } \in (1 \dots n) \tag{40}
$$

$$V^{+} = \left\{ \left( \max V\_{\text{ij}} \left| \mathbf{j} \in I\_{1} \right. \right), \left( \min V\_{\text{ij}} \left| \mathbf{j} \in I\_{2} \right. \right) \right\} \forall \mathbf{i} \in \left( 1 \ldots \mathbf{n} \right) \tag{41}$$

$$V^- = \left\{ (\min V\_{\text{ij}} | \mathbf{j} \in I\_1), \ (\max V\_{\text{ij}} | \mathbf{j} \in I\_2) \right\} \forall \mathbf{i} \in (1 \ldots \mathbf{n}) \tag{42}$$

$$D^{+} = \sqrt{\sum\_{j=1}^{n} \left(V\_{lj} - V\_{j}^{+}\right)^{2}} \forall \text{ i } \in \left(1 \ldots \text{n}\right) \tag{43}$$

$$D^{-} = \sqrt{\sum\_{j=1}^{n} \left(V\_{lj} - V\_{j}^{-}\right)^{2}} \forall \text{ i } \in \left(1 \dots m\right) \tag{44}$$

$$\mathcal{C}\_{\mathbf{i}} = \frac{D^-}{D^+ - D^-} \,\forall \,\mathbf{i} \in (1 \dots m) \tag{45}$$

#### **3. Case Study**

In this Section, the proposed optimization framework is applied to a specific case of hybrid system described in Section 2.1. The considered input data is described in Section 3.1. Four different scenarios are defined in Section 3.2.

#### *3.1. Input Data*

The considered heat demand, ambient temperature, supply and return temperatures are field data retrieved in 2017 from an existing DH plant located in France. The hourly market electricity price was imported from EPEX SPOT for the year 2017 [29] and local climate data of solar radiation were retrieved from PVGIS for the same year [30]. The impact of the uncertainty on the main parameters will be investigated in Section 4.4.

However, the annual profile with 8760 time steps makes is difficult to solve due to complexity of optimization model [31–33]. To limit the computational time without significantly impacting the simulation results [32], the approach of typical periods is used in this work [33], a full year period was discretized into 12 typical days, each of them being representative for a month.

Figure 8 shows that the heat demand varies from 1.44 MWth to 16.9 MWth. The average heat load and heat load variations are larger in the winter. The corresponding supply and return temperatures range from 75 to 90 °C and from 55 to 75 °C, respectively. The average electricity price is also slightly higher in the winter, as illustrated in Figure 9. For each typical day, the heating load peak is observed between 8 and 10 AM and between 8 and 10 PM, while the peak of electricity price is observed between 8 and 9 AM and between 6 and 7 PM. The hourly variations of ambient temperature and solar irradiance are also presented in Figure 10. The heat collected by the STC was calculated following the methodology of Schmidt (Ref. [10]). The considered tilt angle of the collector was 40◦ .

**Figure 8.** Heat Load in: (**a**) one year; (**b**) monthly typical days.

**Figure 9.** Electricity price in: (**a**) one year; (**b**) monthly typical days [29].

**Figure 10.** Hourly variation of ambient temperature and solar irradiance over the year [30].

The considered technical parameters of the heating and the thermal storage systems are provided in Tables 1 and 2. The related economic parameters, including investment and maintenance costs, lifetime and startup costs are presented in Table 3. The startup costs actually vary with the size of the units, but the constant values from [34] were considered here as they match the considered range of unit size.

CO<sup>2</sup> emissions are calculated based on the specific emission factors to the primary consumption [1]: 350 kg CO2/MWh for coal and 200 kg CO2/MWh for natural gas [35]. The CO<sup>2</sup> price in France in 2017 is collected from World Bank Carbon Pricing dashboard 30.5 € per tonne [36]. The fuel costs are assumed constant: 28 €/MWh for Coal and 32 €/MWh for natural gas [34].

The following assumption were also made on the efficiency of the systems: the effect of incidence angle was neglected in the design of the STC system and the CHP operation efficiency was considered constant.

**Table 1.** Technical parameters of each unit.


**Table 2.** Technical parameters of TES [23].


**Table 3.** Economic data of each unit [34].


#### *3.2. Scenarios*

Different scenarios are investigated in this study to analyze the impact of the integration of TES and STC systems into the existing CHP system. Scenario 1 covers the operation of the existing system without the addition of TES nor STC. Both systems are then considered and optimized in Scenario 2. In Scenarios 3 and 4, the addition of TES or STC alone is studied. The scenarios are summarized in Table 4.

**Table 4.** Overview of each Scenario.


#### **4. Results and Discussion**

In this Section, the results of the optimization performed for the 4 Scenarios of the case study are presented and discussed. In this study, the MILP model implemented in Python and solved with IBM DOcplex toolbox. Section 4.1 gives the results of the Pareto Frontiers for all Scenarios. In Section 4.2, the optimal solutions chosen using the TOPSIS-entropy method are presented. Section 4.3 gives more detailed information on the hourly operation strategy, and Section 4.4 presents a sensitivity analysis on the main parameters.

#### *4.1. Pareto Frontiers*

Figure 11 shows the Pareto frontiers computed for Scenarios 2 to 4 in terms of *ATC* and CO<sup>2</sup> emissions, expressed relatively to the reference Scenario 1. In Scenario 1, the *ATC* is 4380 k€/year and the CO<sup>2</sup> emissions are 43,435 ton/year.

**Figure 11.** Relative Pareto frontiers for Scenarios relative to Scenario 1.

Considering the addition of TES only (Scenario 3) quickly allows for a reduction of the system cost (5%) and a moderate reduction of the CO<sup>2</sup> emissions (up to 1% less). This is due to the larger consumption of the produced heat hen heat storage is available (see Section 4.3). When even lower CO<sup>2</sup> emissions must be reached with TES only, the price of the system however increases very sharp: it becomes higher than the reference cost for CO<sup>2</sup> emissions reductions around 7% and increases exponentially afterwards: +30% compared to the reference scenario for a 10% CO<sup>2</sup> emissions reduction.

On the other hand, considering STC only (Scenario 4) immediately leads to a higher cost, because of the higher price of STC. This system however becomes cheaper than TES only to reach CO<sup>2</sup> emissions reductions larger than 8%, although the cost also increases very rapidly for larger emission savings.

Figure 11 also shows that, combining TES and STC (Scenario 2) allows decreasing the *ATC* while reaching much larger CO<sup>2</sup> emissions reduction. The combination of these systems indeed leads to the valorization of a higher amount of heat collected by the STC system by storing it and making it available when the demand is high. Parity with the reference scenario in terms of costs is reached for 12% less CO<sup>2</sup> emissions. For even lower emissions, the cost increase is also significantly reduced compared to TES or STC only: 17% less CO<sup>2</sup> is emitted for cost increase of 7%. The installation of the additional equipment is made more cost-effective by their complementarity.

#### *4.2. TOPSIS-Entropy Method Analysis*

The TOPSIS-entropy method was used to choose the optimal solution on the Pareto frontiers obtained from the MILP optimizations. The weights calculated by the entropy method for each Scenario are very similar: around 0.54 and 0.46 for *ATC* and CO<sup>2</sup> emissions respectively. The solutions presenting the maximum relative quality for each Scenario are given in Table 5 and illustrated on the Pareto frontiers of Figure 11. The results generated for 12 representative days, see Section 3.1, have been extrapolated to a full year. Compared to the reference Scenario 1, the identified, optimal solution for Scenario 2 corresponds to a decrease of both the CO<sup>2</sup> emissions (10%) and the *ATC* (3%), which is an excellent trade-off. The total efficiency of the system increases from 87% to 92%, while the share of renewable energy in the system reaches 10%. The optimal solution for TES only (Scenario 3) is selected before the strong increase of the system cost on the Pareto frontier. The CO<sup>2</sup> emissions are reduced by 5% while the cost is reduced by 4%. When only the STC system is installed (Scenario 4), the optimal solution is again selected before the strong increase of the cost. In that case, the cost increases with 2%, for a CO<sup>2</sup> emissions reduction of 7%. The *ATC* is higher than for Scenario 2 because of the lower proportion of renewable heat that can be consumed. The addition of TES indeed allows for the displacement of the consumption of the heat collected by the STC to the periods when the demand is higher.


**Table 5.** Optimal solutions by TOPSIS-entropy method for each scenarios.

#### *4.3. Hourly Operation Strategy*

In this Section, the hourly operation of the optimal systems selected for the 4 scenarios during the typical days are presented and discussed. Figure 12 shows the hourly heat production per system and the hourly operation modes of the CHP on its operational map, over the 12 typical days in a row. It is obvious that integration of TES and STC significantly improve system operation flexibility and efficiency.

**Figure 12.** *Cont*.

**Figure 12.** Hourly heat production for all Scenarios over the 12 typical days in a row: (**a**) Hourly Heat production of Scenario 1; (**b**) CHP operation field of Scenario 1; (**c**) Hourly Heat production of Scenario 2; (**d**) CHP operation field of Scenario 2; (**e**) Hourly Heat production of Scenario 3; (**f**) CHP operation field of Scenario 3; (**g**) Hourly Heat production of Scenario 4; (**h**) CHP operation field of Scenario 4.

First, it can be observed that the TES and the STC systems play an important role in increasing the system flexibility. The system with TES is more flexible as it can achieve peak load shaving and load valley filling, which results in a decrease of the HOB operation [10]. The HOB's are operated up to 52% less when TES is implemented (Scenarios 2 and 3), as shown in Table 5. In the winter period, the thermal energy supplied by HOB during peak load time in Scenarios 1 is replaced by TES in Scenarios 2 and 3. During the summer period, the daytime heat demand is covered by STC in Scenarios 2 and 4. As expected, the excess heat produced by STC in Scenario 2 is stored in TES and discharged later on when STC cannot meet the demand.

Secondly, the operational flexibility of the CHP unit is also increased by the integration of TES and STC. Figure 12 shows that the CHP operates more on the left-hand part of the A-B line (minimum fuel consumption) in Scenario 2 than in Scenario 1, which helps increasing the electrical power generation for the same fuel consumption and, hence, the same CO<sup>2</sup> emissions.

The detailed performances of the TES system are also shown in Figure 13. TES enables the operation of the system associated with electricity prices, to maximize the profits from selling electricity without affect heat supply safety. Furthermore, TES is more active in the summer period when used in combination with STC. The optimal TES size in Scenario 2 is therefore twice larger than for Scenario 3, see Table 5.

#### *4.4. Sensitive Analysis*

The input parameters used in the case study above are prone to uncertainty. To assess the impact of these uncertainties, a sensitivity analysis is carried out for the optimal solution of Scenario 2, for the following key input parameters [37]: heat demand, carbon price, electricity price, fuel price and interest rate.

Figure 14 shows the performance of the optimal solutions in terms of *ATC*, CO<sup>2</sup> emissions and size of TES and STC, with the variation of the inputs (±20%). It must be noted that each of these results corresponds to an optimal solution computed for the new set of input parameters, and not to the results obtained for the reference optimal solution for other input parameters.

**Figure 13.** Charging or discharging load and heat storage levels of TES over the 12 typical days in a row: (**a**) TES charging and discharging load of Scenario 2; (**b**) TES heat storage levels of Scenario 2; (**c**) TES charging and discharging load of Scenario 3; (**d**) TES heat storage levels of Scenario 3.

Although they affect the optimal sizes of the TES and STC systems, the influence of economic parameters on the amount of CO<sup>2</sup> emitted in the optimal cases is negligible compared to the influence of the heat demand, see Figure 14a. As far as *ATC* is concerned (Figure 14b), heat demand and fuel price have a similar impact: *ATC* increased by 15% when the input parameter increases by 20%. The price of electricity exhibits the opposite trend: the *ATC* is 12.5% higher when it decreases with 20%, due to the lower revenues from the CHP unit. The price of CO<sup>2</sup> and the interest rate should not be neglected, although they have a lower impact on *ATC*.

In terms of impact on the systems design in Figure 14c,d, the TES volume is less sensitive to the uncertainties on the inputs than the STC area, which significantly varies with heat demand, fuel price and interest rate. Although this study showed that their combination with TES can result in a profitable reduction of CO<sup>2</sup> emissions, this shows that economic aspects can be a limiting factor in the implementation of STC.

**Figure 14.** Sensitivity of the main results to the variation of key input parameters: (**a**) CO<sup>2</sup> generation; (**b**) *ATC*; (**c**) TES volume; (**d**) STC area.

#### **5. Conclusions and Future Work**

A framework for the multi-objective optimization of the integration of Solar Thermal Collectors (STC) and Thermal Energy Storage (TES) systems in existing fossil-fuel based heat and power production systems was presented. The proposed method was applied to the representative case of a medium-scale CHP system coupled to a District Heating network. As a comparison, the integration of TES or STC alone was also considered.

The proposed TOPSIS-entropy method has been proved to be efficient to select the optimal design in terms of trade-off between cost and CO<sup>2</sup> reduction. Our results show that, while the addition of TES or STC alone results in limited economic and environmental performances and exhibits a rapid increase of the cost with the targeted CO<sup>2</sup> emission reduction, the optimal combination of TES and STC can lead to a reduction of both the cost and the CO<sup>2</sup> emissions: respectively 3% and 10% in the studied case. For larger CO<sup>2</sup> emissions savings, beyond the optimal trade-off, the additional cost remains limited compared to the other solutions. The integration of TES and STC also significantly improves the system flexibility and efficiency. TES allows for peak load shaving and load valley filling, resulting in up to 52% less operation of peak units. For the optimal design, the total efficiency of the system increases from 87% to 92%. The share of renewable energy reaches 10% when both TES and STC are integrated, compared to 7% with STC alone.

The operational flexibility of the CHP unit itself is also increased by the integration of TES and STC, which helps increasing the electrical power generation.

A sensitivity analysis shows that only the heat demand has a significant impact on the environmental performance, while both the heat demand and the fuel price have a significant influence on the economic performances. Furthermore, the optimal TES volume is less sensitive to the uncertainties on the inputs than the STC surface, that is more impacted by the economic parameters.

In future works, the same methodology could be applied to the integration of more renewable energy sources such as heat pumps and power-to-X based on renewable electricity production. The case of a new CHP unit will also be investigated instead of a retrofit, taking into account the optimal sizing of this unit and the related investment costs. Moreover, the impact of the uncertainties on the input data could be studied using advanced Uncertainty Quantification and Robust Design techniques.

**Author Contributions:** G.W.; conceptualization, data curation, methodology, software, formal analysis, investigation, writing—original draft, writing—review and editing, visualization. J.B.; conceptualization, methodology, investigation, writing—review and editing, supervision, project administration, funding acquisition. All authors have read and agreed to the published version of the manuscript.

**Funding:** The research received no external funding.

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study.

**Conflicts of Interest:** The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

#### **Nomenclature**



#### **References**


## *Article* **Design of a Neural Super-Twisting Controller to Emulate a Flywheel Energy Storage System**

**Daniel A. Magallón <sup>1</sup> , Carlos E. Castañeda 1,\*, Francisco Jurado <sup>2</sup> and Onofre A. Morfin <sup>3</sup>**


**Abstract:** In this work, a neural super-twisting algorithm is applied to the design of a controller for a flywheel energy storage system (FESS) emulator. Emulation of the FESS is achieved through driving a Permanent Magnet Synchronous Machine (PMSM) coupled to a shaft to shaft DC-motor. The emulation of the FESS is carried out by controlling the velocity of the PMSM in the energy storage stag and then by controlling the DC-motor velocity in the energy feedback stage, where the plant's states of both electrical machines are identified via a neural network. For the neural identification, a Recurrent Wavelet First-Order Neural Network (RWFONN) is proposed. For the design of the velocity controller, a super-twisting algorithm is applied by using a sliding surface as the argument; the latter is designed based on the states of the RWFONN, in combination with the block control linearization technique to the control of the angular velocity from both machines in their respective operation stage. The RWFONN is trained online using the filtered error algorithm. Closed-loop stability analysis is included when assuming boundedness of the synaptic weights. The results obtained from Matlab/Simulink validate the performance of the proposal in the control of an FESS.

**Keywords:** wavelet neural network; block control form; filtered error algorithm; neural supertwisting control; flywheel energy storage system

### **1. Introduction**

For many years, flywheels made of stone or other primitive materials have been used for the operation of different mechanisms. Currently, there are mechanisms built from metal flywheels, powered by electric motors, which can work in both motor and generator mode and whose main function is to store energy in the flywheel; because of this property, these mechanisms are known as Flywheel Energy Storage Systems (FESSs) [1]. Induction motors and synchronous motors are some types of electrical machines used in the application of FESSs. For the proper functioning of the electrical grid, it is essential to have a balance between consumption and supply of electrical energy. Due to its rapid response of operation, an FESS has the ability to reduce some of the problems caused by power variations. The control and comparison between different electrical machines used in FESSs are shown in [2]. The FESSs have been studied by different authors showing the performance from different control strategies. Furthermore, in several published works, a complete FESS has been emulated through the use of electric motors where the main operation of the flywheel is the storage of mechanical energy in a rotating flywheel, which can then be converted to electrical energy using an electrical machine with power inverters [3]. An FESS has several advantages in contrast with other energy storage systems, including long useful life, and an FESS has multiple applications such as in wind generators, electric vehicles, etc. [4]. In [5], a vector control strategy was presented to emulate a FESS using a DC permanent magnet machine, where the machine was coupled to a PMSM to control the angular velocity of the flywheel in the charging and discharging stages. In [6],

**Citation:** Magallón, D.A.; Castañeda, C.E.; Jurado, F.; Morfin, O.A. Design of a Neural Super-Twisting Controller to Emulate a Flywheel Energy Storage System. *Energies* **2021**, *14*, 6416. https://doi.org/10.3390/en14196416

Academic Editors: Victor Becerra and Ahmed Rachid

Received: 3 September 2021 Accepted: 29 September 2021 Published: 7 October 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

an integral sliding mode controller with a super-twisting algorithm was presented. In that work, the rotor side converter in the doubly-fed induction generator is used for controlling a wind generator, where the sliding surfaces are generated such that these are compatible with the errors in the stator active and reactive powers. In [7], emulation of an FESS using reconfigurable hardware test-bed of power converters was presented, where the emulation was carried out to the acceleration, deceleration and standby modes. The flywheel's model is described in the *dq* domain with relations among voltage, current and rotating speed. In [8], an adaptive control for high-speed FESSs was proposed where the controller has the task of controlling the DC-link voltage for the FESS and controlling the active power exchange between the FESS and the grid, whose advantage of inertia emulation is that the inertia and damping can be altered in real-time. In [9], under the assumption that all the states are measurable, a sliding mode control system for a 4-quadrant DC–DC converter was designed. Furthermore, a low voltage prototype for a flywheel application was constructed.

Recently, a control scheme applied to an FESS for the improvement of the dynamic performance of the utility grid, based on a PMSM incorporated into a multi-machine system, was presented in [10]. In that work, the speed and voltages generated were monitored by the FESS in order to generate the real and reactive power. Such control strategy was designed against three-phase faults.

On the one hand, considering that the fundamental parts of an energy storage system are the electric motors, the PMSM is a kind of synchronous motor that features low inertia, low noise, high power density and high efficiency, simple structure, and easy control. PMSMs have been used for the control of robotic movements, electric and aerospace propulsions, etc. [11]. The nonlinear mathematical model of a PMSM is strongly coupled. In addition, its parameters are not fixed but are kept within an operating range depending on the work environment. Angular velocity and torque can fluctuate in some parameter regions [12]. Recently, a feedback technique of dynamic surface control for a chaotic PMSM, based on nonlinear mappings using neural networks (NNs) to approximate the system dynamics as well as to estimate the unknown parameters, has been proposed in [13]. In [14], NNs were implemented in a PMSM for the detection of damage or failure in motor bearings by analyzing mechanical vibrations, comparing different types of NNs and the effectiveness of failure detection.

On the other hand, DC machines are generators that convert mechanical energy into DC electrical energy and motors that convert DC electrical energy into mechanical energy. Most DC machines are similar to alternating current machines in that they have AC voltages and currents inside them. DC machines have a DC output only because there is a mechanism that converts internal AC voltages into DC voltages at their terminals; since this mechanism is called a commutator, DC machinery is also known as a commutator or commutator machinery [15].

Implementation of the sliding-mode control to DC motors has been studied by different authors. In [16], an adaptive neural controller in discrete-time was proposed, where a NN was used to identify the plant model, using the Kalman extended filter training algorithm, when applying the block control in sliding mode for angular velocity control. A speed control scheme for a DC motor, based on state observers of sliding modes, was presented in [17]. In that work, the state observers estimate the angular velocity and the load torque; thus, in this way, it was possible to apply the controller, designed from the application of the exact linearization technique in combination with the supertwisting control.

The application of super-twisting control has been recently studied in [18], introducing the control to roll suppression of marine vessels subjected to harmonic wave excitations, where some of the main control objectives were to reduce the phenomenon of vibration over dynamic behaviors such as chaos, due to parametric variation.

The main contribution of this work is the proposal of a methodology to emulate a flywheel energy storage system by driving a motor-generator group formed by two machines,

namely, a PMSM coupled with a DC-motor. The state variables of the system are identified via recurrent wavelet NNs of first-order trained online by the filtered error algorithm. By using these NNs, it is possible to involve unmodeled dynamics and parameter variations, avoiding uncertainty in the machine parameters. The PMSM velocity controller is designed to emulate the stored kinetic energy of the flywheel, while the DC-motor velocity controller is used to emulate the energy that is discharged towards the utility grid. Both controllers are synthesized based on the proposed NN model transformed into the block controllable form to define a sliding surface that is steered to the origin in finite time by the action of the super-twisting control algorithm.

This paper is organized as follows. In Section 2, the mathematical background is presented, which is related to the kinetic energy storage, identification of the nonlinear system via recurrent wavelet first order neural network (RWFONN) using the filtered error training algorithm, and the super-twisting control algorithm; Section 3 presents the proposed methodology, where the procedures of charge/discharge are explained in detail; in Section 4, the simulation results are shown, from which the proposed methodology is validated; some important discussions are presented in Section 5; conclusion and some remarks about the application of the Neural Super-Twisting Control (NSTC) are drawn in Section 6. Furthermore, in Appendices A and B, the boundedness of the identification error and the complete closed-loop stability analysis are shown, respectively.

#### **2. Mathematical Background**

This section describes the energy storage in an FESS, the nonlinear mathematical representation of a permanent magnet synchronous motor and a DC-motor, the mathematical model of the dynamic behavior for an RWFONN and summarizes the filtered error algorithm, the block control linearization technique, and the super-twisting control algorithm.

#### *2.1. FESS*

The main function of FESSs is to store rotational kinetic energy, which can then be converted into electrical energy by means of an electric machine. In order to generate rotational kinetic energy, the flywheel must be driven by an electric machine that meets the requirement of operating in both motor and generator mode, allowing to convert rotational kinetic energy into electric energy and vice versa [4]. The main components from an FESS are the steering wheel, a vacuum containment to avoid loss by air friction, magnetic or metallic bearings, an electric machine capable of operating in both operation modes, i.e., motor and generator mode, and a three-phase converter to connect the FESS with the utility-grid [1]. The kinetic energy stored in the flywheel can be approximated by the following equation [19]

$$E\_k = \frac{1}{2} J\_m \omega\_{m'}^2 \tag{1}$$

where *E<sup>k</sup>* is the stored kinetic energy in *J*, *J<sup>m</sup>* is the inertial moment in N·m·s 2 , and *ω<sup>m</sup>* is the rated angular velocity in rad/s. In order to emulate the FESS deceleration, it is considered the next equation

$$\frac{d}{dt}\omega\_m = \frac{-B\_m\omega\_m}{J\_m},\tag{2}$$

which defines the mechanical behavior of the flywheel when the energy is delivered to the utility grid [20], with *B<sup>m</sup>* the frictional coefficient in N·m·s.

#### *2.2. Nonlinear Systems*

A class of nonlinear multi-input/multi-output system given in the general form

$$\dot{\mathbf{x}} = \mathbf{f}(\mathbf{x}\_{\prime}t) + \mathbf{B}(\mathbf{x}\_{\prime}t)\mathbf{u} + \mathbf{g}(\mathbf{x}\_{\prime}t) \tag{3}$$

where *<sup>χ</sup>* <sup>∈</sup> **<sup>X</sup>** <sup>⊂</sup> <sup>R</sup>*<sup>n</sup>* is the state vector of the system, **f**(*χ*, *t*) is the nonlinear system function vector; **<sup>u</sup>** <sup>∈</sup> **<sup>U</sup>** <sup>⊂</sup> <sup>R</sup>*<sup>m</sup>* is the control input vector, **<sup>B</sup>**(*χ*, *<sup>t</sup>*) is a non-singular matrix that characterizes the system, and **g**(*χ*, *t*) is a smooth function vector that represents external disturbances and parametric variations. It must be noticed that in the present work, both PMSM and DC-motor mathematical models match with that given in the general form (3). It is important to mention that due to the purpose of this work, these mathematical models are not shown here.

#### *2.3. RWFONN*

First-order NNs are used for system identification as well as to design neural controllers for some electrical, mechanical or electromechanical systems. A novel neural identification scheme, namely a Recurrent Morlet Wavelet Neural Network (RMWNN) also known as RWFONN, based on a recurrent high-order neural network (RHONN) structure was proposed in [21]. In the NN propose in this work, the classical sigmoid activation function has been replaced by a Morlet wavelet function. The general structure from an RWFONN is given by [22]

$$\dot{\mathfrak{x}}^i\_j = -a^i\_j \mathfrak{x}^i\_j + (w^i\_{jk})^\top \psi^i\_{jk} \tag{4}$$

where *x i j* is the state of the *i*-th neuron, *a i <sup>j</sup>* > 0 for *i* = 1, 2, . . ., *n* is part of the underlying network architecture and it is fixed during the training process, *w i jk* is the *k*-th adjustable synaptic weight connecting the *j*-th state to the *i*-th neuron, and *ψ i jk* is a Morlet wavelet activation function. The dynamic behavior of the whole neural network can be described by expressing Equation (4) in the vector notation

$$
\dot{\mathbf{x}} = \mathbf{A}\mathbf{x} + \mathbf{w}^{\top}\boldsymbol{\psi},\tag{5}
$$

where **x** = h *x* 1 *j x* 2 *j* . . . *x n j* i> <sup>∈</sup> <sup>R</sup>*<sup>n</sup>* is the state vector of the NN, **w** = h *w* 1 *<sup>j</sup> w* 2 *j* . . . *w n j* i> <sup>∈</sup> <sup>R</sup>*L*×*<sup>n</sup>* is the synaptic weights vector, with *L* representing the total number of weights used to identify the plant behavior, **<sup>A</sup>** <sup>=</sup> diag<sup>h</sup> −*a* 1 *<sup>j</sup>* − *a* 2 *j* . . . − *a n j* i <sup>∈</sup> <sup>R</sup>*n*×*<sup>n</sup>* is the parameter (Hurwitz) matrix, and *ψ* is the regressor vector. In this work, *ψ*(*χ*) = *e* (−*χ* <sup>2</sup>/*β*) cos(*λχ*) is the Morlet wavelet function used here as the activation function where *χ* is the state from the plant (Equation (3)) and the parameters *β*, *λ* are the expansion and dilation terms [22], respectively. Thus, the nonlinear system (3) is identified online by using an RWFONN (Equation (5)), where the synaptic weights vector **w** is adjusted via a filtered error algorithm.

#### *2.4. Filtered Error Training Algorithm*

For the application of the filtered error algorithm, this study begins under the assumption of an unknown system (3), which can be modeled (identified) by using an RWFONN structure (5). The synaptic weights are adjusted according to the following update law [23,24]

$$
\omega^i\_{jk} = -\Gamma^i\_{jk}\psi^i\_{jk}\mathfrak{xi}^i\_{jk\prime} \tag{6}
$$

where <sup>Γ</sup> <sup>∈</sup> <sup>R</sup>*L*×*<sup>L</sup>* is a definite positive gain matrix, *ψ* is the Morlet wavelet activation function, and *ξ* is the identification error, i.e., the difference between the states from the RWFONN (Equation (5)) and those from the plant (3), namely, *ξ i <sup>j</sup>* = *x i <sup>j</sup>* − *χ i j* . The update law (Equation (6)) corresponds to the filtered error training algorithm [23]. Since the RWFONN is given in a generalized way from the structure for a RHONN, the following theorem establishes that identification and parametric errors for the RWFONN are bounded.

**Theorem 1.** *Consider the RWFONN model whose weights are adjustable according to Equation (6) for each i* = 1, 2, . . ., *n;*


#### **Proof.** See [24,25].

Appendix A presents the boundedness of the identification error *ξ i j* given by the synaptic weights *w i j* .

#### *2.5. Nonlinear Block Controllable Form*

One of the main characteristics of the block control linearization technique is to transform a nonlinear system into an equivalent form expressed by first-order subsystems consisting of *r* blocks, which can be solved independently one from each other. Such equivalent form is commonly known as the block controlled form whose representation is described by [26]

$$\begin{aligned} \dot{\mathbf{x}}\_{1} &= \mathbf{f}\_{1}(\mathbf{x}\_{1}, t) + \mathbf{B}\_{1}(\mathbf{x}\_{1}, t)\mathbf{x}\_{2} + \mathbf{g}\_{1}(\mathbf{x}\_{1}, t), \\ \dot{\mathbf{x}}\_{\hat{i}} &= \mathbf{f}\_{\hat{i}}(\bar{\mathbf{x}}\_{\hat{i}}, t) + \mathbf{B}\_{\hat{i}}(\bar{\mathbf{x}}\_{\hat{i}}, t)\mathbf{x}\_{\hat{i}+1} + \mathbf{g}\_{\hat{i}}(\bar{\mathbf{x}}\_{\hat{i}}, t), \hat{i} = \mathbf{2}, \dots, r-1, \\ \dot{\mathbf{x}}\_{r} &= \mathbf{f}\_{r}(\bar{\mathbf{x}}\_{r}, t) + \mathbf{B}\_{r}(\bar{\mathbf{x}}\_{r}, t)\mathbf{u} + \mathbf{g}\_{r}(\bar{\mathbf{x}}\_{r}, t), \end{aligned} \tag{7}$$

where **f**(**x**, *t*) is a smooth and bounded mapping, **x** = [*x*<sup>1</sup> *x*2. . . *xr*] > is the state vector decomposed, **¯xˆi** = - *x*<sup>1</sup> . . . *x*ˆ*<sup>i</sup>* > , for <sup>ˆ</sup>*<sup>i</sup>* <sup>=</sup> 2, . . .,*r*, **<sup>x</sup>**ˆ*<sup>i</sup>* is a *n*ˆ*<sup>i</sup>* × 1 vector, and the subscripts (*n*1, *n*2, . . ., *nr*) define the structure of the system. The matrix **B**ˆ*<sup>i</sup>* , since the fictitious **x**ˆ*i*+<sup>1</sup> for each <sup>ˆ</sup>*i*-th block has full rank, rank(**B**ˆ*<sup>i</sup>* ) = *n*ˆ*<sup>i</sup>* , <sup>∀</sup>**<sup>x</sup>** <sup>∈</sup> *<sup>X</sup>* <sup>⊂</sup> <sup>R</sup>*<sup>n</sup>* and *<sup>t</sup>* <sup>∈</sup> [0, <sup>∞</sup>], ˆ*i* = 1, . . .,*r*. Therefore, in this work, the block control technique is applied to the NN structure (5) to define a sliding surface, which is the argument of the super-twisting controller.

#### *2.6. Super-Twisting Control Algorithm*

The first-order sliding mode has long been used as a robust control technique to cancel nonlinear terms and uncertainties due to external disturbances in a system, but its main disadvantage is the presence of the chattering effect, which induces vibrations in the controlled system. In order to reduce the chattering effect, high-order sliding modes have been used. A particular case is the super-twisting algorithm (STA), which has been playing a very important role in sliding mode controllers. The super-twisting control (STC) algorithm of second-order can be applied to systems where the control appears in the first derivative of the sliding surface. The application of the STC allows to cancel perturbations and nonlinearities, forcing the sliding surface toward zero in finite time. Furthermore, the states that define the sliding surface are directed to zero with asymptotic motion. The dynamics of the sliding surface are known as sliding mode movement, and it is characterized by being a reduced-order system [27,28].

The sliding surface for the NSTC is defined from the nonlinear system once the latter resembles the block controllable form (7) for the variable on which the control input of the system appears. Therefore, the sliding surface is [29]

$$\mathbf{s} = \mathbf{x}\_{\mathbf{r}} \tag{8}$$

where the dynamics of the sliding surface (Equation (8)), involving the STC algorithm and the system (7), take the form

$$\begin{aligned} \dot{\mathbf{s}} &= \mathbf{f}\_r(\bar{\mathbf{x}}\_{r\prime}t) - \mathbf{B}\_r(\bar{\mathbf{x}}\_{r\prime}t)(\lambda\_s|\mathbf{s}|^{\frac{1}{2}}\text{sign}(\mathbf{s}) + \mathbf{u}\_s) + \mathbf{g}\_r(\bar{\mathbf{x}}\_{r\prime}t), \\ \dot{\mathbf{u}}\_s &= \mathbf{a}\_s \text{sign}(\mathbf{s}), \end{aligned} \tag{9}$$

with *λ<sup>s</sup>* = *λs*<sup>11</sup> 0 <sup>0</sup> *<sup>λ</sup>s*22 and *α<sup>s</sup>* = *αs*<sup>11</sup> 0 <sup>0</sup> *<sup>α</sup>s*22 diagonal matrices. Then, the super-twisting control law provides finite-time convergence to zero, simultaneously **s** and **˙s**, where the term *λ<sup>s</sup>* |**s**| 1 <sup>2</sup> sign(**s**) commutes its gain at high frequency for giving robustness to annul the nonlinearities **f***r*(**¯x***<sup>r</sup>* , *t*), and to cancel the external disturbances **g***<sup>r</sup>* (**¯x***<sup>r</sup>* , *t*), while the term **u***<sup>s</sup>* reduces the chattering effect and the steady state error [28].

In order to ensure the stability of the system with the super-twisting control law (Equation (9)), the control matrix gains *λ* and *α* are holding to the following restrictions:

$$k\_i > 2\delta\_{i\prime} \quad k\_i > \frac{1}{2} \frac{k\_i^2(\delta\_i - k\_i)}{k\_i - 2\delta\_i} \tag{10}$$

where the disturbance norm is defined as |**f***r*(**¯x***<sup>r</sup>* , *t*) + **g***<sup>r</sup>* (**¯x***<sup>r</sup>* , *t*)| = *δ*|*s*| 1/2, for *δ* > 0. The inequalities in Equation (10) are defined in the stability analysis presented in Appendix B.

#### **3. Proposed Methodology**

This section describes the procedure for emulating the FESS through a group motorgenerator formed by two electrical machines mechanically coupled. The angular velocity of the PMSM, when it is working in motor mode, is controlled in order to emulate the storing of energy, whereas the velocity for the DC-machine is controlled emulating the discharge of the flywheel where the PMSM is now working in generator mode. The FESS's emulator scheme is shown in Figure 1. The procedure followed to the emulation of the FESS is explained next:


In this work, the sliding surface defined in Equation (8) is designed through the transformation of the RWFONN to the block controllable form involving the dynamics of the tracking error, such that this transformation resembles the structure described in Equation (7). By applying the super-twisting algorithm to the system (7) and using the RWFONN (Equation (5)), trained with the filtered error halgorithm (6), the result is the NSTC system, which is uniformly ultimately bounded (UUB). The stability proof is carried out in Appendix B.

Note that from Figure 1, the dotted blue block contains the motor-side converter gray block, where such block represents a future real-time implementation, while the dotted red block contains the gray blocks of the SVPWM and 3-phase inverter, which also represent a future real-time implementation.

**Figure 1.** FESS's emulator scheme.

Additionally, Figure 2 shows a general flowchart of the proposed algorithm applied to the control of both electrical machines, namely, the PMSM and the DC-motor. Both identification and control processes are indicated separately. Note that in this figure, there are particular notations for the PMSM (*ξ<sup>i</sup>* , *x*1, **u**1*<sup>s</sup>* , and *ε*1) and for the DC-motor (*ξidc*, *x*1*dc*, *usdc*, and *ζ*1). The particular notations and the processes of identification and control will be explained in the following subsections.

**Figure 2.** General flowchart of the proposed control algorithm.

#### *3.1. Flywheel Storage System*

In this section, the mathematical analysis of the stored kinetic energy from the flywheel, emulated by the DC-motor, is developed. Under the assumption that the mass is uniformly distributed and the axis of rotation passes through the mass center, the mass moment of inertia for a solid disk is given by [31]

$$J\_m = \frac{1}{2}mr^2,\tag{11}$$

where *J<sup>m</sup>* is the mass moment of inertia and *m*, *r* are the mass and radius of the disk, respectively. Then, for the numerical calculation of the mass moment of inertia of the disk, when considering *m* = 2 kg and *r* = 0.3 m, Equation (11) yields

$$J\_m = 0.09 \text{ kg} \cdot \text{m}^2.\tag{12}$$

Now, the kinetic energy storage equation is calculated when substituting Equation (12) into (1). Therefore, considering the angular velocity as *ω<sup>m</sup>* = 130 rad/s, the energy stored in the flywheel results

$$E\_k = 760.5 \text{ J} \tag{13}$$

when considering that the deceleration caused by the kinetic energy stored in the flywheel is emulated by the DC-motor and its drive. To calculate the deceleration of the emulated flywheel, from the knowledge of the mass moment of inertia in Equation (12), replacing *J<sup>m</sup>* in Equation (2), it is then possible to calculate the reference angular velocity *ωmre f* for the DC-motor and, consequently, to be able to apply the NSTC.

#### *3.2. Nonlinear Identification and NSTC Applied to PMSM*

To convert the electrical energy applied to the PMSM into kinetic energy, the PMSM is activated in motor mode with the inputs *v<sup>d</sup>* and *vq*. In the following subsections, the neural identification and the NSTC applied to the PMSM in motor mode are presented.

#### 3.2.1. PMSM Neural Identification

In this work, the mathematical model of the PMSM is taken from [30], which consists of a third order nonlinear system given in the general form (3), which is modeled in a *dq* coordinated frame rotating at the rotor velocity. Then, the RWFONN with structure (5), proposed to identify the dynamic system of the PMSM mathematical model, is defined as

$$\begin{aligned} \dot{\chi}\_1 &= -a\_1 \mathfrak{x}\_1 + b\_1 w\_1 \psi\_1(\chi\_1) + \mathfrak{x}\_2 + \mathfrak{x}\_3, \\ \dot{\mathfrak{x}}\_2 &= -a\_2 \mathfrak{x}\_2 + b\_2 w\_2 \psi\_2(\chi\_2) + v\_{d\prime} \\ \dot{\mathfrak{x}}\_3 &= -a\_3 \mathfrak{x}\_3 + b\_3 w\_3 \psi\_3(\chi\_3) + v\_{q\prime} \end{aligned} \tag{14}$$

where the RWFONN's states *x*1, *x*2, and *x*<sup>3</sup> identify the states *ω<sup>r</sup>* = *χ*<sup>1</sup> (angular velocity), *i<sup>d</sup>* = *χ*<sup>2</sup> (*d*-current), and *i<sup>q</sup>* = *χ*<sup>3</sup> (*q*-current) from the PMSM, respectively. *a*1, *a*2, *a*3, *b*1, *b*2, and *b*<sup>3</sup> are positive fixed parameters; *w*1, *w*2, *w*<sup>3</sup> are the synaptic weights; *ψ*1(*χ*1) = *e* (−*χ* 2 1 /*β*<sup>1</sup> ) cos(*λ*1*χ*1), *ψ*2(*χ*2) = *e* (−*χ* 2 2 /*β*2) cos(*λ*2*χ*2), and *ψ*3(*χ*3) = *e* (−*χ* 2 3 /*β*3) cos(*λ*3*χ*3) are Morlet wavelet activation functions. The RWFONN's inputs are the same as PMSM, *v<sup>d</sup>* and *vq*. The filtered error algorithm used for training the RWFONN (Equation (14)) is performed as *w*<sup>1</sup> = *γ*1*ψ*1(*χ*1)*ξ*1, *w*<sup>2</sup> = *γ*2*ψ*2(*χ*2)*ξ*<sup>2</sup> and *w*<sup>3</sup> = *γ*3*ψ*3(*χ*3)*ξ*3, with *γ*1, *γ*2, *γ*<sup>3</sup> adjustable parameters, where *ξ*<sup>1</sup> = *x*<sup>1</sup> − *χ*1, *ξ*<sup>2</sup> = *x*<sup>2</sup> − *χ*<sup>2</sup> and *ξ*<sup>3</sup> = *x*<sup>3</sup> − *χ*<sup>3</sup> are the respective identification errors. According to Equation (5), *i* = *j* = *k* = 1, 2, 3.

It should be noticed that the proposed structure (14) has only one neuron for each state, which represents a low computational burden in contrast with that from structures for high-order neural networks (HONNs). The parameters from the PMSM mathematical model are mentioned in [30], and the values are shown in Table 1 [32].

#### 3.2.2. NSTC Application to PMSM

In order to control the angular velocity *ω<sup>r</sup>* of the PMSM, a first step toward that goal consists of transforming the RWFONN (Equation (14)) into the form (7) by using a recursive transformation of the tracking error *ε<sup>i</sup>* [29]. For this, the structure of the system (14) is rewritten in the vector form

$$\begin{aligned} \dot{\mathbf{x}}\_1 &= -a\_1 \mathbf{x}\_1 + b\_1 w\_1 \psi\_1(\chi\_1) + \mathbf{a}\_1 \mathbf{x}\_2, \\ \dot{\mathbf{x}}\_2 &= \mathbf{A}\_2 \mathbf{x}\_2 + \mathbf{A}\_3 \mathbf{b}\_1 + \mathbf{B}\_2 \mathbf{v}\_2 \end{aligned} \tag{15}$$

with **a**<sup>1</sup> = - 1 1 , **A**<sup>2</sup> = −*a*<sup>2</sup> 0 0 −*a*<sup>3</sup> , **A**<sup>3</sup> = *b*2*w*2*ψ*2(*χ*2) 0 0 *b*3*w*3*ψ*3(*χ*3) , **b**<sup>1</sup> = - 1 1<sup>&</sup>gt; , **B**<sup>2</sup> = 1 0 0 1 , **x**<sup>2</sup> = - *x*<sup>2</sup> *x*<sup>3</sup> > , and **v** = - *v<sup>d</sup> v<sup>q</sup>* > . Now, by applying the recursive transformation to the structure (15) to resemble it into the block control form, the tracking error *ε*<sup>1</sup> is defined by

$$
\varepsilon\_1 = \omega\_{rref} - \mathfrak{x}\_{1\prime} \tag{16}
$$

where *ωrre f* is the reference angular velocity to the PMSM, and *x*<sup>1</sup> represents the state of the neural network (15). Therefore, the tracking error dynamics are given as

$$
\dot{\varepsilon}\_1 = \dot{\omega}\_{ref} - \dot{\mathfrak{x}}\_1. \tag{17}
$$

**Table 1.** Values and parameters of PMSM [32] and DC-motor [33].


Substituting *x*˙<sup>1</sup> Equation (15) into (17), and assigning the desired dynamics *k*1*ε*1, it yields

$$
\dot{\varepsilon}\_1 = \dot{\omega}\_{rref} - \left(-a\_1 \mathbf{x}\_1 + b\_1 w\_1 \psi\_1(\chi\_1) + \mathbf{a}\_1 \mathbf{x}\_2\right) = -k\_1 \varepsilon\_1 \tag{18}
$$

where *k*<sup>1</sup> is adjusted to vanish the tracking error. In this way, it is possible to synthesize a control law through the vector state **x**2, where the reference vector is defined as

$$\mathbf{x}\_{2ref} = \mathbf{a}\_1^{\text{pinv}} [\dot{\omega}\_{mef} + a\_1 \omega\_{mef} - a\_1 \varepsilon\_1 - b\_1 w\_1 \psi\_1(\chi\_1) + k\_1 \varepsilon\_1]. \tag{19}$$

where the superscript "pinv" denotes the pseudo-inverse. Furthermore, the second error in vector form is defined as

$$
\varepsilon\_2 = \mathbf{x}\_{2ref} - \mathbf{x}\_2 \tag{20}
$$

with dynamics

$$
\dot{\varepsilon}\_2 = \dot{\mathbf{x}}\_{2ref} - \dot{\mathbf{x}}\_2. \tag{21}
$$

Substituting Equation (15) into (21) yields

$$
\dot{\varepsilon}\_2 = \dot{\mathbf{x}}\_{2ref} - \mathbf{A}\_2 \mathbf{x}\_{2ref} + \mathbf{A}\_2 \varepsilon\_2 - \mathbf{A}\_3 \mathbf{b}\_1 - \mathbf{B}\_2 \mathbf{v}. \tag{22}
$$

Moreover, using Equations (18) and (22), the representation in the block control form from Equation (15), in terms of the tracking error *ε<sup>i</sup>* , is obtained as

$$\begin{aligned} \dot{\varepsilon}\_{1} &= -k\_{1}\varepsilon\_{1} + \mathbf{a}\_{1}\mathbf{e}\_{2} \\ \dot{\varepsilon}\_{2} &= \mathbf{a}\_{1}^{\text{pinv}} \left[ \dot{\boldsymbol{\omega}}\_{ref} + a\_{1}\boldsymbol{\omega}\_{ref} - b\_{1}\boldsymbol{w}\_{1}\boldsymbol{\psi}\_{1}(\boldsymbol{\chi}\_{1}) \right] \\ &- \mathbf{A}\_{2} \left( \mathbf{a}\_{1}^{\text{pinv}} \left[ \dot{\boldsymbol{\omega}}\_{ref} + a\_{1}\boldsymbol{\omega}\_{ref} - b\_{1}\boldsymbol{w}\_{1}\boldsymbol{\psi}\_{1}(\boldsymbol{\chi}\_{1}) \right] \right) \\ &+ 0.5 \begin{bmatrix} a\_{1}k\_{1} - k\_{1}^{2} + a\_{1}a\_{2}k\_{1} - a\_{2}k\_{1}^{2} \\ a\_{1}k\_{1} - k\_{1}^{2} + a\_{1}a\_{3}k\_{1} - a\_{3}k\_{1}^{2} \end{bmatrix} \boldsymbol{\varepsilon}\_{1} + \mathbf{A}\_{2}\boldsymbol{\varepsilon}\_{2} - \mathbf{A}\_{3}\mathbf{b}\_{1} - \mathbf{B}\_{2}\mathbf{v}\_{1} \end{aligned} \tag{23}$$

Furthermore, defining the sliding surface **s** = *ε*2, with the relative degree from Equation (23) equals to 1, and applying the NSTC Equation (9) in (23) results

$$\begin{aligned} \dot{\varepsilon}\_{1} &= -k\_{1}\varepsilon\_{1} + \mathbf{a}\_{1}\varepsilon\_{2}, \\ \dot{\varepsilon}\_{2} &= \mathbf{a}\_{1}^{\mathrm{pinv}} \left[ \dot{\boldsymbol{\omega}}\_{\mathrm{ref}f} + a\_{1}\dot{\boldsymbol{\omega}}\_{\mathrm{ref}f} - b\_{1}\boldsymbol{w}\_{1}\boldsymbol{\psi}\_{1}(\boldsymbol{\chi}\_{1}) \right] \\ &- \mathbf{A}\_{2} \left( \mathbf{a}\_{1}^{\mathrm{pinv}} \left[ \dot{\boldsymbol{\omega}}\_{\mathrm{ref}f} + a\_{1}\dot{\boldsymbol{\omega}}\_{\mathrm{ref}f} - b\_{1}\boldsymbol{w}\_{1}\boldsymbol{\psi}\_{1}(\boldsymbol{\chi}\_{1}) \right] \right) \\ &+ 0.5 \begin{bmatrix} a\_{1}\mathbf{k}\_{1} - \mathbf{k}\_{1}^{2} + a\_{1}a\_{2}\mathbf{k}\_{1} - a\_{2}\mathbf{k}\_{1}^{2} \\ a\_{1}\mathbf{k}\_{1} - \mathbf{k}\_{1}^{2} + a\_{1}a\_{3}\mathbf{k}\_{1} - a\_{3}\mathbf{k}\_{1}^{2} \end{bmatrix} \boldsymbol{\varepsilon}\_{1} + \mathbf{A}\_{2}\boldsymbol{\varepsilon}\_{2} - \mathbf{A}\_{3}\mathbf{b}\_{1} - \mathbf{B}\_{2}(\boldsymbol{\lambda}\_{s}|\mathbf{s}|^{\frac{1}{2}}\text{sign}(\mathbf{s}) + \mathbf{u}\_{1s}), \end{aligned} \tag{24}$$

where **u**1*<sup>s</sup>* represents the control vector. Note that in Equation (23), the control input vector **v** = - *v<sup>d</sup> v<sup>q</sup>* > is replaced by the STC resulting Equation (24) under the fact that the tracking error variables *ε*<sup>1</sup> and *ε*<sup>2</sup> are decoupled. This latter will guarantee that the tracking errors be steered to zero in finite time. From Equation (24), selecting the sliding surface **s** = **x<sup>2</sup>** arises the system given by [29]

$$\begin{split} \mathbf{s} &= \mathbf{a}\_1^{\text{pinv}} \left[ \dot{\omega}\_{m\ell} + a\_1 \dot{\omega}\_{m\ell} - b\_1 w\_1 \psi\_1(\chi\_1) \right] \\ &- \mathbf{A}\_2 \left( \mathbf{a}\_1^{\text{pinv}} \left[ \dot{\omega}\_{m\ell} + a\_1 \dot{\omega}\_{m\ell} - b\_1 w\_1 \psi\_1(\chi\_1) \right] \right) \\ &+ 0.5 \begin{bmatrix} a\_1 k\_1 - k\_1^2 + a\_1 a\_2 k\_1 - a\_2 k\_1^2 \\ a\_1 k\_1 - k\_1^2 + a\_1 a\_3 k\_1 - a\_3 k\_1^2 \end{bmatrix} \varepsilon\_1 + \mathbf{A}\_2 \varepsilon\_2 - \mathbf{A}\_3 \mathbf{b}\_1 - \mathbf{B}\_2(\lambda\_s |\mathbf{s}|^{\frac{1}{2}} \text{sign}(\mathbf{s}) + \mathbf{u}\_{1s}), \\ &\varepsilon\_1 - \varsigma\_1 \text{sign}(\mathbf{s}) \end{split} \tag{25}$$

**˙u**1*<sup>s</sup>* =*α<sup>s</sup>* sign(**s**).

where Equation (25) is the NSTC applied to the PMSM through the RWFONN structure.

#### *3.3. Nonlinear Identification and NSTC Applied to DC-Motor*

In order to convert the kinetic energy stored by the flywheel into electrical energy, the PMSM must be operated in generator mode. The discharge of the kinetic energy is controlled by the angular velocity of the DC-motor, this can be achieved by decelerating the angular velocity taking into account the reference velocity generated by Equation (2).

#### 3.3.1. DC-Motor Neural Identification

The mathematical model of the DC-motor is taken from [16], which consists of a three state nonlinear system with structure (3). Note that due to the linearity between the effective field current and its constant magnetizing flux, the mathematical model of the DC-motor becomes a linear system of two differential equations. Therefore, the RWFONN here proposed, following the structure (5), to identify the dynamics of the DC-motor mathematical model is defined as

$$\begin{aligned} \dot{\mathfrak{x}}\_{1dc} &= -a\_{1dc}\mathfrak{x}\_{1dc} + b\_{1dc}w\_{1dc}\psi\_{1dc}(\chi\_{1dc}) + \mathfrak{x}\_{2dc} \\ \dot{\mathfrak{x}}\_{2dc} &= -a\_{2dc}\mathfrak{x}\_{2dc} + b\_{2dc}w\_{2dc}\psi\_{2dc}(\chi\_{2dc}) + \mathfrak{u}\_{adc} \end{aligned} \tag{26}$$

where the RWFONN's states *x*1*dc* and *x*2*dc* identify the states *ω<sup>m</sup>* = *χ*1*dc* (angular velocity) and *i<sup>a</sup>* = *χ*2*dc* (armature current) from the DC-motor, respectively. *a*1*dc*, *a*2*dc*, *b*1*dc*, and *b*2*dc* are positive constant parameters; *w*1*dc* and *w*2*dc* are the synaptic weights; *ψ*1*dc*(*χ*1*dc*) = *e* (−*χ* 2 <sup>1</sup>*dc*/*β*1*dc* ) cos(*λ*1*dcχ*1*dc*) and *ψ*2*dc*(*χ*2*dc*) = *e* (−*χ* 2 <sup>2</sup>*dc*/*β*2*dc* ) cos(*λ*2*dcχ*2*dc*) are Morlet wavelet activation functions. The RWFONN's input *uadc* is the same as *u<sup>a</sup>* (armature voltage) to the DC-motor. The filtered error algorithm used for training the RWFONN (Equation (26)) is performed through *w*1*dc* = *γ*1*dcψ*1*dc*(*χ*1*dc*)*ξ*1*dc* and *w*2*dc* = *γ*2*dcψ*2*dc*(*χ*2*dc*)*ξ*2*dc* with *γ*1*dc*, *γ*2*dc* adjustable parameters, and *ξ*1*dc* = *x*1*dc* − *χ*1*dc*, *ξ*2*dc* = *x*2*dc* − *χ*2*dc* are the respective identification errors. From Equation (5), *i* = *j* = *k* = 1, 2.

It must be noticed that the proposed RWFONN (Equation (26)) has only one neuron for each state, in a similar way as in Equation (14), which represents a low computational burden in contrast with HONN. The parameters of the DC-motor mathematical model are mentioned in [16], whose values are shown in Table 1.

#### 3.3.2. NSTC Application to DC-Motor

By following the methodology as that in Section 3.2.2, the recursive transformation is now applied to the structure (26), where the generation of the tracking error *ζ*<sup>1</sup> is defined by

$$\mathcal{J}\_1 = \omega\_{mref} - \mathfrak{x}\_{1dc} \tag{27}$$

where *ωmre f* represents the reference angular velocity of the DC-motor and *x*1*dc* is the state variable from Equation (26). The dynamics of the tracking error *ζ*<sup>1</sup> is given by

$$
\not\subsetneq \omega\_{mref} - \dot{\mathfrak{x}}\_{1dc}.\tag{28}
$$

By substituting Equation (26) into (28) yields

$$\dot{\mathcal{L}}\_1 = \dot{\omega}\_{mref} - (-a\_{1dc}\mathbf{x}\_{1dc} + b\_{1dc}w\_{1dc}\psi\_{1dc}(\chi\_{1dc}) + \mathbf{x}\_{2dc}) = -k\_{1dc}\zeta\_{1\prime} \tag{29}$$

where *k*1*dcζ*<sup>1</sup> constitutes the desired dynamics to vanish the tracking error. In this way, it is possible to synthesize a control law through the state *x*2*dc*, so, to such an end consider

$$
\omega\_{2dcref} = \dot{\omega}\_{mref} + a\_{1dc}\omega\_{mref} - a\_{1dc}\tilde{\zeta}\_1 - b\_{1dc}w\_{1dc}\psi\_{1dc}(\chi\_{1dc}) + k\_{1dc}\tilde{\zeta}\_1. \tag{30}
$$

Furthermore, the second error is defined as

$$
\mathcal{L}\_2 = \mathfrak{x}\_{2dcref} - \mathfrak{x}\_{2dcr} \tag{31}
$$

with error dynamics

$$
\dot{\mathcal{L}}\_2 = \dot{\mathfrak{x}}\_{2dcref} - \dot{\mathfrak{x}}\_{2dc}.\tag{32}
$$

By substituting Equation (26) in (32) it yields

$$\dot{\zeta}\_2 = \dot{\varkappa}\_{2dcref} - \left(-a\_{2dc}\varkappa\_{2dc} + b\_{2dc}w\_{2dc}\psi\_{2dc}(\chi\_{2dc}) + \mathfrak{u}\_{adc}\right). \tag{33}$$

Moreover, using Equations (29) and (33), the block control form for Equation (26) is defined as

$$\begin{aligned} \dot{\zeta}\_1 &= -k\_{1dc}\zeta\_1 + \zeta\_2, \\ \dot{\zeta}\_2 &= \dot{\chi}\_{2dcref} + a\_{2dc}\chi\_{2dc} - b\_{2dc}w\_{2dc}\psi\_{2dc}(\chi\_{2dc}) - u\_{adc}. \end{aligned} \tag{34}$$

Thus, based on the error dynamics *ζ<sup>i</sup>* , the Equation (34) represents the block control transformation with structure (26). Furthermore, from the system (26) with relative degree equals to 1, the NSTC is given by

$$\begin{split} \dot{\zeta}\_{1} &= -k\_{1dc}\zeta\_{1} + \zeta\_{2} \\ \dot{\zeta}\_{2} &= \dot{\omega}\_{mref} + (-a\_{1dc} + a\_{2dc})\dot{\omega}\_{mref} + (a\_{1dc}a\_{2dc})\omega\_{mref} \\ &- (a\_{1dc}k\_{1dc} + k\_{1}^{2} + a\_{1dc}a\_{2dc} - a\_{2dc}k\_{1dc})\zeta\_{1} - a\_{2dc}\zeta\_{2} - a\_{2dc}b\_{1dc}w\_{1dc}\psi\_{1dc}(\chi\_{1dc}) - \\ &b\_{2dc}w\_{2dc}\psi\_{2dc}(\chi\_{2dc}) - (\lambda\_{\text{sdc}}|s\_{\text{dc}}|\frac{1}{2}\text{sign}(s\_{\text{dc}}) + u\_{\text{sdc}}) \end{split} \tag{35}$$

It should be noticed from Equation (35), regarding the tracking error variables *ζ*<sup>1</sup> and *ζ*2, that the control input *usdc* = *uadc* guarantees that these errors will be steered to zero in finite time.

#### **4. Simulation Results**

This work presents the simulation results of the NSTC for controlling the angular velocity of a PMSM and a DC-motor to emulate an FESS. The angular velocity of the PMSM is controlled to store kinetic energy in the flywheel, meanwhile the angular velocity of the DC-machine is controlled to emulate the deceleration of the flywheel by transferring kinetic energy to the PMSM working in generator mode, where it is converted into electrical energy and delivered to the utility grid. The used parameters of the motors are shown in Table 1. The simulation was performed using Matlab/Simulink (*MatlabTM*) with a Runge–Kutta solver and a step size of 100 µs.

#### *4.1. Neural Identification*

This subsection presents the simulation results of the neural identification of the PMSM state variables, as well as the DC-motor state variables using the RWFONN structure.

#### 4.1.1. PMSM States Identification

For the neural identification of the PMSM states, the parameter values are: *a*<sup>1</sup> = 6000, *a*<sup>2</sup> = *a*<sup>3</sup> = 4000 and *b*<sup>1</sup> = 6000, *b*<sup>2</sup> = *b*<sup>3</sup> = 4000; the parameters for the Morlet Wavelet activation functions are: *<sup>β</sup>*<sup>1</sup> <sup>=</sup> <sup>75</sup> <sup>×</sup> <sup>10</sup><sup>3</sup> , *<sup>β</sup>*<sup>2</sup> <sup>=</sup> <sup>22</sup> <sup>×</sup> <sup>10</sup><sup>3</sup> , *<sup>β</sup>*<sup>3</sup> <sup>=</sup> <sup>35</sup> <sup>×</sup> <sup>10</sup><sup>3</sup> , *λ*<sup>1</sup> = *λ*<sup>2</sup> = *<sup>λ</sup>*<sup>3</sup> <sup>=</sup> 0.001; the parameters of the filtered error are: *<sup>γ</sup>*<sup>1</sup> <sup>=</sup> *<sup>γ</sup>*<sup>2</sup> <sup>=</sup> *<sup>γ</sup>*<sup>3</sup> <sup>=</sup> <sup>855</sup> <sup>×</sup> <sup>10</sup><sup>2</sup> . In Figures 3–5, the neuronal identification of the PMSM states is shown, where, in Figure 3 the identification of the angular velocity is the red dashed line that represents the angular velocity of the PMSM (*ωr*) and the blue continuous line represents the state of RWFONN (*x*1). In order to show the convergence of these results, the initial conditions are given as: *ω<sup>r</sup>* = 0 and *x*<sup>1</sup> = −1. Note that in the detail of the figure, the identification convergence is given in 0.0002 s, approximately. The transient is shown when the PMSM is started in the energy storage, where the velocity response has a settling time of 0.15 s, approximately.

The identification of current *i<sup>d</sup>* is presented in Figure 4, where the red dashed line represents the current of the PMSM (*i<sup>d</sup>* ) and the blue continuous line represents the state of RWFONN (*x*2); the initial conditions are given as: *i<sup>d</sup>* = 0 and *x*<sup>2</sup> = 0.5. The identification convergence is given in 0.0003 s approximately, as is shown in the detail of the Figure 4. In the transient response, the inrush current *i<sup>d</sup>* achieves a maximum value of the 132 A with an oscillation frequency of 0.0014 Hz. As it is shown, the identification process is effectively realized.

In Figure 5 the identification of current *q* is shown, where the red dashed line represents the current *i<sup>q</sup>* of the PMSM, and the blue continuous line represents the state of the

RWFONN (*x*3), and where the initial conditions are given as: *i<sup>q</sup>* = 0 and *x*<sup>3</sup> = −0.5. The identification convergence is given in 0.0003 s, approximately. Figure 5 shows the transient when the PMSM is started, and in the process of energy storage, the inrush current *i<sup>q</sup>* achieves a maximum value of the 153 A with an oscillation frequency of 0.0489 Hz. In the transient, the identification procedure is more demanding and is effectively realized.

**Figure 3.** Angular velocity and neural identification behavior: *ω<sup>r</sup>* and *x*<sup>1</sup> .

**Figure 4.** Current and neural identification behavior: *i<sup>d</sup>* and *x*2.

**Figure 5.** Current and neural identification behavior: *i<sup>q</sup>* and *x*3.

4.1.2. DC-motor States Identification

In this section, the neuronal identification of the DC-motor states is carried out, where the values of the parameters used in the RWFONN are defined as follows: *a*1*dc* = *a*2*dc* = 6000 and *b*1*dc* = *b*2*dc* = 6000; the parameters for the Morlet Wavelet activation functions are: *<sup>β</sup>*1*dc* <sup>=</sup> <sup>85</sup> <sup>×</sup> <sup>10</sup><sup>3</sup> , *<sup>β</sup>*2*dc* <sup>=</sup> <sup>75</sup> <sup>×</sup> <sup>10</sup><sup>3</sup> , *λ*1*dc* = 0.001, *λ*2*dc* = 0.01; the parameters of the filtered error are: *<sup>γ</sup>*1*dc* <sup>=</sup> *<sup>γ</sup>*2*dc* <sup>=</sup> <sup>855</sup> <sup>×</sup> <sup>10</sup><sup>2</sup> .

In Figure 6, the neural identification of the angular velocity of the DC-motor is shown. In this figure, *ω<sup>m</sup>* is a state variable of the motor, and *x*1*dc* is a state variable of the NN. As it can be seen in the detail of the figure, the NN identifies the state of the motor with a convergence time of 0.0003 s. The initial conditions are: *ω<sup>m</sup>* = 0 and *x*1*dc* = 1. The figure shows the transient when the DC-motor is starting. The velocity response in the transient has an overshoot of 1.13% and a settling time of 0.05 s.

**Figure 6.** Angular velocity and neural identification behavior: *ω<sup>m</sup>* and *x*1*dc*.

Figure 7 shows the neural identification of the armature current of DC-motor, where *i<sup>a</sup>* is the current of the motor and *x*2*dc* is the state of the network; the initial conditions are: *i<sup>a</sup>* = 0 and *x*2*dc* = 0.5, and obtaining the neural identification with a time of convergence 0.0004 s, approximately. In this figure, the transient presents the typical inrush current of a DC-motor when it is fed with nominal voltage. The current has a maximum value of 21.7 A and achieves the steady-state value of 0.53 A at 0.08 s.

**Figure 7.** Armature current and neural identification behavior: *i<sup>a</sup>* and *x*2*dc*.

In order to make a comparison of the performance of the RWFONN used in this work, numerical simulation results about the identification process from both Recurrent Sigmoid First-Order Neural Network (RSFONN) and RHONN have been obtained. This comparison consists of using the RHONN and RSFONN structures, instead of that for the RWFONN, to the dynamics identification of the plant, training them via a filtered error algorithm. Then, from the data obtained by the identification process, the root mean square (RMS) error is computed, which represents the difference between the actual state and the estimated state when using approaches with different NNs. The comparison is made by estimating the states from both PMSM and DC-motor, where Tables 2 and 3 show the RMS

for each state from both motors, respectively. It can be seen in Table 2 that the lowest RMS values (bold values) are for *ω<sup>r</sup>* and *i<sup>d</sup>* from the RSFONN and for *i<sup>q</sup>* from the RHONN. In Table 3, the results show that the lowest RMS values (bold values) are for *ω<sup>m</sup>* from the RSFONN and for *i<sup>a</sup>* from the RWFONN.

**Table 2.** RMS error for PMSM.


**Table 3.** RMS error for DC-motor.


Even when different NNs obtain the lowest RMS in the identification process, there are different reasons to consider when choosing the NN structure. Such is the case with the RWFONN, having a very single structure, for which it is easy to apply the STA, contrary to the case with the RHONN or even the RSFONN. Another important thing is that the computational burden is lower for simple structures such as the RWFONN but not for the RHONN or when increasing the high-order terms of the latter.

#### *4.2. PMSM and DC-motor Controller-Emulation of Flywheel*

In this subsection, the NSTC for PMSM and a DC-motor controller for emulating the complete system of the flywheel are presented. For the case of the PMSM, the controller is designed to control the angular velocity (*ωm*) indirectly through the states of the NN, where the parameters and gains for the NSTC of PMSM are defined as: *k*<sup>1</sup> = 8000, *λ*1*<sup>s</sup>* = 0.1, *λ*2*<sup>s</sup>* = 2.5, and *α*1*<sup>s</sup>* = 0.1, *α*2*<sup>s</sup>* = 2.5.

In Figure 8, the behavior of the PMSM in motor and generator modes is shown. In the case of motor mode, the velocity tracking performances from 0 to 10 and from 30 to 52 s are shown, where the blue dashed line represents *ωrre f* , and black continuous line is *ωm*. In this time, the tracking is to *ωrre f* = 130 rad/s, and there is kinetic energy storage during this operation mode. A detail of the transient is shown where a fast convergence can be seen. Due to consideration of a grid failure, the case of PMSM as a generator mode is plotted during the period of time from 10 to 30 and from 52 to 70 s. In this time, the deceleration of the flywheel is emulated by controlling the angular velocity of the DC-motor (see Figure 9 on the sign named Discharge of flywheel) and in this figure, the velocity reference *ωmre f* (blue dashed line) is described by Equation (2) and meanwhile, *ω<sup>m</sup>* is the gray continuous line; a detail is shown in the middle of the figure where the very good results of the tracking can be noted. To ensure the NSTC of the DC-motor, the parameters and gains are defined as: *k*1*dc* = 8000, *λsdc* = 3 and *αsdc* = 3.

**Figure 8.** PMSM velocity tracking to emulate the energy storage.

**Figure 9.** DC-motor velocity tracking to emulate the flywheel deceleration.

Figure 10 shows the PMSM velocity controller performance, where tracking velocity error *ε*<sup>1</sup> (Figure 10a) is steered to zero effectively by the super-twisting control law *v<sup>q</sup>* (Figure 10a). Note that the transient response at 30 s corresponds to the change from the generator to motor mode. The error diminishes to zero in 0.015 s, as can be seen in the detail of Figure 10a.

Figure 11 shows the DC-motor velocity controller performance. Figure 11a displays the tracking error signal in the intervals (10–30 s) and (52–70 s), which is steered to zero in finite time by the action of the super-twisting control law shown in Figure 11b. When the tracking error is zero, then the velocity control of the DC-motor is achieved, and the control goal is fulfilled, which consists of emulating the deceleration of the flywheel.

#### *4.3. Power Delivery*

In Figure 12, the kinetic energy stored in the flywheel is shown from 0 to 10 s and from 30 to 52 s due to the fact that PMSM is in motor mode, generating 760.5 J in steady-state. From 10 to 30 s and from 52 to 70 s, the PMSM is in generator mode, and it shows the discharge of the energy when considering a fault in the utility grid or when the flywheel is required to compensate for the utility grid with active power to solve a problem of the electrical office.

**Figure 10.** PMSM velocity controller performance: (**a**) velocity error *ε*<sup>1</sup> , (**b**) control signal *vq*.

**Figure 11.** DC-motor velocity tracking performance: (**a**) velocity error *ζ*<sup>1</sup> . (**b**) control signal *usdc* (10–30 and 52–70 s).

**Figure 12.** Storage and discharge kinetic energy.

#### **5. Discussion**

There exists a variety of three-phase electrical machines that can be implemented in an FESS, such as wound squirrel cage [34] and doubly-fed asynchronous machines [35]. In this work, we used the PMSM to emulate a flywheel storage system because, based on the literature, the small and medium capacity applications of the flywheels make it more convenient to use the PMSM since it is more efficient and less difficult to control. The doubly-fed induction generator is used more in large capacities because the control is carried out by the rotor circuit, where the power flow is proportional to the slip of the machine. Consequently, the electronic power drive is of lower capacity, near 30% of the flywheel capacity, which allows a considerable economic saving in the converter.

This proposal can be extended to other applications in energy storage systems such as power compensators in wind generators as well as wind diesel power systems, electric vehicles, direct electrical grid applications, i.e., applications where it is possible to involve charge/discharge cycles.

Regarding the advantages, this work can be used as a general methodology where it is possible to work as a motor/generator, such as wound squirrel cage, doubly fed asynchronous machines, and of course the PMSM. It is clear that this is possible by making the corresponding changes for the mathematical model of the machine to be used, which all depends on the range of speeds that it is going to operate at, and taking advantage of the benefits offered by the material with which the machine is built (low-speed FESS, or high-speed FESS [35]), and the particular features (electrical, mechanical) of the three-phase machine, etc. It is possible to control nonlinear and linear systems using the NSTC based on RWFONNs trained via a filtered error algorithm. These plants to be controlled are considered unknown regarding the parameters, and only the state-space variables are measured. This work can be extended to other three-phase machines, such as doubly fed asynchronous machines or wound squirrel cage motors.

The main limitation is that this work is an emulated system in simulation, and even when different real scenarios are considered, a real system will always show all the benefits of the proposal. Furthermore, this work is limited by the structure of the RWFONN, as even when the obtained numerical results are good, the computational process can be degraded.

The following interpretation of the results are made: the identification (Figures 3–7) results offer a good approximation of the plants by the RWFONN, and the filtered error algorithm that causes the tracking results in Figures 8 and 9 is good, including the tracking error maintained at zero for *ε*<sup>1</sup> and *ζ*1, Figures 10 and 11, respectively. This demonstrates that the proposed methodology is viable for use in the applications already mentioned. It is clear that they are considered ideal switching processes when the grid failure is presented and there is the change from motor to generator in the PMSM also in the DC-motor, such as

in the case in Figures 8 and 9, from which a good tracking is noted when there is a change in the storage–discharge modes.

The perspective of this work is obtaining results when considering more real scenarios in order to have a better approximation of a real-system. This will contribute to making real-time experiments with this proposal considering different structures of NN with the same or even other control algorithms.

#### **6. Conclusions**

Simulation results from this work show the emulation of the FESS through identification and control of the states from the models for two different electrical machines, namely, the PMSM and the DC-motor. This is achieved through the use of a RWFONN, the block control linearization technique, and the NSTC, by controlling the velocity of the PMSM in the energy storage stage and controlling the velocity of the DC-motor in the energy feedback stage. The stability analysis demonstrates the convergence of the whole system in a closed-loop using the boundedness of the identification error. It is worth mentioning that the authors intend to continue with the study of this work in real-time until laboratories can be accessed and tests can be performed.

**Author Contributions:** D.A.M. contributed with the design process of Neural Controller, methodology, validation and editing; C.E.C. contributed with conceptualization, review, and editing; F.J. contributed with conceptualization, review, and editing; O.A.M. validation of simulation results, review and editing. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Data Availability Statement:** Not applicable.

**Acknowledgments:** The first author acknowledges support from Consejo Nacional de Ciencia y Tecnologia (CONACYT) and Universidad de Guadalajara Centro Universitario de los Lagos.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **Abbreviations**

The following abbreviations are used in this manuscript:


#### **Appendix A. Identification Error Boundedness**

**Theorem A1.** *Suppose that the system (3) and the model (5) are initially at the same state* **x**(0) = *χ*(0)*; then, for any e* > 0 *and any finite T* > 0 *there exist an integer L and a matrix* **<sup>w</sup>**∗ ∈ <sup>R</sup>*L*×*<sup>n</sup> such that the state* **x**(*t*) *of the RWFONN model (5) and weight values* **w** = **w**∗ *satisfies*

$$\sup\_{0 \le t \le T} |\mathbf{x}(t) - \mathbf{x}(t)| \le \epsilon.$$

Then, using the Bellman–Gronwall Lemma [36], it is obtained that the identification error *ξ* = **x** − *χ* is bounded by

$$\|\mathfrak{f}\| \le \frac{\mathfrak{e}}{2}.\tag{A1}$$

**Proof.** See Reference [37].

#### **Appendix B. Closed-Loop Stability Analysis**

In order to analyze the stability of Equations (25) and (35), a representation in scalar form is made as follows [29]:

$$\begin{aligned} s\_h &= -|k\_h|s\_h|^{\frac{1}{2}} \text{sign}(s\_h) + u\_h + f\_h \\ \dot{u}\_h &= -k\_h \text{sign}(s\_h), \; h = 1, 2, \dots, d \end{aligned} \tag{A2}$$

Considering *µ* > *<sup>h</sup>* = h |*x*1| 1 <sup>2</sup> sign(*x*1), *x*<sup>2</sup> i as the state vector of the super-twisting algorithm, *µ<sup>h</sup>* can be rewritten in terms of *s<sup>h</sup>* , then *µ* > *<sup>h</sup>* = h |*sh* | 1 <sup>2</sup> sign(*s<sup>h</sup>* ) *u<sup>h</sup>* i and considering that the matrix **P** is positive definite, the derivative of *µ<sup>h</sup>* is defined as [38]:

$$\boldsymbol{\mu}\_{\boldsymbol{h}} = \begin{bmatrix} \frac{1}{2} \frac{1}{|\boldsymbol{s}\_{\boldsymbol{h}}|^{\frac{1}{2}}} ((-k\_{1\boldsymbol{h}}) |\boldsymbol{s}\_{\boldsymbol{h}}|^{\frac{1}{2}} \text{sign}(\boldsymbol{s}\_{\boldsymbol{h}}) + \boldsymbol{u}\_{\boldsymbol{h}})\\ -k\_{2\boldsymbol{h}} \text{sign}(\boldsymbol{s}\_{\boldsymbol{h}}) \end{bmatrix} = \frac{1}{|\boldsymbol{s}\_{\boldsymbol{h}}|} \mathbf{A}\_{\boldsymbol{h}} \boldsymbol{\mu}\_{\boldsymbol{h}} \tag{A3}$$

Equation (A3) can be expressed as the following linear system:

$$
\dot{\mu}\_{\text{h}} = \mathbf{A}\_{\text{h}} \mu\_{\text{h}} + \rho\_{\text{h}} \tag{A4}
$$

From Equation (A2), *f<sup>h</sup>* is bounded with the following restrictions:

$$|f\_{\hbar}| \le \delta\_{\hbar} |s\_{\hbar}|^{\frac{1}{2}} \, |f\_{\hbar}| \le \delta\_{\hbar} |\mu\_{\hbar}| \, \delta\_{\hbar} \ge 0 \tag{A5}$$

Assuming that *f<sup>h</sup>* in (A2) is an external disturbance bounded by:

$$f\_{\hbar} = \delta\_{\hbar} |s\_{\hbar}|^{\frac{1}{2}} \text{sign}(s\_{\hbar}) = \delta\_{\hbar} \mu\_{\hbar} \tag{A6}$$

To construct a complete family of strong Lyapunov functions for Equation (A2) of the form (A8) the matrix **P** = **P** > > 0, the Lyapunov function can be reduced to the solution of an algebraic Lyapunov equation (ALE) [38] as:

$$\mathbf{A}^\top \mathbf{P} + \mathbf{P}\mathbf{A} = -\mathbf{Q} \tag{A7}$$

Lyapunov's candidate function can be written as follows [29,38]:

$$V\_{\hbar}(\mu) = \mu\_{\hbar}^{\perp} \mathbf{P}\_{\hbar} \mu\_{\hbar} \tag{A8}$$

Considering that the matrix **A** = <sup>1</sup> |*µ<sup>h</sup>* | −1 2 *kh* 1 2 −*k<sup>h</sup>* 0 and *ρ<sup>h</sup>* = " 1 2|*µ<sup>h</sup>* | *fh* 0 # where *k<sup>h</sup>* > 0,

then **A** is Hurwitz. For every symmetric and positive definite matrix **Q** = **Q**> > 0, and **P** = **P** > > 0 is the unique symmetric and positve definite solution of the ALE

(Equation (A7)). Moreover, the derivative *V*˙ of the Lyapunov function satisfies the differential inequality almost every where:

$$
\dot{V} \le -\gamma(\mathbf{Q})V^{\frac{1}{2}}(\mathbf{s})\tag{A9}
$$

where

$$\gamma(\mathbf{Q}) = \frac{\lambda\_{\min}(\mathbf{Q}) \lambda\_{\min}^{\frac{1}{2}}(\mathbf{P})}{\lambda\_{\max}(\mathbf{P})} \tag{A10}$$

is a scalar depending on the selection of the **Q** matrix [38].

**Proof.** The Lyapunov function (A8) is continuous, positive definite and radially unbounded in R<sup>2</sup> , it is differentiable everywhere, except on the line *s*<sup>1</sup> = 0. As the trajectories of Equation (A2) cannot stay on this set, before reaching the origin, the derivative of *V*(*µ*) can be written from Equation (A3) as follows:

$$\dot{V}\_{\hbar}(\mu) = \frac{1}{|\mu\_{\hbar}|} \mu\_{\hbar}^{\top} (\mathbf{A}\_{\hbar}^{\top} \mathbf{P}\_{\hbar} + \mathbf{P}\_{\hbar} \mathbf{A}\_{\hbar}) \mu\_{\hbar} + 2 \mu\_{\hbar}^{\top} \mathbf{P}\_{\hbar} \rho\_{\hbar} \tag{A11}$$

Involving Equation (A6) in the term *ρ* of Equation (A11), it yields:

$$\dot{V}\_{\hbar}(\mu) = -\frac{1}{|\mu\_{\hbar}|} \mu\_{\hbar}^{\top} \mathbf{Q}\_{\hbar} \mu\_{\hbar} \tag{A12}$$

Now, defining *P* = 1 2 *k* 2 *<sup>h</sup>* <sup>+</sup> <sup>2</sup>*k<sup>h</sup>* <sup>−</sup><sup>1</sup> 2 *kh* −1 2 *k<sup>h</sup>* 1 and using the standard inequality for quadratic forms

$$\lambda\_{\min}\{\mathbf{P}\}\|\mu\|\_{2}^{2} \le V\_{\hbar}(\mu) \le \lambda\_{\max}\{\mathbf{P}\}\|\mu\|\_{2}^{2} \tag{A13}$$

where k*µ*k 2 <sup>2</sup> = *µ* 2 <sup>1</sup> + *µ* 2 <sup>2</sup> = |*s*1| + *s* 2 2 is the Euclidean norm of *µ*. Equation (A11) along the solutions of the system (A2) can be rewritten as follows:

$$\dot{V}\_h(\mu) = -\frac{1}{|s\_1|^{\frac{1}{2}}} \mu^\top \mathbf{Q} \mu \le -\frac{1}{|s\_1|^{\frac{1}{2}}} \lambda\_{\text{min}} \{\mathbf{Q}\} ||\mu|| \tag{A14}$$

*V*˙ *h* (*µ*) is negative definite if *Q* > 0, which is the case when *k<sup>h</sup>* > 0.

$$\text{Where } \mathbf{Q} = \frac{k\_{\hbar}}{2} \begin{bmatrix} k\_{\hbar}^2 + 2k\_{\hbar} - \delta\_{\hbar}(k\_{\hbar} + 4\frac{k\_{\hbar}}{k\_{\hbar}}) & -k\_{\hbar} \\ -(k\_{\hbar} - \delta\_{\hbar}) & 1 \end{bmatrix} \text{, therefore the matrix } \mathbf{Q} \text{ is positive.}$$

$$\text{define if } k\_{\hbar} \text{ satisfies the following condition: } k\_{\hbar} > 2\delta\_{\hbar} \text{ also } k\_{\hbar} > \frac{1}{2} \frac{k\_{\hbar}^2(\delta\_{\hbar} - k\_{\hbar})}{k\_{\hbar} - 2\delta\_{\hbar}}. \text{ Using}$$

2 *kh*−2*δ<sup>h</sup>* Equations (A13) and (A14) and the fact that |*s*1| 1 <sup>2</sup> ≤ k*µ*k<sup>2</sup> ≤ *V* 1 2 (*s*) *λ* 1 2 *min*{*P*} , the inequality (A9) is satisfied.

Now, the convergence of *ξ i j* should be considered, involving in this way the identification error boundedness (Equation (A1)), then, the Lyapunov function (A8) can be rewritten as follows:

$$V(\mu, \xi) = \mu^\top P \mu + \frac{1}{2} (\xi\_1^i)^2 \tag{A15}$$

where the time derivative of Equation (A15) is:

$$\dot{V}(\mu,\xi) = -\frac{1}{|\mu\_h|} \mu\_h^\top \mathbf{Q}\_h \mu\_h + \xi\_1^i \xi\_1^i \tag{A16}$$

substituting the time derivative of *ξ i* <sup>1</sup> = *x i* <sup>1</sup> − *χ i* 1 , which is obtained through the filtered error algorithm (6) and is applied in Equation (A16)

$$\dot{\mathcal{V}}(\mu,\xi) = -\frac{1}{|\mu\_h|} \mu\_h^\top \mathbf{Q}\_h \mu\_h + \tilde{\xi}\_1^i \left[ (-a\_1^i \mathbf{x}\_1^i + b\_1^i w\_1^i \boldsymbol{\psi}\_1^i + \mathbf{x}\_2^i + \mathbf{x}\_3^i) - \dot{\chi}\_1^i \right] \tag{A17}$$

In order to guarantee that Equation (A17) is negative definite, the desired dynamics for *x*2*<sup>d</sup>* is

$$\mathbf{x}\_{2d}^i = -c\_1 \mathfrak{z}\_1^i + a\_1^i \mathfrak{x}\_1^i - b\_1^i w\_1^i \psi\_1^i - \mathfrak{x}\_3^i + \dot{\chi}\_1^i \tag{A18}$$

thus,

$$\begin{split} \dot{\mathcal{W}}(\mu\_{\prime}\xi) &= -\frac{1}{|\mu\_{h}|}\mu\_{h}^{\top}\mathbf{Q}\_{h}\mu\_{h} + \\ \xi\_{1}^{i}(-a\_{1}^{i}\mathbf{x}\_{1}^{i} + b\_{1}^{i}w\_{1}^{i}\boldsymbol{\psi}\_{1}^{i} + (-c\_{1}\xi\_{1}^{i} + a\_{1}^{i}\mathbf{x}\_{1}^{i} - b\_{1}^{i}w\_{1}^{i}\boldsymbol{\psi}\_{1}^{i} - \boldsymbol{\chi}\_{3}^{i} + \boldsymbol{\chi}\_{3}^{i} + \boldsymbol{\chi}\_{1}^{i}) - \dot{\chi}\_{1}^{i}) \\ &= -\frac{1}{|\mu\_{h}|}\mu\_{h}^{\top}\mathbf{Q}\_{h}\mu\_{h} - c\_{1}\tilde{\mathfrak{z}}\_{1}^{2} \end{split} \tag{A19}$$

with *c*<sup>1</sup> > 0 real value. In this way, by Theorem (1) and Equation (A1), the uniformly ultimately bounded stability of the complete system is ensured.

#### **References**


MDPI St. Alban-Anlage 66 4052 Basel Switzerland Tel. +41 61 683 77 34 Fax +41 61 302 89 18 www.mdpi.com

MDPI Books Editorial Office E-mail: books@mdpi.com www.mdpi.com/books

MDPI St. Alban-Anlage 66 4052 Basel Switzerland

Tel: +41 61 683 77 34 Fax: +41 61 302 89 18

www.mdpi.com ISBN 978-3-0365-5586-7