Challenges in Smartizing Operational Management of Functionally-Smart Inverters for Distributed Energy Resources: A Review on Machine Learning Aspects

Abstract

The widespread introduction of functionally-smart inverters will be an indispensable factor for the large-scale penetration of distributed energy resources (DERs) via the power system. On the other hand, further smartization based on the data-centric operation of smart inverters (S-INVs) is required to cost-effectively achieve the same level of power system operational performance as before under circumstances where the spatio-temporal behavior of power flow is becoming significantly complex due to the penetration of DERs. This review provides an overview of current ambitious efforts toward smartization of operational management of DER inverters, clarifies the expected contribution of machine learning technology to the smart operation of DER inverters, and attempts to identify the issues currently open and areas where research is expected to be promoted in the future.

1. Introduction

The global goal of carbon neutrality and related policies accelerate the diffusion of distributed energy resources (DERs) such as photovoltaic solar power generation systems (PVs), battery energy storage systems (BESSs), electric vehicles (EVs), and heat-pump water heaters (HPs) to the consumer side. For example, the global renewable net capacity addition in 2022 is estimated to exceed 300 GW for the first time [1]; in particular, about 20% of this capacity will come from new distributed PV installations, such as those installed in residential houses. The International Energy Agency (IEA) also reported that nearly 600 GW of BESS capacity will be needed worldwide in 2030 [2] to achieve the net-zero emissions by 2050 scenario (NZE) [3] and that 16.5 million EVs have already been installed by 2021 [4]. The IEA also reported that the global heat pump capacity needs to be increased to nearly 3000 GW in 2030 [5] to realize NZE, although its compatibility with regional heat demand characteristics needs to be taken into account. Such a remarkable change in the components of the whole power system fundamentally alters the unidirectional nature of power flow behavior in the conventional power grid, from the generation side to the demand side, resulting in spatio-temporally complex power flows that change bi-directionally depending on weather conditions and the DER utilization patterns of various customers [6]. For example, in the task of maintaining power quality at the distribution system, a voltage regulation framework based on on-load tap changers (OLTCs), which uniformly raises/lowers the voltage of a target section of a distribution line throughout the tap stepping operation [7], has conventionally functioned well. However, the increasing complexity of the power flow behavior due to the spatially-biased DER deployment [8] makes it extremely difficult to maintain the voltage at the various user endpoints of the distribution system within an appropriate range through such a uniform control. Similarly, the task of maintaining frequency in the power system, which has been performed by large-scale controllable generators using governor-free and load frequency control functions to regulate output, will become further difficult to achieve stably as renewable energy connected to the grid replaces conventional synchronous generators; this is one of the major challenges that can be expected to emerge with the widespread penetration of inverter-interfaced DERs, and many researchers [9,10] have mentioned this concern of insufficient inertia to maintain grid frequency.

On the other hand, the recent progress of digital transformation (DX) of power systems [11] provides important elemental technologies to address the above issues. For instance, concepts such as a wide area monitoring system using phasor measurement units (PMUs) [12] will play an important role in understanding the stability of a wide area power system. Meanwhile, in the distribution system, which is the capillary that distributes power to end-users, the penetration of sensor built-in sectionalizing switches [13] and smart meters with communication capabilities [14] is presenting new possibilities for understanding the complex behavior of the power system. In particular, from the perspective of improving controllability to cope with the increasing complexity of power flow in power grids, attempts to add various ancillary service functions (e.g., power factor control, volt-watt control, volt-var control, frequency-watt control, etc.) to the DER inverters [15] are being promoted. These smart inverters (S-INVs) can flexibly realize a wide variety of autonomous output controls that respond to individual DERs on the millisecond order by setting control parameters via external communication. Therefore, S-INVs hold great promise, for example, in terms of generating virtual inertia through appropriate control of DER output to maintain frequency and voltage during sudden load-balance fluctuations. However, in the real world, only the functional smartness of inverters associated with DERs, mainly in terms of communication and configurability of various control parameters, has been well-studied in advance, and we have yet to reach a common view on the important issue of how to smartly utilize these S-INVs in operation.

Machine learning (ML) is expected to provide a powerful way to properly orchestrate such smart components in the DX of such power systems. In particular, the realization of smart operational management of DER inverters in a data-centric manner is believed to further enhance the value of functionally-smart inverters in terms of their contribution to ancillary services. Therefore, research on the smarter operation of S-INVs in the system is considered to be indispensable to achieve solid operation of the next-generation power system as before, which is extremely difficult due to large-scale installation of DERs, and many researchers have eagerly worked on related fields in recent years. This review paper aims to provide an overview of the various efforts to apply ML techniques that are expected to lead to the smart operation of functionally-smart DER inverters; we will then attempt to identify scientific challenges in related fields and research topics that are expected to be promoted in the future.

The rest of this manuscript is organized as follows. In Section 2, we give a brief overview of S-INVs and trends in their real-world deployment and follow up with general observations on the roadmap for a step-wise transition of the implementation phase. This section also provides an overview of the recent efforts to realize operational smartization of S-INVs and roughly categorizes their application scenarios. In Section 3, based on the categorization results, we organize the scientific challenges targeted by research in related fields that propose the application of ML techniques. Section 4 summarizes this survey and outlines the open scientific issues that ML techniques are expected to address in the future to further smartize the operation of smart inverters.

2. Smartization of the Operation of Smart Inverters

2.1. Smart Inverters

Inverters are responsible for converting DC power sources of DERs to AC, which is required when connecting to the power grid. Especially in the early stages of PV deployment, there has been much discussion about how to implement maximum power point tracking (MPPT) to achieve high system efficiency and smart functionality from the perspective of the power generation system. Later, however, the need for grid-supporting functions to realize ancillary services such as voltage regulation in addition to general functions began to be actively discussed. So-called “smart inverters” provide such ancillary services by offering flexible control of the active/reactive power of DERs. On the other hand, current source inverters, which have been often used in PV interconnection, cannot operate without AC voltage generated by other generators connected to the grid. These inverters themselves generally do not generate voltage and follow the voltage on the grid side, so they are called grid-following (GFL) inverters. These GFL inverters have been used not only for PV but also for BESS interconnection. In many real-world implementations, wind power interconnection is also being considered, in which the AC generated is converted to DC and sent to the grid via a GFL-type inverter. Therefore, there is a concern that a large number of grid-following-type DER inverters will lead to a relative lack of inertia contributed by conventional power generation facilities on a grid scale. Thus, DER inverters that have a voltage source called grid-forming (GFM) inverters, which create such inertial forces and enable frequency control and voltage magnitude control, are being developed to provide even smarter functionality.

Regarding the characteristics required for such DER inverters to provide ancillary services, ref. [15] has provided a well-organized review in terms of self-security, self-adapting, self-governing, and self-healing. For example, the self-governing feature categorized by them represents the capability of inverters to operate in grid-following and/or grid-forming control modes [16]. Meanwhile, the self-adapting feature represents the flexibility realized by adaptive inverter controllers for stable dynamics under various grid conditions. Typical functionalities that are often discussed from this perspective include the following:

• Constant power factor control: a function to ensure that leading reactive power is output at a set power factor to suppress the increase in distribution line voltage due to the active power supplied to the grid from various neighboring energy resources.
• Active power limitation: a function to design the maximum active power that can be output through the inverter.
• Active power control: a function to immediately control the active power output by command from the distributed energy resource management system (DERMS).
• Ramp rate control: a function to mitigate the impact on the power system by limiting the rate of change of active and reactive power during DER interconnection and disconnection operation.
• Freq-watt control: a function to reduce the active power output of DER for suppressing the increase in frequency when a large number of loads drop off the power system due to, e.g., an accident on a transmission line, resulting in a suppression of frequency increase.
• Volt-watt control: a function to reduce the active power output of DER when the voltage of the connecting point increases, thereby suppressing the voltage increase.
• Volt-var control: a function to suppress voltage rise/drop by supplying reactive power when the voltage of the connecting point increases/decreases.
• Dynamic reactive power control: a function to suppress voltage fluctuations by supplying reactive power in the direction of canceling out the fluctuation when the voltage suddenly changes: leading reactive power when the voltage rises, and lagging reactive power when the voltage falls.
• Reactive power control: a function to immediately control the reactive power output by command from the DERMS.

In addition, in the context of self-securing in their categorization, the following functionalities may be naturally important.

• Monitoring: a function to remotely monitor DER inverter status and measurements with DERMS.
• Communication: a function to establish intercommunication with external systems such as DERMS.
• Data handling: a function to handle specific data models and protocols.

On the other hand, from the perspective of self-healing, which has been defined as fault-tolerance and stress reduction under abnormal conditions, the following functionalities will be important.

• Scheduling: a function to schedule for DER connection/disconnection, control modes, and control parameters.
• Soft start: a function to mitigate the impact on the power system by limiting the rate of change of active power during reconnection.
• Disconnection/reconnection: a function to disconnect and reconnect DERs from the power system with remote control from DERMS.
• Fault ride-through (FRT): a function to prevent DERs from disconnection in response to voltage and frequency fluctuations while adhering to the conditions for continued interconnection operation.
• Islanding detection: a function to detect that the target distribution system has been disconnected from the grid power supply and properly disconnect the DER.
• Maintenance: a function to maintain inverter and DER systems.

For S-INVs with the various functionalities described here to be introduced into the real world, their impact should be empirically demonstrated in multiple stages. In particular, as the role of DERs in the stable supply of electricity changes with the spread of DER installations, grid codes are also being developed based on this change. For example, the U.S. state of California, where DER has been massively adopted, has declared the requirements for grid interconnection of PVs and BESSs through Rule 21 [17], a grid code established by the independent system operator: in this grid code, it is noted that the functionalities of S-INVs will be equipped in a step-wise manner, generally as shown in

Table 1. Summary of functional requirements for S-INVs.

Phase	Description
Phase 1 (autonomous functions): implementation of autonomous control function.	Dynamic volt-var control Constant power factor control Low/high voltage ride-through control Low/high frequency ride-through control Anti-islanding control Reconnect soft-start methods Ramp rates control
Phase 2 (communication functions): implementation of intercommunication function among S-INVs and other systems.	Communication interface Transport protocols Mapping to application protocols Data model Transport and user cyber security
Phase 3 (additional advanced functions): implementation of advanced DER control functions utilizing communication functions.	Volt-watt control Limit maximum active power mode Set active power mode Scheduling power Frequency-watt control Dynamic reactive power support Smooth frequency deviations DER disconnect and reconnect command Monitor alarms Monitor key DER data Provide DER information at interconnection Start up Scheduling modes and parameters

Source: authors’ own elaboration based on CA Rule 21 [17].

. In the real world, the functionality of the DER inverter is advancing in this way due to innovations in hardware technology and the organization of communication protocols.

2.2. Smartization of the Management of “Smart” Inverters

ML technology is considered to play a very important role in realizing a data-centric framework for appropriate control based on the exchange of data realized through limited communication, and a great number of related technologies have been discussed in recent years. Firstly, we focus on representative keywords in recently published related papers to give a brief overview of the research field on the smartization of the S-INV operation. Figure 1 represents a word cloud generated from the abstracts of 130 papers related to S-INVs published since 2017 in some representative journals to provide an overview of recent research topics related to S-INVs. We have included 55 journal papers published in IEEE Xplore (https://ieeexplore.ieee.org/Xplore/home.jsp accessed on 28 October 2022) containing smart inverter as index terms, 30 journal papers published on ScienceDirect (https://www.sciencedirect.com/ accessed on 28 October 2022) containing smart inverter as keywords, and 45 papers published in Energies (https://www.mdpi.com/journal/energies accessed on 28 October 2022) containing smart inverter as keywords. All the articles were retrieved on 28 October 2022. The figure shows several rough trends of recent studies on S-INVs. The first thing that can be seen from this figure is that the voltage control aspect of the power system is expected to be prominent. While the contribution of DER inverters to the frequency control of power systems has been discussed well, there seems to have been even more active research areas in recent years that claim to contribute to the voltage control aspect of the power system. As for functionalities of the inverters, topics related to specific operations such as volt-var control (VVC) seem to be mentioned frequently. The inclusion of other voltage regulators, such as on-load tap changers (OLTCs), suggests that the scientific interest of the community has been focused on topics related to the sharing of roles and coordination with other control systems that have been responsible for voltage regulation in conventional distribution system operation. The appearance of the words communication and centralized may suggest the need for coordinated operation among multiple inverters and existing voltage regulators. In addition, the frequent use of words such as hosting as well as photovoltaic and solar confirms the expectation that the smartization efforts of inverters will promote the introduction of renewable energy.

Based on these research trends and the ongoing discussion of functional requirements for S-INVs in the w world, we categorize the research related to the smartization of DER inverter operations into the following six categories, focusing on the main application scenarios of the developed technology; each category has the following aspects:

• Individual DER system operation: methodology development to support interconnection and operation of individual DER systems.
• Wide-area grid support: methodology development for the provision of ancillary services in wide-area operations expected with the mass introduction of DERs.
• Voltage regulation: methodology development to regulate the voltage around the interconnection point during the operation of the DER systems.
• Emergency control: methodology development aimed at DER operation during emergencies caused by physical factors from a power system perspective.
• Security/anomaly detection: methodology development for cyber security and anomaly detection during operation via information and communication systems.
• Utilization of probe data: methodology development to utilize the data acquired by DER inverters for further service operation.

3. Machine Learning Challenges

In this section, we look at the big picture of related research for smartizing the operation of DER inverters for each of the categories corresponding to the six application scenarios described in Section 2.2, with particular attention to the aspects where ML methods are being actively applied in realizing a data-centric approach. Figure 2 summarizes the targets that each category primarily covers. We will then clarify the motivation and awareness of the issues involved in the introduction of ML technology in each category, accompanied by a survey of relevant studies published by authors affiliated with research institutions located in the various regions shown in Figure 3.

3.1. Individual DER System Operation

Here, we focus on the perspective of smartization for the sustainable operation of individual DER systems that are directly controlled by the inverter. The studies shown in Table 2 are those that focus on the perspective of individual DER systems aiming for sustainable operation. Particularly, inverter control for PV systems has been the subject of many studies as part of the smartization of control strategy [18], with a variety of data-centric control mechanisms being considered.

One of the typical smart features required for inverters used in renewable energy interconnection is the MPPT; this operation strategy allows the output of the renewable energy source to always follow the optimum operating point under changing weather conditions. To realize such an operation, various promising approaches have been proposed, such as tracking the optimal power with a hill climbing approach based on the P–V curve [19]. Among them, some research groups have proposed the application of ML frameworks such as an artificial neural network (ANN) [20,21] and reinforcement learning (RL) [22,23] to realize MPPT in a data-centric manner to achieve even faster tracking of the optimal operating point, which changes rapidly with solar radiation, as in PV generation; in these frameworks, MPPT is achieved by learning a model that outputs inverter control parameters to achieve the maximum power point based on input information such as solar irradiation and panel temperature at each interconnection point. Thus, MPPT is one representative field where ML approaches can contribute to the smartization of the operation of DER inverters.

To sustainably and smartly use the installed DER inverter, attempts to properly quantify the efficiency realized by the inverter [24] and assist maintenance [25,26] are also important topics. For example, the system performance can be understood by analyzing the monitoring results of the outputs of neighboring multiple DER systems [25], and the failure modes common to PV inverters can be analyzed based on the maintenance records, e.g., by using the term frequency-inverse document frequency (TF-IDF) features extracted from records containing maintenance details, to make decisions regarding maintenance implementation [26]. The application of ML techniques is strongly expected in this research area as well, in terms of estimating situation-specific nonlinear response control results based on collected data, and, therefore, these types of smartization schemes will play an important role in sustainable DER system operation.

Table 2. Relevant studies on ML-based individual DER system operation.

Main Background/Target	ML Perspective	Refs.
MPPT	ANN-based MPPT control.	[20,21]
	RL-based MPPT control.	[22,23]
Assist of maintenance	Mixed-effect model-based identification scheme of the aging of PV inverters.	[25]
	SVM and LDA (topic model) for maintenance record analysis.	[26]
	Inverter efficiency estimation based on linear model.	[24]

Table 2. Relevant studies on ML-based individual DER system operation.

Main Background/Target	ML Perspective	Refs.
MPPT	ANN-based MPPT control.	[20,21]
	RL-based MPPT control.	[22,23]
Assist of maintenance	Mixed-effect model-based identification scheme of the aging of PV inverters.	[25]
	SVM and LDA (topic model) for maintenance record analysis.	[26]
	Inverter efficiency estimation based on linear model.	[24]

We should emphasize that the smartization of individual DER system operations is gradually being applied to real-world operations. For example, MPPT technology in PV inverters is widely implemented in the real world, with average conversion efficiencies of around 94–97% [27]. Deterioration diagnosis is another area where real-world services are being implemented; diagnostic tools for DER system operation, including inverters, are becoming popular, especially for PV systems [28]. Thus, the individual DER system operation is the area where the application of ML technology has been most studied with a view to practical use.

3.2. Wide-Area Grid Support

As mentioned in Section 2, one of the smart functionalities that is particularly promising for DER inverters is the support of daily grid operation. Table 3 summarizes representative studies for the realization of ancillary services where the connected DER inverters are intended to contribute rather to the entire power system. Harmonics control, in the sense of countermeasures against harmonic distortion that can occur due to DER output, is one of the key smart technologies expected in inverter operation. For example, there is a popular index called total harmonic distortion (THD) that measures the impact of distortion due to harmonics in AC power supply:

$$THD=\frac{\sqrt{I_2^2+I_3^2+I_4^2+\cdots +I_N^2}}{I_1},$$

(1)

where $I_n(n=1,\dots ,N)$ is the root mean square value of the n-th harmonics corresponding to the Nyquist frequency; in particular, $I_1$ represents the root mean square value of the fundamental source current. Taking a specific real-world implementation context as an example, IEEE Std. 519 defines recommended values for THD derived from source current. Several popular frameworks have been proposed to suppress the impact of such higher-order frequencies, including proportional integral (PI) control via a power filter and an approach that defines control quantities as a solution to a kind of optimization problem [29,30]. Furthermore, in an attempt to flexibly and appropriately suppress the effects of dynamic behavior derived from DER in a data-centric manner, some machine learning approaches have been proposed, such as the quasi-real-time derivation of control target values and control schemes using ANNs [31,32,33]. For instance, ref. [31] proposed a framework for learning an ML-based mechanism that monitors the output voltage of the S-INV at the interconnection point, derives the modulation index, and then derives the desired inverter switching state. For another instance, a framework using convolutional neural networks (CNNs), which have been reported to be highly effective in the context of image processing [34] and DER output prediction [35], has also been reported to work effectively in the context of harmonics control for the inverter [36]. In addition, it has been strongly expected that predicting the immediate frequency trend will be useful to achieve appropriate control, especially for dynamic harmonics. For this reason, many efforts have been reported to implement model predictive control (MPC) [37,38] schemes based on predictors built on ML methodologies such as ANNs [39,40,41,42]. Note that most studies assume that the model for such smart operation can be learned offline in advance. On the other hand, some research groups have argued the importance of the dynamic updatability of the model from the viewpoint of sustainable operation. Batch training of ML models, e.g., ANNs, typically takes several hours to several days of computation time, depending on the architecture and the amount of data. Moreover, if such a model is to be updated at the grid edge, an ML-based framework that requires excessively rich computational resources would be difficult to implement widely in the real world. For example, a framework such as the one proposed by [43] that allows models to be updated online in a computationally inexpensive manner based on the latest data may provide a possible solution that could support efficient contribution to the real-world operation.

On the other hand, many other efforts have been reported that aim at the effective operation of DER inverters from the viewpoint of further active realization of virtual synchronous generators (VSGs), and propose operation methods from the viewpoints of frequency control and grid synchronization. The potential contribution of such functionally-smart inverters has been pioneered especially in the context of the microgrid [44]. Approaches such as droop control, derived by analogy with the conventional synchronous power generation models, may be one possible framework. However, to achieve further adaptive and desirable control, ML approaches such as (deep) ANNs [45,46,47], radial basis function (RBF) neural networks [48], and RL [49,50,51] frameworks have been proposed. In particular, the plausible estimation of the providable virtual inertia of other DERs under various conditions and the derivation of the appropriate outputs based on this estimation play important roles in practical operation scenes [52]. For such a problem, ref. [53] has proposed a framework that realizes real-time estimation of inertia in the current power system by using time-series frequency data collected from multiple PMUs and derives control based on a regression model constructed with random forest (RF) [54]. In addition, as described in Section 2.1, GFL and GFM inverters are each expected to play an important role in the next-generation power system, and coordination between them will be particularly important in a DER-dominated system. To achieve this type of coordination, ref. [55], for example, has proposed a framework for agent-based consensus control of GFM-GFL coordinated secondary control for a microgrid with no other major power sources; such an attempt will be also a topic that is expected to become very important as DERs become massively penetrated.

Table 3. Relevant studies on ML-based wide-area grid support.

Refs.	Background/Target *			ML Perspective
	HC	FC	Other Target
[43]	✓	-	-	ANN-based online learning scheme for dynamic harmonic compensation.
[47]	-	-	Transient stability assessment.	DNN-based online assessment.
[56]	-	✓	-	ANN-based RL for frequency control.
[57]	✓	-	-	Fourier analysis and various optimization schemes to minimize THD.
[31]	✓	-	-	ELM-based harmonic elimination control.
[30]	✓	-	-	GA-based optimization to minimize THD.
[32]	✓	-	-	ANN-based output control to minimize THD.
[50]	-	✓	-	RL for adaptive VSG control.
[39]	✓	-	-	ANN-based MPC for reducing THD.
[29]	✓	-	-	Nature-inspired optimization to minimize THD.
[33]	✓	-	-	Adaptive FNN-based control to reduce THD.
[58]	✓	-	Voltage control.	ANN-based harmonics control.
[45]	-	✓	-	DNN for adaptive VSG control.
[36]	✓	-	-	Deep CNN-based control to minimize leakage current.
[49]	-	✓	-	RL for adaptive VSG control.
[53]	-	✓	-	RF-based estimation of inertia and ANN-based surrogate model for evaluation.
[59]	-	-	Current tracking.	FNN-based control.
[48]	-	✓	-	RBF NN for adaptive VSG control.
[60]	-	✓	Voltage control.	GP-based inference for decision-making in feasible control.
[40]	✓	-	-	ANN-based MPC to minimize THD.
[61]	-	✓	-	RF-based frequency response estimation.
[62]	✓	-	Voltage control.	Iterative learning control to mitigate THD.
[63]	-	-	Grid-forming.	Multi-armed bandits framework for online learning of control parameter configuration.
[64]	✓	-	Voltage control.	Fuzzy inference for fractional order control.
[65]	-	✓	-	Deep belief network for frequency control.
[55]	-	-	Coordination of GFM and GFL inverters.	Multi agent-based consensus control.
[66]	-	✓	-	Deep RL for adaptive VSG control.
[67]	✓	-	-	DBN-based MPC to reduce THD.
[51]	-	✓	-	RL for adaptive VSG control.
[41]	✓	-	-	ANN and TDNN for surrogating controller in MPC.
[46]	-		Voltage control.	ANN-based active/reactive power control.

* HA: harmonic control, FC: frequency control/synchronization.

Table 3. Relevant studies on ML-based wide-area grid support.

Refs.	Background/Target *			ML Perspective
	HC	FC	Other Target
[43]	✓	-	-	ANN-based online learning scheme for dynamic harmonic compensation.
[47]	-	-	Transient stability assessment.	DNN-based online assessment.
[56]	-	✓	-	ANN-based RL for frequency control.
[57]	✓	-	-	Fourier analysis and various optimization schemes to minimize THD.
[31]	✓	-	-	ELM-based harmonic elimination control.
[30]	✓	-	-	GA-based optimization to minimize THD.
[32]	✓	-	-	ANN-based output control to minimize THD.
[50]	-	✓	-	RL for adaptive VSG control.
[39]	✓	-	-	ANN-based MPC for reducing THD.
[29]	✓	-	-	Nature-inspired optimization to minimize THD.
[33]	✓	-	-	Adaptive FNN-based control to reduce THD.
[58]	✓	-	Voltage control.	ANN-based harmonics control.
[45]	-	✓	-	DNN for adaptive VSG control.
[36]	✓	-	-	Deep CNN-based control to minimize leakage current.
[49]	-	✓	-	RL for adaptive VSG control.
[53]	-	✓	-	RF-based estimation of inertia and ANN-based surrogate model for evaluation.
[59]	-	-	Current tracking.	FNN-based control.
[48]	-	✓	-	RBF NN for adaptive VSG control.
[60]	-	✓	Voltage control.	GP-based inference for decision-making in feasible control.
[40]	✓	-	-	ANN-based MPC to minimize THD.
[61]	-	✓	-	RF-based frequency response estimation.
[62]	✓	-	Voltage control.	Iterative learning control to mitigate THD.
[63]	-	-	Grid-forming.	Multi-armed bandits framework for online learning of control parameter configuration.
[64]	✓	-	Voltage control.	Fuzzy inference for fractional order control.
[65]	-	✓	-	Deep belief network for frequency control.
[55]	-	-	Coordination of GFM and GFL inverters.	Multi agent-based consensus control.
[66]	-	✓	-	Deep RL for adaptive VSG control.
[67]	✓	-	-	DBN-based MPC to reduce THD.
[51]	-	✓	-	RL for adaptive VSG control.
[41]	✓	-	-	ANN and TDNN for surrogating controller in MPC.
[46]	-		Voltage control.	ANN-based active/reactive power control.

* HA: harmonic control, FC: frequency control/synchronization.

3.3. Voltage Regulation

As described in Section 2, functionally-smart inverters can realize various output control mechanisms, such as active and reactive power, utilizing DER. In power systems with a large number of renewable energy sources, the role of inverters will become increasingly important in terms of fine-tuning grid operations in rather local areas, such as stabilizing voltage [68] and improving the three-phase imbalance in the distribution system [69]; such controllability is also expected to have a significant impact on the hosting capacity of renewable energy [70]. Table 4 shows representative studies on the sophistication of local area voltage control through the control of such S-INVs.

From the viewpoint of voltage control support in the distribution system, a uniform output control scheme based only on information collected at the connecting point is not necessarily appropriate, and effective realization will be possible only when each DER performs appropriate control by estimating the impact on the behavior of local power flow while also taking into account the interaction with the control of other surrounding energy resources. Motivated by these points, a great number of researchers have addressed the topic of coordinated control frameworks among massively deployed S-INVs and with existing voltage control equipment, and many applications of machine learning techniques have been attempted in this context. In particular, ML is expected to provide two main benefits: estimation and prediction of power flow uncertainties, and data-centric derivation of optimal control parameters and strategies under these uncertainties. For the former perspective, the primary concern is the prediction of the output of the DERs connected to the grid via individual inverters. For example, ref. [71] has proposed a framework that models the spatio-temporal behavior of solar radiation based on copula [72] and uses it for reactive power control to enhance voltage control for PV-induced voltage fluctuations. Estimation and prediction using linear regression models [73,74], kernel regression models [75], time series models [76], and DNNs [77,78] have provided important components in the frameworks for dynamically tuning the control parameters to establish the sophisticated voltage regulation [79,80]; especially the studies introducing the concept of MPC [81,82,83,84,85] have been well discussed. In particular, such prediction is effectively achieved not only by using the information on the DER output and end-point voltage, which can be monitored by the individual S-INV, but also by using a variety of other information as input, including weather measurement data such as solar irradiance [71,74] and demand data measured at each residential house [78] and various points [76]. Meanwhile, for the latter perspective, many frameworks have been proposed to derive the appropriate control depending on the situation in a data-centric manner using SVM [86] and ANNs [87,88,89]. One persistent view is that edge implementation [90] is preferable from a feasibility standpoint for controlling a large number of DER systems connected to the power grid, and a framework has been proposed in which the parameters of the individual inverters are tuned autonomously only using the information collected at each installation site for the appropriate control [88]. On the other hand, from the viewpoint of a large-scale optimization problem in which the parameters of a large number of inverters are to be determined appropriately, some strategies have been proposed, such as a derivation based on fuzzy inference [91], application of nature-inspired optimization schemes [92,93,94,95], optimization from a game theoretical viewpoint [96], and Bayes optimization [97].

Meanwhile, in recent years, the application of RL frameworks [78,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115] has shown great promise as an effort to comprehensively achieve these two tasks: estimating the uncertainty of the surrounding situation and searching for an optimal parameter set that depends on it. In particular, cooperative optimization with other conventional voltage control equipment is considered to be a very challenging topic in terms of a huge optimization problem with a high degree of freedom and complex constraints; for example, the viewpoint of coordination between various regulators, e.g., on-load tap changers (OLTCs), and S-INVs, brute force approaches [116], heuristic approaches [117], and multilevel optimization frameworks [118,119] has been studied. The RL scheme provides one promising approach for learning appropriate operational procedures in situations with such complex interferences [110,120,121,122]; compared to the ordinary supervised learning-based framework (see Figure 4a), where the control guideline of S-INVs are derived based on previously observed data, the RL framework allows the active search for appropriate parameters in dynamic situations involving uncertainty. Meanwhile, one notable difficulty in adopting such an RL framework in practical situations would be the evaluation of operational performance during the learning process. Such an implementation, shown in Figure 4b, generally takes a very long period of data acquisition to derive a good control guideline through exploratory evaluation in the actual system; in some cases, control with inappropriate parameters may adversely affect the real-world power system operation during the learning process. To address this difficulty, most RL frameworks have considered the use of power system simulation models (e.g., [123]) that aim to simulate the real-world power system response, in the form shown in Figure 4c. Furthermore, the concepts called surrogate models [124] and response surface approximation have recently been intensively used in the context of operational sophistication of power systems of various scales [125,126,127] to simulate plausible system responses without access to detailed information on physical characteristics and still allow for fast and numerous trial evaluations. Hence, the physical model-free RL scheme [110] introducing a surrogate model of the power system, as shown in Figure 4d, is expected to be further sophisticated as an important approach to consider dynamic and computationally cost-effective optimization under various conditions, taking into account the real-world power flows that dynamically change according to the DER penetration phase. Another important aspect of practical application of the framework shown in Figure 4 is the reliability of the simulation/surrogate model to reproduce actual power system behavior. For example, to realize the power flow of a real-world power distribution system, a precise understanding of various physical characteristics such as network topology, equipment, and response characteristics of control appliances is required. Even if this information is known accurately, the actual power flow behavior and the simulation results, which are approximated during the model construction process, do not always match perfectly. Furthermore, surrogate models, which are constructed by focusing on the statistical relationships among various measurements, allow for faster acquisition of system response results but may extrapolate responses that do not reflect the actual physical phenomena in extreme situations that are not included in the training data set. Therefore, there is a concern that the results tuned by such a model may result in very poor control of the real system. To address such issues, some researchers have begun to propose optimization schemes that actively estimate the regions of infeasible parameters in the optimization process and search for safe parameters while avoiding infeasible regions [97]. In realizing decentralized cooperative control of DER systems in a data-centric manner, appropriate handling of the effect of deviations in the responses of such models from the real system behavior will become an important topic in the future.

Table 4. Relevant studies on ML-based voltage regulation.

Refs.	Background/Target *							ML Perspective
	STF	VB	P	Q	CO	CO+	Other Target
[87]	-	-	✓	✓	-	-	-	ANN-based active/reactive power control.
[88]	-	-	-	✓	✓	-	-	ANN-based edge implementation of volt-var power control.
[98]	-	-	-	✓	✓	-	-	RL for coordinated voltage regulation.
[81]	✓	-	-	✓	✓	-	-	MPC scheme for state estimation-based coordinated volt-var control.
[75]	-	-	-	✓	-	-	Loss min.	Kernel regression for reactive power control.
[99]	-	-	-	✓	✓	-	-	Deep RL for coordinated voltage regulation.
[92]	-	-	✓	✓	-	-	-	Nature-inspired parameter optimization.
[128]	-	-	✓	-	✓	-	-	Multi-agent deep RL for coordinated voltage regulation.
[100]	-	-	-	✓	✓	-	Loss min.	RL-based coordinated reactive power control.
[79]	-	-	-	✓	✓	-	-	SVM-based coordinated reactive power control.
[94]	-	-	-	✓	-	-	-	Nature-inspired optimization for parameter search.
[101]	-	-	-	✓	✓	-	Loss min.	Deep RL-based coordinated volt-var power control.
[93]	-	✓	✓	✓	-	-	-	K-means load clustering for nature-inspired optimization-based parameter selection to balance voltage.
[96]	-	-	-	✓	✓	-	-	Linear regression for online game-theoretic coordinated volt-var control.
[129]	-	-	✓	✓	-	✓	-	Deep RL for coordinated voltage regulation.
[120]	-	-	-	✓	✓	-	Loss min.	DNN-based reactive power control (model-free approach).
[130]	-	-	-	✓	-	✓	-	Mixture model-based scenario generation for representation of uncertainty in the power grid behavior.
[73]	✓	-	-	-	-	-	-	Linear regression with elastic net regularizer for prediction of voltage behavior.
[83]	✓	✓	✓	✓	-	-	Multiobjective control.	MPC scheme based on nature-inspired optimization.
[131]	-	-	-	✓	-	-	-	Online deep RL for volt-var control of individual S-INV.
[102]	-	-	-	✓	-	✓	-	Multi-agent deep RL for volt-var control.
[132]	-	-	-	✓	-	-	-	ANN-based reactive power control.
[84]	✓	✓	✓	✓	-	-	-	MPC-based control of BESS-interfaced S-INV.
[133]	✓	✓	✓	✓	-	-	Multiobjective control.	MPC-based control of BESS-interfaced S-INV.
[134]	-	-	✓	✓	✓	-	-	RL for coordinated voltage regulation.
[85]	✓	-	-	✓	✓	-	-	MPC-based volt-var control.
[135]	-	-	-	✓	✓	-	-	Nature-inspired optimization-based volt-var control.
[136]	✓	✓	-	✓	✓	-	-	Multi-agent deep RL-based volt-var control for voltage balancing.
[121]	-	-	-	✓	-	✓	-	Deep RL for coordinated volt-var control.
[137]	-	-	✓	✓	-	-	-	Online learning for active/reactive power control of individual S-INV.
[95]	-	✓	-	✓	-	✓	-	Nature-inspired optimization for coordinated voltage regulation.
[104]	-	-	-	✓	✓	-	-	Multi-agent RL-based coordinated online volt-var power control.
[122]	-	-	-	✓	-	✓	-	Multi-agent deep RL-based volt-var control coordinated with the other regulators.
[105]	-	-	-	✓	✓	-	-	Multi-agent RL-based coordinated online volt-var power control.
[107]	-	-	✓	✓	✓	-	-	Deep RL for coordinated voltage regulation.
[76]	✓	-	✓	✓	✓	-	-	AR model for forthcoming voltage prediction.
[108]	-	-	✓	✓	-	-	-	DNN-based RL for regulation parameter search.
[77]	-	-	-	✓	-	✓	Loss min.	Deep CNN-based reactive power control for PVs.
[106]	-	-	-	✓	✓	-	-	Multi-agent RL for coordinated voltage regulation.
[97]	-	-	✓	✓	-	-	-	Bayes optimization for optimal search of inverter parameters.
[109]	-	✓	-	✓	✓	-	-	Deep RL-based volt-var control for voltage regulation in unbalanced distribution network.
[78]	✓	✓	-	✓	✓	-	-	DNN-based load prediction.
[110]	✓	-	-	✓	-	✓	-	GP-based prediction and multi-agent deep RL-based volt-var control coordinated with the other regulators (model-free approach).
[111]	-	-	-	✓	✓	-	-	RL for coordinated voltage regulation.
[74]	-	-	-	✓	✓	-	-	DNN-based inverter control policy.
[112]	-	-	-	✓	✓	-	-	Multi-agent deep RN-based coordinated volt-var control.
[113]	-	-	✓	✓	-	✓	-	Multi-agent deep RL-based active and reactive power control.
[114]	-	-	✓	✓	✓	-	-	Deep RL for coordinated voltage regulation.
[89]	✓	-	✓	✓	-	-	-	ANN-based load forecast and decentralized control.
[80]	✓	-	-	✓	✓	-	Loss min. and peak shaving control.	Deep RN-based control (model-free approach).
[86]	-	-	-	✓	-	-	-	Linear regression and SVM for online reactive power control.
[71]	-	-	-	✓	✓	-	-	Copula-based relationship modelling of spatio-temporal behavior of solar irradiance for evaluation of reactive power control effect.
[115]	-			-		-	-	RL for coordinated voltage regulation.

* STF: short-term forecast, VB: voltage balancing, P: active power control, Q: reactive power control, CO: cooperation among S-INVs, CO+: cooperation w/ other regulators.

Table 4. Relevant studies on ML-based voltage regulation.

Refs.	Background/Target *							ML Perspective
	STF	VB	P	Q	CO	CO+	Other Target
[87]	-	-	✓	✓	-	-	-	ANN-based active/reactive power control.
[88]	-	-	-	✓	✓	-	-	ANN-based edge implementation of volt-var power control.
[98]	-	-	-	✓	✓	-	-	RL for coordinated voltage regulation.
[81]	✓	-	-	✓	✓	-	-	MPC scheme for state estimation-based coordinated volt-var control.
[75]	-	-	-	✓	-	-	Loss min.	Kernel regression for reactive power control.
[99]	-	-	-	✓	✓	-	-	Deep RL for coordinated voltage regulation.
[92]	-	-	✓	✓	-	-	-	Nature-inspired parameter optimization.
[128]	-	-	✓	-	✓	-	-	Multi-agent deep RL for coordinated voltage regulation.
[100]	-	-	-	✓	✓	-	Loss min.	RL-based coordinated reactive power control.
[79]	-	-	-	✓	✓	-	-	SVM-based coordinated reactive power control.
[94]	-	-	-	✓	-	-	-	Nature-inspired optimization for parameter search.
[101]	-	-	-	✓	✓	-	Loss min.	Deep RL-based coordinated volt-var power control.
[93]	-	✓	✓	✓	-	-	-	K-means load clustering for nature-inspired optimization-based parameter selection to balance voltage.
[96]	-	-	-	✓	✓	-	-	Linear regression for online game-theoretic coordinated volt-var control.
[129]	-	-	✓	✓	-	✓	-	Deep RL for coordinated voltage regulation.
[120]	-	-	-	✓	✓	-	Loss min.	DNN-based reactive power control (model-free approach).
[130]	-	-	-	✓	-	✓	-	Mixture model-based scenario generation for representation of uncertainty in the power grid behavior.
[73]	✓	-	-	-	-	-	-	Linear regression with elastic net regularizer for prediction of voltage behavior.
[83]	✓	✓	✓	✓	-	-	Multiobjective control.	MPC scheme based on nature-inspired optimization.
[131]	-	-	-	✓	-	-	-	Online deep RL for volt-var control of individual S-INV.
[102]	-	-	-	✓	-	✓	-	Multi-agent deep RL for volt-var control.
[132]	-	-	-	✓	-	-	-	ANN-based reactive power control.
[84]	✓	✓	✓	✓	-	-	-	MPC-based control of BESS-interfaced S-INV.
[133]	✓	✓	✓	✓	-	-	Multiobjective control.	MPC-based control of BESS-interfaced S-INV.
[134]	-	-	✓	✓	✓	-	-	RL for coordinated voltage regulation.
[85]	✓	-	-	✓	✓	-	-	MPC-based volt-var control.
[135]	-	-	-	✓	✓	-	-	Nature-inspired optimization-based volt-var control.
[136]	✓	✓	-	✓	✓	-	-	Multi-agent deep RL-based volt-var control for voltage balancing.
[121]	-	-	-	✓	-	✓	-	Deep RL for coordinated volt-var control.
[137]	-	-	✓	✓	-	-	-	Online learning for active/reactive power control of individual S-INV.
[95]	-	✓	-	✓	-	✓	-	Nature-inspired optimization for coordinated voltage regulation.
[104]	-	-	-	✓	✓	-	-	Multi-agent RL-based coordinated online volt-var power control.
[122]	-	-	-	✓	-	✓	-	Multi-agent deep RL-based volt-var control coordinated with the other regulators.
[105]	-	-	-	✓	✓	-	-	Multi-agent RL-based coordinated online volt-var power control.
[107]	-	-	✓	✓	✓	-	-	Deep RL for coordinated voltage regulation.
[76]	✓	-	✓	✓	✓	-	-	AR model for forthcoming voltage prediction.
[108]	-	-	✓	✓	-	-	-	DNN-based RL for regulation parameter search.
[77]	-	-	-	✓	-	✓	Loss min.	Deep CNN-based reactive power control for PVs.
[106]	-	-	-	✓	✓	-	-	Multi-agent RL for coordinated voltage regulation.
[97]	-	-	✓	✓	-	-	-	Bayes optimization for optimal search of inverter parameters.
[109]	-	✓	-	✓	✓	-	-	Deep RL-based volt-var control for voltage regulation in unbalanced distribution network.
[78]	✓	✓	-	✓	✓	-	-	DNN-based load prediction.
[110]	✓	-	-	✓	-	✓	-	GP-based prediction and multi-agent deep RL-based volt-var control coordinated with the other regulators (model-free approach).
[111]	-	-	-	✓	✓	-	-	RL for coordinated voltage regulation.
[74]	-	-	-	✓	✓	-	-	DNN-based inverter control policy.
[112]	-	-	-	✓	✓	-	-	Multi-agent deep RN-based coordinated volt-var control.
[113]	-	-	✓	✓	-	✓	-	Multi-agent deep RL-based active and reactive power control.
[114]	-	-	✓	✓	✓	-	-	Deep RL for coordinated voltage regulation.
[89]	✓	-	✓	✓	-	-	-	ANN-based load forecast and decentralized control.
[80]	✓	-	-	✓	✓	-	Loss min. and peak shaving control.	Deep RN-based control (model-free approach).
[86]	-	-	-	✓	-	-	-	Linear regression and SVM for online reactive power control.
[71]	-	-	-	✓	✓	-	-	Copula-based relationship modelling of spatio-temporal behavior of solar irradiance for evaluation of reactive power control effect.
[115]	-			-		-	-	RL for coordinated voltage regulation.

* STF: short-term forecast, VB: voltage balancing, P: active power control, Q: reactive power control, CO: cooperation among S-INVs, CO+: cooperation w/ other regulators.

Especially in a framework aiming to appropriately control the voltage affected by the uncertainty of the power flow, which is estimated based only on sensor information at a limited number of points, by controlling many inverters that have very complex dependencies on each other, flexible decision-making is expected to be performed in real time in a data-centric manner. The utilization of S-INVs, which can be controlled in various ways according to the information observed at their endpoints while taking into account the uncertainties in the power flow behavior in large-scale grids, for fine-tuned voltage control, remains a very challenging research topic even today; however, that is precisely why this research field is expected to see breakthroughs via the application of ML methodologies.

3.4. Emergency Control

Efforts for S-INV operations have not only targeted daily operations, but also infrequent emergency operations. Table 5 summarizes representative studies related to these topics and involving ML perspectives. For instance, the fault ride-through (FRT) [138], which is often specified in grid codes for renewable energy interconnection, is a function that requires power supplies to continue operation without stopping in the event of a voltage sag or frequency fluctuation disturbance in the event of a grid accident. In this context, ref. [82] has proposed an MPC-based active/reactive power control to realize data-centric FRT besides the general voltage control. Detection of grid accident events [139] is also important in the decision-making process for emergency operations. In order to detect such system accidents, ML approaches such as the autoregressive (AR) model [140], K-nearest neighbor (K-NN) approach [141,142], SVM [140,142,143,144,145], Random Forest [142,146,147], Bayesian network [148], adaptive neuro fuzzy inference system (ANFIS) [149], auto-encoder [150], CNN [151,152], and LSTM [153] have been applied in various studies. In particular, ref. [154] considered the application of a bootstrap-based ensemble learning scheme in the decision-making process for control against fault-induced delayed recovery [155]. In addition, in a framework that identifies events that are expected to occur infrequently, such as power system accidents, by learning discriminators in a data-centric manner, the difficulty of achieving learning with high generalization performance due to significantly skewed data [156,157], known as the class imbalance problem [158], can be a barrier [142], for example, has focused on such a class imbalance problem, which can be serious in fault diagnosis, and proposed the application of the synthetic minority over-sampling technique (SMOTE) [158] to resolve the imbalance.

Islanding detection [159] is another important topic, in terms of the smartization of inverter operation in emergencies, that has been studied intensively, especially in the context of microgrids. In an attempt to realize islanding detection in a data-centric manner, many ML approaches have been considered, including the AR model [160], LSTM [161], SVM [145,160], sparse model [162], ANN [163], probabilistic fuzzy neural network (FNN) [164], self-organization model (SOM) [165], ANFIS [166], and auto-encoder [167]. For example, ref. [168] has proposed an ANN-based mode control scheme to realize mode transition control, in which the PV inverter should be operated in grid-connected mode or standalone mode by collecting the time-series information of voltage and current at the DER interconnection point. Thus, there are high expectations for ML in efforts to smartize the operation of DER inverters so that they can operate appropriately even in a kind of emergency.

Table 5. Relevant studies on ML-based fault/islanding detection.

Refs.	Background/Target *			ML Perspective
	FD	ID	Other Target
[166]	-	✓	-	ANFIS-based detection.
[160]	-	✓	-	AR- and SVM-based detection.
[143]	-	-	Fault diagnosis.	Fourier analysis and SVM-based diagnosis.
[165]	-	✓	-	SOM-based detection.
[164]	-	✓	-	Probabilistic FNN-based detection.
[154]	-	-	Fault-induced delayed recovery.	Bootstrap-based ensemble learning for decision-making model.
[161]	-	✓	-	LSTM-based islanding detection.
[145]	✓	✓	-	SVM-based detection.
[82]	-	-	Active/reactive power control for FRT.	MPC-based FRT scheme.
[144]	✓	-	-	SVM-based fault classification.
[151]	✓	-	-	CNN-based detection.
[146]	✓	-	-	Wavelet transformation and RF-based detection.
[167]	-	✓	-	Auto-encoder-based detection.
[162]	-	✓	-	Fourier and wavelet transform and sparse representation-based classification.
[153]	✓	-	-	Fault prediction based on LSTM.
[163]	-	✓	-	Wavelet transform and ANN-based detection.
[149]	✓	-	Fault elimination.	ANFIS-based detection.
[141]	-	-	Fault diagnosis.	K-NN-based diagnosis.
[142]	✓	-	-	SMOTE and various classification approaches (DT, SVM, K-NN, and RF) for detection.
[150]	✓	-	-	Auto-encoder-based detection.
[147]	✓	-	-	Boosting/bagging DTs and KDE for detection.
[168]	-	-	Mode transition control.	ANN-based control.
[148]	-	-	Fault diagnosis.	BN-based diagnosis.
[140]	✓	-	-	AR- and SVM-based detection.
[152]	-	-	Fault diagnosis.	CNN-based diagnosis.

* FD: fault detection, ID: islanding detection.

Table 5. Relevant studies on ML-based fault/islanding detection.

Refs.	Background/Target *			ML Perspective
	FD	ID	Other Target
[166]	-	✓	-	ANFIS-based detection.
[160]	-	✓	-	AR- and SVM-based detection.
[143]	-	-	Fault diagnosis.	Fourier analysis and SVM-based diagnosis.
[165]	-	✓	-	SOM-based detection.
[164]	-	✓	-	Probabilistic FNN-based detection.
[154]	-	-	Fault-induced delayed recovery.	Bootstrap-based ensemble learning for decision-making model.
[161]	-	✓	-	LSTM-based islanding detection.
[145]	✓	✓	-	SVM-based detection.
[82]	-	-	Active/reactive power control for FRT.	MPC-based FRT scheme.
[144]	✓	-	-	SVM-based fault classification.
[151]	✓	-	-	CNN-based detection.
[146]	✓	-	-	Wavelet transformation and RF-based detection.
[167]	-	✓	-	Auto-encoder-based detection.
[162]	-	✓	-	Fourier and wavelet transform and sparse representation-based classification.
[153]	✓	-	-	Fault prediction based on LSTM.
[163]	-	✓	-	Wavelet transform and ANN-based detection.
[149]	✓	-	Fault elimination.	ANFIS-based detection.
[141]	-	-	Fault diagnosis.	K-NN-based diagnosis.
[142]	✓	-	-	SMOTE and various classification approaches (DT, SVM, K-NN, and RF) for detection.
[150]	✓	-	-	Auto-encoder-based detection.
[147]	✓	-	-	Boosting/bagging DTs and KDE for detection.
[168]	-	-	Mode transition control.	ANN-based control.
[148]	-	-	Fault diagnosis.	BN-based diagnosis.
[140]	✓	-	-	AR- and SVM-based detection.
[152]	-	-	Fault diagnosis.	CNN-based diagnosis.

* FD: fault detection, ID: islanding detection.

3.5. Security/Anomaly Detection

Various security assurance frameworks [169,170] have begun to be considered from a cyber-physical perspective for the introduction of DERs. Discussion of the impact of the controllability and information provided by such resources on the power system has been of broad interest in the field [171,172,173]. Particularly, in recent years, there has been much discussion of security and anomaly detection in terms of the control of individual inverters, as shown in Table 6.

For example, in a typical study in the context of anomaly detection, ref. [174] has focused on the erroneous voltage data detection task in the voltage data monitoring process and proposed a volt-var and volt-watt inverter control scheme based on the idea of least absolute shrinkage and selection operator (LASSO) [175], which is a typical framework for models assuming sparseness and is one of the core technologies for black hole shadow observations by an Event Horizon Telescope [176], and has often been used in the physical-model-free description of energy systems [177,178]. In the context of anomaly detection, other approaches based on, e.g., SVM [179] and LSTM [180], also have been proposed.

Meanwhile, efforts to detect cyber/physical attacks [181,182], such as false data injection [183] and data integrity attacks [184], have been gradually increasing in recent years. For example, binary matrix factorization [185] is an approach to analyze patterns in signals and is one of the techniques that has been reported to be useful for power system applications [186]; ref. [182] has discussed the data-centric identifiability of attacks by classifying voltage and current features monitored at multiple points in the power system based on this binary matrix factorization technique. For another example, ref. [183] has proposed the application of a federated learning mechanism [187], i.e., a distributed ML framework, for the detection of false data injection attacks on inverters at multiple solar farms; although their research was less focused on achieving decentralized model learning with less intercommunication and more motivated by data privacy secure, such a framework is also expected to be effective to achieve decentralized cooperative control for individual inverters for proper daily operation. These studies suggest great promise for the contribution of recent advances in ML technology to smartize the operation of S-INVs.

Table 6. Relevant studies on cyber security of S-INVs.

Refs.	Background/Target	ML Perspective
[179]	Anomaly detection.	Anomaly detection based on SVM.
[184]	Detection of data integrity attack.	LSTM-based detection.
[180]	Anomaly detection.	MPC-based anomaly detection by using LSTM.
[174]	Erroneous voltage data detection in volt-var, and volt-watt control.	Linear model and Lasso approach for S-INV control.
[182]	Cyber and physical attack detection.	Matrix factorization-based detection (and t-SNE-based visualization).
[181]	Cyber attack mitigation.	Deep RL for attack detection.
[183]	Detecting false data injection attack.	Federated learning for cyber attack detection.

Table 6. Relevant studies on cyber security of S-INVs.

Refs.	Background/Target	ML Perspective
[179]	Anomaly detection.	Anomaly detection based on SVM.
[184]	Detection of data integrity attack.	LSTM-based detection.
[180]	Anomaly detection.	MPC-based anomaly detection by using LSTM.
[174]	Erroneous voltage data detection in volt-var, and volt-watt control.	Linear model and Lasso approach for S-INV control.
[182]	Cyber and physical attack detection.	Matrix factorization-based detection (and t-SNE-based visualization).
[181]	Cyber attack mitigation.	Deep RL for attack detection.
[183]	Detecting false data injection attack.	Federated learning for cyber attack detection.

3.6. Utilization of Probe Data

In the context of operational smartization of S-INVs, the topics presented in Section 3.1, Section 3.2, Section 3.3, Section 3.4 and Section 3.5 represent the major research trends that have been discussed in related fields. However, several frameworks, such as those shown in

Table 7. Relevant studies on inverter probing.

Target	ML Perspective	Refs.
Load identification.	Probing-to-Learn approach for load identification.	[188]
Distribution network topology processing.	Graph Laplacian-based network topology inference by inverter probing data.	[190]
Impedance identification.	ANN-based identification.	[189]
Evaluation of system configurations of S-INV and DER system.	Bootstrap- and linear regression-based evaluation.	[192]

, have been proposed to utilize data obtained via inverters, i.e., inverter probing data, to identify the physical components of the power system. For example, ref. [188] has discussed an idea to learn the estimation mechanism for other loads based on voltage response behavior to inverter injection changes; their proposed framework will be a promising way to identify non-metered loads connected to the grid by using probing data from S-INVs. Ref. [189] has proposed an ANN-based framework for grid impedance identification based on inverter measurements, and ref. [190] has focused on a framework for network topology inference based on inverter probing data using graph Laplacian [191], and proposed an estimation of distribution network topology using this framework.

The information provided by this type of framework may be trivial to DSOs having detailed system information, e.g., network configuration; however, such a framework may become particularly important for servicers who are in a position to evaluate system configurations with many DERs from a data-centric perspective. Furthermore, the approach of utilizing inverter probing data in conjunction with data measured by PMUs, smart meters, and sensor built-in sectionalizing switches to understand physical characteristics and identify power system response may become a core element of technology in terms of fully data-centric modeling of power system behavior, which is important in the context of S-INV control, as described in Section 3.2 and Section 3.3.

4. Conclusions

The development of functionally-smart inverters with flexible output controllability has been a major driver of the penetration of DERs, and some of these technologies have already been implemented in the real world. On the other hand, however, in order to actively introduce and utilize more DERs in society via power grids, it is essential to further smartize the operation of inverters to make the most effective use of their functionality. ML corresponds to one of the core technologies contributing to smartize operation of such DER inverters, and it is of great significance to explore current research trends in order to understand the awareness of implementation challenges, research gaps, and expected contributions of ML technologies.

This study reviewed relevant research from the aspect of machine learning, which is being considered for introduction with the expectation of maximizing the functionality of DER inverters and contributing to advanced system operation. In this paper, the context of related research on the smartization of inverter operation was divided into six major categories: (1) individual DER system operation, (2) wide-area grid support, (3) voltage regulation, (4) security/anomaly detection, (5), emergency control, and (6) utilization of probe data. The trends in the particularly active research areas (1)–(5) can be described as follows:

In the context of individual DER system operation, ML techniques have been used in research from the perspective of improving system efficiency and maintenance support. In particular, from the viewpoint of improving system efficiency, the applications of machine learning techniques have been proposed for learning optimal control for steep changes in the output of renewable energy and high-speed dynamic response. Meanwhile, sustainable operation of the installed DER system is expected to become important in the future. ML will be expected to contribute to maintenance support by utilizing the data accumulated locally during operation.

The role of ML technology in harmonics and frequency control, which is becoming increasingly important in the context of wide-area grid support, is that of a brain for appropriate high-speed control. Many research groups believe that the MPC-like framework is particularly important for dynamic and appropriate control of harmonics; thus, many applications of prediction frameworks based on machine learning techniques have been proposed. Meanwhile, in the context of frequency control, it is especially necessary to estimate the inertia that the power grid itself has and the contribution that each DER control result makes, to appropriately control each DER. Recent research trends in this area suggest that there is strong hope for a data-centric framework that defines appropriate control guidelines to circumvent the difficulty of having complete a priori information about the detailed physical characteristics of such power systems.

In the context of voltage regulation, there has been a great deal of activity in recent years in attempts to smartize the operation of DER inverters; ML is expected to make particularly significant contributions in this area. Especially in distribution systems, which are directly affected by the dynamics of various loads and outputs of DERs around the interconnection point, ML schemes have been actively applied to predict the impact of power flow and to realize appropriate control of DERs. From the viewpoint of how to utilize a very large number of DER inverters deployed in a distributed manner for voltage control, there are several open barriers, such as the realization of a distributed and cooperative mechanism and the sharing of optimal control on a large scale including existing other types of voltage regulators; however, some ambitious studies have begun to overcome these barriers and realize data-centric control. This is an area where breakthroughs are expected through the application of machine learning frameworks.

In the context of emergency control, many attempts have been made to perform fault detection and islanding detection in a data-centric manner. This is a very important area for safety and security, and, in particular, there are concerns about interference with inverter functions that try to contribute to system inertia, so there are great expectations for the application of ML technology to support emergency control.

Some research topics on system security and anomaly detection in the framework of inverter control via communication and information systems have come to our attention. These topics will become important as DER inverters become more widely used and as data-centric group control mechanisms are implemented via communications. The awareness of this issue has led to research on the application of secure distributed cooperative ML techniques, which will be an inevitable issue for the future deployment of smart DER inverters.

In terms of the perceived challenges to be addressed in the related field and the expected future research directions, the key findings of this review study can be summarized as follows:

• Learning at grid edge: Not all DERs deployed in a distributed manner will have rich computational resources. The keys to realization will be the derivation of appropriate control parameters for each S-INV with limited computational resources and the personalization at local points of the decision-making mechanism involved in the operation.
• Distributed learning: an effective learning scheme via limited communication to achieve proper operation of the entire system cooperatively considering the mutual control effects of individual DERs and other facilities via limited communication will be important.
• Utilization of system models: Simulation and surrogate models used to tune the control parameters of S-INVs generally do not always match actual system behavior. This concern will need to be addressed in the practical application for the real-world system.
• Robustness to data perturbation: In power system operation, where the influence of data-centric control of DERs is dominant, operational robustness against data modification/loss is required. The ML frameworks used in parameter derivation and decision-making processes will also need to be robust.

The social deployment of DERs using functionally-smart inverters will become even more essential for many countries. In this research field, there will be a particular need to take advantage of technological advances in the field of ML and apply them to the expansion of the utilization of DERs, including renewable energy sources.