Recent Trends and Issues of Energy Management Systems Using Machine Learning

Abstract

Energy management systems (EMSs) are regarded as essential components within smart grids. In pursuit of efficiency, reliability, stability, and sustainability, an integrated EMS empowered by machine learning (ML) has been addressed as a promising solution. A comprehensive review of current literature and trends has been conducted with a focus on key areas, such as distributed energy resources, energy management information systems, energy storage systems, energy trading risk management systems, demand-side management systems, grid automation, and self-healing systems. The application of ML in EMS is discussed, highlighting enhancements in data analytics, improvements in system stability, facilitation of efficient energy distribution and optimization of energy flow. Moreover, architectural frameworks, operational constraints, and challenging issues in ML-based EMS are explored by focusing on its effectiveness, efficiency, and suitability. This paper is intended to provide valuable insights into the future of EMS.

1. Introduction

Smart grids are widely acknowledged as a viable solution to enhancing grid stability, reliability, and efficiency. A significant advancement has been made in the attempt to develop energy infrastructure and sustainable energy solutions [1]. The International Energy Agency (IEA) emphasizes innovative power system planning as essential for effectively incorporating expanding renewable energy sources (RESs) [2]. Therefore, the global decarbonization process is considered crucial due to its ability to improve system adaptability. The U.S. Department of Energy characterizes smart grids as advanced systems that enhance the operational effectiveness and efficiency of grid utilities. Smart grids are designed for efficient energy flow management, reduced power outages, enhanced operational control, and increased prosumer involvement in electricity management [3]. In recent years, the need for a transformation to renewable energy utilization has been driven by climate and energy security challenges [3]. The implementation of RES can be crucial due to the increasing significance of smart grids in electricity markets [3,4]. By 2040, additions or refurbishments to more than 80 million kilometers of electrical grids are anticipated to be required, supporting decarbonization and the integration of RES [4]. Development of future grids may be crucial in the effective integration of RES and the development of flexible power systems, which are projected to account for over 80% of the global power capacity increase in the next two decades [4]. However, challenges such as cost reduction, integration of various energy sources, and large-scale electrical grid management must be effectively addressed to sustain the growth of RES and meet the demands of future smart grids.

Distributed energy resources (DERs) are defined as small-scale units of local power generation connected to a large-scale power grid at the distribution level [1,3]. RES, traditional generators, electric vehicles (EVs), and controllable loads can be regarded as DERs. The flexibility offered by RES, including solar panels, wind turbines, and small natural gas-fueled generators, is recognized for its potential to alleviate the load burden on central grids [5]. Consequently, DERs are acknowledged as playing a pivotal role in the power systems, enhancing resilience, and facilitating the transition towards RES.

Various applications and critical component systems are employed in smart grids, including energy management systems (EMS), distribution management systems, outage management systems, big data analytics, and load forecasting [6]. Regarding the main components of energy management, EMS is employed to effectively coordinate the operations of DERs and other involved systems. Specifically, EMS is defined as a computerized system that includes a software platform offering essential support services along with various applications, according to the standard IEC-61970 of the International Electromechanical Commission (IEC) [7].

The future implementation of energy management is contingent upon transitioning from conventional grids to smart grids. EMS applications are designed to ensure the efficient operation of electrical generation and transmission facilities, with the core aim of stabilizing the energy supply at minimal costs in smart grids [8]. Attributed to the pivotal role of energy management, there is a marked enhancement in the reliability and efficiency of supply and distribution systems. This enhancement can be realized through the employment of intelligent algorithms and advanced control systems utilizing mathematical programming, heuristics, and machine learning (ML) to schedule loads and demands and optimize power flow [9,10].

The objectives of EMS are systematically delineated as follows: (i) the monitoring and processing of the grid energy information, ensuring the comprehensive and continuous analysis of grid states; (ii) the precise forecasting of load demands, utilizing ML algorithms to predict and manage future energy requirements; (iii) the optimization of energy flow across the grid, aimed at enhancing efficiency and minimizing losses; (iv) the efficient integration of DERs, harmonizing diverse component systems to build robustness of grid; (v) the strategic management of demand response (DR), adjusting energy generation and consumption patterns in response to grid uncertainties; (vi) the maintenance of grid reliability and stability, ensuring consistent energy supply and resilience against disturbances; and (vii) the development of sophisticated energy scheduling and control systems to optimize energy distribution and utilization [8,9,10,11,12,13,14,15,16,17]. To meet the above requirements, key component systems of EMS may encompass an energy management information system (EMIS), grid autonomation and self-healing system (GASHS), energy storage system (ESS), energy trading risk management system (ETRMS), and demand-side management system (DSMS).

The main contributions of this paper are:

• In this paper, a comprehensive analysis is provided on the construction of ML-based EMS, encompassing a variety of key component systems.
• Enhancements in stability, efficiency, and reliability within EMS are attributed to the integration of ML technologies, highlighting their pivotal role in optimizing energy flow and efficient system operations.
• An in-depth comparison of EMS framework characteristics is presented, along with suggestions for promising ML candidates tailored to specific frameworks.
• The EMS frameworks, recent trends, operational constraints, and challenging issues of ML-based EMS are discussed to provide insights into future developments and enhancements.

This paper is structured into five sections. Section 1 is the introduction to EMS. The architectural frameworks of EMS operations are discussed in Section 2. In Section 3, the key component systems of ML-based EMS are discussed. Section 4 presents the operational constraints, challenging issues, and further studies related to ML-based EMS, emphasizing their role in ensuring efficient operation. Finally, the conclusions are presented in Section 5.

2. Architectural Frameworks of EMS

In the internet of energy domain, EMS is a prominent research topic. Numerous significant papers have been published addressing various aspects such as electricity generation, monitoring, storage, trading, control, and self-healing systems. An overview of the key component systems in EMS architectures is shown in Figure 1.

EMIS is integral to EMS, serving as the backbone for data-driven decision-making and operational efficiency [18,19]. EMIS collects, processes, and analyzes vast amounts of energy data, enabling real-time monitoring, predictive maintenance, and strategic planning, which are essential for optimizing EMS operations and facilitating the transition to more sustainable energy practices. Within EMIS, advanced metering infrastructure (AMI) is identified as a crucial component, offering critical capabilities for real-time data collection and communication [20,21]. AMI can play a vital role in the optimization of power distribution networks in terms of efficient grid management, effective DR programs, and the integration of RESs. In addition, the facilitation of enhanced monitoring and control of energy usage can be achieved through AMI.

To address the increasing complexity and demands of power systems, GASHS are emerging for their ability of improving the functionality and resilience of EMS [22]. Grid automation can automate grid operations, enhancing efficiency, reliability, stability, and adaptability to changing energy demands through the deployment of advanced control technologies. Faults within key component systems in EMS are autonomously detected, diagnosed, and rectified, significantly enhancing the reliability and efficiency of EMS.

ESS has been researched for managing demand fluctuations and energy supply within integrated RESs and various distributed sources in EMS [23]. ESS enables the effective storage and release of energy, facilitating the integration of intermittent RES, enhancing grid stability, and ensuring a reliable and continuous energy supply. The capability of the ESS is pivotal for the efficient operation of EMS, particularly in balancing the variability inherent in RES.

ETRMS has been regarded as a crucial component in the evolving and interconnected power market [17]. Through ETRMS, the operational efficiency of EMS can be enhanced by the effective management of revenue and the minimization of associated risks. Furthermore, ETRMS is instrumental in providing accurate forecasts of market trends, optimizing trading strategies, and mitigating financial risks. The systems for forecasting market trends, optimizing trading strategies, and mitigating financial risks are provided by ETRMS. As a result, ensuring the economic efficiency of EMS operations can be accomplished. ETRMS can also play a role in the context of integrating RES and managing fluctuating energy demand in dynamic electricity markets.

DSMS is implemented to regulate and control the amount and timing of energy consumption on the demand side [24]. Contrary to traditional power systems, which primarily focus on augmenting supply to meet demand, DSMS aims at adjusting or reducing demand to align with the available supply. DR has been recognized as a pivotal component of DSMS, playing a crucial role in balancing supply and demand within the electricity market [24,25]. DR can be categorized as incentive-based and price-based programs in retail markets, as well as capacity, ancillary, and energy market programs in wholesale markets [25]. The approach of DSMS can be conducive to more efficient energy utilization, enhancing reliability and stability of the power grid, leading to cost savings, and yielding environmental benefits.

EMS operations are responsible for overall monitoring, scheduling, controlling, and effective interconnection among various component systems. Based on the nature of the supervisory operation, EMS can be categorized into four types: centralized, decentralized, distributed, and hierarchical architectures, as shown in Figure 2 [8,9,13,14,26]. The selection of the four types of architectures can be influenced by factors such as grid size, the variety of energy sources, grid topology configuration, and requirements for resilience and adaptability. In the implementation of ML-based EMS, it is necessary that the frameworks of control and supervision are considered, along with the characteristics of the applied ML techniques, to ensure the effective integration of ML approaches within the varied EMS architectures.

2.1. Centralized EMS Framework

In the centralized framework (Figure 2a), overall decision-making and control processes are managed by a single central unit. The central unit is characterized by its ability to gather all necessary energy information and execute optimal power scheduling for establishing an efficient EMS. Subsequently, decisions are communicated to local units from the central unit. The optimized operation of the integrated system is made possible through a single control center, where all data are managed, enabling systematic decision-making and processing [13,14]. The adoption of the optimized integrated system enables global optimization of the entire grid, encompassing load balancing, fault management, and efficiency enhancements. The occurrence of conflicting tasks is minimized by maintaining a consistent control strategy across the centralized grid, thereby ensuring high reliability. In the centralized framework, real-time observation of the entire system and straightforward implementation can be obtained, offering significant operational advantages. However, with overall energy information being managed by a central device for decision-making, the necessity for robust data security and management arises [14,26,27]. Furthermore, in terms of scalability, the necessity to address challenges such as computational load and recovery and response delays in the power grid due to central system failures is recognized as requiring resolution [14].

2.2. Decentralized EMS Framework

In the decentralized framework (Figure 2b), multiple controllers are tasked with distinct segments of the system, each handling specific control functions. In the decentralized framework, energy information is independently gathered by local units, contributing to power scheduling [13,14,27,28,29,30,31]. These systems are capable of rapid response and adaptation to local conditions, reducing communication costs and increasing the efficiency of the EMS [28]. Load balancing, fault resolution, increased flexibility, and overall reliability are promoted through the application of optimal scheduling in a decentralized framework [14,31,32,33]. Decentralized frameworks, particularly beneficial for complex systems, such as DERs, can allow significant benefits to be added through local control [34]. However, advanced coordination among units is necessary in decentralized control for efficient operation, and the challenge of consistency between local and global objectives is acknowledged [35]. Therefore, the implementation of both standalone and cooperative modes in local controllers can be considered crucial for effective operation.

2.3. Distributed EMS Framework

In the distributed framework (Figure 2c), control functions are further decentralized through distribution across a network of interconnected controllers [13,15]. Numerous units in systems are interconnected to process energy information and execute decisions independently based on their environment [36]. By sharing information and decisions among distributed control units, the scalability and efficiency of the system can be improved [26,36]. Additionally, the impact on the entire grid of the failure of a single controller can be minimized by automatic distribution system recovery. However, distributed frameworks can face the challenges of distributed control in terms of computational complexity, communication overhead, and privacy concerns. The lack of centralized control in a system consisting of several participating entities may lead to conflicting decision-making. In addition, the number of controllers can increase vulnerabilities in terms of cybersecurity. To maintain stability and reliability, it is necessary to ensure efficient adaptation and scalability through various methods. Therefore, advanced techniques based on ML are required to manage the increasing interactions and massive data processing in a distributed framework [37].

2.4. Hierarchical EMS Framework

In the hierarchical framework (Figure 2d), a combination of centralized and decentralized control mechanisms is employed, creating a layered approach to system management [14]. Efficient and scalable control across the EMS can be promoted by a hierarchical framework that incorporates the advantages of both centralized and decentralized systems [27,38]. In the hierarchical framework, the coordination and management functions of a centralized system are combined with the local task processing of a distributed system. At the top level, overall planning and coordination are handled by centralized control, while at the local level, the independent achievement of local goals by distributed units is realized. The resilience of the EMS to single points of failure can be enhanced, and the reliability of the system can be improved by combining centralized control with distributed execution. Furthermore, through optimal control at each level, the scalability of the EMS can be improved [38,39,40]. However, the complexity inherent in managing a hierarchical framework poses challenges in managing data integration and cross-layer communication. Therefore, effective integration of multiple systems is required to ensure high adaptability, reliability, efficiency, and scalability in the hierarchical framework.

When ML methods are applied to each EMS framework, the processing time and memory requirements are considered crucial. These requirements are influenced by various factors, such as the processing capability of the hardware and the computational complexity of the algorithm. In ML-based EMS frameworks, prolonged processing times are often necessary for handling large-scale systems. In the centralized EMS framework, increased memory requirements and extended processing times are encountered due to data handling from the entire network. In contrast, in decentralized and distributed EMS frameworks, a substantial reduction in processing time and memory requirements can be achieved, attributed to the parallel processing capabilities of distributed local controllers.

3. ML-Based EMS Approaches

In order to implement an efficient EMS, various sources and systems could be interconnected and integrated to ensure system reliability. EMIS, GASHS, ESS, ETRMS, and DSMS are the key components needed to achieve optimal power flow and efficient energy management. In this subsection, a detailed description is presented regarding the integration of ML with each critical component of EMS, including EMIS, GASHS, ESS, ETRMS, and DSMS. It is demonstrated that energy management capabilities can be enhanced by adopting ML-based EMS, thereby contributing to improvements in efficiency, adaptability, reliability, and overall performance. In

Table 1. Promising ML candidates for each EMS framework.

Framework	Characteristic	Promising ML Candidate
Centralized EMS	Advantages - Centralized control over model training and deployment - Consistency in data processing - Efficient data management - Robust computational resources Drawbacks - Handling highly distributed or localized data - Potential for a single point of failure - Limitations of scalability	Naive Bayes - Simplicity, efficiency, and effectiveness in handling high-dimensional data for classification - Fast processing time and memory efficiency Principal component analysis - Dimensionality reduction and enhanced data interpretability - Fast processing time and low memory requirements by removing redundant features Regression analysis - Precise estimation and simple implementation - Moderate processing time and memory requirements Supervised learning - Enhanced interpretability by a direct feedback mechanism - Requiring labeled data and high memory requirements for large datasets Support vector machine - Effectiveness in high-dimensional spaces - Slow processing time for large datasets and hard-to-apply large datasets
Decentralized EMS	Advantages - Adaptability to local conditions - Enhanced privacy and security - Potential for scaling across distributed nodes - Resilience to single points of failure Drawbacks - Increased complexity in coordination and communication between nodes - Data consistency - Possible inefficiencies due to data fragmentation	Learning-based neural ant colony optimization - Adaptable to complex and high-dimensional optimization problems - High computational costs and sensitivity to the number of hidden-layer nodes Multi-agent reinforcement learning - Optimization of complex problems and learning from diverse perspectives - Increased operation costs due to local optimization Representation learning - Efficient handling of unstructured data and adaptability to variations in data - Requiring substantial computational resources Reinforcement learning - Effectiveness in control tasks and sequential decision-making - High computational costs and memory intensive for complex environments Self-organizing map - Efficient decentralized data clustering and visualization - Challenges in dynamic and real-time applications
Distributed EMS	Advantages - Ability to process large and complex datasets - Enhanced privacy and security through data localization - Flexibility in handling diverse computational tasks - Scalability Drawbacks - Increased complexity in managing distributed systems - Potential challenges in data synchronization - Possible inefficiencies due to network latency	Contrastive learning - Effectiveness in learning robust representations - Slow processing time and high memory requirements due to the need for processing pairs DL-driven particle swarm optimization - Effectiveness of finding global optima in complex spaces - High computational cost and memory requirements due to the iterative nature of both DL and swarm optimization Distributed learning - Facilitating parallel processing and enhancing scalability - Fast processing time and high memory requirements due to parallelism Federated learning - Enabling collaborative learning across multiple nodes while maintaining data privacy - High computational costs due to iterative model updates and less efficient than centralized learning in terms of convergence speed Unsupervised learning - Handling unlabeled data and finding hidden patterns - Moderate to high computational costs due to difficulty in capturing complex relationships
Hierarchical EMS	Advantages - Efficient data processing and decision-making by structured layer - Fragmentation of complex tasks - Facilitating modular and scalable model design Drawbacks - Challenges in managing and tuning in multiple layers - Potential issues related to data flow and integration across different levels	Auto-encoder - Efficient dimensionality reduction, denoising data, and feature extraction - Slow processing time and high memory requirements due to complex training Ensemble learning - Enhancement of robustness and reduction in variance and bias - Slow processing time and high memory requirements due to training multiple models Transfer learning - Providing reduced training time and overcoming overfitting - Fast processing time and lower data requirements by utilizing pre-trained models Semi-supervised learning - Efficient use of available labeled data and improvement over time with more data - Requiring moderate to high computational resources Self-supervised learning - Learning useful representations without labeled data - High computational costs as it often involves complex large-scale models

, promising ML candidates are presented as potential candidates, selected based on the characteristics of each EMS framework.

3.1. ML-Based EMIS

Data collected from users via AMI are leveraged by EMIS, an intelligent data management system, to boost operational efficiency. Core technologies for EMIS can be categorized into two groups: enhancing system stability and facilitating data collection applications. Central to system reliability, technologies focused on fault detection and cybersecurity are considered. The importance of cybersecurity has been highlighted by considering cyber-attacks on supervisory control and data acquisition (SCADA) systems [41,42]. Smart infrastructures, fundamental to smart cities, are closely connected to vital services, such as transportation, water, and gas distribution [43]. A threat to these infrastructures can lead to significant socio-economic losses. Therefore, the exploration of ML-based security methods is imperative.

In addressing cybersecurity for EMS, a strategy involving the isolation forest model has been proposed [44]. By employing unsupervised ML, data integrity attacks can be identified within communication networks. A notable improvement in detection capability has been demonstrated by the whale optimization-based artificial neural networks (ANN) method, outperforming random forest and support vector machines (SVMs) methods and achieving high accuracy [44]. Additionally, a high detection rate has been reported in research utilizing ANN to detect man-in-the-middle attacks [45]. To counter the challenges posed by imbalanced datasets, enhanced performance is demonstrated by a model based on stacked auto-encoders, surpassing traditional methods like random forests, AdaBoost, and deep neural networks (DNNs) [46].

To ensure the reliability and safety of EMS operations, capabilities for identifying malfunctions may be required with data collected from various information systems [47]. Therefore, to minimize malfunctions and enhance the performance of intelligent power systems, fault detection in EMS may be vital. ML-based intelligent control research has been studied to enhance the reliability of EMS. Fuzzy reinforcement encoders have been employed as an effective method for fault detection and classification [48]. To predict faults, a fuzzy-based fault diagnosis method has been employed at the edge of the network, gathering signals from cloud-based smart sensors, and achieving high accuracy. Furthermore, using a 1D-convolutional neural network (CNN) for fault classification has been shown to improve the performance of fault detection and classification compared to traditional methods [49].

Data collected from EMIS are utilized in applications such as energy theft detection, load monitoring, and energy forecasting. Substantial economic losses are incurred annually worldwide due to energy theft [50]. Consequently, techniques for energy theft detection have been extensively studied to minimize these losses. Time-series data are predicted and analyzed using recurrent neural network (RNN)-based methods to identify anomalous behavior [51,52]. Methods have been proposed to perform energy theft detection by combining long short-term memory (LSTM) networks with clustering algorithms [51] or by combining gated recurrent unit (GRU) networks with CNNs [52]. However, RNN-based methods may suffer from the vanishing gradient problem in long sequence data. The transformer-based EMIS method has been employed to extract relevant data features by mitigating the vanishing gradient problem [53]. By addressing the vanishing gradient problem, transformer-based methods allow for a more concise extraction of data features compared to RNN-based methods [53]. In general, the data imbalance problem may occur in energy theft data, as there are fewer theft users compared to normal users. The biased number of users can lead to biased training of deep learning models, which can degrade model performance in detecting unseen data. To mitigate the biased learning problem, data augmentation with a variational auto-encoder [54] or methods with a generative model [55] have been proposed, which can mitigate the data bias problem compared to traditional methods.

Load monitoring refers to the method of measuring the electricity load of users using meters, such as AMI, to identify and monitor the usage of electrical appliances. Intrusive load monitoring (ILM) and non-intrusive load monitoring (NILM) are two general classifications of load monitoring [56]. In ILM, energy usage can be directly measured by installing a separate sensor on each appliance. The ILM-based method can provide accurate energy usage for each appliance, but it requires many sensors and is expensive to install. Conversely, only a single or small number of sensors are used to measure the electricity usage of an entire home or building and to estimate the usage of individual appliances in NILM. The combination of ML and NILM methods has been studied, attributed to their low installation costs and potential for enhanced analytics through data analysis. ML-based NILM methods have been proposed based on feature extraction, such as SVMs [57] and decision trees [58]. Recently, deep learning-based methods utilizing CNN and LSTM algorithms have been proposed to improve performance by capturing nonlinear features [59]. Furthermore, the Transformer-based NILM method has been proposed to improve prediction accuracy by capturing long-term dependencies in sequential data [60].

As the demand for sustainable energy increases, research on the prediction of renewable energy production has been conducted [61]. Due to the unpredictable and uncertain nature of RES, challenges in representing the intricate properties have been encountered by conventional statistical [62] and physics-based [63] predictive models. Attempts have been developed to address these issues by employing the implementation of ML algorithms in forecasting generation [63,64,65,66]. Regression analysis has been proposed based on a support vector regression [64] algorithm for forecasting generation prediction of multiple RES. A CNN-based model [65] has been developed for forecasting hourly global horizontal solar irradiance. Additionally, approaches incorporating convolutional long short-term memory networks (CNN-LSTM) [66], taking both spatial and temporal features into account, have been proposed.

3.2. ML-Based GASHS

To improve EMS efficiency, resilience, continuity, and reliability, the role of GASHS in implementing self-healing systems has been emphasized [22]. Particularly in distributed architectures, where complexity is inherent, self-healing control systems are essential. These systems play a crucial role in swiftly restoring operations affected by faults and facilitating alternative operations within distributed networks, thereby increasing the continuity of energy services [37,67].

Self-learning systems that utilize artificial intelligence and cloud computing are regarded as critical components for establishing flexible and adaptive automation systems [15]. A novel distributed ML approach based on feature selection for detecting dynamic power system events has been proposed [68]. This approach, which enables autonomous post-fault restoration without central intervention, involves classifying distinct events using multiclass classification algorithms applied to generator data in multi-generation schemes. In addition, a real-time risk analysis and early warning system for the EMS, based on a stack self-coding neural network prediction model, have been proposed [69]. This system aids in load dispatching and reduces generation costs. The complexity in decision-making for energy production and distribution is attributed to the challenges posed by uncertainties in energy load forecasting. To address these challenges, self-learning data-based forecasting models employing regression and clustering methods have been proposed for cost-efficient operation [70]. Moreover, load forecasting methods based on LSTM and gradient tree boosting algorithms have been developed to design autonomous EMS with integrated uncertainty estimation methods [71]. In [72], ML-based distributed offline-online architecture has been proposed to enhance decision-making in system restoration and resiliency. This approach, which leverages relevant features from a reduced time series dataset, not only enhances the accuracy of decision-making but also contributes to the development of a self-healing structure in power systems. In

Table 2. Characteristics of EMIS and GASHS in terms of key technologies and applications.

Component Systems	EMS Framework	Key Technology	Application
EMIS	- Centralized: Integrated control and decision-making requirements	- AMI - Big data analytics - Fault detection	- Real-time monitoring - Predictive maintenance - Strategic planning
GASHS	- Distributed or Hierarchical: Allowing for localized decision-making and balancing central supervision	- Self-healing code - Cloud computing - Cybersecurity	- Autonomous post-fault restoration - Real-time risk analysis - Load dispatching

, the characteristics of EMIS and GASHS are summarized by EMS frameworks regarding key technologies and applications.

3.3. ML-Based ESS

ESS has been recognized as a main component in enhancing operational efficiency in EMS integrated with variable generation resources [73]. In the context of ESS, research related to ML has focused on three aspects: the optimization of operational costs with scheduling, enhancement of the DER with frequency regulation, and revenue with power trading. In [74], the state of health of ESS is diagnosed by principal component analysis, thereby facilitating improved accuracy and efficiency in system maintenance and operation.

Scheduling methods have been explored to maximize the utility of surplus electricity in residential ESS applications. Conventional control strategies, such as model predictive control [75] and dynamic programming methods [76], have been applied, particularly in residential solar applications. However, significant mismatches between supply and demand may result from these conventional methods when the model is inaccurate, leading to the waste of surplus power and demonstrating scalability limitations. To address conventional problems and diminish prediction errors without dependence on specific models, investigations into ML-based ESS scheduling approaches have been conducted. Notably, one such approach is the application of batch reinforcement learning (RL)-based Q-iteration algorithms, which aim to identify optimal scheduling plans for batteries [77].

The accelerated deployment of RES has been identified as one of the contributors to the degradation of system frequency stability. Various studies have been conducted to address the issue of frequency stability, focusing on the enhancement of frequency stability with ESS [78,79,80]. Response-driven frequency control (RDFC) has been the primary focus of previous studies [78]. However, the flexibility of RDFC is limited, as it has been found incapable of discerning the nature of specific incidents. Therefore, proposals have emerged advocating the use of ML algorithms to predict appropriate load shedding in response to specific events [79].

In the context of power trading, research has been conducted on ESS as a battery resource for energy trading and management. ESS has been integrated into centralized decision-making models for scheduling [80], and game-theoretic models have been utilized to optimize auction mechanisms between the ESS and the trading market [81]. However, previous approaches may lack flexibility in rapidly changing systems. To enhance the flexibility in charging systems, ML-based methods have been proposed in power trading [82,83]. For efficient energy management in microgrids, a dual-level energy management strategy involving centralized RL agents has been proposed in cooperative microgrid scenarios [82]. Additionally, a multi-agent RL approach has been proposed, allowing each microgrid to autonomously make decisions, thereby optimizing the utility of ESS and energy trading simultaneously [83].

3.4. ML-Based ETRMS

The utilization of RES has been identified as an effective strategy to reduce the cost of power generation and transmission over wide areas. There has been a shift from centralized to decentralized systems, with microgrids emerging as critical components in the efficient management of distributed energy resources [84]. By integrating microgrids into the power system, market participants have been able to participate in energy trading markets with the dual objectives of generating revenue and mitigating energy shortages. Various peer-to-peer trading methods have been researched to efficiently manage energy transactions in markets integrated with DERs [17]. It has been identified that decentralized models can be more suitable for the energy trading market compared to centralized systems [85] due to their flexibility in adapting to new technologies, including RES, ESS, and DR.

Approaches for energy trading have been researched, including auction processes [86], game theory [87], blockchain [88], single-agent methods [89], and multi-agent methods [90]. In auction processes, ML algorithms have been employed to predict energy production [86]. Furthermore, benefits for participants in game theory-based trading markets have been improved through the application of model-free RL and market reputation indices [87]. A consensus algorithm has been proposed to detect fraudulent nodes. In this algorithm, future consumption is predicted based on historical data aggregated in the blockchain, utilizing a GRU-based algorithm [88]. By employing the GRU-based algorithm, the facilitation of a fair energy trading platform can be achievable. Situations involving uncertain participation by numerous players have been addressed in extensive studies on multi-agent methods [90]. Comparative analysis has demonstrated that multi-agent methods are more effective in generating greater revenue than single-agent methods [90]. A multi-agent RL-based EMS has been proposed to optimize resource usage and maximize profits while maintaining reliability and efficiency in a decentralized EMS framework [28].

3.5. ML-Based DSMS

The improvement of power system performance, enhancement of stability, reliability, security, efficiency, and reduction of environmental effects can be achieved by DSMS in EMS operations [24]. Techniques employed in DSMS are categorized into several types, namely peak clipping, flexible load shaping, valley filling, strategic growth, load shifting, and load reduction. In [25], DR schemes are discussed using goal-oriented classifications, such as market-based, motivation-based, and system-based approaches, where the benefits and barriers of DR are also introduced. Focusing on large-scale integration in automated distributed EMS, the utilization of advanced artificial intelligence techniques in DR has been reviewed [15].

An hour-ahead DR algorithm employing an ANN-based price prediction model and multi-agent RL-based decision-making has been proposed [91]. In [92], a novel DSMS based on hybrid bacterial foraging and particle swarm optimization is developed to optimize electricity costs, peak-to-average ratio (PAR), carbon dioxide emissions, and user comfort. ML-based DR programs have been reviewed for optimal dynamic pricing, scheduling, and control strategies using deep learning, supervised, unsupervised, and RL algorithms [93,94]. The traditional model-based EMS struggles to adapt to the varying operational conditions in the DER, leading to inaccurate energy consumption schedules. Consequently, a novel approach employing federated RL within a hierarchically distributed EMS framework has been proposed [95]. This methodology is characterized not only by its capability to enable optimal energy scheduling but also by its substantial reduction in communication overhead while maintaining stringent data privacy. A regression analysis-based DSMS has been proposed in the centralized EMS framework for minimizing grid power injection and maximizing the efficiency of hybrid energy sources [96].

Due to the nature of model-free RL algorithms, power systems can be controlled by using algorithms even without the given explicit model. Therefore, the RL-based method has attracted the attention of researchers and has been employed for large-scale online decision-making systems to improve EMS stability [97]. A real-time multi-agent DSMS based on home area networks is proposed, integrating deep Q-network agents for adaptively controlling appliance usage and ESS, which successfully reduces PAR and electricity costs [98]. An intelligent multi-microgrid energy management method that employs a data-driven DNN and model-free RL has been proposed [99]. This method enables a distribution system operator to optimize retail pricing and demand-side PAR without direct access to user information, thereby ensuring both security and efficiency. A novel privacy-preserving load control scheme for managing various home appliances while safeguarding user privacy has been introduced [100]. This scheme, achieved through a partially observable Markov decision process and deep RL with a credit assignment mechanism, shows enhanced efficiency, cost-effectiveness, and user experience. A two-step real-time DR model has been developed [101], utilizing a cloud-fog-based system, a generative adversarial network with a Q-learning model, and cloud computing, focusing on privacy protection. The first step involves scheduling and an incentive scheme based on a discounted stochastic game, while the second step addresses privacy concerns from the strategy analysis of the DR model.

An SVM-based forecasting model has been proposed to enhance forecast accuracy and stability in the day-ahead market due to the scarcity of studies forecasting the available aggregated DR capacity from a load aggregator perspective [102]. A deep RNN with a GRU-based forecasting model for load demand with a one-hour resolution has been developed [103] to accommodate short- to medium-term dependencies and achieve high accuracy. An optimal load dispatch model, aimed at promoting supply–demand balance and aiding in cost reduction and system reliability improvement, is based on forecasting with a deep RNN using the LSTM algorithm [104]. Furthermore, an LSTM-based real-time pricing DR model has been proposed [105] aimed at improving the efficiency of energy usage and the effectiveness of DR strategies. To implement reliable DR by capturing internal changes and improving prediction accuracy, a multivariate load prediction model based on a pre-attention mechanism and CNN has been proposed [106]. To mitigate the mismatch between energy supply and demand and improve profitability and system efficiency, a real-time incentive-based DR scheme integrating RL and DNN has been proposed [107].

In [108], an incentive-based DR program combining RNN and RL has been introduced to optimize profits for both energy service providers and end users, using an RNN-based forecasting model to predict day-ahead wholesale electricity prices, photovoltaic power output, and power load. A real-time DR management strategy based on the twin delayed deep deterministic policy gradient learning approach has been proposed [109] to overcome inaccuracies in the operating models of DERs and the probabilistic modeling of uncertain parameters. This approach has been validated as effective in reducing energy costs and adeptly handling uncertainties from electricity prices and the supply–demand dynamics of DERs. In [110], an integrated energy management system has been proposed to improve energy management performance in terms of profitability, stability, and burden reduction to the grid. This system is optimized using the proximal policy optimization algorithm, known for its effective learning stability and complexity. In addition, a novel performance metric, the burden of load and generation, has been proposed to evaluate comprehensive energy management performance, incorporating it into reward settings for optimizing the management of multi-action controls. In

Table 3. Characteristics of ESS, ETRMS, and DSMS in terms of key technologies and applications.

Component Systems	EMS Framework	Key Technology	Application
ESS	- Centralized: Control for energy supply management - Hierarchical: Decision-making for balancing DER	- Optimization of operational costs - Voltage and frequency regulation	- Managing demand fluctuations - Enhancing grid stability - Integration of intermittent RES
ETRMS	- Decentralized: Flexibility in adapting to dynamic market conditions and decision-making for energy trading	- Market trend forecasting - Trading strategy optimization	- Managing profits and risks in energy trading - Integrating RES - Managing energy demands
DSMS	- Distributed: Efficient demand-side energy usage control in DER	- Incentive-based and price-based DR programs	- Regulating energy consumption - Aligning demand with supply - Enhancing power grid reliability and stability

, the characteristics of ESS, ETRMS, and DSMS are summarized by EMS frameworks regarding key technologies and applications.

4. Operational Constraints and Challenging Issues

4.1. Operational Constraints in ML-Based EMS

Integrating ML-based EMS approaches, a diverse array of operational constraints has been recognized as pivotal for ensuring the safe and reliable operation of energy systems [12,14,111,112,113,114,115,116,117]. These constraints, varying with specific applications and components within EMS, typically encompass power balance constraints [14,17], grid connection constraints [114], energy storage constraints [115], economic constraints [116], and environmental constraints [117].

Efficient load allocations are ensured by power balance constraints, which confirm that the total power supplied to the system is equal to the total power consumed, considering system losses. This balancing act is fundamental to maintaining a stable equilibrium between energy supply and demand, thereby preventing power outages and circumventing the overloading of system components.

Energy storage constraints, formulated to prolong the lifespan of ESS batteries, involve regulating energy capacity, efficiency, and charging and discharging rates. These constraints are pivotal in optimizing ESS operations and preventing overcharging or excessive discharging, which can cause damage and reduce the system’s lifespan. Additionally, with the expansion of RES, energy storage constraints have become instrumental in effectively managing fluctuations in generation and load variations.

Grid connection constraints, including limits on maximum power exchange and frequency response requirements necessary for grid stability, are critical in EMS optimization. These constraints ensure safe and reliable operation, especially regarding voltage and frequency regulation. Parameters must be maintained within specific limits to safeguard the safety and reliability of electrical components. In hybrid EMS models, the control of generators and inverters is carefully optimized to maintain voltage and frequency within acceptable ranges.

Environmental constraints, considering the impact of renewable energy utilization, are influenced by external factors, such as weather conditions and geographic location. These constraints are critical in optimizing EMS to ensure a reliable and cost-effective energy supply while adhering to environmental standards. Economic constraints, encompassing operational costs, infrastructure investment, and the dynamics of energy trading, are crucial for economic viability. Therefore, models to analyze market trends, predict costs, and optimize pricing strategies are needed to enhance economic efficiency.

Technological constraints in the integration of advanced ML-based approaches within EMS may include limitations in computational capabilities, data availability, and the accuracy of predictive models. Continuous development and refinement of advanced technologies are undertaken to address these technological constraints, enhancing the overall efficiency and effectiveness of EMS.

Human factors and user behavior also influence EMS design and optimization. Understanding and predicting user behavior, energy usage patterns, and responses to DR programs are essential for creating user-centric energy management strategies. Increasingly, these factors are being incorporated into predictive models to tailor energy solutions to specific user needs and preferences.

In addition to operational constraints, inherent challenges and specific limitations are also introduced by the architectural frameworks of EMS. In a centralized EMS framework, where control and decision-making are integrated at a single point, constraints related to scalability and resilience are encountered. As the size and complexity expand, these systems are often overwhelmed, leading to potential inefficiencies and vulnerabilities. Moreover, centralized systems are characterized by a single point of failure, which can compromise the stability of the entire grid in the event of cyber-attacks or system faults. Decentralized EMS frameworks, which distribute control across various subsystems or regions, face challenges in coordination and communication. The consistent implementation of policies and effective communication between decentralized units in the absence of a central authority are difficult to ensure. Distributed EMS frameworks decentralize control further, often to individual energy resources like solar panels or storage units. These systems are challenged by significant constraints in data management and processing arising from the high volume of data generated by various distributed sources. The effective integration of this data for informed decision-making across the grid, while maintaining privacy and security, poses a major challenge. Hierarchical EMS frameworks, combining elements of centralized and decentralized systems, introduce complexity in system design and operation. Balancing authority and decision-making power between different hierarchical levels and ensuring efficient communication and coordination across these levels are recognized as significant constraints.

Interoperability is identified as a key constraint across all these architectures. The integration of legacy systems with newer technologies in EMS necessitates seamless interaction and data exchange between various components and systems. This integration requires standardization in communication protocols and data formats, a challenge that is often difficult to achieve universally. Cybersecurity constraints are overarching across all EMS architectures. With the grid becoming more interconnected and reliant on digital technologies, the risk of cyber-attacks escalates. Robust security measures to protect against threats and vulnerabilities are essential for all EMS frameworks.

Regulatory constraints play a significant role, with legislation and policies governing energy usage, distribution, and conservation impacting the design and operation of EMS. Compliance with these regulations is crucial for the lawful and ethical functioning of energy systems. Regulations concerning energy distribution, data privacy, and security can limit the design and operational choices available for EMS.

The various operational constraints within EMS are both multifaceted and critical, encompassing a broad spectrum from power balance to user behavior. These constraints play an essential role in shaping the design and optimization of EMS architectures, ensuring that the systems operate safely, reliably, and in compliance with regulations while maintaining economic viability. This may include leveraging advancements in artificial intelligence technologies, which are instrumental in enhancing the system’s capabilities and adapting to the dynamic nature of energy management. Additionally, a deep understanding of various factors, such as technological, regulatory, and human behavioral aspects, is continually integrated into the development of EMS. A comparison of the major operational constraints in ML-based EMS is given in

Table 4. Comparison of major operational constraints in ML-based EMS.

Constraints	Primary Issues	Implications for EMS Design
Architectural Framework	- Scalability, efficiency, and resilience for each EMS framework	- Tailored approaches for robustness and efficiency in each EMS framework
Cybersecurity	- Cyber threats in digital-dependent systems	- Essential security across all components of EMS architectures
Economics	- Operational costs, infrastructure investment, and energy trading dynamics	- Economic viability models based on cost prediction and pricing strategies
Energy Storage	- Regulation of energy capacity, efficiency, and charge/discharge rates for battery life extension	- Optimization of ESS operations and management of generation and load fluctuations
Environment	- RES utilization influenced by weather and geographic location	- Reliable and cost-effective energy supply with adherence to environmental standards
Grid Connection	- Power exchange and frequency response requirements for grid stability	- Maintaining voltage and frequency regulation
Human Factors	- Dynamic user behavior and energy usage patterns	- User-centric energy management strategies
Interoperability	- Standardization in communication protocols and data formats for system integration	- Achievement of universal standardization
Regulation	- Restrictions imposed by regulations on energy distribution, data privacy, and security	- ML-based EMS design and operational choices through regulations
Power Balance	- Equilibrium of power supply and demand, accounting for system losses	- Stability of EMS by prevention of outages and component overloading
Technology	- Computational capabilities, data availability, and accuracy of predictive models	- Enhancement of system efficiency and effectiveness by technological advancement

.

4.2. Operational Challenging Issues in ML-Based EMS

In EMS, the integration of advanced energy information and communication technologies with various power networks has become essential, enabling adaptive power control and connectivity within distributed and non-stationary environments. However, optimal energy management by EMS faces numerous challenges, encompassing not only complexity and uncertainty but also legal, technical, economic, societal, and sustainability issues [115]. Furthermore, scalability issues emerge as a pivotal challenge for the successful implementation and operation of advanced EMS in increasingly complex and expanding energy management environments. Scalability issues are multifaceted and influenced by the size and integration of additional systems, the inherent complexity of these systems, the volume of data processed, and the dynamic nature of grid environments. Therefore, ML-based predictive analytics is required to develop scalable solutions that efficiently adapt to changing conditions in large-scale environments and the nature of uncertainty. Energy forecast models have been proposed for predicting energy consumption patterns and managing power quality issues, thereby enhancing the scalability of EMS. To address scalability, information availability, and network latency issues in EMS, fog-computing infrastructure and fuzzy logic paradigms have been employed, effectively managing the challenges associated with incorporating various input parameters [118]. In [119], security and privacy issues related to the incorporation of ML-based applications into EMS have been discussed. In particular, the importance of research on data security issues in model updates, memory storage issues, information error issues, and response systems has been increasingly recognized.

While the incorporation of ML techniques into EMS presents considerable advantages, it additionally happens that it presents susceptibilities that could be profited from via model updates. Therefore, an update to an integrity-enforced operation is critically required to prevent malicious activities, such as the injection of false data or code that could compromise the stability of the grid. Regarding memory storage issues, the vast amount of data can be identified as a lucrative target for cyberattacks, which includes private and operational information collected and stored in EMIS. In this context, the implementation of measures to safeguard the privacy of EMS data is recognized as essential. These measures are necessary not only to protect the data from unauthorized access but also to ensure its confidentiality, which is important in maintaining the reliability and security of EMS. Furthermore, concerns regarding information error issues in EMS have been raised, particularly about their implications for decision-making. The research focused on identifying unusual patterns indicative of a cyberattack and performing appropriate responses in real-time for the development of robust error detection and correction mechanisms.

From the viewpoint of EMS, the complexity lies in its multi-dimensional nature, which involves diverse interactions between energy control systems, non-stationary demand and supply patterns, handling uncertainty, and fluctuating market dynamics. In [120], the need to balance economic profitability, computational complexity, and security in designing effective EMS using forecasting-based optimization has been emphasized. Research on complexity management in energy system analysis has been conducted to consider fine-tuning the trade-off of model performance [121]. In addition, the four dimensions of temporal, spatial, mathematical, and system scope modeling have been analyzed with complexity profiles. Consequently, handling these challenging issues is crucial for the successful implementation and operation of advanced EMS in increasingly complex and expanding environments. In

Table 5. Primary challenging issues in ML-based EMS in terms of operational issues.

Operational Index	Challenging Issue
Scalability	- Adaptive ML algorithms for scalable solutions - Reduction of computational complexity in ML models - Handling continuous data streams for real-time analysis - Effective ML operation within integrated systems - Optimized ML models in non-stationary environments - Developing efficient ML-based EMS architectures
Security	- Incorporating ML to enhance cybersecurity and data privacy - Enhanced fault tolerance in ML-based EMS - Modified encryption methods for data-in-transit and data-at-rest - Implementing robust ML-based security protocols to prevent unauthorized access and breaches - Utilizing ML for real-time anomaly detection and potential threats - ML-based self-restoration from technical and operational faults
Storage	- Data lifecycle management in line with regulatory requirements - Effective data compression methods by ML-based feature extraction - Enhanced ML-based monitoring and management systems - Hybrid storage solutions for multi-objectives of ML-based EMS - ML-based key information selection and sharing technologies - Handling large-scale ML-based EMS data in limited memory capacity

, primary challenging issues in ML-based EMS are provided in terms of the operational index.

4.3. Technical Challenging Issues in ML-Based EMS

In this section, the technical challenges of assessing various performance metrics in ML-based EMS are discussed, including accuracy, fault tolerance, model robustness, computational efficiency, and the impact on overall system performance. ML-based EMS are faced with challenges in model evaluation, which may include difficulties in assessing performance effectively in a comprehensive system. These challenges are further compounded by the consideration of a broad spectrum of performance metrics, which may include PAR, energy consumption, maximum demand, load reduction, carbon dioxide emissions, delay time, battery degradation, battery depth of discharge, state of charge, available energy by plug-in electric vehicles, profit, total operating cost, voltage, and frequency fluctuation, etc. In addition, forecasting error, burden on the grid, and sensitivities of operational reserves with load reduction and cost can be utilized to evaluate sustainability. The accurate measurement and interpretation of these metrics are regarded as essential for comprehensively understanding the performance of EMS. However, the interaction among these diverse metrics poses a significant challenge in achieving an overall perspective on the efficiency and effectiveness of EMS. To address these challenges in performance evaluations, developing performance metrics aligned with dynamic requirements is required.

Existing performance metrics for evaluating grid burden reduction have traditionally been evaluated in partial aspects, such as PAR. However, this approach often overlooks comprehensive energy management and the dynamic interactions within it. To address these issues, a general performance metric for evaluating grid burden is required that encompasses the entire architecture and assesses the system based on its interactions. While PAR is a crucial metric for measuring the efficiency of load management, it does not necessarily reflect the responsiveness of the EMS to fluctuating demands or the integration efficiency of various RES.

In the evaluation of ML-based EMS performance from an ML perspective, several key performance indicators are recognized as essential. In the forecasting model, the alignment of accuracy and precision with actual energy consumption and demand patterns is evaluated. An inaccuracy in prediction models of EMS can lead to inefficiencies, such as over- or under-generation of power, impacting overall sustainability. The assessment of fault tolerance and error handling capabilities should also be considered critical to ensure minimal performance degradation in the presence of data anomalies.

The evaluation of model robustness necessitates the consideration of complexities in data processing and management, particularly focusing on the models’ resilience against data variability and their ability to sustain consistent performance amid fluctuating conditions. Therefore, performance metrics may be proposed to include metrics for data accuracy, processing speed, and the ability to handle large-scale data from diverse sources. In particular, computational efficiency, which is crucial for real-time applications, is required to be evaluated in terms of data processing speed and decision-making latency. In scenarios requiring real-time decision-making, ML techniques are evaluated not only for their computational efficiency but also for their effectiveness in making optimal decisions under varying grid conditions. This involves assessing the system’s ability to dynamically balance energy supply and demand while minimizing costs and maintaining grid stability.

To evaluate the scalability of the model, it is necessary to assess its generalizability and adaptability to different sizes and types of grid topologies and environments. Furthermore, the impact of ML algorithms on overall EMS performance can be measured, including improvements in energy efficiency, cost savings, and reductions in carbon dioxide emissions. Lastly, the reliability of ML models in maintaining data security and user privacy is regarded as a crucial performance metric, underscoring the importance of safeguarding sensitive information in ML-based systems.

A shift from traditional performance metrics to more comprehensive evaluation metrics is required for evaluating EMS integrated with ML methods. This approach should encompass the entire system architecture, consider inter-action-based performance, and specifically address the challenges posed by ML integration. Consequently, the successful implementation and operation of optimized EMS can be achieved by addressing these challenging issues, with comprehensive performance metrics being obtained from both energy systems and ML viewpoints in increasingly complex and expanding sizes and types of grid topologies and environments. In

Table 6. Primary challenging issues in ML-based EMS in terms of technical issues.

Technical Index	Challenging Issue
Performance Measure	- Balancing multi-objective optimization in ML models - Developing resilient ML-based EMS to deal with uncertainties - Establishing and agreeing on standardized performance metrics - Enhanced decision-making in integrated ML-based EMS - Combining ML with domain expertise to improve decision-making and model accuracy in complex energy scenarios - Regulatory and operational constraints in complex environment
Reliability and Robustness	- Developing self-sufficient ML algorithms capable of continuous learning and adaptation in EMS - ML-enhanced EMS with fault tolerance to ensure operational continuity - Developing ML-based predictive maintenance system - Sensitivities of operational reserves for sustainable ML-based EMS - Designing ML-based anomaly robustness model with minimum forecasting error and stable voltage and frequency fluctuation - Validation under extreme operational conditions on ML-based EMS with low-performance degradation in terms of operational costs, delay time, and battery life
Integration and Interoperability	- Creating ML-optimized protocols for efficient data sharing between different entities and systems within the EMS - Developing universal application programming interfaces (APIs) that facilitate easier integration of ML models across various platforms - Cross-domain interoperability within ML-based EMS - Ensuring compatibility of ML models across diverse systems and components within the EMS framework - ML-based enhanced communication and coordination between various component systems in the EMS - ML-based EMS integration with existing infrastructure and systems

, primary challenging issues in ML-based EMS are provided in terms of the technical index.

4.4. Challenges of Case Studies in ML-Based EMS

Integrating various components into EMS has been regarded as a crucial technical challenge for ensuring the efficient operation of real-world ML-based EMS applications. Furthermore, research into model generalization and scalability is essential for guaranteeing the effectiveness of ML-based EMS models, not only in controlled environments but also across diverse real-world settings. An extreme ML-based EMS was developed with a focus on maximizing distributed energy resource usage and integrating storage [122]. A case study was conducted to validate the performance of EMS, which involved measuring the actual load and demand values in Laguna, Sorsogon, and Quezon City. The results from the two-day experiment, aimed at evaluating the actual implementation, indicated increased self-consumption of solar PV generation and a reduction in grid dependence. In [70], a case study is presented in which data-driven ML models are applied to optimize EMS in a real-world household involving house appliances. In this case study, the validation process has been executed in a real-world environment, demonstrating the superiority of ML in terms of energy efficiency and management effectiveness. Consequently, this approach demonstrates the applicability of ML methods in real-world scenarios, emphasizing their potential for optimizing energy management in residential settings.

For further studies that can significantly enhance the efficiency, reliability, scalability, stability, and adaptability of EMS, key areas for future research may include adaptive control topology optimization, uncertainty, imbalance management, advanced GASHS for enhanced reliability, large-scale integration, integration of EV, edge computing, ML advancement, and multi-agent-based system dynamics. In adaptive control topology optimization, investigating appropriate control topologies for various grid types is required to improve efficiency and adaptability. With the opening of the electricity market and the increased decision-making power of prosumers, it is essential to develop strategies that manage the variability, uncertainty, and imbalances caused by DER, aiming to stabilize the grid. Research on large-scale EMS, exploring edge computing and advanced ML algorithms, is required to enhance grid resilience and scalability, as well as to analyze the impact of their integration with existing infrastructure. Elaborate constraints are needed to tackle scalability challenges that arise from the integration of EV, ESS, and RES, with a particular focus on balancing supply and demand to maintain stability.

The emergence of EVs as a response to escalating air pollution and their benefits in terms of operational costs and support for utility power grids have been underscored. EVs can function in two modes: grid-to-vehicle and vehicle-to-grid (V2G), offering energy storage and aiding the grid in various capacities, such as managing peak loads and mitigating RES intermittency. In V2G, several challenging issues have been studied, including impacts on distribution networks, battery degradation, investment costs, energy losses, and effects on distributed infrastructure. Consequently, the necessity for advanced charging-discharging control strategies and large-scale EMS integration is emphasized to tackle these challenges. Furthermore, there is a need to study the dynamics of single- and multi-agent systems in EMS frameworks, particularly focusing on cooperative and competitive interactions, to optimize energy management performance and power flow.

5. Conclusions

In this paper, a comprehensive overview of the integration of ML technologies within EMS frameworks is presented. The integration of ML into EMS has been reviewed, with an emphasis on the enhancement of management performance and optimization. A pivotal aspect of this study is the in-depth investigation of ML methods according to EMS frameworks, demonstrating the suitability and effectiveness of various ML approaches. Furthermore, the promising ML candidates for each EMS framework are tabulated. The characteristics of various ML approaches within key component systems of EMS, including EMIS, GASHS, ESS, ETRMS, and DSMS, have been described. These components are identified as primary for achieving optimal power flow and efficient energy management. In particular, the transformative potential of ML in revolutionizing forecasting, decision-making processes, and control mechanisms in EMS has been highlighted.

In the implementation of ML-based EMS, several operational constraints and challenges need to be considered. To ensure stability, reliability, and sustainability while integrating advanced ML technologies into EMS, it is essential to effectively address these operational constraints and comprehensive challenges. In this paper, the major operational constraints and challenging issues in ML-based EMS are described. These include data management, security concerns, the development of robust and scalable ML models and ensuring interoperability, among others. To clarify these various operational constraints and challenges, the contents have been tabulated.

Future studies should be directed towards the development of advanced ML technologies specifically tailored for EMS applications. Ensuring their resilience and adaptability in practical scenarios is identified as crucial. Furthermore, collaborations among various stakeholders, including policymakers, technologists, and energy sector entities, are deemed necessary to create an environment conducive to the establishment of sustainable and efficient EMS. Therefore, this paper presents insights and frameworks that not only enhance the current understanding of ML applications in EMS but also provide suggestions for future research in this promising field.