Next Article in Journal
The Trade of Woody Biomass in the Context of Environmental Economics in Poland
Previous Article in Journal
Addressing the Renewable Energy Challenges through the Lens of Monetary Policy—Insights from the Literature
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

ADPA Optimization for Real-Time Energy Management Using Deep Learning

1
Energy Development Research Institute, China Southern Power Grid, Guangzhou 510530, China
2
Central Southern China Electric Power Design Institute Co., Ltd. of China Power Engineering Consulting Group, Wuhan 430071, China
*
Author to whom correspondence should be addressed.
Energies 2024, 17(19), 4821; https://doi.org/10.3390/en17194821
Submission received: 5 September 2024 / Revised: 23 September 2024 / Accepted: 24 September 2024 / Published: 26 September 2024

Abstract

:
The current generation of renewable energy remains insufficient to meet the demands of users within the network, leading to the necessity of curtailing flexible loads and underscoring the urgent need for optimized microgrid energy management. In this study, the deep learning-based Adaptive Dynamic Programming Algorithm (ADPA) was introduced to integrate real-time pricing into the optimization of demand-side energy management for microgrids. This approach not only achieved a dynamic balance between supply and demand, along with peak shaving and valley filling, but it also enhanced the rationality of energy management strategies, thereby ensuring stable microgrid operation. Simulations of the Real-Time Electricity Price (REP) management model under demand-side response conditions validated the effectiveness and feasibility of this approach in microgrid energy management. Based on the deep neural network model, optimization of the objective function was achieved with merely 54 epochs, suggesting a highly efficient computational process. Furthermore, the integration of microgrid energy management with the REP conformed to the distributed multi-source power supply microgrid energy management and scheduling and improved the efficiency of clean energy utilization significantly, supporting the implementation of national policies aimed at the development of a sustainable power grid.

1. Introduction

Microgrids, as fundamental units within the Energy Internet, facilitate the integration of surplus clean energy into the grid by connecting to the distribution network, allowing for the sale of excess energy. During periods of insufficient supply, microgrids also enable the purchase of electricity from the distribution network. This makes them a crucial component for utilizing distributed energy to supply power to end users [1,2]. Due to the substantial presence of distributed power sources within microgrids, their energy management strategies diverge significantly from those of traditional distribution networks. Furthermore, evolving trends in power grid energy management and dispatching, characterized by a shift towards a hierarchical distribution and market-oriented development from the distribution network to the microgrid, underscore the urgent need to optimize energy management [3]. In the quest to address the challenges stemming from the intermittent nature of the energy sources and variable loads within microgrids, certain researchers [4,5] have endeavored to develop online algorithms aimed at optimizing management and mitigating the long-term costs associated with the renewable energy supply. Nonetheless, this approach appears to overlook the operational constraints inherent in distributed power systems that are integral to microgrids, thereby potentially limiting its practical applicability in real-world control scenarios. Currently, there is a dearth of studies focusing on guiding flexible-load energy consumption in power grids via REP mechanisms from the perspective of microgrid operational optimization. By integrating a demand-side response price mechanism, which incorporates distributed power generation with real-time energy management control, the energy management objective function of microgrids can be effectively optimized. This approach not only enhances the overall optimization of microgrid operations, reduces operating costs, and increases the utilization of renewable energy, but it also significantly mitigates carbon emissions and environmental pollution associated with extensive fossil fuel use [6,7]. Additionally, this aligns with the overarching strategic objectives of energy management and utilization within the power system, positioning it as a focal point and primary direction of research in power system energy management. Consequently, the investigation of real-time microgrid energy management and control in the context of the Energy Internet is of substantial significance.
The foundational concept of deep learning lies in the hierarchical processing of data through the stacking of multiple layers, wherein the output of each layer serves as the input for the subsequent layer, allowing information to be incrementally transformed until the desired output is achieved [8,9]. The ADPA plays a crucial role in approximating the cost function and control strategy within dynamic programming frameworks, thereby addressing optimal control problems in nonlinear systems through function approximation techniques [10]. Liu et al. [11] elucidate the synergistic potential of integrating deep learning algorithms with ADPA frameworks. This convergence is posited to culminate in the construction of learning mechanisms that exhibit enhanced cognitive acuity and augmented precision. Building on these principles, this study introduces a real-time energy management strategy for microgrids, integrating deep learning with the ADPA. This approach effectively mitigates the limitations of traditional optimization algorithms regarding real-time performance and regulation, offering a simplified algorithmic structure, enhanced closed-loop control precision, and superior online real-time capabilities [12]. Consequently, it is well suited to the real-time energy management requirements of microgrids. Moreover, the integration of microgrid energy management with the REP not only enhances the efficiency of clean energy utilization and encourages utilities to increase their investments in renewable energy generation but also aligns with national environmental policies aimed at reducing coal-based power generation.
In response to the challenges of flexible-load management arising from the insufficient power generation within contemporary microgrid systems, this work examined an REP mechanism tailored to address the demand-side responses within the microgrid energy management framework. The online microgrid energy management strategy that leverages the REP was introduced to incorporate deep learning and the ADPA. This strategy not only enhanced self-learning capabilities but also met the stringent real-time operational requirements. The objectives of this work were threefold: (1) to optimize the multi-source power supply within the microgrid, thereby increasing the utilization of renewable energy; (2) to strategically adjust flexible power supplies within the microgrid, minimizing the reliance on thermal power and reducing the overall operational costs; and (3) to achieve targets for the development of a sustainable power grid.

2. Materials and Methods

2.1. Establishment of Microgrid Real-Time Energy Management Control Model

To ensure a satisfactory level of customer satisfaction in electricity consumption, the microgrid operator is required to procure a portion of the electrical energy from the distribution network to meet the demands of certain flexible loads within the system. This procurement of electricity from the distribution network results in associated costs for the microgrid operator [13]:
C m ( p m t ) = ρ p m t
where ρ denotes the benchmark electricity price set by the distribution network for the sale of electric energy to the microgrid operator. Meanwhile, pm(t) represents the quantity of electricity procured by the microgrid operator from the distribution network to satisfy a portion of the flexible-load demand when the distributed power supply is insufficient.
To optimize its operational costs, a microgrid operator may resort to curtailing flexible loads. However, the extent of this curtailment is inherently constrained, necessitating the development of a flexible-load curtailment limit ratio function [14]:
μ ( t ) = 1 T p l max ( t ) E ( p l ( t ) ) p l max ( t ) p l min ( t ) σ
where plmax(t) denotes the peak power demand across all users within the microgrid, while plmin(t) represents the demand attributable to non-deferrable loads. The variable T serves as a temporal constraint, which, under real-time data acquisition, is consistently set to 1. The parameter plmax(t) − plmin(t) signifies the aggregation potential for flexible-load reduction, with plmax(t) − E(pl(t)) indicating the specific portion of flexible load that can be curtailed. E(pl(t) is the expectation of the load power supply, and σ is introduced as a positive scalar representing the ratio of flexible-load curtailment.
Simultaneously, the optimization of microgrid operational costs must account for the associated cost factors and constraints linked to distributed energy resources. Accordingly, a minimization objective function can be formulated to optimize the generation and operational costs within a microgrid framework [15,16]:
min μ ( t ) ( ϕ w w G w C w ( p w ( t ) ) + ϕ g g G g C g ( p g ( t ) ) + ϕ s s S C s ( p s ( t ) ) + ϕ l l L C l ( p l ( t ) ) + ϕ m C m ( p m ( t ) )
To ensure that the solution to the objective function aligns more closely with the actual system operations, it is essential that the optimization process adheres to the following constraints:
s . t .   0 p w ( t ) p w max , p w ( t ) p w ( t 1 ) p w max 0 p g ( t ) p g max , p g ( t ) p g ( t 1 ) p g max p d max p s ( t ) p c max , E s min E s ( t ) E s max p l min p l ( t ) p l max ( t )

2.2. Establishment of Microgrid REP Energy Management Control Model

2.2.1. Relationship between REP and Distribution Network Electricity Price

Building on the previously developed real-time energy management control model for microgrids, the subsequent phase involved formulating an REP energy management control model for these systems. This model elucidates the interrelationship between the REP within the microgrid and the electricity price in the distribution network, as detailed in reference [17]:
ρ r ( t ) = e K p m ( t ) p l max ( t ) p l min ( t ) ρ l
where ρr(t) represents the REP levied by the microgrid operator to satisfy the consumption needs of flexible-load users after procuring electricity from the distribution network in instances where the microgrid’s distributed power sources are inadequate. pm(t) denotes the thermal power acquired by the operator from the distribution network when the distributed power generation within the microgrid falls short. K is a constant proportional coefficient. plmax(t) indicates the total-load demand within the microgrid, while plmin(t) represents the rigid-load demand. plmax(t) − plmin(t) signifies the demand for flexible-load power, and ρl is the sale price of power from the distribution network.
Here, the following REP constraints are used to fully reflect this principle [18]:
0 ρ r ( t ) ξ ρ l
where ξ is the rational factor. For the selection of the purchased power (pm(t)) under the condition of making full use of the distributed energy in the microgrid, it satisfies two correlation conditions [19]:
(1)
When plmin(t)ps(t) + pw(t) + pg(t), the distributed power generation in the microgrid is less, so that the rigid-load power demand of the microgrid cannot be met. At this time, the microgrid operator needs to charge the REP for the electricity purchased to supply the flexible-load users after meeting the rigid-load demand. Therefore, the value range of the pm(t) is as follows:
p m 0 , p l max ( t ) p l min ( t )
(2)
When plmin(t)ps(t) + pw(t) + pg(t), the distributed power generation in the microgrid can not only meet the rigid-load demand in the network but can also provide part of the remaining power to the flexible load in the network; however, it cannot fully meet the needs of the flexible load in the network. At this time, for the part that does not meet the demand for the flexible load, the microgrid operator must purchase electricity from the distribution network and charge the REP for the electricity supplied to the flexible-load user. Therefore, the value range of the pm(t) of the purchased electricity at this time is as follows:
p m 0 , p l max ( t ) p w ( t ) + p g ( t ) + p s ( t )

2.2.2. Cost and Profit of REP

The primary objective of microgrid operations is to minimize the overall long-term cost associated with power generation. This encompasses optimizing both the cost function and the profit function across the various components involved.
(1)
Distributed generation cost and energy storage conversion cost
This part of the cost is composed of the cost of hydroelectric power generation, the cost of gas power generation, and the charge and discharge cost of wind power and photovoltaic power generation in the energy storage battery pack [20,21]:
F l ( p w ( t ) ) , p g ( t ) , p s ( t ) ) = w G w C w p w ( t ) + g G g C g p g ( t ) + s S C s p s ( t )
Among them, Cw(pw(t)) is the cost generated by hydroelectric power generation, Cg(pg(t)) is the cost generated by gas power generation, and Cs(ps(t)) is the cost generated by storage battery pack charging and discharging. In order for microgrid operators to minimize the cost of the power supply, this part of the cost should be as small as possible [20,21]:
min F l p w t , p g t , p s t
The constraint conditions are as follows:
s . t .   0 p w ( t ) p w max , p w ( t ) p w ( t 1 ) p w max 0 p g ( t ) p g max , p g ( t ) p g ( t 1 ) p g max p d max p s ( t ) p c max , E s min E s ( t ) E s max
(2)
The cost of purchasing electricity from the distribution network
This part of the cost is the cost generated by the microgrid operator to purchase electricity from the distribution network in order to meet the needs of the power users in the network [22]:
F 2 ( p m t ) = ρ l p m t
where pm(t) is the purchase of electricity, and ρl is the distribution network sales price. For microgrid operators to achieve the minimum cost of power generation, naturally, the less electricity purchased the better:
min F 2 ( p m ( t ) )
The constraint conditions are as follows:
s . t .   p m ( t ) 0 , p l max ( t ) p l min ( t ) or   p m ( t ) 0 , p l max ( t ) p w ( t ) + p g ( t ) + p s ( t )
(3)
REP profit
Given that the REP exceeds the purchase price of electricity acquired by the microgrid operator from the distribution network, the operator stands to realize a profit through the differential between these rates [23,24]:
F 3 p m t = ρ r p m t ρ l p m t
where pm(t) is the purchase of electricity, and pr(t) is the REP. Because the REP of the microgrid is higher than the purchase of electricity from the distribution network, the higher the REP price of the microgrid, the larger the profit of the microgrid operator, and the more electricity purchased, the higher the profit:
max F 3 p m t
The constraint conditions are as follows:
0 < ρ r t ξ ρ l
To minimize the overall power generation cost of the microgrid, its objective function can be written as follows:
min F 1 p w t , p g t , p s t , F 2 p m t , F 3 p m t
In order to facilitate calculation and solution, the above multi-objective function is converted into a single-objective function by the weighted method for solving, and the minimum objective optimization function of the power generation cost of the microgrid operator can be obtained as follows:
min ϕ w w G w C w ( p w ( t ) ) + ϕ g g G g C g ( p g ( t ) ) + ϕ s s S C s ( p s ( t ) ) + ϕ l l L C l ( p l ( t ) ) + φ m ρ l ρ r ( t ) p m ( t )
At the same time, the acquisition of the objective function also needs to meet the following constraints:
s . t .   0 p w ( t ) p w max , p w ( t ) p w ( t 1 ) p w max 0 p g ( t ) p g max , p g ( t ) p g ( t 1 ) p g max p d max p s ( t ) p c max , E s min E s ( t ) E s max 0 < ρ r t ξ . ρ l p m ( t ) 0 , p l max ( t ) p l min ( t ) or   p m ( t ) 0 , p l max ( t ) p w ( t ) + p g ( t ) + p s ( t )

3. Results and Discussion

3.1. Deep Learning ADPA-REP Energy Management Strategy

The energy management control strategy necessitates that the system elucidates the coupling dynamics between the flexible-load users’ power demands and the REP within the microgrid, as derived from the instantaneous generation data of distributed energy sources and the real-time load demands of the microgrid. This insight enables the microgrid operator to optimize the scheduling and control of the distributed power generation, coal-fired power procurement, and load demands, effectively aligning these with the REP and power purchases [25,26]. To enhance both the real-time responsiveness and stability of the energy management strategy of the microgrid under fluctuating electricity prices, based on the architecture of the heuristic dynamic programming structure, an integrative approach combining the ADPA with deep learning is employed. This approach not only adeptly addresses the challenges associated with traditional optimization algorithms, which often fail to adhere to real-time constraints and exhibit suboptimal regulatory capabilities, but it also boasts a streamlined algorithmic framework and enhanced precision in closed-loop control. This methodological framework is illustrated in Figure 1.
In this framework (Figure 1a), the input (X(k)) to the action network comprises the state variables of the controlled system, including the real-time measurements of power (pw(t), pg(t), and ps(t)), the maximum power demand, and the rigid load imposed by the distributed energy sources within the microgrid. The output (u(k)) represents the control strategy of the system, specifically, the REP needed by the microgrid operator to minimize the objective function associated with the operational costs of power generation and supply, alongside the quantity of electricity procured from the distribution network. The REP energy management control strategy aims to modulate the power supply within the microgrid by leveraging REP signals. The multi-hidden deep learning neural network is used in the action network and the critic network. This approach facilitates the achievement of supply–demand equilibrium through price mechanisms [27]. Additionally, when the REP exceeds the psychological price threshold of certain flexible-load consumers, these consumers are incentivized to adopt strategies that curtail their electricity consumption. Consequently, the REP mechanism not only ensures supply–demand balance but also contributes to peak shaving and valley filling for flexible loads.
To minimize the operational costs of the system, it is imperative to optimize two variables: the REP (ρr(t)) and the quantity of electricity (pm(t)) procured by the microgrid from the distribution network. The action network’s outputs (u1(t) = pm(t) and u2(t) = ρr(t)) must adhere to specified constraints and optimization criteria. Therefore, the outputs pm(t) and ρr(t) should train the system, evaluate the network input as the state and control strategy of the controlled object, and the output as the cost function, and the utility function can be defined according to the control target. To enhance the system’s approximation to the objective function and achieve superior computational precision, stringent control over the output discrepancies of both the action network and the critic network is imperative. In this investigation, we employed a deep learning architecture featuring multiple hidden layers and dual-output for each input (Figure 1b), which was leveraged to manage the action network and critic network. This approach was designed to minimize their error values to the threshold of acceptability, adhering to the principle of minimizing control system errors.
During the implementation of the deep learning-based ADPA, the critic network’s output comprises both the optimal objective function value and the corresponding control strategy, as determined by the action network. Consequently, the formulation of the critic network’s objective function and the iterative training of the deep neural network are of paramount importance. Algorithm 1 illustrates the construction of the objective function and the parameter-setting procedures within the critic network, as well as the updated main program for the deep learning ADPA training iterations.
Algorithm 1. Parameter-setting program
INPUT: None
OUTPUT: Fitness value (fx)
BEGIN
  DEFINE global variables
         glo.aw=5; glo.bw=12; glo.kw=0; glo.ag=0.2;
         glo.ap=2.5; glo.bp=1; glo.kp=0;
         glo.as=1; glo.ks=0; glo.kexi=3.4;
         glo.fw=1; glo.fg=l; glo.fs=1:glo.fl=1; glo.fp=1;
         glo.k=2.5;
         glo.rl=0.45;
  CALCULATE plt
         plt=glo.pmax-glo.pwt-glo.pgt-glo.pst-glo.pxt;
  INITIALIZE rowRT
         rowRT=1;
  CALL fitness function to calculate fitness value
         fx=fitness2019_07_22_23 15_01(glo.pwt,glo.pgt,glo.pst,plt,rowRT)
  DEFINE fitness function
         fumction fx=fitness2019_07_22_23_15_01(pwt,pgt,pst,plt,rowRT)
                BEGIN
       DEFINE local variables
         cw=glo.aw.*pwt*pwt+glo.bw.*pwt+glo.kw;
         cs=glo.as.*pst*pst+glo.ks;
         cg=glo.ag.*pgt;
         cp=((glo.ap*plt)^2+glo.bp*plt+glo.kp)^(1/2);
         rrt=exp((glo.k*plt)/(glo.pmax-glo.pmin))*glo.rl;
         cl=plt*(glo.rl-rrt);
       CALCULATE fitness value
         fx=glo.fw,*cw+glo.fg.*cg+glo.fs.*cs+glo.fl.*cl+glo.fp*cp;
       RETURN fx
      End
End
In the iterative training process of the proposed algorithm, upon each completion of the action network training phase, the network generates the REP (ρr(t)) and the associated purchased electricity (pm(t)). These outputs, along with the control functions and state variables (such as the distributed power generation), are subsequently fed into the critic network to determine the objective function. The training terminates if the control and state variables satisfy the constraints and yield an optimal objective function. If the criteria are not met, the control and critic networks are iteratively updated until the optimal objective function value is achieved, thereby deriving the real-time energy optimization management control strategy. Jesus et al. [28] conducted a comparative analysis of the proficiency of deep learning methodologies against that of conventional algorithms for the forecasting of spot electricity prices, indicating that deep learning, a subset of machine learning, typically exhibits a superior predictive accuracy in comparison to statistical models. Furthermore, deep learning models have demonstrated superior performances over traditional machine learning approaches, yielding statistically significant outcomes. This substantiates the robustness and precision of the models developed in the present investigation.
In essence, the dynamic energy management control paradigm for microgrids is adept at harnessing real-time correlational insights between demand and supply dynamics. This strategy adeptly optimizes the deployment of microgrid power sources in response to the fluctuating nature of distributed generation, thereby facilitating the equilibrium of supply and demand, as well as effectuating peak reduction and valley filling. Consequently, this approach ensures the operational safety and stability of the microgrid. Furthermore, the integration of the ADPA with real-time pricing mechanisms enhances the efficacy of demand-side response initiatives, encouraging consumers to modulate their electricity consumption by adopting strategies that align with periods of peak and off-peak demand.

3.2. Simulation Case Analysis

In accordance with the power grid company’s standards, data for the microgrid distributed power supply within a regional power grid in Guangxi, a southwestern province in China, were collected at 15 min intervals over a period totaling 1085 min. The proposed management control strategy was evaluated through simulation. The microgrid integrates hydropower, gas, wind, and photovoltaic sources into the distributed power supply of an energy storage battery pack at specified intervals, as depicted in Figure 2. The horizontal axis represents the time of data acquisition (in minutes), while the vertical axis denotes the distributed power generation (in MW). Recent reports by the International Renewable Energy Agency (IRENA) indicate a substantial reduction in the cost of renewable electricity generation. In this study, the parameters for the distributed power cost model were defined as follows: hydropower cost coefficients: αw = 4~5, βw = 10~12, and Kw = 0; and gas power cost coefficient: αg = 0.1~0.25. The rational factor for the REP is ξ = 3.5, with a proportion coefficient of k = 2.5. The storage battery cost function coefficient is αs = 1, Ks = 0 [19].
Figure 3a illustrates the convergence behavior between the target minimum cost of microgrid power generation and the corresponding output derived from the deep learning ADPA throughout the training iterations. Figure 3b depicts the regression trajectory of the system over the entire training and optimization duration, demonstrating an ideal alignment between the target and output values. Although an increased iteration count can enhance a model’s training data performance, an excessive number of iterations may precipitate overfitting. This phenomenon is characterized by a superior model performance on the training set but suboptimal generalization to new, unseen data. Consequently, it is imperative to strike an optimal balance in the iteration count to mitigate the risk of overfitting. Figure 3c presents the iteration count required to achieve the minimum mean-square error between the target microgrid power generation cost and the deep learning ADPA output. Furthermore, Figure 3 encompasses the complete iterative optimization process of the system’s target values and output errors within the optimization algorithm framework, revealing that the deep neural network necessitates only 54 epochs to achieve objective function optimization. When optimal computational precision is attainable, the computational load of the model is considerably reduced in comparison to those of traditional machine learning methodologies [28]. An epoch represents a full pass of the dataset through the neural network layers—input, hidden, and output—followed by a return through the same layers, so the system checks the results six times. The entire iterative optimization process is completed in 14.6 s, with a mean-square error of 4.1321 × 10−23, underscoring the efficiency of the proposed deep learning ADPA. In comparison to the 15 min sampling interval, this strategy satisfies real-time energy management control requirements effectively.
When implementing the proposed deep learning ADPA within the framework depicted in Figure 1, it is possible to derive both the optimized REP and the corresponding electricity purchases. Additionally, this approach enables the formulation of an effective REP energy management control strategy. As illustrated in Figure 4, the results obtained from this control strategy, post-system operation, display the relationship between the purchased electricity and REP. The data reveal that, under conditions of limited microgrid power supply, the REP mechanism facilitates the regulation of both the supply- and demand-side power loads [29,30]. Furthermore, during periods of insufficient distributed power supply, flexible-load users face elevated energy purchase costs compared to conventional distribution network prices [21], thereby supporting the rational regulation and control of microgrid energy management.
An REP-based microgrid energy optimization management control strategy was developed and assessed through system operations. Figure 5a,b illustrate the dynamics of this strategy, particularly highlighting scenarios where the distributed power within the microgrid fails to meet the aggregate demand of all users. The figures provide insights into the relationships among the REP, power supply, maximum demand, and rigid loads. Additionally, they show how the total distributed power generation, REP, and total microgrid demand interrelate.
The implementation of an REP-based energy management control strategy demonstrates its efficacy in balancing the power supply and demand within the microgrid by leveraging economic price mechanisms. This strategy facilitates the adjustment of flexible-load users’ electricity consumption in response to the REP. When there is a significant shortfall in the distributed energy relative to the total demand, the optimized REP ensures that the supply–demand balance is adjusted efficiently. Although this approach does not restrict electricity consumption during periods of imbalance, it provides a mechanism for microgrid operators to procure additional energy from the distribution network if necessary, adjusting the REP within rational limits to manage the demand effectively [31]. This method underscores the role of price leveraging in achieving equilibrium in microgrid energy management.

4. Conclusions

In this work, the REP mechanism was designed to enhance the demand-side response and flexible power supply management within the microgrid energy management framework. An online REP-based control strategy for microgrid energy management was proposed, which not only upheld real-time responsiveness and stability but also promoted a more rational energy management approach. This strategy facilitated an effective interactive mechanism between microgrid operators and power users, thereby preventing unilateral reductions in flexible power loads by operators aimed at maintaining long-term optimal generation costs. Simulation results demonstrated that adjusting the flexible-load energy consumption behaviors of users through the REP could mitigate the issue of curtailed flexible loads due to insufficient renewable energy generation. Additionally, it reduced the need for extensive thermal power purchases from the distribution network to meet flexible-load demands, thereby controlling operational costs. Furthermore, the algorithm is notably efficient, with the accomplishment of the iterative optimization procedure within a mere 13 s. This expedited computation, when juxtaposed with the 15 min sampling interval, signifies that the strategy is well suited to the demands of real-time energy management control scheduling. Overall, this strategy supports microgrid operators in refining their grid operation and management processes, offering a solid foundation for optimal design and scheduling decisions. This approach optimally harnesses the generative capacity of distributed power sources, significantly enhancing the efficiency of clean energy utilization and facilitating the achievement of the power system’s objectives in energy conservation and emission reduction.

Author Contributions

Methodology, Y.H.; software, C.L.; formal analysis, Z.W.; investigation, Y.H.; resources, C.L.; data curation, Z.W.; writing—original draft, Z.W.; writing—review and editing, Y.H., L.W. and C.L.; project administration, L.W. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors on request.

Conflicts of Interest

Authors Zhengdong Wan, Yan Huang and Liangzheng Wu were employed by the Energy Development Research Institute, China Southern Power Grid. Author Chengwei Liu was employed by the Central Southern China Electric Power Design Institute Co., Ltd. of China Power Engineering Consulting Group.

References

  1. Katiraei, F.; Iravani, R.; Hatziargyriou, N.; Dimeas, A. Micro-grids management. IEEE Power Energy Mag. 2008, 6, 54–65. [Google Scholar] [CrossRef]
  2. Santhosh Krishna, B.V.; Pauline, S.; Sivakumar, S.; Palagan, C.A.; Talasila, V.; Krishna, M.M.S. Enhanced efficiency in smart grid energy systems through advanced AI-based thermal modeling. Therm. Sci. Eng. Prog. 2024, 53, 102765. [Google Scholar] [CrossRef]
  3. de Lima, T.D.; Lezama, F.; Soares, J.; Franco, J.F.; Vale, Z. Modern distribution system expansion planning considering new market designs: Review and future directions. Renew. Sustain. Energy Rev. 2024, 202, 114709. [Google Scholar] [CrossRef]
  4. Elsied, M.; Oukaour, A.; Gualous, H.; Brutto, O.A.L. Optimal economic and environment operation of micro-grid power systems. Energy Convers. Manag. 2016, 122, 182–194. [Google Scholar] [CrossRef]
  5. Huang, T.; Liu, D.-E. A self-learning scheme for residential energy system control and management. Neural Comput. Appl. 2013, 2, 259–269. [Google Scholar] [CrossRef]
  6. Wang, J.; Gao, Y.; Li, R. Reinforcement learning based bilevel real-time pricing strategy for a smart grid with distributed energy resources. Appl. Soft Comput. 2024, 155, 111474. [Google Scholar] [CrossRef]
  7. Li, B.; Roche, R.; Paire, D.; Miraoui, A. A price decision approach for multiple multi-energy-supply microgrids considering demand response. Energy 2019, 167, 117–135. [Google Scholar] [CrossRef]
  8. Ali, R.; Cha, Y.-J. Subsurface damage detection of a steel bridge using deep learning and uncooled micro-bolometer. Constr. Build. Mater. 2019, 226, 376–387. [Google Scholar] [CrossRef]
  9. Li, D.; Channa, I.A.; Chen, X.; Dou, L.; Khokhar, S.; Ab Azar, N. A new deep learning method for classification of power quality disturbances using DWT-MRA in utility smart grid. Comput. Electr. Eng. 2024, 117, 109290. [Google Scholar] [CrossRef]
  10. Zhang, J.; Zhang, H.; Liu, Z.; Wang, Y. Model-free optimal controller design for continuous-time nonlinear systems by adaptive dynamic programming based on a precompensator. ISA Trans. 2015, 57, 63–70. [Google Scholar] [CrossRef]
  11. Liu, D.; Ha, M.; Xue, S. State of the Art of Adaptive Dynamic Programming and Reinforcement Learning. CAAI Artif. Intell. Res. 2022, 1, 93–110. [Google Scholar] [CrossRef]
  12. Huang, Y.; Wang, Y.; Liu, N. A two-stage energy management for heat-electricity integrated energy system considering dynamic pricing of Stackelberg game and operation strategy optimization. Energy 2022, 244, 122576. [Google Scholar] [CrossRef]
  13. Wu, N.; Wang, H. Deep learning adaptive dynamic programming for real time energy management and control strategy of micro-grid. J. Clean. Prod. 2018, 204, 1169–1177. [Google Scholar] [CrossRef]
  14. Ye, Y.; Yuan, Q.; Tang, Y.; Strbac, G. Decentralized Coordination Parameters Optimization in Microgrids Mitigating Demand Response Synchronization Effect of Flexible Loads. Zhongguo Dianji Gongcheng Xuebao/Proc. Chin. Soc. Electr. Eng. 2022, 42, 1748–1759. [Google Scholar]
  15. Turdybek, B.; Tostado-Véliz, M.; Mansouri, S.A.; Jordehi, A.R.; Jurado, F. A local electricity market mechanism for flexibility provision in industrial parks involving Heterogenous flexible loads. Appl. Energy 2024, 359, 122748. [Google Scholar] [CrossRef]
  16. Sun, S.; Dong, M.; Liang, B. Joint supply, demand, and energy storage management towards microgrid cost minimization. In Proceedings of the 2014 IEEE International Conference on Smart Grid Communications (SmartGridComm), Venice, Italy, 3–6 November 2014. [Google Scholar]
  17. Ullah, Z.; Wang, S.; Wu, G.; Xiao, M.; Lai, J.; Elkadeem, M.R. Advanced energy management strategy for microgrid using real-time monitoring interface. J. Energy Storage 2022, 52, 104814. [Google Scholar] [CrossRef]
  18. Cui, G.; Jia, Q.S.; Guan, X. Energy Management of Networked Microgrids With Real-Time Pricing by Reinforcement Learning. IEEE Trans. Smart Grid 2024, 15, 570–580. [Google Scholar] [CrossRef]
  19. Wu, N.; Wang, H.; Yin, L.; Yuan, X.; Leng, X. Application Conditions of Bounded Rationality and a Microgrid Energy Management Control Strategy Combining Real-Time Power Price and Demand-Side Response. IEEE Access 2020, 8, 227327–227339. [Google Scholar] [CrossRef]
  20. Bishwajit, D.; Gulshan, S.; Bokoro, P.N. Economic Management of Microgrid Using Flexible Non-Linear Load Models Based on Price-Based Demand Response Strategies. Results Eng. 2024, 102993. [Google Scholar] [CrossRef]
  21. He, Y.; Zhang, J. Real-time electricity pricing mechanism in China based on system dynamics. Energy Convers. Manag. 2015, 94, 394–405. [Google Scholar] [CrossRef]
  22. Wang, C.; Li, X. Optimization scheduling of microgrid comprehensive demand response load considering user satisfaction. Sci. Rep. 2024, 14, 16034. [Google Scholar] [CrossRef] [PubMed]
  23. Wang, D.; Fan, R.; Yang, P.; Du, K.; Xu, X.; Chen, R. Research on floating real-time pricing strategy for microgrid operator in local energy market considering shared energy storage leasing. Appl. Energy 2024, 368, 123412. [Google Scholar] [CrossRef]
  24. Wesseh, P.K.; Lin, B. A time-of-use pricing model of the electricity market considering system flexibility. Energy Rep. 2022, 8, 1457–1470. [Google Scholar] [CrossRef]
  25. Fagundes, T.A.; Fuzato, G.H.F.; Magossi, R.F.Q.; Flores, M.A.B.; Vasquez, J.C.; Guerrero, J.M.; Machado, R.Q. Economic Operation Optimization Under Real-Time Pricing for an Energy Management System in a Redundancy-Based Microgrid. IEEE Trans. Ind. Electron. 2024, 71, 8872–8882. [Google Scholar] [CrossRef]
  26. Li, B.; Zhao, R.; Lu, J.; Xin, K.; Huang, J.; Lin, G.; Chen, J.; Pang, X. Energy management method for microgrids based on improved Stackelberg game real-time pricing model. Energy Rep. 2023, 9, 1247–1257. [Google Scholar] [CrossRef]
  27. Xiong, S.; Liu, D.; Chen, Y.; Zhang, Y.; Cai, X. A deep reinforcement learning approach based energy management strategy for home energy system considering the time-of-use price and real-time control of energy storage system. Energy Rep. 2024, 11, 3501–3508. [Google Scholar] [CrossRef]
  28. Jesus, L.; Fjo, D.R.; Bart, D.S. Forecasting spot electricity prices: Deep learning approaches and empirical comparison of traditional algorithms. Appl. Energy 2018, 221, 386–405. [Google Scholar]
  29. Guo, Z.; Xu, W.; Yan, Y.; Sun, M. How to realize the power demand side actively matching the supply side?—A virtual real-time electricity prices optimization model based on credit mechanism. Appl. Energy 2023, 343, 121223. [Google Scholar] [CrossRef]
  30. Jiang, J.; Kou, Y.; Bie, Z.; Li, G. Optimal Real-Time Pricing of Electricity Based on Demand Response. Energy Procedia 2019, 159, 304–308. [Google Scholar] [CrossRef]
  31. Singh, A.R.; Kumar, R.S.; Bajaj, M.; Khadse, C.B.; Zaitsev, I. Machine learning-based energy management and power forecasting in grid-connected microgrids with multiple distributed energy sources. Sci. Rep. 2024, 14, 19207. [Google Scholar] [CrossRef]
Figure 1. Deep learning ADPA: (a) fundamental framework; (b) multilayer neural network topology.
Figure 1. Deep learning ADPA: (a) fundamental framework; (b) multilayer neural network topology.
Energies 17 04821 g001
Figure 2. Power generation: (a) hydroelectric power generation; (b) gas power generation; (c) distributed power storage device.
Figure 2. Power generation: (a) hydroelectric power generation; (b) gas power generation; (c) distributed power storage device.
Energies 17 04821 g002
Figure 3. Objective function setting and output training process: (a) error optimization process; (b) iterative regression process; and (c) the number of iterations of the best-fit parameter under the minimum error.
Figure 3. Objective function setting and output training process: (a) error optimization process; (b) iterative regression process; and (c) the number of iterations of the best-fit parameter under the minimum error.
Energies 17 04821 g003
Figure 4. REP of microgrid and corresponding purchased electricity: (a) REP corresponding to the purchase of electricity; (b) REP corresponding to supply–demand balance.
Figure 4. REP of microgrid and corresponding purchased electricity: (a) REP corresponding to the purchase of electricity; (b) REP corresponding to supply–demand balance.
Energies 17 04821 g004
Figure 5. Relationships between REP and microgrid power supply and demand: (a) the relationship between the total supply and total demand; (b) the relationship between the power and rigid-load and maximum power demands.
Figure 5. Relationships between REP and microgrid power supply and demand: (a) the relationship between the total supply and total demand; (b) the relationship between the power and rigid-load and maximum power demands.
Energies 17 04821 g005
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Wan, Z.; Huang, Y.; Wu, L.; Liu, C. ADPA Optimization for Real-Time Energy Management Using Deep Learning. Energies 2024, 17, 4821. https://doi.org/10.3390/en17194821

AMA Style

Wan Z, Huang Y, Wu L, Liu C. ADPA Optimization for Real-Time Energy Management Using Deep Learning. Energies. 2024; 17(19):4821. https://doi.org/10.3390/en17194821

Chicago/Turabian Style

Wan, Zhengdong, Yan Huang, Liangzheng Wu, and Chengwei Liu. 2024. "ADPA Optimization for Real-Time Energy Management Using Deep Learning" Energies 17, no. 19: 4821. https://doi.org/10.3390/en17194821

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop