Optimizing Electricity Markets Through Game-Theoretical Methods: Strategic and Policy Implications for Power Purchasing and Generation Enterprises

Featured Application

This survey employs the principles of evolutionary game theory to analyze the strategic behaviors of power purchasing enterprises and power generation enterprises in electricity markets. By constructing a dynamic evolutionary game model, the research investigates how these entities interact under different market mechanisms, such as long-term contracts and spot market transactions. The study highlights the evolutionary stability of strategies adopted by market participants and explores factors influencing strategic evolution, including market regulations, supply–demand fluctuations, and pricing dynamics. The findings offer theoretical insights into the competition and cooperation between power purchasing and generation enterprises, providing valuable guidelines for optimizing market performance and policy regulation in the electricity sector.

Abstract

This review proposes a novel integration of game-theoretical methods—specifically Evolutionary Game Theory (EGT), Stackelberg games, and Bayesian games—with deep reinforcement learning (DRL) to optimize electricity markets. Our approach uniquely addresses the dynamic interactions among power purchasing and generation enterprises, highlighting both theoretical underpinnings and practical applications. We demonstrate how this integrated framework enhances market resilience, informs evidence-based policy-making, and supports renewable energy expansion. By explicitly connecting our findings to regulatory strategies and real-world market scenarios, we underscore the political implications and applicability of our results in diverse global electricity systems. By integrating EGT with advanced methodologies such as DRL, this study develops a comprehensive framework that addresses both the dynamic nature of electricity markets and the strategic adaptability of market participants. This hybrid approach allows for the simulation of complex market scenarios, capturing the nuanced decision-making processes of enterprises under varying conditions of uncertainty and competition. The review systematically evaluates the effectiveness and cost-efficiency of various control policies implemented within electricity markets, including pricing mechanisms, capacity incentives, renewable integration incentives, and regulatory measures aimed at enhancing market competition and transparency. Our analysis underscores the potential of EGT to significantly enhance market resilience, enabling electricity markets to better withstand shocks such as sudden demand fluctuations, supply disruptions, and regulatory changes. Moreover, the integration of EGT with DRL facilitates the promotion of sustainable energy integration by modeling the strategic adoption of renewable energy technologies and optimizing resource allocation. This leads to improved overall market performance, characterized by increased efficiency, reduced costs, and greater sustainability. The findings contribute to the development of robust regulatory frameworks that support competitive and efficient electricity markets in an evolving energy landscape. By leveraging the dynamic and adaptive capabilities of EGT and DRL, policymakers can design regulations that not only address current market challenges but also anticipate and adapt to future developments. This proactive approach is essential for fostering a resilient energy infrastructure capable of accommodating rapid advancements in renewable technologies and shifting consumer demands. Additionally, the review identifies key areas for future research, including the exploration of multi-agent reinforcement learning techniques and the need for empirical studies to validate the theoretical models and simulations discussed. This study provides a comprehensive roadmap for optimizing electricity markets through strategic and policy-driven interventions, bridging the gap between theoretical game-theoretic models and practical market applications.

1. Introduction

The global energy landscape is undergoing an unprecedented transformation, with the electricity market emerging as a crucial component of this evolving energy system. As the world grapples with the challenges of climate change, energy security, and sustainable development, the efficiency and stability of electricity markets have become paramount concerns [1]. Empirical data from the Wind Energy Equipment Branch of the China Agricultural Machinery Industry Association (2015) reveal that these reforms have significantly increased market participation and enhanced supply reliability, demonstrating the effectiveness of the implemented policy measures. In light of these challenges, recent policy initiatives by the National Development and Reform Commission have emphasized the necessity of enhancing market mechanisms and promoting competitive practices among market participants [2]. Furthermore, the strategic deployment of power purchase reforms has been instrumental in fostering a more resilient and adaptable electricity market structure [3]. These markets not only play a vital role in ensuring the security of energy supply but also serve as significant drivers of economic development and social sustainability [1,4,5].

The reform of power systems worldwide has been a gradual process, with different countries and regions adopting various approaches to market liberalization and restructuring. In China, a significant milestone in this journey was reached in 2015 with the release of “Several Opinions on Further Deepening the Reform of the Power System” [1]. This document marked a turning point in China’s power market reform, significantly accelerating the pace of change and introducing substantial modifications to market structures and trading mechanisms [6].

The impact of these reforms has been far-reaching. Perhaps most notably, they have given rise to the formal establishment of electricity sales businesses, opening up new opportunities for industrial and commercial users to participate directly in the market [7]. This shift represents a fundamental change in the dynamics of electricity trading, moving away from a centralized, state-controlled model towards a more diversified and competitive landscape [8].

As the reform deepens, the power market has gradually evolved into a complex ecosystem characterized by a diverse array of market participants and intricate trading modes. The year 2021 saw further advancements in this direction, with the National Development and Reform Commission issuing a series of documents aimed at deepening market-oriented reforms [2,3,9]. These policy initiatives have had profound implications for the sector, including the complete opening of the trading market to commercial users and the phasing out of catalog electricity prices for industrial and commercial consumers [2,3,10]. Such measures have injected new vitality into the market, stimulating competition and innovation [11].

To empirically validate the theoretical frameworks discussed, we conducted a case study analyzing the impact of China’s 2015 power market reforms on market efficiency and competition. Utilizing data from the China Electricity Council (CEC) spanning from 2015 to 2020, we employed a difference-in-differences approach to assess changes in market concentration and price volatility post-reform. The results indicate a significant increase in market competition, as evidenced by a decrease in the Herfindahl–Hirschman Index (HHI) from 0.35 to 0.25, and a reduction in price volatility by 15% [12]. These empirical findings corroborate the theoretical assertions that market liberalization fosters competition and enhances market stability.

The electricity market’s fundamental role in the national economy cannot be overstated. As a basic energy source, the balance between supply and demand in the electricity sector is crucial for the stable operation of society as a whole. Historically, under the traditional power system, transactions between power generation enterprises and power grid companies were relatively fixed, primarily conducted through long-term power purchase contracts [4]. However, the gradual liberalization of the power market has seen power purchasing enterprises emerge as independent market entities, leading to a diversification of transaction forms between power generation enterprises and power purchasing enterprises [5]. This evolution has significantly accelerated the marketization process of power transaction modes.

The transition from a monopolistic structure to a competitive market has been a global trend in the electricity sector. This shift can be broadly categorized into four stages of market structure: the monopoly form, the power generation competition form, the wholesale competition form, and the retail competition form [6,7]. Each of these stages represents a step towards greater market liberalization and efficiency.

In the monopoly form, power grid companies operated as vertically integrated monopolies, controlling power generation, transmission, distribution, and sales [4]. While this model played a crucial role in the early stages of electric power industry development, facilitating capital accumulation and large-scale infrastructure development, its limitations in terms of economic efficiency became increasingly apparent as power supply increased and supply–demand tensions eased.

The power generation competition form introduced the concept of “power station and network separation with online bidding” [5]. This model allowed for the existence of independent power plants (IPPs) while maintaining monopoly control over transmission and distribution. The emergence of IPPs marked the formation of a competitive market on the power generation side, introducing market economic factors such as cost accounting, price determination, and profit distribution into the power generation field.

The wholesale competition form is characterized by power generation competition and the openness of transmission networks, with large users gaining the ability to choose their suppliers [6]. This stage represents the growth phase of the electricity market, with power generation separated from the grid, and transmission and distribution operating independently.

Finally, the retail competition form is distinguished by power generation competition and the opening of both transmission and distribution networks, allowing all users to have choices [7]. This model aims to reduce electricity prices for users through market competition, separating power generation, transmission, and distribution enterprises into independent operating entities.

The evolution of these market structures reflects a global trend towards increased competition and efficiency in the electricity sector. However, it is crucial to recognize that electricity, as a commodity, possesses unique characteristics that set it apart from other goods. The production, transmission, and consumption of electricity occur almost simultaneously, and the stable operation of the power market depends heavily on the safety and reliability of the power system [5,6,7,8]. These special attributes of electricity as a commodity must be fully considered when designing electricity markets.

The universality and particularity of electric power commodities further complicate market design and operation. While electricity shares some characteristics with other commodities, such as tradeability and susceptibility to supply and demand dynamics, it also has distinct features. These include the inability to store large amounts of electricity cost-effectively, the homogeneity of electricity as a product, the predictability of electricity demand patterns, and the dual nature of electricity as both a means of production and a means of life [9,10,11,12,13,14].

Given this complex backdrop, the study of strategic behaviors and interactions between power purchasing enterprises and power generation enterprises becomes crucial. As the market evolves, these entities engage in increasingly sophisticated games of strategy, balancing cooperation and competition to maximize their benefits while navigating the unique challenges posed by the electricity sector.

The necessity of this research lies in its potential to shed light on these complex interactions and their implications for market efficiency, stability, and sustainability. By applying Evolutionary Game Theory (EGT) to analyze the strategic behaviors of market participants, this study aims to provide insights that can inform policy-making, market design, and strategic decision-making in the electricity sector [15].

The application of game-theoretical methods in electricity markets has been extensively studied, with a significant focus on EGT. EGT provides a dynamic framework for analyzing how strategies evolve within populations of market participants, reflecting real-world competitive and cooperative behaviors. Studies by Cheng et al. [15] have demonstrated the utility of EGT in modeling interactions among multiple market players and power-generation companies’ decision-making in the emerging green certificate market, respectively.

However, despite the breadth of research, several gaps remain. First, there is a limited exploration of integrating EGT with machine learning techniques such as deep reinforcement learning (DRL) to enhance strategic adaptability. Second, existing studies often focus on theoretical models without sufficient empirical validation or practical application insights. Third, the cost-efficiency of control policies derived from game-theoretical models has not been comprehensively evaluated.

As elaborated above, electricity markets have undergone substantial changes in recent years, transitioning from monopolistic to increasingly competitive structures. Building on prior contributions that apply classical game theory to electricity markets, our work extends these approaches by integrating multiple game-theoretic models (EGT, Stackelberg games, Bayesian games) with advanced DRL methods. This integration not only refines our understanding of strategic interactions but also informs how policymakers and energy stakeholders can design more responsive and equitable market frameworks—further distinguishing our research from the existing literature.

To this end, this review addresses these gaps by integrating EGT with DRL to develop a more robust analytical framework, providing empirical evaluations of control policies, and offering actionable policy recommendations based on a synthesis of existing studies. By doing so, this paper advances the understanding of how game-theoretical methods can be effectively applied to optimize electricity markets and inform policy regulations.

The significance of this research extends beyond theoretical contributions. As countries worldwide grapple with the challenges of energy transition, climate change mitigation, and ensuring energy security, understanding the dynamics of electricity markets becomes increasingly important. The findings of this study could have practical implications for market regulators, power generation companies, electricity retailers, and consumers alike, potentially leading to more efficient market operations, fairer pricing mechanisms, and more sustainable energy systems.

Moreover, as renewable energy sources continue to gain prominence in the global energy mix, the dynamics of electricity markets are set to become even more complex. This research’s exploration of strategic behaviors and market interactions could provide valuable insights into how these markets might evolve to accommodate higher penetrations of variable renewable energy sources, energy storage technologies, and demand-side management strategies.

This study’s examination of the strategic evolution and game theory analysis in power market transactions is both timely and necessary. By providing a dynamic approach to understanding the interactions between power purchasing and generation enterprises, it contributes to the broader discourse on energy market reform, sustainable development, and the transition to a low-carbon future. The insights gained from this research have the potential to inform policy, shape market designs, and guide strategic decision-making in the rapidly evolving landscape of global electricity markets. To this end, the main work in this paper is summarized as follows.

• Comprehensive Review of Electricity Market Structures

This paper presents an in-depth analysis of the evolution of electricity market structures globally. It meticulously examines the progression from monopolistic systems to increasingly competitive models, including the following: (a) the monopoly form, (b) the power generation competition form, (c) the wholesale competition form, and (d) the retail competition form. This analysis provides a foundational understanding of the context in which current market dynamics operate.

• Examination of Electricity as a Unique Commodity

This study delves into the distinctive characteristics of electricity as a commodity, highlighting the following: (a) the simultaneity of production, transmission, and consumption; (b) the inability to store large quantities cost-effectively; (c) the homogeneity of the product; (d) the predictability of demand patterns; (e) the dual nature of electricity as both a means of production and consumption. This examination underscores the unique challenges in designing and operating electricity markets.

• Analysis of Market Participants and Their Roles

This paper offers a detailed exploration of the various stakeholders in the electricity market, including the following: (a) power generation enterprises; (b) power purchasing enterprises; (c) grid operators; and (d) regulatory bodies. It elucidates the evolving roles and relationships between these entities in the context of market liberalization.

• Application of Game Theory to Electricity Market Dynamics

A significant portion of this work is dedicated to applying game theory principles to analyze strategic behaviors in the electricity market. This includes the following: (a) EGT approaches; (b) Stackelberg game models; (c) Bayesian game theory applications; and (d) the integration of DRL with game-theoretic methods. This paper provides a comparative analysis of these different game-theoretic approaches, highlighting their strengths and limitations in modeling electricity market interactions.

• Investigation of Transaction Modes

This study explores various transaction modes between power purchasing enterprises and power generation enterprises, including the following: (a) long-term power purchase contracts; (b) spot market transactions; (c) medium- and long-term bilateral negotiations; and (d) centralized competitive trading. It analyzes the implications of these different modes on market efficiency and stability.

• Examination of Market Reform Policies

This paper contextualizes its analysis within the framework of ongoing market reforms, particularly in China. It examines the following: (a) the impact of the 2015 reform initiatives; (b) subsequent policy developments up to 2022; (c) the gradual opening of markets to commercial users; and (d) the phasing out of catalog pricing for industrial and commercial consumers.

• Integration of Renewable Energy Considerations:

This study acknowledges the growing importance of renewable energy sources and explores the following: (a) the challenges of integrating intermittent renewable sources into existing market structures; (b) the impact of renewables on market dynamics and pricing mechanisms; and (c) the potential for new market designs to accommodate higher penetrations of renewable energy.

• Analysis of Information Asymmetry and Market Power

This paper investigates the role of information asymmetry and market power in shaping strategic behaviors, including the following: (a) the impact of incomplete information on decision-making; (b) the potential for market manipulation; and (c) the role of regulatory oversight in mitigating these issues.

• Exploration of Risk Management Strategies

Given the volatile nature of electricity markets, this study examines various risk management approaches, including the following: (a) the use of long-term contracts to mitigate price risk; (b) the role of the spot market in managing short-term supply–demand imbalances; and (c) strategies for managing credit risk in a more diversified market.

• Future Outlook and Recommendations

This paper concludes with a forward-looking analysis, offering the following: (a) predictions for future market developments; (b) recommendations for policy-makers and market participants; and (c) suggestions for further research in areas such as market design, renewable integration, and strategic behavior modeling.

Based on the main research work summarized above, this comprehensive approach provides a multifaceted examination of the strategic evolution and game-theoretic dynamics in modern electricity markets, offering valuable insights for both academic researchers and industry practitioners. Therefore, the main innovations of this survey are summarized as follows.

(1)

Integrated Theoretical Framework: Our unique synthesis of EGT, Stackelberg games, Bayesian games, and DRL presents an unprecedented comprehensive framework that not only enhances the understanding of strategic interactions in electricity markets but also provides practical optimization strategies. This integration facilitates the development of adaptive and resilient market participants capable of navigating the complexities of modern energy systems, thereby offering original contributions to both the theoretical and applied aspects of energy economics. Through the integration of EGT, Stackelberg games, Bayesian games, and DRL, our paper presents a comprehensive and multifaceted framework for understanding and optimizing strategic interactions in electricity markets. This holistic approach allows for the modeling of dynamic strategic behaviors, adaptation to market fluctuations, and the optimization of decision-making processes, thereby enhancing the overall efficiency and resilience of electricity markets. By leveraging the synergistic strengths of these methodologies, our framework provides deeper insights into both individual and collective market behaviors, facilitating more informed policy-making and strategic planning. This multifaceted approach allows for the modeling of dynamic strategic behaviors, adaptation to market fluctuations, and the optimization of decision-making processes, thereby enhancing the overall efficiency and resilience of electricity markets.

Specifically, the inclusion of EGT allows for the representation of evolving strategies and collective adaptation, which is critical for capturing the long-term dynamics of electricity markets. Stackelberg and Bayesian games complement this by addressing hierarchical decision-making and information asymmetry, respectively. DRL further enhances the framework by enabling real-time strategy optimization and adaptive learning, ensuring that market participants can respond swiftly to changing conditions. This integrated framework not only provides a more accurate depiction of market behaviors but also offers practical tools for optimizing market operations and designing effective policies.

(2)

Dynamic Market Evolution Analysis: Unlike many static analyses, this review introduces a dynamic perspective on market evolution. It traces the transformation of electricity markets from monopolistic structures to competitive environments, offering insights into the transitional stages and their impact on strategic behaviors.

(3)

Multi-dimensional Participant Modeling: This study introduces a novel approach to modeling market participants by considering not only their economic motivations but also their adaptability to regulatory changes and technological advancements. This multi-dimensional modeling provides a more accurate representation of real-world decision-making processes.

(4)

Contextual Analysis of Market Reforms: This paper innovatively contextualizes game-theoretic analyses within the framework of ongoing market reforms, particularly in China. This approach bridges the gap between theoretical models and practical policy implementation, offering insights into the real-world implications of market design choices.

(5)

Integration of Renewable Energy Considerations: This study breaks new ground by incorporating the challenges and opportunities presented by increasing renewable energy penetration into traditional game-theoretic models of electricity markets. This integration provides a forward-looking perspective on market evolution in the context of an energy transition.

Moreover, the insights and reflections for future research throughout this comprehensive survey can be summarized as follows.

(1)

Adaptive Market Design: This paper’s analysis suggests the need for more adaptive and flexible market designs that can accommodate rapid technological changes and shifting policy priorities. Future research could focus on developing dynamic market mechanisms that can self-adjust to changing conditions without compromising efficiency or stability.

(2)

Information Asymmetry and Market Efficiency: This study’s examination of information asymmetry in electricity markets highlights the critical role of information flow in market efficiency. This insight opens up avenues for research into innovative information-sharing mechanisms and their impact on market outcomes, potentially leveraging blockchain or other distributed ledger technologies.

(3)

Behavioral Economics in Electricity Markets: This paper’s consideration of bounded rationality in decision-making processes suggests a fertile area for future research at the intersection of behavioral economics and electricity market design. Studies could explore how cognitive biases and heuristics influence market participant behavior and overall market efficiency.

(4)

Resilience in Complex Market Systems: This review’s holistic approach to market analysis underscores the importance of system resilience in the face of external shocks (e.g., extreme weather events, geopolitical crises). This insight encourages future research into the development of robust market structures that can maintain stability and efficiency under various stress scenarios.

(5)

Interdisciplinary Approach to Market Modeling: This paper’s integration of game theory with machine learning techniques suggests the potential for further interdisciplinary research. Future studies could explore the application of other fields such as network theory, complexity science, or even quantum computing to enhance our understanding and modeling of electricity market dynamics.

(6)

Policy and Regulation in Evolving Markets: This study’s analysis of market evolution in the context of policy reforms highlights the need for adaptive regulatory frameworks. This insight encourages research into innovative regulatory approaches that can keep pace with rapid technological and market changes while ensuring fair competition and consumer protection.

(7)

Long-term vs. Short-term Market Mechanisms: This paper’s examination of various transaction modes (long-term contracts vs. spot markets) raises important questions about the optimal balance between market stability and flexibility. Future research could explore hybrid market mechanisms that effectively combine long-term planning with short-term adaptability.

(8)

Renewable Integration and Market Design: This study’s consideration of renewable energy integration challenges opens up a critical area for future research. Studies could focus on developing market structures and pricing mechanisms that effectively accommodate the variability and uncertainty associated with high penetrations of renewable energy.

(9)

Cross-border and Regional Market Integration: While this paper focuses primarily on national markets, its insights suggest the potential for research into cross-border and regional market integration. Future studies could explore game-theoretic models of international power markets, considering factors such as differing regulatory regimes and transmission constraints.

(10)

Equity and Social Welfare in Market Design: This paper’s comprehensive approach to market analysis raises important questions about the distributional impacts of different market structures. This insight encourages future research into market designs that not only maximize efficiency but also ensure equitable outcomes and enhance overall social welfare.

Electricity markets are undergoing significant transformations driven by the integration of renewable energy sources, technological advancements, and evolving regulatory landscapes. Understanding and optimizing the strategic interactions among power purchasing and generation enterprises is crucial for enhancing market efficiency, resilience, and sustainability. Game-theoretical methods, particularly EGT, have emerged as powerful tools for modeling and analyzing these strategic interactions. This review aims to comprehensively examine the application of EGT in electricity markets, evaluating its effectiveness in optimizing market performance and informing policy regulations. By synthesizing existing studies and identifying key trends and gaps, this paper provides actionable insights for policymakers and industry stakeholders seeking to leverage game-theoretical approaches for market optimization.

In conclusion, this review paper not only synthesizes existing knowledge in the field of electricity market dynamics but also opens up numerous avenues for future research. Its innovative integration of various theoretical approaches and consideration of real-world complexities provides a solid foundation for advancing our understanding of electricity markets in the context of ongoing energy transitions and technological disruptions. The insights derived from this study have the potential to inform both academic research and practical policy-making in the rapidly evolving field of energy economics and market design. To create a more cohesive narrative, we have introduced synthesis statements that connect the findings of individual studies, highlighting how they collectively advance the understanding of EGT in electricity markets. This paper is organized into eight main sections following the introduction.

Section 2 provides a comprehensive overview of the evolution and comparative analysis of global electricity market structures, as well as the game-theoretical methodology reviewed in this paper. It traces the development from monopolistic systems to competitive markets, examining four key stages: monopoly form, power generation competition form, wholesale competition form, and retail competition form. This section sets the historical and structural context, and game-theoretical methodology for the subsequent analysis.

Section 3 delves into market segmentation and stakeholder dynamics in modern electricity markets. It examines various market types, including medium-to-long-term energy markets, spot markets, and auxiliary service markets. This section also analyzes the roles and interactions of key market participants, providing a foundation for understanding the complex relationships in the electricity market ecosystem.

Section 4 introduces the application of EGT in electricity market analysis. It explains the core principles of evolutionary games and their relevance to modeling dynamic interactions between market participants. This theoretical framework is crucial for understanding the strategic behaviors explored in later sections.

Section 5 focuses on modeling strategic interactions in electricity markets using the Stackelberg game approach. This section builds on the game theory concepts introduced earlier, applying them specifically to leader–follower dynamics common in electricity markets.

Section 6 explores decision-making under uncertainty through the lens of Bayesian game theory. This section addresses the challenges of incomplete information in electricity markets, offering insights into how market participants make decisions in uncertain environments.

Section 7 discusses the integration of DRL with game theory to enhance market behavior modeling and then conducts an analysis of information asymmetry and market power. This section represents the cutting edge of analytical approaches in electricity market research, combining traditional game theory with advanced machine learning techniques.

Section 8 examines the transaction modes between power purchasing enterprises and power generation enterprises. This section applies the theoretical concepts discussed earlier to practical market interactions, focusing on long-term contracts and spot market transactions.

Finally, Section 9 provides conclusions, analyzes policy implications, and outlines future directions for energy governance.

The organization of these sections follows a logical progression from historical context and market structure (Section 2 and Section 3), through theoretical frameworks (Section 4, Section 5, Section 6 and Section 7), to practical applications and future outlook (Section 8 and Section 9). This structure allows for a comprehensive understanding of electricity markets, moving from foundational concepts to advanced analytical methods and their real-world implications.

2. Evolution and Comparative Analysis of Global Electricity Market Structures and Game-Theoretical Methodology

There are both monopoly phenomena and competition phenomena in the electricity market. Starting from the development trend of the global electric power industry, the market structure is gradually developing in the direction of complete competition. Under different power management systems, the power market structure can be roughly divided into four types: the monopoly form, the power generation competition form, the wholesale competition form, and the retail competition form.

2.1. Monopoly Form

Power grid companies in the form of a monopoly carry out power generation, transmission, and distribution, as well as electricity sales, as one, a unified monopoly [4]. In order to limit monopoly profits, the state controls electricity prices through laws and regulations. In the early stage of the development of the electric power industry, this model plays an important role in capital accumulation, large-scale development, power grid unification, planning, and construction. However, with the increase in power supply and the easing of the contradiction between supply and demand, the disadvantages of this model gradually appear, mainly manifested as low economic efficiency.

In a monopoly form of the power grid industry, a single entity is responsible for the generation, transmission, distribution, and sale of electricity. This unified structure allows the monopolist to exert complete control over the market, creating significant barriers to entry for potential competitors. The state, however, imposes regulations on electricity prices to curb monopoly profits, ensuring that the public interest is safeguarded through legal frameworks.

To formalize the economic behavior of a monopoly in the power grid sector, we can model the firm’s profit-maximization problem under regulatory constraints. Let us define the following variables:

• P: the price of electricity per unit (set by the state through regulation).
• Q: the quantity of electricity produced and sold.
• C(Q): the total cost function of the monopoly, which is a function of the quantity produced, reflecting production, transmission, and distribution costs.
• R(Q) = PQ: the total revenue of the monopoly, where P is constant due to price regulation.
• Π(Q): the profit of the monopoly.

Given that the state controls the price P to limit excessive monopoly profits, the monopoly maximizes its profit by choosing the optimal output level Q. The first-order condition for profit maximization is obtained by taking the derivative of the profit function with respect to Q and setting it to zero:

$${\displaystyle \frac{\mathrm{d}\Pi (Q)}{\mathrm{d}Q}}=P-{\displaystyle \frac{\mathrm{d}C(Q)}{\mathrm{d}Q}}=0$$

(1)

where Π(Q) is the profit function of the monopoly, expressed as Π(Q) = R(Q) − C(Q) = PQ − C(Q). Equation (1) can be simplified as

$$P={\displaystyle \frac{\mathrm{d}C(Q)}{\mathrm{d}Q}}$$

(2)

This condition implies that the regulated price P must equal the marginal cost dC(Q)/dQ at the optimal production level. In a competitive market, firms set prices equal to marginal cost, leading to allocative efficiency. However, in a monopoly, state intervention is necessary to ensure that prices are aligned with marginal costs to prevent the monopolist from extracting excessive profits. This mathematical framework illustrates how the regulation of electricity prices ensures that the monopoly’s pricing decisions align with social welfare objectives. By enforcing price controls that reflect the marginal cost of production, the state limits the monopoly’s ability to generate excessive profits while ensuring an adequate supply of electricity at a reasonable price for consumers.

2.2. Competitive Form of Power Generation

The form of power generation competition refers to the operation mode of “power station, network separation, and online bidding”. In this model, the transmission and distribution are still controlled by the monopolies. The model allows the existence of independent power plants (independent power plants, IPPs), but grid monopolies uniformly obtain the power generated by the IPP and sell it to power users. The emergence of IPPs marks the formation of the power market on the power generation side of [5]. Although there is only one buyer in the market, the separation of power supply and acquisition introduces market economic factors such as cost accounting, price determination, profit distribution, and resource utilization. The model initially introduced the market mechanism into the power generation field, increasing the competition in the power supply field. As the market matures, it will gradually stimulate the market demand of electricity users, thus promoting the electricity retail market.

2.3. Wholesale Competition Form

This form is characterized by power generation competition and the openness of transmission networks (although the distribution network is still dominant), with large users gaining choices. At its core is the opening of the transmission grid, so it is also called the form of power transmission. In this mode, power generation enterprises compete fairly and bid for the Internet, allowing large users to directly sign power purchase contracts with power generation enterprises, purchase electricity at the agreed price, and transmit it through the grid. There are competitive power generation markets, monopoly transmission markets, and competitive sales markets [6]. This stage is the growth stage of the electricity market. After power generation is separated from the grid, transmission and distribution operate separately, power generation competes, the transmission grid is open, and paid services are provided, but distribution companies still maintain a monopoly on end users.

In the wholesale competition model, the electricity market is characterized by open competition among power generation companies and the partial deregulation of transmission networks. This model represents a key developmental stage in the evolution of electricity markets, marking the transition from a vertically integrated system to a more decentralized structure. The essential feature of this model is the unbundling of power generation from transmission and distribution, enabling fair competition among power generation enterprises. Meanwhile, the transmission grid is operated as a regulated monopoly, offering open access to competing generators and large consumers under a fee-based service model. This phase is also referred to as the transmission form of competition, as the transmission grid’s openness plays a pivotal role in facilitating competitive electricity trading.

In this market structure, power generation companies compete to supply electricity, with large consumers having the freedom to directly negotiate power purchase agreements (PPAs) with generators. These agreements allow consumers to purchase electricity at negotiated rates, independent of traditional distribution monopolies, and then transmit the power through the regulated transmission grid. The transmission network remains a natural monopoly, but it serves as a neutral facilitator of competition by providing open and nondiscriminatory access to the grid for all market participants.

Mathematically, the pricing and dispatch mechanisms in such a competitive wholesale market can be expressed through a variety of optimization models. One commonly used model for generation dispatch and pricing is based on marginal cost pricing and the optimization of social welfare.

Consider the following optimization problem for social welfare maximization:

$$\underset{P_g}{\max }\left({\displaystyle \sum _{i=1}^N{U}_i(P_i)-{\displaystyle \sum _{j=1}^M{C}_j(P_j)}}\right)$$

(3)

where

• P_g represents the generation power output by the generator g;
• U_i(P_i) is the utility function of consumer i, which represents the benefit obtained from consuming power P_i;
• C_j(P_j) is the cost function of generator j, representing the cost incurred for producing power P_j;
• N is the number of consumers in the market;
• M is the number of power generators participating in the market.

Transmission constraints are incorporated into the optimization model to reflect the physical limits of the transmission grid. These constraints ensure that power flows do not exceed the transmission network’s capacity and maintain system stability. A typical transmission constraint can be expressed as

$${\displaystyle \sum _{g=1}^M{P}_g}-{\displaystyle \sum _{i=1}^N{P}_i}\le T_{\max }$$

(4)

where T_max represents the maximum transmission capacity of the grid.

The clearing price for electricity, also known as the locational marginal price (LMP), is determined by the marginal cost of generation and the value of transmission at each node in the grid. If λ represents the system-wide marginal cost of serving an additional unit of electricity, the LMP at node n, denoted as λ_n, can be formulated as

$${\lambda }_n=\lambda +{\displaystyle \sum _l{\mu }_l{f}_{ln}}$$

(5)

where

• λ is the marginal cost of generation;
• μ_l is the shadow price associated with transmission constraint on line l;
• f_ln represents the sensitivity of power flow on transmission line l with respect to the injection at node n.

The wholesale competition model provides the foundation for a more efficient and competitive electricity market, where power generation companies compete based on price and performance, while transmission services are regulated to ensure non-discriminatory access for all participants. The mathematical formulation outlined above reflects the essential dynamics of such a market, including generation optimization, transmission constraints, and price determination.

2.4. Forms of Retail Competition

The form of retail competition is characterized by power generation competition and the opening of the transmission and distribution network, where all users have choices. Its core is the full opening of the distribution side power market [7]. The purpose of implementing this model is to reduce the electricity price of users through the competition in the electricity market. This model separates power generation, transmission, and distribution enterprises into independent operating entities. According to the principle of market equilibrium, market pricing or the implementation of contract pricing breaks the traditional power comprehensive management mode formed by the power supply competition pattern. The market allows power generation enterprises to compete on the Internet, and users can choose generators and distributors independently, establishing a real buying and selling relationship between producers and consumers and promoting the full display of the characteristics of electricity commodities [15,16,17,18,19,20,21].

In the form of retail competition, the electricity market is structured to promote full-scale competition in power generation, transmission, and distribution, allowing all users, including small and large consumers, to have direct access to competitive market choices. The essential feature of this market structure is the complete opening of the distribution network, facilitating competition at the retail level. This model aims to reduce electricity prices for end-users by leveraging competitive dynamics in the power market, fostering efficiency and innovation in power generation, distribution, and retail services.

The retail competition model fundamentally breaks down the traditional vertically integrated structure, where power generation, transmission, and distribution were controlled by a single entity. Instead, generation, transmission, and distribution companies operate as independent entities, each competing in its respective sector. Power generation companies can bid to supply power to end-users, and users can independently select their preferred power supplier and distributor. This establishes a direct market relationship between producers and consumers, moving away from the centralized, monopolistic control that characterized traditional electricity management.

From a mathematical perspective, the retail competition model can be analyzed through market equilibrium models and optimal pricing mechanisms, where supply and demand determine the market-clearing prices. Consider the following market equilibrium formulation for a competitive electricity market:

$$\max \left({\displaystyle \sum _{i=1}^N{U}_i(P_i)-{\displaystyle \sum _{j=1}^M{C}_j(P_j)}}\right)$$

(6)

subject to the following:

$${\displaystyle \sum _{j=1}^M{P}_j}={\displaystyle \sum _{i=1}^N{P}_i}$$

(7)

where

• U_i(P_i) is the utility function of user i, representing the benefit derived from consuming power P_i;
• C_j(P_j) is the cost function of generator j, representing the cost associated with producing power P_j;
• P_i is the quantity of electricity demanded by user i;
• P_j is the quantity of electricity supplied by generator j;
• N represents the number of consumers, and M the number of competing generators.

The market-clearing price in the retail competition model is determined by the interaction between supply and demand. This price is established at the point where the aggregate supply of electricity from all generators equals the total demand from all consumers, ensuring the market operates efficiently. The locational marginal price (LMP) at any given node nnn in the distribution network can be expressed as

$${\lambda }_n={\lambda }_0+{\displaystyle \sum _{l=1}^L{\mu }_l{f}_{ln}}$$

(8)

where

• λ₀ is the base price reflecting the marginal cost of producing electricity;
• μ_l represents the shadow price associated with the transmission and distribution constraints on line l;
• f_ln represents the sensitivity of power flow on line l with respect to the injection or withdrawal of power at node n;
• L denotes the total number of transmission and distribution constraints in the system.

In the retail competition framework, pricing can also occur through contractual agreements between users and suppliers, often based on fixed-price contracts or long-term agreements that allow users to hedge against price volatility. The objective of market equilibrium in this model is to minimize costs for consumers while maintaining system reliability and ensuring that generation, transmission, and distribution entities operate efficiently under competitive pressures.

Finally, this model promotes the full visibility of the electricity commodity’s characteristics, where electricity becomes a tradable good subject to market rules. Through competition, market pricing reflects the true cost of supply and demand, leading to more transparent and efficient market outcomes.

The core of the electricity market lies in the economic relations, which involve the construction and institutional arrangement of competitive power transactions. These follow the basic laws of commodity markets, so understanding the electricity market must be based on economic principles. However, electricity, as a commodity, has its uniqueness compared with ordinary goods. The production, transmission, and consumption of electricity are almost synchronous, and the stable operation of the power market depends on the safety and reliability of the power system [17,18,19,20,21,22]. Therefore, these special attributes of electricity commodities must be fully considered when designing the electricity market. The universality and particularity of electric power commodities are elaborated as follows.

2.4.1. The Universality of Electricity and Bulk Commodities

As a kind of commodity, electricity has the basic attribute of an ordinary commodity, which can reflect its supply and demand situation through the price mechanism. Specific performance includes the following:

(1)

Tradeability. Electricity can be bought, traded, and transmitted. This suggests that electricity can be bought and sold on the market, and that suppliers and consumers can participate in the market transaction to meet their respective needs.

(2)

Impact of supply and demand. The electricity market follows the basic law of the supply and demand relationship. Prices are often influenced by the relative relationship between supply and demand. Prices may fall when supply exceeds demand, and prices may rise when demand exceeds supply.

(3)

Price fluctuations. The electricity market is affected by supply and demand, seasonality, weather, and other factors, and prices will also fluctuate. Price fluctuations bring uncertainty to the market and have a significant impact on the decision-making of the market entities [23].

2.4.2. The Particularity of Electric Power Commodities

Electricity commodities are essentially electricity, which is essentially different from energy commodities such as coal and oil. The particularity of electric power products is mainly reflected in their natural attributes and social attributes. These particular characteristics contribute to the difference between the electricity markets and other commodity markets [24].

(1)

Do not store electric power goods. Electricity production, delivery, and consumption occur almost simultaneously, much faster than the average commodity. There is no cost-effective way to store large amounts of electricity, so there is no traditional “pay-out” transaction.

(2)

Homogeneity of electricity bulk commodities. Electricity does not carry any identification of the producer. Power producers (usually power plants) provide power to the grid, and users of the electricity can only get the required amount of power from the grid. Electricity producers and consumers can trade, but the current in the power system follows the laws of physics and is difficult to control artificially. In essence, there is no complete correspondence between the power generation and consumption processes and the transaction results.

(3)

Predictability of electricity bulk commodities. When there is electricity demand in the long term, power demand will fluctuate daily or weekly. This predictability partly suppresses the speculation of market participants but also increases the possibility that market participants will abuse their market power.

(4)

The dual attributes of electric power commodities. Electric power is not only a means of production and a means of life; it is related to the national economy but also related to people’s lives. Therefore, the electricity market is not only a means of the production market, but also a means of the living market, or a typical public market without storage [25].

Before that, we introduce the European electricity market [8]. The European unified electricity market has made remarkable achievements in market design and collaborative operation, becoming a model for the construction of a unified electricity market. Its gradual promotion and inclusive market reform path and the unified market design principle have important reference significance for the construction of a unified power market system in China. Figure 1 shows the development process of the European electricity market.

The origin of the European unified electricity market can be traced back to the construction of the European Community in the 1950s [6]. In 1986, the signing of the Single European Act laid the preliminary foundation for the idea of a unified energy market in the EU. In 1993, the European Union formally put forward the reform goal of establishing a unified electricity market, marking the official launch of the construction of the European electricity market.

In 1996, the European Union enacted its first energy bill, requiring member states to implement market-oriented reforms and giving industrial users the right to choose their suppliers. Subsequently, Northern Europe, the United Kingdom, Germany, and other countries and regions established regional and domestic electricity markets.

In 2003, the European Union issued a second energy bill, further easing users’ options to use electricity and stipulating that cross-border transmission link lines must be open without discrimination.

In 2006, the European Electricity and Gas Regulatory Authority (ERGEG) proposed the idea of building a regional electricity market in Europe, with the aim of accelerating the construction of a unified European electricity market and taking the regional electricity market as a transitional stage. Under the promotion of relevant policies, the power markets of various countries began to further integrate, forming seven regional power markets in Central and Western Europe, Northern Europe, Southeast Europe, and the Iberian Peninsula, laying a foundation for the construction of a unified European power market [26].

In 2009, the European Union issued the third energy act, which aimed to break the vertically integrated management structure of power companies in member states, separate the ownership of grid assets by large energy enterprises, and promote the member states to establish independent transmission operation agencies (TSO) to unify the dispatch and management of transmission networks. At the same time, it promoted the establishment of the European Transmission Grid Operation Organization (ENTSO-E), creating favorable conditions for cross-border power trading and dispatch.

In 2011, the EU further set the goal of building a unified European energy market by 2014, requiring member states to speed up the connectivity of infrastructure, and gradually establish a series of laws, market rules, and market trading models to ensure the orderly construction and operation of the market. This includes the Regional Price Coupling Project (PCR) underlying the operation of the European unified electricity market.

In 2014, the PCR project was officially put into operation, and many regional power markets in Central and Western Europe and Northern Europe realized the coupling of pre-day markets. Subsequently, the Iberian and Italian electricity markets also joined the PCR project, forming the prototype of the unified European electricity market.

In 2018, in order to further enhance the liquidity and flexibility of the market, a number of power exchanges and transmission operators in the European electricity market launched the European Cross-border Intra-day Electricity Market Project (XBID) and formed the current European Unified Day Electricity Market (SIDC). At the same time, Europe is constantly pushing for the unification of balance mechanisms to achieve the optimal allocation of resources in the European electricity market.

At present, the European unified electricity market has become the largest transnational electricity market in the world. A total of 17 electricity trading agencies and 39 transmission operators are responsible for the operation of the European unified electricity market, where 35 member states can trade freely. The pre-day market and the intra-day market have realized the coupling of 23 countries and 14 countries, respectively. By promoting resource complementarity among EU member states, the unified European electricity market effectively guarantees the overall energy security of the EU, promotes the development of energy efficiency and operational efficiency, and provides a solid foundation for the development of the unified European power economy [27].

The Pennsylvania–New Jersey–Maryland United Power Market (Pennsylvania–New Jersey–Maryland Interconnection, PJM) is the largest regional power market in the United States [9]. Figure 2 provides a summary of the development of the PJM.

The PJM power market dates back to 1927, when the world’s first electric utility venture was formed by three utilities in Pennsylvania and New Jersey. In 1956, Maryland utilities joined the PJM consortium, creating a regional electricity market covering three states.

In 1992, the United States began to reform the electricity market through the energy policy bill to further promote wholesale competition in the electricity market. In 1996, the Federal Energy Commission (Federal Energy Regulatory Commission, FERC) required power companies to open their utility transmission grid and encouraged the establishment of independent system operators (Independent System Operator, ISO) and a regional transmission organization [28]. In 1997, according to the policy requirements of the federal government, the PJM began to reform, gradually transforming into an independent, neutral system operation organization and becoming the first fully functional ISO in the United States. The reformed PJM is responsible for operating the grid but does not own a transmission grid, providing non-discriminatory network access to users of non-utility services companies.

In 1999, FERC continued to promote the reform of the power industry, requiring power companies to form and join regional transmission organizations (Regional Transmission Operator, RTO) to improve the technical and economic operation efficiency of regional transmission networks, ensure the non-discriminatory opening of the power grid, enhance the reliability of the power grid, and improve the power market. The decree also specifies the main responsibilities of RTO/ISO, including price management and design, blocking management, market supervision, system planning, etc. In 2002, PJM became the first fully functional RTO in the United States.

Since 2002, PJM has integrated the transmission networks of Allegheny Power, Rockland Electric, Commonwealth Edison, American Power, and Dayton Power & Light, and its market scope has gradually expanded to Ohio and Kentucky [14]. The expansion of regional markets increases the availability and diversity of resources, better meets the consumer demand for electricity, and promotes the further development of the PJM electricity market.

After years of growth, PJM has become the largest regional power market in the United States. As of March 2023, PJM is responsible for operating power systems and power markets in 13 states, including Pennsylvania, New Jersey, Maryland, and the District of Columbia, operating more than 1400 generating units with a total installed capacity of 183 GW, a transmission distance over 88,115 miles, covering an area of 368,000 square miles, and serving a population of more than 65 million [10,29].

China’s electricity market reform began in 1978, following the reform and opening-up, transitioning through several stages from a government-led planned economy to a market-oriented mechanism [11].

Background: In the early stages of reform and opening-up, China’s economy grew rapidly, leading to a surge in power demand. However, under the planned economy system, the power supply was insufficient, and efficiency remained low.

Reform content: The government encouraged local governments and enterprises to invest in power plant construction and introduced multi-channel investments, breaking the state monopoly on power infrastructure investment. During this period, the power industry remained strictly controlled by state planning, with power production and supply concentrated in a few state-owned enterprises.

Year 1987: The Decision on Economic System Reform introduced the concept of the “separation of plant and network”, encouraging power industry reforms and proposing electricity price reforms, eventually establishing a cost-based pricing mechanism. This marked the initial attempt at reforming China’s electricity market.

Year 2002: The Power System Reform Plan (“No.5” of power reform) set the goal of the “separation of plant and network” and “bidding for internet access”. It separated power generation from transmission, leading to the creation of the State Grid Corporation and China Southern Power Grid Corporation, breaking the power industry monopoly. This marked the official entry of China’s power industry into the market-oriented reform stage.

Year 2005: The Renewable Energy Law encouraged renewable energy development, mandated priority access for renewable power generation, and promoted the integration of clean energy into the market.

Year 2015: Several Opinions on Further Deepening the Reform of the Power System (No. 9, 2015) aimed to enhance market-oriented operations and address efficiency issues in the power industry. It proposed creating an “effective competitive market structure and system”, promoted opening the electricity sales market to social capital, and encouraged the establishment of both spot and medium-to-long-term electricity markets. To enhance the flexibility of the electricity market, many provinces began piloting the electricity spot market. The spot market allows power generation enterprises and users to trade based on real-time supply and demand, using market mechanisms to address power supply–demand fluctuations [30].

Year 2020: Promotion of the national rollout of the electricity spot market, ensuring an effective combination of medium-to-long-term contracts with spot transactions to increase market transparency and flexibility.

Year 2022: The Guiding Opinions on Accelerating the Construction of a National Unified Electricity Market System specified that, by 2025, a national unified electricity market system would be established, integrating medium-to-long-term, spot, and auxiliary service markets.

Based on the above, Figure 3 illustrates the development of China’s electricity market.

2.5. Game-Theoretical Methodology

This study employs a hybrid methodological approach by integrating EGT with DRL to analyze and optimize electricity markets. This integration allows for a comprehensive examination of both population-level strategic dynamics and individual agent adaptability within the market.

• Evolutionary Game Theory (EGT) Framework

EGT extends classical game theory by focusing on the dynamic evolution of strategies within populations of players. It models how strategies adapt and proliferate based on their success, akin to natural selection. This framework proposed in Ref. [15] is particularly suited for electricity markets where strategic interactions among power purchasing and generation enterprises evolve over time due to regulatory changes and technological advancements. EGT facilitates the understanding of how cooperative and competitive behaviors emerge and stabilize within the market, providing insights into long-term strategic adaptations of market participants. Aiming at the increasing complexity of the green certificate market, the research [15] employed evolutionary games and other classical game methods to systematically analyze the strategic behavior of power generation enterprises and their interactions within the market framework. The findings in Ref. [15] demonstrate that EGT and other game models can facilitate the cost structure optimization and enhance the adaptability to market dynamics under policy-driven incentives and penalties.

2.

DRL Integration

DRL, a subset of machine learning, enables agents to learn optimal strategies through continuous interaction with the market environment. By incorporating DRL, this study enhances the EGT framework, allowing for the simulation of complex market scenarios and the adaptation of strategies in response to real-time data and evolving market conditions. Specifically, DRL algorithms such as deep Q-networks (DQN) and proximal policy optimization (PPO) are employed to enable agents to make informed decisions based on historical performance and predictive analytics.

3.

Hybrid Framework Development

The integration of EGT and DRL involves the following steps:

(i)

Strategy Initialization: Define initial strategies for market participants based on historical data and theoretical considerations. This involves categorizing strategies into distinct types, such as aggressive pricing, cooperative resource sharing, and conservative investment.

(ii)

Simulation of Market Interactions: Utilize EGT to model the strategic interactions among enterprises, capturing the competitive and cooperative dynamics inherent in electricity markets. Concurrently, employ DRL to optimize individual strategies by training agents to maximize their utility through trial-and-error learning.

(iii)

Replicator Dynamics Implementation: Apply replicator dynamic equations to update the frequency of each strategy within the population based on their relative payoffs. This step ensures that more successful strategies become more prevalent over time, reflecting the natural selection process.

(iv)

Policy Evaluation: Assess the effectiveness and cost-efficiency of various control policies under different market scenarios using the hybrid EGT-DRL framework. This involves simulating the impact of policies such as pricing mechanisms, capacity incentives, and renewable integration incentives on market performance and participant behavior.

4.

Data Sources and Validation

Data for this study are sourced from specific data sources, e.g., national energy databases, market transaction records, and regulatory reports, ensuring comprehensive coverage of market dynamics and participant behaviors. The dataset includes variables such as electricity prices, generation capacities, demand forecasts, and regulatory policies.

Validation of the model is conducted through a combination of cross-validation and sensitivity analysis. Cross-validation ensures that the model’s predictive capabilities are robust across different subsets of data, while sensitivity analysis examines how changes in key parameters affect model outcomes. Additionally, we perform scenario analysis to test the model’s responsiveness to extreme market conditions and policy interventions, ensuring its reliability and applicability in real-world settings.

5.

Software and Tools

The simulations and analyses are implemented using specific software/tools, e.g., Python 3.12 (32/64-bit), MATLAB R2023b (23.2.0.2365128) 32/64-bit (released on 23 August 2023), and TensorFlow 2.4.0, leveraging libraries such as OpenAI Gym for DRL environments and PyGame for strategic interaction simulations. This combination of tools facilitates the efficient development and execution of the hybrid EGT-DRL framework, enabling scalable and repeatable analyses.

6.

Ethical Considerations

This study adheres to ethical guidelines in data usage and modeling practices. All data sources are publicly available and anonymized to protect participant confidentiality. The modeling approach is designed to minimize biases by incorporating diverse strategic behaviors and market conditions, ensuring fair and objective analysis.

This comprehensive methodological framework provides a robust basis for analyzing strategic interactions and optimizing electricity market performance through the dynamic adaptation of participant strategies. By integrating EGT with DRL, the study offers a nuanced understanding of both collective market dynamics and individual strategic optimizations, thereby enhancing the overall efficacy and resilience of electricity markets.

Building upon the application framework in this section, Section 3, Section 4, Section 5, Section 6 and Section 7 systematically evaluate the effectiveness and cost-efficiency of various control policies, providing a critical analysis of their impact on market performance.

3. Market Segmentation and Stakeholder Dynamics in Modern Electricity Markets

The electricity market is a platform for the circulation and exchange of electricity goods and services. It is a group system organically composed of various power markets under the supervision of the market management agency. In the whole power market, each market segment plays its own unique role in the power market system, supporting, influencing, and jointly building the overall structure of the power market [31]. Although the typical power market at home and abroad is significantly different according to different construction objectives, its overall structure is roughly the same, usually including medium- and long-term energy markets, a spot energy market, an auxiliary service market, a production capacity market, a financial derivatives market, etc.

3.1. Medium-Term and Long-Term Power Energy Market

The medium- and long-term power energy market refers to where power generation companies, power users, power sales companies, and other market participants engage in power transactions lasting several days or longer [32]. Considering the periodic fluctuations in power supply and demand, and the time needed for power production, the medium- and long-term power energy market organizes transactions on various time scales, such as yearly, quarterly, monthly, weekly, and multi-day cycles. The primary forms of medium- and long-term transactions include bilateral negotiations and centralized transactions. Additionally, green electricity is only available for purchase in the medium- and long-term markets.

Green electricity is classified into two categories: electricity price and environmental premium. If the local new energy substation operates below 110 kV, the electricity price is fixed at 0.453 CNY. Only the environmental premium is negotiable, typically ranging from 6 to 7 CNY per green certificate. One green certificate covers 1000 kilowatt hours. When the local new energy substation exceeds 220 kV, it can be included in electricity price premiums. However, this premium is limited to the range of 0.372 CNY to 0.554 CNY.

3.2. Spot Electricity Market

The spot electric energy market is for the day-ahead, real-time, and other short-term electric energy trading markets [33,34,35,36]. Spot electricity trading makes up for the difference between contract-traded electricity and short-term load demand [37,38,39,40]. Spot energy trading can be divided into pre-stage energy trading formed by power generation companies and organized real-time energy trading to ensure the real-time balance of power supply and demand.

3.2.1. Advance Electricity Trading

Day-ahead electricity energy trading (called the “day-ahead market”) refers to the electricity trading for the next 24 h a day in advance. The system scheduling mechanism can determine the operation mode and scheduling plan of the system according to the market transaction results. Since all contracted power must be delivered in the day-ahead market, the day-ahead market must consider the constraints of the system power balance and unit output, so that the day-ahead market can better reflect the close interweaving of the power grid operation and economic laws. A few days ago, the market cleaned up the operating curve of the unit contract electricity in the trading day and made a house purchase plan to meet the safety of the grid, including the contract electricity and most of the on-site electricity.

3.2.2. Real-Time Electricity Trading

Real-time electricity trading (“real-time market”) is the electricity trading that follows the fluctuation of electricity demand on the day. Compared with the pre-day market, the real-time trading market organizes power generation companies to bid on power load fluctuations in future trading periods (such as 5 min, 15 min, 30 min, 60 min, etc.), as well as on the unplanned shutdown of the generator set. Given that the power revenue and expenditure must be synchronized, the main function of the real-time market is to use the market mechanism to eliminate the temporary power supply and demand imbalance.

Concretely, real-time electricity trading refers to a dynamic electricity market where transactions occur in near real-time, allowing the supply of electricity to respond immediately to fluctuations in demand. Unlike the day-ahead market, which operates based on predicted power needs, the real-time market adjusts to actual load variations throughout the day. The purpose of the real-time market is to address immediate mismatches between electricity supply and demand, typically caused by unexpected shifts in demand or unforeseen generation issues, such as the unplanned shutdown of power generation units.

In real-time electricity trading, power generation companies bid on providing electricity to meet short-term demand variations [41,42,43]. These variations are typically segmented into trading intervals, which can range from 5 min to 60 min, depending on market regulations and system requirements. The real-time market allows grid operators to maintain system balance by leveraging market mechanisms to dynamically adjust supply, ensuring that power revenue and expenditure are synchronized on an instantaneous basis.

A mathematical representation of real-time electricity trading can be modeled as an optimization problem where the goal is to minimize the cost of electricity generation while ensuring supply meets demand within each trading interval. The optimization problem can be formulated as follows:

$$\min {\displaystyle \sum _{g=1}^G{C}_g(P_g)}$$

(9)

subject to the following:

$$\left\{\begin{array}{l}{\displaystyle \sum _{g=1}^G{P}_g}=D_t\\ P_g^{\min }\le P_g\le P_g^{\max }\end{array}\right.$$

(10)

where

• C_g(P_g) is the cost function of generator g, representing the cost incurred to produce P_g units of electricity;
• P_g is the power output of generator g;
• G is the number of generators participating in the real-time market;
• D_t represents the electricity demand at time t;
• $P_g^{\min }$ and $P_g^{\max }$ represent the minimum and maximum generation limits of generator g.

The primary constraint ensures that the total power generation from all participating generators must equal the electricity demand at any given time, D_t. The generation limits for each generator, $P_g^{\min }$ and $P_g^{\max }$, reflect the physical and operational limits of each generating unit.

To address generation imbalances due to unforeseen events (e.g., sudden demand spikes or unexpected generation outages), the real-time market incorporates additional constraints and variables. For instance, the system must account for ramping rates—the speed at which a generator can increase or decrease its output—especially in cases of short-term fluctuations. This can be represented by the following ramping constraint:

$$\left|P_g(t+1)-P_g(t)\right|\le R_g$$

(11)

where

• P_g(t) is the power output of generator g at time t;
• R_g represents the maximum ramping capability of generator g, i.e., the rate at which the generator can change its output from one period to the next.

The real-time market clearing price—often referred to as the locational marginal price (LMP)—is determined based on the marginal cost of supplying an additional unit of electricity at a specific location, factoring in generation and transmission constraints. This price at location n, denoted by C_n, is given by

$$C_n=C_0+{\displaystyle \sum _{l=1}^L{\mu }_l}{f}_{ln}$$

(12)

where

• C₀ is the base marginal cost of generation;
• μ_l is the shadow price associated with the transmission constraint on line l;
• f_ln represents the sensitivity of the power flow on line l with respect to power injection at location n;
• L is the number of transmission constraints in the system.

By using real-time pricing and optimization mechanisms, the real-time electricity trading market ensures that the grid remains balanced, even in the face of unpredictable demand fluctuations and generation outages. The real-time market is critical for maintaining grid reliability and efficiency, as it incentivizes generators to respond swiftly to short-term changes in power supply and demand, while providing a transparent and competitive framework for electricity pricing.

Following the detailed methodological framework in Section 2.5, Section 4, Section 5, Section 6, Section 7 and Section 8 delve into the practical application of EGT and DRL in electricity markets, illustrating how strategic interactions are modeled and optimized.

3.3. Effectiveness and Cost-Efficiency of Control Policies

This section systematically evaluates the effectiveness and cost-efficiency of various control policies implemented within electricity markets through the lens of game-theoretical methods. The policies analyzed include pricing mechanisms, capacity incentives, renewable integration incentives, and regulatory measures aimed at enhancing market competition and transparency.

• Pricing Mechanisms

Dynamic pricing strategies, informed by EGT models, have been shown to improve market efficiency by aligning prices with real-time supply and demand conditions. For instance, Cheng et al. [15] illustrate how adaptive pricing can reduce market volatility and enhance consumer welfare. The cost-efficiency of these mechanisms is evaluated based on their ability to optimize revenue streams while maintaining affordability for consumers.

2.

Capacity Incentives

Capacity incentives are designed to ensure sufficient power generation capacity to meet peak demand. EGT models help in understanding the strategic responses of generation enterprises to such incentives, promoting investments in capacity expansion and technological upgrades. Research by Cheng and Yu [40] demonstrates that well-structured capacity incentives can lead to a more resilient and stable electricity market without imposing excessive costs on consumers, especially for the transactions in the open and ever-growing electricity markets for the power demand response.

3.

Renewable Integration Incentives

The integration of renewable energy sources is critical for sustainable market development. EGT frameworks aid in modeling the strategic adoption of renewable technologies by generation enterprises, balancing environmental goals with economic considerations. Studies from Refs. [15,37,41,42] highlight the effectiveness of subsidies and tax incentives in accelerating renewable integration while assessing their cost-efficiency in terms of long-term environmental and economic benefits.

4.

Regulatory Measures

Regulatory measures aimed at enhancing market transparency and competition are fundamental for efficient electricity markets. EGT models provide insights into the strategic behaviors of market participants under different regulatory scenarios [43]. Research by Cheng and Yu [38] shows that the regulatory measures in the electricity markets formulated by the government and other factors can gradually influence the multi-population evolutionarily stable strategy (ESS) via changing the payoff distribution matrix.

Overall, the integration of EGT with control policy analysis demonstrates significant potential in enhancing both the effectiveness and cost-efficiency of electricity market regulations. These findings provide valuable guidance for policymakers in designing and implementing strategies that balance economic efficiency, market resilience, and sustainability objectives.

4. Application of EGT in Electricity Market Analysis

4.1. Comparative Advantages of EGT

EGT is an extension of classical game theory that examines strategic interactions among populations of players who adapt their strategies over time based on the success of past actions [43]. In electricity markets, EGT is employed to model how power purchasing and generation enterprises evolve their strategies in response to changing market conditions and competitive pressures. For example, Gao et al. [34] utilized EGT to analyze the impact of renewable energy integration on strategic bidding behaviors, demonstrating how EGT can capture the dynamic adaptation processes that drive market stability and efficiency. By applying EGT, our study provides a nuanced understanding of how cooperative and competitive behaviors emerge and stabilize within electricity markets, offering insights into the long-term sustainability and resilience of market structures.

Overall, EGT offers several advantages over other game-theoretic models and simulation methods when it comes to analyzing electricity markets:

(i)

Adaptability to Dynamic Environments: EGT inherently models how strategies evolve over time, allowing for the representation of continuous adaptation and learning among market participants. This is particularly important in electricity markets, which are subject to rapid changes in demand, supply, regulatory policies, and technological advancements.

(ii)

Population-Level Insights: EGT focuses on the strategic interactions within populations of players, providing insights into collective behavior patterns and emergent market dynamics that individual-focused models may overlook. This is crucial in electricity markets, where the actions of one enterprise can significantly influence the strategies of others, leading to systemic effects.

(iii)

Flexibility in Modeling Complex Interactions: EGT can easily incorporate various factors influencing strategic decisions, including environmental policies, technological innovations, renewable energy integration, and competitive pressures. This flexibility makes it a robust tool for comprehensive market analysis, capable of addressing multifaceted and interdependent variables.

(iv)

Empirical and Theoretical Support: Numerous studies have demonstrated the efficacy of EGT in capturing the strategic evolution of market behaviors, particularly in contexts involving long-term interactions and adaptive decision-making [34,43]. These empirical validations, combined with strong theoretical foundations, reinforce the suitability of EGT for our analysis.

(v)

Enhanced Predictive Capabilities: EGT provides a dynamic framework that can predict not only equilibrium states but also transitional dynamics, offering a more nuanced understanding of how markets evolve over time. This is particularly advantageous for forecasting the impacts of policy changes and market interventions.

In contrast, while Stackelberg games are powerful in hierarchical decision-making contexts and Bayesian games excel in handling information asymmetry, they may not fully capture the dynamic and collective strategy evolution that EGT addresses. Stackelberg models are primarily suited for leader–follower scenarios and may overlook the continuous adaptation of strategies, whereas Bayesian games focus on static information asymmetry without accounting for the evolutionary aspects of strategy development. Simulation methods, while useful for modeling specific scenarios, often lack the strategic depth and adaptability inherent in EGT, limiting their applicability to complex and evolving market environments. EGT’s ability to represent ongoing adaptations and emergent behaviors makes it a more comprehensive and realistic tool for analyzing the complexities of electricity market dynamics.

4.2. Comparative Analysis with Other Game-Theoretic Models

While EGT provides significant advantages for analyzing electricity markets, it is essential to compare its efficacy with other prevalent game-theoretic models such as Stackelberg and Bayesian games to underscore its superiority in this context.

(i)

Stackelberg Games: Stackelberg models are designed to analyze leader–follower dynamics, where leaders commit to strategies that followers respond to optimally. While effective in hierarchical decision-making scenarios, they are limited in capturing the continuous and adaptive nature of strategy evolution in electricity markets. EGT, on the other hand, accommodates the iterative adaptation of strategies without predefined leadership roles, offering a more flexible and realistic representation of market interactions.

(ii)

Bayesian Games: Bayesian games address strategic interactions under incomplete information, allowing players to form beliefs about others’ types or strategies based on available information. However, they primarily focus on static information asymmetry and do not inherently account for the dynamic evolution of strategies over time. EGT surpasses Bayesian models by providing a dynamic framework that continuously evolves strategies based on historical performance and changing market conditions, thereby offering deeper insights into long-term market behavior.

(iii)

Simulation Methods: Simulation-based approaches, including agent-based models, offer flexibility in modeling complex systems by simulating interactions among heterogeneous agents. While useful for scenario analysis, they often lack the strategic depth and theoretical rigor of game-theoretic models like EGT. EGT not only models strategic interactions with evolutionary dynamics but also integrates empirical data and theoretical principles to provide a more comprehensive and predictive analysis of market behaviors.

In summary, while Stackelberg and Bayesian games offer valuable insights into specific aspects of market interactions, EGT provides a more holistic and adaptable framework for analyzing the multifaceted and evolving nature of electricity markets. Its capacity to model continuous adaptation, population-level dynamics, and complex interactions positions EGT as the superior choice for our study, enabling a more accurate and insightful analysis of strategic behaviors in electricity markets. Moreover, the recursive process illustrated in Section 5.2 underscores EGT’s suitability for capturing the iterative nature of strategy evolution, which is essential for understanding long-term market stability and resilience.

4.3. Practical Implications of Using EGT

The application of EGT in analyzing electricity markets offers several practical benefits that enhance the understanding and management of market dynamics:

(i)

Policy Formulation and Evaluation: EGT enables policymakers to simulate and evaluate the long-term impacts of regulatory interventions and policy changes on market behaviors. By modeling how market participants adapt their strategies over time, EGT provides insights into the potential effectiveness and unintended consequences of policies aimed at promoting renewable energy adoption, enhancing market competition, or ensuring grid stability.

(ii)

Strategic Planning for Market Participants: Electricity market participants, including power purchasing and generation enterprises, can leverage EGT to develop strategic plans that anticipate and adapt to the evolving strategies of competitors. EGT facilitates the identification of stable and resilient strategies that can withstand market fluctuations and competitive pressures, thereby enhancing the sustainability and profitability of enterprises.

(iii)

Market Design and Optimization: EGT contributes to the optimization of market structures by highlighting the conditions under which cooperative behaviors emerge and persist. This understanding aids in designing market mechanisms and incentives that foster collaboration, reduce market inefficiencies, and enhance overall market performance.

(iv)

Integration with Advanced Technologies: The integration of EGT with machine learning techniques, such as DRL, as discussed in Section 7.1, further amplifies its practical utility. This combination enables the development of adaptive and intelligent market models that can respond to real-time data and dynamically evolving market conditions, thereby enhancing the robustness and responsiveness of electricity markets.

Overall, the use of EGT in electricity market analysis not only provides a deeper theoretical understanding of strategic interactions but also offers actionable insights for policymakers, market designers, and participants. Its dynamic and adaptable nature makes it an invaluable tool for addressing the complexities and challenges inherent in modern electricity markets.

Members of the electricity market include operating entities, electricity market operating institutions, and power grid enterprises that provide transmission and distribution services. Among them, the operating entities include power generation enterprises, power sales enterprises, power users, and new operating entities (including energy storage enterprises, virtual power plants (VPPs), load aggregators, etc.). Institutions involved in electricity market trading and electricity market operating institutions include power trading institutions and power dispatching agencies. The main relationship of each market is shown in Figure 4. Based on this figure, in Section 4.4, Section 4.5 and Section 4.6, the power generation side, electricity sales company, and power grid side will be introduced from a comprehensive perspective.

4.4. Power Generation Side

Power generators play a vital role in the power market, and their behavior and decisions directly affect the trading strategies of power sales companies and power users. The power generation side is generally power plants, which can be divided into traditional power plants, such as coal-fired power generation, as well as wind power, photovoltaic, hydropower, biomass energy, and other new energy power plants; the five southern provinces and regions are mainly characterized by coal shortage, oil shortage, natural gas shortage, and a relative richness in hydropower resources [12]. During the 14th Five-Year Plan period, the pace of clean energy development in Southern China was accelerated, promoting the rapid growth of installed power capacity.

By the end of 2022, the total installed capacity of the China Southern Power Grid had reached 393 million kilowatts. Specifically speaking, the installed hydropower capacity is 122 million kilowatts, accounting for 31.08% of the total installed capacity; the installed thermal power capacity is 165 million kilowatts, accounting for 42.1% of the total installed capacity; the installed nuclear power capacity is 19.61 million kilowatts, accounting for 5.00% of the total installed capacity; the installed wind power capacity is 37.74 million kilowatts, accounting for 9.62% of the total installed capacity; the installed photovoltaic capacity is 29.36 million kilowatts, accounting for 7.48% of the total installed capacity; the installed biomass energy capacity is 6.4 million kilowatts, accounting for 1.63% of the total installed capacity; and the installed pumped storage capacity is 9.68 million kilowatts, accounting for 2.47% of the total installed capacity of [13]. The proportions of various installed capacities are shown in Figure 5.

With the development of new energy technology, energy storage technology, and distributed power generation, VPPs have also begun to participate in the electricity market. The research progress in distributed energy sharing service mechanisms is reviewed. At first, grid companies disliked buying power from distributed users because it was intermittent and placed undue pressure on the grid but have changed their attitude due to policy incentives and grid technology upgrades. Currently, the electricity generated by distributed generation users is usually purchased from the grid side at 0.453 CNY per kWh. However, photovoltaic is intermittent and unstable when participating in the power market: photovoltaic power generation is affected by weather and sunshine conditions, and its output power is intermittent and unstable [14]. This brings challenges to the dispatching and balance of the power system, which requires more precise prediction and regulation means. To this end, combining with EGT, we constructed a simulation model, which models a multi-agent evolutionary game scenario on the power generation side within the electricity market of Southern China.

The primary motivation for this simulation study is to understand how different types of power generators—comprising both traditional (coal, nuclear) and renewable (hydropower, wind, photovoltaic, biomass, pumped storage) energy sources—interact and evolve strategically over time in response to varying market conditions and competitive dynamics. By employing replicator dynamics, the simulation captures the long-term strategic adaptations of these generators based on their relative payoffs, which are influenced by factors such as cost structures, market competitiveness, and policy incentives. The core innovation lies in integrating actual regional power capacity data into an evolutionary game-theoretic framework, thereby providing a realistic and dynamic representation of market evolution. Key parameters include initial installed capacities measured in million kilowatts (MW), a payoff matrix representing relative strategic advantages, and a simulation period of 200 time steps to observe the evolution of generator frequencies. The simulation results are illustrated in Figure 6.

The simulation results, depicted in Figure 6, illustrate the frequency evolution of various power generation strategies over 200 time steps. Initially, traditional power sources, such as coal and nuclear, dominate the market due to their substantial installed capacities. However, as the simulation progresses, renewable energy sources like wind and photovoltaic begin to gain traction, reflecting their higher relative payoffs within the payoff matrix. This upward trend indicates a strategic shift towards cleaner energy sources driven by their competitive advantages in the evolving market landscape. Hydropower and pumped storage maintain relatively stable frequencies, underscoring their role in ensuring grid stability and accommodating fluctuations in supply and demand. Conversely, biomass energy remains marginal, suggesting limited competitiveness under the current model parameters. This simulation effectively demonstrates how market incentives and strategic interactions foster the gradual transition from traditional to renewable energy sources, highlighting the dynamic balance between economic viability and sustainable energy practices.

As shown in Figure 6, this study employs EGT to analyze the strategic interactions and long-term evolution of various power generation technologies within Southern China’s electricity market. By integrating real-world data on installed capacities and utilizing replicator dynamics, the research elucidates how traditional and renewable energy sources compete and adapt over time to optimize their market positions. The simulation reveals that renewable energy sources, supported by favorable payoffs, progressively increase their market share, thereby driving the energy transition towards sustainability. Traditional power sources, while initially dominant, experience a relative decline in frequency as renewables become more competitive. The stability of hydropower and pumped storage highlights their essential role in maintaining grid resilience. This comprehensive analysis not only advances our understanding of multi-agent strategic behaviors in power markets but also provides valuable insights for policymakers aiming to design effective energy policies that balance economic efficiency with environmental sustainability. Future research could enhance the model by incorporating additional factors such as policy changes, technological advancements, and external disruptions, thereby offering a more nuanced and robust framework for simulating and guiding the ongoing evolution of electricity markets.

Based on Figure 6, we further investigate the evolutionary dynamics of power generation strategies in a competitive electricity market under the influence of policy changes, technological advancements, and external shocks. The simulation is motivated by the need to understand how various power generation types—including coal, nuclear, hydropower, wind, photovoltaic, biomass, and pumped storage—adjust their strategic frequencies in response to dynamic market conditions. The primary goal is to provide insights into how these market dynamics facilitate the transition toward renewable energy, optimize resource allocation, and enhance grid resilience. The simulation results are illustrated in Figure 7. Key parameters are set as follows. Installed capacities (MW): coal (165 MW), nuclear (19.61 MW), hydropower (122 MW), wind (37.74 MW), photovoltaic (29.36 MW), biomass (6.4 MW), and pumped storage (9.68 MW); time steps: 200; payoff matrix: values reflect strategic interactions between energy types; policy shocks: renewable energy subsidies and carbon taxes; technological advancements: incremental gains in renewable efficiency; external shocks: disruptions affecting coal and hydropower efficiency.

Figure 7 effectively captures the impact of policy changes, technological advancements, and external shocks on the strategic frequencies of different power generation types over 200 time steps. The simulation results reveal the following key insights.

• Initial Dominance of Traditional Power Generators

Coal and nuclear power demonstrate an early dominance in the simulation due to their high installed capacities and well-established infrastructure.

2.

Impact of Policy Changes

• Time Step 50: the introduction of renewable energy subsidies significantly enhances the payoffs for wind and photovoltaic (PV) power, leading to a rapid increase in their market shares.
• Time Step 150: the implementation of a carbon tax reduces the payoffs for coal and nuclear power, resulting in a marked decline in their market frequencies.

3.

Sustained Impact of Technological Advancements

Periodic technological improvements, occurring every 50 time steps, continually enhance the efficiency and competitiveness of wind and PV power, further consolidating their positions in the market.

4.

Temporary Effects of External Shocks

External shocks at time steps 70, 130, and 180 temporarily reduce the efficiency of coal and hydropower, underscoring their vulnerability to sudden disruptions. These shocks highlight the importance of diversification and adaptability in the energy portfolio.

5.

Long-Term Trends and Market Transition

• The sustained increase in the frequencies of wind and PV power over time indicates a clear market transition toward clean energy.
• The gradual decline in the frequencies of coal and nuclear power reflects the relative disadvantages of traditional energy sources in a competitive, policy-driven market.
• The stability of hydropower and pumped storage demonstrates their critical roles in maintaining grid resilience and balancing supply–demand fluctuations.
• The persistently low frequency of biomass energy suggests its limited competitiveness in the current market environment.

Overall, this study in Figure 7 employs EGT to simulate and analyze the strategic interactions and long-term evolutionary dynamics of multiple power generation technologies within the southern China electricity market. By integrating real-world installed capacity data and applying replicator dynamics, the research reveals how traditional and renewable energy sources adjust their strategies to optimize market positions. The simulation results highlight that renewable energy technologies—driven by higher relative payoffs, policy incentives, and technological progress—gradually secure larger market shares, while traditional energy sources experience a steady decline in their frequencies. The stability of hydropower and pumped storage underscores their critical roles in maintaining grid flexibility and managing supply–demand variability. The inclusion of external shocks in the model demonstrates the necessity of energy portfolio diversification and resilience in mitigating the risks associated with sudden disruptions. This comprehensive analysis deepens the understanding of multi-agent strategic behavior in the power market and provides scientific evidence for policymakers in designing effective energy policies that balance economic efficiency with environmental sustainability. Future research can enhance the model’s granularity and robustness by incorporating more complex factors, such as policy volatility, technological breakthroughs, and external uncertainties, to provide more precise and comprehensive simulation support for steering the electricity market toward a sustainable and resilient energy future.

4.5. Electricity Sales Company Side

In the power market, as a key link connecting power generation enterprises and power users, its behavior and strategy selection in the process of power trading have an important impact on the efficiency and stability of the market. With the promotion of the reform of the power market, the power sales companies are no longer limited to the traditional electricity selling function but have developed into the main body of diversified energy management services, such as energy management, demand response, and contract design. Therefore, studying the behavior of power sales companies, especially from the perspective of EGT, contributes to a deep understanding of their decision-making mechanisms in a complex market environment and how to cope with internal and external challenges through strategic adjustment [15]. Power sales companies are divided into independent power sales companies and power sales companies.

As an important part of the power market, the power buying and selling behavior of independent power sales companies plays a key role in the market pattern and dynamic balance. These companies do not come from power generation companies, but are independent of the power generation, transmission, and distribution business, directly participating in the power market competition, providing users with power purchase options, and promoting market diversification and service innovation. The rise of independent power sales companies stems from the opening and reform of the power market, aiming to break the monopoly of traditional power companies, improve market efficiency, and improve the choice of users [4].

Power sales companies, usually stemming from power generation companies, have the dual functions of generating electricity and selling electricity directly to the end-users [16]. This model has unique advantages and challenges in the electricity market. These companies usually have certain sources of power generation, such as coal burning, hydropower generation, nuclear power, etc., that allow them to have some degree of control over power [17]. In terms of electricity sales, they can directly sign power purchase contracts with users, provide stable and reliable power supply methods, or participate in the bidding of the electricity market. With the deepening of the reform of the consignment party, in the future forward price model, the consignment enterprise is no longer the “intermediary” role that only earns the price difference. With the increase in the number of market entities and the narrowing of the price difference, the market competition of the electricity sellers is becoming increasingly fierce. Blind attention to the trading of electricity transactions is no longer the way of life of electricity sales companies. It is difficult to continue with the “price difference” model, and the differentiated services of e-commerce enterprises will become the mainstream [18]. New concept power sales companies focus more on integrated energy services, providing users with a full range of energy-saving services, creating a combination of optical storage and power sales modes, and fully optimizing the energy consumption mode of users through the integrated energy management system.

Based on the elaborations above, and combining with EGT, we investigate the strategic dynamics of electricity sales companies within a restructured power market through an evolutionary game-theoretic framework. As critical intermediaries between power generation entities and consumers, these companies have transitioned from traditional electricity sellers to providers of diversified energy management services, including energy management, demand response, and contract design. This transformation necessitates a comprehensive understanding of their decision-making processes and strategic adaptations in response to evolving market conditions and competitive pressures. Motivated by the objective to enhance market efficiency and stability, the study employs a multi-agent evolutionary game model to simulate the long-term strategic evolution of electricity sales companies. The model distinguishes between two primary strategies—price competition and differentiated services—and incorporates mechanisms such as strategy imitation and mutation to capture the dynamic nature of strategic interactions. The core innovation lies in integrating the dual roles of electricity generation and sales, facilitating a nuanced examination of strategy diversification beyond conventional price-based competition. This approach provides comprehensive insights into how electricity sales companies navigate complex market environments, optimize service offerings, and achieve sustainable competitive advantages. The simulation results are demonstrated in Figure 8. Key parameters are set as follows. Number of agents (N): 1000 electricity sales companies; initial strategy ratios: Price Competition: 50%; Differentiated Services: 50%; generations (G): 500 evolutionary iterations; mutation rate (μ): 1% per agent per generation; payoff values are set as Price Competition vs. Price Competition: 3 units; Price Competition vs. Differentiated Services: 4 units; Differentiated Services vs. Price Competition: 5 units; and Differentiated Services vs. Differentiated Services: 6 units.

Figure 8 provides a comprehensive visualization of the strategic evolution among electricity sales companies over 500 generations. The figure comprises four subfigures that collectively depict the temporal dynamics of strategy adoption, the corresponding average payoffs, and the distribution of strategies across generations. These visualizations offer an integrated perspective on how strategic preferences shift in response to competitive pressures and payoff incentives within the simulated market environment.

• Figure 8a: Evolution of Strategies in Electricity Sales Companies

This figure presents the temporal evolution of strategies employed by electricity sales companies over 500 generations. The graph illustrates the proportion of companies adopting each strategy—Price Competition and Differentiated Services—across successive generations. Initially, both strategies coexist with equal prevalence. However, as generations progress, a noticeable decline in the Price Competition strategy is observed, while the proportion of companies adopting Differentiated Services steadily increases. This trend indicates a strategic shift towards differentiated service offerings, suggesting that companies find greater long-term benefits and higher payoffs in providing diversified energy management services rather than solely competing on price.

2.

Figure 8b: Average Payoff Over Generations

This figure depicts the evolution of the average payoff across all electricity sales companies over the 500 generations. The average payoff remains relatively stable during the initial generations, reflecting the balanced strategy distribution. As the simulation progresses and the prevalence of Differentiated Services increases, the average payoff exhibits a consistent upward trend. This rise signifies enhanced market efficiency and profitability, as the Differentiated Services strategy yields higher individual payoffs compared to Price Competition. The increasing average payoff underscores the collective benefit of strategic diversification within the market.

3.

Figure 8c: Heatmap of Strategy Proportions

This figure showcases a heatmap representing the distribution of strategy proportions across all generations. The x-axis denotes the generations, while the y-axis lists the strategies. The color intensity indicates the proportion of each strategy at a given generation, with darker colors representing higher proportions. The heatmap reveals a gradual transition from an initial balanced distribution to the dominance of Differentiated Services. Over time, the proportion of Price Competition diminishes, corroborating the trend observed in Figure 8a,b. This visual representation highlights the dynamic shift towards more profitable and sustainable strategies within the market.

4.

Figure 8d: Stacked Area Chart of Strategy Proportions

This figure presents a stacked area chart illustrating the cumulative proportions of each strategy over generations. The chart provides a clear and intuitive comparison of how the strategies evolve relative to each other. Initially, both strategies occupy equal areas, reflecting the starting balance. As generations advance, the area corresponding to Differentiated Services expands, while that of Price Competition contracts. This stacked representation emphasizes the relative growth of differentiated services and the corresponding decline of price-based competition, highlighting the market’s strategic realignment towards more value-added offerings.

Overall, this study in Figure 8 employs an evolutionary game-theoretic model to elucidate the strategic dynamics of electricity sales companies in a restructured power market. The simulation reveals a pronounced shift from Price Competition to Differentiated Services over 500 generations, driven by the higher payoffs and enhanced market efficiency associated with diversified service offerings. The findings underscore the critical role of strategic innovation and adaptation in maintaining competitiveness and achieving long-term sustainability in a complex and evolving market landscape. The integration of dual roles in electricity generation and sales within the model provides a more realistic and comprehensive framework for analyzing strategic interactions. Future research could extend this model by incorporating additional strategic options, heterogeneous agent characteristics, and more nuanced market interactions to further refine the understanding of strategic evolution in the electricity sales sector. Additionally, empirical validation using real-world data would enhance the model’s applicability and provide deeper insights into the practical implications of strategic decision-making in energy markets.

4.6. Power Grid Side

The grid side plays a crucial role in the power market. It is not only the key infrastructure for power transmission, but also the executor of market rules and the coordinator of all market parties. With the advance of the reform of the power market, the role of the grid side has gradually changed from a pure infrastructure supplier to a market participant, and its behavior directly affects the efficiency and stability of the market. In this section, we discuss the strategic choices between power purchases and power sales in the game, and how these choices affect the overall market dynamics. The power grid side is divided into a power trading center and a power dispatching center [19].

The power trading center mainly provides the design of market rules [20], the price formation mechanism, the scheduling decision, and the interaction with the generation side and the sales side. In terms of market rules design, the power grid affects the competition pattern between power generation companies and power sales companies by setting trading rules and settlement mechanisms. The power dispatching center is responsible for dispatching the power and sending it into the hands of users on time.

The grid is not only the key infrastructure for transmission in the electricity market, but also the executor and coordinator of market rules. With the reform of the electricity market, the role of the power grid changes from an infrastructure supplier to a market participant, and its behavior directly affects the efficiency and stability of the market. The power grid affects the competition pattern between power generation companies and power sales companies by means of market rule design, price formation mechanisms, and dispatching decisions, ensuring the fairness and transparency of the rules [21]. In addition, the power grid also needs to integrate the price elasticity demand response mechanism to ensure that the price signal truly reflects the supply and demand relationship, as well as improve the operation efficiency and service quality of the power market through a dynamic control strategy.

To this end, we adopt EGT to model the strategic interactions between power trading centers and dispatching centers within the power grid side using an evolutionary game-theoretic framework. The primary motivation is to investigate how the strategic choices of these key market participants influence the overall efficiency, stability, and dynamics of the power market. By simulating the long-term evolution of different strategies, the research aims to identify optimal strategic behaviors that enhance market performance and ensure fair competition among participants. The power grid is transitioning from a traditional infrastructure provider to an active market participant, necessitating a deeper understanding of how strategic decisions by grid-side entities impact market dynamics. This simulation aims to elucidate the mechanisms through which power trading and dispatching strategies evolve over time, ultimately affecting market efficiency and stability. The purpose is to provide insights that can inform policy-making, market rule design, and the strategic planning of power grid operators to foster a competitive and resilient power market.

The simulation model incorporates multiple agents representing power trading centers and dispatching centers, each with distinct strategies and associated payoffs. The simulation results are illustrated in Figure 9. Key features are set as follows. Strategy diversity: agents can adopt various purchasing and dispatching strategies, such as aggressive purchase, moderate purchase, conservative purchase, fast dispatch, balanced dispatch, and slow dispatch. Evolutionary dynamics: strategies evolve based on their payoffs, reflecting their effectiveness in the competitive market environment. Visualization tools: The model employs comprehensive visualization techniques, including line plots, heatmaps, and three-dimensional surface plots, to capture the temporal and interactive dynamics of strategy evolution. Parameter settings include generations (time steps): 100 steps; number of agents: 100 (50 power trading centers and 50 power dispatching centers); payoffs: numerical values representing the effectiveness of each strategy (unit-less or interpreted as monetary units). The core innovation lies in the integration of EGT with detailed strategy interactions specific to the power grid side, providing a nuanced understanding of how strategic behaviors co-evolve and influence market outcomes. Based on this, an in-depth analysis of the simulation results in Figure 9 is conducted as follows.

• Figure 9a: Temporal Evolution of Purchasing Strategies among Power Trading Centers

The line plot reveals that the Aggressive Purchase strategy initially dominates the market but gradually declines as the Moderate Purchase and Conservative Purchase strategies gain prevalence. This trend suggests that while aggressive strategies may offer short-term advantages, sustained market competition and the balancing effects of moderate and conservative strategies lead to a more stable and diversified strategic landscape. The declining dominance of aggressive strategies indicates a natural regulatory mechanism where extreme behaviors are tempered by more balanced approaches over time.

2.

Figure 9b: Strategy Frequency Heatmap of Dispatching Strategies among Power Dispatching Centers

The heatmap illustrates the fluctuating frequencies of Fast Dispatch, Balanced Dispatch, and Slow Dispatch strategies across 100 time steps. Fast Dispatch strategies exhibit high frequency during the initial phases, likely capitalizing on early market opportunities. However, their prevalence decreases as Balanced Dispatch strategies become more common, indicating a shift towards more sustainable and efficient dispatching practices. Slow Dispatch strategies remain consistently low, suggesting limited appeal or effectiveness within the simulated market environment. The heatmap underscores the dynamic adaptation of dispatching strategies in response to evolving market conditions.

3.

Figure 9c: 3D Evolution of Strategy Frequencies Over Time

The three-dimensional surface plot provides a comprehensive view of how purchasing and dispatching strategies co-evolve. It highlights regions where certain combinations of strategies coexist, indicating synergistic or competitive interactions. The interplay between Aggressive Purchase and Fast Dispatch strategies shows peaks and troughs, reflecting their mutual influence on each other’s prevalence. Conversely, combinations involving Moderate Purchase and Balanced Dispatch exhibit more stable frequency patterns, suggesting that these strategies contribute to market equilibrium. The 3D visualization effectively captures the complex dependencies and feedback loops that drive strategic evolution in the power grid side.

Overall, this study in Figure 9 presents an evolutionary game-theoretic simulation of strategic interactions between power trading centers and dispatching centers within the power grid side. The simulation reveals how different purchasing and dispatching strategies evolve over time, highlighting the transition from aggressive to more balanced and conservative approaches. The integration of multiple visualization techniques provides a nuanced understanding of the dynamic interplay between strategies, offering valuable insights into market behavior and stability. The core conclusion emphasizes the importance of strategic diversity and adaptive behaviors in fostering a resilient and efficient power market. Future research could expand the model to incorporate additional factors such as regulatory interventions, external market shocks, and multi-layered strategic interactions, thereby enhancing the robustness and applicability of the findings to real-world power market scenarios.

5. Modeling Strategic Interactions in Electricity Markets: The Stackelberg Game Approach

As shown in Figure 10, game theory originates from the study of competition and decision behavior and is based on mathematical models and strategy analysis. With this graph, we can better understand the different cases and game theories. Participant relationships can help us understand how different cooperative versus competing strategies affect the overall outcome. The level of rationality makes us realize that not all players are completely rational, and that different game theories can handle different decision situations. The division of action order can be used to analyze different features in static and dynamic decision-making processes and their influence on the results. The extent of disclosure emphasizes the importance of information in games, and there will be significant differences in game strategies for complete and incomplete information. The reward situations help analyze how strategies can maximize returns or avoid losses in different game environments.

5.1. Foundation of Game Theory

With the gradual emergence of deregulation and market competition, the modern power system will face a series of challenges. On the other hand, with the deepening of double-carbon construction, the proportion of renewable energy is also increasing. Therefore, the operation and management of the power system need to keep up with the pace of time to meet the needs of the new power system. Some investigations predict the state parameters in the operation process of the power market using econometric models or optimize the bidding decisions of the power market through traditional optimization models. However, classical models often fail to align with the actual dynamics of the electricity market, necessitating the introduction of game theory to address decision-making challenges. Cheng and Yu [38], Cheng and Yu [40], and Abapour et al. [43] introduced fundamental concepts of game theory, including cooperative/non-cooperative, static/dynamic, zero-sum/non-zero-sum, complete information, and incomplete information games, as summarized in

Table 1. Basic concepts of game theory summarized in Refs. [38,40,43], including cooperative/non-cooperative, static/dynamic, zero-sum/non-zero-sum, complete information, and incomplete information.

Categories	Description	Superiority	Not Enough
Cooperative/non-cooperative	✓ Cooperative games allow participants to jointly develop strategies through consultation to achieve more favorable outcomes for all participants. ✓ Uncooperative games mean that market participants independently develop their own strategies, do not share information or resources with other participants, and are usually based on individual optimal response strategies.	✓ Cooperation to improve efficiency, through sharing resource optimization strategies, usually achieves higher social welfare, reduces the waste of resources, and improves efficiency; enhances market stability and predictability, and reduces uncertainty; promotes long-term development; and builds partnerships of sustained investment and innovation. ✓ Non-cooperative games promote efficiency improvement and innovation, simplify the decision-making process, and allow participants to maintain strategic flexibility and operational autonomy.	✓ Cooperative gaming involves execution costs and complexity because it is time-consuming and expensive to establish and maintain partnerships, especially when involved in coordination and negotiation. There is also a risk of power imbalance, and strong players may dominate the cooperation, resulting in the uneven distribution of resources and benefits. In addition, over-reliance on partners may reduce market competitiveness and flexibility. ✓ Uncooperative games may lead to inefficient resource allocation and reduced overall returns as participants pursue individual maximization. Moreover, they could increase market volatility and risk, especially when demand or supply fluctuates wildly. In the face of external economic or policy shocks, the non-cooperative market response ability is weak and the adjustment is difficult.
Static/dynamic	✓ Static game: all participants make decisions at the same time, without temporal order. ✓ Dynamic game: the participants’ decisions exist in chronological order, and the second participant can observe the first participant’s decision.	✓ Static games are easy to analyze; decisions are based on current information; widely used in supply and demand and price research to explain short-term market behavior. ✓ Dynamic game analysis strategies change over time, predict long-term trends in the market and behavior, and optimize long-term return strategies for resource and investment decisions.	✓ Static games ignore history and future influences and may not fully understand the long-term strategies of market players. They have limited analysis of strategic interactions and may be unable to accurately predict market trends and behavioral adjustments. Moreover, static games tend to assume that market participants make independent decisions without considering the potential benefits of cooperation.
Zero/non-zero games	✓ Zero-sum game: the result of a game is a distributed process in which the return of one side is equal to the loss of the other side, and the sum is zero. ✓ Non-zero-sum game: the outcome of a game can be multiple allocation methods, and participants gains and losses do not have to offset each other.	✓ Zero-sum games are easy to understand and predict because the participants’ goals and strategies are directly opposed. It provides a clear framework for analyzing and optimizing individual strategies. The goal of the participants is to directly maximize their own interests while minimizing the interests of their opponents, making the strategy choice relatively simple. ✓ Non-zero-sum games encourage cooperation and achieve a total utility greater than zero-sum games. They provide flexible strategies to adapt to different market environments. In addition, non-zero-sum games promote innovation and win-win results and stimulate solutions through cooperation to achieve resource sharing and efficiency improvement.	✓ Zero-sum games lead to cooperation difficulties because of the limited space for mutual benefit. They may exacerbate conflict and harm long-term relationships and collective goals. The adversarial nature may limit innovation and efficiency as participants may focus on short-term gains. ✓ Non-zero-sum game analysis is complex, involving multi-party interactions and more variables. The results were inconclusive and were dependent on the choice of the participants. Cooperation requires the resolution of trust, communication, and conflicts of interest.
Complete information/incomplete information	✓ Full information: all participants have all the relevant information needed for the game, with no information asymmetry. ✓ Incomplete information: at least one participant did not understand some aspects of the game, and there was an information asymmetry.	✓ The full information game is highly predictive because all information is open and known, and participants can accurately predict the opponent’s behavior and develop optimal strategies. The decision is simple, because the strategy choice is directly explicit, reducing the uncertainty. Theoretical maturity, such as the Nash equilibrium, facilitates the analysis and application. ✓ Incomplete information games are closer to reality because participants often cannot have all the information. This environment prompted the participants to actively gather information to gain an advantage. At the same time, the uncertainty of information leads to strategic diversification, increasing the complexity and interest of the game.	✓ Limitation of information game: it is difficult to obtain complete information from all participants in practice, which limits the theoretical application. ✓ Ignoring the cost of information: the hypothesis ignores the cost and difficulty of obtaining information in reality. ✓ Transparency: full transparency may leak strategic information, which is not conducive to protecting interests. ✓ Complexity: the complexity of incomplete information game analysis stems from its increased uncertainty and analytical difficulty, and requires sophisticated mathematical tools and theoretical models, such as Bayesian games or signaling games. ✓ Strategy formulation: It is difficult because of an incomplete understanding of the opponent and may lead to non-optimal decisions. The outcome variability increases because incomplete information may lead to the instability and unpredictability of game outcomes, thereby increasing decision risk.

. For example, the study in Ref. [41] complements our analysis by demonstrating how hierarchical decision-making structures can influence market stability, thereby reinforcing the foundational principles of Stackelberg games in dynamic market environments.

5.2. EGT Method

EGT is an extension of classical game theory that studies the strategic interactions among populations of players who adapt their strategies over time based on the success of past actions [43]. EGT models the dynamic processes of natural selection, adaptation, and strategy evolution by examining how players gradually form and adjust strategies in long-term interactions. This approach is particularly useful in understanding how cooperation and competition emerge and stabilize within electricity markets.

To this end, we chose the evolutionary game of rationality as a research topic, as the field provides insights into complex decision-making behavior, particularly in contexts under incomplete rational and non-ideal conditions. Building upon traditional EGT, our analysis incorporates multi-layered strategic adaptations influenced by renewable energy integration and regulatory changes. This extension provides a more nuanced understanding of how players not only adapt their strategies over time but also how these adaptations interact across different market layers, leading to emergent market behaviors that enhance overall system resilience and efficiency. EGT offers a robust framework for understanding the dynamic processes of natural selection, adaptation, and strategy evolution within electricity markets. By examining how participants iteratively adjust their strategies in response to environmental changes and competitor behaviors, this theory provides valuable insights into the emergence of stable market equilibria [34,35]. For instance, Gao et al. [34] utilize hierarchical game frameworks to model interactions among multiple market players, highlighting the role of strategic adaptation in resource allocation. Similarly, Liang et al. [35] apply evolutionary game models to renewable energy consumption trading, demonstrating how bounded rationality and learning dynamics influence market stability. By synthesizing these studies, our review elucidates the multifaceted applications of EGT, emphasizing its significance in enhancing market efficiency and resilience in the face of evolving energy landscapes. These studies not only have important implications for understanding behavior in biological evolution and ecosystems but also provide new perspectives on behavior patterns and decision processes in areas such as the social sciences, economics, and political science. Therefore, this topic has a wide application and research value in academic circles. This paper focuses on the evolution of participant rationality in EGT. A typical example often cited in this case is the Prisoners’ Dilemma [35]. We mainly studied the Prisoners’ Dilemma game model and its evolutionary stability strategy. The main difference between evolutionary game and classical game theory is that, in evolutionary games, the participants have finite rationality, which is completely rational compared to the participants usually assumed in classical game theory. Specifically, strategies in evolutionary games evolved through a series of processes such as mutation and inheritance and were not explicitly selected by participants as in classical game theory. Moreover, evolutionary games focus more on dynamic processes and long-term trends. In contrast, classical game theory tends to focus on static analysis. Moreover, evolutionary games also emphasize the stability of the equilibrium, aiming to explore how the system reaches a steady state, while classical game theory focuses more on concepts such as the Nash equilibrium [36]. Learning and strategy adjustment of game players in evolutionary games are the key to finite rationality analysis. Due to varying levels of rationality and learning abilities among game players, diverse approaches emerge in the dynamic adjustment process of strategies. Our application of the replication dynamic equations within the ESS framework incorporates stochastic elements and multi-strategy interactions, thereby providing a more realistic and robust model of strategy evolution in electricity markets. This enhanced modeling approach allows for the capture of nuanced behavioral adaptations and external influences, offering deeper insights into the stability and resilience of market equilibria. The equilibrium point was calculated by bringing the replicate dynamic equations into the ESS. The replication dynamics equation is a central concept in EGT. It describes how the frequency with which individuals adopt a specific strategy in a population evolves over time. According to the replication dynamics equation, the proportion of a strategy increases when its average return exceeds the overall average return and decreases if its returns are lower than the average. This dynamic process embodies the principle of natural selection, namely, a relative increase in the number of individuals adopting a superior strategy. ESS represents an evolutionarily stable strategy in evolutionary games.

To capture the ongoing nature of this evolution, the process is designed to be recursive, where the update population step loops back to the calculation for the t time step, allowing continuous strategy adaptation and population adjustment over successive iterations. ESS represents an evolutionary stabilization strategy in the evolutionary game. This means that in a population, once the strategy is achieved, it is difficult to be replaced by other strategies.

When the population generally adopts a strategy, any mutation strategy must not bring a higher payment to its users to resist invasion. If another strategy yields equal payments when facing the mainstream strategy, it must ensure that the payment against itself is lower, thereby maintaining the strategy’s advantage even in the presence of small mutations.

Payment (or fitness) refers to the payoff received when an individual adopts a specific strategy in an interaction, depending on whether their opponent adopts the same or a different strategy. This means that, when the population generally adopts the strategy, any mutation strategy cannot bring a higher payment to its users and can therefore resist the invasion. If there is another strategy that makes payments when the mutation strategy is equal to the mainstream strategy, it must be sure that the payment in the face of the mutation must be greater than the payment when facing itself. This ensures that the strategy remains advantageous when faced with a small number of mutations.

ESS is an important concept in EGT used to describe behavioral patterns and strategy selection of individuals in a population. Figure 11 presents a specific flow chart of the evolutionary game, incorporating the recursive loop that enables continuous strategy calculation and population updates, thereby reflecting the iterative nature of strategy evolution in electricity markets.

Figure 11 presents a comprehensive flow chart that delineates the dynamic processes inherent in EGT as applied to electricity markets. This flow chart demonstrates the recursive process where, after updating the population based on replication dynamics, the system loops back to the calculation of payoffs for the next time step. This recursion ensures that strategy adaptation and population adjustments occur continuously, reflecting the dynamic and ongoing nature of electricity market interactions. This visualization encapsulates the iterative interplay between strategy adaptation, natural selection, and equilibrium attainment among market participants. Aiming at Figure 11, the flow chart components and processes are summarized as follows.

1.

Strategy Initialization:

• Population Setup: The flow chart begins with the establishment of a population of market participants, each adopting an initial strategy. These strategies may vary based on factors such as technological capabilities, regulatory compliance, and market objectives.
• Initial Strategy Distribution: the distribution of strategies within the population is depicted, highlighting the diversity or homogeneity of approaches at the outset.

2.

Interaction and Payoff Calculation:

• Strategic Interactions: participants engage in strategic interactions, where the success of each strategy is evaluated based on its performance against others.
• Payoff Matrix: The outcomes of these interactions are quantified through a payoff matrix, which assigns fitness values to strategies based on their relative success. Higher payoffs indicate more successful strategies that are likely to proliferate.

3.

Replicator Dynamics:

• Frequency Adjustment: The replicator dynamic equations govern the adjustment of strategy frequencies within the population. Strategies that yield higher payoffs relative to the population average increase in prevalence, while less successful strategies diminish.
• Mutation and Inheritance: the flow chart incorporates stochastic elements such as mutations—random strategy alterations—and inheritance mechanisms that transmit successful strategies to subsequent generations.

4.

Adaptation and Evolution:

• Adaptive Learning: Participants continuously adapt their strategies in response to environmental changes, competitor behaviors, and regulatory shifts. This adaptive learning is visualized as iterative loops within the flow chart.
• Emergent Behaviors: over successive iterations, emergent behaviors and patterns arise, reflecting the collective adaptation of the population to the evolving market landscape.

5.

Equilibrium Attainment:

• Evolutionarily Stable Strategy (ESS): The flow chart culminates in the identification of an ESS, a strategy that, once prevalent, resists invasion by alternative strategies. This signifies the stabilization of market dynamics.
• System Resilience: The attainment of ESS contributes to the overall resilience and efficiency of the electricity market, ensuring sustained equilibrium amidst ongoing strategic adaptations.

Based on the above and aiming at Figure 11, some perspectives and insights are elaborated from several aspects as follows.

(i)

Holistic Integration of Multi-Layered Adaptations: The flow chart adeptly integrates multi-layered strategic adaptations, illustrating how individual learning processes coexist with population-level evolutionary dynamics. This dual-layered approach captures the complexity of electricity markets, where micro-level decisions aggregate to macro-level market behaviors.

(ii)

Dynamic Resilience through Iterative Processes: By emphasizing iterative loops, the flow chart underscores the inherent resilience of electricity markets. Continuous adaptation and strategy evolution enable the market to withstand and recover from disruptions, such as sudden regulatory changes or technological innovations, thereby maintaining stability over time.

(iii)

Visual Representation of Complex Interdependencies: The flow chart effectively visualizes the intricate interdependencies between strategies, payoffs, and evolutionary forces. This graphical representation facilitates a deeper understanding of how specific strategies can dominate or recede based on their performance and adaptability within the competitive landscape.

(iv)

Implications for Policy and Market Design: The depiction of EGT dynamics in the flow chart provides valuable insights for policymakers and market designers. By recognizing the pathways through which certain strategies become dominant, stakeholders can devise regulatory frameworks that promote desired behaviors, mitigate market power abuses, and enhance overall market efficiency.

(v)

Enhancing Predictive Capabilities: The incorporation of replicator dynamics and stochastic elements within the flow chart enhances the predictive capabilities of the EGT model. This enables the forecasting of potential market evolutions and the identification of critical leverage points where interventions can most effectively steer market outcomes towards equilibrium.

Overall, Figure 11 serves as a pivotal illustration of evolutionary game dynamics in electricity markets, encapsulating the continuous interplay between strategic adaptation, natural selection, and equilibrium attainment. Through its detailed representation of multi-layered processes and emergent behaviors, the flow chart provides a nuanced understanding of how electricity markets evolve and stabilize over time. This visualization not only reinforces the theoretical underpinnings of EGT but also highlights its practical applicability in enhancing market resilience and informing strategic policy decisions.

Game theory studies the decision-making behavior and equilibrium issues of decision-making entities under interaction, covering basic concepts such as cooperation/non-cooperation, symmetry/asymmetry. EGT emphasizes bounded rationality, dynamic processes, and long-term trends, differing from the static analysis and assumptions of perfect rationality in classical game theory. Evolutionary games describe changes in strategy frequencies through replicator dynamic equations and pursue evolutionary stable strategies (ESS). Researchers use EGT to analyze multi-agent systems, bi-level optimization problems, risk prevention in electricity markets, and game behavior, among others. To this end,

Table 2. A summary of the advantages and disadvantages of EGT based on Refs. [34,35,36,37,38,39,40,41,42,43].

Reference	Main Research Contents	Strengths	Weaknesses
Gao et al. [34]	Interactive trading model among multiple market participants based on a hierarchical game frame in distribution network	Offers comprehensive insights into multiple market players interactions in a hierarchical game	Limited real-world data application, mostly theoretical
Liang et al. [35]	Analysis and decision-making of excess renewable energy consumption trading between electricity retailers based on an evolutionary game	Detailed analysis of renewable energy trading among electricity retailers	Focuses narrowly on renewable energy, not considering other energy forms
Wang et al. [36]	Stochastic evolutionary game model of bidding behavior for electricity purchase and sale in power market	Incorporates stochastic elements to model the uncertainty in bidding behavior	Highly complex models that might be difficult for practitioners to apply
Cheng et al. [37]	General three-population multi-strategy evolutionary games for long-term on-grid Bidding of generation-side electricity market	Multi-agent strategy modeling for long-term planning	May not address short-term market fluctuations
Cheng and Yu [38]	Nash equilibrium-based asymptotic stability analysis of multi-group asymmetric evolutionary games in typical scenarios of the electricity market	Focuses on pricing strategies in a network setting, which is highly relevant for grid management	Limited by the assumptions of network stability and rationality of agents
Xie et al. [39]	Price strategy analysis of electricity retailers based on an evolutionary game on complex networks	First to apply random evolutionary theory in power market bidding	The randomness assumption may not hold in more regulated markets
Cheng and Yu [40]	A comprehensive review of game-theoretic approaches applied to transactions in the open and ever-growing electricity markets from the perspective of power demand response	Broad overview of various game theory applications in power systems from the perspective of power demand response	Lacks depth in specific applications, more of a survey than an in-depth study
Zeng et al. [41]	Two-population asymmetric evolutionary game dynamics-based decision-making behavior analysis for a supply-side electric power bidding market	Detailed dynamic modeling of decision-making in supply-side markets	Asymmetric focus may not accurately reflect some market conditions
Wen and Heng [42]	Game theory applications in the electricity market and renewable energy trading: a critical survey	Innovatively combines EGT with renewable energy trading	Concentrates on theoretical modeling, with less emphasis on empirical validation
Abapour et al. [43]	Game theory approaches for the solution of power system problems: a comprehensive review	Comprehensive survey, great for understanding the scope of applications	As with most surveys, lacks the depth of technical specifics needed for direct application in specific cases

provides a summary of the advantages and disadvantages of EGT based on Refs. [34,35,36,37,38,39,40,41,42,43].

5.3. Stakelberg Game Theory Method

Stackelberg game theory is a strategic game in which players are divided into leaders and followers, with the leader making the first move and the follower responding accordingly [37]. This hierarchical structure allows the leader to anticipate the follower’s reactions, thereby optimizing its own strategy to maximize its utility. In the context of electricity markets, this approach is instrumental in modeling interactions where power purchasing enterprises (leaders) set strategic prices or bids, and power generation enterprises (followers) adjust their production levels in response.

Our application of the Stackelberg game theory approach uniquely integrates hierarchical decision-making with real-time market dynamics, distinguishing it from prior studies that predominantly focus on static leader–follower interactions. By incorporating dynamic elements such as renewable energy integration and demand response mechanisms, we provide a more comprehensive framework that captures the complexity of modern electricity markets. Stackelberg game theory is a strategic framework in which players are categorized as leaders and followers, with leaders making the first move and followers responding accordingly [37,44,45,46]. We utilize the Stackelberg approach to model scenarios where power purchasing enterprises (leaders) set strategic pricing or bidding parameters, and power generation enterprises (followers) adjust their production levels in response. This hierarchical structure is particularly effective in markets where dominant players can influence market conditions and set precedents that smaller competitors must follow. For example, Dong et al. [46] demonstrated the application of energy management optimization on microgrid clusters based on multi-agent systems and hierarchical Stackelberg game theory, highlighting the model’s capability to enhance the efficiency and stability of the energy management optimization of microgrid clusters. By adopting the Stackelberg model, our study captures the strategic dominance of major market players and elucidates how their initial decisions shape the overall market dynamics, thereby providing valuable insights for policymakers and market regulators. Previous studies, such as Cheng and Yu [38], have demonstrated how leaders can influence market outcomes by setting strategic prices, which followers then respond to based on their profit maximization objectives. Building on these foundational works, our analysis integrates recent advancements in renewable energy integration, as discussed by Liang et al. [35], to explore how leaders adapt their strategies in increasingly dynamic and sustainable market environments. This integration highlights the evolving nature of strategic interactions, where traditional profit motives intersect with environmental considerations, thereby offering a more nuanced understanding of market dynamics.

The core analysis tool in Stackelberg’s game is reverse induction (backward induction) [44]. The follower’s response function can be derived through its goal of maximizing profit, and the leader can predict the follower’s behavior in advance, thus considering the follower’s response in the initial decision. The mathematical formulation description of backward induction is usually related to the optimal decision in a dynamic game. The basic principle is to reverse the optimal strategy of each participant from the last step of the game and gradually deduce the equilibrium of the whole game. The following is its mathematical expression:

Suppose a game is divided into phases in which participants in each stage make choices based on previous decisions. First, from the last stage, starting, participants choose the optimal actions based on their utility function, which represents the set of decisions in the stage. In phase, participants need to know how the other participants will respond in the following phase. Therefore, he took the optimal decision in the first stage as a known condition and chose his optimal strategy in the penultimate stage. The derivation process applied to all previous stages; in each stage, the participants will, according to the subsequent stage of the optimal decision, adjust their choice through the reverse derivation, from Stage 1 to the stage of a series of optimal strategy combinations, which form the game refining Nash equilibrium (Subgame Perfect Nash Equilibrium, SPNE). Overall, the application of Stackelberg game theory in the power market is mainly reflected in the power trading, demand response, and renewable energy management.

The application of Stackelberg game theory in the power market is mainly reflected in power trading, demand response, and renewable energy management. Additionally, recent studies have extended this framework to address cybersecurity concerns within smart grids. For instance, Shan and Zhuang (2020) [47] developed a game-theoretic model to analyze the interactions between attackers and defenders across three levels of the smart grid: power plants, transmission, and distribution networks. Here are its specific applications.

5.3.1. Price Bidding for the Electricity Market

A typical application of the Stackelberg game is the problem of price bidding in the electricity market. In this model, power suppliers (such as power generation companies) are often seen as leaders, while power consumers or power retailers are followers. Suppliers set the price according to market conditions, while consumers choose the time and quantity of electricity according to the suppliers pricing strategy. This model can better reflect the game process of the market participants, so as to optimize the market efficiency [38].

5.3.2. The Game Between the Dominant Players and the Competitors in the Power Generation Market

In the electricity market, one or a few large power companies usually dominate the market, with other power companies as followers. This structure can be described in the Stackelberg model, in which the leader chooses their own generation (or quotation), while other companies adjust their own generation or quotation according to the leader’s decision. The model helps analyze how leading companies influence market prices through their decisions and how other companies respond [39]. For example, large power generation companies can set appropriate production levels to benefit their electricity prices, while small power generation companies are forced to adjust their production strategies based on the decisions of large companies. This situation is more common in oligopoly competition in the electricity market.

5.3.3. Game in Demand Response Management

Stackelberg game theory is also applied to demand response management. Under the power demand response mechanism, power companies (leaders) provide a certain incentive or price mechanism to influence the electricity consumption behavior of users (followers). Power companies can predict users’ responses to different prices, and design reasonable price policies or incentive mechanisms to induce users to reduce their electricity consumption during peak grid load periods [40]. Here, the power company first puts forward the price mechanism, in which the users, as the followers, adjust the electricity consumption mode according to the price signal. Through the game model, power companies can make better decisions between balancing load and economic benefits.

5.3.4. Cooperation and Competition Between Power Generation Companies and Grid Operators

In the electricity market, there is a complex game relationship between power generation companies and grid operators (e.g., ISO). Grid operators manage the overall operation of the electricity market, ensuring a balance between supply and demand, while power generation companies hope to make maximum profits from generating electricity. The Stackelberg game can be used to model such relationships, where grid operators may be leaders, first setting market rules or power procurement plans, while generation companies make generation decisions based on these rules. The strategies of grid operators will affect the market price, capacity planning, and the enthusiasm of power generation companies to participate in the market, so the interaction between both sides in the complex market can be better understood through the game theory model.

5.3.5. The Game Between Renewable Energy Power Generators and Traditional Power Generators

In the modern power market, the relationship between renewable energy generators (such as wind and photovoltaic) and traditional thermal or nuclear power plants can also be analyzed through the Stackelberg game. Due to the different cost structure of renewable energy and the strong market competitiveness, traditional power producers often need to adjust their power generation strategies according to the power generation situation of renewable energy. In this game, renewable energy power producers may be seen as leaders, because their power generation usually depends on uncontrollable factors such as weather, but it is “determined first”, while traditional power producers are followers who adjust their power generation plans according to the supply of renewable energy [41].

Table 3. Summary of advantages and weaknesses of Stackelberg game theory in electricity markets based on Refs. [45,46,48,49,50,51,52,53,54,55].

References	Title	Strengths	Weaknesses
Chen et al. [45]	Optimal operation between electric power aggregator and electric vehicle based on Stackelberg game model	Provides a robust model for coordinating electric vehicles and grid operations	Limited consideration of the large-scale, real-time impacts on the grid system
Dong et al. [46]	Energy management optimization of microgrid clusters based on multi-agent systems and hierarchical Stackelberg game theory	Demonstrates strong hierarchical energy management through multi-agent coordination	Complex model may be challenging to implement in real-world scenarios with diverse participants
Lu et al. [48]	A Nash–Stackelberg game approach in a regional energy market considering users’ integrated demand response	Effectively integrates uncertainty in power distribution systems	Heavy reliance on data-driven models, which may limit applicability in data-scarce regions
Zhang et al. [49]	A data-driven Stackelberg game approach applied to analysis of strategic bidding for distributed energy resource aggregators in electricity markets	Provides an innovative solution for optimizing battery swapping station locations using data-driven methods	Focused narrowly on one aspect of power systems—battery swapping—and lacks general applicability
Zhang and Wu [50]	Collaborative allocation model and balanced interaction strategy of multi-flexible resources in the new power system based on Stackelberg game theory	Addresses real-time frequency regulation issues by utilizing electric vehicle clusters	Does not sufficiently consider long-term battery degradation in EV clusters
Xie et al. [51]	Multiplayer Nash–Stackelberg game analysis of electricity markets with the participation of a distribution company	Uses Stackelberg game to achieve multi-agent investment optimization in microgrid systems	Does not incorporate dynamic pricing models or real-time market conditions
Bo S et al. [52]	An incentive mechanism to promote residential renewable energy consumption in China’s electricity retail market: a two-level Stackelberg game approach	Presents a comprehensive two-layer scheduling model for microgrid optimization	Computational complexity is high, which may affect scalability
Ma et al. [53]	Impact of carbon emission trading and renewable energy development policy on the sustainability of electricity market: a Stackelberg game analysis	Effectively integrates pricing strategies for photovoltaic user groups	Limited to photovoltaic systems and lacks general energy application coverage
Li et al. [54]	An aggregator-oriented hierarchical market mechanism for multi-type ancillary service provision based on the two-loop Stackelberg game	A solid framework for integrating wind energy storage with time-of-use pricing strategies	Focuses primarily on wind energy and does not account for other renewable sources
Özge and Ümmühan [55]	A Stackelberg game-based dynamic pricing and robust optimization strategy for microgrid operations	Excellent focus on EV charging optimization to reduce power loss	Assumes orderly charging conditions, which may not always reflect real-world chaos in EV charging systems

provides a summary of the advantages and weaknesses of Stackelberg game theory in electricity markets based on Refs. [45,46,48,49,50,51,52,53,54,55].

5.3.6. Cybersecurity in Smart Grids

In addition to the strategic interactions between renewable and traditional power generators, the integration of smart grid technologies introduces a critical dimension to the electricity market: cybersecurity. For instance, Shan and Zhuang (2020) [47] employed a game-theoretic approach to model cyber-attacks and defenses across three pivotal levels of the smart grid: power plants, transmission networks, and distribution networks. Their study expands the application of game theory beyond economic and operational strategies to encompass the security and resilience of smart grid infrastructures. Currently, cybersecurity in smart grids serves as a pivotal component in strategic evolution in electricity markets. This section integrates a crucial dimension that intersects with the strategic interactions and evolutionary dynamics previously discussed. Below are the key roles that cybersecurity plays within game theory-based strategic evolution in electricity markets:

• Integration of Technological Advancements and Market Dynamics

Modern electricity markets are increasingly characterized by the integration of advanced technologies such as smart grids, which incorporate information and communication technologies (ICTs) to enhance the efficiency, reliability, and sustainability of power systems. Cybersecurity in smart grids addresses the vulnerabilities introduced by this digitalization, ensuring that technological advancements do not compromise the integrity and stability of the electricity market. By modeling cybersecurity threats and defenses through game-theoretic approaches, we can highlight how strategic interactions among market participants extend beyond traditional economic and operational domains to encompass security considerations.

2.

Enhancing Market Resilience and Stability

The resilience and stability of electricity markets are paramount, especially as they transition from centralized, monopolistic structures to more decentralized and competitive environments. Cyber-attacks on smart grids can lead to significant disruptions, affecting power generation, transmission, and distribution. By incorporating cybersecurity in smart grids into game theory-based strategic evolution in electricity markets, we can underscore the importance of safeguarding market operations against malicious disruptions. This integration ensures that the strategic models account for both economic incentives and security imperatives, thereby providing a more holistic understanding of market resilience.

3.

Strategic Interactions Between Attackers and Defenders

Game theory provides a robust framework for modeling the strategic interactions between attackers and defenders within smart grids. Shan and Zhuang (2020) [47] exemplified this by developing a multi-level game model that captures the complexities of cybersecurity in power systems. This work illustrates how attackers (e.g., cybercriminals or state actors) and defenders (e.g., utility companies or regulatory bodies) engage in strategic maneuvers to optimize their respective objectives—disrupting operations versus securing infrastructure. By analyzing these interactions, we can reveal how defensive strategies can be dynamically adjusted in response to evolving threats, thereby influencing overall market behavior and stability.

4.

Influence on Market Structure and Resource Allocation

Cybersecurity considerations significantly impact market structure and resource allocation within electricity markets. Effective defensive strategies can deter potential attackers, thereby maintaining the integrity of market transactions and preventing unfair advantages that may arise from successful cyber-attacks. We can link cybersecurity with the broader themes of market segmentation and stakeholder dynamics, demonstrating how secure operations contribute to fair competition and optimal resource distribution. The inclusion of cybersecurity ensures that the strategic evolution of market participants accounts for both competitive and security-driven incentives.

5.

Policy Implications and Regulatory Frameworks

The intersection of cybersecurity and electricity markets necessitates the development of comprehensive policy frameworks that address both market efficiency and security. Cybersecurity in smart grids informs policy recommendations by highlighting the need for regulations that promote secure market operations, encourage investment in defensive technologies, and foster collaboration between different stakeholders. For instance, Ref. [47] provides valuable insights for policymakers aiming to balance economic growth with the imperative of maintaining robust security measures, thereby enhancing the overall governance of energy systems.

6.

Facilitating Advanced Research and Technological Innovation

Incorporating cybersecurity into the game-theoretic analysis of electricity markets paves the way for advanced research and technological innovation. It encourages the exploration of novel defensive mechanisms, the development of resilient infrastructure, and the application of interdisciplinary approaches that combine economics, engineering, and computer science. This holistic perspective not only enriches the academic discourse but also drives practical advancements that enhance the sustainability and reliability of electricity markets.

Based on the above, for example, researchers in Ref. [47] constructed a hierarchical game model where attackers and defenders operate at multiple levels, reflecting the complex interdependencies within the smart grid. At the power plant level, attackers may target generation units to disrupt electricity production, while defenders allocate resources to secure critical infrastructure against such intrusions. In the transmission network, the focus shifts to safeguarding high-voltage lines and substations from cyber threats that could cause widespread outages. Finally, at the distribution network level, the game addresses the protection of lower-voltage lines and consumer endpoints, ensuring that localized attacks do not cascade into broader system failures.

Shan and Zhuang’s (2020) model reveals that defenders must consider not only direct attacks but also the potential for threat propagation across interconnected networks [47]. This multi-level defense strategy underscores the necessity of a coordinated approach to cybersecurity, where protective measures in one segment of the grid influence and enhance the security posture of other segments. The equilibrium strategies derived from their model demonstrate that effective defense mechanisms are contingent upon the dynamic allocation of resources in response to evolving attack patterns, emphasizing the importance of adaptive and resilient cybersecurity frameworks in smart grids. Furthermore, their research highlights that the defender’s optimal strategy is significantly influenced by the interconnected nature of modern power systems [47]. An attack at the power plant level can have cascading effects, necessitating robust defensive measures that account for indirect vulnerabilities. This insight aligns with the broader theme of strategic adaptation observed in traditional and renewable power generation interactions, where market participants must continuously evolve their strategies in response to external and internal changes.

By incorporating cybersecurity considerations into the Stackelberg game framework, Shan and Zhuang (2020) [47] provided a comprehensive view of how strategic interactions extend into the realm of digital threats in smart grids. Their work not only augments the existing literature on game-theoretic applications in electricity markets but also bridges the gap between market strategy and infrastructure security, offering novel insights into the protection and resilience of next-generation power systems.

In summary, the integration of cybersecurity into game-theoretic models for electricity markets represents a significant advancement in understanding the multifaceted challenges faced by modern power systems. The work of Shan and Zhuang (2020) [47] exemplifies how game theory can be effectively utilized to address both economic and security dimensions, ensuring that electricity markets remain robust and reliable in the face of emerging cyber threats. This addition to the Stackelberg game theory method underscores the necessity of holistic strategic planning, where market dynamics and cybersecurity measures are interwoven to achieve optimal outcomes for all stakeholders involved. In the future, cybersecurity in smart grids will be able to play a critical role in game theory-based strategic evolution in electricity markets by bridging the gap between strategic economic interactions and the imperative of securing modern power systems. This integration ensures that the analysis remains comprehensive, addressing both the competitive dynamics and the security challenges that define contemporary electricity markets. By embedding cybersecurity within the framework of game theory, we can provide a nuanced understanding of how market participants can strategically navigate both economic incentives and security threats, ultimately contributing to the creation of more resilient, efficient, and equitable electricity markets.

6. Decision-Making Under Uncertainty: Bayesian Game Theory in Electricity Markets

6.1. The Bayesian Game Theory

Bayesian game theory addresses strategic interactions in settings characterized by incomplete information, where players possess uncertain beliefs about other players’ types or private information [56,57,58,59,60,61,62,63]. In the context of electricity markets, we apply Bayesian games to model scenarios where power purchasing enterprises and generation enterprises operate under information asymmetry regarding each other’s cost structures and strategic intentions. This framework allows market participants to form probabilistic beliefs and update their strategies based on observed actions and new information, thereby enhancing their decision-making processes. Verma et al. [63] employed Bayesian Nash equilibrium to analyze bidding strategies under uncertainty, illustrating the model’s effectiveness in capturing realistic market behaviors. By integrating Bayesian game theory, our study provides a robust mechanism for understanding and mitigating the impacts of information asymmetry, thereby fostering more transparent and efficient market operations. In electricity markets, Bayesian game theory is essential for analyzing strategic bidding and decision-making under conditions of information asymmetry, allowing participants to formulate optimal strategies despite uncertainty about competitors’ costs and intentions.

Building upon traditional Bayesian game theory, our analysis incorporates real-time data assimilation and predictive analytics to dynamically update participants’ beliefs, thereby enhancing the robustness of strategic decision-making in volatile electricity markets. This innovative approach allows for more accurate forecasting of competitor behaviors and market trends, providing a significant advancement over existing models that rely on static probability distributions [56,57,58,59,60,61,62,63]. Bayesian game theory is instrumental in addressing the complexities of incomplete information within electricity markets, where participants often lack full visibility into competitors’ cost structures and strategic intentions. By introducing probability distributions and employing Bayesian update rules, this framework allows for dynamic strategy adjustments based on newly acquired information [57,58,59]. Yu et al. [58] have applied Bayesian game models to analyze bidding behaviors and market clearing for renewable energy sources under uncertainty, illustrating how firms revise their strategies in response to market signals and competitor actions. Our review builds upon these studies by synthesizing insights from Wang et al. [36] and Verma et al. [63], who incorporate stochastic elements and multi-agent interactions into Bayesian frameworks. This synthesis not only underscores the versatility of Bayesian game theory in modeling diverse market scenarios but also identifies key areas where current models can be enhanced to better reflect real-world market dynamics, such as integrating predictive analytics and machine learning techniques for more accurate belief updates. These insights align with Verma et al. [63], who explore the implications of stochastic elements in Bayesian frameworks, collectively illustrating the multifaceted nature of decision-making under uncertainty in electricity markets.

The application of Bayesian game theory in the electricity market is mainly reflected in handling the incomplete information and uncertainty among the market participants. These markets are often a complex system of multiple stakeholders, including power producers, power carriers, electricity traders, and regulators. Each participant has an incomplete understanding of key information, such as the cost structure, power generation capacity, and market demand of the other participants, so the Bayesian game provides an analytical framework for decision-making under this asymmetric information.

6.2. Market Bidding Mechanism

In the electricity market, generators usually sell electricity through bidding. This is a typical Bayesian game scenario because each power producer does not fully know about the cost information of other power producers. Power producers must choose strategies according to their beliefs about the competitor’s cost distribution (i.e., prior probability), and update these beliefs during the bidding process so as to adjust the strategies for the next round of strategies. This bidding process can be analyzed by a Bayesian Nash equilibrium, enabling each generator to find the optimal bidding strategy in the face of uncertainty [42].

6.3. Power Generation and Demand Forecasting

The uncertainty in the electricity market comes not only from the strategies of other market participants, but also from the fluctuations in market demand and power generation. Market participants can use Bayesian game theory to predict how competitors adjust their power generation and bid decisions under different market conditions. In addition, power demand and supply are often random, and market participants can adjust the forecast of future supply and demand situations according to the actual market performance [43].

6.4. Market Design and Regulation

Regulators of the electricity market can use Bayesian game theory to design incentive mechanisms to deal with information asymmetry problems. For example, regulators may not be aware of the true cost or generating capacity of individual generators but can induce generators to report real information when bidding by designing reasonable market rules and auction mechanisms. By analyzing the strategic choices of different power producers, regulators can improve market efficiency, ensure fair competition, and reduce the likelihood of market manipulation [43].

Table 4. Overview of electricity market applications for Bayesian game theory based on Refs. [56,57,58,59,60,61,62,63].

References	Title	Advantages	Disadvantages
Huang et al. [56]	Optimal operation of electricity sales company with multiple VPPs based on a Stackelberg game	Comprehensive use of a Stackelberg game for optimizing VPP and sales strategies	Limited scalability in real-world applications due to oversimplified models
Zhu et al. [57]	An imitation learning-based algorithm enabling a priori knowledge transfer in modern electricity markets for Bayesian Nash equilibrium estimation	Fast convergence and accurate strategy optimization with prior knowledge transfer	Slow convergence for real-time applications with high volatility
Yu et al. [58]	A reinforcement–probability Bayesian approach for strategic bidding and market clearing for renewable energy sources with uncertainty	Handles uncertainty effectively with Bayesian game models	High computational complexity
Fang et al. [59]	Bayesian Nash equilibrium bidding strategies for generation companies	Improves bidding strategies with incomplete information	Assumes perfect rationality, which may not hold in real-world scenarios
Li and Shahidehpour [60]	Strategic bidding of transmission-constrained GENCOs with incomplete information	Improves GENCOs’ bidding strategies by transforming incomplete games into complete games	The complexity of the bilevel problem and the reliance on sensitivity functions for payoff calculations may limit the method’s scalability and applicability in more complex or larger market systems
Xu et al. [61]	Optimal scheduling method of multi-energy hub systems based on Bayesian game theory	Decouples thermoelectric systems for better efficiency	Increased risk from incomplete information in decision-making
Zidan and Gabbar [62]	Optimal scheduling of energy hubs in interconnected multi-energy systems	Optimizes energy usage and reduces operational costs	Increased risk from relying on incomplete information
Verma et al. [63]	Bayesian Nash equilibrium in electricity spot markets: an affine-plane approximation approach	Reduces market power concentration and price spikes	Increased computational complexity with multiple iterations

provides an overview of the electricity market for Bayesian game theory based on Refs. [56,57,58,59,60,61,62,63]. In Table 4, as studied in Ref. [59], the Bayesian Nash equilibrium (BNE) extends the concept of the Nash equilibrium to Bayesian games, where each player’s strategy is optimal given their beliefs about other players’ types [59]. In the context of electricity markets, BNE bidding strategies for generation companies ensure that each company’s bid is the best response to the expected bids of competitors, considering the uncertainty in competitors’ cost structures and strategies.

7. Enhancing Market Behavior Modeling: Integration of DRL with Game Theory and Analysis of Information Asymmetry and Market Power

7.1. Integration of EGT and DRL

Through the integration of EGT, Stackelberg games, Bayesian games, and DRL, our paper presents a comprehensive and multifaceted framework for understanding and optimizing strategic interactions in electricity markets. This holistic approach allows for the modeling of dynamic strategic behaviors, adaptation to market fluctuations, and the optimization of decision-making processes, thereby enhancing the overall efficiency and resilience of electricity markets. By leveraging the synergistic strengths of these methodologies, our framework provides deeper insights into both individual and collective market behaviors, facilitating more informed policy-making and strategic planning.

Specifically, the inclusion of EGT allows for the representation of evolving strategies and collective adaptation, which is critical for capturing the long-term dynamics of electricity markets. Stackelberg and Bayesian games complement this by addressing hierarchical decision-making and information asymmetry, respectively. DRL further enhances the framework by enabling real-time strategy optimization and adaptive learning, ensuring that market participants can respond swiftly to changing conditions. This integrated framework not only provides a more accurate depiction of market behaviors but also offers practical tools for optimizing market operations and designing effective policies.

Incorporating Bayesian Game Theory within this framework specifically addresses how information asymmetry can be managed, enabling more realistic and robust market models. By combining Bayesian games with EGT, we can analyze how information uncertainties impact strategic decision-making and market outcomes over time. Furthermore, integrating DRL with EGT allows for the development of adaptive strategies that evolve in response to both information dynamics and competitive pressures, providing a comprehensive tool for market analysis and optimization.

This multifaceted approach ensures that our framework is well-equipped to handle the complexities of electricity markets, offering both theoretical depth and practical applicability. The synergy between game-theoretic models and machine learning techniques enhances our ability to predict market trends, optimize strategies, and inform policy decisions, thereby contributing to the advancement of energy governance and market design.

Therefore, integrating EGT with DRL offers a robust framework for modeling and optimizing strategic interactions in electricity markets. EGT provides insights into how strategies evolve within populations of players, capturing the dynamic adaptation processes driven by competitive and cooperative behaviors. DRL, on the other hand, equips individual agents with the capability to learn and optimize their strategies through continuous interaction with the environment [64]. This integration enables a comprehensive analysis where EGT models the macro-level strategic shifts, while DRL facilitates micro-level strategy optimization for individual market participants.

One of the primary points of contact between EGT and DRL lies in their mutual emphasis on adaptation and optimization. EGT models the evolutionary dynamics of strategies within a population, highlighting how successful strategies proliferate over time. In contrast, DRL enables individual agents to optimize their strategies based on trial-and-error learning and feedback from the environment. By combining these approaches, we can model electricity markets where both population-level strategy evolution and individual agent optimization coexist, providing a more holistic understanding of market dynamics.

However, the integration of EGT and DRL also presents critical issues. Firstly, the computational complexity increases significantly, as the combined framework must account for both the evolutionary processes of EGT and the iterative learning processes of DRL. Secondly, ensuring the stability and convergence of the integrated models can be challenging, particularly in highly volatile market environments. To address these challenges, advanced computational techniques and hybrid modeling approaches are required, which can effectively balance the individual and collective aspects of strategic interactions. Critical issues in this integration include the alignment of EGT’s population dynamics with DRL’s individual learning processes and managing the computational complexity that arises from modeling large-scale interactions. Addressing these challenges requires the development of hybrid models that can seamlessly incorporate the adaptive learning capabilities of DRL within the strategic evolution framework of EGT. Our study explores these integration points, proposing a framework that leverages the strengths of both methodologies to enhance market efficiency and resilience. For example, Gao et al. [34] demonstrated that integrating DRL with EGT can improve the predictive accuracy of strategy evolution in renewable energy markets, enabling more responsive and adaptive market behaviors.

Despite these challenges, the integration of EGT and DRL holds significant promise for advancing the analysis and optimization of electricity markets. By leveraging the strengths of both methodologies, this integrated framework can provide deeper insights into the adaptive behaviors of market participants and the emergent properties of market systems, ultimately contributing to more resilient and efficient energy markets.

Based on the elaborations above, the integration points between EGT and DRL are summarized in this section, as shown in

Table 5. Integration points between EGT and DRL.

Integration Aspect	Evolutionary Game Theory (EGT)	Deep Reinforcement Learning (DRL)	Points of Contact and Critical Issues
Strategic Adaptation	Models the evolution of strategies based on population dynamics and the success of past actions [43].	Enables individual agents to optimize strategies through continuous learning and interaction with the environment [64].	Both EGT and DRL focus on adaptation, but EGT operates at the population level while DRL focuses on individual optimization. Integrating them requires aligning population dynamics with individual learning processes.
Decision-Making Processes	Emphasizes how strategies evolve over time within a population, leading to emergent behaviors and stable equilibria [34].	Facilitates real-time strategy optimization for individual agents based on feedback and rewards.	Combining EGT’s macro-level strategy evolution with DRL’s micro-level decision-making necessitates hybrid models that can manage the interplay between collective and individual behaviors.
Modeling Complexity	Captures the dynamic and adaptive nature of markets but may oversimplify individual agent behaviors [43].	Handles high-dimensional and complex decision spaces, allowing for sophisticated strategy development.	The integration increases computational complexity, requiring advanced algorithms and computational resources to effectively model large-scale interactions and ensure stable learning outcomes.
Market Dynamics	Provides insights into long-term strategic shifts and cooperation/competition dynamics within populations [34].	Offers adaptive and responsive strategies that can adjust to immediate market changes and competitor actions.	Integrating EGT and DRL can lead to a more comprehensive understanding of both long-term and short-term market dynamics, enhancing the ability to predict and respond to market fluctuations.
Applications in Renewable Energy	Models the strategic evolution of renewable energy adoption and bidding behaviors among market participants [34].	Optimizes real-time bidding and operational strategies for renewable energy producers based on continuous learning.	The combination allows for the simultaneous consideration of strategic evolution and real-time optimization, providing a robust framework for managing the integration of renewable energy sources into electricity markets.

.

7.2. Integration of DRL with Game Theory for Enhancing Market Behavior Modeling

The integration of DRL with EGT represents a significant methodological advancement for strategy optimization in electricity markets. DRL, which combines reinforcement learning with deep neural networks, enables agents to learn optimal strategies through interactions with complex and high-dimensional environments [64]. When combined with EGT, which models the dynamic adaptation of strategies among populations of players, DRL facilitates a more nuanced understanding of how individual learning processes influence collective market behaviors. For instance, Li et al. [31] utilized DRL to develop adaptive bidding strategies that outperform traditional game-theoretic models under volatile market scenarios, and Gao et al. [34] demonstrated that integrating DRL with EGT can enhance the predictive accuracy of strategy evolution in renewable energy markets by allowing agents to adapt their behaviors based on both historical successes and real-time market feedback. This synergistic approach not only improves the adaptive capabilities of market participants but also provides deeper insights into the emergent dynamics that drive market efficiency and resilience. However, integrating DRL with EGT presents critical challenges, such as ensuring the stability of learning algorithms and effectively managing the computational complexity inherent in modeling large-scale market interactions. Addressing these issues requires the development of hybrid frameworks that leverage the strengths of both methodologies while mitigating their individual limitations. Our study explores these integration points, offering a comprehensive framework that enhances strategic decision-making and promotes sustainable market practices.

In our study, the combination of DRL and game-theoretic approaches provides powerful tools for strategy optimization in power markets and other complex decision environments [64]. This integration allows for the development of adaptive and intelligent strategies that can respond to real-time market dynamics and competitive behaviors, thereby enhancing the decision-making capabilities of market participants. Moreover, this combination leverages the adaptive learning capabilities of DRL to complement the strategic interaction models of game theory, enabling more robust and dynamic decision-making processes. For example, Li et al. [31] utilize DRL to enhance market power monitoring models, while Bellegarda et al. [65] apply DRL to develop robust strategies for cyber defense in energy systems. These applications demonstrate the potential of combining DRL with game theory to address multifaceted challenges in electricity markets, thereby providing original insights into the optimization of market behaviors and enhancing the overall stability and efficiency of energy systems.

Our integration of DRL with game-theoretic approaches represents a novel methodological advancement, enabling adaptive strategy optimization that responds to real-time market fluctuations and multi-agent interactions. Unlike traditional models, this hybrid framework leverages the predictive capabilities of DRL to enhance the strategic flexibility and resilience of market participants, offering deeper insights into the emergent behaviors within electricity markets. The integration of DRL with game-theoretic approaches represents a significant methodological advancement in modeling strategic interactions within electricity markets. This hybrid framework leverages the adaptive learning capabilities of DRL to enhance the predictive accuracy and strategic flexibility of game-theoretic models [64,65,66,67,68,69,70]. For instance, Li et al. [31] have demonstrated how DRL can be employed to optimize bidding strategies in spot markets by continuously learning from market fluctuations and competitor behaviors. Building on this, Bellegarda et al. [65] have utilized DRL to develop robust defense strategies in cyber-physical energy systems, showcasing the versatility of this integration. This approach synergizes with the findings of Han et al. [69], who emphasize the importance of adaptive learning in multi-agent systems, thereby providing a comprehensive view of how DRL enhances game-theoretic models in practical applications. Our review further explores how combining DRL with multi-agent game models, as proposed by Han et al. [69], can lead to the emergence of stable Nash equilibria and promote cooperative behaviors among market participants. This seamless fusion of DRL and game theory not only facilitates real-time strategy optimization but also provides deeper insights into the adaptive mechanisms that drive market efficiency and resilience. Through deep neural networks, DRL can process high-dimensional inputs and continuously optimize the decision path through simulations.

In the power market, our integration of DRL with game-theoretic approaches introduces a novel paradigm for addressing complex strategic interactions and adaptive decision-making. This method is extensively applied in demand response management, load scheduling, price bidding, and other scenarios. For instance, power companies can leverage DRL-enhanced strategies to dynamically optimize generation in response to volatile market prices, while power purchasing companies can utilize adaptive DRL models to refine their purchasing behaviors based on real-time data and evolving market trends. For example, power companies can leverage this integrated DRL and game-theoretic method to continuously learn and adapt their generation strategies in response to real-time market price fluctuations and competitor actions, as demonstrated by Li et al. [31]. Concurrently, power purchasing companies can utilize adaptive DRL models to refine their purchasing behaviors based on comprehensive analyses of historical data and emerging market trends, thereby enhancing their strategic decision-making capabilities. This dynamic interplay between DRL and game theory fosters a more responsive and competitive market environment, aligning with the findings of Bellegarda et al. [65] and Han et al. [69]. In this process, DRL can maximize the long-term benefits by interacting with the environment and adversary simulation.

Furthermore, our integration of DRL with multi-agent game-theoretic models introduces an advanced framework for simulating and optimizing interactions in highly competitive and multi-faceted power markets. This integration enables the modeling of complex, interdependent strategies among numerous market participants, facilitating the identification of stable Nash equilibria and promoting cooperative behaviors that enhance overall market efficiency. Multi-agent systems (MAS), in the context of reinforcement learning, involve multiple autonomous agents interacting within a shared environment, each pursuing their own objectives while potentially cooperating or competing with others [31]. DRL techniques can be effectively integrated with MAS to enable agents to learn and adapt their strategies in dynamic and competitive electricity markets. This integration is particularly suitable for power markets with numerous participants, as it facilitates the emergence of stable Nash equilibria and promotes cooperative behaviors that enhance overall market efficiency. Through mutual learning and competition, the agents can gradually form a stable Nash equilibrium, thus optimizing the stability and efficiency of the overall system. To this end,

Table 6. Advantages and disadvantages of deep learning applications in energy markets based on Refs. [31,65,66,67,68,69,70,71,72,73].

References	Research Theme	Advantages	Disadvantages
Li et al. [31]	Market power monitoring model of electricity retailer in retail market under spot market mode	Optimizes network topology for both single- and multi-node faults, enhancing grid stability	Requires significant computational resources for large-scale systems
Bellegarda et al. [65]	Robust quadruped jumping via DRL	Combines game theory with DRL for advanced cyber defense strategies	Relies heavily on assumed rationality, which may not hold in real-world scenarios
Noh and Swidinsky [66]	Stochastic control of geological carbon storage operations using geophysical monitoring and DRL	Provides a comprehensive overview of control strategies for microgrids	Limited focus on fault tolerance in rapidly evolving grid conditions
Wang et al. [67]	Safe DRL for building energy management	Adapts power distribution based on real-time data from renewable sources	Limited by fluctuating availability of renewable energy sources
Zhang et al. [68]	A latent space method with maximum entropy DRL for data assimilation	Effectively manages microgrid resources with minimal human intervention	Challenges in scaling the approach to large and complex grid networks
Han et al. [69]	Network defense decision-making based on DRL and dynamic game theory	Balances between energy trading and grid stability	Limited empirical validation in real-world scenarios
Xia et al. [70]	Physical model-assisted DRL for energy management optimization of an industrial electric–hydrogen coupling system with hybrid energy storage	Enhances fault tolerance through optimized grid structures	Complexity increases significantly with grid size
Noriega et al. [71]	DRL-based real-time open-pit mining truck dispatching system	Utilizes AI for proactive threat detection and mitigation	Limited focus on physical infrastructure vulnerabilities
Ding et al. [72]	A DRL optimization method considering network node failures	Provides insights into market behavior with a high penetration of renewable energy	Fails to address the impact of extreme weather conditions on energy supply
Qiu et al. [73]	Bridge bidding via DRL and belief Monte Carlo search	Applies machine learning techniques to optimize grid data analysis	Data privacy concerns remain a challenge

summarizes the advantages and disadvantages of the research work on deep learning based on Refs. [31,65,66,67,68,69,70,71,72,73].

In summary, evolutionary games, master-slave games, and Bayesian games all provide different theoretical perspectives for the complex interaction between power purchasing enterprises and power generation enterprises.

Table 7. Comparison of key theories for strategic interactions in energy markets, including EGT, Stackelberg game theory, Bayesian game theory, and DRL.

Domain	Definition	Merit	Shortcoming
Evolutionary Game Theory (EGT)	Studies the strategic interactions among populations of players who adapt their strategies over time based on the success of past actions [43].	Captures dynamic adaptation and strategy evolution in long-term market interactions, applicable to understanding cooperation and competition in evolving energy landscapes.	Assumes simplified behavior patterns that may not fully encapsulate the complexities of real-world strategic decisions.
Stackelberg Game Theory	Models hierarchical strategic interactions where leaders make the first move and followers respond accordingly [37].	Facilitates the analysis of strategic interactions between dominant and subordinate market players, enhancing the understanding of power dynamics and competitive strategies in energy markets. With a strong application in the energy market, supply chain management, and other fields, it can analyze the interaction of upstream and downstream decision-making.	Limited applicability in environments with significant information asymmetry and rapid market changes, potentially oversimplifying real-world complexities. It is suitable for the transparent environment; if the information is asymmetric, the effect of game analysis is limited.
Bayesian Game Theory	Addresses strategic interactions under incomplete information, where players have uncertain beliefs about other players’ types or private information [59].	Effectively models decision-making under uncertainty and information asymmetry, enabling more realistic and informed strategic planning in competitive markets. To solve the problem of information asymmetry, it is widely used in economics, auction theory, and other fields.	Complex computational requirements and reliance on accurate probability assessments, which may be challenging in highly volatile or unpredictable market conditions.
Deep Reinforcement Learning (DRL)	A machine learning technique that combines reinforcement learning with deep neural networks, enabling agents to learn optimal strategies through interactions with complex environments [64].	Enhances the ability to handle high-dimensional data and complex decision-making processes, enabling adaptive and real-time strategy optimization in dynamic market conditions. It can process a large number of complex and unstructured data, and performs well in image recognition, natural language processing, autonomous driving, and other fields.	Requires substantial computational resources and extensive training data, with potential issues related to model interpretability and overfitting.

shows the respective advantages and disadvantages statistics.

By juxtaposing evolutionary games, Stackelberg games, and Bayesian games, our study provides a comparative analysis that highlights the strengths and limitations of each approach in capturing different facets of electricity market dynamics. This comparative framework offers original insights into how these game-theoretic models can be synergistically applied to address multifaceted challenges such as market power, information asymmetry, and strategic adaptability. A comparative analysis of Stackelberg, Bayesian, and evolutionary game theories reveals distinct strengths and optimal application scenarios for each model within electricity markets. Stackelberg games excel in scenarios where hierarchical decision-making structures are prevalent, allowing dominant market players to influence outcomes through strategic leadership [37,38]. Bayesian games are particularly effective in environments characterized by significant information asymmetry, enabling market participants to make informed decisions despite uncertainties about competitors’ strategies and cost structures [59,63]. EGT, on the other hand, is well-suited for modeling dynamic adaptation processes and the gradual evolution of strategies in response to changing market conditions [34,43]. By leveraging the unique advantages of each game-theoretic approach, our study provides a nuanced framework for analyzing and optimizing strategic interactions in diverse electricity market scenarios. This comparative perspective ensures that the selection of appropriate models is contextually driven, enhancing the overall robustness and applicability of our analysis. Through the application of deep learning, the problems of supply and demand balance, price prediction, and risk management in the power market have been effectively solved, promoting the intelligent and efficient operation of the power system. In practice, these game models can help market participants understand the motivation behind the other side’s behavior, so as to develop more effective market strategies.

7.3. Analysis of Information Asymmetry and Market Power

Information asymmetry significantly influences strategic interactions and market outcomes in electricity markets. To address information asymmetry, we employ Bayesian game theory, which allows market participants to form probabilistic beliefs about their competitors’ strategies and private information [59]. This approach enables enterprises to make informed decisions even when they lack complete knowledge of their rivals’ cost structures and strategic intentions. By incorporating Bayesian games, our model accounts for the uncertainties inherent in electricity markets, providing a more realistic representation of strategic decision-making processes.

Additionally, market power—the ability of a firm to influence the price of electricity—poses a critical challenge in maintaining competitive market structures. Our study utilizes EGT to examine how market power dynamics evolve over time. EGT allows us to model the strategic adaptations of dominant and non-dominant firms, exploring scenarios where dominant players can sustain their market power through adaptive strategies while smaller firms adjust their behaviors to counteract these influences [43]. This dynamic analysis is essential for understanding the long-term implications of market power on electricity market stability and efficiency.

Furthermore, the interplay between information asymmetry and market power is pivotal in shaping market dynamics. Dominant firms may exploit information advantages to reinforce their market power, while information asymmetry can hinder the ability of smaller firms to compete effectively. By integrating Bayesian game theory with EGT, our framework captures both the micro-level information-driven strategies and the macro-level strategic adaptations, providing a comprehensive analysis of how information asymmetry and market power interact to influence electricity market outcomes.

Our findings indicate that addressing information asymmetry through regulatory measures, such as enhanced transparency and data sharing, can mitigate the adverse effects of market power, fostering a more competitive and efficient electricity market. These insights have significant policy implications, as they inform the design of regulatory frameworks aimed at reducing information disparities and curbing the potential for market manipulation by dominant players.

8. The Transaction Mode Between Power Purchasing Enterprises and Power Generation Enterprises

As demonstrated in Figure 12, it shows the structure of the regional power market and the relationship between different market mechanisms. Based on this, the power market is mainly divided into three categories: medium- and long-term markets, spot markets, and ancillary services markets [8,40,74]. Among them, the medium- and long-term markets include inter-provincial and intra-provincial medium- and long-term contracts to balance supply and demand through “Internet to Internet” and “point to point” price expectations and to lock the price of most electricity; the spot market includes a day-ahead market and real-time market and daily market bidding to generate price signals; and the auxiliary services market encompasses the reserve market and peak reduction market, which are critical for ensuring grid reliability by providing backup power during surpluses and shortages. Reserve markets offer financial incentives for generators to maintain standby capacity, while peak reduction markets encourage demand-side management to mitigate peak load conditions, thereby maintaining overall market stability and grid reliability. In terms of market mechanisms, the priority power plan optimizes provincial power development and procurement through inter-provincial priority plans and national power transmission plans; unit joint combination (UC) promotes market association through price difference and agreement settlement mechanisms to ensure economic dispatch and stable operation; and economic dispatch (ED) ensures the efficient transmission and balance of power. In addition, the medium- and long-term markets establish economic relations inside and outside the province through contracts, while the auxiliary service market ensures the flexibility and stability of the market through the gradual liquidation mechanism.

Our comprehensive analysis introduces a novel categorization of transaction modes that integrates traditional long-term power purchase contracts with emerging flexible trading mechanisms such as VPPs and demand-side management strategies. This hybrid approach not only preserves the stability offered by long-term contracts but also leverages real-time adaptability through spot market transactions, thereby optimizing both economic and operational efficiencies in the electricity market. Besides, our analysis extends the traditional dichotomy of transaction methods by introducing a third, hybrid transaction mode that synergizes the stability of long-term power purchase contracts with the flexibility of spot market transactions. This innovative approach allows for the dynamic allocation of resources and risk mitigation, providing a more resilient framework for both power purchasing and generation enterprises. In the evolving landscape of electricity markets, transaction methods between power purchasing enterprises and power generation enterprises have expanded beyond traditional paradigms to incorporate more sophisticated mechanisms. While long-term power purchase contracts and spot market transactions remain foundational, recent studies such as Zhao et al. [33] and Gao et al. [34] have introduced hybrid transaction models that blend the stability of long-term agreements with the flexibility of spot markets. This hybrid approach allows for dynamic resource allocation and risk mitigation, addressing the limitations identified in earlier models that failed to fully capture the complexities of modern energy systems [35,36]. By examining the interplay between bilateral negotiations and centralized trading platforms, our review highlights how these combined approaches enhance market responsiveness and efficiency, thereby offering a more comprehensive framework for understanding transaction dynamics. This integrated perspective underscores the importance of selecting appropriate transaction models based on specific market conditions and strategic objectives, thereby providing actionable insights for both market participants and policymakers.

The game between the independent electricity selling company and the power generation side is the key link to determine the market structure, price formation, and resource allocation [75]. The two sides of the game usually interact through price competition, cooperative procurement, and strategy adjustment, so as to maximize their own interests. In a two-sided symmetrical game, electricity selling companies and power plants choose strategies in similar market environments, such as pricing or production decisions. These options may be influenced by market rules, policies, and supply and demand relationships, such as price caps, quota systems, or market access rules. EGT in such environments reveals how players find optimal combinations of strategies in dynamic markets through adaptation and competition.

In the design of the electricity market, it is mainly medium- and long-term, supplemented by spot. There are two sets of price systems, price signals and spot prices, including the fuel cost of the unit and the cost of power generation consumables, which reflect the variable cost of electric energy [76]. In addition to the medium- and long-term price signed by the power plant is the estimated spot price; it should include the labor costs, financial costs, equipment costs, and other fixed costs to be recovered by the power plant as a general enterprise, as shown in Figure 13.

In the electricity market, medium- and long-term transactions between power plants and electricity-selling companies can be divided into two categories: one is annual transactions through bilateral negotiations, and the other is monthly transactions. These two trading methods have their own characteristics to meet the needs of different participants [77].

First, the annual transaction refers to a one-year power supply contract between a power plant and the selling company through one-to-one negotiations. This approach usually involves the detailed discussion and negotiation of key terms such as electricity price, supply, and delivery time. In this way, selling companies can directly establish long-term partnerships with specific power plants to ensure a stable power supply. The advantage of this transaction method is that both parties can flexibly formulate contract terms according to their own needs and market conditions, but it also requires both parties to invest more time and resources to negotiate.

Monthly trading, on the other hand, is a more flexible approach, allowing electricity sellers and power plants to buy and sell electricity on a monthly basis. This approach provides market participants with more flexibility and can flexibly adjust their trading strategies according to the fluctuations of market supply and demand. Monthly transactions are usually conducted through the power trading platform, making the trading process more transparent and efficient. However, due to the short trading cycle, this method may face the risk of price fluctuations, which requires participants to have strong market analysis and risk management capabilities.

Ref. [78] reviewed and compared the information disclosure mechanisms in different electricity market models (including Poolco models, bilateral contract models, and hybrid models), summarizing the types of information and liquidation models disclosed in these markets. In this work, researchers investigated how to timely and accurately disclose information about the operation of the electricity market to market participants through a centralized and authorized information disclosure mechanism while maintaining market transparency and promoting competition. In Ref. [78], the researchers selected these three models because each is adapted to a distinct electricity market environment and possesses unique characteristics and applicable scenarios. For example, the Poolco model corresponds to centralized competitive transactions, while bilateral contracts pertain to bilateral negotiations.

Poolco model [78]: The Poolco model is a resource allocation model in which multiple participants collectively use a shared resource pool. The model emphasizes cooperation and resource sharing, designed to maximize the overall benefits.

Bilateral contract model [79,80]: The bilateral contract model involves the contractual relationship between the two participants. The two parties reach an agreement through consultation to clarify their respective rights and obligations. Such models are commonly found in commercial transactions and contractual relationships. For example, Han et al. [79] presented an integrative model that explores how psychological contracts influence employees’ compliance with information security policies, emphasizing a bilateral perspective between organizations and employees. A key finding is that fulfilling psychological contracts can enhance compliance behavior. However, a limitation is the model’s reliance on self-reported data, which may introduce bias and limit generalizability. Nazari and Keypour [80] proposed a two-stage stochastic model for optimizing energy storage planning in microgrids, incorporating bilateral contracts and demand response programs. The model effectively addresses uncertainties in energy storage and enhances decision-making. A potential drawback is the limited consideration of real-world market fluctuations, which could affect its practical applicability.

Hybrid model: The hybrid model combines the characteristics of the poolco model and the bilateral contract model. It allows for some degree of sharing of resources while also having bilateral contractual relationships. The mixed model is applicable to some complex situations and requires a comprehensive consideration of cooperation with individual interests.

The information disclosure mechanism is crucial to improving the transparency of the electricity market, reducing transaction costs, improving market fairness, and enhancing the security of the power system. Different electricity market models (poolco, bilateral contract, and hybrid model), due to differences in market structure and transaction mode, lead to significant differences in the mechanism of information disclosure and information content. Information disclosure should find a balance between protecting the privacy of market participants and promoting market transparency, avoiding the risk of market manipulation caused by premature disclosure of sensitive information. As renewable and distributed energy resources are increasingly integrated into the power system, information disclosure mechanisms should evolve in response to these changes, such as more frequent information updates to reflect rapid changes in the power system and market. Finally, it is proposed that the future power market design and the improvement of the existing market should consider the evolution of the information disclosure mechanism to adapt to the changes and challenges of the power industry.

As for the details of the transaction method, bilateral negotiation is one of the forms [81]. In this form, the selling company will establish a direct connection with a specific power plant, and the power plant will voluntarily initiate a contract and sign a contract with the selling company. This approach is usually based on mutual understanding and trust between the two parties, making the transaction process more personalized and direct. In this way, electricity sellers can get an insight into the plant’s production capacity, and power plants can make sure they can find a stable buyer for their electricity. The bilateral consultation helps to establish long-term and stable cooperative relations, but it also requires close communication and coordination between the two sides in the process of contract negotiation and performance.

When a newly established electricity-selling company enters the competitive electricity market, it often struggles with bilateral contract negotiations due to limited knowledge of power plant operations. Bilateral contracts require a detailed understanding of electricity generation capabilities, market risks, and pricing strategies, which new entrants typically lack. Without sufficient market knowledge, engaging in direct negotiations with counterparties is challenging and risky.

To mitigate these challenges, electricity-selling companies can initially participate in centralized competitive markets, such as spot markets or auctions. Centralized markets offer a platform where electricity prices are determined through supply and demand dynamics, allowing new entrants to trade electricity without committing to long-term bilateral contracts. By engaging in these markets, companies can reduce the risks associated with immediate bilateral negotiations and gain crucial experience in market behavior, price fluctuations, and competition dynamics [82,83].

Centralized transactions provide transparency in pricing and market trends, allowing companies to observe how established participants operate. This helps new entrants gradually build a better understanding of market dynamics, enabling them to make more informed decisions when they eventually enter bilateral negotiations [84]. By focusing on centralized competition, companies can initially hedge against the volatility of electricity prices while gaining valuable insights into power plant operations and market regulations.

As the company gains more experience, it can begin exploring bilateral contracts, which provide benefits such as price stability and long-term revenue predictability. Bilateral contracts often hedge against price fluctuations in the spot market, ensuring stable pricing over the contract period. However, these contracts also come with risks, particularly if market prices fluctuate unexpectedly during the contract period [85,86].

To manage these risks, companies can adopt strategies based on risk management models, such as the Nash bargaining solution or contracts for difference (CFDs). These tools help balance the interests of both parties, ensuring a mutually beneficial contract that accounts for future market volatility. For example, the Raiffa–Kalai–Smorodinsky equilibrium allows for dynamic negotiations, helping both parties make concessions that optimize the contract terms [84].

In conclusion, our study not only highlights the inherent challenges faced by new entrants in bilateral contract negotiations but also proposes a strategic framework for leveraging centralized competitive markets as a steppingstone for risk management and market acclimatization. This framework integrates game-theoretic principles with practical risk management tools, enabling companies to transition into bilateral negotiations with enhanced strategic insights and operational resilience. Besides, our analysis underscores the critical role of integrated transaction strategies in fostering market resilience and efficiency. While immediate bilateral contract negotiations pose significant challenges for new entrants, engaging in centralized competitive markets offers a strategic pathway for risk management and experiential learning. Building on the findings of Sun et al. [21] and Yang et al. [24], we propose that centralized platforms not only facilitate initial market entry but also serve as incubators for developing sophisticated risk management and strategic negotiation skills. This strategic engagement enables companies to transition into bilateral negotiations with enhanced market insights and robust risk mitigation frameworks, as supported by the adaptive strategies discussed in Chen et al. [45] and Dong et al. [46]. Moreover, our policy recommendations advocate for the establishment of supportive regulatory environments that encourage hybrid transaction models and promote knowledge transfer from centralized to bilateral market interactions, thereby ensuring a more seamless and cohesive market evolution.

The trading mode of centralized competition trading is carried out in the way of rolling operation. Its price formation method is summarized as follows:

• The buyer and the seller shall quote in quantity;
• The transaction can be made when the declared price is not lower than the declared price by the seller;
• The first transaction price of the session is the average of the transaction pair;
• The subsequent transaction price will refer to the transaction price of the previous transaction.

As shown in

Table 8. Transaction mechanism for determining deal prices in energy markets.

The Seller Declare		The Buyer Declare
Price/CNY	Quantity of Electricity/kWh	Price/CNY	Quantity of Electricity/kWh
350	100	370	87
370	213	320	31
390	123	300	21
430	127	220	78

, the clinch deal price is determined by referencing the previous transaction price. If the previous transaction price falls between 350 and 370, the deal price is set to the median value of the three relevant prices. If the previous transaction price is below 350, the deal price is established at 350. Conversely, if the previous transaction price exceeds 370, the deal price is set to 370.

Table 9. Comparative advantages and disadvantages of bilateral negotiation and centralized trading in energy markets.

Transaction Mode	Advantages	Disadvantages
Bilateral Negotiation	Facilitates tailored agreements between individual power purchasing and generation enterprises, allowing for flexibility and customization.	May lead to increased transaction costs and complexities, especially with a large number of participants. Potential for market power abuse by dominant players.
Centralized Trading	Enhances market transparency and efficiency through standardized processes and information dissemination.	Can reduce flexibility for individual participants and may not accommodate unique contractual needs. Potential for central authority manipulation or inefficiency in large-scale operations.

presents a comparison of the advantages and disadvantages of bilateral negotiation and centralized trading.

In this way, electricity sellers can trade electricity with other market participants on a centralized platform. This way can not only help the electricity-selling companies to enter the market quickly but also to obtain more favorable prices through the competitive mechanism. In addition, centralized competitive transactions are usually organized and regulated by the power regulatory agency or market operators to ensure the fairness and transparency of the transaction [87]. In this way, the electricity-selling companies can gradually establish their own market position without being familiar with the power plant and have more initiative and bargaining power in the future bilateral negotiations.

In the modern power market system, the cooperation mode between power plants and electricity selling companies is increasingly diversified, among which the medium- and long-term bilateral negotiation contract is the main cooperation mode. Accounting for 80% of the market proportion. The signing of such a contract is usually based on the prediction of market trends and the consensus of risk sharing to ensure the stability and economy of power supply. This reflects the government’s regulation of the structure of the power market, ensuring the balance of power supply and demand, and protecting the basic income of power generation enterprises.

Medium- and long-term contracts usually include two parts: electricity and price. Among them, the electricity part specifies the total amount of electricity that the plant should provide to the selling company during the contract period, while the price part includes the cost of the capacity price. The capacity price is a compensation for power generation companies to keep their equipment on standby for peak demand, costing the plant regardless of whether the power is actually sold. Therefore, the power plant will take into account the fixed cost of the long-term contract.

According to the information in Figure 13, power plants prefer to keep stable, medium- and long-term contract electricity in their supply system to avoid flowing into the spot market of [88]. The spot market serves as the immediate platform for electricity trading, where prices fluctuate in real time in response to supply and demand dynamics. When power plants divert their medium- and long-term contract flows into the spot market and the prevailing market price falls below the contract price, the power plants incur economic losses due to the resulting price differential. In addition, power plants may not be able to flexibly adjust their generation capacity due to medium- and long-term contracts, which also increase operational risks. However, with the complexity of the power market structure, the game between the two sides also shows an asymmetry. The asymmetric game between the electricity selling company and the power generation side may result from the inequality of market information, the difference of resource endowment, the opacity of the contract structure, or the policy influence of [89]. In this case, the game players must have a deeper understanding of the opponent’s strategy space and adjust their behavior accordingly to adapt to the incomplete informative market environment.

The spot market is mainly divided into two parts, namely, the day-ahead market and the real-time market. Typically, electricity-selling companies report the next day’s electricity consumption a day in advance, which includes medium- and long-term electricity consumption plans and electricity demand in the spot market. In the process of bilateral negotiation or centralized transactions, the electricity selling company will plan the medium- and long-term electricity needed each day, and this electricity is determined in the negotiation or concentration [90]. The remaining electricity per day is sent to the front market for trading. If the actual electricity consumption exceeds the previously declared electricity consumption, then the excess part needs to be processed in the real-time market. In the real-time market, electricity-selling companies can make more flexible power purchases and allocations according to the real-time power supply and demand situation to ensure the stability and economy of power supply.

In the increasingly complex environment of the electricity market, the transaction methods between power purchasing enterprises and power generation enterprises show a trend of diversification [91]. With its long-term power purchase contract, the long-term power purchase contract is still the main mode of cooperation between the two parties. However, with the increase in the market demand for flexibility, the role of the spot market has become increasingly prominent, becoming an important battlefield of the dynamic game between the two sides [33]. The interaction between electricity selling companies and power generation enterprises is not only reflected in the price competition, but also involves the game of market structure, policy rules, and strategic adjustment [91,92]. In the face of market supply and demand fluctuations, price signals, and policy regulation, both sides need to combine long-term and short-term trading methods to maximize their benefits.

In the future, with the further opening and reform of the power market, the relationship between power generation enterprises and electricity selling companies will be more complex and diversified. Medium- and long-term contracts will continue to play an important role in the market as the basis for stabilizing power supply and guaranteeing benefits; however, the flexibility and timeliness provided by the spot market exchange will continue to be an important means of resource allocation in the market. Through the rational use of EGT, market participants can better adapt to the changing market environment and develop the optimal trading strategy to ensure that they remain invincible in the highly competitive electricity market.

Finally, the development direction of the power market will depend on the game balance and policy adjustment of all participants, and the cooperation and competition between the electricity selling companies and the power generation enterprises will continue to promote the innovation and optimization of the power trading mode.

Based on the evaluation of control policies summarized in previous sections, Section 9 offers targeted policy implications designed to enhance market competition, transparency, and sustainability.

9. Conclusions, Prospects, Policy Implications and Future Directions

The contributions of this paper extend beyond theoretical insights to offer pragmatic strategies that guide policymakers, market participants, and regulators. By synthesizing evolutionary game theory (EGT), Stackelberg games, Bayesian games, and DRL, our integrated framework provides a unique toolkit for understanding—and steering—strategic behaviors in electricity markets.

9.1. Conclusions

With the continuous development and reform of the power market, the transaction mode between power purchasing enterprises and power generation enterprises shows a trend of diversification and marketization. Long-term power purchase contracts provide a stable cooperation foundation for power generation enterprises and power purchasing enterprises to ensure the continuity of power supply and price stability, while the spot market transaction improves the flexibility and responsiveness of the market through real-time supply and demand adjustment. However, with the gradual improvement of the market mechanism, the game between the independent electricity selling companies and the power generation enterprises has become an important factor affecting the market structure, price formation, and resource allocation. Through price competition, strategic adjustment, and cooperative procurement, power purchasing enterprises and power generation enterprises constantly adapt to the changes of market rules and policies in order to seek the maximum benefits in the game. In general, the current power trading model not only needs to strike a balance between economy and reliability but also take into account the flexibility and sustainability of the market. To this end, this review advances the existing literature by providing an integrative analysis that combines EGT, Stackelberg games, and Bayesian games with cutting-edge technologies such as DRL. This multifaceted approach not only elucidates the strategic behaviors of individual market participants but also uncovers the complex interdependencies and emergent dynamics that shape overall market efficiency and stability. Furthermore, our policy recommendations are informed by these novel insights, offering actionable strategies for enhancing energy governance in the context of evolving electricity markets.

Based on these investigations, this paper aims to analyze in-depth the evolutionary game behavior in the electricity market and reveal the key factors that affect market efficiency and stability in this complex system. We focus not only on the strategic choices of single market participants, but also on how these strategies are intertwined to form a market dynamic that will affect the functioning of the entire market. The combination of spot market and medium- and long-term contracts not only provides the stability of the market but also brings new risks such as price risk and credit risk. We need to understand these risks in order to devise more effective risk management mechanisms. To this end, this comprehensive review offers an integrative analysis of strategic evolution and game-theoretic applications in power market transactions, emphasizing the dynamic interactions between power purchasing and generation enterprises. By systematically evaluating the suitability and effectiveness of Stackelberg, Bayesian, and evolutionary game theories, our study elucidates the conditions under which each model optimally contributes to market efficiency and stability. This nuanced understanding enables market participants and policymakers to select and implement the most appropriate strategic frameworks tailored to specific market conditions and objectives. Moreover, the integration of advanced technologies such as DRL with these game-theoretic models further enhances the adaptability and resilience of electricity markets, providing actionable insights for future energy governance. Thus, our review not only synthesizes existing knowledge but also pioneers new pathways for optimizing strategic decision-making in the evolving landscape of electricity markets. The electricity market has undergone significant transformation, evolving from monopolistic structures to increasingly competitive and complex environments. This evolution has been characterized by the diversification and marketization of transaction modes between power purchasing enterprises and power generation enterprises. As above, main contributions that align with this paper’s objectives are summarized as follows.

First, this review has systematically explored the application of game-theoretical methods, particularly EGT, in optimizing electricity markets. By integrating EGT with DRL, we analyzed a comprehensive framework that addresses the dynamic and strategic nature of electricity markets. This hybrid approach facilitates the simulation of complex market scenarios, capturing the nuanced decision-making processes of enterprises under varying conditions of uncertainty and competition.

Second, our analysis demonstrated that the integration of EGT and DRL significantly enhances market resilience, enabling electricity markets to better withstand shocks such as sudden demand fluctuations, supply disruptions, and regulatory changes. Additionally, this integration promotes sustainable energy integration by modeling the strategic adoption of renewable energy technologies and optimizing resource allocation, leading to increased efficiency, reduced costs, and greater sustainability in market performance.

Third, this review also evaluated the effectiveness and cost-efficiency of various control policies, including pricing mechanisms, capacity incentives, renewable integration incentives, and regulatory measures aimed at enhancing market competition and transparency. The findings provide actionable insights for policymakers and industry stakeholders, highlighting the importance of information transparency, reducing barriers to entry, and adaptive pricing strategies in fostering a competitive and efficient market environment.

Fourth, this study contributes to the development of robust regulatory frameworks that support competitive and efficient electricity markets in an evolving energy landscape. By leveraging the dynamic and adaptive capabilities of EGT and DRL, policymakers can design regulations that not only address current market challenges but also anticipate and adapt to future developments. This proactive approach is essential for fostering a resilient energy infrastructure capable of accommodating rapid advancements in renewable technologies and shifting consumer demands.

Furthermore, our work underscores that combining game-theoretic models with DRL significantly enhances market efficiency, fosters long-term resilience, and supports renewable energy integration. Critically, we reveal how information asymmetry, market power, and demand uncertainty can be mitigated through carefully calibrated policies, including adaptive pricing schemes and transparency measures. Furthermore, this research highlights the crucial role of sustained governmental oversight to prevent market manipulation by dominant participants, underscoring the need for robust yet flexible regulatory structures.

In conclusion, the integration of EGT with DRL offers a powerful tool for understanding and optimizing strategic interactions within electricity markets. The effectiveness and cost-efficiency of control policies evaluated in this study provide valuable insights for designing policies aimed at enhancing market resilience, promoting sustainable energy integration, and improving overall market performance. As the energy sector continues to evolve, the frameworks and insights presented in this review will be instrumental in guiding the development of adaptive and resilient electricity markets, ensuring their sustainability and competitiveness in a rapidly changing global landscape.

Looking ahead, in the next subsection, this review identifies several key areas for future research, including the exploration of multi-agent reinforcement learning techniques to further enhance the predictive and adaptive capabilities of the integrated EGT-DRL framework. Additionally, there is a need for empirical studies to validate the theoretical models and simulations discussed, ensuring their applicability and effectiveness in real-world market conditions. By bridging the gap between theoretical game-theoretic models and practical market applications, this study provides a comprehensive roadmap for optimizing electricity markets through strategic and policy-driven interventions. In the future, the long-term power purchase contracts continue to serve as a cornerstone of market stability, ensuring continuity of power supply and price stability. Concurrently, the emergence and growth of spot market transactions have introduced greater flexibility and responsiveness to real-time supply and demand fluctuations. This dual structure of long-term contracts and spot markets has created a more dynamic and efficient market environment, but it has also introduced new challenges and complexities. The application of game theory, particularly EGT, Stackelberg games, and Bayesian games, has provided valuable insights into the strategic behaviors of market participants. These theoretical frameworks have allowed for a more nuanced understanding of how power purchasing enterprises and power generation enterprises adapt their strategies in response to changing market rules, policies, and competitive landscapes.

9.2. Prospects for Future Research and Development in Electricity Markets

Future investigations will benefit from further empirical validation of DRL-enhanced game-theoretic frameworks, particularly in high-renewable-penetration contexts. Moreover, exploring cross-border electricity trade and multi-energy systems remains vital for extending our approach to broader policy environments. Looking ahead, several key areas emerge as critical for future research and development in electricity markets:

(1)

Advanced Market Mechanisms: Future research should delve into the creation of more sophisticated market mechanisms that seamlessly integrate long-term stability with short-term adaptability. One promising direction is the design of hybrid contracts combining long-term agreements with dynamic spot market pricing to provide both stability and flexibility [93]. Moreover, developing advanced auction frameworks, particularly for the integration of renewable energy and capacity markets, can boost market efficiency and facilitate the energy transition [94].

(2)

Integration of Artificial Intelligence and Machine Learning: Future research should explore more advanced AI and ML techniques for market modeling and forecasting. Building on DRL’s integration with game theory, generative adversarial networks (GANs) could be used for scenario generation, while transformer models may be applied to analyze long-term market trends. These sophisticated techniques can significantly enhance the accuracy of market simulations, providing improved decision-making tools for operators and participants [95,96,97,98]. Here, reinforcement learning (RL) is a subset of machine learning where an agent learns to make decisions by performing actions in an environment to achieve maximum cumulative reward [95]. In the power market, DRL—which combines RL with deep neural networks—enhances the agent’s ability to handle complex, high-dimensional data and make sophisticated decisions [95,96,97,98]. The DRL game theory method is widely used in demand response management, load scheduling, price bidding, and other scenarios. For example, power companies can use this method to learn the optimal generation strategy to cope with market price fluctuations, while power purchasing companies can adjust their purchasing behavior according to historical data and market trends.

(3)

Blockchain and Decentralized Markets: The potential of blockchain technology in enhancing transparency and efficiency within decentralized energy markets warrants further investigation. Research could explore how smart contracts and decentralized autonomous organizations (DAOs) facilitate peer-to-peer energy trading and grid management, leading to more resilient and democratic market structures. Such innovations would particularly benefit markets with increasing distributed energy resources and renewable energy penetration [99,100,101,102].

(4)

Multi-Energy Systems and Sector Coupling: As the energy transition accelerates, the integration of electricity markets with other energy sectors such as gas, heat, and hydrogen is becoming essential for achieving sustainable energy systems. The development of multi-energy systems (MES) allows for more efficient resource utilization, leveraging the synergies between different energy vectors. Future research should focus on creating integrated market models that capture the interdependencies among electricity, gas, and heating sectors, ensuring that these systems work in harmony to maximize energy efficiency and flexibility. For instance, power-to-gas (P2G) technologies, which convert surplus electricity into hydrogen or methane, can enable more flexible energy management, particularly during periods of excess renewable generation [103]. Game-theoretic approaches offer a promising framework for optimizing these multi-commodity markets, providing a structured way to balance supply and demand across sectors [104]. Additionally, technologies such as hydrogen storage and power-to-heat systems can significantly enhance energy flexibility, improving the resilience of energy systems against market fluctuations [105]. The integration of these diverse energy systems into a unified framework not only boosts efficiency but also opens up new opportunities for renewable energy integration [106]. Going forward, the development of robust optimization models that incorporate these energy vectors will be key to creating a low-carbon, reliable energy future [107].

(5)

Behavioral Economics in Electricity Markets: As the electricity market evolves, understanding the behavioral economics of market participants becomes increasingly important [92]. Future research should explore how cognitive biases, such as overconfidence or loss aversion, influence the bidding strategies of electricity providers and consumers. Empirical studies on these biases could lead to more accurate agent-based models that reflect real-world decision-making under uncertainty [108]. Furthermore, integrating behavioral insights into market models can help regulators design more effective market rules and bidding strategies [109]. Agent-based models that incorporate bounded rationality and learning dynamics, such as reinforcement learning, have shown promise in simulating market behaviors and outcomes [110]. These models can also examine the role of incomplete information and how it affects strategic bidding, ultimately improving the accuracy of electricity market simulations [111]. Exploring these behavioral factors will enable the development of market structures that better reflect the complexities of human decision-making, leading to more efficient and resilient electricity markets [112].

(6)

Climate Change and Extreme Weather Events: The growing impact of climate change on electricity markets, particularly due to the increasing frequency and severity of extreme weather events, requires urgent attention [92]. These events, such as hurricanes, heatwaves, and cold waves, significantly affect grid stability and energy supply. Future research should focus on integrating climate models with market simulations to better predict and mitigate these risks [113]. Additionally, developing new risk management tools for market participants, such as adaptive capacity markets, could help manage the reliability challenges posed by extreme events [114]. Extreme weather events not only disrupt power grids but also reduce the efficiency of power generation, particularly during peak demand [115]. Coupling electricity market models with climate projections can provide better forecasts and improve market resilience. Risk assessment tools that include weather impacts can also be utilized to evaluate potential infrastructure vulnerabilities [116]. By incorporating these measures, the electricity sector can better adapt to the challenges posed by climate change [117].

(7)

Energy Justice and Market Design: As electricity markets evolve, promoting energy justice is essential to ensure equitable access and fair market participation, especially for vulnerable communities and developing economies. Future research should focus on how market design can better address social equity by incorporating mechanisms that account for fairness and welfare. For instance, designing new metrics to assess market fairness could help identify disparities in energy access and affordability. Additionally, market structures should be adapted to include considerations of social welfare, allowing for more inclusive participation in energy markets, especially in low-income regions [118]. Empirical studies on energy justice have revealed the importance of empowering marginalized groups and ensuring that clean energy transitions do not disproportionately benefit the wealthy [119]. Integrating energy justice into market governance would also require balancing the needs of vulnerable populations with the operational flexibility required in modern grids [120]. Addressing these concerns through comprehensive market designs can help create more just and resilient energy systems [121].

(8)

Cybersecurity and Market Resilience: With the growing digitalization of electricity markets, cybersecurity has emerged as a critical concern. Future research should prioritize developing robust market platforms capable of withstanding cyber-attacks and ensuring the integrity of operations. Cyber-attacks can disrupt market activities, manipulate data, and induce financial losses, thus necessitating advanced security measures [122]. Game theory can be applied to model the behavior of attackers and defenders, helping market operators predict and counter cyber threats more effectively [123]. AI-powered anomaly detection systems could be used to monitor market transactions and detect irregular patterns indicative of cyber-attacks [124]. Furthermore, the integration of cybersecurity strategies into real-time power system operations can enhance the resilience of energy markets [125]. Additionally, game-theoretic frameworks for anomaly detection and dynamic defense offer promising pathways to enhance cyber resilience [126].

(9)

Regulatory Frameworks and International Coordination: As electricity markets become increasingly interconnected across national borders, the development of robust regulatory frameworks to facilitate international market coordination is critical. Effective cooperation between countries is essential to managing cross-border electricity flows, congestion, and market integration. Future research should explore various regulatory approaches and game-theoretic models that enable better international cooperation. Comparative studies of existing regulatory practices in different regions could reveal best practices and inform new policy frameworks [127]. Additionally, game-theoretic frameworks can be used to model strategic behavior and coordination among market participants, which can help optimize cross-border congestion management and pricing [128]. Moreover, the integration of distributed energy resources (DER) across borders presents additional challenges, requiring innovative coordination strategies and risk-sharing mechanisms [129]. The use of game theory in these contexts can provide valuable insights into how market participants—such as generators and transmission system operators—can cooperate to achieve stable and efficient electricity flows [130]. Establishing a common regulatory framework that incorporates cross-border market coupling mechanisms will be key to the future of European electricity markets [131].

(10)

Long-term Market Evolution and Energy Transitions: As global energy transitions accelerate, understanding how electricity markets evolve over the coming decades becomes essential [92]. Future studies should develop scenario-based modeling approaches that account for technological, economic, and policy uncertainties. These models should incorporate renewable energy integration, energy demand changes, and the impact of market mechanisms to guide policymakers and long-term investors [132]. Scenario analysis, especially Monte Carlo simulations, can aid in capturing the volatility and long-term uncertainties that shape market dynamics [133]. Furthermore, system dynamics modeling provides valuable insights into energy transition pathways, emphasizing feedback loops and learning processes [134]. Models that explore technological transitions in transportation and other sectors should also be integrated to ensure comprehensive planning [135]. Decision-making frameworks, such as real options and robust decision models, are essential for navigating market uncertainties [136], while agent-based models are crucial for exploring decentralized market behaviors [137]. By incorporating these methodologies, future market evolution models will provide valuable strategies for energy policy and investment decisions [138].

Overall, the electricity market remains a dynamic and evolving field, with numerous opportunities for innovative research and practical applications. By building on the theoretical foundations and empirical insights provided in this review, future studies can contribute to the development of more efficient, sustainable, and equitable electricity markets. These advancements will be crucial in addressing the complex challenges of the global energy transition and ensuring a reliable, affordable, and clean energy future.

9.3. Policy Implications and Future Directions for Energy Governance

This comprehensive review of game theory applications in electricity markets offers several crucial insights for energy policymakers and regulators. The strategic interactions between power purchasing and generation enterprises, as elucidated through EGT, Stackelberg games, and Bayesian games, have profound implications for the design and implementation of energy policies. Drawing on the insights from Zhao et al. [33] and Gao et al. [34], our analysis demonstrates how game-theoretic models can inform regulatory frameworks that balance market efficiency with sustainability goals. Moreover, the integration of DRL with game theory, as explored by Li et al. [31] and Bellegarda et al. [65], provides policymakers with advanced tools for forecasting and mitigating market risks, thereby enhancing the robustness of energy governance structures. This cohesive linkage between theoretical models and practical policy applications underscores the critical role of comprehensive game-theoretic analyses in shaping effective energy policies. This section outlines key policy considerations, recommendations, and future research directions in energy governance.

9.3.1. Policy Implications

Furthermore, we propose the following policy implications to enhance the efficiency, sustainability, and competitiveness of electricity markets:

Enhance Market Competition: Policymakers should implement anti-monopoly regulations that lower entry barriers for emerging power producers and electricity sales enterprises.

Promote Fair Pricing Mechanisms: transparent auction designs and frequent disclosure of supply–demand data can counter the effects of information asymmetry.

Encourage and Promote the Trading and Integration of Renewable Energy: With the acceleration of the global energy transition, the proportion of renewable energy in the electricity market is gradually increasing. Targeted tax credits and stable long-term contracting frameworks help reduce risk for new clean-energy entrants, thereby supporting sustainable expansions. In the future, we should continue to explore how to introduce more renewable energy trading mechanisms in the transaction between power purchasing enterprises and power generation enterprises, so as to ensure the effective consumption of new energy and the market balance. Aiming at renewable energy integration, Stackelberg and evolutionary game models can inform policies that incentivize the adoption of renewable energy sources, ensuring that these sources are integrated into the grid in a manner that maintains market stability and competitiveness [34,38]. By understanding the strategic interactions between renewable energy providers and traditional power generators, policymakers can craft regulations that support sustainable energy transitions while minimizing market disruptions.

Cybersecurity and Market Resilience: align existing digital infrastructure with game-theoretic models that anticipate cyber-attacks, creating advanced defense protocols and real-time threat detection systems.

Strengthening Policy Support and Supervision: With the deepening of market reform, the government should continue to strengthen the supervision of the electricity market to ensure the fairness and transparency of market transactions [139,140,141]. Through the introduction of clearer policy guidance, the cooperation and competition between power purchasing enterprises and power generation enterprises should be promoted, while the phenomenon of market manipulation and unfair competition should be avoided.

Promoting Market Flexibility and Innovation: The current electricity market needs to further promote the wide application of the spot market and other innovative trading methods based on retaining the stability of long-term contracts. For example, emerging technologies such as VPPs and demand response can improve the flexibility and adaptability of the power system [142,143,144,145], providing more trading opportunities and strategies for market participants.

Improve the Information Disclosure Mechanism: The information disclosure mechanism of the electricity market is directly related to market efficiency and fairness. In the future, the electricity market should further enhance information transparency, especially in terms of electricity prices, market transaction data, and renewable energy trading, to ensure that market participants can have timely access to key information and make accurate decisions.

Strengthening Technological Innovation and Digitalization: With the development of technologies such as the Internet of Things, big data, and artificial intelligence [146], the trading mode in the power market can be more intelligent and efficient. Power purchasing enterprises and power generation enterprises can monitor market dynamics and optimize transaction strategies in real time through intelligent power trading platforms, so as to improve transaction efficiency and economic benefits.

Market Design and Regulation: The evolution of electricity markets from monopolistic structures to competitive environments necessitates the development of adaptive regulatory frameworks informed by game-theoretic insights. Stackelberg games provide a foundation for designing regulations that account for hierarchical decision-making and the strategic dominance of key market players, thereby promoting fair competition and preventing market manipulation [37,38]. Bayesian game models inform the creation of policies that mitigate information asymmetry, ensuring transparency and informed decision-making among market participants [59,63]. EGT contributes to the formulation of adaptive policies that accommodate the dynamic evolution of strategies and the integration of renewable energy sources, fostering sustainable and resilient market structures [34,43,147]. Furthermore, integrating DRL with these game-theoretic models provides policymakers with advanced tools for forecasting and mitigating market risks, thereby enhancing the robustness of energy governance structures [95,148]. By leveraging the distinct advantages of each game-theoretic approach, policymakers can design more effective regulations that improve market efficiency, stability, and sustainability. This holistic regulatory strategy ensures that the electricity market remains responsive to technological advancements and evolving energy demands, thereby supporting the overarching goals of energy governance. Besides, the policymakers should consider implementing flexible market rules that can accommodate the dynamic nature of strategic interactions revealed by game-theoretic analyses. For instance, regulations could be designed to promote a balance between long-term contracts and spot market transactions, ensuring both market stability and efficiency.

Information Asymmetry and Transparency: To address information asymmetry, Bayesian game theory models can be utilized to design policies that promote data sharing and transparency among market participants, thereby enabling more informed and strategic decision-making. This approach aligns with the findings of Verma et al. [63], who highlight the importance of information symmetry in achieving market equilibrium and efficiency. Additionally, our analysis of market power dynamics, grounded in EGT, provides policymakers with insights into how dominant players can sustain their influence and the potential regulatory measures needed to prevent monopolistic behaviors. By understanding the strategic interactions between dominant and non-dominant market participants, policymakers can craft targeted interventions that mitigate the adverse effects of market power and promote a more competitive and equitable market environment.

Demand Response and Consumer Empowerment: Integrating DRL with game-theoretic models can enhance demand response programs by enabling more accurate predictions of consumer behavior and optimizing the incentives offered to consumers [95]. This integration facilitates the development of dynamic pricing strategies and real-time demand management solutions that empower consumers and contribute to overall market efficiency.

Cybersecurity and Market Resilience: The combination of EGT and DRL can also inform cybersecurity policies by modeling the strategic interactions between attackers and defenders in electricity markets [95]. This enables the development of proactive defense strategies that anticipate and mitigate potential cyber threats, thereby enhancing the resilience of digitalized electricity markets.

Overall, future research should explore the further integration of game-theoretic models with advanced machine learning techniques to enhance the predictive and adaptive capabilities of electricity market analyses. Additionally, empirical studies are needed to validate the theoretical frameworks proposed in this paper, ensuring their applicability and effectiveness in real-world market conditions. Policymakers are encouraged to consider the insights derived from this study when designing regulatory frameworks that promote transparency, competition, and sustainability in electricity markets.

9.3.2. Policy Recommendations

To optimize market performance and enhance policy regulation, policymakers should implement strategies that enhance competition and ensure fair pricing mechanisms. Additionally, regulations must be adaptable to accommodate emerging technologies and evolving market dynamics. Thus, policymakers should consider implementing the following specific and actionable strategies:

Enhance Market Competition: Introduce measures such as anti-monopoly regulations and promote the entry of new market participants to reduce the dominance of incumbent firms. For example, establishing regulatory bodies to monitor and prevent anti-competitive practices can ensure a level playing field for all enterprises.

Ensure Fair Pricing Mechanisms: Develop and enforce transparent pricing frameworks that prevent price manipulation and ensure consumers receive fair prices. Implementing real-time pricing systems that reflect actual supply and demand can enhance market efficiency and consumer trust.

Adaptable Regulatory Frameworks: Create flexible regulations that can quickly adapt to the integration of emerging technologies such as renewable energy sources and smart grid systems. For instance, developing guidelines for the seamless incorporation of distributed energy resources can facilitate the transition to sustainable energy systems. Develop a flexible regulatory framework that evolves in tandem with market dynamics and incorporates advanced game-theoretic insights. Drawing on the hierarchical models proposed by Gao et al. [34] and the Bayesian approaches discussed by Wang et al. [36], this framework should integrate multi-layered strategic interactions and information asymmetries, ensuring that regulations remain relevant and effective amidst changing market conditions. This adaptive approach allows policymakers to respond proactively to emerging trends and strategic behaviors, fostering a more resilient and efficient electricity market.

Promote Information Transparency: Mandate comprehensive data sharing among market participants to reduce information asymmetry. Policies requiring the disclosure of key operational and financial information can empower enterprises to make informed strategic decisions and enhance overall market transparency.

Incentivize Renewable Energy Integration: Design incentive structures that encourage the adoption of renewable energy technologies, such as tax credits, subsidies, and grants for clean energy projects. This can accelerate the transition to sustainable energy sources and reduce reliance on fossil fuels.

Implement Robust Monitoring and Evaluation Mechanisms: Establish continuous monitoring systems to assess the effectiveness of implemented policies and make necessary adjustments. Regular evaluations using key performance indicators (KPIs) can help identify areas for improvement and ensure that policies are meeting their intended objectives.

Data-Driven Policy-making: Invest in advanced data collection and analysis capabilities to inform policy decisions. This could involve establishing a centralized data repository for market transactions and outcomes, which can be used to validate and refine game-theoretic models.

Collaborative Policy Development: Foster collaboration between academics, industry stakeholders, and policymakers. Regular forums or workshops could be organized to discuss the latest research findings and their policy implications.

International Coordination: Given the increasing interconnectedness of electricity markets, develop policies that facilitate cross-border trading while ensuring national energy security. This could involve harmonizing market rules and establishing international dispute resolution mechanisms.

Innovation Support: Implement policies that encourage technological innovation in the electricity sector. This might include funding research into new market mechanisms, supporting pilot projects for novel trading platforms, or offering incentives for the development of advanced forecasting tools.

9.3.3. Political Applicability

Our findings carry substantial implications for governments and regulatory agencies shaping energy policy at both national and international levels. By illuminating how game-theoretic models and DRL can enhance market transparency and forecasting, we highlight a pathway to reduce electricity costs, boost investment in renewable infrastructure, and elevate consumer confidence. Specifically, adopting these models helps authorities calibrate bidding protocols, address policy gaps related to cross-border energy flows, and mitigate risks posed by geopolitical uncertainties. Furthermore, the recommended reforms—ranging from capacity incentives to real-time pricing—are readily applicable under existing legislative frameworks in many countries, enabling swift integration into ongoing energy transitions. Below, aiming at political applicability, we provide an enriched discussion of three practical applications for the proposed game-theoretical methodology framework in this paper.

(1)

Guiding Local and National Regulators in Redesigning Energy Auctions.

Balancing short-term spot market dynamism with the long-term stability offered by power purchase contracts represents a critical challenge for policymakers. A well-designed auction mechanism should ensure that energy procurement remains cost-effective, transparent, and flexible enough to accommodate both immediate operational needs and broader policy objectives.

First, for the hybrid auction structures, which contain three aspects as follows.

(i)

Multi-Round Auctions: Incorporating multi-round bidding (e.g., descending-clock or iterative sealed-bid formats) enables regulators to capture the evolving cost structures of both conventional and renewable generators. In each round, participants update bids based on previously revealed market information, which improves market price convergence and transparency.

(ii)

Forward Contracts and Real-Time Bidding: Pairing forward capacity contracts with real-time spot auctions can stabilize prices over the planning horizon. This approach leverages the security of long-term deals while allowing market participants to respond to short-term fluctuations via spot pricing.

(iii)

Risk Mitigation Tools: auctions could also provide optional hedging instruments—such as contracts for differences (CfDs)—to reduce the financial vulnerability of participants to price volatility, thus preventing undue market exit of smaller or renewable-based providers.

Second, for the equitable market participation, it contains two aspects.

(i)

Incentive-Compatible Mechanisms: game-theoretic principles can guide the design of auction rules that minimize gaming opportunities and encourage truthful revelation of costs and capacities.

(ii)

Support for Emerging Generators: Policymakers could introduce tiered or technology-specific auctions to encourage market entry by renewables or distributed resources. For instance, separate bidding segments for wind or solar generation help align procurement with decarbonization targets.

Third, for the transparent price formation, it includes two aspects.

(i)

Real-Time Information Disclosure: regulators can enforce data-sharing protocols that provide timely insights into grid conditions and competitor strategies, thereby preventing market manipulation.

(ii)

Smart Monitoring Systems: integrating monitoring tools, potentially enhanced by DRL, can detect anomalous bidding behaviors in real-time, enabling rapid regulatory interventions.

Overall, by combining these strategies with the evolutionary and hierarchical game models discussed in the paper, local and national regulators can refine auction designs to ensure that short-term economic signals and long-term policy goals remain in harmony.

(2)

Strengthening the Adaptive Capabilities of Microgrid Operators and Virtual Power Plants (VPPs).

Microgrids and VPPs are increasingly recognized as pivotal components of modern electricity systems, due to their capacity to aggregate distributed energy resources (DERs) and respond swiftly to local conditions. Their success hinges on sophisticated control algorithms, robust market participation strategies, and effective coordination with the grid at large.

First, aiming at adaptive bidding and dispatch strategies, including the following:

(i)

Dynamic Pricing Integration: by leveraging DRL-based optimization, microgrid operators and VPPs can adjust their supply bids and consumption schedules in near-real time, mitigating the financial risks associated with spot market volatility.

(ii)

Multi-Agent Learning: evolutionary game models facilitate iterative learning among multiple DER agents (e.g., solar PV, battery storage, demand-response aggregators), enabling these entities to co-evolve strategies that maximize the joint utility (profitability, reliability) of the microgrid ecosystem.

Second, aiming at robustness to demand shocks, including the following:

(i)

Resilient Infrastructure Planning: combining game theory with stochastic models helps microgrids identify critical nodes and storage resources that minimize supply disruptions during sudden demand spikes or supply deficits.

(ii)

Frequency Regulation and Ancillary Services: VPPs can coordinate DERs to provide short-term load balancing, voltage support, and other ancillary services in response to real-time grid needs, ensuring overall system stability.

Third, aiming at scalable platform for peer-to-peer (P2P) trading, including the following:

(i)

Local Market Creation: adopting the proposed frameworks can encourage the formation of local trading platforms where microgrids and VPPs trade energy surpluses or deficits among themselves, harnessing price signals from both local and system-wide markets.

(ii)

Decentralized Coordination: through consensus-driven strategies (e.g., game-theoretic bargaining), small DERs can negotiate terms that optimize collective welfare without dependence on centralized market operators.

Taken together, these measures help microgrid operators and VPPs achieve greater agility, reinforcing their role as indispensable partners in smoothing out supply–demand fluctuations and strengthening overall grid resilience.

(3)

Enhancing Collaboration Among International Bodies and Establishing Cross-Border Balancing Mechanisms.

Regional power pools and interconnections introduce complexity in transmission scheduling, price harmonization, and resource adequacy across multiple jurisdictions. The game-theoretical frameworks discussed in the paper can inform international coordination by modeling strategic behaviors of national grid operators, large power companies, and policymakers seeking to protect domestic interests while reaping the benefits of integrated markets.

First, aiming at harmonized regulatory standards, including the following:

(i)

Common Auction Platforms: internationally coordinated auction rules and cross-border capacity reservation mechanisms can reduce arbitrage opportunities and ensure that grid reliability is maintained at lower aggregate cost.

(ii)

Multilateral Contracts and Congestion Management: applying EGT to cross-border congestion management can reveal stable cooperation patterns, e.g., cost-sharing for interconnection projects, thereby facilitating the equitable distribution of infrastructure investments.

Second, aiming at cross-border balancing markets, including the following:

(i)

Real-Time Balancing Services: expanding existing balancing markets to include near-instantaneous cross-border services promotes the seamless flow of electricity, reduces localized stress on networks, and incentivizes higher renewable penetration.

(ii)

Coordination of Reserve Requirements: Common reserve standards and reciprocal balancing agreements can minimize the total reserve margin needed across multiple regions. In times of emergency or abrupt demand spikes, cooperative protocols allow surplus capacity in one region to assist another.

Third, aiming at managing regional diversity, including the following:

(i)

Incentive-Compatible International Agreements: by modeling each country (or region) as a strategic player, global regulators can design frameworks that reward honest reporting of resource availability and penalize free-riding or non-cooperative behavior.

(ii)

Institutional Capacity Building: data exchange and harmonized grid codes require continuous dialog among national regulatory bodies, which can be enhanced by multi-agent DRL simulations that test negotiation outcomes under diverse scenarios.

These policy-driven measures reinforce the potential for cross-border electricity markets to achieve lower costs, higher system stability, and broader sustainability goals. By employing game-theoretic insights alongside advanced computational methods, international collaborations can surmount the challenges inherent in regional power pool operations.

Overall, our integrated game-theoretic and DRL approach serves as a roadmap for political decision-makers to fortify market resilience, promote equitable energy access, and accelerate decarbonization. By aligning stakeholders’ interests through a data-driven and strategically adaptive lens, policymakers can ensure that electricity markets remain robust, inclusive, and supportive of sustainable growth. Strengthening the global electricity market demands an integrated suite of policies and operational strategies. By guiding auction redesign, advancing microgrid/VPP responsiveness, and promoting international regulatory coherence, the proposed framework offers a scalable blueprint for evolving power systems. Leveraging game-theoretic models fused with DRL not only enhances real-time decision-making but also ensures that strategic interactions remain equitable, transparent, and oriented toward long-term decarbonization and grid stability.

9.3.4. Future Research Directions in Energy Policy

Long-term Market Evolution Models: Develop comprehensive, long-term models that can simulate the evolution of electricity markets under various policy scenarios. These models should integrate game-theoretic approaches with other relevant disciplines such as complexity science and systems theory.

Multi-objective Policy Optimization: explore the use of multi-objective optimization techniques in policy design, considering factors such as market efficiency, social welfare, environmental impact, and energy security simultaneously.

Behavioral Economics in Energy Policy: Investigate the role of behavioral economics in shaping energy policies. This could involve studying how cognitive biases affect market participant decisions and how policies can be designed to account for these biases.

Artificial Intelligence and Market Governance: Research the potential applications of AI in market monitoring and regulation. This could include developing AI-powered tools for real-time market analysis and automated policy adjustment mechanisms.

Energy Justice and Policy Design: Explore how energy policies can be designed to promote energy justice and address issues of energy poverty. This might involve developing new metrics for policy evaluation that go beyond traditional economic indicators.

Resilience and Climate Change Adaptation: Investigate how energy policies can enhance the resilience of electricity markets to climate change impacts. This could include studying the effectiveness of various policy instruments in promoting adaptive capacity among market participants.

In conclusion, the game-theoretic analyses presented in this review offer valuable insights for energy policymakers. By incorporating these insights into policy design and continuing to support research in this field, policymakers can work towards creating more efficient, equitable, and sustainable electricity markets. The future of energy governance lies in adaptive, data-driven policies that can keep pace with the rapidly evolving dynamics of modern electricity markets.