Optimal Bidding Framework for Integrated Renewable-Storage Plant in High-Dimensional Real-Time Markets

Abstract

With the development of electricity spot markets, the integrated renewable-storage plant (IRSP) has emerged as a crucial entity in real-time energy markets due to its flexible regulation capability. However, traditional methods face computational inefficiency in high-dimensional bidding scenarios caused by expansive decision spaces, limiting online generation of multi-segment optimal quotation curves. This paper proposes a policy migration-based optimization framework for high-dimensional IRSP bidding: First, a real-time market clearing model with IRSP participation and an operational constraint-integrated bidding model are established. Second, we rigorously prove the monotonic mapping relationship between the cleared output and the real-time locational marginal price (LMP) under the market clearing condition and establish mathematical foundations for migrating the self-dispatch policy to the quotation curve based on value function concavity theory. Finally, a generalized inverse construction method is proposed to decompose the high-dimensional quotation curve optimization into optimal power response subproblems within price parameter space, substantially reducing decision space dimensionality. The case study validates the framework effectiveness through performance evaluation of policy migration for a wind-dual energy storage plant, demonstrating that the proposed method achieves 90% of the ideal revenue with a 5% prediction error and enables reinforcement learning algorithms to increase their performance from 65.1% to 84.2% of the optimal revenue. The research provides theoretical support for resolving the “dimensionality–efficiency–revenue” dilemma in high-dimensional bidding and expands policy possibilities for IRSP participation in real-time markets.

1. Introduction

In recent years, the renewable energy industry, mainly wind power and photovoltaic, has developed rapidly, and its installed capacity has been growing [1]. By 2024, China’s installed solar power capacity had reached 890 GW, up 45.2% year on year. And the installed capacity of wind power had reached 520 GW, an increase of 18.0% [2]. However, the inherent volatility and intermittency of wind power and photovoltaics pose significant challenges to the safe and stable operation of power grids [3]. In this context, energy storage systems, with their bidirectional power regulation capabilities and rapid response characteristics, have become a key solution to smooth the fluctuations of renewable energy sources and enhance the flexibility of the grid [4].

With the continuous advancement of China’s electricity market reform, the scale of its electricity market continues to expand. In 2024, China’s market-traded electricity will reach 6.1 trillion kWh, accounting for 63% of the whole society’s electricity consumption [5]. As an important part of the electricity spot market, the short-term clearing mechanism of the real-time energy market can reflect the real-time supply and demand changes and marginal cost of the system and provide a highly volatile revenue window for flexible resources [6]. Although China’s real-time energy market is still in the pilot stage and relatively small in scale compared with the pioneer countries [7], benefiting from the maturity of simulation [8,9] and control [10,11] technology in transmission and distribution, it has considerable development potential.

IRSP as a new type of clean energy power generation system has significant advantages such as high energy conversion efficiency, good operational flexibility, and superior comprehensive economy [12,13]. Through the cooperative optimization control of renewable energy and an energy storage system, IRSP can realize a wide range of power adjustments and a fast response capability [14]. This enables IRSP to track real-time price fluctuations, as well as power demand, making it ideal for participating in real-time energy markets.

The existing research is extensive on the issue of quotation strategies for renewable energy-storage systems to participate in the energy market. In terms of problem modeling, it can be divided into two main categories. The first category regards the research object as the price taker and ignores the impact of bidding on the LMPs. Reference [15] studies the bidding strategies of virtual power plants including wind power and energy storage in the day-ahead and real-time market, analyzing the impact of LMP on bidding behavior from the perspective of price taker. Reference [16] studies the market optimal policy of a distributed energy storage system. The LMP is regarded as an exogenous variable. The second type of pricing problem is modeled as a Stackelberg game between operators and the market. In [17], a three-layer Stackelberg game model of energy pricing is constructed on the two-stage model of a user-side wind–battery coupling system. In [18], a master–slave game model between energy storage operators and power generators is established. The optimal pricing policy under the condition of market equilibrium is analyzed. However, for independent operators, the market parameters and boundary conditions are often unknown, and these methods are mainly used for economic analysis or market equilibrium research, which makes it difficult to provide operators with online quotation strategies that can be directly applied.

In terms of solving methods, they can be mainly divided into three categories. The first method is based on stochastic programming, which focuses on the average performance under known probability distribution. Reference [19] studies the optimal quotation of a wind–heat–CAES system in the energy and reserve market, introducing CVaR into the objective function. Reference [20] formulates a two-stage model of a wind-pumped storage system in the market. The second method is based on robust optimization, which focuses on the performance guarantee of uncertain sets in the worst case. Reference [21] develops a mixed-integer linear programming model. An adjustable robust bidding policy is proposed for optimizing joint bidding of energy storage in both day-ahead and real-time markets. In [22], an adjustable robust bidding policy is proposed. The third kind of method is based on approximate dynamic programming represented by reinforcement learning, which determines the optimal action in each stage through a training value function. Reference [23] adopts stochastic dynamic programming to solve the optimal decision problem of a light-energy storage system. Reference [24] uses a deep reinforcement learning algorithm to develop the adaptive bidding policy of a new energy-energy storage system in the real-time market. In [25], a multi-agent reinforcement learning decision framework for PV-ESS participation in both day-ahead and real-time markets is proposed.

With the improvement of the refinement of the electricity market, the “quantity quotation” mode [26,27] has become the mainstream mechanism of the real-time energy market; that is, the combination of different electricity and corresponding prices is declared in the form of the quotation curve formed by quantity and price [28]. High-dimensional bidding refers to the situation where the dimensionality of the quantity and price pair is large, and usually the dimensionality of the decision-making space of “quantity-price” is not less than 20 [29]. High-dimensional bidding enables market players to respond to market fluctuations more flexibly, manage opportunity costs, optimize revenue and reduce risks, and enhance market game ability. However, the above research still has shortcomings when applied to the high-dimensional policy of IRSP participating in the real-time market. Some of the methods can provide the optimal output action of IRSP in self-scheduling mode or the optimal quotation action in pure price bidding mode, but the problem of “quantity quotation” is more complicated, resulting in the fact that these originally efficient online strategies cannot be directly migrated to the online setting of the multi-segment quotation curve. The other part of the method can be applied to the setting of the quotation curve, but in the requirement of high-dimensional bidding, the spatial dimension of decision variables is very high, and the solvability of the model limits the efficiency.

When dealing with the bidding problem of IRSPs in high-dimensional real-time markets, the aforementioned methods often have some shortcomings, as shown in

Table 1. Comparative analysis of methodologies.

Methodology	Key Characteristics	Limitations and Research Gaps
Stochastic Programming	Optimizes the expected outcome; provides optimal curve reference.	Offline; unsuitable for real-time sequential decisions.
Robust Optimization	Optimizes for the worst-case; ensures reliability.	Computationally conservative; not for real time.
Reinforcement Learning	Enables adaptive online learning and decision-making.	Sample inefficient; struggles with high-dimensional action spaces.
Model Predictive Control	Online optimization using a receding horizon.	Computational complexity prohibitive in high dimensions.
Proposed Framework	Designed for high-dimensional online decision-making.

. Stochastic programming [30] and robust programming [31] can provide a reference for the optimal bidding curve to some extent, but they cannot formulate the optimal action for each period online. Therefore, these two methods are more often used for bidding in the day-ahead market and do not match the real-time market. Although reinforcement learning methods can achieve online decision-making, they are currently mostly limited to low-dimensional bidding problems [32,33]. The difficulty in applying them to this problem lies in the high-dimensional action space corresponding to the high-dimensional bidding problem in the Markov decision problem, and the training of reinforcement learning requires high-dimensional action networks and exponentially increasing samples. Rolling optimization algorithms represented by model predictive control (MPC) [34] are widely used in the online optimization control problem of IPSP, but the multiple decision variables and constraints brought by high-dimensional problems reduce the solution efficiency of MPC. To the best of our current knowledge, no one has applied the MPC algorithm to the problem discussed in this paper. In conclusion, there is currently a lack of frameworks that can effectively address high-dimensional challenges and maintain compatibility with existing online optimization algorithms.

In view of the above problems, in order to overcome the “dimensionality–efficiency–benefit” triplex contradiction faced by IRSP operators. Based on policy migration, this paper proposes a framework for formulating the optimal offer for IRSP to participate in the high-dimensional bidding real-time market. The main contributions of this paper include the following:

(1) The bidding model of IRSP participating in the real-time energy market is constructed, and the rolling clearance mechanism on the market side and the operation constraint on the power station side are integrated into the optimization framework.

(2) The analytical relationship between cleared output and LMP under real-time market clearing conditions and the concavity of the IRSP bidding problem value function are proved, and the theoretical basis for constructing a high-dimensional quotation curve based on policy migration is established.

(3) The optimization problem of the high-dimensional quotation curve is mapped to the optimal response subproblem of the output in the LMP parameter space, and the optimal quotation framework of the IRSP participating in the real-time energy market is constructed.

The research contributes to sustainability in at least three aspects. In terms of energy sustainability, the effective operation of IRSPs can mitigate renewable curtailment, thereby increasing the utilization rate of renewable energy. This is conducive to reducing reliance on fossil energy and lowering carbon emissions. In terms of the sustainability of economic development, the improvement of the profitability of IRSPs in the real-time markets helps to enhance the economic feasibility of related projects and increase their investment attractiveness, adding market-driven force to the energy transition. In terms of the formulation of sustainability policies, the theoretical and simulation results can provide reference for policymakers, facilitating the design of more effective market rules and incentive mechanisms.

This paper is organized as follows. Section 2 introduces the bidding model of IRSP participating in the real-time energy market, and Section 3 proposes optimal bidding framework based on policy migration, Section 4 draws the case study. Section 5 draws the conclusions.

2. The Bidding Model of IRSP Participating in the Real-Time Energy Market

This paper assumes that IRSP is a commercial power station operating independently and participates in the real-time energy market in the form of “quantity quotation”. For bidding period $t_0$, the real-time energy market model with IRSP participation is as follows:

$$\begin{array}{l}\min \begin{array}{c}{\displaystyle \sum _{t=t_0}^{t_0+T^{\mathrm{N}}-1}\left({\displaystyle \sum _{k=1}^{n^{\mathrm{u}}}\mathit{p}{\mathit{r}}_k^{\mathrm{u}\mathrm{T}}{\mathit{P}}_{t,k}^{\mathrm{G},\mathrm{u}}+{\displaystyle \sum _{k=1}^{n^{\mathrm{p}}}\mathit{p}{\mathit{r}}_k^{\mathrm{p}\mathrm{T}}{\mathit{P}}_{t,k}^{\mathrm{S},\mathrm{p}} }}\right.}\\ \left.-{\displaystyle \sum _{k=1}^{n^{\mathrm{d}}}\mathit{p}{\mathit{r}}_k^{\mathrm{d}\mathrm{T}}{\mathit{P}}_{t,k}^{\mathrm{G},\mathrm{d}}- {\displaystyle \sum _{k=1}^{n^{\mathrm{n}}}\mathit{p}{\mathit{r}}_k^{\mathrm{n}\mathrm{T}}{\mathit{P}}_{t,k}^{\mathrm{S},\mathrm{n}}}}\right)\Delta t+M^{\mathrm{T}}(\mathit{S}{\mathit{L}}^{-}+\mathit{S}{\mathit{L}}^{+})\end{array}\\ \mathrm{s}.\mathrm{t}. 1^{\mathrm{T}}{\mathit{P}}_t^{\mathrm{G}}-1^{\mathrm{T}}{\mathit{P}}_t^{\mathrm{D}}+1^{\mathrm{T}}{\mathit{P}}_t^{\mathrm{PV}}+1^{\mathrm{T}}{\mathit{P}}_t^{\mathrm{W}}+1^{\mathrm{T}}{\mathit{P}}_t^{\mathrm{S},\mathrm{p}}-1^{\mathrm{T}}{\mathit{P}}_t^{\mathrm{S},\mathrm{n}}=0:{\mathsf{\lambda }}_t,\forall t\\ -{\mathit{P}}_t^{\mathrm{L}}+\mathit{S}{\mathit{L}}^{+}-\mathit{S}{\mathit{L}}^{-}\le -{\mathit{F}}^{\mathrm{L},\max }:{\mathsf{\mu }}_t^{-},\forall t\\ {\mathit{P}}_t^{\mathrm{L}}-\mathit{S}{\mathit{L}}^{+}+\mathit{S}{\mathit{L}}^{-}\le {\mathit{F}}^{\mathrm{L},\max }:{\mathsf{\mu }}_t^{+},\forall t\\ {\mathit{P}}_t^{\mathrm{L}}=H\left(A^{\mathrm{G}}{\mathit{P}}_t^{\mathrm{G}}-{\mathit{P}}_t^{\mathrm{D}}+A^{\mathrm{S}}{\mathit{P}}_t^{\mathrm{S},\mathrm{p}}-A^{\mathrm{S}}{\mathit{P}}_t^{\mathrm{S},\mathrm{n}}+A^{\mathrm{PV}}{\mathit{P}}_t^{\mathrm{PV}}+A^{\mathrm{W}}{\mathit{P}}_t^{\mathrm{W}}\right),\forall t\\ \mathit{S}{\mathit{L}}^{-}\ge 0,\mathit{S}{\mathit{L}}^{+}\ge 0\\ (2),(3)\\ \mathrm{together} \mathrm{with}\\ {\mathsf{\pi }}_{t_0}={\mathsf{\lambda }}_{t_0}\cdot 1/\Delta t-H^{\mathrm{T}}\left({\mathsf{\mu }}_{t_0}^{+}-{\mathsf{\mu }}_{t_0}^{-}\right)/\Delta t\end{array}$$

(1)

where $T^{\mathrm{N}}$ represents the time domain length of the real-time market rolling optimization; $n^{\mathrm{u}}$, $n^{\mathrm{d}}$, $n^{\mathrm{p}}$, and $n^{\mathrm{n}}$ represent the number of bidding segments of the conventional generator’s up and down output and the number of bidding segments of the IRSP’s positive and negative output; $\mathit{p}{\mathit{r}}_k^{\mathrm{u}}$, $\mathit{p}{\mathit{r}}_k^{\mathrm{d}}$, $\mathit{p}{\mathit{r}}_k^{\mathrm{p}}$, and $\mathit{p}{\mathit{r}}_k^{\mathrm{n}}$ represent the declared electricity price of each segment of the generator’s increased and reduced output and the declared electricity price of each segment of the IRSP’s positive and negative output; ${\mathit{P}}_{t,k}^{\mathrm{G},\mathrm{u}}$, ${\mathit{P}}_{t,k}^{\mathrm{G},\mathrm{d}}$, ${\mathit{P}}_{t,k}^{\mathrm{S},\mathrm{p}}$, and ${\mathit{P}}_{t,k}^{\mathrm{S},\mathrm{n}}$ represent the bid power of each segment of the generator’s increased and reduced output and the bid power of each segment of the IRSP’s positive and negative output; $\mathit{S}{\mathit{L}}^{-}$ and $\mathit{S}{\mathit{L}}^{+}$ represent the negative relaxation factor and positive relaxation factor of the power flow capacity constraint; $M$ is the penalty weight; and $\Delta t$ is the time slot. In this paper, 15 min is taken; that is, the real-time market will organize a bid and clearance every 15 min. The real-time active power values ${\mathit{P}}_t^{\mathrm{G}}$, ${\mathit{P}}_t^{\mathrm{D}}$, ${\mathit{P}}_t^{\mathrm{PV}}$, ${\mathit{P}}_t^{\mathrm{W}}$, ${\mathit{P}}_t^{\mathrm{S},\mathrm{p}}$, ${\mathit{P}}_t^{\mathrm{S},\mathrm{n}}$, and ${\mathit{P}}_t^{\mathrm{L}}$ represent the generator output, load demand, PV output, wind power output, IRSP positive power, IRSP negative power, and line flow. Among them, the output of the generator and IRSP is determined by market clearing; load demand, PV output, and wind power output come from ultra-short-term forecasts, which fluctuate around the results of the previous day’s bidding. ${\mathit{F}}^{\mathrm{L},\max }$, $H$, $A^{\mathrm{G}}$, $A^{\mathrm{S}}$, $A^{\mathrm{PV}}$, and $A^{\mathrm{W}}$ represent the upper limits of the line power flow, the power flow transfer distribution factor matrix and the index conversion matrix of the generator, IRSP, PV, and wind power. The goal of the real-time market is to minimize the cost of purchasing power. As renewable energy sources, wind and PV should be guaranteed priority clearance, so the associated power purchase costs do not appear in the objective function. The first constraint represents the balance of supply and demand, while the second and third constraints are the lower and upper flow constraints. The dual variables corresponding to these three constraints are ${\mathsf{\lambda }}_t$, ${\mathsf{\mu }}_t^{-}$, and ${\mathsf{\mu }}_t^{+}$, respectively, and their values in the period $t_0$ determine the corresponding real-time LMP ${\mathsf{\pi }}_{t_0}$. The fourth constraint is the power flow equilibrium constraint. The fifth constraint represents that the relaxation factor of the power flow capacity constraint is non-negative.

In addition, the real-time market model also needs to consider the operating constraints of the generator (2):

$$\begin{array}{l}{\mathit{P}}_t^{\mathrm{G}}={\mathit{P}}_t^{\mathrm{G},\mathrm{DA}}+{\mathit{P}}_t^{\mathrm{G},\mathrm{u}}-{\mathit{P}}_t^{\mathrm{G},\mathrm{d}},\forall t\\ -{\mathit{R}}^{\mathrm{G}}\le {\mathit{P}}_t^{\mathrm{G}}-{\mathit{P}}_{t-1}^{\mathrm{G}}\le {\mathit{R}}^{\mathrm{G}},\forall t\\ {\mathit{P}}^{\mathrm{G},\min }\le {\mathit{P}}_t^{\mathrm{G}}\le {\mathit{P}}^{\mathrm{G},\max },\forall t\\ 0\le {\mathit{P}}_{t,k}^{\mathrm{G},\mathrm{u}}\le ({\kappa }_{k+1}^{\mathrm{G},\mathrm{u}}-{\kappa }_k^{\mathrm{G},\mathrm{u}}){\mathit{P}}_t^{\mathrm{uG},\max },\forall t,\forall k=1,2,\dots ,n^{\mathrm{u}}\\ 0\le {\mathit{P}}_{t,k}^{\mathrm{G},\mathrm{d}}\le ({\kappa }_{k+1}^{\mathrm{G},\mathrm{d}}-{\kappa }_k^{\mathrm{G},\mathrm{d}}){\mathit{P}}_t^{\mathrm{dG},\max },\forall t,\forall k=1,2,\dots ,n^{\mathrm{d}}\\ {\displaystyle \sum _{k=1}^{n^{\mathrm{d}}}{\mathit{P}}_{t,k}^{\mathrm{G},\mathrm{d}}}={\mathit{P}}_t^{\mathrm{G},\mathrm{d}},{\displaystyle \sum _{k=1}^{n^{\mathrm{u}}}{\mathit{P}}_{t,k}^{\mathrm{G},\mathrm{u}}}={\mathit{P}}_t^{\mathrm{G},\mathrm{u}},\forall t\end{array}$$

(2)

where ${\mathit{P}}_t^{\mathrm{G},\mathrm{DA}}$, ${\mathit{P}}_t^{\mathrm{G},\mathrm{u}}$, ${\mathit{P}}_t^{\mathrm{G},\mathrm{d}}$, ${\mathit{R}}^{\mathrm{G}}$, ${\mathit{P}}_t^{\mathrm{uG},\max }$, ${\mathit{P}}_t^{\mathrm{dG},\max }$, ${\kappa }_k^{\mathrm{G},\mathrm{u}}$, ${\kappa }_k^{\mathrm{G},\mathrm{d}}$, ${\mathit{P}}^{\mathrm{G},\min }$, and ${\mathit{P}}^{\mathrm{G},\max }$ represent the output of the generator bid in the day-ahead market, the up-rated and down-rated output in the real-time market, the climbing rate, the declared maximum up-rated technical output and down-rated technical output, the output percentage boundary corresponding to each segment, and the lower limit and upper limit of the output. The first constraint means that the real-time output of the generator is jointly determined by the day-ahead cleared output and the adjusted cleared output in the real-time market. The second constraint is the climbing constraint of the generator. The third constraint is the output capacity constraint of the generator. The fourth constraint and the fifth constraint mean that the adjusted output of the bid in each segment of the generator should not exceed the corresponding boundary of that segment. The sixth constraint reflects the relationship between the segments and the sum of the adjusted output of the bid generator segment.

In addition, the real-time market model needs to consider the operational constraints of the IRSP. Since the movement of in-station storage and the power abandonment of renewable energy are flexible and adjustable, IRSPs are similar to virtual power plants and are equivalent to conventional units with varying output boundaries in the market:

$$\begin{array}{l}0\le {\mathit{P}}_t^{\mathrm{S},\mathrm{p}}\le {\mathit{P}}_t^{\mathrm{S},\mathrm{p},\max },0\le {\mathit{P}}_t^{\mathrm{S},\mathrm{n}}\le {\mathit{P}}_t^{\mathrm{S},\mathrm{n},\max },\forall t\\ 0\le {\mathit{P}}_{t,k}^{\mathrm{S},\mathrm{n}}\le ({\kappa }_{k+1}^{\mathrm{S},\mathrm{n}}-{\kappa }_k^{\mathrm{S},\mathrm{n}}){\mathit{P}}_t^{\mathrm{S},\mathrm{n},\max },\forall t,\forall k=1,2,\dots ,n^{\mathrm{n}}\\ 0\le {\mathit{P}}_{t,k}^{\mathrm{S},\mathrm{p}}\le ({\kappa }_{k+1}^{\mathrm{S},\mathrm{p}}-{\kappa }_k^{\mathrm{S},\mathrm{p}}){\mathit{P}}_t^{\mathrm{S},\mathrm{p},\max },\forall t,\forall k=1,2,\dots ,n^{\mathrm{p}}\\ {\displaystyle \sum _{k=1}^{n^{\mathrm{p}}}{\mathit{P}}_{t,k}^{\mathrm{S},\mathrm{p}}}={\mathit{P}}_t^{\mathrm{S},\mathrm{p}},{\displaystyle \sum _{k=1}^{n^{\mathrm{n}}}{\mathit{P}}_{t,k}^{\mathrm{S},\mathrm{n}}}={\mathit{P}}_t^{\mathrm{S},\mathrm{n}},\forall t\end{array}$$

(3)

where ${\mathit{P}}_t^{\mathrm{S},\mathrm{p},\max }$, ${\mathit{P}}_t^{\mathrm{S},\mathrm{n},\max }$, ${\kappa }_k^{\mathrm{S},\mathrm{p}}$, and ${\kappa }_k^{\mathrm{S},\mathrm{n}}$ are, respectively, the positive maximum technical output and negative maximum technical output declared by the IRSP and the output percentage boundary corresponding to the positive and negative segments. The first constraint means that the total cleared output of the IRSP should not exceed the declared output boundary, and the second constraint and the third constraint mean that the cleared output of each IRSP paragraph should not exceed the corresponding boundary of that paragraph. The fourth constraint reflects the relationship between the segments and summations of the awarded IRSP contributions.

For simplicity, this paper assumes that there is no IRSP participation in the day-ahead market:

$$\begin{array}{l}\min {\displaystyle \sum _{t=0}^{T_m}{\displaystyle \sum _{k=1}^{n^{\mathrm{ad}}}\mathit{p}{\mathit{r}}_k^{\mathrm{ad}\mathrm{T}}{\mathit{P}}_{t,k}^{\mathrm{G},\mathrm{ad}}}\Delta t+M^{\mathrm{T}}(\mathit{S}{\mathit{L}}^{-}+\mathit{S}{\mathit{L}}^{+})}\\ \mathrm{s}.\mathrm{t}. 1^{\mathrm{T}}{\mathit{P}}_t^{\mathrm{G},\mathrm{ad}}+1^{\mathrm{T}}{\mathit{P}}_t^{\mathrm{PV},\mathrm{ad}}+1^{\mathrm{T}}{\mathit{P}}_t^{\mathrm{W},\mathrm{ad}}-1^{\mathrm{T}}{\mathit{P}}_t^{\mathrm{D},\mathrm{ad}}=0:{\mathsf{\lambda }}_t^{\mathrm{ad}},\forall t\\ -{\mathit{P}}_t^{\mathrm{L},\mathrm{ad}}+\mathit{S}{\mathit{L}}^{+}-\mathit{S}{\mathit{L}}^{-}\le -{\mathit{F}}^{\mathrm{L},\min }:{\mathsf{\mu }}_t^{\mathrm{ad},-},\forall t\\ {\mathit{P}}_t^{\mathrm{L},\mathrm{ad}}-\mathit{S}{\mathit{L}}^{+}+\mathit{S}{\mathit{L}}^{-}\le {\mathit{F}}^{\mathrm{L},\max }:{\mathsf{\mu }}_t^{\mathrm{ad},+},\forall t\\ {\mathit{P}}_t^{\mathrm{L},\mathrm{ad}}=H\left(A^{\mathrm{G}}{\mathit{P}}_t^{\mathrm{G},\mathrm{ad}}-{\mathit{P}}_t^{\mathrm{D},\mathrm{ad}}+A^{\mathrm{PV}}{\mathit{P}}_t^{\mathrm{PV},\mathrm{ad}}+A^{\mathrm{W}}{\mathit{P}}_t^{\mathrm{W},\mathrm{ad}}\right),\forall t\\ \mathit{S}{\mathit{L}}^{-}\ge 0,\mathit{S}{\mathit{L}}^{+}\ge 0\\ 0\le {\mathit{P}}_t^{\mathrm{PV},\mathrm{ad}}\le {\mathsf{\xi }}_t^{\mathrm{PV},\mathrm{ad}},\forall t\\ 0\le {\mathit{P}}_t^{\mathrm{W},\mathrm{ad}}\le {\mathsf{\xi }}_t^{\mathrm{W},\mathrm{ad}},\forall t\\ -{\mathit{R}}^{\mathrm{G}}\le {\mathit{P}}_t^{\mathrm{G},\mathrm{ad}}-{\mathit{P}}_{t-1}^{\mathrm{G},\mathrm{ad}}\le {\mathit{R}}^{\mathrm{G}},\forall t\\ {\mathit{P}}^{\mathrm{G},\min }\le {\mathit{P}}_t^{\mathrm{G},\mathrm{ad}}\le {\mathit{P}}^{\mathrm{G},\max },\forall t\\ 0\le {\mathit{P}}_{t,k}^{\mathrm{G},\mathrm{ad}}\le ({\kappa }_{k+1}^{\mathrm{G},\mathrm{ad}}-{\kappa }_k^{\mathrm{G},\mathrm{ad}})({\mathit{P}}^{\mathrm{G},\max }-{\mathit{P}}^{\mathrm{G},\min }),\forall t,\forall k=1,2,\dots ,n^{\mathrm{ad}}\\ {\mathit{P}}^{\mathrm{G},\min }+{\displaystyle \sum _{k=1}^{n^{\mathrm{da}}}{\mathit{P}}_{t,k}^{\mathrm{G},\mathrm{ad}}}={\mathit{P}}_t^{\mathrm{G},\mathrm{ad}},\forall t\\ \mathrm{together} \mathrm{with}\\ {\mathsf{\pi }}_t^{\mathrm{ad}}={\mathsf{\lambda }}_t^{\mathrm{ad}}\cdot 1/\Delta t-H^{\mathrm{T}}\left({\mathsf{\mu }}_t^{\mathrm{ad},+}-{\mathsf{\mu }}_t^{\mathrm{ad},-}\right)/\Delta t,\forall t\end{array}$$

(4)

where $n^{\mathrm{ad}}$, $\mathit{p}{\mathit{r}}_k^{\mathrm{ad}}$, ${\mathit{P}}_{t,k}^{\mathrm{G},\mathrm{ad}}$, ${\mathit{P}}_t^{\mathrm{G},\mathrm{ad}}$, ${\mathit{P}}_t^{\mathrm{PV},\mathrm{ad}}$, ${\mathit{P}}_t^{\mathrm{W},\mathrm{ad}}$, ${\mathit{P}}_t^{\mathrm{D},\mathrm{ad}}$, ${\mathit{P}}_t^{\mathrm{L},\mathrm{ad}}$, ${\mathsf{\xi }}_t^{\mathrm{PV},\mathrm{ad}}$, ${\mathsf{\xi }}_t^{\mathrm{W},\mathrm{ad}}$, and ${\kappa }_k^{\mathrm{G},\mathrm{da}}$ represent the number of bidding segments of the day-ahead generator output, the declared price of each segment, the day-ahead cleared output, the total day-ahead cleared output, the day-ahead cleared output of photovoltaic and wind power, the day-ahead load forecast, the active power flow, the day-ahead of the PV and wind power forecast, and the output percentage boundary corresponding to each segment of the generator. The first constraint is the supply and demand balance constraint, the second to fifth constraints represent the power flow constraint, and the eighth to eleventh constraints represent the generator constraint. These constraints are similar to the real-time market. The sixth and seventh constraints are the boundary constraints representing the cleared output of renewable energy. The day-ahead LMP ${\mathsf{\pi }}_t^{\mathrm{ad}}$ is determined by the dual variables ${\mathsf{\lambda }}_t^{\mathrm{ad}}$, ${\mathsf{\mu }}_t^{\mathrm{ad},-}$, and ${\mathsf{\mu }}_t^{\mathrm{ad},+}$ of the first three constraints.

The structure of the IRSP is shown in Figure 1, which contains $N^{\mathrm{S}}$ energy storage units. For simplicity, consider the multiple renewable energy units that may exist within it as a whole. The IRSP is treated as a single entity in the market, which means that its operator has a unified scheduling authority for each unit within the unit, and autonomously distributes the power of each storage unit and renewable energy. The public bus of an IRSP is connected to the grid via a grid-connected transmission line, which has a limited capacity.

The power of the IRSP is flexible, and the quotation curve of each period affects the clearing results of the real-time market, which further affects the revenue of each period. Therefore, for operators, there exists the problem of the optimal online quotation policy $f_0$. For any bidding period $t_0$, the goal is to find the optimal quotation curve to maximize the revenue expectation of the future period in the running cycle:

$$\begin{array}{l}{f}_0:\underset{{\varphi }_t^{\mathrm{p}},{\varphi }_t^{\mathrm{n}}}{\max }\mathbb{E}[{\displaystyle \sum _{t=t_0}^{T_m}{\pi }_t{P}_t^{\mathrm{S}}\Delta t}]+R_{t_0}\\ s.t.0\le P_t^{\mathrm{R}}\le {\xi }_t^{\mathrm{R}}\\ {\mathit{E}}_t={\mathit{E}}_{t-1}-{\mathsf{\eta }}^{-1}{\mathit{P}}_t^{\mathrm{dis}}\Delta t+\mathsf{\eta }{\mathit{P}}_t^{\mathrm{ch}}\Delta t\\ 0\le {\mathit{E}}_t\le {\mathit{E}}^{\max }\\ 0\le {\mathit{P}}_t^{\mathrm{dis}}\le {\mathit{P}}^{\mathrm{dis},\max }\\ 0\le {\mathit{P}}_t^{\mathrm{ch}}\le {\mathit{P}}^{\mathrm{ch},\max }\\ P_t^{\mathrm{S}}=P_t^{\mathrm{R}}+1.{\mathit{P}}_t^{\mathrm{dis}}-1.{\mathit{P}}_t^{\mathrm{ch}}\\ -P_t^{\mathrm{S},\mathrm{n},\max }\le P_t^{\mathrm{S}}\le P_t^{\mathrm{S},\mathrm{p},\max }\\ P_t^{\mathrm{S},\mathrm{p},\max }=\min (1.{\mathit{P}}^{\mathrm{dis},\max }+{\xi }_t^{\mathrm{R}},1.\mathsf{\eta }{\mathit{E}}_t/\Delta t+{\xi }_t^{\mathrm{R}},P^{\mathrm{Tr}})\\ P_t^{\mathrm{S},\mathrm{n},\max }=\min (1.{\mathit{P}}^{\mathrm{ch},\max },1.{\mathsf{\eta }}^{-1}({\mathit{E}}^{\max }-{\mathit{E}}_t)/\Delta t,P^{\mathrm{Tr}})\\ P_t^{\mathrm{S}}=P_t^{\mathrm{S},\mathrm{p}}-P_t^{\mathrm{S},\mathrm{n}}\\ ({[p{r}_k^{\mathrm{p}}]}_t)={\varphi }_t^{\mathrm{p}}(P_t^{\mathrm{S},\mathrm{p}})\\ ({[p{r}_k^{\mathrm{n}}]}_t)={\varphi }_t^{\mathrm{n}}(P_t^{\mathrm{S},\mathrm{n}})\\ (1),(2),(3),(4)\\ t_0\le t\le T_m\end{array}$$

(5)

where $P_t^{\mathrm{S}}$, $T_m$, ${\varphi }_t^{\mathrm{p}}$, ${\varphi }_t^{\mathrm{n}}$, $P_t^{\mathrm{R}}$, ${\xi }_t^{\mathrm{R}}$, ${\mathit{E}}_t$, $\mathsf{\eta }$, ${\mathit{P}}_t^{\mathrm{dis}}$, ${\mathit{P}}_t^{\mathrm{ch}}$, ${\mathit{E}}^{\max }$, ${\mathit{P}}^{\mathrm{dis},\max }$, ${\mathit{P}}^{\mathrm{ch},\max }$, and $P^{\mathrm{Tr}}$ represent the net output of the IRSP, the total period of the operating cycle, the positive output quotation curve and negative output quotation curve declared by the IRSP, the actual output of renewable energy, the ultra-short-term forecast value of the full power of renewable energy, the energy of each energy storage unit, the efficiency matrix of each energy storage unit, the discharge power and charging power of each energy storage unit, the energy capacity of each energy storage unit, the maximum discharge power and charging power of each energy storage unit, and the transmission line capacity. The first constraint is the capacity constraint of renewable energy. The second constraint is the energy transfer constraint of storage. The third constraint is the energy capacity constraint of energy storage. The fourth and fifth constraints are the power capacity constraints of energy storage. The sixth constraint is the power balance constraint of the common bus. The seventh constraint is the net output boundary constraint of the IRSP. The eighth constraint means that the positive maximum technical output is determined by the ultra-short-term forecast power of renewable energy, the maximum discharge power of storage, the energy stored, and the capacity of the grid-connected transmission line. The ninth constraint means that the negative maximum technical output is jointly determined by the maximum charging power of the storage, the energy stored, and the capacity of the transmission line of the grid-connected line. The tenth constraint represents the relationship between the net IRSP output and the cleared positive and negative output. The 11th and 12th constraints are the quotation curve constraints. ${\varphi }_t^{\mathrm{p}}$ is a monotonically non-decreasing stepped curve, the independent variable is the cleared positive output of the IRSP, ranging from 0 to the maximum positive technical output, and the dependent variable is the lowest acceptable price of the power station. ${\varphi }_t^{\mathrm{n}}$ is a monotonically non-increasing stepped curve, the independent variable is the cleared negative output of the IRSP, the range is 0 to the negative maximum technical output, and the dependent variable is the lowest acceptable price of the power station. In actual declaration, according to the number of declaration segments and the corresponding percentage boundary of each segment, the quotation curve is discretized into a multi-segment electricity price sequence ${[p{r}_k^{\mathrm{p}}]}_t$ and ${[p{r}_k^{\mathrm{n}}]}_t$ for declaration. The independent variable $R_{t_0}$ is a constant, including the cumulative net income obtained by the IRSP due to charge and discharge between the start period of the operation cycle and the $t_0-1$ period:

$$R_{t_0}={\displaystyle \sum _{t=0}^{t_0-1}{\pi }_t{P}_t^{\mathrm{S}}\Delta t}$$

(6)

There are many limitations in solving the problem $f_0$. In addition to the limitation that the market boundary and parameters are unknown to independent IRSP operators, more importantly, in the scenario of high-dimensional bidding, operators need to formulate the optimal multi-segment quotation and the corresponding percentage boundary online. Such high-dimensional optimization limits the efficiency of the solution algorithm.

3. Optimal Bid Framework Based on Policy Migration

To address the difficulties caused by high-dimensional optimization mentioned above, this section proposes an optimal offer framework based on policy migration in “autonomous decision” mode, which significantly reduces the dimensions of action space.

First, we propose the clearing conditions for IRSP participation in high-dimensional bidding. Under this condition, the cleared output of the IRSP in the real-time market can be regarded as a function of the LMP, which is an inevitable result of the physical meaning of the market clearing rule and the quotation curve. As shown in Theorem 1,

Theorem 1.

When the number of declared segments tends to infinity, the cleared output of each IRSP in the $t_0$ period and the corresponding LMP meet

$${\mathit{P}}_{t_0}^{\mathrm{S},\mathrm{p}}({\mathsf{\pi }}_{t_0})=\sup \left\{\mathit{x}\right|{\mathsf{\varphi }}_{t_0}^{\mathrm{p}}(\mathit{x})\le {\mathsf{\pi }}_{t_0}\},{\mathit{P}}_{t_0}^{\mathrm{S},\mathrm{n}}({\mathsf{\pi }}_{t_0})=\inf \{\mathit{x}|{\mathsf{\varphi }}_{t_0}^{\mathrm{n}}(\mathit{x})\le {\mathsf{\pi }}_{t_0}\}$$

(7)

Proof. 

Assume the research object is the IRSP $j$ connected to the bus $i$ with a positive bid.

From the KKT conditions of (1), we have

$${\displaystyle \frac{\partial \mathcal{L}}{\partial P_{t,k,j}^{\mathrm{S},\mathrm{p}}}}=p{r}_{k,j}^{\mathrm{p}}\mathsf{\Delta }t-{\lambda }_t+{[H^{\mathrm{T}}\left({\mathsf{\mu }}_t^{+}-{\mathsf{\mu }}_t^{-}\right)]}_i+{\nu }_{t,k,j}^{+}-{\nu }_{t,k,j}^{-}=0$$

(8)

where $\mathcal{L}$ is the Lagrangian function and ${\nu }_{t,k,j}^{+}$ and ${\nu }_{t,k,j}^{-}$ are the dual variables associated with the constraint $0\le {\mathit{P}}_{t,k}^{\mathrm{S},\mathrm{p}}\le ({\kappa }_{k+1}^{\mathrm{S},\mathrm{p}}-{\kappa }_k^{\mathrm{S},\mathrm{p}}){\mathit{P}}_t^{\mathrm{S},\mathrm{p},\max }$.

Based on the definition of LMP in (1), we have

$${\pi }_{t,i}\mathsf{\Delta }t={\lambda }_t-{[H^{\mathrm{T}}\left({\mathsf{\mu }}_t^{+}-{\mathsf{\mu }}_t^{-}\right)]}_i$$

(9)

Combining these equations yields

$$(p{r}_{k,j}^{\mathrm{p}}-{\pi }_{t,i})\mathsf{\Delta }t={\nu }_{t,k,j}^{-}-{\nu }_{t,k,j}^{+}$$

(10)

Let $\mathsf{\Delta }{P}_{k,j}^{\mathrm{S},\mathrm{p}}\triangleq ({\kappa }_{k+1,j}^{\mathrm{S},\mathrm{p}}-{\kappa }_{k,j}^{\mathrm{S},\mathrm{p}}){\mathit{P}}_{t,j}^{\mathrm{S},\mathrm{p},\max }$. According to the complementary slackness conditions,

$$P_{t,k,j}^{\mathrm{S},\mathrm{p}}\cdot {\nu }_{t,k,j}^{-}=0,\left(\mathsf{\Delta }{P}_{k,j}^{\mathrm{S},\mathrm{p}}-P_{t,k,j}^{\mathrm{S},\mathrm{p}}\right)\cdot {\nu }_{t,k,j}^{+}=0.$$

(11)

Thus

a. When $p{r}_{k,j}^{\mathrm{p}}-{\pi }_{t,i}>0$, we have ${\nu }_{t,k,j}^{-}>0$ and $P_{t,k,j}^{\mathrm{S},\mathrm{p}}=0$.

b. When $p{r}_{k,j}^{\mathrm{p}}-{\pi }_{t,i}<0$, we have ${\nu }_{t,k,j}^{+}>0$ and $P_{t,k,j}^{\mathrm{S},\mathrm{p}}=\mathsf{\Delta }{P}_{k,j}^{\mathrm{S},\mathrm{p}}$.

Therefore, the cleared output $P_{t,j}^{\mathrm{S},\mathrm{p}}$ satisfies (12)

$${\displaystyle \sum }_{k-1}\mathsf{\Delta }{P}_{k,j}^{\mathrm{S},\mathrm{p}}\le P_{t,j}^{\mathrm{S},\mathrm{p}}\le {\displaystyle \sum }_k\mathsf{\Delta }{P}_{k,j}^{\mathrm{S},\mathrm{p}},k\ge 1,p{r}_{k,j}^{\mathrm{p}}\le {\pi }_{t,i}$$

(12)

Note that

$${\varphi }_{t,j}^{\mathrm{p}}(x)={\displaystyle \sum }_{k=1}^{n^{\mathrm{p}}}p{r}_{k,j}^{\mathrm{pT}}\cdot 1({\displaystyle \sum }_{k-1}\mathsf{\Delta }{P}_{k,j}^{\mathrm{S},\mathrm{p}}<x\le {\displaystyle \sum }_k\mathsf{\Delta }{P}_{k,j}^{\mathrm{S},\mathrm{p}})$$

(13)

Hence,

$${\displaystyle \sum }_k\mathsf{\Delta }{P}_{k,j}^{\mathrm{S},\mathrm{p}}=\sup \{x|{\mathsf{\varphi }}_{t,j}^{\mathrm{p}}(x)\le {\pi }_{t,i}\},k\ge 1,p{r}_{k,j}^{\mathrm{p}}\le {\pi }_{t,i}$$

(14)

When $n^{\mathrm{p}}\to \infty$ and $\mathsf{\Delta }{P}_{k,j}^{\mathrm{S},\mathrm{p}}\to 0$, applying (12) and the squeeze theorem gives

$$P_{t,j}^{\mathrm{S},\mathrm{p}}=\sup \left\{x\right|{\mathsf{\varphi }}_{t,j}^{\mathrm{p}}(x)\le {\pi }_{t,i}\}$$

(15)

Since $j$ and $t$ are arbitrary, the above conclusion holds for all IRSPs in time period $t_0$:

$${\mathit{P}}_{t_0}^{\mathrm{S},\mathrm{p}}({\mathsf{\pi }}_{t_0})=\sup \left\{\mathit{x}\right|{\mathsf{\varphi }}_{t_0}^{\mathrm{p}}(\mathit{x})\le {\mathsf{\pi }}_{t_0}\}$$

(16)

Similarly, when the cleared output is negative, the same reasoning applies, yielding

$${\mathit{P}}_{t_0}^{\mathrm{S},\mathrm{n}}({\mathsf{\pi }}_{t_0})=\inf \left\{\mathit{x}\right|{\mathsf{\varphi }}_{t_0}^{\mathrm{n}}(\mathit{x})\le {\mathsf{\pi }}_{t_0}\}$$

(17)

□

When the IRSP capacity is small, we can approximate that bidding has almost no effect on the LMP distribution [29], that is, the real-time LMP is exogenous. At the same time, we approximately believe that the clearance condition under high-dimensional bidding is valid, and the problem $f_0$ at this time is transformed into

$$\begin{array}{l}{f}_1:\underset{{\varphi }_t^{\mathrm{p}},{\varphi }_t^{\mathrm{n}}}{\max }\underset{{\pi }_t~{\sigma }_t}{\mathbb{E}}[{\displaystyle \sum _{t=t_0}^{T_m}{\pi }_t{P}_t^{\mathrm{S}}({\pi }_t)\Delta t}]+R_{t_0}\\ s.t.0\le P_t^{\mathrm{R}}\le {\xi }_t^{\mathrm{R}}\\ {\mathit{E}}_t={\mathit{E}}_{t-1}-{\mathsf{\eta }}^{-1}{\mathit{P}}_t^{\mathrm{dis}}\Delta t+\mathsf{\eta }{\mathit{P}}_t^{\mathrm{ch}}\Delta t\\ 0\le {\mathit{E}}_t\le {\mathit{E}}^{\max }\\ 0\le {\mathit{P}}_t^{\mathrm{dis}}\le {\mathit{P}}^{\mathrm{dis},\max }\\ 0\le {\mathit{P}}_t^{\mathrm{ch}}\le {\mathit{P}}^{\mathrm{ch},\max }\\ P_t^{\mathrm{S}}=P_t^{\mathrm{R}}+1.{\mathit{P}}_t^{\mathrm{dis}}-1.{\mathit{P}}_t^{\mathrm{ch}}\\ -P_t^{\mathrm{S},\mathrm{n},\max }\le P_t^{\mathrm{S}}\le P_t^{\mathrm{S},\mathrm{p},\max }\\ P_t^{\mathrm{S},\mathrm{p},\max }=\min (1.{\mathit{P}}^{\mathrm{dis},\max }+{\xi }_t^{\mathrm{R}},1.\mathsf{\eta }{\mathit{E}}_t/\Delta t+{\xi }_t^{\mathrm{R}},P^{\mathrm{Tr}})\\ P_t^{\mathrm{S},\mathrm{n},\max }=\min (1.{\mathit{P}}^{\mathrm{ch},\max },1.{\mathsf{\eta }}^{-1}({\mathit{E}}^{\max }-{\mathit{E}}_t)/\Delta t,P^{\mathrm{Tr}})\\ P_t^{\mathrm{S}}=P_t^{\mathrm{S},\mathrm{p}}-P_t^{\mathrm{S},\mathrm{n}}\\ P_t^{\mathrm{S},\mathrm{p}}=\sup \left\{x\right|{\varphi }_t^{\mathrm{p}}(x)\le {\pi }_t\}\\ P_t^{\mathrm{S},\mathrm{n}}=\inf \left\{x\right|{\varphi }_t^{\mathrm{n}}(x)\le {\pi }_t\}\\ t_0\le t\le T_m\end{array}$$

(18)

where ${\sigma }_t$ is the distribution of the real-time LMP, which depends on the distribution of the market boundary and parameters and has nothing to do with the bidding of the IRSP.

Aiming at the optimal online quotation policy problem $f_1$ in the mode of “quantity quotation”, the corresponding optimal online decision problem $f_2$ in the mode of “autonomous decision” is proposed:

$$\begin{array}{l}{f}_2:\underset{P_t({\mathit{E}}_t,{\pi }_t)}{\max }\underset{{\pi }_t~{\sigma }_t}{\mathbb{E}}[{\displaystyle \sum _{t=t_0}^{T_m}{\pi }_t{P}_t^{\mathrm{S}}\Delta t}]+R_{t_0}\\ s.t.0\le P_t^{\mathrm{R}}\le {\xi }_t^{\mathrm{R}}\\ {\mathit{E}}_t={\mathit{E}}_{t-1}-{\mathsf{\eta }}^{-1}{\mathit{P}}_t^{\mathrm{dis}}\Delta t+\mathsf{\eta }{\mathit{P}}_t^{\mathrm{ch}}\Delta t\\ 0\le {\mathit{E}}_t\le {\mathit{E}}^{\max }\\ 0\le {\mathit{P}}_t^{\mathrm{dis}}\le {\mathit{P}}^{\mathrm{dis},\max }\\ 0\le {\mathit{P}}_t^{\mathrm{ch}}\le {\mathit{P}}^{\mathrm{ch},\max }\\ P_t^{\mathrm{S}}=P_t^{\mathrm{R}}+1.{\mathit{P}}_t^{\mathrm{dis}}-1.{\mathit{P}}_t^{\mathrm{ch}}\\ -P_t^{\mathrm{S},\mathrm{n},\max }\le P_t^{\mathrm{S}}\le P_t^{\mathrm{S},\mathrm{p},\max }\\ P_t^{\mathrm{S},\mathrm{p},\max }=\min (1.{\mathit{P}}^{\mathrm{dis},\max }+{\xi }_t^{\mathrm{R}},1.\mathsf{\eta }{\mathit{E}}_t/\Delta t+{\xi }_t^{\mathrm{R}},P^{\mathrm{Tr}})\\ P_t^{\mathrm{S},\mathrm{n},\max }=\min (1.{\mathit{P}}^{\mathrm{ch},\max },1.{\mathsf{\eta }}^{-1}({\mathit{E}}^{\max }-{\mathit{E}}_t)/\Delta t,P^{\mathrm{Tr}})\\ P_t^{\mathrm{S}}=P_t({\mathit{E}}_t,{\pi }_t)\\ t_0\le t\le T_m\end{array}$$

(19)

“Autonomous decision” mode and “quantity quotation” mode have the same IRSP operating boundary and the same value function under the optimal policy. The difference between the two is that in the former mode, the operator can observe the real-time LMP ${\pi }_{t_0}$ of $t_0$ in the current decision-making period and all the historical LMPs (the future price is unknown) in advance and can independently decide the current cleared output $P_t^{\mathrm{S}}$. In the latter mode, the operator does not know the LMP of $t_0$ in advance, and the output is determined by the market clearing and cannot be directly determined. Next, Theorem 2 will be proved, which shows that the optimal policy in “autonomous decision” mode can obtain the optimal price curve in “quantity quote” mode solely through migration.

For simplicity’s sake, combine ${\varphi }_t^{\mathrm{p}}$ and ${\varphi }_t^{\mathrm{n}}$ at boundary 0 into a monotone, undiminished overall quote curve:

$${\varphi }_t(P_t^{\mathrm{S}})\left\{\begin{array}{c}{\varphi }_t^{\mathrm{p}}(P_t^{\mathrm{S}}),P_t^{\mathrm{S}}\ge 0\\ {\varphi }_t^{\mathrm{n}}(-P_t^{\mathrm{S}}),P_t^{\mathrm{S}}<0\end{array}\right.$$

(20)

It is not difficult to verify that the clearing condition is equivalent to

$$P_t^{\mathrm{S}}=\sup \left\{x\right|{\varphi }_t(x)\le {\pi }_t\}$$

(21)

To better prove the theorem, we first prove Lemma 1 by induction, which states that the value function of the original problem is a concave function of stored energy.

Lemma 1.

The optimal policy $P_t^{\mathrm{S}}=P_t^{*}({\mathit{E}}_t,{\pi }_t)$ for problem $f_2$ corresponds to a value function $V({\mathit{E}}_t,{\pi }_t)$ that is concave with respect to ${\mathit{E}}_t$.

Proof. 

When $t=T_m$, the value function is given by

$$V({\mathit{E}}_{T_m},{\pi }_{T_m})=\underset{x\in X({\mathit{E}}_{T_m},{\xi }_{T_m}^{\mathrm{R}})}{\max }{\pi }_{T_m}x\Delta t$$

(22)

where $X({\mathit{E}}_{T_m},{\xi }_{T_m}^{\mathrm{R}})$ denotes the feasible region.

To maximize revenue, the energy storage should discharge along the boundary of the feasible region. Thus,

$$\begin{array}{l}V({\mathit{E}}_{T_m},{\pi }_{T_m})\\ ={\pi }_{T_m}{P}_{T_m}^{\mathrm{S},\mathrm{p},\max }\Delta t\\ ={\pi }_{T_m}\min (1.{\mathit{P}}^{\mathrm{dis},\max }+{\xi }_{T_m}^{\mathrm{R}},1.\mathsf{\eta }{\mathit{E}}_{T_m}/\Delta t+{\xi }_{T_m}^{\mathrm{R}},P^{\mathrm{Tr}})\Delta t\end{array}$$

(23)

Since the min function is concave and the summation preserves concavity, $V({\mathit{E}}_{T_m},{\pi }_{T_m})$ is a concave function of ${\mathit{E}}_t$.

Assume that for time period $t+1$, given ${\pi }_t~{\sigma }_t$, $V({\mathit{E}}_{t+1},{\pi }_{t+1})$ is concave in ${\mathit{E}}_{t+1}$. Define

$$J_t(x,{\mathit{E}}_t,{\pi }_t)={\pi }_{t}x\Delta t+\underset{{\pi }_t~{\sigma }_t}{\mathbb{E}}[V({\mathit{E}}_{t+1},{\pi }_{t+1})]$$

(24)

Since the expectation over ${\pi }_{t+1}$ does not affect concavity and ${\pi }_{t}x\Delta t$ is linear, $J_t(x,{\mathit{E}}_t,{\pi }_t)$ is jointly concave in $(x,{\mathit{E}}_t)$.

The optimal policy is

$$P_t^{*}({\mathit{E}}_t,{\pi }_t)=\underset{x\in X({\mathit{E}}_t,{\xi }_t^{\mathrm{R}})}{\arg \max }{J}_t(x,{\mathit{E}}_t,{\pi }_t)$$

(25)

We have

$$V({\mathit{E}}_t,{\pi }_t)=\underset{x\in X({\mathit{E}}_t,{\xi }_t^{\mathrm{R}})}{\max }{J}_t(x,{\mathit{E}}_t,{\pi }_t)$$

(26)

Note that the feasible region $X({\mathit{E}}_t,{\xi }_t^{\mathrm{R}})$ is a convex set with respect to ${\mathit{E}}_t$. By the concavity preservation theorem for maximum function [35], we conclude $V({\mathit{E}}_t,{\pi }_t)$ is a concave function of ${\mathit{E}}_t$. □

On this basis, we can prove Theorem 2, which is divided into two steps. The first part proves that the price curve obtained by the optimal policy in the mode of “autonomous decision” is a monotonic and non-decreasing compliance price curve, and the second part proves that the price curve is the optimal price curve.

Theorem 2.

The generalized inverse function of the optimal policy $P_t^{\mathrm{S}}=P_t^{*}({\mathit{E}}_t,{\pi }_t)$ about ${\pi }_t$ for problem $f_2$ is the optimal quotation curve ${\varphi }_t^{*}(P_t^{\mathrm{S}})$ for problem $f_1$:

$${\varphi }_t^{*}(P_t^{\mathrm{S}})=\inf \left\{{\pi }_t\right|P_t^{*}({\mathit{E}}_t,{\pi }_t)\ge P_t^{\mathrm{S}}\}$$

(27)

Proof (Compliance).

Assume there exist ${\pi }_t^{-}>{\pi }_t$ such that

$$x^{-}=P_t^{*}({\mathit{E}}_t,{\pi }_t^{-})<P_t^{*}({\mathit{E}}_t,{\pi }_t)=x$$

(28)

Under the optimal policy, $P_t^{\mathrm{R}}$ attains the boundary value of the feasible region, leaving no room for adjustment. Therefore, since ${\mathit{E}}_{t+1}^{-}\ge {\mathit{E}}_t$, the optimal policy $P_t^{*}({\mathit{E}}_t,{\pi }_t)$ satisfies

$$\begin{array}{l}{J}_t({\pi }_t,x)={\pi }_{t}x\Delta t+\underset{{\pi }_t~{\sigma }_t}{\mathbb{E}}[V({\mathit{E}}_{t+1},{\pi }_{t+1})]\\ P_t^{*}({\mathit{E}}_t,{\pi }_t)=\underset{x\in X({\mathit{E}}_t,{\xi }_t^{\mathrm{R}})}{\arg \max }{J}_t({\pi }_t,x)\end{array}$$

(29)

Thus, the first-order optimality conditions are

$$\begin{array}{l}{\displaystyle \frac{\partial J_t({\pi }_t^{-},x^{-})}{\partial x^{-}}}={\pi }_t^{-}\Delta t+\underset{{\pi }_t~{\sigma }_t}{\mathbb{E}}[{\displaystyle \frac{\partial V({\mathit{E}}_{t+1}^{-},{\pi }_{t+1})}{\partial x^{-}}}]=0\\ {\displaystyle \frac{\partial J_t({\pi }_t,x)}{\partial x}}={\pi }_t\Delta t+\underset{{\pi }_t~{\sigma }_t}{\mathbb{E}}[{\displaystyle \frac{\partial V({\mathit{E}}_{t+1},{\pi }_{t+1})}{\partial x}}]=0\end{array}$$

(30)

Subtracting the first-order conditions gives

$$({\pi }_t^{-}-{\pi }_t)\Delta t=\underset{{\pi }_t~{\sigma }_t}{\mathbb{E}}[{\displaystyle \frac{\partial V({\mathit{E}}_{t+1},{\pi }_{t+1})}{\partial x}}-{\displaystyle \frac{\partial V({\mathit{E}}_{t+1}^{-},{\pi }_{t+1})}{\partial x^{-}}}]$$

(31)

By the lemma, $V({\mathit{E}}_t,{\pi }_t)$ is concave in ${\mathit{E}}_t$, implying that ${\displaystyle \frac{\partial V({\mathit{E}}_t,{\pi }_t)}{\partial {\mathit{E}}_t}}$ is non-increasing in ${\mathit{E}}_t$. From the relationships $P_t^{\mathrm{S}}=P_t^{\mathrm{R}}+1.{\mathit{P}}_t^{\mathrm{dis}}-1.{\mathit{P}}_t^{\mathrm{ch}}$ and ${\mathit{E}}_t={\mathit{E}}_{t-1}-{\mathsf{\eta }}^{-1}{\mathit{P}}_t^{\mathrm{dis}}\Delta t+\mathsf{\eta }{\mathit{P}}_t^{\mathrm{ch}}\Delta t$, we derive

$$\begin{array}{l}{\displaystyle \frac{\partial V({\mathit{E}}_t,{\pi }_t)}{\partial {\mathit{E}}_t}}\\ ={\displaystyle \frac{\partial V({\mathit{E}}_t,{\pi }_t)}{\partial x}}.{\displaystyle \frac{\partial (P_t^{\mathrm{R}}+1.{\mathit{P}}_t^{\mathrm{dis}}-1.{\mathit{P}}_t^{\mathrm{ch}})}{\partial {\mathit{E}}_t}}\\ =-1.(\mathsf{\eta }+{\mathsf{\eta }}^{-1}){\displaystyle \frac{\partial V({\mathit{E}}_t,{\pi }_t)}{\partial x}}\end{array}$$

(32)

Thus, ${\displaystyle \frac{\partial V({\mathit{E}}_t,{\pi }_t)}{\partial x}}$ decreases monotonically with ${\mathit{E}}_t$, leading to

$${\displaystyle \frac{\partial V({\mathit{E}}_{t+1}^{-},{\pi }_t)}{\partial x}}\ge {\displaystyle \frac{\partial V({\mathit{E}}_{t+1},{\pi }_t)}{\partial x}}$$

(33)

Therefore,

$$({\pi }_t^{-}-{\pi }_t)\Delta t=\underset{{\pi }_t~{\sigma }_t}{\mathbb{E}}[{\displaystyle \frac{\partial V({\mathit{E}}_{t+1},{\pi }_{t+1})}{\partial x}}-{\displaystyle \frac{\partial V({\mathit{E}}_{t+1}^{-},{\pi }_{t+1})}{\partial x^{-}}}]\le 0$$

(34)

However, because ${\pi }_t^{-}>{\pi }_t$, we have $({\pi }_t^{-}-{\pi }_t)\Delta t>0$.

This is a contradiction. Hence, the optimal policy $P_t^{*}({\mathit{E}}_t,{\pi }_t)$ is non-decreasing in ${\pi }_t$.

Since the monotonicity of the inverse function remains unchanged, ${\varphi }_t^{*}(P_t^{\mathrm{S}})$ is non-decreasing in $P_t^{\mathrm{S}}$—forming a compliant quotation curve. □

Proof (Optimality).

Assume there exists another optimal quotation curve ${\varphi }_t^1(P_t^{\mathrm{S}})\ne {\varphi }_t^{*}(P_t^{\mathrm{S}})$. We have

$$\underset{{\pi }_t~{\sigma }_t}{\mathbb{E}}[{\displaystyle \sum _t^{T_m}{\pi }_t{P}_t^1({\pi }_t)\Delta t}]>\underset{{\pi }_t~{\sigma }_t}{\mathbb{E}}[{\displaystyle \sum _t^{T_m}{\pi }_t{P}_t^2({\pi }_t)\Delta t}]$$

(35)

where $P_t^1({\pi }_t)$ and $P_t^2({\pi }_t)$ represent the cleared outputs under the clearing conditions corresponding to quotation curves ${\varphi }_t^1(P_t^{\mathrm{S}})$ and ${\varphi }_t^{*}(P_t^{\mathrm{S}})$, respectively, given by

$$\begin{array}{l}{P}_t^1({\pi }_t)=\sup \left\{x\right|{\varphi }_t^1(x)\le {\pi }_t\}\\ P_t^2({\pi }_t)=\sup \left\{x\right|{\varphi }_t^{*}(x)\le {\pi }_t\}\end{array}$$

(36)

Since the generalized inverse function of $P_t^{\mathrm{S}}=P_t^{*}({\mathit{E}}_t,{\pi }_t)$ is ${\varphi }_t^{*}(P_t^{\mathrm{S}})$, which is non-decreasing, it follows $P_t^{*}({\mathit{E}}_t,{\pi }_t)=P_t^2({\pi }_t)$.

Thus, we derive

$$\underset{{\pi }_t~{\sigma }_t}{\mathbb{E}}[{\displaystyle \sum _t^{T_m}{\pi }_t{P}_t^1({\pi }_t)\Delta t}]>\underset{{\pi }_t~{\sigma }_t}{\mathbb{E}}[{\displaystyle \sum _t^{T_m}{\pi }_t{P}_t^{*}({\mathit{E}}_t,{\pi }_t)\Delta t}]$$

(37)

Note that $P_t^{*}({\mathit{E}}_t,{\pi }_t)$ is the optimal policy of $f_2$, we have

$$\begin{array}{l}\underset{{\pi }_t~{\sigma }_t}{\mathbb{E}}[{\displaystyle \sum _t^{T_m}{\pi }_t{P}_t^{*}({\mathit{E}}_t,{\pi }_t)\Delta t}]\\ =\underset{P_t\in X({\mathit{E}}_t,{\xi }_t^{\mathrm{R}})}{\max }\underset{{\pi }_t~{\sigma }_t}{\mathbb{E}}[{\displaystyle \sum _t^{T_m}{\pi }_t{P}_t\Delta t}]\\ \ge \underset{{\pi }_t~{\sigma }_t}{\mathbb{E}}[{\displaystyle \sum _t^{T_m}{\pi }_t{P}_t^1({\pi }_t)\Delta t}]\end{array}$$

(38)

This leads to a contradiction. Therefore, ${\varphi }_t^{*}(P_t^{\mathrm{S}})$ is the optimal quotation curve. □

To sum up, the optimal bidding framework proposed in this section is as follows: for the bidding period $t_0$, the corresponding decision problem $f_2$ under the mode of “autonomous decision” is first constructed, and then the optimal policy of $f_2$ is formulated to obtain $P_{t_0}^{*}({\mathit{E}}_{t_0},{\pi }_{t_0})$. Finally, the generalized inverse function ${\varphi }_t^{*}(P_t^{\mathrm{S}})$ about ${\pi }_{t_0}$ is obtained from (27), which is the optimal quotation curve of this period. It is worth noting that $f_2$ is a typical online sequential decision problem. Due to the uncertainty of future LMPs and renewable energy output, the optimal policy is only an ideal situation, and generally cannot be obtained [36]. Therefore, a common practice is to use an efficient online algorithm as an alternative, and then migrate to obtain the corresponding suboptimal quotation curve, and the final performance is positively correlated with the performance of the used algorithm. Although the framework proposed in this paper does not involve specific online algorithms, it significantly reduces the dimensions of the decision variables by transforming the original problem of solving the high-dimensional optimal quotation curve into a subproblem of solving the optimal output under different price parameters, which is conducive to improving the computational efficiency of the relevant algorithms and increasing the possibility of adapting algorithms that could not originally handle high-dimensional problems.

4. Case Study

In order to verify the correctness of the theory proposed in this paper, a combined wind storage power station with typical parameters is taken as an IRSP to carry out a case study. The power station contains one wind power unit and two battery energy storage units, and the installed capacity of wind power is 40 MW. The power market selects the reconstructed IEEE 39-bus New England system, which includes 10 generator units, 4 centralized photovoltaic units, and 4 centralized wind turbines. The diagram and parameters of the system can be found in Appendix A. Among them, the wind power units are connected to BUS4, BUS12, BUS21, and BUS25, with a capacity of 1500 MW, and the photovoltaic units are connected to BUS12, BUS18, BUS28, and BUS39, with a capacity of 1500 MW. The relevant parameters of IRSP and the market are shown in

Table 2. Parameters of the IRSP and the market.

Parameters	Value	Parameters	Value
$\Delta t$	15 min	$T_m$	95
${\eta }_1$	0.95	${\eta }_2$	0.9
$E_1^{\max }$	40 MWh	$E_2^{\max }$	60 MWh
$P_1^{\mathrm{dis},\max }$	10 MW	$P_2^{\mathrm{dis},\max }$	30 MW
$P_1^{\mathrm{ch},\max }$	10 MW	$P_2^{\mathrm{ch},\max }$	30 MW
$T^{\mathrm{N}}$	8	$P^{\mathrm{Tr}}$	60 MW
$\mathrm{M}$	1 × 106	$n^{\mathrm{u}}$	5
$n^{\mathrm{ad}}$	10	$n^{\mathrm{n}}$	5

where the decision time slot is 15 min, and the time domain of real-time market optimization is the next two hours.

In this paper, the Gurobi solver based on Matlab R2019b is used to calculate the idealized offline optimal policy and realize the day-ahead market and real-time market clearing calculation. For the boundary data of the market, PV capacity factors and wind capacity factors come from the historical measured data of France [37,38], and the average MAPE error of the day-ahead forecast is 5% so as to generate the day-ahead forecast data and real-time output data of wind power and photovoltaic energy. The pre-forecast data and real-time data of load are based on the standard load of each node of the IEEE39 node calculation example and are generated according to the real pre-forecast data and real-time data of France, respectively [39]. The day-ahead market quotation curve of each generator set is constant and unchanged, the real-time market quotation curve takes the day-ahead LMP as the center to carry out stepped floating quotation, and the floating value of the quotation is also constant and unchanged. Figure 2 shows the real-time data for a certain year—in order, the wind power output in the IRSP station, the centralized PV output connected to BUS12, and the load of BUS39. The blue dashed lines represent the lower envelope, and the red dashed lines represent the upper envelope.

The IRSP was connected to BUS1, BUS16, BUS25, and BUS33, respectively; $n^{\mathrm{n}}=10$ and $n^{\mathrm{d}}=10$ were set; the real-time market model was solved; and the clearing simulation was carried out for the whole year. The LMP result is shown in Figure 3, and the cleared net output of the IRSP was shown in Figure 4. The quotation policy of the IRSP is a simple linear policy. The generation method of the quotation curve is as follows: the output is evenly divided into $n^{\mathrm{n}}$ segments from $P_t^{\mathrm{S},\mathrm{n},\max }$ to 0, and the corresponding quotation is equally divided into $n^{\mathrm{n}}$ segments from low price (100 CNY/MWh) to 0, thus generating the quotation curve ${\varphi }_t^{\mathrm{n}}$. ${\varphi }_t^{\mathrm{p}}$ is generated in a similar way, with borders from 0 to 800 CNY/MWh and from 0 to $P_t^{\mathrm{S},\mathrm{p},\max }$. After the bid net output is determined, the IRSP determines the internal power action in a way that minimizes energy loss.

Figure 3a shows the LMPs when the IRSP is connected to four nodes, respectively, and Figure 3b shows the LMPs when no IRSP is connected. It can be seen that the two graphs are basically the same, which indicates that the access of a small-capacity IRSP basically does not affect the distribution of LMPs and verifies the rationality of the hypothesis of exogenous price distribution.

Figure 4a shows the corresponding net cleared output when four nodes are connected to the IRSP, respectively, and Figure 4b shows the net cleared output derived from the LMP, quotation curve, and Formula (7). It can be seen that the two graphs are exactly the same, which further validates the correctness of the clearing condition proposed by Theorem 1.

In order to further verify the correctness of the optimal quotation framework proposed in this paper, the offline optimal policy that can accurately predict the future LMPs and wind power under ideal circumstances is taken as the optimal policy of problem $f_2$, and the optimal quotation curve is generated by policy migration. Figure 5 shows the offline optimal policy curve and the corresponding offer curve at 0:15 under the operational results of a typical day (2 January). It can be obtained that the maximum forward technical output declared by the IRSP at this time is 42.94 MW and the maximum reverse technical output is 30.38 MW, and the corresponding critical LMPs are 419.8 CNY/MWh and 327.3 CNY/MWh, respectively. The optimal policy curve and the corresponding quotation curve are monotonic, which confirms the compliance of the quotation framework proposed in this paper.

Since offline calculation represents the ideal situation, Figure 5b is the optimal solution of the quotation curve ${\varphi }_t(P_t^{\mathrm{S}})$ represented by Model (18). Under the proposed method, the original problem is transformed into finding the response of the optimal solution of Model (19) to the change in parameter ${\pi }_{t_0}$, which is a parameter linear programming problem. Thus, Figure 5a is obtained, and quotation curve is obtained by using the method of finding the generalized inverse. If Model (18) is directly solved, the original problem will be an optimization problem with a decision variable scale of $O((T_m-t_0+1)*(n^n+n^p))$, containing a number of nonlinear constraints, which is difficult to solve.

This demonstrates the simplifying effect of the “dimension reduction” method proposed in this paper. It is worth noting that this part is to verify the correctness and effectiveness of the method in an ideal condition where the uncertain quantities are known. Under actual conditions, the algorithm design directly facing the original problem will be more difficult, while the “dimension reduction” proposed framework can effectively accommodate typical online optimization algorithms. In the final part of the section, the MPC and reinforcement learning algorithms will be used for verification.

Figure 6 and Figure 7 show the impact of bidding segments on cumulative revenue and cleared output on a typical day (2 January), where $n^{\mathrm{all}}=n^{\mathrm{n}}+n^{\mathrm{p}}$ represents the total of the bidding segments of the positive and negative forces. The quotation curve ${\varphi }_t^{*}(P_t^{\mathrm{S}})$ obtained by offline optimal policy migration is adopted in all quotation strategies of the IRSP. Among them, the red dashed line in the Figure 6a represents the offline optimal revenue of the IRSP in “autonomous decision” mode, which is 85.18 thousand CNY. The blue line represents the change in the cumulative revenue with the bidding dimension when the IRSP uses ${\varphi }_t^{*}(P_t^{\mathrm{S}})$ as the quotation curve in “quantity quotation” mode. It can be seen that with the growth of the bidding dimension, the cumulative income under “quantity and quotation” mode continues to increase and converges to the optimal offline income under “autonomous decision-making” mode. This shows that in high-dimensional real-time markets, the optimal pricing curve in “quantity quotation” mode can be obtained by migrating the optimal policy in “autonomous decision-making” mode, which proves the correctness of the framework proposed in this paper. Figure 6b uses the 2-norm as the deviation index to show the difference between the cleared output of the IRSP in “quantity quotation” mode and the offline optimal action in “autonomous decision” mode. It can be seen that with the increase in bid dimension, the deviation of the two gradually converges to 0. Specifically, in Figure 7, the red line represents the offline optimal action in “autonomous decision” mode, while the blue line and the green line represent the bidding output of the bidding segments of 2 and 10, respectively. It can be seen that when the bidding segments are small, the flexibility of bidding is limited, and there is a large deviation between the cleared output and the optimal action except in the peak period, thus affecting the revenue. When the bidding segments are larger, the bias almost disappears. This shows that in the high-dimensional bidding market, after ${\varphi }_t^{*}(P_t^{\mathrm{S}})$ declaration, the cleared output converges to the offline optimal action in the mode of an “autonomous decision”.

Table 3. Comparison of typical daily revenue for different algorithms.

Typical Day Revenue (Thousand CNY/Day)	RL	RL + FP	MPC1 + FP	MPC2 + FP	OP
2-January	55.08	71.24	67.86	77.25	84.61
21-March	49.47	60.42	56.80	64.85	72.54
12-July	46.76	57.04	51.90	59.57	66.71
17-September	46.75	59.82	56.01	64.05	70.62

shows the typical daily revenue of the proposed framework applied to the reinforcement learning algorithm and the MPC algorithm, and the winter, spring, summer, and autumn are selected for testing on each day. The IRSP accesses BUS1, and $n_{\mathrm{all}}$ is 10.

(1) RL: Uses the classical deep deterministic policy gradient (DDPG) [40] algorithm directly for Model (18). The state vector includes the bidding period, the single point prediction of the new energy output in the station in the period, the declared positive maximum technical output and negative maximum technical output during the period, and the initial value of each stored energy in the period. The action vector is of 10 dimensions for the 10 prices of the quote curve of the corresponding period. The reward function includes the market payoff and the action out-of-bounds penalty. The sample pool size is 30,000, and the training is performed until convergence.

(2) RL + FP: Uses the classical DDPG algorithm for Model (19), then makes a quotation curve using the proposed framework. The state vector adds one dimension representing the node electricity price parameter of the bidding period, and the rest is unchanged. The action vector is one-dimensional for the optimal IRSP output under the electricity price parameters. The reward function and training method are the same as the previous algorithm.

(3) MPC1 + FP: Uses the MPC algorithm for Model (19), then makes a quotation curve using the proposed framework. The prediction domain is 4 h, and the MAPE error of the forecast of new energy and electricity price is 30%. It is used to simulate the situation with poor prediction conditions.

(4) MPC2 + FP: Uses the MPC algorithm for Model (19), then makes a quotation curve using the proposed framework. The prediction domain is 4 h, and the MAPE error of the forecast of new energy and electricity price is 5%. It is used to simulate the situation with good prediction conditions.

(5) OP: Offline optimal results.

It can be seen from Table 3 that the typical day in winter has the highest revenue of 84.61 thousand CNY/day, and the typical day in summer has the lowest revenue of 66.71 thousand CNY/day. The comparison of the data in the first two columns shows that the performance of reinforcement learning is significantly improved after applying the proposed framework. Taking 12 July as an example, it increased from 46.76 thousand CNY/day to 57.04 thousand CNY/day, that is, from 70.1% of the optimal revenue to 85.5%. It shows that the dimensionality reduction in the problem can effectively reduce the training difficulty of reinforcement learning, thereby improving the training effect. The data in the third and fourth columns illustrate that the more accurate the prediction model used, the higher the revenue, and when the prediction error is 5%, the revenue is generally about 90% of the optimal revenue. In general, the proposed framework can be effectively compatible with classical algorithms such as reinforcement learning and MPC, which enables these methods to be effectively applied under high-dimensional challenges, bringing considerable benefits to the IRSP.

5. Conclusions

Aiming at the problem of an IRSP participating in the online bidding of the high-dimensional real-time energy market, this paper proposes an optimization framework based on policy migration. The main research conclusions are as follows.

Firstly, in theory, it is proved that the optimal quotation curve of an IRSP can be obtained from the optimal output action curve of “autonomous decision” mode by generalized inverse operation.

Secondly, in terms of method, the framework proposed can transform the problem of high-dimensional quotation curve optimization into a subproblem of optimal output response in the price parameter space, avoiding the “dimensional disaster” faced by the traditional method.

Thirdly, the case study shows that with the bidding segments increasing, the application effect of the proposed framework improves, and the clearing result of the obtained ideal quotation curve gradually converges to the optimal solution. Numerical tests on a IEEE 39-bus system demonstrate it robustly achieves approximately 90% of the ideal revenue under realistic prediction errors (5% MAPE) and significantly enhances the performance of the DDPG algorithm by 15 percentage points.

These obtained results underscore the framework’s capability to effectively resolve the dimensionality–efficiency–revenue trilemma for IRSPs in high-dimensional real-time markets.

There may be some challenges in practical deployment, including the impact of prediction errors of new energy output on output boundary declaration, the handling of market rule changes in different regions, and communication delays in market participation. Future work will focus on specific types of IRSPs to improve the robustness of the algorithm in practice.