Bi-Level Optimization-Based Bidding Strategy for Energy Storage Using Two-Stage Stochastic Programming

Abstract

Energy storage will play an important role in the new power system with a high penetration of renewable energy due to its flexibility. Large-scale energy storage can participate in electricity market clearing, and knowing how to make more profits through bidding strategies in various types of electricity markets is crucial for encouraging its market participation. This paper considers differentiated bidding parameters for energy storage in a two-stage market with wind power integration, and transforms the market clearing process, which is represented by a two-stage bi-level model, into a single-level model using Karush–Kuhn–Tucker conditions. Nonlinear terms are addressed using binary expansion and the big-M method to facilitate the model solution. Numerical verification is conducted on the modified IEEE RTS-24 and 118-bus systems. The results show that compared to bidding as a price-taker and with marginal cost, the proposed mothod can bring a 16.73% and 13.02% increase in total market revenue, respectively. The case studies of systems with different scales verify the effectiveness and scalability of the proposed method.

1. Introduction

The massive penetration of renewable energy has brought new vitality into the power system, but has also caused problems in terms of stability and energy efficiency. As a representative of flexible resources, energy storage can play an important role in maintaining the stability of the power system and promoting the consumption of renewable energy [1]. In the electricity market, aggregated energy storage can not only directly provide energy for trading, but can also provide an energy reserve service to prevent problems such as power imbalance in the real-time market.

Energy storage, due to the different services it can provide to the power system, is capable of participating in various types of markets, such as the energy market, the reserve market, and the ancillary services market [2,3]. However, in traditional power markets, energy storage has typically been limited by its capacity and thus has mostly participated passively as a price-taker. In recent years, advancements in energy storage generation technology and the development of integrated configuration solutions have led to an increase in the scale of it. The growth has enabled energy storage to influence market clearing and price formation, gradually shifting its role from price-taker to price-maker. Developing effective strategies to maximize the profit of energy storage in these markets has become an issue worth studying.

To overcome these problems, we first equate the electricity market, including the market stage and operation stage, to a two-stage stochastic programming model to quantify the impact of uncertainty. Then, the market clearing process of the electricity market is represented by a bi-level model. In the upper level, energy storage maximizes their profits through different market bidding parameters, while the lower level completes market clearing to achieve a minimum total cost. The proposed bi-level model is difficult to solve directly so we convert it into a single-level model using KKT (Karush–Kuhn–Tucker) conditions, and finally turn it into a MILP (Mixed Integer Linear Programming) model through linearization steps, so that it can be handled with commercial solvers.

1.1. Related Works

The rapid growth of renewable energy sources in power systems has led to a greater need for flexibility [4]. As a result, the number of energy storage devices deployed and their maximum capacity are increasing rapidly [5]. Numerous studies have been conducted on coordinating the participation of energy storage resources in electricity market transactions. For example, Gomes et al. [6] considered the uncertainty of market prices and renewable energy output, and created a two-stage stochastic programming model to optimize bidding strategies for energy storage devices in the day-ahead market. In [7], a model for maximizing energy storage profits based on price forecasts was developed, where Zheng et al. differentiated between bidding strategies in the wholesale electricity market according to the SoC (State of Charge) of energy storage. Xie et al. [8] used a robust approach to maximize energy storage arbitrage by representing wind power output and electricity prices using predefined uncertainty sets. Additionally, Xu et al. [9] established a marginal cost model for battery energy storage participating in the electricity market, considering battery aging mechanisms and optimizing dispatch designs according to market prices. Similar studies have also been conducted by Shuai [10] and Tang [11], where the former optimized microgrid energy management and market strategies based on proposed dynamic stochastic algorithms integrating AC power flow and energy storage characteristics, while the latter employed data-driven reinforcement learning methods to assist energy storage providers in making bidding decisions in coupled multiple markets. These studies provide valuable insights for energy storage providers in decision-making and arbitrage within electricity market transactions. Nevertheless, the majority of these studies operate under the assumption of limited market influence by energy storage providers, treating them simply as price-takers and ignoring their influence over market clearing and price formation processes. In deterministic scenarios, the market clearing price is directly used for energy storage resource allocation, while in uncertain scenarios, the price is set as a variable parameter to simulate market stochastic processes for further optimization. These assumptions are reasonable when the capacity of energy storage devices is relatively small compared to the total system generation capacity. However, as the capacity of energy storage continues to increase with the continuous construction of storage stations and advances in storage aggregation technologies, its impact on market prices gradually becomes significant, necessitating a shift in role in the market from price-takers to price-makers. Based on this, this paper takes into account the active behavior of energy storage in the electricity market and studies the bidding strategy to obtain the maximum profit from the perspective of price-makers.

The bi-level model is often used in the electricity market to help power generators formulate strategic behaviors [12] and achieve the goal of maximizing revenue through parameter iteration and game processes between the upper and lower levels. Researchers in the electrical power system employ it in bidding strategy formulation and resource allocation processes for RES (Renewable Energy Source) aggregators [13], FTRs (Financial Transmission Rights) [14], thermal power [15], hydropower [16], virtual power plants [17], electric vehicles [18], and even more general virtual bidders [19]. Similarly, the bi-level model can also play an important role in helping energy storage providers formulate market strategies [20,21,22]. In related research, Smets et al. [23] combined a bi-level optimization model with recurrent neural networks to optimize the charging and discharging decisions and risk indicators of energy storage devices in the European balancing market. In [24], Xia et al. discussed arbitrage schemes for energy storage in the energy and primary frequency regulation markets considering the impact of battery degradation, established upon the bi-level model of energy storage and joint markets. Furthermore, the bi-level model is also utilized in determining the siting and investment scale of energy storage, such as the research by Hrvoje et al. in [25] and Nasrolahpour et al. in [26]. The transaction process of energy storage participating in both the energy market and the reserve market is affected by the uncertainty of renewable energy output. The decision-making for market arbitrage contains uncertain parameters, which are not extensively discussed in the above literature. Two-stage stochastic programming is commonly employed to address two-stage decision-making problems influenced by uncertainty [27,28,29]. To fill this research gap, this paper combines the bi-level model with two-stage stochastic programming to address the bidding strategy problem of energy storage participating in joint clearing markets. Specifically, a detailed comparison between our proposed method and existing research is shown in

Table 1. Literature review.

Ref. No.	Market Style	Price Acceptance Method	Uncertainty Consideration	Revenue Sources
[6]	Day-ahead	Price-taker	×	Energy
[7]	Day-ahead	Price-taker	✔	Energy
[8]	Day-ahead	Price-taker	×	Energy
[9]	Day-ahead	Price-taker	×	Energy&Reserve
[10]	Day-ahead&Real-time	Price-taker	×	Energy
[22]	Real-time	Price-maker	×	Energy
[23]	Day-ahead&Real-time	Price-taker	×	Energy
[24]	Day-ahead&Real-time	Price-maker	×	Energy
[25]	Day-ahead	Price-maker	✔	Energy
Our work	Day-ahead&Real-time	Price-maker	✔	Energy&Reserve

.

1.2. Our Contributions

The contributions of this paper are as follows:

1

Based on the day-ahead and real-time market timeline analysis and the types of services provided by energy storage, a trading framework for energy storage to participate in the day-ahead electricity market, day-ahead reserve market and real-time balancing market is established.

2

The uncertainty of wind power output is quantified using output scenarios to establish a two-stage stochastic optimization model. The bi-level model is equivalently transformed into a single-level model through Lagrangian duality and KKT conditions. Nonlinear terms within the model are linearized using binary expansion and the big-M method, thereby reducing the complexity of solving the problem.

3

The proposed method is tested on the modified IEEE RTS-24 and 118-bus systems. The impact of bidding strategies on energy storage market revenue is analyzed. Compared with participating in the market as a price-taker and bidding according to marginal cost, the method proposed in this paper can bring a 16.73% and 13.02% increase in total market revenue, respectively.

Section 2 establishes a two-stage bi-level model based on energy storage services and market timeline analysis. Section 3 introduces the model transformation steps and designs linearization methods for different nonlinear terms. Section 4 verifies the proposed method on the modified IEEE RTS-24 and 118-bus systems and analyzes the impact of bidding strategy, energy storage configuration, and linearization parameters. Finally, the conclusions are presented in Section 5.

2. Market Model

Energy storage contains many different forms, including battery storage, compressed-air storage, hydrogen energy storage, etc. When the energy storage scale is large enough, or through aggregation to meet market access conditions, it can participate in the electricity market as a price-maker. The diagram of aggregated energy storage is shown in Figure 1. It is worth noting that the energy storage studied in this article is large-capacity independent or aggregated energy storage mainly based on electrochemistry, and energy storage in other forms such as pumped storage is not considered for the time being.

Given the inherent flexibility of energy storage and the different services it can provide, a two-timescale market framework including the day-ahead and real-time markets is adopted. In the day-ahead market stage, the energy storage operator submits power bids and declares its rapidly dispatchable capacity as a spinning reserve. In the real-time market stage, leveraging its fast response and bidirectional ramping capability, the energy storage is called on from the committed reserve to correct renewable energy forecast output errors, and it is remunerated through real-time balancing payments. By dynamically allocating capacity and available power across these two time horizons, the energy storage operator can simultaneously capture energy-arbitrage and ancillary-service revenues. The overall scheduling and optimization framework is depicted in Figure 2.

Small-scale energy storage, constrained by limited market power, generally acts as a price-taker when it participates in electricity markets. By contrast, large-scale energy storage can exert a noticeable influence on the market-clearing process. Consequently, a bi-level optimization framework is adopted to capture the additional profit opportunities that arise from the mutual interaction between energy storage dispatch decisions and the market-clearing process.

In order to highlight the research focus and simplify the problem, several reasonable assumptions were made about the market model, which are detailed below:

1

The non-convex characteristics of thermal power generators, such as on/off states and fixed costs, are not considered.

2

The costs associated with reserves include reserve allocation costs and utilization costs. In this study, it is assumed that the former is obtained from the reserve balance equation, and the latter is consistent with the clearing price in the real-time stage, which refers to the European spot market reserve pricing scheme [30,31,32].

3

Wind power has no marginal cost. Energy storage charging and discharging efficiencies are consistent, and constraints on the rates of energy storage charging and discharging are not considered.

4

In the market, energy storage can engage in strategic bidding, while other market participants can only bid based on their marginal costs.

2.1. Upper-Level Model

Energy storage is located at the upper layer of the bi-level model. It aims to achieve the goal of maximizing market revenue by adjusting the bidding strategy, which includes three parts: day-ahead electricity market revenue, day-ahead reserve allocation revenue, and real-time balancing market revenue. The objective function is as follows:

$$\begin{array}{cc}\hfill & \begin{array}{cc}max& {\displaystyle \sum _t}\Delta t\left[{\lambda }_{nt}\left(Q_{bt}^{dis}-Q_{bt}^{chs}\right)-C_b\left(Q_{bt}^{dis}+Q_{bt}^{chs}\right)\right]\end{array}\hfill \\ \hfill & +\sum _t\Delta t\left[{\lambda }_{nt}^{up}\left(R_{bt}^{dis,up}+R_{bt}^{chs,up}\right)+{\lambda }_{nt}^{dn}\left(R_{bt}^{dis,dn}+R_{bt}^{chs,dn}\right)\right]\hfill \\ \hfill & +\sum _{t,\omega }{\pi }_{\omega }\Delta t·\left[\left({\lambda }_{nt\omega }+C_b\right)\left(R_{bt\omega }^{chs,up}+R_{bt\omega }^{dis,dn}\right)+\left({\lambda }_{nt\omega }-C_b\right)\left(R_{bt\omega }^{chs,dn}+R_{bt\omega }^{dis,up}\right)\right]\hfill \end{array}$$

(1)

Energy storage uses different strategic parameters to quote the electricity and reserve services it provides based on its marginal cost. The constraints that need to be met are as follows:

$$1\le k_{b1}\le \overline{k_{b1}}$$

(2)

$$1\le k_{b2}\le \overline{k_{b2}}$$

(3)

2.2. Lower-Level Model

The market operator is located at the lower-level of the model. Its goal is to ensure the maximum social welfare, that is, the minimum power generation cost, while completing the clearing of electrical energy and reserve. Since the power generation cost of wind generators in the market is not considered, the total cost is generated by energy storage and thermal generators, and both include the cost related to electrical energy and the reserve. The expression of its objective function is espressed as follows:

$$\begin{array}{cc}\hfill & \begin{array}{cc}min& {\displaystyle \sum _{b,t,\omega }}\Delta t·\left[k_{b1}{C}_b\left(Q_{bt}^{dis}-Q_{bt}^{chs}+R_{bt}^{chs,up}+R_{bt}^{dis,up}+R_{bt}^{chs,dn}+R_{bt}^{dis,dn}\right)\right]\end{array}\hfill \\ \hfill & +\sum _{b,t,\omega }{\pi }_{\omega }\Delta t·k_{b2}{C}_b·\left(R_{bt\omega }^{chs,up}-R_{bt\omega }^{chs,dn}+R_{bt\omega }^{dis,up}-R_{bt\omega }^{dis,dn}\right)\hfill \\ \hfill & +\sum _{i,t,\omega }{\pi }_{\omega }\Delta t·C_i\left(P_{it}+P_{it}^{up}-P_{it}^{dn}+P_{it\omega }^{up}-P_{it\omega }^{dn}\right)\hfill \end{array}$$

(4)

where the first item is the electricity cost and reserve deployment cost of energy storage in the day-ahead market, the second item is the reserve usage cost of energy storage in the real-time market, and the third item is the total cost of thermal generators in the day-ahead and real-time markets.

The constraints that the lower-level model needs to satisfy are as follows, where the parameters in the right-most brackets are the Lagrange multipliers corresponding to the constraints:

$$\sum _{i\in {\Psi }_n^I}{P}_{it}+\sum _{q\in {\Psi }_n^Q}{W}_{qt}+\sum _{b\in {\Psi }_n^B}\left(Q_{bt}^{dis}-Q_{bt}^{chs}\right)-\sum _{j\in {\Psi }_n^J}{L}_{jt}=\sum _{m\in {\Theta }_n}{B}_{nm}\left({\theta }_{nt}-{\theta }_{mt}\right):\left({\lambda }_{nt}\right),\forall n,t$$

(5)

$$\sum _{i\in {\Psi }_n^I}{p}_{it}^{up}+\sum _{b\in {\Psi }_n^B}\left(R_{bt}^{chs,up}+R_{bt}^{dis,up}\right)=R_t^{up}:\left({\lambda }_{nt}^{up}\right),\forall n,t$$

(6)

$$\sum _{i\in {\Psi }_n^I}{p}_{it}^{dn}+\sum _{b\in {\Psi }_n^B}\left(R_{bt}^{chs,dn}+R_{bt}^{dis,dn}\right)=R_t^{dn}:\left({\lambda }_{nt}^{dn}\right),\forall n,t$$

(7)

$$\begin{array}{cc}\hfill & \sum _{i\in {\Psi }_n^I}\left(p_{it\omega }^{up}-p_{it\omega }^{dn}\right)+\sum _{q\in {\Psi }_n^Q}\left(W_{qt\omega }-W_{qt}-W_{qt\omega }^{spill}\right)\hfill \\ \hfill & +\sum _{b\in {\Psi }_n^B}\left(R_{bt\omega }^{chs,up}-R_{bt\omega }^{chs,dn}+R_{bt\omega }^{dis,up}-R_{bt\omega }^{dis,dn}\right)\hfill \\ \hfill & =\sum _{m\in {\Theta }_n}{B}_{nm}\left[\left({\theta }_{nt\omega }-{\theta }_{mt\omega }\right)-\left({\theta }_{nt}-{\theta }_{mt}\right)\right]:\left({\lambda }_{nt\omega }\right),\forall n,t,\omega \hfill \end{array}$$

(8)

$$B_{nm}\left({\theta }_{nt}-{\theta }_{mt}\right)\le \overline{P_{nm}}:\left({\alpha }_{nm}\right),\forall m\in {\Theta }_n,\forall n,t$$

(9)

$$B_{nm}\left({\theta }_{nt\omega }-{\theta }_{mt\omega }\right)\le \overline{P_{nm}}:\left({\alpha }_{nm\omega }\right),\forall m\in {\Theta }_n,\forall n,t,\omega$$

(10)

$${\theta }_{1t}=0:\left({\sigma }_{1t}\right),\forall t$$

(11)

$${\theta }_{1t\omega }=0:\left({\sigma }_{1t\omega }\right),\forall t,\forall \omega$$

(12)

$$\underline{P_i}\le P_{it}\le \overline{P_i}:\left({\beta }_{it}^{min},{\beta }_{it}^{max}\right),\forall i,t$$

(13)

$$\underline{P_i}\le P_{it}+p_{it\omega }^{up}-p_{it\omega }^{dn}\le \overline{P_i}:\left({\beta }_{it\omega }^{min},{\beta }_{it\omega }^{max}\right),\forall i,t,\omega$$

(14)

$$p_{it\omega }^{up}\le \overline{p_{it}^{up}}:\left({\delta }_{it\omega }^{up}\right),\forall i,t,\omega$$

(15)

$$p_{it\omega }^{dn}\le \overline{p_{it}^{dn}}:\left({\delta }_{it\omega }^{dn}\right),\forall i,t,\omega$$

(16)

$$-R{D}_i\le P_{it}-P_{i,t-1}\le R{U}_i,\left({\gamma }_{it}^D,{\gamma }_{it}^U\right),\forall i,t$$

(17)

$$-R{D}_i\le P_{it}+p_{it\omega }^{up}-p_{it\omega }^{dn}-P_{i,t-1}-p_{i,t-1,\omega }^{up}+p_{i,t-1,\omega }^{dn}\le R{U}_i,\left({\gamma }_{it\omega }^D,{\gamma }_{it\omega }^U\right),\forall i,t$$

(18)

$$W_{qt}\le \overline{W_{qt}}:\left({\varphi }_{qt}\right),\forall q,t$$

(19)

$$W_{qt\omega }^{spill}\le W_{qt\omega }:\left({\psi }_{qt\omega }\right),\forall q,t,\omega$$

(20)

$$\begin{array}{cc}\hfill & \underline{So{C}_b}\le \Delta t\sum _{t^{\prime }=1}^t\left[\eta \left(Q_{bt}^{chs}-R_{bt}^{chs,up}+R_{bt}^{chs,dn}\right)-\left(Q_{bt}^{dis}+R_{bt}^{dis,up}-R_{bt}^{dis,dn}\right)/\eta \right]\hfill \\ \hfill & +So{C_b}^{init}:\left({\epsilon }_{bt}^{min}\right),\forall b,t\hfill \end{array}$$

(21)

$$\begin{array}{cc}\hfill & So{C_b}^{init}+\Delta t\sum _{t^{\prime }=1}^t\left[\eta \left(Q_{bt}^{chs}-R_{bt}^{chs,up}+R_{bt}^{chs,dn}\right)-\left(Q_{bt}^{dis}+R_{bt}^{dis,up}-R_{bt}^{dis,dn}\right)/\eta \right]\hfill \\ \hfill & \le \overline{So{C}_b}:\left({\epsilon }_{bt}^{max}\right),\forall b,t\hfill \end{array}$$

(22)

$$\begin{array}{cc}\hfill & \underline{So{C}_b}\le \Delta t\sum _{t^{\prime }=1}^t\left[\eta \left(Q_{bt}^{chs}+R_{bt\omega }^{chs,dn}-R_{bt\omega }^{chs,up}\right)-\left(Q_{bt}^{dis}+R_{bt\omega }^{dis,up}-R_{bt\omega }^{dis,dn}\right)/\eta \right]\hfill \\ \hfill & +So{C_b}^{init}:\left({\epsilon }_{bt\omega }^{min}\right),\forall b,t,\omega \hfill \end{array}$$

(23)

$$\begin{array}{cc}\hfill & So{C_b}^{init}+\Delta t\sum _{t^{\prime }=1}^t\left[\eta \left(Q_{bt}^{chs}+R_{bt\omega }^{chs,dn}-R_{bt\omega }^{chs,up}\right)-\left(Q_{bt}^{dis}+R_{bt\omega }^{dis,up}-R_{bt\omega }^{dis,dn}\right)/\eta \right]\hfill \\ \hfill & \le \overline{So{C}_b}:\left({\epsilon }_{bt\omega }^{max}\right),\forall b,t,\omega \hfill \end{array}$$

(24)

$$0\le Q_{bt}^{chs}\le \overline{Q_b^{chs}}{u}_{bt}^{chs}:\left({\upsilon }_{bt}^{chs}\right)$$

(25)

$$0\le Q_{bt}^{dis}\le \overline{Q_b^{dis}}{u}_{bt}^{dis}:\left({\upsilon }_{bt}^{dis}\right)$$

(26)

$$u_{bt}^{chs},u_{bt}^{dis}\in \left\{0,1\right\}$$

(27)

$$u_{bt}^{chs}+u_{bt}^{dis}\le 1:\left({\tau }_{bt}\right)$$

(28)

where Equations (5)–(7) are the electrical power, upward reserve, and downward reserve balance of the day-ahead market, respectively. In the day-ahead stage, the energy and reserve deployment prices are obtained from the shadow prices of the energy and reserve balance equations, as shown in Equations (5)–(7). Equation (8) is the electrical power balance in the real-time market. Constraints (9) and (10) limit the maximum power of the transmission line. Equations (11) and (12) specify the power angle reference values in the day-ahead and real-time stages. Constraints (13)–(18) are the output, reserve, and ramp limits of thermal generators. Wind power output and wind curtailment limits are constraints (19) and (20). Constraints (21)–(24) limit the capacity range of energy storage in the day-ahead and real-time stages. The limit of energy storage charge and discharge power is represented by constraints (25) and (26). Equations (27) and (28) indicate that the energy storage cannot be in the charging and discharging status at the same time.

3. Solution Methodology

Due to the nested structure of the bi-level model, a direct solution will lead to exponential computational complexity, and it is not easy to obtain convergent results when the variable dimension is high. Therefore, this section considers converting the bi-level model into a single-level model through the strong duality theorem and the KKT condition, and then solving it.

3.1. Model Transformation

The optimal solution of the lower-level model is determined using KKT conditions, including stability, feasibility, and complementary relaxation conditions, which are used as constraints for the upper-level model. To this end, the dual form of the optimization problem of the lower-level model needs to be determined first. The objective function of the dual problem is as follows:

$$\begin{array}{cc}\hfill max& \sum _{n,t}\left(-\sum _{j\in {\Psi }_n^J}{\lambda }_{nt}{L}_{jt}+\sum _{\omega }\sum _{q\in {\psi }_n^Q}{\lambda }_{nt\omega }{W}_{qt\omega }+{\lambda }_{nt}^{up}{R}_t^{up}+{\lambda }_{nt}^{up}{R}_t^{dn}\right)\hfill \\ \hfill & -\sum _{n,m\in {\Theta }_n}\left({\alpha }_{nm}\overline{P_{nm}}+\sum _{\omega }{\alpha }_{nm\omega }\overline{P_{nm}}\right)-\sum _{q\in {\Psi }_n^Q,t}\left(\overline{W_{qt}}{\varphi }_{qt}+\sum _{\omega }{W}_{qt\omega }{\psi }_{qt\omega }\right)\hfill \\ \hfill & +\sum _{i\in {\Psi }_n^I,t}\left[{\beta }_{it}^{min}\underline{P_{it}}-{\beta }_{it}^{max}\overline{P_{it}}+\sum _{\omega }\left({\beta }_{it\omega }^{min}\underline{P_{it}}-{\beta }_{it\omega }^{max}\overline{P_{it}}-{\delta }_{it\omega }^U\overline{P_{it}^U}-{\delta }_{it\omega }^D\overline{P_{it}^D}\right)\right]\hfill \\ \hfill & -\sum _{i\in {\Psi }_n^I,t}\left[{\gamma }_{it}^{D}R{D}_i+{\gamma }_{it}^{U}R{U}_i+\sum _{\omega }\left({\gamma }_{it\omega }^{D}R{D}_i+{\gamma }_{it\omega }^{U}R{U}_i\right)\right]\hfill \\ \hfill & +\sum _{b,t}\left[{\epsilon }_{bt}^{min}\left(\underline{So{C}_b}-So{C}^{init}\right)+{\epsilon }_{bt}^{max}\left(So{C}^{init}-\overline{So{C}_b}\right)\right]\hfill \\ \hfill & +\sum _{b,t,\omega }\left[{\epsilon }_{bt\omega }^{min}\left(\underline{So{C}_b}-So{C}^{init}\right)+{\epsilon }_{bt\omega }^{max}\left(So{C}^{init}-\overline{So{C}_b}\right)\right]\hfill \end{array}$$

(29)

The constraints include

$$\begin{array}{cc}\hfill & C_i\Delta t+{\lambda }_{nt}+{\beta }_{it}^{max}-{\beta }_{it}^{min}+{\gamma }_{it}^U-{\gamma }_{i,t+1}^U-{\gamma }_{it}^D+{\gamma }_{i,t+1}^D\hfill \\ \hfill & +\sum _{\omega }\left({\beta }_{it\omega }^{max}-{\beta }_{it\omega }^{min}+{\gamma }_{it\omega }^U-{\gamma }_{i,t+1,\omega }^U-{\gamma }_{it\omega }^D+{\gamma }_{i,t+1,\omega }^D\right)\ge 0,\forall i\in {\Psi }_n^I,\forall n,t\hfill \end{array}$$

(30)

$$\begin{array}{c}{\pi }_{\omega }{C}_i\Delta t+{\lambda }_{nt\omega }+{\beta }_{it\omega }^{max}-{\beta }_{it\omega }^{min}+{\gamma }_{it\omega }^U-{\gamma }_{i,t+1,\omega }^U-{\gamma }_{it\omega }^D+{\gamma }_{i,t+1,\omega }^D\hfill \\ +{\delta }_{it\omega }^U\ge 0,\forall i\in {\Psi }_n^I,\forall n,t,\omega \hfill \end{array}$$

(31)

$$\begin{array}{cc}\hfill & -{\pi }_{\omega }{C}_i\Delta t-{\lambda }_{nt\omega }-{\beta }_{it\omega }^{max}+{\beta }_{it\omega }^{min}-{\gamma }_{it\omega }^U+{\gamma }_{i,t+1,\omega }^U+{\gamma }_{it\omega }^D-{\gamma }_{i,t+1,\omega }^D\hfill \\ \hfill & +{\delta }_{it\omega }^D\ge 0,\forall i\in {\Psi }_n^I,\forall n,t,\omega \hfill \end{array}$$

(32)

$${\displaystyle \frac{1}{\eta }}\sum _t^{N_T}\left[{\epsilon }_{bt}^{min}-{\epsilon }_{bt}^{max}+\sum _{\omega }\left({\epsilon }_{bt\omega }^{min}-{\epsilon }_{bt\omega }^{max}\right)\right]+{\lambda }_{nt}+k_{b1}{C}_b\Delta t\ge 0,\forall b\in {\Psi }_n^B,\forall n,t,\omega$$

(33)

$$\eta \sum _t^{N_T}\left[{\epsilon }_{bt}^{max}-{\epsilon }_{bt}^{min}+\sum _{\omega }\left({\epsilon }_{bt\omega }^{max}-{\epsilon }_{bt\omega }^{min}\right)\right]-{\lambda }_{nt}-k_{b1}{C}_b\Delta t\ge 0,\forall b\in {\Psi }_n^B,\forall n,t,\omega$$

(34)

$${\pi }_{\omega }{k}_{b2}{C}_b\Delta t+{\lambda }_{nt\omega }+{\displaystyle \frac{1}{\eta }}\sum _t^{N_T}\left({\epsilon }_{bt\omega }^{min}-{\epsilon }_{bt\omega }^{max}\right)\ge 0,\forall b\in {\Psi }_n^B,\forall n,t,\omega$$

(35)

$$-{\pi }_{\omega }{k}_{b2}{C}_b\Delta t-{\lambda }_{nt\omega }+\eta \sum _t^{N_T}\left({\epsilon }_{bt\omega }^{max}-{\epsilon }_{bt\omega }^{min}\right)\ge 0,\forall b\in {\Psi }_n^B,\forall n,t,\omega$$

(36)

$${\lambda }_{nt}-\sum _{\omega }{\lambda }_{nt\omega }+{\varphi }_{qt}\ge 0,\forall q\in {\Psi }_n^Q,\forall n,t,\omega$$

(37)

$$-{\lambda }_{nt\omega }+{\psi }_{qt\omega }\ge 0,\forall q\in {\Psi }_n^Q,\forall n,t,\omega$$

(38)

$$\sum _{m\in \Theta n}{B}_{nm}\left[-{\lambda }_{nt}+{\lambda }_{mt}+\sum _{\omega }\left({\lambda }_{nt\omega }-{\lambda }_{mt\omega }\right)+{\alpha }_{nm}-{\alpha }_{mn}\right]+{\left.{\sigma }_{1t}\right|}_{n=1}=0,\forall m\in {\Theta }_n,\forall n,t,\omega$$

(39)

$$\sum _{m\in \Theta n}{B}_{nm}\left(-{\lambda }_{nt\omega }+{\lambda }_{mt\omega }+{\alpha }_{nm\omega }-{\alpha }_{mn\omega }\right)+{\left.{\sigma }_{1t\omega }\right|}_{n=1}=0,\forall m\in {\Theta }_n,\forall n,t,\omega$$

(40)

Thus, according to the constraints in the original optimization problem of the lower-level model and its dual form, the stability condition and feasibility condition in the KKT condition can be obtained.

By multiplying the Lagrange multiplier with the original feasibility condition, we can get the form of the complementary relaxation condition in the KKT condition that the lower-level model needs to satisfy. Taking constraints (9) and (10) as examples, the following can be obatained:

$${\alpha }_{nm}\left[B_{nm}\left({\theta }_{nt}-{\theta }_{mt}\right)-\overline{P_{nm}}\right]=0,\forall m\in {\Theta }_n,\forall n,t$$

(41)

$${\alpha }_{nm\omega }\left[B_{nm}\left({\theta }_{nt\omega }-{\theta }_{mt\omega }\right)-\overline{P_{nm}}\right]=0,\forall m\in {\Theta }_n,\forall n,t,\omega$$

(42)

According to the strong duality theorem and the KKT conditions corresponding to the lower-level model, we can obtain the conditions which the optimal solution of the lower-level optimization problem needs to meet. Substituting them into the constraints of the upper-level model, the bi-level model can be transformed into a single-level model.

3.2. Linearization

Given that the objective function (4) and the complementarity constraints (41)–(42) contain nonlinear terms, it is necessary to linearize these elements first. Analysis of the two different sources of nonlinearity is conducted below:

(1)

The nonlinear terms in the objective function.

The nonlinear terms in objective function (4) arise from the product of Lagrangian multipliers ${\lambda }_{nt}$, ${\lambda }_{nt\omega }$ and continuous variables $Q_{bt}^{dis},Q_{bt}^{chs},R_{bt\omega }^{dis},R_{bt\omega }^{chs}$. They can be linearized using a binary expansion method [33,34]. Taking ${\lambda }_{nt}{Q}_{bt}^{dis}$ as an example, the specific process is as follows:

$$Q_{bt}^{dis}=\sum _k{x}_{btk}^{qdis}{\widehat{q}}_{btk}^{dis},\forall b,t$$

(43)

$$\sum _k{x}_{btk}^{qdis}=1,\forall b,t$$

(44)

$$x_{btk}^{qdis}=\left\{0,1\right\},\forall b,t,k$$

(45)

$$0\le {\lambda }_{nt}-y_{btk}^{qdis}\le G\left(1-x_{btk}^{qdis}\right),\forall b\in {\Psi }_n^B,\forall t,k$$

(46)

$$0\le y_{btk}^{qdis}\le G{x}_{btk}^{qdis},\forall b,t,k$$

(47)

$${\lambda }_{nt}{Q}_{bt}^{dis}=\sum _k{y}_{btk}^{qdis}{\widehat{q}}_{btk}^{dis}$$

(48)

(2)

The nonlinear terms in complementarity constraints.

The nonlinear terms in the complementarity constraints (41)–(42) originate from the product of dual variables and continuous variables, which can be linearized using the big-M method [35]. Details are as follows:

$$\left\{\begin{array}{c}{\alpha }_{nm}\le M{a}_{nm}\hfill \\ \overline{P_{nm}}-B_{nm}\left({\theta }_{nt}-{\theta }_{mt}\right)\le M\left(1-a_{nm}\right)\hfill \end{array}\right.,\forall m\in {\Theta }_n,\forall n,t$$

(49)

$$\left\{\begin{array}{c}{\alpha }_{nm\omega }\le M{a}_{nm\omega }\hfill \\ \overline{P_{nm}}-B_{nm}\left({\theta }_{nt\omega }-{\theta }_{mt\omega }\right)\le M\left(1-a_{nm\omega }\right)\hfill \end{array}\right.,\forall m\in {\Theta }_n,\forall n,t,\omega$$

(50)

By performing similar processing on other nonlinear terms in the transformed single-level model according to the steps of the above-mentioned binary decomposition method and the big-M method, the nonlinearity in the model can be eliminated, thereby converting the problem into a MILP (Mixed Integer Linear Programming) problem that can be solved by commercial solvers such as Gurobi or Cplex. The detailed process of establishing a bi-level model and transforming it into a single-level model is shown in Figure 3.

4. Numerical Studies

This section introduces the modified RTS-24 system and the IEEE 118-bus system as test systems to verify the effectiveness of the proposed bi-level model of energy storage participating in day-ahead and real-time market transactions.

4.1. System Data

The modified IEEE RTS-24 system topology is shown in Figure 4. It contains 24 nodes and is equipped with 10 thermal generators, 1 wind generator and 1 energy storage device. The wind generator is located at node 1 and the energy storage device is located at node 15. The total maximum capacity of the thermal generators is 3405 MW. The specific operating parameters of each thermal generator are shown in

Table 2. Operation parameters of thermal generators.

Unit No.	Node No.	$\underline{{\mathit{P}}_{\mathit{i}}}$ (MW)	$\overline{{\mathit{P}}_{\mathit{i}}}$ (MW)	$\mathit{R}{\mathit{D}}_{\mathit{i}}$ (MW/min)	$\mathit{R}{\mathit{U}}_{\mathit{i}}$ (MW/min)	${\mathit{C}}_{\mathit{i}}$ (JPY/MW)
P1	1	62.4	192	2	2	278.67
P2	2	62.4	192	2	2	278.67
P3	7	75	300	7	7	305.62
P4	13	207	591	3	3	340.06
P5	15	66.3	215	3	3	173.04
P6	16	54.3	155	3	3	86.73
P7	18	100	400	6.67	6.67	30.94
P8	21	100	400	6.67	6.67	30.94
P9	22	60	300	5	5	0.0007
P10	23	248.6	660	3	3	84.7

. The predicted wind power output is taken from the 24 h output data of a wind farm in Northwest China [36]. The settings of other parameters in the system are shown in

Table 3. Other parameters of IEEE RTS-24 system.

Parameter	Value	Parameter	Value	Parameter	Value
$\overline{k_{b1}}$	10	$\overline{P_{nm}}$	120 MW	$\overline{Q_b^{dis}}$	100 MW
$\overline{k_{b2}}$	10	$\underline{So{C}_b}$	10 MWh	$C_b$	50 JPY/MW
$R_t^{up}$	150 MW	$\overline{So{C}_b}$	500 WMh	$\eta$	0.9
$R_t^{dn}$	150 MW	$\overline{Q_b^{chs}}$	100 MW	K	60

.

The wind power output in the day-ahead stage is recorded as scenario 1 and scenario 2, with probabilities of 0.6 and 0.4 respectively. The uncertainty of the load is not considered. The data of wind output and load are shown in Figure 5.

4.2. Analysis of Market Results

The day-ahead energy and reserve clearing prices are shown in Figure 6. It can be seen that the peak energy price in the day-ahead market occurs at 12:00–14:00 and 18:00–19:00, respectively, which is basically consistent with the peak load. The peak and valley periods of energy prices change significantly, with the maximum price difference of 240.52 JPY/MWh, which can provide a large space for energy storage arbitrage in the market. The reserve price is lower than the energy price, and the upper reserve price is higher than the lower reserve price.

The day-ahead dispatch results of electricity power are shown in Figure 7. It can be seen that energy storage charging is concentrated in the two periods of 22:00–5:00 and 14:00–15:00, which corresponds to the low period of the clearing price. Energy storage discharge is carried out in the peak period of 8:00–13:00 and 18:00–19:00, so that it can obtain the maximum market arbitrage benefits while meeting the system power balance. Figure 8 shows the energy storage dispatch results of two scenarios in the day-ahead and real-time stage. As can be seen from Figure 8, energy storage can earn arbitrage profits by charging at low prices and discharging at peak prices in the day-ahead stage. After considering the use of energy storage reserve in the real-time stage in different scenarios, it is observed that the energy storage charging and discharging are more frequent, and continuous charging and discharging state conversion may occur, indicating that the reserve provided by energy storage can be effectively called upon in the real-time market, thereby giving full play to the flexible regulation role of energy storage.

4.3. Comparative Analysis of Market Types

In order to compare the profits of energy storage participating in different types of markets, the following three cases are set for comparative analysis:

Case 1: Energy storage only participates in the day-ahead electricity energy market and makes profits through the arbitrage of the day-ahead market price difference.

Case 2: Energy storage participates in the joint clearing of the day-ahead electricity energy and reserve markets, and the profits include day-ahead arbitrage and reserve deployment benefits.

Case 3: Energy storage also participates in the real-time balancing market, and the reserve use compensation is added to the energy storage profits.

The day-ahead electricity market clearing prices under the three cases are shown in Figure 9. It is worth noting that due to the different wind power output scenarios, the benefits of the real-time market will change accordingly, so the benefits at this stage are expressed in the form of expected values. The comparison of the upper-reserve and down-reserve prices and capacities corresponding to scenarios 2 and 3 is shown in Figure 10 and Figure 11. To quantitatively demonstrate the market benefits of energy storage under different cases, the relevant data for the three cases are listed in

Table 4. Comparison of energy storage market profits.

Case No.	Energy Market Profit (JPY)	Reserve Market Profit (JPY)	Real-Time Market Profit (JPY)	Total Profit (JPY)
Case 1	2,097,459	/	/	2,097,459
Case 2	2,176,601	872,833	/	3,049,424
Case 3	1,643,922	1,121,841	568,953	3,334,716

, where the total profit is composed of energy market profit, reserve market profit, and real-time market profit as mentioned in Section 2.

From the above charts, it can be seen that compared with Case 1, energy storage needs to participate in the reserve market in Case 2, which leads to a reduction in the input of the energy market, making the energy price higher and increasing the income of the energy market by 3.77% and the total income by 45.39%. Comparing Case 3 with the other cases, the peak energy price is higher than the others, while the valley energy price is lower than the others, indicating that the price arbitrage space for energy storage in the day-ahead market is the largest. However, due to the further reduction of the energy market input, the income of the energy market has decreased by 24.47%, while the income of the reserve market has increased by 28.53%, and the total income has increased by 9.36%, compared to Case 2. In general, Case 1 has the least profit-making channels for energy storage, resulting in the lowest total market revenue. Case 2 increases the total market profit by increasing the day-ahead reserve market revenue. Case 3, although it has the lowest energy market profit, further increases the expected revenue of the real-time market so that energy storage can obtain the highest total market revenue.

4.4. Analysis of the Impact of Bidding Strategies

Based on the above Case 3, the energy storage quotation will also have an impact on market profits. Three different bidding strategies are set for comparative analysis:

Case 3.1: Energy storage only participates in the day-ahead and real-time markets as a price-taker, that is, it bids quantity but not price.

Case 3.2: Energy storage bids according to its marginal cost.

Case 3.3: Energy storage bids with quantity and price, and different bidding strategy parameters are used in the day-ahead and real-time markets, which is denoted as $k_{b1}$ and $k_{b2}$.

Table 5. Comparison of energy storage market profits under different bidding strategies.

Case No.	${\mathit{k}}_{\mathit{b}1}$	${\mathit{k}}_{\mathit{b}2}$	Energy Market Profit (JPY)	Reserve Market Profit (JPY)	Real-Time Market Profit (JPY)	Total Profit (JPY)
Case 3.1	/	/	1,397,494	736,113	393,981	2,527,588
Case 3.2	1	1	1,575,371	887,607	487,494	2,950,472
Case 3.3	5.48	2.30	1,643,922	1,121,841	568,953	3,334,716

lists the profits of energy storage participating in the day-ahead and real-time markets under the above three cases. It can be seen that compared with Case 3.1, energy storage actively participates in the market in Case 3.2 and Case 3.3 and can bring a 16.73% and 31.93% increase in total market revenue, respectively. Case 3.2 is equivalent to the situation in which the bidding strategy parameters $k_{b1}$ and $k_{b2}$ are both 1 in Case 3.3 when bidding with marginal cost. The market revenue of Case 3.3 at each stage is higher than that of Case 3.2, and the total revenue increase by 13.02%. The reason for this is that Case 3.3 can find the optimal quotation in the day-ahead and real-time markets through the iteration of the upper-level bidding strategy parameters to achieve the global maximum value of market profits.

4.5. Analysis of Model Scalability

In order to verify the scalability of the model, a more complex IEEE 118-bus system is selected for analysis, and its topology is shown in Figure 12. The system consists of 54 thermal generators with a total maximum output of 4628 MW. A wind generator is set at node 37, and two energy storage devices are set at nodes 8 and 66, respectively, with a variable cost of 50 JPY/MW. The wind power output and load data are the same as those of the aforementioned IEEE RTS-24 node system. The capacity range of the two energy storage devices is 10–250 MWh, the maximum charge and discharge power is 50 MW, and the charge and discharge efficiency is set to 0.9.

(1)

Impact of energy storage configuration

Energy storage configuration parameters include the maximum power and maximum capacity. In order to study their impact on the total revenue of energy storage participating in the market, the configuration parameters of the two energy storages are changed within the range of 50–250 MW and 50–250 MWh, and the total revenues are shown in Figure 13.

As can be seen from Figure 13, the increase in energy storage maximum power and capacity both have a positive effect on the improvement of market revenue. When the maximum power is fixed at 250 MW, and the capacity increases from 50 MWh to 250 MWh, the total market revenue increases by 4.19 times. When the maximum capacity is fixed at 250 MWh, and the power increases from 50 MW to 250 MW, the total market profit increases by 3.84 times. The energy storage maximum power determines the market participation ability. On the one hand, the peak realization ability of electricity prices in the energy market is affected by the upper limit of discharge power. On the other hand, the reserve bid in the market cannot exceed the power upper limit. Therefore, when the maximum power is low, the market profit will be affected and remain at a low level. The energy storage maximum capacity determines the market participation time. Taking the energy market as an example, small-capacity energy storage can only participate in peak arbitrage in some time periods, while large-capacity energy storage can be discharged continuously at multiple electricity price peaks, broadening the arbitrage space, thereby achieving an increase in total revenue.

(2)

Analysis of computation performance

Computation performance is also an important indicator of model scalability. The number of discrete data points K introduced when the binary expansion method is used in the linearization step in Section 3.2 will affect the optimization results and computation time of the model. Comparing the changes in the computational performances of the two models under different K values, the results are shown in Figure 14. It can be seen that the solution time of the IEEE 118-bus system is higher than that of the IEEE RTS-24 system, and the computation time and total profit of the two systems increase with increasing K. The computation time data under different K values are shown in the figure. It can be seen that when the K value increases from 10 to 120, the computation time of the two test systems increases by 22 times and 27 times, respectively. The calculation time difference between the two systems gradually increases from 25.89s to 559.45 s. The larger the K value, the smaller the step size of the discrete approximation selection in the binary expansion method, which results in an increase in computation complexity and time, and the impact on the computation time of complex systems is more obvious. At the same time, a larger K means more discrete approximate data points of the optimization parameters, which increases the feasible domain, so that a better optimization result can be obtained, that is, a greater total market profit.

5. Conclusions

This paper establishes a two-stage bi-level model for energy storage to participate in the day-ahead and real-time markets, considering two types of services: energy and reserve. The upper-level formulates a bidding strategy with the goal of maximizing energy storage market profit, while the lower-level completes joint market clearing with the goal of minimizing total cost. Based on the KKT condition and linearization method, the model is transformed into a single-level MILP model that is easier to solve. Numerical analysis is carried out on IEEE RTS-24 and IEEE 118-bus systems. The results show that energy storage participation in both the reserve market and the real-time market can increase total profit by 45.39% and 9.36%, respectively. Compared to bidding as a price-taker and with marginal cost, the proposed method can bring a 16.73% and 13.02% increase in total market profit, respectively. The energy storage maximum power determines the market participation ability, and the maximum capacity determines the market participation time, both of which have a positive effect on improving market total profit. Numerical studies of two different scale markets verify the effectiveness and scalability of the proposed method, and the choice of discrete data point K in the binary expansion method affects the computation time and objective value.

The research conducted in this manuscript is based on two standard system topologies. The demonstration based on the actual case of a relevant electricity market will be the direction of our future research. At the same time, in order to further improve the market participation of energy storage, it is also worthwhile to add the research on the auxiliary market such as peak shaving and frequency modulation, and to consider demand response and other market mechanisms.