Optimal Battery Storage Participation in European Energy and Reserves Markets

Abstract

Battery energy storage is becoming an important asset in modern power systems. Considering the market prices and battery storage characteristics, reserve provision is a tempting play fields for such assets. This paper aims at filling the gap by developing a mathematically rigorous model and applying it to the existing and future electricity market design in Europe. The paper presents a bilevel model for optimal battery storage participation in day-ahead energy market as a price taker, and reserve capacity and activation market as a price maker. It uses an accurate battery charging model to reliably represent the behavior of real-life lithium-ion battery storage. The proposed bilevel model is converted into a mixed-integer linear program by using the Karush–Kuhn–Tucker optimality conditions. The case study uses real-life data on reserve capacity and activation costs and quantities in German markets. The reserves activation quantities and activation prices are modeled by a set of credible scenarios in the lower-level problem. Finally, a sensitivity analysis is conducted to comprehend to what extent do battery storage bidding prices affect its overall profit.

1. Introduction

The European power sector is characterized by an ongoing liberalization and integration of national markets into one common marketplace. After the successful introduction of national electricity exchanges, followed by their coupling, the focus switched to the provision of ancillary services. The frequency reserves, as fairly location-independent services, were first in line to be governed by the market laws. Most of the European systems already have well-organized reserve markets, but their harmonization, which is the foundation for the integrated European reserve markets, is yet to be initiated. Reserve markets will use the same cross-border interconnection capacities as the energy market, and therefore these two markets must be co-optimized. The most recent European Union energy package incorporates detailed rules on how the reserve markets are to be organized, co-optimized and coupled, forming a cornerstone for all future reserve market research [1].

The reserve markets, depending on the type of reserve and different countries’ regulations, are organized as either single-stage capacity-only markets or two-stage capacity and activation markets. The former type includes only capacity auction where the reserve providers’ bids consist of capacity volume (in MW) and price (€/MW). Using a merit order list (MOL), the the transmission system operator (TSO) accepts the cheapest bids until the required capacity is reached. A reserve provider must take into account the potential activated energy cost within its capacity bid as it is usually not separately remunerated (it could also be remunerated based on a regulated price). Such capacity is activated based on uniform price or some other rule. Usually, the frequency containment reserve (FCR) and sometimes automatic frequency restoration reserve (aFRR) are modeled this way. The latter type, along with the capacity procurement, includes the activated energy auction as well. A reserve provider’s bid consist of energy volume (in MWh) and price (in €/MW). Using the MOL, energy offers are activated when needed. Such pricing activates the cheapest units first and therefore yields lower overall cost. Usually, manual restoration and replacement reserves and often automatic restoration reserves are modeled this way.

Both stages can be modeled in either a pay-as-bid or marginal pricing manner. In the current German secondary reserve market, both the capacity and the activated energy are priced as pay-as-bid [2,3]. However, the PICASSO project published a report with a conclusion that the pricing of aFRR activated energy in a future European-wide aFRR activation platform will be guided by the marginal pricing rule [4], which is adopted in this paper as well.

The capacity of the installed battery storage worldwide was around 10 GWh in 2017 [5]. In Germany alone, as one of the leaders in battery installations, in 2018 the capacity of home storage systems was around 930 MWh and large storage systems around 550 MWh [5]. The capacity of industrial storage systems is hard to estimate due to a lack of information. It is estimated that by the end of 2030 the battery capacity would rise to 181–421 GWh worldwide [5]. Most of the large storage systems operate in FCR markets. The FCR markets, in developed countries such as Germany and UK, are coming close to saturation, but new revenue streams are unlocking such as grid deferral and aFRR markets [6].

Coupling of national reserve markets and their co-optimization with energy markets creates new possibilities for battery storage as they could sell their services cross-border and position themselves in multiple markets. The battery storage as a fully flexible resource must be able to simultaneously bid in both the energy and reserve markets and must maintain its state-of-energy (SOE) within the allowed, i.e., feasible, range. Energy markets include a large number of different units, both capacity- and technology-wise, and its size is considerably larger then one battery storage. For example, the French power system had the minimum demand of 30.4 GW in 2018 during the summer and the peak demand of 96.6 GW during the winter [7]. Battery storage impact on such large market is negligible and therefore it can be seen as a price taker. However, reserve markets are smaller in size. For example, the German aFRR market has total demand of above 2 GW, while German FCR market is somewhat higher than 0.5 GW [8]. The battery storage trading on those markets should be modeled as price maker as its behavior could affect the prices.

In this paper, a novel battery storage scheduling algorithm for joint participation on energy and reserve market is designed and validated on a realistic test case. The battery storage acts as a price taker in the day-ahead energy market and as a price maker in the reserves market. Such algorithms are deemed to be the backbone for future battery scheduling in the large coupled and co-optimized energy and reserve markets in Europe. The focus of the paper is on aFRR markets as they are becoming a new source of revenue for the battery storage systems. However, the developed algorithm can easily be adjusted for other types of reserves.

2. Literature Review and Contributions

Depending on its capacity with respect to the total system load, energy storage can be considered too small to affect market prices, i.e., price taker, or to have a sufficient capacity to alter the market outcomes, thus becoming a price maker. Some early studies model the energy storage as a price taker, which means the prices in the models are known upfront [9,10].

Arbitrage alone might not be sufficient to justify the investment cost of energy storage. The authors in [11] prove that large-scale energy storage will dampen the price difference between on- and off-peak hours when performing arbitrage. It hereby reduces the profit it can make in the energy market, suggesting that energy storage should be used for ancillary services as well.

In [12], the authors model a profit-seeking price-taker energy storage that participates in energy and reserve day-ahead market and energy hour-ahead market. Stochastic unit commitment is used to derive scenarios for the cost of power and reserve in the hour-ahead market, as well as the actual reserve activation quantities. The uncertain parameters arise from the wind power plant output uncertainty. An optimal energy storage bidding model considering day-ahead energy, spinning reserve and regulation markets is presented in [13]. The price-taker energy storage considers uncertainties of predicted market prices and energy deployment in spinning reserve and regulation markets. The optimal bidding schedule is secured against realization of uncertainties using robust optimization framework.

Optimal bidding strategies are studied for battery energy storage systems in the reserve market with battery aging constraints in [14,15]. On the other hand, [16] combines power from unpredictable wind and photovoltaic sources with energy storage in the day-ahead electricity market using a stochastic two-stage programming environment, where the first stage is the day-ahead market, while the second stage simulates the balancing market using multiple scenario sets with historical data. An interested reader may find a comprehensive overview of operating models of energy storage is available in [17].

The Alberta Electric System Operator (AESO) compared sequential clearing of the energy and reserve market with their co-optimization and concluded that co-optimization was more cost-efficient then sequential clearing [18]. Authors in [19] propose a model that co-optimizes energy and reserve market for a combined cycle plant using a mixed-integer linear program (MILP). Paper [20] proposes a nonlinear model for co-optimization of energy and reserves in competitive electricity markets including nonlinear constraints such as power flow losses, unreliability and generation repair time. The authors in [21] clarify two approaches used in the literature to formulate the reserve requirements. The first one is by pre-defining the necessary reserve requirements using ad-hoc rules, such as the 3 + 5% rule [22], and setting the reserve requirements as parameters in the optimization problem. The second approach incorporates the power balance and transmission constraints both at the day-ahead and the balancing stage. These approaches are studied and evaluated in the MISO (Midwestern Independent System Operator) system in [23]. Another model that proposes an optimal dispatch of the energy and reserve capacity, but considering uncertain demand, is presented in [24]. The effects of co-optimized and individual clearing of the energy and reserve markets are investigated.

Despite a large body of literature focused on either theoretical or US-market based participation of energy storage, there are very few papers that replicate the operation of European markets and integrate them in a rigorous and scientific framework. One of the pivotal papers in modeling battery storage providing primary frequency response in the European setting is [25]. The presented optimization problem and the case study is focused and based on data for the German market. German energy and reserves market was also targeted in [26], where the pay-as-bid feature as well as longer time steps for providing reserve (4–12 h) was adopted. German aFRR market was the main topic in papers [27,28]. The former paper tackles the aFRR activation duration and price forecasting while the latter one deals with the bidding process in the German energy and aFRR reserve markets. The model in the paper [28] creates bids for storage to participate in the aFRR market based on price and activation forecasts meaning that it does not observe energy storage as a price forming factor but as a price taker.

With respect to the examined literature, this paper aims at filling the gap by combining a mathematically rigorous mathematical model with application to the existing and future electricity markets currently designed in Europe. Contribution of the paper is threefold. First, we develop a bilevel model for optimal battery storage participation in day-ahead energy market as a price taker, and reserve capacity and activation market as a price maker. Conceptually, this paper is an alternative to the approach of price maker algorithms for the German aFRR presented in [27,28]. As opposed to the majority of the literature that uses a generic energy storage model, we use an accurate battery charging model to reliably represent the behavior of actual battery storage. The proposed bilevel model is converted into a mixed-integer linear program by using the Karush–Kuhn–Tucker (KKT) optimality conditions. Second, we use real-life data on reserve capacity and activation costs and quantities to bring relevant conclusions. The reserves activation quantities and, consequently, the activation price is modeled by a set of credible scenarios. Thirdly, we provide a sensitivity analysis to comprehend to what extent do battery storage bidding prices affect its overall profit.

In the following chapter we first define the indices, parameters and variables used in the model and then present the model itself. The KKT optimality conditions and linearization technique are also presented. In Section 4 we present a case study based on the German market. This section also includes a sensitivity analysis for different sets of battery storage bidding prices. Finally, the relevant conclusions are drawn in the final section.

3. Mathematical Formulation

3.1. Nomenclature

Sets:

I	Set of generation units, indexed by i.
J	Set of battery charging curve linear parts, indexed by j.
S	Set of reserve activation scenarios, indexed by s.
T	Set of time periods, indexed by t.

Parameters:

$C_i^{\mathrm{a}\downarrow }$	Generator i down reserve activation price (€/MWh).
$C_i^{\mathrm{a}\uparrow }$	Generator i up reserve activation price (€/MWh).
$C^{\mathrm{b},\mathrm{a}\downarrow }$	Battery storage down reserve activation price (€/MWh).
$C^{\mathrm{b},\mathrm{a}\uparrow }$	Battery storage up reserve activation price (€/MWh).
$C^{\mathrm{b},\mathrm{cap}\downarrow }$	Battery storage down reserve capacity price (€/MW).
$C^{\mathrm{b},\mathrm{cap}\uparrow }$	Battery storage up reserve capacity price (€/MW).
$C_i^{\mathrm{cap}\downarrow }$	Generator i down reserve capacity price (€/MW).
$C_i^{\mathrm{cap}\uparrow }$	Generator i up reserve capacity price (€/MW).
$G_{t,i}^{\downarrow }$	Generator i maximum down reserve capacity (MW).
$G_{t,i}^{\uparrow }$	Generator i maximum up reserve capacity (MW).
$F_j$	Maximum amount of energy that can be charged at specific state-of-energy breakpoint $R_j$ as a portion of $SOE$.
P	Battery storage maximum charging and discharging power (MW).
$R_j$	Capacity of each state-of-energy segment j as a portion of the maximum state-of-energy $SOE$.
$R_t^{\mathrm{cap}\downarrow }$	Required down reserve capacity (MW).
$R_t^{\mathrm{cap}\uparrow }$	Required up reserve capacity (MW).
$R_{t,s}^{\mathrm{a}\downarrow }$	Activated down reserve energy (MWh).
$R_{t,s}^{\mathrm{a}\uparrow }$	Activated up reserve energy (MWh).
${\eta }^{\mathrm{ch}}$	Battery storage charging efficiency.
${\eta }^{\mathrm{dis}}$	Battery storage discharging efficiency.
${\lambda }_t^{\mathrm{da}}$	Day-ahead market price (€/MW).

Variables:

$g_{t,i,s}^{\mathrm{a}\uparrow }$	Generator i activated down energy (MWh).
$g_{t,i,s}^{\mathrm{a}\downarrow }$	Generator i activated up energy (MWh).
$g_{t,i}^{\mathrm{cap}\downarrow }$	Generator i down capacity reserved quantity (MW).
$g_{t,i}^{\mathrm{cap}\uparrow }$	Generator i up capacity reserved quantity (MW).
${\overline{q}}_t^{\downarrow }$	Battery storage down reserve capacity bid (MW).
${\overline{q}}_t^{\uparrow }$	Battery storage up reserve capacity bid (MW).
$q_{t,s}^{\mathrm{a}\downarrow }$	Battery storage activated down reserve quantity in scenario s (MWh).
$q_{t,s}^{\mathrm{a}\uparrow }$	Battery storage activated up reserve quantity in scenario s (MWh).
$q_t^{\mathrm{cap}\downarrow }$	Battery storage down reserved capacity (MW).
$q_t^{\mathrm{cap}\uparrow }$	Battery storage up reserved capacity (MW).
$q_t^{\mathrm{ch}}$	Battery storage charging quantity (MW).
$q_t^{\mathrm{dis}}$	Battery storage discharging quantity (MW).
$so{e}_{t,s}$	Battery storage state-of-energy (MWh).
${\lambda }_{t,s}^{\mathrm{a}\downarrow }$	Down reserve activation clearing price in scenario s (€/MWh).
${\lambda }_{t,s}^{\mathrm{a}\uparrow }$	Up reserve activation clearing price in scenario s (€/MWh).
${\lambda }_t^{\mathrm{cap}\downarrow }$	Down reserve capacity clearing price (€/MW).
${\lambda }_t^{\mathrm{cap}\uparrow }$	Up reserve capacity clearing price (€/MW).

3.2. Initial Problem Formulation

The proposed battery storage optimal bidding problem is formulated as follows:

$$\underset{{\mathsf{\Xi }}^{\mathrm{UL}}}{\mathrm{Maximize}}\sum _{t\in \mathcal{T}}\left[{\lambda }_t^{\mathrm{da}}(q_t^{\mathrm{dis}}-q_t^{\mathrm{ch}})+\left({\lambda }_t^{\mathrm{cap}\uparrow }·q_t^{\mathrm{cap}\uparrow }+{\lambda }_t^{\mathrm{cap}\downarrow }·q_t^{\mathrm{cap}\downarrow }\right)+\left({\lambda }_{t,s}^{\mathrm{a}\uparrow }·q_{t,s}^{\mathrm{a}\uparrow }+{\lambda }_{t,s}^{\mathrm{a}\downarrow }·q_{t,s}^{\mathrm{a}\downarrow }\right)\right]$$

(1)

subject to:

$$0\le q_t^{\mathrm{ch}}\le \frac{\Delta so{e}_t}{\Delta t·{\eta }^{\mathrm{ch}}},\forall t$$

(2)

$$0\le q_t^{\mathrm{dis}}\le P·{\eta }^{\mathrm{dis}},\forall t$$

(3)

$$q_t^{\mathrm{ch}}-q_t^{\mathrm{dis}}+{\overline{q}}_t^{\downarrow }\le \frac{\Delta so{e}_{t,s}}{\Delta t·{\eta }^{\mathrm{ch}}},\forall t,s$$

(4)

$$-q_t^{\mathrm{ch}}+q_t^{\mathrm{dis}}+{\overline{q}}_t^{\uparrow }\le P·{\eta }^{\mathrm{dis}},\forall t$$

(5)

$$so{e}_{t,s}=so{e}_{t-1,s}+\Delta t·q_t^{\mathrm{ch}}·{\eta }^{\mathrm{ch}}+q_{t,s}^{\mathrm{a}\downarrow }·{\eta }^{\mathrm{ch}}-\Delta t·q_t^{\mathrm{dis}}/{\eta }^{\mathrm{dis}}-q_{t,s}^{\mathrm{a}\uparrow }/{\eta }^{\mathrm{dis}},\forall t,s$$

(6)

$$0\le so{e}_{t,s}-\Delta t·{\overline{q}}_t^{\uparrow },\forall t,s$$

(7)

$$so{e}_{t,s}+\Delta t·{\overline{q}}_t^{\downarrow }\le SOE,\forall t,s$$

(8)

$$so{e}_{t,s}=\sum _{j=1}^{J-1}so{e}_{t,j,s},\forall t$$

(9)

$$0\le so{e}_{t,j,s}\le (R_{j+1}-R_j)·SOE,\forall t,j,s$$

(10)

$$\Delta so{e}_{t,s}=F_1·SOE+\sum _{j=1}^{J-1}\frac{F_{j+1}-F_j}{R_{j+1}-R_j}·so{e}_{t-1,j,s},\forall t,s$$

(11)

where ${\mathsf{\Xi }}_{UL}$ = {$q_t^{\mathrm{ch}}$, $q_t^{\mathrm{dis}}$, $q_t^{\mathrm{cap}\uparrow }$, $q_t^{\mathrm{cap}\downarrow }$, $q_{t,s}^{\mathrm{a}\downarrow }$, $q_{t,s}^{\mathrm{a}\uparrow }$, $so{e}_{t,s}$, $so{e}_{t,j,s}$, $\Delta so{e}_{t,s}$}.

Battery storage in objective function (1) draws benefits from three streams. The first part is the day-ahead market, where it performs energy arbitrage as a price taker. The battery storage can be either discharged, $q_t^{\mathrm{dis}}$, or charged, $q_t^{\mathrm{ch}}$, at the day-ahead market price ${\lambda }_t^{\mathrm{da}}$. The second part is the capacity reservation market. Since this market is much smaller than the day-ahead market, battery storage is modeled as a price maker, i.e., the up and down capacity reservation prices ${\lambda }_t^{\mathrm{cap}\uparrow }$ and ${\lambda }_t^{\mathrm{cap}\downarrow }$ are dual variables whose values are decided in the lower-level problem considering the battery’s bids. The final part of the objective function (1) displays the benefits of both the up and down reserve activation $q_{t,s}^{\mathrm{a}\uparrow }$ and $q_{t,s}^{\mathrm{a}\downarrow }$ at prices ${\lambda }_{t,s}^{\mathrm{a}\uparrow }$ and ${\lambda }_{t,s}^{\mathrm{a}\downarrow }$, respectively.

Constraints (2) and (3) limit the day-ahead charging and discharging power. The fact that the battery charging ability reduces with high state-of-energy values is considered by limiting the battery charging power in (2) by the maximum amount of energy the battery can charge in a single time-step, $\Delta so{e}_{t,s}$, divided by the length of the time-step to convert energy to power. On the other hand, the discharging battery ability in (3) is constant regardless of the state-of-energy. Constraints (4) and (5) impose charging and discharging limits to down and up reserve bids so the charging and discharging battery capacity is not exceeded. Down reserve in (4) can be provided by increasing the charging power from the day-ahead stage (in this case $q_t^{\mathrm{ch}}$ is positive and $q_t^{\mathrm{dis}}$ is zero) or by reducing or fully stopping the discharging power from the day-ahead stage and possibly starting to charge instead (in this case $q_t^{\mathrm{ch}}$ is zero and $q_t^{\mathrm{dis}}$ is positive). Similarly, up reserve in (5) can be provided by reducing the day-ahead charging power and/or increasing the day-ahead discharging power. Equation (6) calculates the state-of-energy per each reserve activation scenario. Since $q_t^{\mathrm{ch}}$ and $q_t^{\mathrm{dis}}$ are power quantities, they are multiplied by an appropriate time step duration $\Delta t$. Since the day-ahead market is on an hourly basis, $q_t^{\mathrm{ch}}$ and $q_t^{\mathrm{dis}}$ are multiplied by 1. The reserve activation quantities $q_{t,s}^{\mathrm{a}\downarrow }$ and $q_{t,s}^{\mathrm{a}\uparrow }$ are energy quantities, the same as the state-of-energy $so{e}_{t,s}$. Constraints (7) and (8) provide the lower and upper bounds on the battery state-of-energy considering the reserve activations per scenario and the bid reserve quantities. This ensures that regardless of the reserve activation scenarios the state-of-energy will remain within the given bounds. Constraints (9)–(11) calculate the amount of energy the battery can charge in a time-step, $\Delta so{e}_{t,s}$. To describe the nonlinear battery charging curve, a piecewise approximation given in Figure 1 is used. This curve shows the amount of energy a lithium-ion battery can withdraw from the grid depending on its current state-of-energy. The given picewise linear approximation divides the state-of-energy in multiple segments, $so{e}_{t,j,s}$, constituting the actual battery state-of-energy $so{e}_{t,s}$. These segments are used in (11) to calculate the amount of energy the battery can charge in time period t. Further details on this procedure are available in [29].

The battery scheduling problem (1) is subject to the following lower-level problem (corresponding dual variables related to each constraint are listed after a colon):

$$\begin{array}{c}\underset{{\mathsf{\Xi }}^{\mathrm{LL}}}{\mathrm{Minimize}}\sum _{t\in \mathcal{T}}\left[\sum _{i\in \mathcal{I}}{C}_i^{\mathrm{cap}\uparrow }·g_{t,i}^{\mathrm{cap}\uparrow }+C^{\mathrm{b},\mathrm{cap}\uparrow }·q_t^{\mathrm{cap}\uparrow }+\sum _{i\in \mathcal{I}}{C}_i^{\mathrm{cap}\downarrow }·g_{t,i}^{\mathrm{cap}\downarrow }+C^{\mathrm{b},\mathrm{cap}\downarrow }·q_t^{\mathrm{cap}\downarrow }\right]+\\ \sum _{t\in \mathcal{T}}\sum _{s\in \mathcal{S}}{\pi }_s·\left[\sum _{i\in \mathcal{I}}{C}_i^{\mathrm{a}\uparrow }·g_{t,i,s}^{\mathrm{a}\uparrow }+C^{\mathrm{b},\mathrm{a}\uparrow }·q_{t,s}^{\mathrm{a}\uparrow }+\sum _{i\in \mathcal{I}}{C}_i^{\mathrm{a}\downarrow }·g_{t,i,s}^{\mathrm{a}\downarrow }+C^{\mathrm{b},\mathrm{a}\downarrow }·q_{t,s}^{\mathrm{a}\downarrow }\right]\end{array}$$

(12)

subject to:

$$\sum _{i\in \mathcal{I}}{g}_{t,i}^{\mathrm{cap}\uparrow }+q_t^{\mathrm{cap}\uparrow }\ge R_t^{\mathrm{cap}\uparrow },\forall t:{\lambda }_t^{\mathrm{cap}\uparrow }$$

(13)

$$\sum _{i\in \mathcal{I}}{g}_{t,i}^{\mathrm{cap}\downarrow }+q_t^{\mathrm{cap}\downarrow }\ge R_t^{\mathrm{cap}\downarrow },\forall t:{\lambda }_t^{\mathrm{cap}\downarrow }$$

(14)

$$-\sum _{i\in \mathcal{I}}{g}_{t,i,s}^{\mathrm{a}\uparrow }-q_{t,s}^{\mathrm{a}\uparrow }+R_{t,s}^{\mathrm{a}\uparrow }=0,\forall t,s:{\lambda }_{t,s}^{\mathrm{a}\uparrow }$$

(15)

$$-\sum _{i\in \mathcal{I}}{g}_{t,i,s}^{\mathrm{a}\downarrow }-q_{t,s}^{\mathrm{a}\downarrow }+R_{t,s}^{\mathrm{a}\downarrow }=0,\forall t,s:{\lambda }_{t,s}^{\mathrm{a}\downarrow }$$

(16)

$$g_{t,i}^{\mathrm{cap}\uparrow }\le G_{t,i}^{\uparrow },\forall t,i:{\psi }_{t,i}^{\uparrow }$$

(17)

$$g_{t,i}^{\mathrm{cap}\downarrow }\le G_{t,i}^{\downarrow },\forall t,i:{\psi }_{t,i}^{\downarrow }$$

(18)

$$g_{t,i,s}^{\mathrm{a}\uparrow }\le g_{t,i}^{\mathrm{cap}\uparrow }·\Delta t,\forall t,i,s:{\kappa }_{t,i,s}^{\uparrow }$$

(19)

$$g_{t,i,s}^{\mathrm{a}\downarrow }\le g_{t,i}^{\mathrm{cap}\downarrow }·\Delta t,\forall t,i,s:{\kappa }_{t,i,s}^{\downarrow }$$

(20)

$$q_t^{\mathrm{cap}\uparrow }\le {\overline{q}}_t^{\uparrow },\forall t:{\zeta }_t^{\uparrow }$$

(21)

$$q_t^{\mathrm{cap}\downarrow }\le {\overline{q}}_t^{\downarrow },\forall t:{\zeta }_t^{\downarrow }$$

(22)

$$q_{t,s}^{\mathrm{a}\uparrow }\le q_t^{\mathrm{cap}\uparrow }·\Delta t,\forall t,s:{\nu }_{t,s}^{\uparrow }$$

(23)

$$q_{t,s}^{\mathrm{a}\downarrow }\le q_t^{\mathrm{cap}\downarrow }·\Delta t,\forall t,s:{\nu }_{t,s}^{\downarrow }$$

(24)

$$g_{t,i,s}^{\mathrm{a}\uparrow },g_{t,i,s}^{\mathrm{a}\downarrow }\ge 0,\forall t,i,s:{\alpha }_{t,i,s}^{\uparrow },{\alpha }_{t,i,s}^{\downarrow }$$

(25)

$$q_{t,s}^{\mathrm{a}\uparrow },q_{t,s}^{\mathrm{a}\downarrow }\ge 0,\forall t,s:{\beta }_{t,s}^{\uparrow },{\beta }_{t,s}^{\downarrow }$$

(26)

$$g_{t,i}^{\mathrm{cap}\uparrow },g_{t,i}^{\mathrm{cap}\downarrow }\ge 0,\forall t,i:{\gamma }_{t,i}^{\uparrow },{\gamma }_{t,i}^{\downarrow }$$

(27)

$$q_t^{\mathrm{cap}\uparrow },q_t^{\mathrm{cap}\downarrow }\ge 0,\forall t:{\delta }_t^{\uparrow },{\delta }_t^{\downarrow }$$

(28)

where ${\mathsf{\Xi }}_{LL}$ = {$g_{t,i}^{\mathrm{cap}\uparrow }$, $g_{t,i}^{\mathrm{cap}\downarrow }$, $g_{t,i,s}^{\mathrm{a}\uparrow }$, $g_{t,i,s}^{\mathrm{a}\downarrow }$, $q_t^{\mathrm{cap}\uparrow }$, $q_t^{\mathrm{cap}\downarrow }$, $q_{t,s}^{\mathrm{a}\uparrow }$, $q_{t,s}^{\mathrm{a}\downarrow }$}.

The lower-level problem objective function (12) is the maximization of the social welfare, which includes minimizing the cost of both generators’ and the battery’s up and down capacity reservation as well as its activation per scenario. Constraints (13) and (14) impose the up and down required reserve capacity volumes, while Equations (15) and (16) decide on the contribution of each asset (generators and the battery storage) to up and down reserve activation per scenario. Up and down generators’ cleared reserve capacities are restricted by their offered capacities in (17) and (18), while the generators’ activated quantities are limited by their reserved capacities in (19) and (20). The same is achieved for the battery with constraints (21)–(24). Finally, nonnegativity of the lower-level variables is imposed in (25)–(28). The dual variables listed after a colon in constraints (13)–(28) indicate if those constraints are binding or not. Dual variables of constraints (13)–(16) take values of marginal cost for up capacity reservation, down capacity reservation, up capacity activation and down capacity activation, respectively, and are used in the upper-level problem to determine the profitability of the battery storage operation. The remaining dual variables defined for constraints (17)–(28) indicate how much this constraint worsen the objective function. If the value of a dual variable is zero, this constraint does not affect the objective function value, i.e., it is not binding.

Problem (1)–(2) is a bilevel problem and cannot be solved directly. Thus, the lower-level problem needs to be replaced by its equivalent constraints. We use Karush–Kuhn–Tucker optimality conditions to convert the initial bilevel problem into a mixed-integer linear program (MILP). An interested reader may find details on this mathematical technique in [30].

3.3. KKT Conditions of the Lower-Level Problem

The dual objective function:

$$\begin{array}{c}\mathrm{Maximize}-\sum _{t\in \mathcal{T}}{\overline{q}}_t^{\uparrow }·{\zeta }_t^{\uparrow }-\sum _{t\in \mathcal{T}}{\overline{q}}_t^{\downarrow }·{\zeta }_t^{\downarrow }+\sum _{t\in \mathcal{T}}{R}_t^{\mathrm{cap}\uparrow }·{\lambda }_t^{\mathrm{cap}\uparrow }+\sum _{t\in \mathcal{T}}{R}_t^{\mathrm{cap}\downarrow }·{\lambda }_t^{\mathrm{cap}\downarrow }\\ +\sum _{t\in \mathcal{T}}\sum _{s\in \mathcal{S}}{R}_{t,s}^{\mathrm{a}\uparrow }·{\lambda }_{t,s}^{\mathrm{a}\uparrow }+\sum _{t\in \mathcal{T}}\sum _{s\in \mathcal{S}}{R}_{t,s}^{\mathrm{a}\downarrow }·{\lambda }_{t,s}^{\mathrm{a}\downarrow }-\sum _{t\in \mathcal{T}}\sum _{i\in \mathcal{I}}{G}_{t,i}^{\uparrow }·{\psi }_{t,i}^{\uparrow }-\sum _{t\in \mathcal{T}}\sum _{i\in \mathcal{I}}{G}_{t,i}^{\downarrow }·{\psi }_{t,i}^{\downarrow }\end{array}$$

(29)

Dual constraints and stationarity conditions:

$$-\sum _{s\in \mathcal{S}}{\kappa }_{t,i,s}^{\uparrow }+C_i^{\mathrm{cap}\uparrow }-{\lambda }_t^{\mathrm{cap}\uparrow }-{\gamma }_{t,i}^{\uparrow }+{\psi }_{t,i}^{\uparrow }=0,\forall t,i$$

(30)

$$-\sum _{s\in \mathcal{S}}{\kappa }_{t,i,s}^{\downarrow }+C_i^{\mathrm{cap}\downarrow }-{\lambda }_t^{\mathrm{cap}\downarrow }-{\gamma }_{t,i}^{\downarrow }+{\psi }_{t,i}^{\downarrow }=0,\forall t,i$$

(31)

$${\pi }_s·C_i^{\mathrm{a}\uparrow }-{\lambda }_{t,s}^{\mathrm{a}\uparrow }-{\alpha }_{t,i,s}^{\uparrow }+{\kappa }_{t,i,s}^{\uparrow }=0,\forall t,i,s$$

(32)

$${\pi }_s·C_i^{\mathrm{a}\downarrow }-{\lambda }_{t,s}^{\mathrm{a}\downarrow }-{\alpha }_{t,i,s}^{\downarrow }+{\kappa }_{t,i,s}^{\downarrow }=0,\forall t,i,s$$

(33)

$$-\sum _{s\in \mathcal{S}}{\nu }_{t,s}^{\uparrow }-{\delta }_t^{\uparrow }+{\zeta }_t^{\uparrow }-{\lambda }_t^{\mathrm{cap}\uparrow }+C^{\mathrm{b},\mathrm{cap}\uparrow }=0,\forall t$$

(34)

$$-\sum _{s\in \mathcal{S}}{\nu }_{t,s}^{\downarrow }-{\delta }_t^{\downarrow }+{\zeta }_t^{\downarrow }-{\lambda }_t^{\mathrm{cap}\downarrow }+C^{\mathrm{b},\mathrm{cap}\downarrow }=0,\forall t$$

(35)

$$-{\beta }_{t,s}^{\uparrow }-{\lambda }_{t,s}^{\mathrm{a}\uparrow }+{\nu }_{t,s}^{\uparrow }+{\pi }_s·C^{\mathrm{b},\mathrm{a}\uparrow }=0,\forall t,s$$

(36)

$$-{\beta }_{t,s}^{\downarrow }-{\lambda }_{t,s}^{\mathrm{a}\downarrow }+{\nu }_{t,s}^{\downarrow }+{\pi }_s·C^{\mathrm{b},\mathrm{a}\downarrow }=0,\forall t,s$$

(37)

Complementarity slackness:

$$(-\sum _{i\in \mathcal{I}}{g}_{t,i}^{\mathrm{cap}\uparrow }-q_t^{\mathrm{cap}\uparrow }+R_t^{\mathrm{cap}\uparrow })\perp {\lambda }_t^{\mathrm{cap}\uparrow },\forall t$$

(38)

$$(-\sum _{i\in \mathcal{I}}{g}_{t,i}^{\mathrm{cap}\downarrow }-q_t^{\mathrm{cap}\downarrow }+R_t^{\mathrm{cap}\downarrow })\perp {\lambda }_t^{\mathrm{cap}\downarrow },\forall t$$

(39)

$$(g_{t,i}^{\mathrm{cap}\uparrow }-G_{t,i}^{\uparrow })\perp {\psi }_{t,i}^{\uparrow },\forall t,i$$

(40)

$$(g_{t,i}^{\mathrm{cap}\downarrow }-G_{t,i}^{\downarrow })\perp {\psi }_{t,i}^{\downarrow },\forall t,i$$

(41)

$$(-g_{t,i}^{\mathrm{cap}\uparrow }·\Delta t+g_{t,i,s}^{\mathrm{a}\uparrow })\perp {\kappa }_{t,i,s}^{\uparrow },\forall t,i,s$$

(42)

$$(-g_{t,i}^{\mathrm{cap}\downarrow }·\Delta t+g_{t,i,s}^{\mathrm{a}\downarrow })\perp {\kappa }_{t,i,s}^{\downarrow },\forall t,i,s$$

(43)

$$(q_t^{\mathrm{cap}\uparrow }-{\overline{q}}_t^{\uparrow })\perp {\zeta }_t^{\uparrow },\forall t$$

(44)

$$(q_t^{\mathrm{cap}\downarrow }-{\overline{q}}_t^{\downarrow })\perp {\zeta }_t^{\downarrow },\forall t$$

(45)

$$(-q_t^{\mathrm{cap}\uparrow }·\Delta t+q_{t,s}^{\mathrm{a}\uparrow })\perp {\nu }_{t,s}^{\uparrow },\forall t,s$$

(46)

$$(-q_t^{\mathrm{cap}\downarrow }·\Delta t+q_{t,s}^{\mathrm{a}\downarrow })\perp {\nu }_{t,s}^{\downarrow },\forall t,s$$

(47)

$$-g_{t,i,s}^{\mathrm{a}\uparrow }\perp {\alpha }_{t,i,s}^{\uparrow },\forall t,i,s$$

(48)

$$-g_{t,i,s}^{\mathrm{a}\downarrow }\perp {\alpha }_{t,i,s}^{\downarrow },\forall t,i,s$$

(49)

$$-q_{t,s}^{\mathrm{a}\uparrow }\perp {\beta }_{t,s}^{\uparrow },\forall t,s$$

(50)

$$-q_{t,s}^{\mathrm{a}\downarrow }\perp {\beta }_{t,s}^{\downarrow },\forall t,s$$

(51)

$$-g_{t,i}^{\mathrm{cap}\uparrow }\perp {\gamma }_{t,i}^{\uparrow },\forall t,i$$

(52)

$$-g_{t,i}^{\mathrm{cap}\downarrow }\perp {\gamma }_{t,i}^{\downarrow },\forall t,i$$

(53)

$$-q_t^{\mathrm{cap}\uparrow }\perp {\delta }_t^{\uparrow },\forall t$$

(54)

$$-q_t^{\mathrm{cap}\downarrow }\perp {\delta }_t^{\downarrow },\forall t$$

(55)

where all dual variables are nonnegative, but ${\lambda }_{t,s}^{\mathrm{a}\uparrow }$ and ${\lambda }_{t,s}^{\mathrm{a}\downarrow }$, which are unrestricted.

The equivalent mixed-integer nonlinear program is (1), (30)–(55). The nonlinearity comes from multiplications of the upper-level variables (cleared battery-related quantities) and lower-level dual variables representing up and down reserve capacity reservation and activation. These are linearized using some of the KKT conditions and the strong duality equation as follows. First, the term ${\lambda }_t^{\mathrm{cap}\uparrow }·q_t^{\mathrm{cap}\uparrow }$ is rewritten using KKT condition (34):

$${\lambda }_t^{\mathrm{cap}\uparrow }·q_t^{\mathrm{cap}\uparrow }=-\sum _{s\in \mathcal{S}}{\nu }_{t,s}^{\uparrow }·q_t^{\mathrm{cap}\uparrow }-{\delta }_t^{\uparrow }·q_t^{\mathrm{cap}\uparrow }+{\zeta }_t^{\uparrow }·q_t^{\mathrm{cap}\uparrow }+C^{\mathrm{b},\mathrm{cap}\uparrow }·q_t^{\mathrm{cap}\uparrow }$$

(56)

Considering (46) and (54), Equation (56) is equal to:

$${\lambda }_t^{\mathrm{cap}\uparrow }·q_t^{\mathrm{cap}\uparrow }=-\sum _{s\in \mathcal{S}}{\nu }_{t,s}^{\uparrow }·q_{t,s}^{\mathrm{a}\uparrow }+{\zeta }_t^{\uparrow }·q_t^{\mathrm{cap}\uparrow }+C^{\mathrm{b},\mathrm{cap}\uparrow }·q_t^{\mathrm{cap}\uparrow }$$

(57)

In a similar way, using (35), (47) and (55), we obtain the following equivalence:

$${\lambda }_t^{\mathrm{cap}\downarrow }·q_t^{\mathrm{cap}\downarrow }=-\sum _{s\in \mathcal{S}}{\nu }_{t,s}^{\downarrow }·q_{t,s}^{\mathrm{a}\downarrow }+{\zeta }_t^{\downarrow }·q_t^{\mathrm{cap}\downarrow }+C^{\mathrm{b},\mathrm{cap}\downarrow }·q_t^{\mathrm{cap}\downarrow }$$

(58)

The term related to up reserve activation can be rewritten using (36):

$${\lambda }_{t,s}^{\mathrm{a}\uparrow }·q_{t,s}^{\mathrm{a}\uparrow }=-{\beta }_{t,s}^{\uparrow }·q_{t,s}^{\mathrm{a}\uparrow }+{\nu }_{t,s}^{\uparrow }·q_{t,s}^{\mathrm{a}\uparrow }+{\pi }_s·C^{\mathrm{b},\mathrm{a}\uparrow }·q_{t,s}^{\mathrm{a}\uparrow }$$

(59)

where ${\beta }_{t,s}^{\uparrow }·q_{t,s}^{\mathrm{a}\uparrow }=0$ follows directly from (50). In a similar fashion, using (37) and (51) we obtain:

$${\lambda }_{t,s}^{\mathrm{a}\downarrow }·q_{t,s}^{\mathrm{a}\downarrow }={\nu }_{t,s}^{\downarrow }·q_{t,s}^{\mathrm{a}\downarrow }+{\pi }_s·C^{\mathrm{b},\mathrm{a}\downarrow }·q_{t,s}^{\mathrm{a}\downarrow }$$

(60)

Finally, combining the obtained equalities (57)–(60) with the strong duality equality (The strong duality theorem states that, under certain conditions which are satisfied for linear optimization problems such as the one at hand, optimal solutions to the primal and the associated dual problem yield the same objective value [30].) (12) = (29), we obtain the following linear objective function of the upper-level problem:

$$\begin{array}{c}\underset{{\mathsf{\Xi }}^{\mathrm{UL}}}{\mathrm{Maximize}}\sum _{t\in \mathcal{T}}\left[{\lambda }_t^{\mathrm{da}}(q_t^{\mathrm{dis}}-q_t^{\mathrm{ch}})+\right.\\ \left(C^{\mathrm{b},\mathrm{cap}\uparrow }·q_t^{\mathrm{cap}\uparrow }+C^{\mathrm{b},\mathrm{cap}\downarrow }·q_t^{\mathrm{cap}\downarrow }\right)+\sum _{s\in \mathcal{S}}\left({\pi }_s·C^{\mathrm{b},\mathrm{a}\uparrow }·q_{t,s}^{\mathrm{a}\uparrow }+{\pi }_s·C^{\mathrm{b},\mathrm{a}\downarrow }·q_{t,s}^{\mathrm{a}\downarrow }\right)-\\ \left(\sum _{i\in \mathcal{I}}{C}_i^{\mathrm{cap}\uparrow }·g_{t,i}^{\mathrm{cap}\uparrow }+C^{\mathrm{b},\mathrm{cap}\uparrow }·q_t^{\mathrm{cap}\uparrow }+\sum _{i\in \mathcal{I}}{C}_i^{\mathrm{cap}\downarrow }·g_{t,i}^{\mathrm{cap}\downarrow }+C^{\mathrm{b},\mathrm{cap}\downarrow }·q_t^{\mathrm{cap}\downarrow }\right)-\\ \sum _{s\in \mathcal{S}}{\pi }_s·\left(\sum _{i\in \mathcal{I}}{C}_i^{\mathrm{a}\uparrow }·g_{t,i,s}^{\mathrm{a}\uparrow }+C^{\mathrm{b},\mathrm{a}\uparrow }·q_{t,s}^{\mathrm{a}\uparrow }+\sum _{i\in \mathcal{I}}{C}_i^{\mathrm{a}\downarrow }·g_{t,i,s}^{\mathrm{a}\downarrow }+C^{\mathrm{b},\mathrm{a}\downarrow }·q_{t,s}^{\mathrm{a}\downarrow }\right)+\\ \left(R_t^{\mathrm{cap}\uparrow }·{\lambda }_t^{\mathrm{cap}\uparrow }+R_t^{\mathrm{cap}\downarrow }·{\lambda }_t^{\mathrm{cap}\downarrow }+\right.\\ \left.\left.\sum _{s\in \mathcal{S}}{R}_{t,s}^{\mathrm{a}\uparrow }·{\lambda }_{t,s}^{\mathrm{a}\uparrow }+\sum _{s\in \mathcal{S}}{R}_{t,s}^{\mathrm{a}\downarrow }·{\lambda }_{t,s}^{\mathrm{a}\downarrow }-\sum _{i\in \mathcal{I}}{G}_{t,i}^{\uparrow }·{\psi }_{t,i}^{\uparrow }-\sum _{i\in \mathcal{I}}{G}_{t,i}^{\downarrow }·{\psi }_{t,i}^{\downarrow }\right)\right]\end{array}$$

(61)

The final MILP formulation is (61) subject to constraints (2)–(11), (13)–(28), (30)–(55), where the orthogonal constraints (38)–(55) are easily linearized using the big M method.

4. Case Study

4.1. Input Data

The proposed model is tested on real data streaming from 1 May 2020. The day-ahead market prices, shown in

Table 1. Day-ahead market prices (${\lambda }_t^{\mathrm{da}}$) on 1 May 2020.

Hour	Price (€/MWh)	Hour	Price (€/MWh)	Hour	Price (€/MWh)	Hour	Price (€/MWh)
1	5.5	7	2.54	13	0.35	19	18.99
2	5.35	8	1.50	14	−2.04	20	23.50
3	3.82	9	−1.57	15	−2.06	21	28.43
4	2.63	10	−2.43	16	−0.04	22	26.88
5	1.56	11	−2.89	17	1.95	23	20.91
6	2.46	12	−2.47	18	7.88	24	16.00

were taken from the German electricity exchange—EPEX, while the capacity and energy bids were gathered from an online German platform for balancing reserves auctions—Regelleistung.net. The former dataset is a series of 24 day-ahead prices, while the latter dataset for automatic frequency restoration reserve (for up and down reserve separately) consists of six 4-h periods, each of them including the following: total aFRR up/down volume and series of volume–price pairs (capacity price–capacity volume–energy price). The first stage in the auction is arranging the capacity price–capacity volume pairs in an ascending order by price, where all bids up to the total required volume (shown in

Table 2. Required up ($R_t^{\mathrm{cap}\uparrow }$) and down ($R_t^{\mathrm{cap}\downarrow }$) reserve per 4-hour periods on 1 May 2020.

	Hours 1–4	Hours 5–8	Hours 9–12	Hours 13–16	Hours 17–20	Hours 21–24
Up reserve (MW)	2359	2334	2355	2344	2357	2360
Down reserve (MW)	2247	2295	2338	2354	2316	2303

) are accepted. Energy prices are used in the second stage in real-time when the TSO activates the accepted reserve providers. It arranges energy price–capacity volume pairs in an ascending order by price and all the bids up to total required energy are activated. For each 4-h period there are up to several hundred bids and many of them are identical both in terms of capacity and energy prices. To ease the computational efforts, we clustered similar ones and obtained between 30 and 90 total bids per timestep. Figure 2 shows the up reserve bids of the generators in the system. All the generators bid up a capacity reservation at zero €/MW (flat blue line), while the up reservation activation bids range from 36 to 2550 €/MWh (orange curve shows the activation bids sorted in ascending order). Figure 3 shows the down reserve bids of the generators in the system. As opposed to the up reserve, the down reserve capacity price is zero only for app. 500 €/MW, while the price of reservation of the remaining down reserve volume increases up to 8.6 €/MW (monotonically increasing blue curve). The corresponding activation prices are indicated with the orange curve. To minimize the operating cost (12), the system operator will activate the cheapest down reserve, i.e., the lowest values of the orange curve.

The data used to test and validate our model is taken from the German auction (www.regelleistung.net) and power system websites (www.smard.de) to accurately define one arbitrary chosen day. For the bids, real data for this specific day accounted for, on average, 283 and 333 bids over all bidding periods for the up and down reserve, respectively. In total in one day, there were 3697 bids for both up and down reserve during all bidding periods. A large number of those bids had the same values for both capacity and energy price or had the same number for one of those features and very similar for the other. To relieve the computational burden, but preserve the same level of accuracy, we aggregated those similar bids (in both features) and obtained on average 64 and 72 bids over all biding periods for the up and down reserve, respectively. This is in total 818 bids in one day for both up and down reserve during all bidding periods. This is still a very high number of bids even though the number of modeled bids was decreased by 88%. However, the accuracy of the case study remained untouched. When it comes to scenarios of activated aFRR, we used 10 scenarios as it is a sufficient number to validate the stochastic nature of the activation. Further increase in the number of scenarios would reduce computational efficiency for very low gains in the captured uncertainty.

Energy prices are used in the second stage in real-time when the TSO activates the accepted reserve providers. It arranges energy price–capacity volume pairs in an ascending order by price and all the bids up to total required energy are activated.

A strategic battery storage (energy capacity 50 MWh; power capacity 50 MW; charging efficiency 1; discharging efficiency 0.82) is then added to the mentioned merit order lists. The system operator in the second stage of the reserve allocation process takes the energy bids, arranges them by price (ascending for up reserve, and descending for down reserve) and activates them one by one until satisfying the balancing energy request at a specific moment. The request for the total activated energy is modeled as an uncertain parameter through scenarios. In the case study, we used the quarter-hour activated aFRR balancing energies taken from the German electricity data transparency platform www.smard.de. The quarter-hours were summarized to an hourly resolution to match the hourly resolution of our model. Note that the same data was also used in papers [27,28]. The data for ten days streaming form May 1 to 10 May 2020 were taken as ten scenarios in our case study. The up and down reserve activation data are shown in Figure 4 and Figure 5. To elaborate, each historical day (with all its hourly values) is shown as one scenario with a probability of 10%. The figures indicate a quite low activated volume, rarely surpassing 400 MWh, as compared to the reserved quantities from Table 2. Those scenarios affect our model results twofold: through the amount of activated reserve and through the price cleared for the activated reserve. In the case of batteries, the amount of activated reserve is relevant for securing a feasible state-of-energy evolution through time. It means that the state-of-energy boundaries will be satisfied regardless of which scenarios are actually realized. The price of activated reserves affects the profitability of reserve provision. The price maker models can be created in a way that their forecasted price is dependent on the activation scenarios as well, but they can not take into account the effect of the battery on the aFRR activated energy price formation.

In the results of the case study, presented in the following subsection, we first analyze the results of the battery storage placing all four of its bids, i.e., up reserve capacity, down reserve capacity, up reserve activation and down reserve activation, at zero price. Note that, due to the marginal pricing, the battery storage will receive the marginal price that can only be better or equal to the one it bid price. After this analysis, we provide a sensitivity analysis with different values that the battery storage bids for the up reserve capacity, down reserve capacity, up reserve activation and down reserve activation.

4.2. Results

The maximum profit battery storage can achieve using the given input data is €22,171.61. While the revenue from providing down reserve capacity is quite high, €6724.47, the revenue from providing up reserve capacity is much lower, €21.03. On the other hand, the activation revenues are similar, €8506.66 for up reserve and €7291.02 for down reserve. The revenue in the day-ahead market is negative €371.57, as the battery storage primarily uses it to charge the energy later used for reserve activation. Figure 6 shows the battery storage day-ahead schedule along with the cleared up and down reserve capacities. Positive values represent the battery charging process, while negative ones the battery discharging process. In the day-ahead market, the battery storage generally charges during the night hours. It occasionally discharges (during hours 6, 8, 9, 12–15 and 19), but never over 18 MW. Provision of up reserve capacity (when activated, the battery discharges), never breaks 18 MW neither. It is significantly lower in volume than the down reserve capacity provision, which reaches 28 MW in hour 15. In some hours, e.g., 15, the system operator reserved both up and down reserve capacity from the battery. The activated amounts will differ based on the reserve activation scenario. For a more detailed explanation of the energy storage reserve activation please consult section 2.2 in [31].

Figure 7 shows the propagation of the battery storage state-of-energy throughout the day for each scenario. Although the day-ahead schedule is the same, the activation direction (up or down) and the amount of activated reserve differs. For instance, in the 15-th hour, the battery storage reserves both up (3 MW) and down (28 MW) capacity. In scenario 3 we have 28 MW activated in the down direction and 1 MW in the up direction, while scenario 7 does not activate any up reserve, but activates 28 MW of down reserve. Since the modeled reserve is aFRR (15-minutes duration), a scenario can have activated both up and down reserve in the same hour (detailed visualization is available in Figure 8 and Figure 9). In all scenarios, the battery storage is quite depleted in hour 15 and charges at 20.6 MW in the day-ahead market in hour 16. In the same hour, five out of ten scenarios provide 5.5 MW of down reserve (compare to Figure 6), enabling the battery storage to further charge in those scenarios (this is seen in Figure 7 as the ensemble of five scenarios with higher values of state-of-energy in hour 16). On the other hand, in the remaining five scenarios the battery activates 13.3 MW of up reserve, which reduces the charging effect from the day-ahead market, and consequently the battery receives less overall charge in those scenarios (this is seen in Figure 7 as the ensemble of five scenarios with lower values of state-of-energy in hour 16). Figure 7 is also useful to illustrate that the ending state-of-energy is highly dependent on the reserve activation scenario and ranges from 6 MWh for scenario 3 to 41.3 MWh for scenario 8.

Activation of the energy storage up and down reserves per scenario are visualized and listed in Figure 8 and Figure 9. The numbers in the tables beneath these figures should be read column-by-column. In the first hour, the up reserve is fully activated (0.4 MW) in 9 out of 10 scenarios (Figure 8) and only in scenario s2 the battery up reserve remains inactive. The most noticeable property of the battery storage up reserve provision is having the activated capacity equal to the reserved capacity in the majority of scenarios. The lowest number of scenarios with fully activated up reserve occurs in hour 16, when only five scenarios experience full activation. Similar properties are observed for down reserve activation shown in Figure 9, where the lowest number of scenarios with fully activated reserve takes place in hour 15 with four full activations.

Generally, such uniform behavior of the battery storage reserve activation increases its utilization, i.e., the revenue of reserve activation, and harmonizes the state-of-energy across all scenarios. Since the last term in objective function (1) considers the weighed activation revenue, if the actual up reserve activation price in a certain hour of a scenario with 10% probability is €50/MWh, the value of the dual variable ${\lambda }_{t,s}^{\mathrm{a}\uparrow }$ would be €5/MWh. This is a direct consequence of scenario probability ${\pi }_s$ multiplying the activation costs in lower-level objective function (12).

To provide a better insight into the role of the battery storage in the overall reserve activation process,

Table 3. Volume of up reserve activation provided by the battery per scenario as a portion of the overall activated reserve (rounded to an integer unless close to zero), in MWh.

Hour	s1	s2	s3	s4	s5	s6	s7	s8	s9	s10
1	0.4/400	0/0	0.4/327	0.4/8	0.4/2	0.4/17	0.4/38	0.4/116	0.4/79	0.4/10
2	16/377	10/10	16/341	16/21	8/8	16/16	13/13	16/70	16/69	16/177
3	10/170	10/10	10/156	10/32	10/23	10/38	10/34	10/28	10/129	10/107
4	17/67	0/0	17/32	7/7	17/68	17/53	12/12	17/74	17/62	2/2
5	1/1	2/2	5/7	0/0	5/11	5/17	5/223	5/5	5/15	4/4
6	0/0	0/3	0/34	0/39	0/4	0/5	0/72	0/0	0/2	0/9
7	2/5	2/13	2/80	2/28	2/52	2/48	2/182	2/2	0/0	2/5
8	0/4	0/47	0/44	0/17	0/48	0/33	0/35	0/0	0/0	0/213
9	1/2	1/17	1/6	1/279	1/138	1/33	1/9	1/1	0/0	1/489
10	0/0	8/20	1/1	8/50	8/54	8/16	8/118	1/1	2/2	8/102
11	7/7	1/1	7/31	7/57	7/263	7/7	7/25	7/13	0/0	7/8
12	0/554	0/2	0/841	0/63	0/108	0/7	0/10	0/0	0/4	0/1
13	1/123	1/23	1/74	1/57	1/58	1/5	1/89	1/17	1/104	1/0
14	4/336	4/192	0/0	4/110	4/22	4/4	4/17	4/125	4/124	0/0
15	3/349	3/484	1/1	3/69	3/148	3/3	0/0	3/15	3/116	3/753
16	13/260	13/86	2/2	13/219	1/1	7/7	3/3	5/5	13/76	13/1119
17	9/477	9/375	9/157	9/50	2/2	2/2	4/4	9/27	3/3	9/37
18	2/97	2/115	2/263	2/27	2/14	2/2	2/89	2/42	2/10	2/165
19	6/20	6/67	6/163	6/6	6/52	6/6	6/139	6/27	6/13	6/74
20	4/98	4/69	4/125	3/3	4/226	4/129	4/103	2/2	4/5	2/2
21	1/157	1/30	1/84	1/20	1/23	1/70	1/93	1/1	1/142	1/5
22	0/0	3/76	3/131	3/15	3/62	3/205	3/508	3/17	3/128	3/3
23	7/12	1/1	7/27	7/43	2/2	7/30	7/68	7/10	7/40	7/158
24	6/74	6/16	6/190	6/8	6/10	6/6	6/23	6/29	6/17	6/148

and

Table 4. Volume of down reserve activation provided by the battery per scenario as a portion of the overall activated reserve (rounded to an integer unless close to zero), in MWh.

Hour	s1	s2	s3	s4	s5	s6	s7	s8	s9	s10
1	1/8	1/209	1/6	1/250	1/430	1/37	1/88	1/57	1/10	1/57
2	0/5	0/132	0/4	0/118	0/72	0/43	0/23	0/15	0/10	0/8
3	0/4	0/28	0/5	0/32	0/65	0/20	0/31	0/12	0/6	0/8
4	0/15	0/166	0/5	0/68	0/20	0/27	0/16	0/6	0/12	0/77
5	0/201	0/48	0/9	0/200	0/18	0/49	0/6	0/56	0/81	0/12
6	18/72	5/40	13/48	18/55	18/69	5/40	0/15	18/407	18/247	18/64
7	0/7	0/100	0/4	0/84	0/21	0/66	0/26	0/373	0/214	0/24
8	0/8	0/4	0/14	2/63	2/65	2/112	2/105	2/516	2/477	0/5
9	16/380	0/29	0/27	0/20	0/32	16/173	16/183	16/403	16/181	0/4
10	11/163	11/57	11/71	11/42	0/15	11/444	0/14	11/648	11/237	10/41
11	2/38	2/148	0/12	0/27	0/4	2/455	2/33	2/369	2/333	2/384
12	0/5	6/84	0/4	0/24	0/5	6/108	6/58	6/827	6/65	6/504
13	10/30	10/119	10/28	4/9	10/89	10/54	2/7	10/169	2/7	10/651
14	9/14	11/20	11/246	11/49	11/179	11/387	11/253	11/70	1/6	11/392
15	0/1	0/2	28/60	16/21	28/180	28/198	28/603	4/9	0/5	9/14
16	0/2	0/4	6/74	0/5	6/559	6/99	6/271	6/32	3/8	1/6
17	0/0	0/2	0/36	0/8	0/165	0/394	0/127	0/119	0/134	0/62
18	0/5	0/8	0/6	0/11	0/186	0/292	0/290	0/231	0/284	0/30
19	11/83	11/32	0/14	11/133	11/127	11/546	11/32	11/104	11/113	0/11
20	1/21	0/9	0/5	7/231	0/19	7/121	7/92	7/293	7/85	7/234
21	0/30	0/12	0/5	0/124	0/9	0/16	0/3	0/165	0/6	0/48
22	7/498	0/40	0/16	7/103	0/36	0/45	0/8	7/159	0/18	7/155
23	10/167	10/274	0/27	1/47	10/341	10/89	3/49	10/223	10/118	0/7
24	0/23	2/125	0/15	2/151	2/145	2/141	2/173	2/163	2/155	2/53

provide ratios of the reserve activation provided by the battery storage and the overall activated reserve for up and down direction. In the first hour, scenarios significantly vary in terms of the activated up reserve (Table 3). For scenario 1 the battery storage provides only 0.1% out of the activated 400 MWh. The same volume of battery’s up activation in scenario 5 consists of 20% of the overall up reserve (0.4/2 MWh). In the second hour, the battery provides up to 16 MWh of the up reserve. In scenarios 2, 5, 6 and 7 this is sufficient to cover the entire required up reserve volume. When it comes to down reserve, the battery does not provide any portion in hours 2–5 (Table 4). In hour 9, it does not provide any reserve in scenarios that require low volumes, but it becomes active once the volumes increase (scenarios 1 and 6–9). This is because the down reserve activation prices of certain generators are negative (see the orange curve in Figure 3) and those are prioritized in the activation phase over the battery storage whose activation price is zero.

To better understand battery storage actions, the prices in different markets are shown in Figure 10. As shown in Table 1, the day-ahead prices are rather low throughout the day, taking the highest values in hours 19–24. The up capacity prices are zero (or slightly positive) throughout the day, which reflects the very low day-ahead market prices. The down capacity prices are much higher, reaching €93/MW in the afternoon hours. The up and down activation prices in Figure 10 are averaged over all scenarios. They are much higher than the day-ahead prices. Despite extremely low up reserve capacity prices, the activation prices are much higher. The peak price €235/MWh is achieved for up reserve activation in hour 16, which is the main reason for the battery storage reserving 13.3 MW of its up capacity and activating it fully in five out of ten scenarios.

4.3. Sensitivity Analysis

This section analyzes the effects of the battery storage bidding prices on its overall profit using the same data as the simulations in the previous section. The sensitivity includes variations in the four bidding parameters related to the reserves market: (i) up capacity reservation price (€/MW), (ii) down capacity reservation price (€/MW), (iii) up capacity activation price (€/MWh), iv) down capacity activation price (€/MWh). The results presented in

Table 5. Effect of the bidding parameters on the battery storage profit (in €); the four numbers in the top cells indicate (i) up capacity reservation price (€/MW), (ii) down capacity reservation price (€/MW), (iii) up capacity activation price (€/MWh), (iv) down capacity activation price (€/MWh).

	Day-Ahead Revenue	Up Capacity Res. Revenue	Up Capacity Act. Revenue	Down Capacity Res. Revenue	Down Capacity Act. Revenue	Overall Revenue
(0,0,0,0)	−372	21	8507	6724	7291	22,172
(1,1,0,0)	−368	22	8835	6511	7163	22,162
(1,1,25,0)	−354	21	8684	6804	7274	22,429
(1,1,25,15)	−366	21	8316	5568	5630	19,168
(5,5,0,0)	−360	22	8588	6703	7205	22,158
(5,5,25,0)	−327	21	8066	7162	7515	22,438
(5,5,25,15)	−360	22	8096	5713	5698	19,169
(5,5,50,0)	−7	5	5386	5605	6352	17,342
(10,10,50,−15)	1033	7	3585	9172	12,416	26,212

indicate that, regardless of the bidding prices, the battery storage utilizes the day-ahead market to charge (thus the day-ahead revenue is always negative), while the profit is made in the capacity reservation and activation stage. The only exception is the bidding strategy (10,10,50,−15), which has a high day-ahead positive revenue. This is the result of very frequent down capacity activation (the revenue is €12,416), which, besides that revenue itself, benefits the battery storage by charging it. This energy is discharged in the evening hours with the highest day-ahead prices to bring additional revenue in the day-ahead market.

Up reserve capacity revenue is generally very low, which is a direct consequence of the very low (mostly zero) up capacity reservation prices (see orange curve in Figure 10). However, the up capacity activation prices are high, especially during hour 16, and in most cases this stream of revenue is the highest. Down capacity reservation revenue is usually slightly higher than the activation revenue thanks to the high down capacity reservation prices during the afternoon hours (see the gray curve in Figure 10). The only exception are the last two cases.

The highest daily profit is achieved for bidding at €10/MW for both up and down capacity reservation, €50/MWh for up reserve activation and −€15/MW for down reserve activation. These bidding prices enable the battery storage to both affect the clearing prices (mostly by increasing them in its favor) and to win the auction in the majority of hours and scenarios. On the other hand, (5,5,50,0) bidding scheme results in the lowest overall profit, mostly because the high up reserve activation price €50/MWh reduced the up capacity activation revenue. However, the down activation bid at €0/MWh is insufficiently low for the battery storage to provide enough down reserve activation revenue to cancel out the negative monetary effects of the high up activation bid. On the other hand, the case with the highest profit (10,10,50,−15) provides sufficiently low down capacity activation bid for the battery storage to be cleared for activation more frequently and results in the highest down reserve activation revenue €12,416. This bidding strategy also results in the highest down capacity reservation revenue.

4.4. Comparison to a Baseline Model

To demonstrate the effectiveness and practical importance of the proposed model, we compare it against a baseline model where the battery storage acts as a price taker in all the markets and disregards its impact on the reserve capacity and activation prices. The baseline model includes only the upper-level problem (1) with capacity reservation and activation prices (${\lambda }_t^{\mathrm{cap}\uparrow }$, ${\lambda }_t^{\mathrm{cap}\downarrow }$, ${\lambda }_{t,s}^{\mathrm{a}\uparrow }$, ${\lambda }_{t,s}^{\mathrm{a}\downarrow }$) treated as parameters. The obtained schedule is included in the market-clearing lower-level problem to obtain the actual profitability of the baseline model. The baseline model assessment procedure is described in the following steps:

• First we solve only the lower-level problem (2) without battery storage bids, i.e., setting ${\overline{q}}_t^{\uparrow }$ and ${\overline{q}}_t^{\downarrow }$ to zero. This is needed to obtain the capacity reservation and activation prices ${\lambda }_t^{\mathrm{cap}\uparrow }$, ${\lambda }_t^{\mathrm{cap}\downarrow }$, ${\lambda }_{t,s}^{\mathrm{a}\uparrow }$ and ${\lambda }_{t,s}^{\mathrm{a}\downarrow }$.
• Then we solve the upper-level problem (1) using the capacity reservation and activation prices ${\lambda }_t^{\mathrm{cap}\uparrow }$, ${\lambda }_t^{\mathrm{cap}\downarrow }$, ${\lambda }_{t,s}^{\mathrm{a}\uparrow }$ and ${\lambda }_{t,s}^{\mathrm{a}\downarrow }$ from the previous step. Note that the capacity reservation and activation prices are treated as parameters as opposed to being treated as variables in the proposed formulation. The outcome is the battery storage day-ahead and reserves bids.
• Finally, we solve the lower-level problem (2) again, but this time with battery storage bids ${\overline{q}}_t^{\uparrow }$ and ${\overline{q}}_t^{\downarrow }$ from the previous step. This calculation provides actual reserve capacity and activation prices (note that these may differ from those obtained in step 1) as well as cleared battery storage quantities and profit.

After running Step 2 using the reserve capacity and activation prices from Step 1, the obtained battery storage profit is €57,518, which is more than two and a half times higher than €22,172 obtained using the proposed model. The obtained battery operation schedule for the baseline model is shown in Figure 11. The battery storage very rarely charges in the day-ahead market, the majority in hour 16. The battery charges primarily through the provision of down-regulation capacity.

The obtained battery storage bidding schedule is then applied to the lower-level problem to calculate the reserve capacity and activation quantities actually accepted in the market and to deliver the true profit as the actual profit is expected to decrease if the battery storage’s bids had an effect on the reserve capacity and activation prices. The obtained actual profit of the battery storage is only €8856, which is almost three times lower than €22,172 obtained using the proposed model. Although all battery storage bids were accepted in the market, the obtained baseline battery scheduling process failed to capture the interaction between the battery storage bids and the market-clearing prices. The result is a much lower profit than when using the proposed model, thus proving the effectiveness of the formulation presented in this paper.

5. Conclusions

The paper presented a model for the optimal bidding strategy of battery storage acting in the day-ahead market as a price taker and in the aFRR market as a price maker. The model accurately captures the essence of the electricity market structure in Europe, which is in the process of shifting toward an hourly marginal-price reserve structure. Although the battery storage from the case study is relatively small in size as compared to the overall reserves market volume (50 MW as opposed to over 2.3 GW), the battery storage can significantly affect aFRR reserve market since the activated energy is usually quite low. The bidding prices of the battery storage may have an adverse effect on its profit. Thus, the bidding prices and quantities need to be carefully chosen so the battery storage affects the market prices in a desirable way, but still stays in the money, i.e., gets cleared to provide reserve capacity and, when necessary, becomes activated.

1 May 2020, the day used in the case study, is characterized by a rather low reserve capacity prices. Despite that, the battery storage profit is significant and bidding in the reserves market is much more profitable than bidding only in the energy market. Since most of the days in the year 2020 have higher reserve capacity prices, these results can be considered conservative, i.e., the lower bound on the profits to be achieved in German markets.

The presented model and results should be useful to project developers and battery storage market participants as the battery storage costs are still quite high and accurately seizing all potential revenue streams is essential for the profitability of such investment.