The Cost Reduction Potential of Demand Response in Balancing Markets from a System Perspective

Abstract

Demand response (DR) can potentially provide a cost-efficient alternative for balancing the electricity grid by replacing fossil-fuelled power plants for the provision of flexible capacity. This paper aims to quantify the cost reduction potential of DR from a system perspective. Historical data of balancing markets are studied using regression and average bid price analysis to quantify the effect of the participation of DR resources on the price of flexible capacity for the provision of balancing reserves by focusing on two case studies in Great Britain and the Netherlands. It is estimated that DR bids are, on average, 35% lower than the market average. The regression analysis concluded that 1% higher participation of DR in balancing markets leads, on average, to a 2.7% lower prices for flexible capacity. The results verify the hypothesis that flexible DR capacity is offered at a lower price on balancing markets compared to conventional generation resources, resulting in lower costs for grid operators to balance the grid, thus reducing societal costs for electricity provision and overall emissions through the integration of low-carbon balancing resources.

1. Introduction

In 2021, the European Union (EU) set the ambitious goal of reaching net-zero greenhouse gas emissions in 2050 [1]. To achieve this goal, a shift towards renewable energy sources (RES) has to be made. Wind and solar resources have the highest potential in generating renewable electricity in the EU [2]. However, the electricity grid will face significant challenges due to a shift towards RES, such as solar and wind energy, as electricity production will be increasingly intermittent, volatile, and unpredictable. Therefore, one of the main challenges of the energy transition is to constantly maintain the balance between the supply and demand of electricity, as this is essential for a reliable, efficient, and safe operation of the electricity system [3]. The traditional way to balance the electricity grid is mainly by adjusting the power output of fossil-fuelled power plants to match the demand. However, the decommissioning of fossil-fuelled power plants is essential for Europe’s low-carbon future, and as such, there is a need for alternative flexible resources to balance the electricity grid in the future. Nowadays, there is a growing consensus among scientists, policymakers, and electricity market participants that demand response (DR) is an essential source of flexibility that needs to be developed to maintain a balanced electricity grid in the future [4]. In contrast to the traditional way, with DR, the imbalance in the electricity system is solved by adjusting the demand for electricity to match it with the supply [5]. With the growing penetration of renewable energy sources in the energy system, DR will play an increasingly important role in balancing the supply and demand of electricity.

In Europe, transmission system operators (TSO) organise and operate single-buyer balancing markets for flexible capacity to balance the grid. In this paper, it is investigated whether DR can reduce costs for balancing the electricity system by having a lower marginal cost than traditional balancing methods. The assumption is that when DR has a lower marginal cost, DR capacity will be offered on the balancing markets for a lower price. With this marginal bidding assumption, it is assumed that the bid price reflects the marginal costs of providing flexible capacity from a particular flexibility source. When more flexible capacity is available for a lower bid price, the price for flexible capacity will decrease. This leads to the hypothesis of this research, that higher participation of DR on balancing markets reduces the settled prices of flexible capacity, and thus the costs incurred by TSOs. Reduced costs for TSOs can lower societal costs for electricity supply.

1.1. Literature Review and Research Gap

The economic benefits of DR from a system perspective are often discussed in academic literature. In this section, the academic and grey literature that assess the cost reduction of DR is reviewed.

In a report of the European Commission, it is stated that DR can offer “cheaper and cleaner solutions to balancing the grid” [6]. However, the report lacks a detailed research methodology. Strbac [7] assessed a modest economic benefit of DR in system flexibility by comparing the operational costs of DR to those of a gas turbine. Ofgem [8], the energy regulator for Great Britain (GB), similarly evaluated the system cost reduction from DR by comparing its operational costs to those of a power generation plant. Nonetheless, both studies did not explore how the reduced operational costs of DR might lower balancing market prices and consequently decrease balancing costs for grid operators. Koliou et al. [9] investigated the cost reduction of DR for grid operators, concluding that the primary economic advantage of DR is the reduction in infrastructural grid costs due to decreased peak load. However, the paper does study the effect of DR participation on balancing market prices, and is therefore not relevant for testing the hypothesis of this research.

Several studies estimate the economic benefit of DR using a modelling approach. Vlachos and Biskas [10] simulated the costs incurred by grid operators on balancing markets by comparing the cost effectiveness of different balancing market designs for a specific region, including DR resources. Nevertheless, their study did not specifically estimate the effect of the participation of DR on the balancing costs. Dietrich, Latorre, Olmos, and Ramos [11], using a modelling approach, found that implementing DR for grid balancing could reduce system costs and be “economically reasonable” from a system operator’s perspective. However, they did not quantify the economic benefit. Gils [12] conducted a case study on Germany, simulating the country’s energy system, and concluded that the main benefit of DR is its ability to replace peak power generation capacity, thereby reducing system costs. Similarly, Xiang et al. [13] and Klaassen et al. [14] performed a cost–benefit analysis of DR, considering the energy system as a whole, without focusing on the specific balancing costs for grid operators. Consequently, these studies cannot be used to test the hypothesis that DR reduces costs for grid operators by lowering balancing market prices.

Walawalkar et al. [15] took a different approach and conducted a case study on the PJM electricity market in the United States. They compared the social welfare gains with the subsidies paid to price-responsive loads. The strength of their paper is that it uses actual load and price data from 2006. However, they still needed to simulate DR bids to estimate the economic benefit of DR participation in the electricity market. Bradley et al. [16] have provided a comprehensive overview of the different system cost reductions of DR but stated that “it was not possible to provide an average annual estimate of the value of DR to avoid the need for generation capacity to provide a reserve for emergencies/unforeseen events”.

The Dutch TSO, TenneT, in its Annual Market Update 2019, reported that the prices for flexible capacity had dropped in the Dutch balancing market due to increased offered capacity since the moment that DR was allowed to participate [17]. However, this does not detail specifically DR, but rather the effect of increased liquidity on these markets. In the Power Responsive reports of GB’s TSO, National Grid, insights have been provided on the average bid prices of DR on balancing markets, which show that DR bid prices are usually lower than traditional flexibility sources [18]. These reports form a sound basis for supporting both the hypothesis and the analysis of the bid prices of DR. However, the depth of analysis is limited, and external factors affecting prices have not been taken into account. Therefore, a more in-depth analysis is needed to estimate the cost reduction of DR on balancing markets.

While different methods have been employed in the existing literature to estimate the cost reduction potential of DR, a generic quantitative method is lacking with which stakeholders can identify relationships between the sources that provide flexibility and the balancing market prices. The literature review clearly indicates the lack of studies that quantitatively estimate the cost reduction of DR in grid balancing using historical balancing market data. This gap can be attributed to the difficulty of conducting such analyses in most countries, where the necessary data are not publicly accessible. As a consequence, current research outputs cannot be used for the estimation of the cost reduction potential of flexibility sources from a system perspective, and cannot be strictly comparable within the context of different countries and balancing markets. The identified research gap sets the basis for the scope of this research and the formulation of the main research question as follows:

What is the cost reduction potential of DR participation in balancing markets from a system perspective?

1.2. Introduction to Methodology

To quantify the cost reduction potential, it is important to estimate the relationship between two main variables, i.e., the price of flexible capacity on balancing markets and the share of DR in the total accepted capacity on balancing markets. This paper presents a method to identify this relationship, which applies a regression analysis in a novel way by using historical data of balancing markets as input. To complement the regression analysis, an average bid price analysis is conducted to compare the average bid price of DR bids with the overall average of the market, which is used as a measure for the cost reduction of DR. These methodological steps are applied to two case studies, corresponding to the manual frequency restoration reserve (mFRR) balancing product in the Netherlands (NL) and the firm frequency response (FFR) balancing product in GB. For NL, undisclosed data were used, specifically made available for this research by the Dutch TSO TenneT. For GB, public data of TSO National Grid were used.

1.3. Paper Layout

The paper is structured as follows: the research method is presented in Section 2, and the results follow in Section 3. The results are discussed in Section 4, and the paper is concluded in Section 5, where future research and policy recommendations are provided.

2. Research Methodology

In this section, the research methodology for the estimation of the cost reduction of DR from a system perspective is presented. The data collection and relevant variables are presented in Section 2.1. The methods for the regression analysis and the average bid analysis are presented in Section 2.2 and Section 2.3, respectively.

2.1. Data Collection and Variables

The data collection addresses the two identified case studies in NL and GB. The balancing sub-markets in scope are listed in

Table 1. Input data for estimation of cost reduction of demand response.

Balancing Sub-Market	Country	Data Source	Period
FFR dynamic (FFRd) FFR static (FFRs)	GB	Post Tender Reports National Grid [19]	04-2018/01-2022
mFRRda Upward mFRRda Downward aFRR Symmetrical aFRR Upward aFRR Downward	NL	Balancing market auction results (undisclosed data, specifically made available for this research by Dutch TSO TenneT)	01-2018/12-2021

, including the data source and the investigated periods.

In contrast to the symmetrical GB balancing sub-markets, the Dutch balancing markets are separated per the specific direction of the flexible capacity. Flexible capacity in the upward direction refers to an increase in production or a decrease in consumption of electricity. Flexible capacity in the downward direction refers to a decrease in production or an increase in consumption of electricity. In a symmetrical balancing market, flexible capacity should be available in both upward and downward directions [20].

The data on the balancing markets in GB and NL are translated into the variables used in this research. The symbols for these variables are listed in

Table 2. Symbols used to define variables and parameters for analyses.

Variable	Symbol	Unit
Flexible capacity	$C$	$MW$
Share of flexibility source in total accepted capacity	$S$	$\%$
Price of flexible capacity	$P$	$\frac{€}{MW/h}$$\left(\mathrm{NL}\right) \mathrm{or} \frac{\pounds }{MW/h}$ (GB)
Average price of flexible capacity	$\overline{P}$	$\frac{€}{MW/h}$$\left(\mathrm{NL}\right) \mathrm{or} \frac{\pounds }{MW/h}$ (GB)
Sub-Script	Symbol	Example Value
Flexibility source	$f$	Battery, gas turbine, demand response, etc.
Balancing sub-market	$m$	GB: FFRd (dynamic), FFRs (static) NL: aFRR, mFRR
Tender period	$t$	Month, day of year, ½ hour
Reference to specific bid	$b$

.

To test the hypothesis of this research, the variables used in this research are defined as follows:

• Average bid price on balancing sub-market $m$ per flexibility source $f$ per tender period $t$ $\left(\overline{P_{m,f,t}}\right)$.
• The share of flexibility source $f$ in the total accepted capacity per tender period $t$ on balancing sub-market $m$ $\left(S_{m,f,t}\right)$.
• The price of accepted flexible capacity bids of balancing sub-market $m$ per tender period $t$ $\left(P_{m,t}\right)$.
• The total flexible capacity accepted on balancing sub-market $m$ per tender period $t$ ($C_{m,t}$).

It is important to note that the variable for flexible capacity, $C_{m,t}$, has a different interpretation per sub-market. The main difference is the time horizon over which it specifies the flexible capacity. For example, in Dutch mFRR markets, the variable defines how much MW of flexible capacity was reserved by the TSO for one month, which, therefore, could be linearly activated by the TSO to balance the grid. For other markets, the time horizon for the variable $C_{m,t}$ is one week or one day. To calculate this variable, only the accepted capacity bids are considered, as the capacity that is bid but rejected does not influence the settled prices of flexible capacity on balancing markets. This capacity does not affect the costs incurred by TSOs to balance the grid and is, therefore, irrelevant to this research.

It is also important to note that the FFR markets in the UK and the Dutch mFRR markets have a pay-as-bid pricing scheme in which there is not a single price per period. Every bidder receives their bid price when their bid is accepted. Therefore, the variable $P_{m,t}$ is, in this case, defined as the average of all the accepted bid prices per period. The Dutch aFRR market has a pay-as-cleared (marginal pricing) pricing scheme in which every accepted bidder receives the bid price of the highest accepted bid. Therefore, the balancing market prices in the two types of markets should be interpreted differently.

2.2. Regression Analysis

With the combination of $S_{m,f,t}$ and $P_{m,t}$, the hypothesis can be tested by identifying the trend between the two variables, with DR as the flexibility source of interest. The trend analysis is performed using regression analysis, which tests whether a higher share of DR in balancing markets (independent variable) is associated with a lower price for flexible capacity (dependent variable) that can support system cost reduction. Variable $C_{m,t}$ is used as a control variable in the regression model to check whether a higher accepted capacity is associated with a higher price for flexible capacity. Other external independent variables, such as gas price (${GP}_t$), are also included in the regression model as control variables when they are shown to be statistically significant. Adding these control variables increases the accuracy of the regression model.

An example regression model in this context is as follows:

$$P_t=a+\beta { S}_{DR,t}+\gamma {GP}_t+\delta C_t+\epsilon$$

(1)

The intercept $a$ and the coefficients $\beta$, $\gamma ,$ and $\delta$ are estimated by the regression model based on the input data. The error term $\epsilon$, or residual, is the difference between the predicted value of the dependent variable and the actual value. The accuracy in predicting the dependent variable using the independent variables is maximised by parameterizing the coefficients to minimise the error term. When the regression model estimates a negative relationship between $P_t$ and ${ S}_{DR,t}$, the results support the hypothesis that DR participation can result in cost reduction for balancing the grid.

2.3. Average Bid Price Analysis

The average bid price analysis is used to complement the regression analysis by providing a different measure for the same estimation of the cost reduction in DR. This analysis is used to validate the regression analysis results by comparing the average bid price of DR flexible capacity ($\overline{P_{DR}}$) with the overall average bid price per balancing market ($\overline{P}$). The variables for the average bid prices are defined in Equations (2) and (3). When DR bids are lower than the market average, TSOs can reduce costs when more flexible capacity is bought from DR resources. This would support the hypothesis of this research.

$$\overline{P_{DR}}=\frac{{\sum }_{b_{DR}=1}^{B_{DR}}{P}_{b,DR}}{B_{DR}}, (b_{DR} \in 1, 2, \dots , B_{DR})$$

(2)

$$\overline{P}=\frac{{\sum }_{b=1}^B{P}_b}{B}, (b \in 1, 2, \dots , B)$$

(3)

Capacity-weighted averages were also considered for calculating $\overline{P}$ and $\overline{P_{DR}}$. These weighted averages yielded results comparable to those obtained using simple averages. Therefore, for the sake of simplicity, simple averages were employed in the average bid price analysis.

3. Results and Analysis

3.1. Results Case Study: Great Britain

The flexibility source associated with DR on the FFR market in GB is referred to as Load Response. To provide some context to the role of this flexibility source in this balancing sub-market, Figure 1 shows the average shares of the flexibility sources in the total accepted volume of the period in scope. The percentage of Load Response is 13% and 13% for the FFRd and FFRs, respectively. Next, Figure 2 shows a scatterplot of the share of Load Response and the price of flexible capacity on the FFR market per tender period in scope. It shows a negative trend for the FFRd market, whereas no visible trend can be identified for the FFRs market.

3.1.1. Regression Analysis: FFR Market

First, it was tested whether the data are suitable for building a regression model. This test shows that a regression model is only suitable for the FFRd, and not for the FFRs market. Data of the static FFRs market only contain a price for flexible capacity for 34 months (excluding outliers). In the other months (25% of the time), no static flexible capacity was bought; thus, no price was settled. The regression model built based on these 34 months fails to have statistically significant results, as shown in

Table 3. Results regression model of FFRs market.

Dependent Variable	Symbol	Unit
Price of flexible capacity	$P_{FFRs}$	£/MW/h
Independent Variables	Symbol	Unit	Coefficient	t Stat	Standard Error	p-Value
Intercept	$a_{FFRs}$		2.6	9.3	0.28	0.000
Load response	$S_{FFRs,Load Response}$	%	0.0031	0.29	0.010	0.771
Diesel	$S_{FFRs,Diesel}$	%	−0.13	−2.68	0.0050	0.012
Distributed generation (for export)	$S_{FFRs,DGexport}$	%	−0.0092	−1.39	0.0066	0.177
Gas price	$GP$	£/therm	0.0044	1.46	0.0030	0.155
Regression Statistics
R2	0.28
Adjusted R2	0.18
Standard error	4.61
F-test (p value)	0.04
Observations	34

. Here, it can be seen that only the participation of the generation-type Diesel has a significant effect on the price, that the observations/number of predictors ratio is too low, and that the predictive value of the model is low (R² = 0.28). Therefore, only the regression results of the FFRd market are analysed further.

The results of the dynamic FFRd regression model can be found in

Table 4. Regression model FFRd market.

Dependent Variable	Symbol	Unit
Price of flexible capacity	$P_{FFRd}$	£/MW/h
Independent Variables	Symbol	Unit	Coefficient	t Stat	Standard Error	p-Value
Intercept	$a_{FFRd}$		7.661	10.787	0.71	0.000
Battery	$S_{FFRd,Battery}$	%	−0.059	−3.241	0.018	0.002
Load response	$S_{FFRd,Load Response}$	%	−0.12	−5.421	0.021	0.000
Gas price	${GP}_{GB}$	£/therm	0.0550	9.0331	0.0072	0.000
Regression Statistics
R2	0.71
Adjusted R2	0.69
Mean absolute percentage error (MAPE)	24%
Standard error	2.19
F-test (p value)	0.00
Observations	45

. The independent variables that have been shown to be statistically significant are $S_{FFRd,Battery}$ and $S_{FFRd,Load Response}$. The variables of the other flexibility sources have been shown to be statistically insignificant. An external independent variable that has appeared to be significant is the GB gas price ${GP}_{GB}$.

Table 5. Results statistical tests on regression model dynamic FFRd market.

Statistical Test	Test	Result	Interpretation Result
Multicollinearity	Correlation matrix	The strongest correlation between Gas price and Price of 0.69	Debatable
	VIF test	1.09	Acceptable
Sample size/number of predictors ratio		11	Acceptable
Normality of residuals	Shapiro–Wilk test	p-value of 0.36	Acceptable
	QQ-plot and residual plot	No trend between residuals and independent variables	Acceptable
Goodness-of-fit	R2	0.71	Reasonably high
	F-test	0.0000	Acceptable

shows the results of the statistical tests, which are used to test the validity of the results of the regression model. The data show that a strong correlation of 0.69 has been found between ${GP}_{GB}$ and $P_{FFRd}$. This is just below the critical value of 0.7, above which multicollinearity in the data is suggested [21]. Multicollinearity could cause problems for the estimation of the coefficients in the regression model. Contrarily, the variance inflation factor analysis (VIF analysis) shows that there is no collinearity. Therefore, the regression model is accepted.

Figure 3 and the regression statistics show that the regression model predicts the price of flexible capacity, $P_{FFRd}$, reasonably well (R² = 0.71 and MAPE = 24%). It shows a negative relationship between $P_{FFRd}$ and $S_{FFRd,Load Response}$ (−0.12 $\frac{\pounds /MW/h}{\%}$) and $S_{FFRd,Battery}$ (−0.059 $\frac{\pounds /MW/h}{\%}$), which supports the hypothesis of this study. A positive relationship has been found between ${GP}_{GB}$ and $P_{FFRd}$, which is according to expectations. For the gas price in GB, the unit £/therm is used, a conventional unit for natural gas companies [22]. Therefore, many data providers also use this unit. A therm is approximately equal to 29.3 kWh.

The plots in Figure 3 show that the price is predicted more accurately at relatively high values than at low values of the independent variables ($S_{FFRd,Battery},S_{FFRd,Load Response}$ and ${GP}_{GB}$). Especially high values for ${GP}_{GB}$ show a clear effect on $P_{FFR}$. It also shows that the accuracy of the model increases over time and that it follows the up-moving trend of $P_{FFRd}$. This can be explained by the effect of ${GP}_{GB}$ on $P_{FFRd}$. The peak of $P_{FFRd}$ in January 2020 is clearly not predicted well. This peak may have been caused by an external factor that is not included in this analysis.

The estimated regression model for the FFRd market is presented in Equation (4).

$$P_{FFRd}=7.66-0.059{ S}_{FFRd,Battery}-0.12{ S}_{FFRd,Load response}+0.055{GP}_{GB}$$

(4)

3.1.2. Average Bid Price Analysis: FFR Market

Figure 4 and Figure 5 show that Load Response bids are lower in the static FFRs market than in the dynamic FFRd market. The figures also show that Load Response providers bid, on average, a lower price than the market average in both the dynamic (GBP 7.10 vs. GBP 8.24; i.e., 14% lower) and the static (GBP 2.50 vs. GBP 8.42; i.e., 70% lower) market. This supports the hypothesis of this research.

3.2. Results Case Study: The Netherlands

Market data were collected for five Dutch balancing sub-markets: aFRR Symmetrical, aFRR Upward, aFRR Downward, mFRR Downward, and mFRRda Upward capacity. The average shares of the different flexibility sources in the total accepted capacity per balancing sub-market are illustrated in Figure 6. It shows that Large-scale production takes most of the share, followed by Mixed Portfolio. DR is attributed to a small share in the total accepted capacity on the markets, especially in the aFRR balancing market, for which $S_{DR}$ is 0. Therefore, the aFRR markets are excluded from the analysis, as DR takes an insignificant share of the total accepted capacity. Figure 7 shows a scatterplot of the DR share and the flexible capacity price on the Dutch balancing sub-market for mFRR provision per tender period in scope. It shows a negative trend for the mFRRda Upward provision, while no visible trend can be recognised for the mFRRda Downward provision.

3.2.1. Regression Analysis: mFRRda Upward and Downward Market

Table 6. Results statistical tests on regression model mFRRda Upward market.

Statistical Test	Test	Result	Interpretation Result
Multicollinearity	Correlation matrix	$\mathrm{Strong} \mathrm{correlation} \mathrm{between} {GP}_{NL}$ and $P_{mFRRda Up}$ of 0.83	Debatable
	VIF test	1.23	Acceptable
Sample size/number of predictors ratio		137.75	Acceptable
Normality of residuals	Shapiro–Wilk test	p-value of 0.00	Not acceptable
Goodness-of-fit	R2	0.70	Acceptable
	F-test	0.00	Acceptable

and

Table 7. Results statistical tests on regression model mFRRda Downward market.

Statistical Test	Test	Result	Interpretation Result
Multicollinearity	Correlation matrix	$\mathrm{Highest} \mathrm{correlation} \mathrm{between} {GP}_{NL}$ and $P_{mFRRda Down}$ of 0.63	Acceptable
	VIF test	1.00	Acceptable
Sample size/number of predictors ratio		137.75	Acceptable
Normality of residuals	Shapiro–Wilk test	p-value of 0.00	Not acceptable
Goodness-of-fit	R2	0.40	Acceptable
	F-test	0.00	Acceptable

show the results of the statistical tests, which are used to test the validity of the results of the regression models for the mFRRda Upward and Downward markets. A strong correlation between ${GP}_{NL}$ and $P$ was found, which is in line with expectations, as high gas prices result in high marginal costs for gas turbines that provide flexible capacity. The VIF analysis shows that this high correlation does not cause multicollinearity. Multicollinearity could cause problems for the estimation of the coefficients in the regression model. On the contrary, the normality of residuals assumption is violated by the regression model. This suggests that the coefficients of the model cannot be estimated accurately.

To understand why the residuals are not normally distributed, the relation between the residuals and the independent variables is illustrated in Figure 8 and Figure 9 for the mFRRda Upward and Downward market. Two things are remarkable, which can explain the violation of the normality of the residual assumption. For both markets, there are many different residuals for 0% $S$. This is due to the fact that in many auctions, DR did not participate, and different prices were settled. When the DR participants join the auction, they often win and provide a significant share of the auction. Next to that, residuals are higher for moments when ${GP}_{NL}$ was around EUR 100/MWh. This can be explained by the market instability caused by relatively high gas prices.

The violation of the normality of residuals assumption makes the estimation of the coefficient less accurate. However, the 95% confidence range of the coefficients is negative for both markets, and the independent variables show a significant effect on the dependent variable, and the violation of the normality of residuals assumption is explainable. Therefore, the results of the mFRRda Upward and Downward regression model are taken into consideration and used to support the conclusion. The inaccuracy caused by the violation of the assumption can be compensated by combining the results of multiple markets.

The results of the regression model of the mFRRda Upward market can be found in

Table 8. Regression model mFRRda Upward market.

Dependent Variable	Symbol	Unit
Price of flexible capacity	$P_{mFRRda Up}$	€/MW/h
Independent Variables	Symbol	Unit	Coefficient	t Stat	Standard Error	p-Value
Intercept	$a_{mFRRda Up}$		4.35085	11.7286	0.37	0.000
DR	$S_{mFRRda Up,DR}$	%	−0.13	−5.1967	0.026	0.000
Gas price	${GP}_{NL}$	€/MWh	0.1474	28.6928	0.0051	0.000
Regression Statistics
R2	0.70
Adjusted R2	0.70
Mean absolute percentage error (MAPE)	30%
Standard error	3.35
F-test (p value)	0.00
Observations	551

. The independent variable that has been shown to be statistically significant is $S_{mFRRda Up,DR}$. The external independent variable that has shown to be significant is the NL gas price ${GP}_{NL}$.

Figure 10 and the regression statistics show that the regression model predicts the price of flexible capacity $P_{mFRRda Up}$ well (R² = 0.70 and MAPE = 30%). It shows a negative relationship between $P_{mFRRda Up}$ and $S_{mFRRda Up,DR}$ (−0.13 $\frac{€/MW/h}{{\%}_S}$), which supports the hypothesis of this study. However, the relationship looks non-linear. This can make the estimation of the effect less accurate. A positive relationship has been found between ${GP}_{NL}$ and $P_{mFRRda Up}$, as expected. Especially high values for ${GP}_{NL}$ show a clear effect on $P_{mFRRda Up}$. This relation does look to be linear.

The estimated regression model of the mFRRda Upward capacity market is presented in Equation (5).

$$P_{mFRRda Up}=2.35-0.13{ S}_{mFRRda Up,DR}+0.15{GP}_{NL}$$

(5)

The results of the regression model of the mFRRda Downwards market can be found in

Table 9. Regression model mFRRda Downward capacity market.

Dependent Variable	Symbol	Unit
Price of flexible capacity	$P_{mFRRda Downward}$	€/MW/h
Independent Variables	Symbol	Unit	Coefficient	t Stat	Standard Error	p-Value
Intercept	$a_{mFRRda Up}$		1.4750	2.7922	0.5282	0.005
DR	$S_{mFRRda Down,DR}$	%	−0.37	−2.323	0.16	0.021
Gas price	${GP}_{NL}$	€/MWh	0.1646	19.096	0.00862	0.000
Regression Statistics
R2	0.40
Adjusted R2	0.40
Mean absolute percentage error (MAPE)	54%
Standard error	6.36
F-test (p value)	0.00
Observations	551

. The independent variable that has been shown to be statistically significant is $S_{mFRRda Down,DR}$. The external independent variable that has shown to be significant is the NL gas price ${GP}_{NL}$ (in €/MWh).

Figure 11 and the regression statistics in Table 6 show that the regression model prediction accuracy is acceptable (R² = 0.40 and MAPE = 54%). It shows a negative relationship between $P_{mFRRda Down}$ and $S_{mFRRda Down,DR}$ (−0.37 $\frac{€/MW/h}{{\%}_S}$), which supports the hypothesis of this study. However, the relationship looks non-linear. A positive relationship has been found between ${GP}_{NL}$ and $P_{mFRRda Down}$, as expected. This relation does look to be linear.

The estimated regression model of the mFRRda Downwards market is presented in Equation (6).

$$P_{mFRRda Down}=2.48-0.37{ S}_{mFRRda Down,DR}+0.16{GP}_{NL}$$

(6)

3.2.2. Average Bid Price Analysis: mFRRda Markets

Figure 12 shows that DR bids are lower than the market average on the mFRRda Downwards market (EUR 4.92/MW/h vs. EUR 7.06/MW/h; i.e., 30% lower) and mFRRda Upwards market (EUR 6.71/MW/h vs. EUR 8.98/MW/h; i.e., 25% lower). Both observations support the hypothesis of this study.

3.3. Cross-Market Comparison and Analysis

In this section, the results of the different analyses and markets are consolidated to conduct a cross-market comparison and analysis between the two case studies of GB and NL.

3.3.1. Regression Analysis

The results of the regression models provide an estimation of the change in price caused by a 1% increase in the share of DR ($\frac{\Delta P}{{\Delta S}_{DR}}$ in $\frac{€/MW/h}{\%}$ or $\frac{\pounds /MW/h}{{\%}_S}$). To fairly compare markets with different price levels and currencies, the relative cost reduction (RCR) is calculated by dividing this number by the average price per market, $\overline{P_m}$. This results in a measure for the percentage change in $P$ caused by a percentage change in $S_{DR}$ ($\frac{\Delta P}{{\Delta S}_{DR}}$ in $\frac{{\%}_P}{{\%}_S}$).

Figure 13 shows the relative cost reduction of DR across the different markets for which a regression analysis (RA) was possible (${RCR}_{RA,DR,m}$). It shows that the relative cost reductions of the FFRd and the mFRRda Upward markets are −1.41% and −1.49%, respectively. The relative cost reduction of the mFRRda Downward capacity market is significantly higher at −5.24%. The average of the different estimations is −2.71%.

Figure 14 shows the estimated price reduction of DR in the period in scope per balancing sub-market. This is calculated by multiplying the relative cost reduction (RCR) of the regression analysis (${RCR}_{RA,DR,m}$) with the average share of DR per market (${\overline{S}}_{DR,m}$). This result can be interpreted as the cost reduction (${CR}_{DR,m}$) for TSOs due to the participation of DR in balancing markets in the period in scope. This cost reduction ranges from −11% to −18%.

3.3.2. Average Bid Price Analysis

The measure of the average bid price analysis should be interpreted differently from the regression analysis. The relative cost reduction of the average bid price (ABA) analysis (${RCR}_{ABA,DR,m}$) shows the percentage difference between the average bid price of DR and the overall average bid price on the market $\Delta P$ (in $\%$).

Figure 15 shows the relative cost reduction for which the average bid price analysis was possible. It shows that the different markets show significantly different results, ranging from −14% for the FFRd market to −70% for the FFRs market. The average of the results is −35%.

Both the results of the regression analysis and the average bid price analysis support the hypothesis that the participation of DR lowers the price for flexible capacity. However, the quantifications show that there is a significant difference in the extent to which DR lowers this price.

4. Discussion

This chapter reflects on the methodology, the assumptions, and the results of this research. By examining the strengths and weaknesses of the study, it is determined how the results can be interpreted to draw conclusions.

4.1. Internal Validity

4.1.1. Regression and Average Bid Price Analysis

Different types of analyses were used to test the hypothesis of this study. Multiple balancing products are included in the scope of the two case studies in GB and NL. The results of the different analyses are discussed per balancing product, after which the different outcomes are compared.

Regarding the GB’s FFRd data, a clear negative relationship is visible between the participation of DR ($S_{Load response,FFRd}$) and the price of flexible capacity on the FFRd market ($P_{FFRd}$). Both the results of the regression and the average bid price analysis support the hypothesis that higher participation of DR leads to lower balancing costs for grid operators.

However, for GB’s FFRs market, a contradiction between the results of the different types of analyses is observed. Remarkably, DR bids are significantly lower in the FFRs market than the market average (−70%), while the regression analysis cannot support the hypothesis as it cannot identify a significant effect of $S_{Load response,FFRs}$ on $P_{FFRs}$. Next to that, a positive correlation (0.28) was found between $S_{Load response,FFRs}$ and $P_{FFRs}$.

When assuming the validity of the hypothesis of this study, i.e., a negative relationship between $S_{Load response,FFRs}$ and $P_{FFRs}$, the positive correlation can be explained by the fact that this type of analysis does not consider other factors. It could be the case that in moments of high $S_{Load response,FFRs}$, an external factor (e.g., the gas price) accidentally was also high, which increased $P_{FFRs}$. This leads to a positive correlation. It could have been tested with a regression analysis to determine whether the external factor caused this correlation. However, this was impossible due to the limited number of observations as input to the regression model. This limitation can also explain why no significant effect was estimated of $S_{Load response,FFRs}$ on $P_{FFRs}$.

When not assuming the validity of the hypothesis, i.e., there is an insignificant or a positive relationship between the two variables of interest, the contradiction in the FFRs market can be explained by the fact that the result of the average bid price analysis is caused by the extreme outlier in the average bid price of the flexibility source DSF: Distributed generation (for export). The average bid of this flexibility source of GBP 47.58/MW/h was significantly higher compared to the market average of GBP 8.42/MW/h. When excluding this flexibility source, Load Response bids the third-highest average bid price on this market of the seven flexibility sources participating in this market.

For both the Dutch mFRRda Upward and Downward capacity markets, a negative trend between the variables of interest is observed. This trend is supported by both the regression and the average bid price analysis, which support the hypothesis of this research. However, two main weaknesses exist in the regression analysis of the mFRRda markets. First, the regression analysis of the mFRRda Downward market is based on low participation of DR resources of 2.3%. Therefore, a limited number of DR participants are represented by these results. Second, the regression models of both the mFRRda Upward and Downward capacity market violate the normality of the residual assumption. This leads to less accurate estimations of the coefficients ${RCR}_{RA,DR,mFRRda Up,ref}$ and ${RCR}_{RA,DR,mFRRda Down,ref}$. This inaccuracy can (partly) be corrected by taking the average over the estimations of multiple regression models.

4.1.2. Cross-Market Comparison

The cross-comparison for the two case studies shows that the highest price reduction due to the participation of DR is estimated in the mFRRda Downward capacity market by both the regression analysis and the average bid price analysis.

The results of the FFRd market show that the lowest price reduction is calculated using both the regression and average bid price analysis in this market. Remarkably, the regression analysis shows a similar result for the FFRd market to the mFRRda Upward capacity market (−1.41 $\frac{{\%}_P}{{\%}_S}$ vs. −1.49 $\frac{{\%}_P}{{\%}_S}$, respectively), while the results of the average bid price analysis are significantly different (−14% vs. −25%, respectively). This can be explained by the difference in the approach of the two types of analyses in estimating the price reduction. The average bid price analysis does not consider the share of the flexible capacity provided by DR. The regression analysis does consider this (indirectly). One thing that is clearly noticeable in the FFR and mFRRda markets is the extreme increase in the price of flexible capacity at the end of 2021, which is probably related to the relatively high gas prices. High volatility of the dependent variable in a regression model leads to less accurate estimations of the regression model [21].

4.2. External Validity

Regarding the country selection bias, similar studies are only possible for countries and markets that allow DR participation for the provision of balancing reserves and for which the appropriate data is available. This creates an inherent selection bias since only countries that meet these criteria can be included in the scope of such analyses.

Regarding the indirect effects of DR, this study mainly focuses on the cost reduction that the participation of DR can realise for TSOs organizing single-buyer balancing markets, and, therefore, the cost reduction potential is from a system perspective. When a price reduction is found using the proposed research methodology, it is suggested that this will lead to lower balancing costs for TSOs. However, other potential costs will likely be increased due to higher participation of DR. When there are more small DR participants, TSOs will need to invest more in internal operations like billing, metering, verification, settlement, communication, software, and administration [16]. An energy system simulation approach would be needed to correct for the indirect effects of higher participation of DR in balancing markets. This lies outside the scope of this research.

As mentioned in the introduction, the relevant literature presents (an indication of) a cost reduction of DR. Now that the results of this research are shown, it is interesting to compare these results with the existing literature. The only source that provides sufficient detailed information is the annual Power Responsive reports of the NationalGrid, GB’s TSO [18]. First, this report confirms the calculational method to calculate the capacity and average bid prices per flexibility source on the FFR market, as it reports the same results. Second, the results of the average bid price analysis can be confirmed by the same report, as it shows that Load Response bids are structurally lower than the market average. The report does not provide relevant information to support the regression and correlation analysis.

4.3. Recommendations for Future Research and Policy

After identifying the most important strengths and weaknesses of this research, policy and future research recommendations can be provided. The recommendations for future research are mainly based on the weaknesses and the part of the research gap that has not been addressed. The policy recommendations are mainly based on the challenges the authors encountered while conducting this research.

Future research should include a broader range of balancing markets in different countries. This improves the external validity by reducing the country selection bias. When appropriate balancing market data are available in more countries, it is recommended to look at the cost reduction of DR in balancing markets with a high share of DR, as this was not the case for all the markets included in the scope of this research. Furthermore, it would be relevant to study the impact of the high gas prices on the balancing markets individually to isolate this effect from other developments that have been studied on the price of flexible capacity. This can be performed by zooming in on the energy crisis period and by conducting a regression analysis to estimate the effect of the gas price on the balancing market prices.

It is also recommended to test whether non-linear models, like exponential or logarithmic models, better explain the relationship between the participation of DR and the price of flexible capacity in balancing markets. The non-linear relationship can be estimated using non-linear regression models. This can increase the accuracy of the estimations.

Lastly, it is recommended to combine the results of a study like this with a study that includes other factors that can lead to extra (or less) costs for TSOs, such as changes to the rules and design of the balancing market auctions. This shall provide a complete overview of the cost reduction of DR from a system perspective.

One of the main challenges of this research was finding appropriate data on balancing markets that can be used for the quantitative research methods used in this study. This challenge limited the scope to only two countries. Preferably, balancing markets of more countries would have been included in the scope to improve the external validity of the study. Therefore, it is recommended that policymakers and energy authorities make balancing markets more transparent to enable future studies on the relationship between the flexibility sources and the balancing market prices. Specifically, data on the mix of flexibility sources participating in the balancing markets is largely missing and would be valuable for future research. The potential challenge of implementing improving data transparency is that it creates an extra burden for grid operators, leading to extra costs. However, with decreasing costs of information technology and clear trends on more interconnectivity between devices, these extra costs are expected to be limited.

5. Conclusions

In this section, the conclusion of this research is formulated by answering the main research question:

What is the cost reduction potential of demand response participation in balancing markets from a system perspective?

Based on existing literature, this research hypothesises that higher participation of DR can reduce costs for balancing the electricity grid. The main research aim was to test this hypothesis using a novel quantification methodology, thus contributing to the existing literature. When appropriate data are available, this novel application of the methodology can be used as a generic quantitative method to compare different balancing markets.

The research question was answered by analysing historical data on balancing markets to identify the (quantitative) relationship between the share of DR in the total accepted capacity and the price of flexible capacity in balancing markets. The markets included in the scope were the FFR markets of GB and the FRR market of NL, using historical data from 2018 up to 2021. The chosen methodology has limitations to the internal and external validity of the results, mainly due to ambiguity in the results, model assumptions, and the country selection bias.

Despite these limitations, the results show that for most balancing markets that were included in the scope, a significant price reduction is estimated due to the participation of DR, which leads to lower costs for TSOs for balancing the electricity grid. It is estimated that DR bids are, on average, 35% lower than the market average. The regression analysis estimated that a 1% higher participation of DR in balancing markets leads, on average, to a 2.7% lower price for flexible capacity. Looking at the average current participation of DR in the investigated markets, it is estimated that the price in these markets has dropped by 10–20% due to DR participation. It can, therefore, be concluded that the results of this research support the hypothesis that higher participation of DR can reduce costs for balancing the electricity grid. This study contributes to the current literature by quantifying the cost reduction of DR participation in grid balancing using historical balancing market data.