Stackelberg-Game-Based Demand Response for Voltage Regulation in Distribution Network with High Penetration of Electric Vehicles

Abstract

With the development of the economy, electricity demand continues to increase, and the time for electricity consumption is concentrated, which leads to increasing pressure on the voltage regulation of the distribution network. For example, a large number of electric vehicles charging during a low-price period may cause the problem of under-voltage of the distribution network. On the other hand, the penetration of distributed power generation of renewable energy may cause over-voltage problems in the distribution network. This study proposes a Stackelberg game model between the distribution system operator and the load aggregator. In the Stackelberg game model, the distribution system operator affects the users’ electricity consumption time by issuing subsidies to decrease the frequency of voltage violations. As the representative of users, the load aggregator helps the users schedule the demand during the subsidized period to maximize profits. Case studies are carried out on the IEEE 33-bus power distribution system. The results show that the time of the subsidy can be optimized based on the Stackelberg game model. Both the distribution system operator and the load aggregator can obtain the optimal economic profits and then comprehensively improve the operating reliability and economy of the power distribution system.

1. Introduction

1.1. Research Questions

Due to the rapid development of society and the economy, residents’ demand for electricity consumption has risen sharply, and demand response has become an important business of the smart grid. As photovoltaic power generations are connected to the distribution network and the number of electric vehicles increases, the distribution network will experience over-voltage and under-voltage problems [1,2]. Therefore, it is necessary to ensure the residents’ electricity demand and alleviate the pressure on the voltage regulation of the distribution network. The voltage control problem of the distribution network has become an important research topic [3,4,5].

There are several voltage control methods on the distribution network side. Distribution system operators use an on-load voltage regulating transformer (OLTC) and shunt capacitors (SC) to adjust the voltage of the distribution network when the load fluctuates [6,7,8]. However, OLTC and SC are low-speed operation devices that cannot cope with the power fluctuations of the photovoltaic power generations. Therefore, traditional voltage regulation methods have limitations and cannot cope with the high penetration of photovoltaic generations and the popularization of electric vehicles. Power electronic regulators such as D-STATCOM [9] are also commonly used. However, D-STATCOMs are expensive to use. According to the voltage sensitivity algorithm, the x/r parameter of the line is the common basis for voltage regulation [10]. Adjusting the active and reactive power can realize the voltage control of the network. The above methods of adjusting the voltage have a certain delay in timeliness, so a new voltage control method based on active power is required.

1.2. Literature Review

With the development of Internet of Things technology [11], demand response utilizing flexible loads can be used for voltage regulation [12]. A better control effect can be obtained by using active power control. Reference [13] have researched the application of utilizing the active power of flexible resources to help the voltage regulation of distribution network. However, most of the flexible resources belong to the users. Many papers have studied the voltage control method of flexible load [14,15], but most of the flexible resources belong to the users. Therefore, it is necessary to manage the user’s electricity consumption time through the method of load aggregator [16]. Encouraging users to offer flexible resources for voltage regulation should be further researched. These existing methods have not considered the voltage problem. Therefore, in this paper, we propose the Stackelberg game method to establish a model between distribution network operators and load aggregators. The voltage problem of the distribution network is solved by this method.

In power systems, load aggregators can help users explore demand response resources to provide auxiliary service to the regulation market [17,18]. Due to the impact of renewable energy on power dispatch, reference [19] establishes a day-to-day hierarchical optimal scheduling model with the participation of multi-load aggregators and wind power, considering the Stackelberg game relationship between the grid dispatching layer and the load aggregators. This optimal scheduling strategy can reduce the overall economic cost of the system while ensuring the income of each load aggregator. Moreover, it enhanced the peak shaving ability of the system to a certain degree. With the development of the smart grid and the energy internet, demand response has become an essential tool for load aggregators to deal with the risks of the electricity market [20,21]. With the development of electric vehicles and renewable energy, the charging behavior of electric vehicles and the penetration of renewable energy sources will put pressure on the voltage regulation of the distribution network [22]. With the application of demand response, a Stackelberg game method can be used to guide the users’ electricity consumption time through subsidies so that the voltage of the distribution network can be kept stable, and both parties can obtain economic benefits. Therefore, it is necessary for the distribution system operators and load aggregators to build a Stackelberg game model for maintaining voltage stability and obtaining the largest profit. This paper proposes a Stackelberg game between the distribution system operator and load aggregators. In the model, the distribution system operator guides load aggregators to adjust users’ power consumption time, eases the pressure on distribution network regulation, ultimately solves the voltage violation problem, and improves system stability and economic profits. Load aggregators help users to arrange the working time of flexible resources to achieve the maximum profit.

In terms of the algorithm for finding the optimal solution in the Stackelberg game, reference [23] designs a belief update model based on reinforcement learning and a decision model based on the searching strategy of an evolutionary game. For the decision-making model, reference [24] proposes a way of solving market equilibrium based on a two-layer particle swarm algorithm. The outer layer particle swarm algorithm searches for the strategy combination in the feasible domain; the inner particle swarm algorithm calculates the value of the Nash fitness function. For a multi-layer optimization problem, reference [25] uses the differential evolution algorithm to coordinate the relationship between the microgrid, distribution network, and load to obtain the minimum cost of the power system. A multi-objective optimal scheduling model is established to achieve minimum network loss and the highest voltage bias qualification rate. However, the above methods do not consider the voltage regulation pressure on the side of the distribution network. Analytical methods are complex and take a lot of time when solving non-linear problems of voltage. Therefore, under the constraint of permissible voltage range, this paper proposes a Stackelberg-game-based demand response for utilizing the flexible resources, so that load aggregators and the distribution system operator can achieve suitable economic profits at the same time. Since the electricity consumption time of the adjustable flexible loads is a discrete relationship and the voltage calculation is a non-linear calculation, this paper formulates a Stackelberg game optimization algorithm and provides a new way to solve complex problems further.

1.3. Contributions

The main contributions of this paper are summarized as follows:

• A Stackelberg game model for distribution system operators and load aggregators is established considering flexible resources such as electric vehicles and photovoltaic power generation on the distribution network. In this model, the distribution system operator provides subsidies to load aggregators to guide the electricity consumption time of the flexible resources. As the representative of all users, the load aggregators help users obtain the largest economic benefits. In this way, the users’ flexible resources can be fully utilized, and the voltage of the distribution network can be maintained at the permissible range.
• This paper considered the voltage violation problem of the Stackelberg game model. Based on the Stackelberg game model, this paper proposes a two-layer optimization algorithm. The genetic algorithm is used on the upper layer to obtain the time of subsidies. The load aggregator uses a split algorithm on the lower layer to obtain the maximum subsidy and establish a standard to ensure that the voltage is within a permissible range. The solution is a win-win situation between the distribution system operator and the load aggregator. Moreover, to prevent users from getting together to use electricity at one time, the algorithm is designed to avoid simultaneous activation. This algorithm can ensure that all users can obtain the subsidy, and through the split algorithm, the stability of the distribution network voltage is ensured.

2. Stackelberg Game Model

The Stackelberg game means that after the leader in the game makes a decision, the follower of the game makes the optimal decision that is beneficial to himself according to the leader’s strategy. Then the leader makes the optimal decision that is beneficial to himself according to the follower’s strategy. The proposed Stackelberg game model is shown in Figure 1.

As shown in Figure 1, it is a background diagram of the game mechanism between the distribution system operator and a load aggregator. It means that the distribution system operator provides subsidies to guide the users’ electricity consumption behavior. The load aggregator arranges the schedules of electricity consumption on behalf of the users. A Stackelberg game model between the distribution system operator and the load aggregator is established based on the above game mechanism. The detailed process of the Stackelberg game is explained as follows: First, the distribution system operator, as the setter of the subsidy price, guides the load aggregator to consume electricity in different periods. The number of voltage violations is crucial for the distribution system operator. It is hoped that the frequency of voltage violation is as small as possible to reduce the pressure on the voltage regulation. When the voltage of the distribution network does not exceed the limit, the distribution system operator obtains the greatest benefit. Secondly, as the users’ representative, the load aggregator arranges the working time of users’ flexible resources during the periods with the largest subsidy to reduce the electricity cost. In this way, the competitive relationship between the distribution system operator and the load aggregator is represented by the Stackelberg game model. When the game is balanced, the profits of the distribution system operator and the load aggregator are maximized.

3. Mathematical Model of the Proposed Stackelberg Game

In the proposed Stackelberg game model, the main problem solves the optimal strategy for voltage adjustment and profitability of the distribution system operator. The lower-level problem solves the arrangement of the electricity consumption of users’ flexible resources to obtain the greatest benefit.

3.1. Mathematical Model of the Main Problem of the Stackelberg Game

The Objective Function of the Distribution System Operator

In the two-layer optimization model, the main problem is to decide on the subsidy of the distribution system operator. The distribution system operator offers subsidies for different periods to guide the users’ electricity consumption. The purpose of the distribution system operator is to minimize the subsidy and the number of voltage violations. Its objective function is (1):

$$\mathrm{Min} {{\displaystyle \sum }}_{t=1}^{T=T}su{b}_{s\_i}{}^t-\sigma N_{ove}$$

(1)

$su{b}_{s\_i}{}^t$ is the subsidy obtained from the distribution system operator, and $N_{ove}$ represents the number of voltage violations of all nodes at a time. When the value of this objective function reaches the minimum value, it means that the number of voltage violations by the distribution system operator and the subsidies to users is the least, and the benefits of the distribution network side are maximized.

In the distribution network, the number of voltage violations is calculated as follows

$$N_{ove}={{\displaystyle \sum }}_{t=1}^T{{\displaystyle \sum }}_{num=1}^{N_c}{v}_{ove}^{num,t}$$

(2)

$$V_{ove}^{num,t}=\{\begin{array}{c}1,V^{num,t}>V_{max} or V^{num,t}<V_{min}\\ 0, V_{min}\le V^{num,t}\le V_{max}\end{array}$$

(3)

Among them, $V_{ove}^{num,t}$ is the flag of whether each time is over the limit, $V^{num,t}$ represents the voltage of each time, and $V_{min}$ and $V_{max}$ represent the minimum and maximum voltage of the standard voltage, respectively. $N_{ove}$ represents the number of voltage violations of all nodes in a time. The node voltage should be in the permissible range.

$$V_{min}\le V^{num,t}\le V_{max}$$

(4)

3.2. Mathematical Model of the Lower-Level Problem of the Stackelberg Game

The Objective Function of Load Aggregator

In the smart grids, the introduction of load aggregators can execute decisions on behalf of users. Load aggregators can help users arrange the working time of flexible resources to minimize electricity costs. Its objective function is:

$$\mathrm{Min} {{\displaystyle \sum }}_{t=1}^{T=T}Fee{s}^t-su{b}_{s\_i}{}^t$$

(5)

$$Fee{s}^t=Bil{l}^t\times P_{Total}^t$$

(6)

In the formula, $Fee{s}^t$ is the electricity fee. When the value of this objective function reaches the minimum, it means that the electricity cost of the load aggregator is the smallest, and the obtained subsidy is the highest. At this time, distribution system operators have minimal voltage violations. $su{b}_{s\_i}{}^t$ is the subsidy obtained from the distribution system operator, t represents the time sequence number, and T represents the total number of time intervals.

The formula of the load aggregator’s total power $P_{Total}^t$ is as follows:

$$P_{Total}^t=-P_{PV}^t+P_{BL}^t+P_{FLT,i}^t+P_{FLT,EV}^t$$

(7)

Among them, $Bil{l}^t$ is the electricity price of different times, $P_{PV}^t$ is the output of photovoltaic power generation, and $P_{BL}^t$ is the residents’ basic load. $P_{FLT,i}^t$ is the residents’ flexible load except for electric vehicles, and $P_{FLT,EV}^t$ is the residents’ flexible load of electric vehicles. Basic load represents the electrical appliances necessary for life, such as television and refrigerator. Flexible load means the controllable loads, such as ordinary flexible loads of rice cooker, washing machine, exhaust fan, air conditioner, dishwasher, etc. Electric vehicles are seen as a particular flexible load. The start time for obtaining the maximum subsidy of the flexible loads is expressed as follows.

$$L_i=\left\{x_{1\_i},x_{2\_i},\dots ,x_{n\_i}\right\}$$

(8)

$$x_{n\_i}=\{\begin{array}{c}0\\ 1\end{array}$$

(9)

$\mathrm{where}, L_i$ is the set of whether it is the maximum subsidy in a period during the start time range of the i-th flexible loads. $x_{n\_i}$ represents the status of maximum subsidy of the corresponding time period. If it obtains the maximum subsidy, $x_{n\_i}$ equals 1, and if it is not, $x_{n\_i}$ equals 0. n represents the total number of the maximum subsidy periods. When $x_{n\_i}$ is 1, part of users start to use flexible loads. Calculation of the specific number of users is as follows:

$$P_{FLT,i}^t=P_{FL,i}^t\times N_{users}/{\displaystyle \sum }{L}_i$$

(10)

$$t_{s\_i}\in \left[\left(t_{s1\_i}, t_{s1\_i}+T_{work}\right),\left(t_{s2\_i},t_{s2\_i}+T_{work}\right),\dots , \left(t_{sL\_i},t_{sL\_i}+T_{work}\right)\right]$$

(11)

$\mathrm{where},$ flexible load $P_{FL,i}^t$ represents the power of one of the flexible loads except electric vehicles, I represents the i-th flexible load. $P_{FL,i}^t$ is specifically expressed as $P_{FL,RC}^t$, $P_{FL,WM}^t$, $P_{FL,EF}^t$, $P_{FL,AC}^t$, $P_{FL,DW}^t$, $P_{FL,EV}^t$ represents the power of the rice cooker, washing machine, exhaust fan, air conditioner, dishwasher, and electric vehicle. $t_{start}^i$ is the start time of the flexible load. $N_{users}$ is the number of all users of the distribution network. $P_{FLT,i}^t$ represents the average power of flexible loads in each period of time, $t_{s\_i}$ represents the time during which the flexible load starts working, and $T_{work}$ represents the working time interval.

The charging period of electric vehicles can be set with charging technology development. After setting the charging time, the state of charge (SOC) limit during charging and discharging needs to be considered, which is shown in (12), and the dynamic formula of SOC is shown in (13):

$$SO{C}_{min}\le SOC\left(t\right)\le SO{C}_{max}$$

(12)

$$SOC\left(t+1\right)=SOC\left(t\right)+\frac{P_{FL,EV}^t}{En{e}_{EV}}\Delta t$$

(13)

Among them, SOC(t) is the value of SOC in the t period, $P_{EV}$ is the charging and discharging power of the electric vehicle, and $En{e}_{EV}$ is the total capacity of the battery. In order to keep the battery of electric vehicles from being damaged, upper and lower limits are set for the charging and discharging of electric vehicles. $SO{C}_{min}$ and $SO{C}_{max}$ respectively represent the upper and lower limits of the charging state of the electric vehicle.

Finally, the subsidy is calculated by:

$$su{b}_{s\_i}={\displaystyle \sum }{C}_{sub}^t{P}_{FL,i}^t+{\displaystyle \sum }{C}_{sub}^t{P}_{FL,EV}^t{\theta }_i,t\in T_1$$

(14)

$${\theta }_i=\{\begin{array}{c}1\\ 0\end{array}, t\in T_{EV}$$

(15)

The method for searching for the start time of an electric vehicle is different from the above. It can be started at any time within the rechargeable period without continuous charging. $P_{FL,EV }^t$ represents the charging power of the electric vehicle, and ${\theta }_i$ represents whether it is charging in a certain period.

4. The Solution Algorithm of the Proposed Stackelberg Game

In this paper, a Stackelberg game model is established. In the main problem, the distribution system operator decides subsidies period to guide the load aggregator. At the lower level, the load aggregator represents users to arrange the time of electricity consumption and obtain maximum benefits. In this Stackelberg game model, a genetic algorithm [26] and a split algorithm are used to solve the optimization problem. Figure 2 shows the flow chart of the optimization. In the main function, the genetic algorithm searches for the optimal subsidy time period, and the fitness function is calculated to evaluate the number of voltage violations. The low-level optimization calculates the users’ largest profit.

Figure 3 is the detailed solution algorithm of the Stackelberg game model. In the main function, the genetic algorithm is used to search the strategy of the time of subsidy, and the value of the fitness function evaluates the quality of the strategy. First, the parameters are set. After the initial population is randomly generated, the individuals of the corresponding population are matched with each other in each generation. In the fitness function, the voltage of the distribution network is calculated by power flow, and the violation number is determined. If the value of the fitness function is not satisfied, then the next individual will continue to match the opponent until the optimal strategy is found. The process ends when the generation number is reached.

In the low-level function, the method of preventing the simultaneous start and the mixed-integer optimization algorithm are used to solve the Stackelberg game model of the load aggregator. According to Figure 3, it can be seen that the load aggregator receives the subsidy strategy from the result of the genetic algorithm. Considering the prevention of simultaneous activation, the load aggregator arranges users’ power consumption strategies based on the calculated maximum subsidies. The mixed-integer algorithm is utilized to solve the user’s minimum cost of electricity consumption, and the result determines whether this strategy is the optimal solution for all individuals in the population. The optimal solution is then sent to the main function to calculate the value of the fitness function.

5. Case Studies

In the case studies, the IEEE 33-bus power distribution system is used for simulation [27]. In the model, it is assumed that there is a total of 100 users at each node. Electric vehicles and photovoltaic power generations are respectively loaded on the user nodes at a ratio of 30% and 70%. The Newton–Raphson method is used for power flow calculation. The parameters of population, crossover, mutation, and generation of the genetic algorithm are 20, 0.8, 0.02, and 1500, respectively. The calculation time of day-ahead planning is 4.20 h by using a personal computer with Intel(R) Core(TM) i5-10210U CPU @ 1.60 GHz 2.11 GHz.

In order to show the effectiveness of the proposed method of voltage control based on the Stackelberg game, this paper designs the following three cases for comparison.

Case 1: The distribution system operator does not interact with the load aggregator. Within a certain time range, the flexible loads are started randomly.

Case 2: The distribution system operator does not interact with the load aggregator. Within a certain range, the flexible loads are started at the moment of the lowest electricity price.

Case 3: The distribution system operator subsidizes the load aggregator, and the subsidy affects the start time of the flexible loads on the user side.

This study selects several commonly used household appliances as flexible loads. When an appliance is used multiple times a day, each use is marked with a different number [28]. The allowable working interval and working time of each electrical appliance refer to real life, shown in

Table 1. Flexible load data on the user side.

Flexible Load Ai		$[\mathit{B}{\mathit{n}}_{{\mathit{A}}_{\mathit{i}}},\mathit{E}{\mathit{d}}_{{\mathit{A}}_{\mathit{i}}}]$	TAi (min)	PAi (kW)
1	Rice cooker	6:00–8:00	60	0.5
2	Ventilator	0:00–23:59	60	0.3
3	Washing machine	0:00–23:59	60	0.4
4	Air conditioner	9:00–12:00	60	1.5
5	Rice cooker	9:00–11:00	60	0.5
6	Air conditioner	12:00–15:00	60	1.5
7	Dishwasher	20:00–6:00	60	0.6
8	Rice cooker	15:00–18:00	60	0.5
9	Electric vehicles	18:00–6:00	360	3

.

Figure 4 shows the power of photovoltaic power generation, basic load, and flexible load within 24 h. In Figure 4, FL stands for flexible load, PV stands for photovoltaic power generation, and BL stands for the basic load. The power of photovoltaic generation is relatively high during 6:00–18:00, and the highest period is 9:00–13:00. The basic load has higher power between 6:00–9:00 and 18:00–21:00. It shows that the consumption time and power of the basic load are relatively fixed. The curve of FL is the electricity consumption that the start time of the flexible loads is random. The working time of flexible resources will be influenced by subsidies. The power consumption of flexible loads during periods of high photovoltaic power generation can prevent over-voltage conditions.

Figure 5 shows the voltage of 32-node of Case 1. In Case 1, the flexible loads on the user side are randomly activated without the intervention of the distribution system operator. From Table 1, it can be seen that the charging time of electric vehicles is between 18:00 and 6:00. As shown in the figure, during 18:00–24:00, the peak power consumption is high, and under-voltage frequently occurs during this period. The result indicates that if the electric vehicles are charging without intervention will put a lot of pressure on the voltage regulation of the distribution system operator. The power distribution system operator needs to adjust the voltage to ensure the voltage quality constantly. In Figure 5, in the electricity consumption mode of the random start of flexible loads, the number of voltage violations is the most, and the number of under-voltage and over-voltage is 109 times.

Figure 6 shows the voltage of 32-node of Case 2. In Case 2, users use electricity according to the peak and valley electricity prices. In this study, the peak period is 8:00–22:00, and the other time is the valley period. In Case 2, the electric vehicles are started to charge after 22:00 when the electricity price is low, so under-voltage conditions seriously occur during this period. In Figure 6, under the influence of the peak-valley electricity price, the start-up time of flexible loads prefers the low-price time period. At this time, the number of voltage violations is 84 times in total.

Figure 7 shows the voltage of 32-node of Case 3. In Case 3, the Stackelberg game is introduced. In the upper-level optimization, a genetic algorithm is utilized to calculate the optimal time for granting subsidies. In the lower level optimization, the load aggregator determines the start time of the flexible loads on the user side after obtaining subsidy information. As shown in Figure 7, the number of voltage violations is zero in this case. After the optimization of the Stackelberg game, users can receive subsidies to save electricity costs, and the distribution system operator does not need to adjust the voltage regulation devices frequently. In this way, a win-win situation can be achieved between the distribution system operator and the user side. In Figure 7, using subsidies to guide the start-up time of flexible loads and performing day-ahead scheduling is a novel method that can allow users to obtain subsidies and keep the voltage of the distribution network stable.

When comparing with Case 1 and Case 2, Case 1 has more occurrences of random under-voltage because the charging time of the electric vehicle is randomly decided during the peak period of power consumption in the evening. In Case 2, users use electricity according to the peak-valley price, so the charging time of the electric vehicle is after 22:00 at night, avoiding the peak period of evening electricity consumption, but the number of under-voltages is relatively high. The cost of electricity in Case 1 is also higher than that in Case 2 because the usage of the flexible loads is not considering the electricity price.

In the three cases, if the voltage is exceeded the limit, the OLTC is supposed to operate the tap to regulate the voltage to the admissible range. Figure 8 shows the number of times of adjusting OLTC taps under the simulation of the three cases. In Case 1 and Case 2, there are different degrees of voltage limit violations. In Case 3, there is no voltage limit violation. As a result, the OLTC needs to adjust the tap in Cases 1 and Cases 2, and the number of OLTC operations are six and four, respectively. The tap of OLTC does not need to adjust in Case 3, indicating that the proposed strategy can regulate the voltage, and the regulation pressure of the distribution system operator is reduced.

Figure 9 shows the effect of subsidies on Case 3. Among the three cases, it can be seen that the peak power of Case 3 is lower than that of Case 1 and Case 2 at the time when the electric vehicle is charging. The trend of power consumption in Case 3 shows that when subsidies are issued, the load aggregator actively responds to schedule users’ flexible loads. The overlap rate between the flexible load usage time and the subsidy time is relatively high, indicating that it successfully guides users to use electricity through subsidies.

6. Conclusions

The electricity consumption of flexible loads such as electric vehicles would be concentrated during the low-price periods, which may cause under-voltage problems in the distribution network. The high penetration of distributed generation of renewable energy sources may cause over-voltage problems in the distribution network. In order to solve these problems, this paper establishes a Stackelberg game model between the distribution system operator and the load aggregator. The load aggregator represents users and helps the users to arrange the flexible loads to achieve maximum profit. The optimal solution of the Stackelberg game model will maximize the benefits of both the leader and the followers. Case studies show that in the Stackelberg game model, the distribution system operator can affect the users’ power consumption time by issuing subsidies, and the load aggregators can help users arrange electricity demand during the subsidy period to maximize profits. In this way, the voltage of the distribution network can avoid violating the permissible range, thereby comprehensively improving the operational reliability and economy of the distribution system. This paper provides a new method for distribution network voltage control. The result shows that there is no voltage limit violation, and the tap of OLTC does not need to adjust in Case 3. it can be seen that the peak power of Case 3 is lower than that of Case 1 and Case 2, which shows that when subsidies are issued, the load aggregator actively responds to schedule users’ flexible loads. The Stackelberg game model studied in this paper can provide a method for realizing the friendly interaction between grids and users.