Research on a Multi-Agent Cooperative Control Method of a Distributed Energy Storage System

Abstract

For the flexible regulation requirements of new power systems with a high proportion of new energy, this paper proposes a multi-point distributed energy storage system control method based on the idea of multi-agent cooperative control. On the one hand, the method transforms and upgrades the strategies of each distributed battery energy storage control system to make it a terminal agent with active response and control functions; on the other hand, a coordination agent is set between the power grid and each terminal agent, the coordinated control of the distributed energy storage system is realized by the information interaction between the coordination agent and the terminal agent. The simulation results show that the method can fully exploit the aggregate regulation ability of the distributed energy storage system, quickly stabilize the frequency fluctuation of new energy, and support the safe and economical operation of the system.

1. Introduction

The vigorous development of wind power, photovoltaic and other new energy is the main way to achieve the “double carbon” goal. However, with the gradual increase in the proportion of new energy access to the public power grid, the intermittence, randomness and volatility of new energy output will inevitably impact the power and energy balance and power quality of the power system [1,2,3,4]. Therefore, at this stage, it is urgent to realize the optimal allocation and reasonable utilization of flexible regulation resources to meet the flexible regulation requirements of the power system.

The optimized configuration and aggregate utilization of the energy storage system is an effective measure to improve the flexible adjustment of demand of the new power system under the high proportion of new energy access. The energy storage (ES) can absorb the excess energy at the peak of new energy generation and supplement the corresponding energy when the new energy generation is insufficient so as to ensure the stable operation of the system. At present, the development of energy storage presents a centralized and distributed trend [5]: large, centralized energy storage facilities are constrained by geography and resource environment, and the construction cost is high; The distributed energy storage facilities are flexible in installation and can be freely combined, but there are problems of scattered locations, different capacities and different forms. For the control of multi-point distributed energy storage, experts and scholars at home and abroad have carried out a lot of research work. In [6], it proposes a power grid frequency adjustment model that includes energy storage. The model adopts a coordinated control strategy for fast and slow frequency adjustment resources that takes into account the energy storage SOC but does not specify the control method for active participation of energy storage in frequency adjustment. In [7], it proposes an overall control strategy for energy storage to participate in primary frequency adjustment and defines evaluation indicators, such as maximum frequency deviation, frequency degradation rate, and so on. In [8], it proposes a fuzzy control-based energy storage frequency adjustment control strategy, which calibrates the system output in the photovoltaic grid-connected model to maintain stable active power output but does not specify the collaborative control mode of energy storage. In [9], aiming at the power distribution problem of energy storage participating in secondary frequency adjustment, a control strategy for maximizing the energy conversion efficiency of energy storage batteries was proposed. In [10], it comprehensively considers the state of charge of large-scale energy storage control systems and the load requirements of power systems and establishes a complete model of secondary frequency adjustment for battery energy storage systems based on state of charge fuzzy control strategy assistance. It achieves the improvement of stability indicators, such as frequency, voltage, and power of battery energy storage systems, but does not consider the application scenario of primary frequency adjustment for energy storage. In [11], in response to the randomness of photovoltaic power generation and the inability to provide inertial support for the system, it proposes a control strategy based on a virtual synchronous generator (VSG) for hybrid energy storage, which can autonomously adjust the frequency and voltage of the system. In [12,13,14,15], they focus on the configuration of energy storage capacity in primary frequency adjustment and study it from frequency deviation, SOC state, real-time energy storage superposition, droop control, high and low-frequency signals, etc. In [16], it combines particle swarm optimization and photovoltaic prediction to propose a new strategy for optimal allocation of energy storage capacity, which improves the frequency adjustment quality of the power grid. In [17], it uses the linear decreasing inertia weight particle swarm optimization algorithm to seek the best efficiency of the distributed energy storage system under the existing constraints, with the goal of minimizing the output of the distributed energy storage system. In [18], it uses model prediction algorithms to analyze the impact of centralized and decentralized energy storage grid connection and analyze the impact of different sampling intervals and the number of power stations on energy storage configuration.

The above research mainly focuses on the control strategy and optimal configuration issues in the primary or secondary frequency adjustment of energy storage, while the control logic combining active control and collaborative control for energy storage is rarely involved. To address this issue, this paper proposes a collaborative control method for distributed energy storage systems based on the idea of multi-agent collaborative control, taking multipoint distributed energy storage as the research object and supporting the frequency regulation of power systems under high permeability photovoltaic access as the goal. This method, on the one hand, makes the distributed battery energy storage control system become a terminal agent with active response and control functions through the transformation and strategy upgrading of each distributed battery energy storage control system. On the other hand, it sets a coordination agent between the power grid and each terminal agent to realize the collaborative control of the distributed energy storage system through the information interaction between the coordination agent and the terminal agent. This paper implements the above model and strategy based on the power system simulation software DIgSILENT/PowerFactory and verifies the correctness and effectiveness of the method.

2. Multi-Agent Collaborative Control Architecture for Energy Storage

The agent has a certain degree of autonomy and adaptability and is more and more widely used in the power grid with a high proportion of intermittent new energy [19]. The basic structure of the agent is shown in the Figure 1.

Multi-Agent System (MAS) contains multiple agents. The lower agents, the upper agents, and the agent clusters contain multiple information flows. Through cooperation and competition, the system as a whole achieves stability and optimization. The working efficiency of different multi-agent system structures is also different. At present, the most commonly used is the hybrid multi-agent structure, as shown in the Figure 2.

This paper uses the advantages of hybrid multi-agent architecture, combines the agent theory with distributed energy storage technology, and proposes a multi-agent collaborative control architecture for energy storage, as shown in Figure 3.

On the one hand, the architecture has transformed and upgraded the distributed battery management system (BMS) into a terminal agent with active control and response functions. Each terminal agent controls the PCS active response frequency and voltage disturbance through the preset active control strategy. If the disturbance is not eliminated by the active control, the terminal agent will feed back relevant information to the coordinating agent and request cooperative control.

On the other hand, a coordination agent is set between the power grid and each terminal agent. On the one hand, the coordination agent selects the optimal interval through the preset algorithm and distributes the power grid dispatching instructions to the energy storage terminal agents; On the other hand, it accepts the request of the terminal agent and selects the appropriate terminal agent for cooperative control.

3. Active Control Strategy of Energy Storage Agent

The energy storage device is connected to the power system through the power electronic converter and does not have damping characteristics and rotational inertia, so it lacks the inherent active frequency regulation and voltage regulation support capability of the conventional synchronous generator. In this paper, the virtual synchronous generator control technology is used here, and the damping, inertia, active frequency adjustment and reactive voltage regulation characteristics of the conventional synchronous generator are simulated through the additional improved control strategy of the energy storage inverter. On the basis of the conventional charging and discharging energy storage/release function, the energy storage agent can actively respond to the frequency and voltage disturbance adjustment function [20]. The control topology is shown in the Figure 4. Where $P_{ref},Q_{ref}$ is the power setting value; $P_e,Q_e$ is the output (instantaneous) power; $u_{abc},i_{abc}$ is the output voltage and current of VSG, and SPWM is sinusoidal pulse width modulation, which controls the on-off of switching devices in the inverter circuit.

3.1. Frequency Control Strategy

Combined with the motion equation of the synchronous generator, set virtual damping and inertia to obtain the frequency adjustment control equation of VSG, as shown in the following equation.

$$\left\{\begin{array}{l}J\frac{d\left(\omega -{\omega }_N\right)}{dt}=\frac{1}{{\omega }_N}\left(P_{ref}-P_e\right)-D\left(\omega -{\omega }_N\right)\\ \omega =\frac{d\theta }{dt}\end{array}\right.$$

(1)

where: ${\omega }_N$ is synchronous electrical angular velocity; $\omega$ is the electrical angular velocity; $\theta$ is the electrical angle; $D$ is the damping coefficient; $J$ is virtual inertia; Other parameters are the same as above; The relationship between electromagnetic torque $T_e$ and power is approximately: $T_e=P_e/{\omega }_N\approx P_e/\omega$.

Laplace transform is performed on Equation (1), and then the term is shifted to obtain:

$$\omega =\frac{\left(P_{ref}-P_e\right)}{{\omega }_N}\times \frac{1}{JS+D}+{\omega }_N$$

(2)

The frequency control block diagram is shown in Figure 5.

3.2. Voltage Control Strategy

VSG voltage control equation is:

$$\dot{E}=\left. [D_q\left(U_n-U_o\right)+\left(Q_{ref}-Q_e\right)\right]/\left(KS\right)$$

(3)

where: $U_n$ is the voltage amplitude; $U_o$ is the output voltage amplitude; $D_q$ is the reactive voltage coefficient; $K$ is the reactive voltage integral coefficient, and other parameters are as above.

The voltage control block diagram of the virtual synchronous generator, according to Equation (3), is shown in Figure 6.

The VSG calculation process is as follows: the AC side collects the electric energy signal $u_{abc},i_{abc}$, calculates the active power, reactive power and other parameters through the power calculation, and calculates the output angular velocity $\omega$ of the VSG grid-connected system according to the frequency adjustment link, integrates it to obtain the phase angle $\theta$, and the excitation potential amplitude $\dot{E}$ is obtained through the voltage regulation link. $\dot{E}$ is transformed by abc/dq coordinate system and combined with the virtual impedance link to obtain the voltage component. Then, the voltage component is decoupled through the voltage and current dual loop control loop to obtain the voltage control signal, which is used as the input signal for SPWM modulation, to obtain the control signal of the switch bridge in the converter. The voltage characteristics of the control energy storage AC side are the same as the power grid.

4. Multi-Agent Collaborative Control Strategy for Energy Storage

4.1. Energy Storage Agent Control Logic

The control strategy of the energy storage agent proposed in this paper is mainly aimed at the terminal agent in the energy storage area, combining with particle swarm optimization algorithm to form a logical agent composed of coordination agent, terminal agent and battery body, as shown in the Figure 7 (taking a single battery as an example).

Among them, information flow ① includes information collected by BMS from the battery body, such as voltage, current, state of charge, etc.;

Information flow ② includes real-time information and historical data of battery body collected by BMS, collaborative control information, etc.;

Information flow ③ includes relevant information in PCS operation control;

Information flow ④ includes PCS active control information, etc.;

Information flow ⑤ includes real-time battery information, active control deviation information, coordinated control deviation information, etc.;

Information flow ⑥ includes energy storage output power demand information, collaborative control request information, etc.;

Information flow ⑦ includes power demand information of power grid dispatching center, etc.;

4.2. Multi-Energy Storage Agent Cooperative Control Algorithm

The preset collaborative control algorithm of the coordination agent is shown in the Figure 8.

The algorithm is as follows:

(1)

Set the total control target value of the regional energy storage coordination agent according to the power grid dispatching requirements $P_0\left(t\right)$.

(2)

Calculate the initial power distribution value of each energy storage terminal agent:

When each energy storage terminal agent calculates the initial value of power distribution, it needs to know the information, such as the state of charge of the battery pack at the last time of each terminal agent.

When the power demand is positive, the initial value $P_i\left(t\right)$ of discharge power distribution calculated by the ith terminal agent is:

$$P_i\left(t\right)=P_0\left(t\right)\left. [\frac{S_i\left(t-1\right)}{{\displaystyle \sum _{i=1}^n{S}_i^{}\left(t-1\right)}}\right]$$

(4)

where: $i=1,2,3\dots n$, n is the number of terminal agents; $S_i^{}\left(t-1\right)$ is the state of charge of the battery pack of the ith energy storage terminal at the last moment.

(3)

Calculate the power reference value of each energy storage terminal agent:

Combined with the maximum charging and discharging power allowed by the energy storage PCS, the power reference value of the energy storage terminal agent is:

$$P_i{}^r\left(t\right)=P_0\left(t\right)\left. [\alpha \frac{S_i\left(t-1\right)}{{\displaystyle \sum _{i=1}^n{S}_i\left(t-1\right)}}+\beta \frac{P_i{}^{\max }}{{\displaystyle \sum _{i=1}^n{P}_i^{\max }}}\right]$$

(5)

where: $P_i^{\max }$ is the maximum allowable charging and discharging power of the ith energy storage terminal intelligent agent PCS; $\alpha ,\beta$ is the power adjustment factor.

Compare the initial power distribution value of the energy storage terminal agent with the maximum allowable charge and discharge power. If the initial power distribution value is greater than the maximum allowable charge and discharge power, then $\alpha =0,\beta =1$; If the initial value of power distribution is less than the maximum allowable charge and discharge power, then $\alpha =0,\beta =1$:

$$P_i^r\left(t\right)=\left\{\begin{array}{l}{P}_0\left(t\right)\frac{S\left(t-1\right)}{{\displaystyle \sum _{i=1}^{n}S\left(t-1\right)}}{P}_i\left(t\right)<P_i^{\max }\\ P_0\left(t\right)\frac{P_i^{\max }}{{\displaystyle \sum _{i=1}^n{P}_i^{\max }}}{P}_i\left(t\right)>P_i^{\max }\end{array}\right.$$

(6)

(4)

Determine the output power target value of each energy storage terminal agent:

From the above equation, the output power target value of the ith energy storage terminal agent is $P_i^r\left(t\right)$, and there are two possibilities:

If the initial value of power distribution is greater than the maximum allowable charge and discharge power, the intellectual output of the energy storage terminal has reached the limit value but has not reached the initial value of power distribution, and there is a deviation. The calculation equation of the total power deviation $P_{\mathrm{deviation}}$ is:

$$P_{\mathrm{deviation}}=P_0\left(t\right)-{\displaystyle \sum _{i=1}^n{P}_i{}^r}\left(t\right)$$

(7)

If the initial value of power distribution is less than the maximum allowable charge and discharge power, the output target value of the energy storage terminal agent reaches the initial value of coordination agent distribution, and there is still margin for additional adjustment. The calculation formula of the power margin $P_{margin}$ of the ith energy storage terminal agent is:

$$P_{margin}=P_i^{\max }-P_i\left(t\right)$$

(8)

The energy storage terminal intelligent agent with power margin and the state of charge not reaching the limit will increase the additional power redistributed on the original basis. The total control target value of the energy storage coordination intelligent agent will be set as the total power deviation, and the values will be updated according to Equations (4) to (8).

The objective function set by the algorithm is shown in Equation (9), and Equation (10) and Equation (11) are the calculation conditions related to the objective function.

$$G=\min \left({\omega }_1{F}_1+{\omega }_2{F}_2\right)$$

(9)

$$F_1={\displaystyle \sum _{i=1}^n\left|P_i{}^{\ast }\left(t\right)-P_i^r\left(t\right)\right|}$$

(10)

$$F_2={\displaystyle \sum _{i=1}^n\left|S_i\left(t-1\right)+\frac{P_i\ast \Delta t}{C_i}-S_r\right|}$$

(11)

where: $F_1$ is the calculated power difference between the total charging and discharging power of the energy storage station and the total regulating command power; $F_2$ is the difference between the regulated state of charge of the energy storage station and the preset state of charge; $P_i{}^{\ast }\left(t\right)$ is the charging and discharging power for the ith energy storage terminal agent; $S_r$ is the preset state of charge of the battery pack; $\Delta t$ is the adjustment time; $C_i$ is the energy storage capacity of the ith energy storage terminal agent; ${\omega }_1,{\omega }_2$ is the weight coefficient, which is used to measure whether the energy storage station is inclined to adjust the state of charge or to track and dispatch. The position of particles in the algorithm is expressed by the charging and discharging power of each energy storage terminal agent, and the speed of particles is expressed by the corresponding state of charge of each energy storage terminal.

The above objective function consists of two parts: one is to minimize the deviation between the total charging and discharging power of the energy storage system and the target value of the total regulation command (where the power deviation is expressed as the per-unit value); The other part is to minimize the deviation between the adjusted state of charge of each distributed energy storage system and the preset ideal state of charge. After continuous iterative optimization using particle swarm optimization, the total output of each energy storage agent approaches the target value of the total regulation command, and the adjusted state of charge of each distributed energy storage agent has the smallest deviation from the preset ideal state of charge.

4.3. Influencing Factors of Collaborative Control

In the actual operation of power grids, the condition of network communication is an important factor that affects the effectiveness of multi-agent collaborative control. At the same time, in the new smart grid, the communication carrier for multipoint distributed energy storage is mainly optical fiber, with high communication reliability. Therefore, the default communication delay between agents in this article is within the allowable error range, regardless of the impact of communication exceptions on the control effect of multiple agents, to ensure the timeliness and effectiveness of information transmission. The entire station communication adopts a dual network redundancy communication arrangement. The station control level communicates with the upper dispatching center data network through gateway, switches, and longitudinal encryption. The station control level is connected to the bay level, the bay level to the process level, and each agent through optical fiber. The IEC 61850 communication protocol is adopted. The specific communication method and architecture are shown in the Figure 9.

On the other hand, the multi-energy storage agent collaborative control algorithm proposed in this paper allocates the regulation demand based on the state of charge and the maximum allowable charge and discharge power of each energy storage unit, which can enable each energy storage unit to fully exert its own regulation ability, achieve a balanced output, effectively guarantee the service life of the equipment, and promote the smooth operation of the actual power grid.

5. Analysis of Numerical Examples

Based on the aforementioned cooperative control strategy and algorithm of multiple energy storage agents, the power system simulation model, including multi-point distributed energy storage, is established using DIgSILENT simulation software. The system includes photovoltaic, energy storage, synchronous motor and load. Nodes 2 and 3 are, respectively, connected with photovoltaic and energy storage devices, node 1 is connected with synchronous motor and energy storage devices, and nodes 5, 6 and 8 are connected with the load. The example simulation system is shown in Figure 10, the VSG block diagram of the energy storage agent in DIgSILENT is shown in Figure 11 (the control quantities in the figure have been described in Section 3.1) and node parameter settings are shown in

Table 1. Node parameter settings.

Node ID	Node Type	Operating Parameters
1	PQ	Active power of synchronous motor: 0.8 MW Reactive power of synchronous motor: 0.3 Mvar Energy storage active power: −0.2 MW Energy storage reactive power: 0 Mvar
2	PQ	PV active power: 0.8 MW PV reactive power: 0.1 Mvar Energy storage active power: 0.08 MW Energy storage reactive power: 0.01 Mvar
3	PQ	PV active power: 0.8 MW PV reactive power: 0.1 Mvar Energy storage active power: 0.08 MW Energy storage reactive power: 0.01 Mvar
5	PQ	Load active power: 1.5 MW Load reactive power: 0.5 Mvar
6	PQ	Load active power: 1.5 MW Load reactive power: 0.5 Mvar
8	PQ	Load active power: 1.5 MW Load reactive power: 0.5 Mvar

.

When the optical storage unit does not apply the virtual synchronization control strategy, the circuit breaker at node 8 trips due to fault, and the circuit breaker has no subsequent action in 1 s, and the system frequency is set to the standard unit value based on 50 Hz. At this time, the dynamic response characteristics of the system frequency and voltage are shown in Figure 12 and Figure 13.

As can be seen from Figure 12, when the circuit breaker at node 8 trips due to fault and the circuit breaker has no subsequent action in 1 s, the system frequency continues to rise in 10 s and does not return to a stable state.

As can be seen from Figure 13, the voltage at Bus 4 increases rapidly after 1 s and tends to stabilize after about 10 s, with a stable value of 0.32 p.u. higher than the initial value.

When the optical storage unit applies the virtual synchronization control strategy, the circuit breaker at eight nodes trips due to a fault and the circuit breaker does not act subsequently, and the system frequency is set to the unit value based on 50 Hz. At this time, the dynamic response characteristics of the system frequency and voltage are shown in Figure 14 and Figure 15.

It can be seen from Figure 14 that the frequency of the system oscillates after 1 s and then tends to stabilize after about 8 s, and the stable value is 0.0001 Hz lower than the initial value. When the virtual synchronization control strategy is applied, the system frequency can oscillate and stabilize after the failure, and the frequency fluctuation range is small.

It can be seen from Figure 15 that the voltage at PV 1 increases rapidly to 1.0057 p.u. after 1 s and then recovers to stability after about 8 s. The stabilized voltage is 0.0018 p.u. higher than the original voltage. The voltage at Bus 4 rose rapidly to 1.0017 p.u. after 1 s and then stabilized after about 8 s. The stabilized voltage was 0.002 p.u. higher than the original voltage. When the virtual synchronization control strategy is applied, the voltage fluctuation range of the system is smaller than that without the virtual synchronization control strategy, and it can recover stability in a certain time.

As can be seen from Figure 16, the power of ES1, 2, and 3 rapidly increases by about 0.006 MW at 1 s and recovers to the initial value after about 3 s. When a disturbance occurs in the system, each energy storage terminal agent actively adjusts the power output by the energy storage agent to provide fast frequency support.

At this time, multiple energy storage agents cooperate to control the function action and start to conduct secondary adjustments to make up for the adjustment deviation of primary active control. The weight coefficient ${\omega }_1,{\omega }_2$ in the cooperative control algorithm of multiple energy storage agents ranges from 0 to 1, the maximum charging and discharging power of energy storage are limited to 0.2 MW, and the upper and lower limits of the SOC of energy storage are 0.1 to 0.8. After many iterations, the optimal value of 0.006036 is obtained and the iterative convergence process of the fitness function is shown in Figure 17

With reference to the relevant requirements in the fast frequency response standard for new energy stations in Northwest China Power Grid, the frequency response time of secondary regulation is set in 2 s, and the frequency deviation dead band is 50 ± 0.01 Hz. After the first active control is stable for 6 s, the calculated power regulation command value is input into each energy storage unit in the form of a parameter event. The power reference value Pref of energy storage, the output power P (taking energy storage three as an example), and the frequency changes of key nodes of the system are shown in the following figures, where the system frequency is set to the standard unit value based on 50 Hz.

In Figure 18, after setting the parameter event, the power reference value of energy storage three suddenly changes to 0.75 MW at 6 s, and the output power oscillates and increases, becoming equal to the reference value at 6.08 s. From the adjustment process of energy storage 3, it can be seen that each energy storage operates according to the power adjustment amount calculated by the collaborative control algorithm, achieving a support effect on the system frequency.

In Figure 19, the frequency of the key nodes of the system is shown in the form of a unit value. The frequency oscillates after 6 s and recovers to stability at 6.09 s. The stabilized frequency is 1.005 p.u. The response time and frequency deviation is in line with the standards set in the previous article, with a good adjustment effect.

It can be seen from the following three situations: the optical storage unit does not apply the virtual synchronization control strategy, the virtual synchronization control strategy, the virtual synchronization control strategy and the cooperative control strategy: when the virtual synchronization control strategy is not applied, when the open circuit fault occurs, and the fault is not cleared, the system frequency does not return to stability, and the system voltage fluctuates greatly. When the virtual synchronization control strategy is applied, the system voltage fluctuation range is small, the system frequency response time is shortened, and the stabilized frequency deviation is smaller, but it still does not meet the standards set in this paper. When the virtual synchronization control strategy and cooperative control strategy are applied, the system frequency response time is greatly shortened, and the stabilized frequency deviation meets the standards set in this paper.

6. Conclusions

In this paper, a multi-agent cooperative control strategy for distributed energy storage systems is proposed considering that the energy storage system can suppress the fluctuation of renewable energy. On the one hand, the strategy is based on the idea of the virtual synchronous generator to propose an active control strategy to actively respond to the disturbance of the system and realize the active control; On the other hand, a cooperative control strategy is proposed based on the idea of multi-agent cooperative control. The multi-agent particle swarm optimization algorithm is used to establish the objective function of the control strategy, find the optimal solution, and send the relevant power regulation instructions to the terminal energy storage agent to achieve cooperative control. The simulation results show that the proposed distributed energy storage control strategy can effectively reduce the frequency fluctuations of the new power system with renewable energy as the main body and ensure the smooth operation of the system. However, this paper does not give a control strategy considering the access location of distributed energy storage, which needs further research.