Profit Maximizing Control of a Microgrid with Renewable Generation and BESS Based on a Battery Cycle Life Model and Energy Price Forecasting

Abstract

In this paper, an optimal control strategy is presented for grid-connected microgrids with renewable generation and battery energy storage systems (BESSs). In order to optimize the energy cost, the proposed approach utilizes predicted data on renewable power, electricity price, and load demand within a future period, and determines the appropriate actions of BESSs to control the actual power dispatched to the utility grid. We formulate the optimization problem as a Markov decision process and solve it with a dynamic programming algorithm under the receding horizon approach. The main contribution in this paper is a novel cost model of batteries derived from their life cycle model, which correlates the charge/discharge actions of batteries with the cost of battery life loss. Most cost models of batteries are constructed based on identifying charge–discharge cycles of batteries on different operating conditions, and the cycle counting methods used are analytical, so cannot be expressed mathematically and used in an optimization problem. As a result, the cost model proposed in this paper is a recursive and additive function over control steps that will be compatible with dynamic programming and can be included in the objective function. We test the proposed approach with actual data from a wind farm and an energy market operator.

1. Introduction

1.1. Motivations

Due to the growing concern about sustainability and demand on energy, renewable generations are receiving more interests from governments, researchers and investors, which leads to an increase in the number of renewable power systems integrated into the current electrical grids. The penetration of renewable energy, however, is mostly hindered by their variability and intermittency, which motivates the development of microgrids supplied by renewable power systems [1]. Either grid-connected or islanded, these decentralized power systems are believed to be the promising solution to achieve higher penetration of clean energy in the future [2]. Given proper control of storage units and communications with the electricity market, the non-dispatchable renewable power can be smoothed and used on demand, therefore reducing the difficulty of power scheduling in the main grid operation [3]. In many studies that focus on the control of renewable power systems [4,5,6,7,8,9,10,11,12,13,14,15], the battery energy storage system (BESS) is essential for controlling the actual power dispatched to the local customers and the grid. Utilizing forecasting data on renewable power and power demand to arrange BESS actions over different periods, the power constraints in the microgrid and the operating parameters in the main grid can be satisfied. Also, in a deregulated energy market with variable electricity price, the profit of power trading with the utility grid can be maximized with appropriate charge/discharge decisions over different price intervals. For instance, part of the renewable power can be used to meet the local demand, and the remaining can either be charged to the BESS or sold to the market. In addition, operators of the microgrid can purchase some energy from the main grid at a low price and stored for further use during high price intervals.

A significant challenge in these studies is the existence of forecasting errors. To optimize the operation of microgrids, predicted data on renewable power, load demand, and electricity price within a future period ranging from hours to days would be required, and the forecasting errors cannot be avoided. A relatively small level of errors would be acceptable, which can be addressed with some online ancillary services such as fast-response generators or operating reserves, while a large error can be detrimental to both microgrid and the main power grid. Another issue is related to battery cost. In these studies, batteries used are likely to experience more charge and discharge cycles than regular tasks, which can accelerate the degradation of batteries, leading to an increased operating cost of batteries. Therefore, the additional cost resulted from a lower than expected lifetime of batteries should not be neglected, and some functions reflecting the cost of cycling batteries due to charge/discharge actions should be included to achieve the optimal dispatch.

1.2. Literature Review

Many control strategies and optimization methods, including model predictive control (MPC) [5,8,14,15], dynamic programming (DP) [3,4,7,16], sliding mode control (SMC) [17,18], reinforcement learning (RL) [9,10], particle swarm optimization (PSO) [11,19,20,21], and mixed-integer linear programming (MILP) [6,13,15], have been proposed for renewable power control under different conditions. The use of MPC is mainly due to forecasting errors. With real-time forecasting data within a short horizon that updated at every control step, the effect of errors can be reduced, and any mismatch in the supply and the demand can be identified promptly and solved in the following control step. In [5], the wind power smoothing problem is formulated to optimize the maximum ramp rate and the battery state with wind power prediction. This model is further investigated in the case of frequency control due to disturbances in the supply–demand balance [8]. Authors of [14] present a MPC scheme to optimize microgrid operations while meeting changing request and operation constraints. The optimization problem is formulated using MILP that can be solved efficiently to meet the real-time operating constraints. An online power scheduling for microgrids with renewable generations, BESS, heating, and cooling units is presented in [15]. The MPC scheme is used with a feedback correlation to compensate for prediction errors. The energy management problem in microgrids can be perceived as controlling different units over multiple time periods under uncertainty, which can be considered as a Markov decision process (MDP) and solved with DP. In [3], the energy management problem in microgrids with renewable power from six different generation sites is considered. The optimization problem is formulated based on different energy prices and solved with DP. A decentralized energy management strategy in microgrids with thermostatically controlled loads, solar power, distributed generators, and BESSs is proposed in [3], which can determine the optimal controls for BESS and distributed generator to minimize the energy cost while maintaining the desired temperature in local buildings. An optimal dispatch strategy for grid-connected wind power plant with BESS is proposed in [7], in which the DP algorithm used can incorporate the prediction of wind power and electricity price simultaneously to determine the optimal controls for BESS to maximize the profit. Authors of [16] develop a recursive DP algorithm to solve the optimal power flow in a microgrid considering limits on storage devices, network currents, and voltages. In the SMC scheme, a desired trajectory in the system will be defined, and the control objective is to track it. In [17], a decentralized SMC-based strategy to improve the performance of microgrids with renewable generations, BESS, non-linear, and unbalanced loads. The sliding surfaces used are predefined trajectories for active/reactive power to minimize fluctuations, compensate negative sequence, and harmonic currents. They further investigate the issue of stability and power-sharing in hybrid AC/DC microgrids with a similar control scheme in [18]. The population-based algorithm, PSO, is used in some studies to optimize the power scheduling in microgrids. A day-ahead multi-objective optimization dispatch in a microgrid considering costs from batteries and carbon emissions is solved with a PSO algorithm in [11]. The algorithm is also applied in microgrids with hybrid renewable generation units to address the optimal power management problems that are complicated, which is to optimize objectives including the annual cost of the system, loss of load expected, loss of energy expected, system costs of investment, replacement, operation, and maintenance, see, e.g., ([19,20,21]). In addition, there are some learning-based algorithms proposed for microgrids. Authors of [9] formulate the energy management problem in a microgrid as a Markov decision process to minimize the daily cost and address it with deep learning. A Q-learning based operation strategy to maximize the profit of a microgrid with community battery storage system is proposed in [10], which combines both centralized and decentralized approaches to control the system units.

Based on these studies, it can be perceived that the cost of batteries will be a key factor to achieve an optimal dispatch in microgrids with renewable generation units. Over the past decades, battery lifetime and degradation modeling have been investigated extensively, partly due to the increasing interest in the electric vehicle, energy arbitrage, and renewable power applications from researchers and investors [22]. According to [23], these models can be categorized into theoretical models and empirical models. The theoretical models are constructed based on the chemical mechanism in the battery cell, such as the aging of active material, chemical decomposition, and surface film modification [24]. In the case of operation planning, authors of [23] believe that the theoretical models may not be suitable as the chemical reaction processes inside the cells can be difficult to correlate with charge and discharge actions. On the other hand, the empirical models are believed to be appropriate for BESS planning and have been studied previously [22,23,25]. These models are developed based on degradation experiments, in which batteries are cycling at different operating conditions until they reach the end of life. The number of charge–discharge cycles that the battery can experience will be counted, and the effects of stress factors, such as temperature, depth of discharge (DOD), average state of charge (SOC), and operating currents on the rate of life cycle loss can be investigated. It should be pointed out that most of the cost models of BESS in previously reviewed studies are either not considered [3,5,6,7,8,9,10,14,15,16] in the optimization problem, or simple models that without consideration on the cycling model under different stress factors [4,11,13].

With these models, calculating the cost of battery cycling would become identifying the number of cycles and their corresponding stress factors. For all cycles and half-cycles (charge/discharge events), the corresponding operating parameters such as DOD, temperature, and current will be compared with a predefined operating condition measured from empirical experiments to determine their actual consumption on the battery life, and the resulting cost can be identified. However, according to the authors of [22], in the case of scheduling the optimal operation of BESS, the relationship between the number of cycles and battery actions can only be expressed analytically with complicated forms and would be difficult to use with an optimization solver. In other words, the models used to calculate the cost of battery cycling would require some algorithms to search the operating cycles of batteries, which are analytical methods that cannot be described with mathematical equations, therefore would be difficult to include in the objective function of optimization problems. As a result, these models are generally used for assessment rather than planning.

1.3. Contribution

In this paper, we aim to develop a control strategy of BESS in a grid-connected microgrid to optimize the cost with consideration on the cost from the battery cycling model. Based on previous studies, the optimization problem is formulated as a MDP, and a dynamic programming algorithm is utilized to optimize the overall cost in the microgrid over each planning horizon. We also employ the receding horizon approach to minimize the effect of forecasting errors and real-time mismatches; therefore only the first action solved will be used as the actual input to the system in the current stage, and the algorithm will repeat in the next step with updated system states and predicted data.

The main contribution of this paper is a novel cost model of battery based on the empirical battery life cycle models. The proposed cost model is recursive and additive over the control stages and can be included in the objective function of the optimization problem. It is also compatible with the DP algorithm that can be used to solve MDP problems, which does not require additional computations than regular DP algorithms.

The remainder of the paper is structured as follows. Section 2 introduces the overall optimization problem. The control system and the cost model of the battery are described in Section 3. The dynamic programming control algorithm and the simulation results are introduced in Section 4 and Section 5, respectively. The last section concludes the paper.

2. Problem Statement

We consider a grid-connected microgrid consisting of a battery energy storage system (BESS) and a renewable generation unit. A deregulated energy market environment is assumed, and the microgrid can communicate with the utility grid to conduct energy trading by controlling the actions of the BESS. The objective is to minimize the operating cost of BESS and maximize the profit of the microgrid. A configuration of the microgrid can be seen in Figure 1.

The receding horizon approach is utilized, and predicted data on renewable power, electricity price, and power demand over every planning horizon are updated at each control step. The cost of power trading with the energy market, and the cost of battery cycling over a planning horizon are used as the cost function of the problem, and a dynamic programming algorithm is used to solve the optimization problem over each horizon. Only the first action solved in each horizon will be used as the actual input to the system, and the algorithm will repeat in the next step with updated forecasting data and system states.

The energy stored in the BESS is considered as the system state, and the charge/discharge power is the control input. We assume multiple batteries are coordinated and charged/discharged with the same amount of power, which is to avoid the long string structure that can lead to charge imbalance [26] and increase the amount of controllable power.

3. System Model and Costs

3.1. Microgrid Model

We consider our system as a discrete-time system with the sampling rate $\delta >0$, and most data used are piecewise constant with constant values over periods $[n\delta ,(n+1\left)\delta \right)$, where n is a non-negative integer.

3.1.1. Predicted Data

It is assumed that three sets of predicted estimates of renewable power, load power demand, and electricity price can be made over every planning horizon, which is supposed to be piecewise constants. We use $P_{rn}\left(k\right)$, $P_{ld}\left(k\right)$ and $C_{ele}\left(k\right)$ to represent the renewable power, load power demand, and electricity price predicted over the period $[k\delta ,(k+1\left)\delta \right)$ respectively. As the model predictive control approach is utilized, these forecasting data will be updated prior to each control step, and fast predicting technique for a short period would be required, see, e.g., [27,28]. Also, we assume that the electricity price is identical for both selling and buying.

3.1.2. Battery Energy Storage System

The dynamics of the battery [13] used in the microgrid is described by the following equation.

$$E_B(k+1)=E_B\left(k\right)-P_B\left(k\right)\Delta \delta -d|P_B\left(k\right)\Delta \delta |,$$

(1)

where $E_B(·)$ is the energy state of the battery, $\Delta \delta$ is the factor used to convert power to energy based on the actual time of each control step, $P_B(·)$ is the charge/discharge power of the BESS, $d>0$ is the charging/discharging loss factors of the BESS. Based on the model, $P_B(·)>0$ indicates the discharging action and $P_B(·)<0$ indicates the charging action of BESS. In addition, we assume n batteries integrated into the microgrid, which are controlled and balanced simultaneously with the same amount of charge/discharge power.

The following constraints limit the operation of the battery,

$$E_B^{min}\le E_B\left(k\right)\le E_B^{max}$$

(2)

$$-P_B^{min}\le P_B\left(k\right)\le P_B^{max}$$

(3)

where $E_B^{min}$ and $E_B^{max}$ are the lower and upper limits of the battery energy state, which are used to avoid overcharge and over-discharge. $P_B^{max}$ and $P_B^{min}$ are the maximum amounts of power that can be charged and discharge from the battery within a control step.

3.1.3. Power Balancing in the Microgrid

Let $P_G$ be the power exchanged with the energy market, and the following equation can represent the power balancing task in the microgrid.

$$P_G\left(k\right)=P_{rn}\left(k\right)-P_{ld}\left(k\right)+n·P_B\left(k\right).$$

(4)

It can be perceived that $P_G>0$ indicates the microgrid is transmitting power to the utility grid, and $P_G<0$ indicates some power is purchased from the main grid to meet the power demand in the microgrid. Then the energy cost will be $C_{ele}\left(k\right)·P_G\left(k\right)$, where a positive value represents the profit gained by selling and a negative value represents the cost to buy power from the main grid, and minimize the term $-C_{ele}\left(k\right)·P_G\left(k\right)$ is equivalent to maximizing the profit. Also, in consideration of the ramp rate requirements on the main power grid, $P_G$ should satisfy the following constraint:

$$-P_G^{min}\le P_G\left(k\right)\le P_G^{max}$$

(5)

where $0<P_G^{min}<P_G^{max}$ are given constants.

3.2. Battery Aging and Cost Models

Our cost model of battery is built on the battery lifetime model used in [22]. One significant feature of the model is to identify the ‘half-cycle’, or the difference between two adjacent local extremes on the curve of the battery energy level. These ‘half-cycles’ contain information on the charge and discharge actions of the battery and can be used to assess its degradation. Another significant factor is the temperature, which has been studied in [23,25]. In our case, we assume a constant operating temperature of the BESS controlled by specific cooling devices.

3.2.1. Cycle Life Model

The basic cycle life model considered in [22] is described as follows:

$$T_{cycle}={\displaystyle \frac{N_d^{fail}}{w·n_d^{day}}}$$

(6)

where $N_d^{fail}$ is the maximum number of charge–discharge cycles that the battery can experience at a specific DOD before its end of life, $n_d^{day}$ is the number of daily cycles that the battery experienced at the DOD, and w is the average number of operating days within a year. So $T_{cycle}$ denotes the estimated lifetime of the battery in years. Furthermore, $N_d^{fail}$ can be expressed as a function of the DOD by fitting the typical empirical data provided by the manufacturers, which is:

$$N_d^{fail}=f\left(d\right)=N_{100}^{fail}{d}^{-kp}$$

(7)

where d is the DOD, $N_{100}^{fail}$ is the number of cycles at 100% DOD, and $kp$ is a constant. The curves of cycle life versus DOD at different $kp$ values are presented in Figure 2.

Assuming $n_d$ cycles of d DOD are experienced by the battery, its cycle life loss ${Ls}_{cycle}(\%)$ can be described as:

$$L{s}_{cycle}={\displaystyle \frac{n_d}{f\left(d\right)}}\times 100\%$$

(8)

As a result, with the same rate of cycle life loss $L{s}_{cycle}$, the equivalent 100%-DOD cycle number, indicated as $n_{100}^{eq}$, at d DOD with $n_d$ cycles can be derived from the following equation.

$${\displaystyle \frac{n_{100}^{eq}}{N_{100}^{fail}}}=L{s}_{cycle}={\displaystyle \frac{n_d}{f\left(d\right)}}$$

(9)

Substituting Equation (7) into (9), the equivalent 100%-DOD cycle can be derived as:

$$n_{100}^{eq}=n_d·d^{kp}.$$

(10)

Based on the equation, the DOD and the number of different cycles can be counted and converted to the equivalent value, and if we assume that the battery would experience the similar pattern of operation within a certain period, the corresponding lifetime can be estimated with Equation (6).

3.2.2. Counting Half Cycles

Instead of counting full cycles, authors of [22] assume the battery completes a half-cycle between two adjacent local maximum and minimum of the energy level, which is also the switching point between a charge and a discharge action. Let $E_{max}$ be the rated energy capacity of the battery, and $E_k$ be the energy level in the battery at the end of k-th half cycle, the corresponding DOD indicated as $d_k^{half}$, can be described as:

$$d_k^{half}=\left|{\displaystyle \frac{E_k-E_{k-1}}{E_{max}}}\right|$$

(11)

Based on Equation (11), the equivalent 100%-DOD cycle number of K half cycles can be calculated as:

$$n_{100}^{eq}=\sum _{k=1}^{K}0.5·{\left(d_k^{half}\right)}^{kp}$$

(12)

3.2.3. Cycle Life Cost Model

As the half cycle can be perceived as the individual charge/discharge action in-between local extremes, the term $L{s}_{cycle}$(%), namely the cycle life loss percentage would be suitable to determine the battery life cost model. Considering a replacement cost $R_c$($) incurred at the end of the battery life, and a cycle life loss percentage $L{s}_{cycle}\left(w\right)$ within a period w, the corresponding cost of battery lifetime consumption within the period will be the product of the two terms. In our case, the period W is the actual time of each planning horizon, and the cycle life loss can be determined from battery actions within the horizon. For a single half-cycle with depth $d_k^{half}$, the cost of cycle life loss $C_{loss}$ can be derived as:

$$C_{loss}\left(d_k^{half}\right)={\displaystyle \frac{n_{100}^{eq}}{N_{100}^{fail}}}{R}_c={\displaystyle \frac{0.5·{\left(d_k^{half}\right)}^{kp}}{N_{100}^{fail}}}{R}_c.$$

(13)

Therefore, the optimization problem can be formulated to minimize the cycle life cost and the energy cost to meet the power demand by making decisions on BESS actions at different electricity prices.

3.3. Redefine the Cost Model of Battery Cycle Life

In this section, we propose a new model for the cost of cycle life loss that can be utilized in the dynamic programming approach.

The dynamic programming approach is a recursive algorithm based on the Bellman principle of optimality [29]. It starts with the last step of the planning horizon and loops over the two adjacent states within the step. For every former state, an action that can optimize the cost within this control step will be determined, and the corresponding cost will be memorized. The algorithm will then proceed to the second last step of the planning horizon and repeat the same procedure. The previously memorized values, which are now the cost of the latter states in the current control step, will be used to determine the optimal costs within the step. Once the algorithm reaches the first step in the planning horizon, the optimal costs for all initial states over the horizon can be retrieved.

One basic principle of using the approach is that the cost of the problem should be additive [30]. In our case, the cost of energy trading is related to the amount of power exchanged with the main power grid, which will be accumulated over control steps. However, the costs from battery cycle life loss are determined by the local extreme points, which cannot be identified within every control step, as the battery could undergo consecutive charge or discharge actions over multiple steps in practice. In other words, the cost of battery life loss, derivable from Equation (13), cannot be calculated from looping over adjacent states in the DP algorithm. As a result, we propose a method to calculate the cost that can be applied in the DP algorithm.

Referring to Figure 3. Let BC and CD be the actions of BESS on step 2 and 3, and A be the former state of SOC to be decided in step 1. Based on Equation (13), the cost over step 2 and 3, or the ‘cost-to-go’, will be ${\displaystyle \frac{0.5R_c}{N_{100}^{fail}}}{\left(\right|B-C\left|\right)}^{kp}+{\displaystyle \frac{0.5R_c}{N_{100}^{fail}}}{\left(\right|C-D\left|\right)}^{kp}$. If A is higher than B, the cost over step 1, 2 and 3 will be ${\displaystyle \frac{0.5R_c}{N_{100}^{fail}}}{\left(\right|A-C\left|\right)}^{kp}+{\displaystyle \frac{0.5R_c}{N_{100}^{fail}}}{\left(\right|C-D\left|\right)}^{kp}$, as $AB$ and $BC$ are two subsequent discharge actions with the local minimum C. On the other hand, if A is lower than B, $AB$ will become a charge action, and the cost over the three steps is ${\displaystyle \frac{0.5R_c}{N_{100}^{fail}}}{\left(\right|A-B\left|\right)}^{kp}+{\displaystyle \frac{0.5R_c}{N_{100}^{fail}}}{\left(\right|B-C\left|\right)}^{kp}+{\displaystyle \frac{0.5R_c}{N_{100}^{fail}}}{\left(\right|C-D\left|\right)}^{kp}$.

Since only the cost within the current step will be evaluated by the dynamic programming algorithm, we propose the following method to calculate the life cycle cost:

Considering a step k within the planning horizon of N steps, $k=0,1,\dots ,N-1$. Based on the battery model introduced in Section 3.1, $E_B\left(k\right)$ and $E_B(k+1)$ will be the former and latter state on step k, and we define ${\sigma }_k$ as the ‘subsequent local extreme’ of step k, which is the following local extreme seen from the state $E_B\left(k\right)$, and determined by the actions after step k. Then the cost of battery life cycle loss on step k can be calculated as:

$$C_{loss}\left(k\right)={\displaystyle \frac{0.5R_c}{N_{100}^{fail}}}\left(\right|{\displaystyle \frac{E_B\left(k\right)-{\sigma }_k}{E_{max}}}{\left|\right)}^{kp}-{\displaystyle \frac{0.5R_c}{N_{100}^{fail}}}\left(\right|{\displaystyle \frac{E_B(k+1)-{\sigma }_k}{E_{max}}}{\left|\right)}^{kp}$$

(14)

Referring to Figure 3, it can be perceived that if $A\ge B$, ${\sigma }_1$ will be state C. Substituting the values into Equation (14), the cost in this step will be ${\displaystyle \frac{0.5R_c}{N_{100}^{fail}}}{\left(\right|A-C\left|\right)}^{kp}-{\displaystyle \frac{0.5R_c}{N_{100}^{fail}}}{\left(\right|B-C\left|\right)}^{kp}$, and the cost over the three steps is ${\displaystyle \frac{0.5R_c}{N_{100}^{fail}}}{\left(\right|A-C\left|\right)}^{kp}-{\displaystyle \frac{0.5R_c}{N_{100}^{fail}}}{\left(\right|B-C\left|\right)}^{kp}+{\displaystyle \frac{0.5R_c}{N_{100}^{fail}}}{\left(\right|B-C\left|\right)}^{kp}+{\displaystyle \frac{0.5R_c}{N_{100}^{fail}}}{\left(\right|C-D\left|\right)}^{kp}$, which is equal to ${\displaystyle \frac{0.5R_c}{N_{100}^{fail}}}{\left(\right|A-C\left|\right)}^{kp}+{\displaystyle \frac{0.5R_c}{N_{100}^{fail}}}{\left(\right|C-D\left|\right)}^{kp}$, the result we discussed previously. Similarly, If $A<B$, ${\sigma }_1$ will be state B and the cost within the step is ${\displaystyle \frac{0.5R_c}{N_{100}^{fail}}}{\left(\right|A-B\left|\right)}^{kp}-{\displaystyle \frac{0.5R_c}{N_{100}^{fail}}}{\left(\right|B-B\left|\right)}^{kp}$. The cost over the three steps will become ${\displaystyle \frac{0.5R_c}{N_{100}^{fail}}}{\left(\right|A-B\left|\right)}^{kp}+{\displaystyle \frac{0.5R_c}{N_{100}^{fail}}}{\left(\right|B-C\left|\right)}^{kp}+{\displaystyle \frac{0.5R_c}{N_{100}^{fail}}}{\left(\right|C-D\left|\right)}^{kp}$, which coincides with the previous result. In the dynamic programming algorithm, an extra array should be created to memorize the ‘subsequent local extreme’ for all states, and a program to update this value should be included as well. We will discuss more on this in the next section.

4. Optimization Technique

4.1. Optimization Problem

Based on the system defined in previous sections, the control inputs are the BESS actions $P_B\left(k\right)$ in n batteries, which determine the cost of battery life cycle cost $C_{loss}$ in Equation (14), and the power exchanged from the microgrid with the energy market. To state the optimization problem, we propose the following cost function based on Equations (4) and (14).

$$\sum _{k=0}^{N-1}n{C}_{loss}\left(k\right)-C_{ele}\left(k\right)P_G\left(k\right)$$

(15)

Then the optimal control problem can be stated as: given $P_{rn}\left(k\right)$, $P_{ld}\left(k\right)$, and $C_{ele}\left(k\right)$ for all k in every planning horizon, find the control input $P_B\left(k\right)$ such that the constraints (2)–(5) hold and the minimum of (15) is achieved.

To solve this problem, we introduce the Bellman function $V(k,E_B)$ as follows: For all $k=0,1,\dots ,N-1$, $E_B\in [E_B^{min},E_B^{max}]$, $P_B\in [-P_B^{min},P_B^{max}]$,

$$\begin{array}{cc}\hfill V(N,E_B):=& 0\forall E_B\in [E_B^{min},E_B^{max}]\hfill \end{array}$$

(16)

$$\begin{array}{cc}\hfill V(k,E_B):=& V((k+1),E_B)+\underset{P_B}{min}(n{C}_{loss}\left(k\right)-C_{ele}\left(k\right)P_G\left(k\right)).\hfill \end{array}$$

(17)

The algorithm can be solved recursively by starting from $k=N-1$ and computing $V(k,E_B)$ for all $E_B$. With a given initial state $E_B\left(0\right)$, the minimum of (15) can be obtained when $k=0$, and the optimal set of $P_B$ over the planning horizon of N steps can be retrieved.

4.2. Updating Local Extremes

As the ‘subsequent local extreme’ ${\sigma }_k$, introduced in Section 3.3, is required to compute the cost of life cycle loss $C_{loss}$. Based on the Bellman function introduced previously, we can assign ${\sigma }_k$ for every $V(k,E_B)$, which can be considered as the next local extreme seen from the state $E_B\left(k\right)$. However, since ${\sigma }_{k-1}$ is required to calculate the values of $V(k-1,E_B)$, we will introduce a method to update ${\sigma }_{(k-1)}$ with ${\sigma }_k$ in a recursive order suitable for the dynamic programming algorithm.

Referring to Figure 4, as the dynamic programming algorithm starts from the last step of the horizon, for all $E_B(N-1)$, the corresponding values of ${\sigma }_{N-1}$ will be equivalent to a state $E_B\left(N\right)$ that generates the minimal cost along the path, since $V(N,E_B)=0$ and ${\sigma }_N$ does not exist. Then the values of ${\sigma }_{N-1}$ will be memorized for the corresponding $E_B(N-1)$. In the remaining iterations, the values of ${\sigma }_k$ can be updated with the following method:

At step $k-1$, and given $E_B(k-1)$, $E_B\left(k\right)$, and ${\sigma }_k$ of $E_B\left(k\right)$ solved previously. There are two conditions. First, if the action between $E_B(k-1)$ and $E_B\left(k\right)$ is opposite to the action from $E_B\left(k\right)$ to ${\sigma }_k$, the ‘subsequent local extreme’ of $V(k-1,E_B(k-1))$ will be $E_B\left(k\right)$, namely ${\sigma }_{k-1}=E_B\left(k\right)$. Second, if both actions are the same type (both charging/discharging), or the action in step $k-1$ is idling, then the ‘subsequent local extreme’ ${\sigma }_{k-1}$ seen from the state $E_B(k-1)$ should update to ${\sigma }_k$.

With this updating method, we can determine the ‘subsequent local extreme’ for every former state in every step in the dynamic programming algorithm, and the cost of cycling can be calculated with Equation (14) to determine the minimum of (15).

It can be perceived that the cost model eliminates the need to run a cycle counting algorithm along the planning horizon and decompose it into a recursive and additive function, which allows the cycle-counting-based battery cost models to be included in the objective function of the optimization problem.

5. Simulation

5.1. Set Up

The proposed strategy is tested with computer simulation. Parameters of the battery used in the simulation are summarized in

Table 1. Microgrid parameters.

$P_B^{max},P_B^{min}$	$P_B^{max},P_B^{min}$	$E_B^{min}$	$E_B^{max}$	$E_{max}$	$R_c$
24 MW	60 MW	1.25 MWh	11.25 MWh	12.5 MWh	2,500,000 $

and

Table 2. Battery cost parameters.

$N_{100}^{fail}$	kp	$\Delta \mathit{\delta }$	d	n
2347	1.1	1/12	0.05	5

, which are based on the data provided in [16,23].

5.2. Parameters and Database

The battery used in [23] can experience 3000 cycles at 80% DOD, assuming the value of $kp$ is 1.1, the cycles to failure at 100% DOD is around 2347. The cost of the battery used is 200 $/kWh. In addition, we choose 90% and 10% state-of-charge of BESS as the upper and lower bound for the BESS energy state to avoid overcharging/discharging.

It is assumed that the actual time of each control step is 5 min, as the predictive data on renewable power, electricity price, and load power are average values based on five-minute observations. The planning horizon considered is two-hour, and there will be 24 control steps on each horizon.

We conducted a one-day simulation, and the renewable power data is retrieved from the Woolnorth wind farm in Tasmania, Australia; the electricity price and power demand data are retrieved from the Australian Energy Market Operator. It should be noted that the load demand data is downscaled to the level of a microgrid.

We test two sets of data. The first set has an overall higher demand than the renewable power generated, and the total amount of power generated in the other one is higher than the demand, which can be seen in Figure 5 and Figure 6.

As the data used are actual observations, we include some errors in the electricity price, renewable power, and load power demand based on the short-term forecasting technique presented in [28] that produces a normalized mean absolute error that ranges between 5% and 14%. As a result, we include the normally distributed random errors with zero mean and the standard deviation equal to 5% of the mean value of the actual data. The actual electricity price and the one with errors are illustrated in Figure 7.

Due to the errors in renewable power and load power, the actual power dispatched to the grid will deviate from the results solved by the algorithm, whereas the cost of battery life cycle loss will not be affected by these errors as the state transition in the battery is deterministic. In our simulation, the data sets with errors are used in the algorithm, and the results are assessed with the actual data sets.

5.3. Simulation Results

With an initial state of 80% SOC (10 MWh), and the electricity data in Figure 7, the control inputs of the BESS and the change in the battery SOC solved under the two data sets can be seen in Figure 8 and Figure 9.

The number of ‘half-cycles’ identified from the first data set was 21, with a long half cycle from the initial state to the lower limit and a number of small cycles near the lower limit, whereas the number of ‘half-cycles’ identified from the second data set was five. Most of the charging actions, namely the negative values in BESS actions in both data sets can be observed around step 90 to 100, and step 200 to 210, which are periods that have relatively higher values of electricity prices than others. As the state of charge before these two periods are close to the lower limit, those spikes in the electricity prices would lead to some charging actions solved by the algorithm to achieve higher profit.

The costs calculated based on the simulation results are summarized in

Table 3. Costs solved with both data sets.

	BESS Cost	Energy Trading Cost	Overall Cost	Cost without BESS
Higher demand	3849.07	410,851.24	414,700.31	433,108.81
Lower demand	2548.24	−394,509.25	−391,961.01	−369,880.45

.

The BESS cost is calculated using Equation (13), and the energy trading cost is determined with $\sum (-C_{ele}\left(k\right)P_G\left(k\right))$ over the simulation period with forecasting errors. Also, we include the results solved with the case of balancing the power differences by exchanging power with the utility grid only, which is the cost without BESS in the table, solved with $\sum (-C_{ele}\left(k\right)(P_{rn}\left(k\right)-P_{ld}\left(k\right)))$.

As the objective function in our problem is to minimize the cost function (15), a lower cost indicates a better result. In the case of higher demand, the overall cost 414,700.31 is the cost spent on buying energy from the energy market, and a reduction of 18,408.5, which is around 4.5% of the daily cost, can be achieved. In the lower demand condition, the negative sign in the cost indicates the profit gained from selling the excessive power, and an increase of 22,080.56 can be observed, which is around 5.6% of the daily profit.

5.4. Discussion

Compared with some population-based heuristics optimizers, such as differential evolution (DE) and PSO [19,20,21], DP could be inefficient due to the well-known ‘curse of dimensionality’. Nevertheless, in the case of optimal control considering battery cost from cycling, these population-based optimizers can be costly for a large size of population, as a cycle counting algorithm will be invoked each time the entities/particles calculate the value of cost function. Also, these algorithms would require some adjustments in their parameters and initial positions to obtain the best results, which could be undesirable for real-time applications. In addition, the major drawback of these population-based algorithms is the lack of a solid mathematical foundation to assess the result [31]. In comparison, the DP algorithm can guarantee the global optimum in the planning horizon, which could be more reliable and consistent in real-time operating. Also, our proposed cost model of battery eliminates the need to execute the cycle counting algorithm at the cost of extra storage space, which reduces the computation complexity and can be used as the benchmark to assess those population-based algorithms.

The proposed approach is a high-level control scheme, and the basic idea is to determine the optimal actions of BESS to maximize the profit in the microgrid. Although it is beyond the scope of this paper to investigate the low-level control, we would like to emphasize the importance of power electronics in the distributed generators (DGs)/batteries structure. In the microgrid environment, a large amount of DG units (e.g., solar panels, microturbines), storage units, and non-linear loads will be integrated, and a network of power inverters connected in parallel will be necessary in order to obtain good power sharing [32] and stabilize system frequency [33]. In our case, the distributed BESS is used, and multiple batteries connected in parallel should be coordinated and synchronized, which raises concerns on power sharing and frequency. We propose the high-level power scheduling control to optimize the cost, and a low-level control on the power inverters would be necessary, such as the hierarchical droop control for parallel-connected inverters introduced in [34,35,36,37].

6. Conclusions

In this paper, we proposed a control strategy to maximizing the profit in a microgrid with a renewable power system and a battery energy storage system. The predicted data on renewable power, load power and electricity price are used to determine the suitable control inputs of the system, and we used the receding horizon approach to alleviate the effects of forecasting errors by updating the prediction constantly. We also considered the cost of battery cycling and developed a recursive cost model. A dynamic programming algorithm is used to solve the optimization problem over each planning horizon, and the cost model is compatible with the algorithm. We tested our algorithm with actual data, and simulation results have shown significant improvements in different conditions.