Electricity Cost Optimization in Energy Storage Systems by Combining a Genetic Algorithm with Dynamic Programming

Abstract

Recently, energy storage systems (ESSs) are becoming more important as renewable and microgrid technologies advance. ESSs can act as a buffer between generation and load and enable commercial and industrial end users to reduce their electricity expenses by controlling the charge/discharge amount. In this paper, to derive efficient charge/discharge schedules of ESSs based on time-of-use pricing with renewable energy, a combination of genetic algorithm and dynamic programming is proposed. The performance of the combined method is improved by adjusting the size of the base units of dynamic programming. We show the effectiveness of the proposed method by simulating experiments with load and generation profiles of various commercial electricity consumers.

1. Introduction

An energy storage system (ESS) is a system that can store energy and provide it for consumer use for a certain time period at an acceptable level. In an electrical grid system, the ESS can be used to adjust the electricity usage and charge. The ESS is charged and discharged when the electricity usage is low and high, respectively. In other words, the overall energy efficiency of the system is improved and the energy flow from the electrical grid connected to the system is stabilized. Reliability is the key to the effective use of smart grid systems and new renewable energy sources [1]. Thus, the demand for ESSs is increasing [2,3,4,5].

The ESS acts as buffer between energy generation and load. New renewable energy sources often generate electricity even when the electrical energy usage is low. To avoid the waste of energy, the energy can be stored in the ESS and withdrawn from the ESS when needed, thereby increasing the energy efficiency. Energy providers benefit from more predictable power generation requirements. The ESS provides reliable and high-quality electricity to all industrial, commercial, and residential users [6,7].

Dynamic electricity pricing has been used with new technologies such as smart meters. In dynamic electricity pricing, the electricity charges vary depending on the time of day and time-of-use (TOU) tariffs. Energy providers can set high prices during times of high energy use, which encourages the consumers to avoid the overuse of energy, thereby preventing emergencies such as power outages [8,9]. In general, the TOU pricing consists of two or three pricing tiers (e.g., light load, heavy load, and overload). The price depends on the time of day. Many utilities in various countries such as the US energy company, Pacific Gas and Electric (PG&E) [10], the Canadian energy company, Hydro Ottawa (HO) [11], Korea Electric Power Corporation (KEPCO) [12], and Taiwan Power Company (TPC) [13] offer TOU pricing for commercial and industrial customers.

In a pricing system in which the prices vary depending on the time of day, consumers can reduce electricity costs by using energy during times with low electricity prices. The ESS plays a crucial role in the dynamic pricing policy. By storing energy during low load periods and using the stored energy during a high pricing tier period, consumers can avoid high electricity bills. To maximize electricity bill savings based on dynamic pricing, various studies have been carried out regarding the scheduling of the charge/discharge amount of the ESS [14,15,16,17] or consumer electricity planning solution [18]. These studies focused on various optimization methods such as dynamic, linear, nonlinear, and mixed integer linear programming as well as stochastic and particle swarm optimization and genetic algorithms.

The most widely used method is dynamic programming (DP), which was first introduced by Maly and Kwan [19] who focused on minimizing electrical energy usage costs without reducing the battery life. Van de Ven et al. [20] focused on minimizing the installation costs of the ESS. They emphasized the user demand and price, such as the Markov decision process, which can be solved by DP. Koutsopoulos et al. [21] proposed an optimal ESS control system from the viewpoint of facility providers and solved the offline problem in a limited time period using DP. Romaus et al. [22] suggested stochastic DP for the energy management of the hybrid ESS for electric vehicles.

In this paper, we propose a method that solves an ESS scheduling problem for electricity cost optimization for enterprise ESSs with dynamic pricing and renewable energy sources. We suggest a DP approach that considers the forecasts of the power generation and load for 24 h. We also aim to improve the performance of the optimization method by combining DP with a genetic algorithm (GA). Although several studies focused on DP, it has some problems when applied to commercial electrical systems. In the case of residential electrical systems, the amount of power used is small such that the memory and time constraints for DP are relatively low, so DP can be a reasonable choice for this case. However, the amount of power used in commercial electrical systems is large. In that case, DP uses large memory sizes and time resources for finding good solutions. We can increase the size of the base unit in DP for reducing memory and time resources. However, in this case, errors will likely increase. To resolve this problem, we employ a genetic algorithm, which is one of the metaheuristic methods that can be used to identify near-optimal values (not the optimal values). By using the solutions of DP with a large base unit as the initial population of a GA, the memory and time constraints of DP can be satisfied.

In addition to charges based on the electrical energy usage, electricity bills may include a demand charge, which is determined by the highest amount of power (kW) during the billing period multiplied by the relevant demand charge rate ($/kW). The demand charge rate is usually fixed when a commercial or industrial customer signs the contract [23]. By this demand charge, utility companies can charge customers consuming large amount of power more fees for their use of extra resources associated with the power maintenance [24]. In a customer’s point of view, the larger the highest amount of power used during the billing period, the larger the demand charge. Hence, customers with demand charge should try to reduce the highest amount of power during the billing period to decrease their electricity bills. Since it is difficult to measure the exact amount of power practically, the highest amount of power is usually measured by calculating the electrical energy drawn during a predetermined time interval. We consider scenarios with demand charge and those without demand charge in this study.

The DP without demand charge produces reasonable dynamic pricing results. However, the design of DP is difficult if there are demand charges because the objective functions become much more complex. A metaheuristic method, such as a GA, can perform better than DP in such a case. In this paper, we compare the performances of DP and a GA for cases with demand charge and those without demand charge, and we propose more effective algorithms for each case by combining DP with the GA.

There have also been studies on the optimization of operation of ESS considering renewable energy in microgrid using various strategies including other metaheuristic algorithms. Wang and Huang [2] proposed a two-period stochastic programming program for the joint optimization of investment and operation of a microgrid, taking the impact of energy storage, renewable energy integration, and demand response into consideration. Mozafari and Mohammadi [3] applied bee swarm optimization algorithm to optimize the operation strategies and capacities of ESS considering various factors. In the study of Li et al. [4], a new optimal scheduling of ESS based on chance-constrained programming has been proposed for minimizing the operating costs of an isolated microgrid. Tushar et al. [5] proposed a real-time decentralized demand-side management in the integration of electric vehicles (EVs), ESSs, and renewable energy sources by formulating a game with mixed strategy between customers. In this paper, we explore a large variety of optimization approaches to energy storage systems especially using the combination of dynamic programming (DP) and genetic algorithms (GA). We conduct more extensive simulations than previous work, with various 18 scenarios, with and without demand charge.

We summarize our contributions in this study as follows: (i) we propose a combined method of DP and GA for electric cost optimization with renewable energy and ESS under TOU with/without demand charge; (ii) we improve the performance of the proposed method by adjusting the size of the base units of DP; (iii) we perform comparative experiments on the proposed method for various industrial electricity load and renewable energy generation profile; and (iv) finally we show that our combined method is effective for both cases with and without demand charge in terms of cost saving and time.

The remainder of the paper is organized as follows: the problem, optimization method used for ESS scheduling, and DP operation process are described in Section 2. Our method combining DP with a GA is presented in Section 3. The savings and computing time associated with DP, a pure GA, and the combination of DP and GA depending on various DP base unit sizes are compared in Section 4. We draw conclusions in Section 5.

2. Dynamic Programming for ESS Scheduling

2.1. Problem Formulation

We formally define ESS scheduling problems with demand charge and that without demand charge in this section. The definitions are similar to those presented in previous work [25]. The load $l_i$ refers to the amount of energy used during the $i$th time interval and $g_i$ refers to the amount of energy generated during the $i$th time interval. The variable $x_i$ refers to the amount of energy stored in the ESS at the $i$th time interval. Instead of SOC (state of charge), which is the level of charge of an electric battery relative to its capacity, we used the amount of energy stored in the ESS as a variable to be optimized. There is a relationship such that SOC at the $i$th time interval is the same as $\frac{x_i}{C}\times 100\%$, where $C$ is the capacity of ESS. So optimizing the value of $x_i$ can be considered the same as optimizing that of SOC after multiplying some coefficients.

Each time interval is defined to be one hour in this study. The amount of energy provided to the ESS at the $i$th time interval is $x_i-x_{i-1}$; thus, the net energy required from the power grid, $E_i$ can be calculated as follows:

$$E_i=x_i-x_{i-1}+l_i-g_i$$

(1)

That is, if the electricity price at the $i$th time interval is $p_i$, the electrical energy charge amount of the $i$th time interval is $E_i·p_i$. If $E_i$ is negative, it means that electricity is sent back to the grid. Although there may be several pricing policies for this feed-in electricity, in this study, we assumed that there is no compensation of the feed-in electricity. That is, the amount of net energy $E_i$ is negative, the cost at that time interval only becomes 0. Thus, the sum of the costs in $T$ time intervals can be represented as ${\sum }_{i=1}^T{I}_{ℝ+}\left(E_i\right)·\left\{E_i·p_i\right\},$ where $I_{ℝ+}\left(x\right)$ is the indicator function that returns 1 if $x$ is a positive real number, otherwise, returns 0. This ensures that the sum is not negative, although the costs in several time intervals can be negative. Therefore, the following equation is the formulation of this ESS scheduling problem.

Minimize:

$${\displaystyle \sum _{i=1}^T}{I}_{ℝ+}\left(E_i\right)·\left\{E_i·p_i\right\}$$

(2)

subject to

$$0\le x_i\le C,i=1,2,\dots ,T$$

(3)

$$-P_d\le x_i-x_{i-1}\le P_c,i=1,2,\dots ,T,$$

(4)

where $C$ is the capacity of a battery, $P_d$ is the amount of maximum battery discharge in an hour, and $P_c$ is the amount of maximum battery charge in an hour. This means that $x_i$ cannot exceed the capacity of the battery and $x_i-x_{i-1}$ must range between $-P_d$ and $P_c$.

The objective function, Equation (2) is the sum of hourly electrical energy costs, and each hourly cost is calculated by multiplying the amount of electrical energy from the power grid during an hour and the electricity price at that time. Only when the amount of electrical energy from the power grid is negative, the cost of that time interval is 0. This property is represented with indicator function $I$. Equation (2) does not have a linear nor a quadratic property because of the existence of the function $I$. The function only produces 0 or 1. Moreover, the objective function is not convex. At some points, gradients cannot be calculated. So general linear or quadratic programming cannot be applied to this problem.

Equation (2) is the objective function when we assume that the battery efficiency can be 100%. In fact, recent battery technology has developed a lot, and it is becoming possible to develop a battery with an efficiency of 99% or more with the lithium-ion battery (Li-ion) [26], lithium-sulfur battery (Li-S) [27], and vanadium redox flow battery (VRFB) [28]. However, this high efficiency can be achieved in an ideal environment, so in practice, there would be battery charge and discharge loss. These losses are likely to lead to some different simulation results. So, in our experiments, we used modified objective function considering battery efficiency $\alpha$. In this case, $E_i^{\prime }$, which is the net energy of the $i$th time interval considering battery efficiency $\alpha$, is calculated as follows:

$$E_i^{\prime }={\alpha }^{-1}(x_i-x_{i-1})+l_i-g_i,$$

(5)

where $0<\alpha <1$ is battery efficiency. That is, to increase the amount of energy stored in the battery from $x_{i-1}$ to $x_i$, the amount of ${\alpha }^{-1}\left(x_i-x_{i-1}\right)$ is required to charge the battery. The objective function of the problem considering battery efficiency can be written using the modified amount of net energy as follows:

$${\displaystyle \sum _{i=1}^T}{I}_{ℝ+}\left(E_i^{\prime }\right)·\left\{E_i^{\prime }·p_i\right\}$$

(6)

With regard to the pricing including the demand charge, the total electrical energy cost is the sum of the energy and demand charges, which is the product of the fixed rate $p^{*}$ and peak demand and can thus be written as: $\underset{1\le i\le T}{{\max }_{1\le i\le T}}{E}_i^{\prime }·p^{*}$ [29]. Peak demand refers to the highest amount of power during the billing period and is represented as kW. However, in practical, the highest amount of power is usually measured by calculating the electrical energy drawn during a predetermined time interval. So, in this study, we define peak demand as the largest hourly electrical energy required from the power grid during the billing period $T$. The problem related to minimizing the total electrical energy cost can then be formulated as follows:

Minimize:

$${\displaystyle \sum _{i=1}^T}{I}_{ℝ+}\left(E_i^{\prime }\right)·\left\{E_i^{\prime }·p_i\right\}+\underset{1\le i\le T}{\max }{E}_i^{\prime }·p^{*}$$

(7)

Equations (2) and (7), which are the objective functions in the case without and with demand charge respectively, do not have a linear or a quadratic property and are not convex. At some points, gradients cannot be calculated. So simple mathematical optimization methods using some gradients cannot be applied to these problems. In this study, we adopted DP and GA to solve the problems because they have some characteristics to fit these problems.

DP is based on splitting the problem into smaller subproblems and there should be an equation that describes the relationship between these subproblems. The problems defined in this study have those properties. The relationship between subproblems are addressed in the next subsection. GA can also be applied to these kinds of problems. It can be easily applied regardless of the type of objective function. GA is a sort of a metaheuristic, which is a higher-level procedure or heuristic designed to find, generate, or select a heuristic that may provide a sufficiently good solution to an optimization problem, especially with incomplete or imperfect information or limited computation capacity [30,31].

2.2. Assumptions and Limitations of the Proposed Problem Formulation

In this subsection, assumptions and limitations of the proposed problem formulation are discussed.

• In relation with the demand charge, the amount of the largest hourly electrical energy required from the power grid during the billing period is used for the value of peak demand instead of the exact amount power. In practical, power is usually measured by calculating the electrical energy drawn during a predetermined time interval. We used an hour as this time interval in this study, however, if we use a shorter time interval, such as five minutes, the result will be more accurate.
• For the electricity that is sent back to the grid, we assumed that there is no compensation of the feed-in electricity. If the other pricing policies for this feed-in electricity are applied, the problem formulation should be modified.
• Another existing study [32] has modeled the problem of scheduling the charge/discharge power of ESS considering power balance constraint. The problem formulation of our study has slightly different view. In our study, we optimize the amount of charge/discharge energy during the unit time interval instead of optimizing the power of the ESS. Therefore, energy balance among generation, load, grid, and ESS is considered instead of power balance. Both models can be applied to ESS scheduling problem considering the other environments.
• In this study, we experimented the proposed method assuming that actual generation and load completely follow the certain predetermined patterns. However, in practical, generation and load may not follow the same pattern every day, so the proposed method should be applied with some predicted generation and load patterns to be used in the field. There have been a number of recent studies on day-ahead prediction of photovoltaic (PV) output [33,34], wind power generation [35], and load [36,37,38,39]. It is expected that the proposed method of combining GA and DP will show a good performance when an ideal prediction algorithm with great accuracy is adopted as we simulated in this study. However, generation and load predictions will usually have errors and simulation results may be different from this study. A statistical analysis of the day-ahead (and two-days-ahead) load forecasting errors have been made in [40] and economic impact assessment of load forecast errors have been discussed in [41]. In the ESS scheduling problem addressed in this paper, if the net energy (the difference between renewable energy and load) is underforecasted, excessive electrical energy may be accumulated in the battery uselessly by the predetermined schedule. On the other hand, the net energy is overforecasted, the energy contained in the battery may be used up in advance, so the consumer may have to buy energy from the grid even when the price is high. In both cases, there can be some economic inefficiencies in practical.

2.3. Dynamic Programming

If there is no demand charge and the price is determined only by energy charge with TOU, near-optimal ESS schedules can be obtained using DP. The DP is a technique that improves the algorithm performance by storing previously computed optimal solutions of subproblems in the memory and reducing the computations based on the information stored in the memory if necessary.

We define the two-dimensional table $D\left[i,w\right]$ as the minimum electrical energy cost when an amount $w$ is stored in the ESS at the $i$th time interval. The $D\left[i,w\right]$ should select the minimum value of $D\left[i-1,x\right]+cost\left(i,x,w\right)$ for all possible values of the residual amount of the battery at the ($i-1)th$ time interval and $x$’s, where $cost\left(i,x,w\right)$ is the elecricity cost when the residual amount of the battery becomes $w$ at the $i$th time interval from $x$ at the ${(i-1)}^{th}$ time interval. Possible $x$ values are in the range of $\max \left(0,w-P_c\right)\le x\le \min (C,w+P_d)$ and $cost\left(i,x,w\right)$ is calculated as $\left(x-w+l_i-g_i\right)·p_i$ using the objective function. Therefore, the recurrence equation used in DP is as follows:

$$D\left[i,w\right]=\underset{w-P_c\le x\le w+P_d}{\min }\left(D\left[i-1,x\right]+cost\left(i,x,w\right)\right)$$

(8)

Therefore, the minimum electrical energy cost is one of $D\left[T,w\right]$’s in the last time interval $T$. Based on backward tracing, the path toward obtaining the optimal value can be found. The pseudo-code for a scheduling algorithm according to $T$ time intervals is given in Figure 1.

If load and power generation can be predicted accurately, the DP can achieve near-optimal solutions. However, in reality, it is very difficult to accurately predict the load and power generation. In addition, based on the proposed method, the algorithm can be performed in a short time period when the capacity of the battery is low such as in the residential power system, but it cannot be performed in a short time period when the capacity of the battery is high such as in the enterprise power system. One way to solve this issue is to use a large base unit in DP. For example, assuming that the capacity of the battery is 1000 kWh, the algorithm can be run faster if a base unit of 10 kWh is used instead of the default unit 1 kWh. Although the use of a large base unit makes the solution less accurate, it has an advantage in terms of the computing time.

Examples of DP with different base units are shown in Figure 2. In this example, DP in Figure 2a has three states (0, 5, and 10 kWh) and that in Figure 2b has 11 states (0–10 kWh). Both have four time intervals, the charging and discharging power is 5 kW, and the initial battery is empty.

The DP in Figure 2a is based on a base unit of 5 kWh and that in Figure 2b is based on a base unit of 1 kWh. The path from each point at the time interval $t$ to each point at the time interval $t+1$ is calculated for each possible pair, consisting of the states in the time intervals $t$ and $t+1$. This simple example shows that DP with smaller base unit is more complex and has higher computational cost than that with larger base unit.

The size of the base unit is important in the design of DP both in terms of solution quality and temporal performance. For example, if the capacity of ESS is 3.5 kWh and we set 1 kWh as the base unit of DP, the proposed DP algorithm can only deal with 0 kWh, 1 kWh, 2 kWh, and 3 kWh as the value of $x_i$, and the amount of 0.5 kWh is not considered. In this case, the proposed DP algorithm is not efficient. On the other hand, if the capacity of ESS is 500.5 kWh and the base unit is also 1 kWh, the left amount of 0.5 kWh is not so critical compared with the former case. If we set 0.1 kWh as the base unit, the obtained solution will be more accurate. As shown in these examples, the smaller the base unit compared with the amount of capacity, the more efficiently the DP algorithm performs. If we set the base unit small enough, the proposed DP can produce near-optimal solution. However, DP with small base unit may have high computational cost so the size of the base unit should be carefully determined considering both the quality of solutions and time cost.

For the pseudocode of the proposed DP in Figure 1, it is easy to calculate the time complexity: $O\left(T·C·(P_c+P_d\right))$. Because $P_c$ and $P_d$ values are proportional to $C$, the complexity can be written as $O\left(T·C^2\right)$. This time complexity is valid for the DP with a base unit of 1 kWh. However, if a base unit of $b$ kWh is used, the time complexity becomes $O(T·\left(C/b{)}^2\right)$.

The proposed DP algorithm yields a near-optimal solution for the case without a demand charge. Moreover, if the domain is limited to set of integers with a given precision, it produces an optimal solution. For example, if we limit possible values of each $x_i$ to only integers that are multiples of 10, an optimal solution is derived by the proposed DP with base unit 10. However, the objective function becomes more complex when a demand charge is included. Thus, in that case, it is difficult to achieve the desired performance with a similar DP method described above. However, DP has a strong advantage of optimizing energy charge represented as the first term in Equation (7), which is the objective function of the problem with demand charge, because the first term is exactly the same as the objective function of the problem without demand charge. DP can optimize the energy charge, but it cannot optimize demand charge. If we combine DP with other appropriate methods to optimize demand charge, we can get good solutions of the problem. GA is adopted as the method to optimize demand charge in this study and the method will be explained in the next section.

3. Genetic Algorithm Process

The GA [42] is a metaheuristic technique that expresses information about solutions in a genetic form and optimizes a given objective function using an evolution process such as crossover and mutation. In addition, GAs for real-valued representations are called real-coded GAs (RCGAs). Real-valued representations were first used to generate a metaoperator and identify the most appropriate parameters related to chemometric problems. Nowadays, RCGA is mainly being used for numerical optimization problems in continuous domains [43,44,45,46].

In this study, the population size of the GA is set to 100. Both parents are randomly selected from the population, and crossover and mutation operations are performed. Subsequently, the worst solution in the population is replaced by the offspring generated by crossover and mutation. This process is repeated up to a maximum of 100,000 generations. The pseudocode of the proposed GA is shown in Figure 3.

• Encoding: in the proposed RCGA, a real-number vector is encoded, with a length of the number $T$ of the maximum time intervals. Unlike in general real encoding, the value of the gene $x_i$ of a solution vector is limited by the value of the gene $x_{i-1}$ of the previous index. Therefore, the range of $x_i$ is as follows:

  $$0\le x_i\le C,$$

  (9)

  $$-P_d\le x_i-x_{i-1}\le P_c\leftrightarrow x_{i-1}-P_d\le x_i\le x_{i-1}+P_c$$

  (10)

  $$\Rightarrow \max \left(0,x_{i-1}-P_d\right)\le x_i\le \min \left(C,x_{i-1}+P_c\right)$$

  (11)
• Evaluation: if there is no demand charge, the objective function of this problem is the same as Equation (2) in Section 2. If there is a demand charge, the function in Equation (4) is used. The lower the function value is, the higher is the possibility to be selected as parents.
• Initialization: an initial population of 100 individuals is generated and the encoding constraint in Equation (11) should be adhered to. The individuals are randomly generated and the limit is not exceeded.
• Crossover operator: in this study, blend crossover (${\mathrm{BLX}}_{\mathsf{\alpha }}$), one of the crossover techniques for real-valued chromosomes, is used, where $\mathsf{\alpha }$ is a non-negative real number. This crossover operation randomly determines genes within the range $\left[C_{min}-\mathsf{\alpha }I,C_{max}+\mathsf{\alpha }I\right]$, where $C_{max}=\max \left(x,y\right)$ and $C_{min}=\min (x,y$). The parameter $\mathsf{\alpha }$ used in this study is 0.5 and $I=C_{max}-C_{min}$. This study includes additional constraints because the encoding conditions should not be violated. Therefore, the range of the $i$th gene of the offspring of the crossover should be set to $\left[\max \left(0,x_{i-1}-P_d\right),\min \left(C,x_{i-1}+P_c\right)\right]$.
• Mutation: the mutation transforms a part of the offspring generated via crossover such that more diverse solutions are generated during the genetic process. Mutation is not performed always but depending on the probability value. In this study, the probability is set to 0.2. The mutation process selects a part of the chromosome index and changes the corresponding part, but it assigns values uniformly and randomly within the range of the encoding constraint.

We described the process of a pure GA in the above. We also examine the performance of a combination of GA and DP. In the proposed combined method, DP can help the GA to identify a better solution than that obtained with a standalone of pure GA or DP. For the combined method, firstly, the solution derived from DP is calculated. After that, the solution obtained by DP is included in the GA population, when constructing an initial population. The solution by DP and other solutions in the GA population evolves to better solutions by GA process through generations. So we can improve the solution quality by using this combined method regardless of base unit of DP. That is, we can obtain near-optimal solutions with more accurate precision, and the obtained solutions are always better than those by stand-alone DP.

As the ESS capacity increases, the temporal performance of DP degrades. However, the temporal performance can be improved by adjusting the size of the base unit in DP. To obtain economic efficiency and reasonable solution quality, the optimal solution of DP with a large base unit is included in the population of GA.

DP algorithm proposed in Section 2.2 produces reasonable results for the problem without a demand charge. However, when a demand charge is included, it is difficult to achieve the desired performance with the proposed DP, because the DP is designed to optimize the energy charge without considering the demand charge. If we combine the proposed DP with GA addressed in this section, we can get good solutions of the problem even for the case with demand charge. GA is a kind of metaheuristics, so it can find appropriate solutions that are fit for given objective functions.

4. Experimental Results

4.1. Experiment Data

The electricity load data used in this study were obtained from the Office of Energy Efficiency and Renewable Energy (EERE) [47] and include information about electricity load of United States (industrial and residential). This dataset contains hourly load profile data for commercial and residential buildings. Hourly load profiles are available for all TMY3 locations in the United States. We only used commercial building load profiles for Anchorage in our experiments. There are three types of commercial buildings (hospital, restaurant, and office). The photovoltaic (PV) watts calculator [48] was developed by the National Renewable Energy Laboratory (NREL). The PV watts calculator uses past PV data and calculates the energy generated from the grid-connected PV system. We used six types of PV generation data considering the combinations of three weather types (cloudy, rainy, and sunny) and two season types (summer and winter).

Table 1. Detailed information for each case.

Case	Season	Weather	Building
1	Summer	Cloudy	Hospital
2	Summer	Rainy	Hospital
3	Summer	Sunny	Hospital
4	Winter	Cloudy	Hospital
5	Winter	Rainy	Hospital
6	Winter	Sunny	Hospital
7	Summer	Cloudy	Office
8	Summer	Rainy	Office
9	Summer	Sunny	Office
10	Winter	Cloudy	Office
11	Winter	Rainy	Office
12	Winter	Sunny	Office
13	Summer	Cloudy	Restaurant
14	Summer	Rainy	Restaurant
15	Summer	Sunny	Restaurant
16	Winter	Cloudy	Restaurant
17	Winter	Rainy	Restaurant
18	Winter	Sunny	Restaurant

shows information about each test case.

Typical TOU prices were generated by simulations using three price levels for summer and winter based on the TOU pricing models of several utility companies. The TOU pricing model that was constructed in this study is given in

Table 2. Hourly pricing for summer and winter.

Hour (from-to)	Summer (Cents/kWh)	Winter (Cents/kWh)
0–1	5	5
1–2	5	5
2–3	5	5
3–4	5	5
4–5	5	5
5–6	5	5
6–7	5	5
7–8	10	15
8–9	10	15
9–10	10	15
10–11	10	15
11–12	15	10
12–13	15	10
13–14	15	10
14–15	15	10
15–16	15	10
16–17	15	10
17–18	10	15
18–19	10	15
19–20	5	5
20–21	5	5
21–22	5	5
22–23	5	5
23–24	5	5

Demand charge rate, $p^{*}=20$.

.

The ESS capacity used for the experiments differs for each building. The capacity of hospitals and offices is 500 kWh. The capacity of restaurants is 250 kWh. The $P_c$ and $P_d$ values are assumed to be one-fifth of the capacity.

4.2. Performace Comparison for the Case without Demand Charge

Table 3. Comparison of cost savings of the proposed methods without demand charge.

Case	DP1	DP10	GA	GA+DP1	GA+DP10	HS	HS+DP1	HS+DP10
1	10.32%	10.29%	8.75% (0.16%)	10.33% (0.00%)	10.29% (0.04%)	8.37% (0.17%)	10.32% (0.00%)	10.31% (0.05%)
	(2.924)	(0.052)	(0.192)	(3.201)	(0.257)	(0.224)	(3.212)	(0.281)
2	7.85%	7.79%	6.61% (0.11%)	7.85% (0.00%)	7.80% (0.03%)	5.22% (0.13%)	7.85% (0.00%)	7.79% (0.01%)
	(2.854)	(0.048)	(0.184)	(3.433)	(0.262)	(0.237)	(3.238)	(0.243)
3	7.36%	7.36%	6.05% (0.10%)	7.52% (0.50%)	7.36% (0.01%)	5.40% (0.10%)	7.36% (0.00%)	7.48% (0.02%)
	(2.994)	(0.047)	(0.164)	(3.452)	(0.273)	(0.242)	(3.228)	(0.201)
4	3.60%	3.60%	2.96% (0.14%)	3.60% (0.00%)	3.60% (0.00%)	1.93% (0.12%)	3.60% (0.00%)	3.60% (0.01%)
	(3.014)	(0.051)	(0.171)	(3.321)	(0.281)	(0.204)	(3.216)	(0.239)
5	3.58%	3.58%	2.95% (0.13%)	3.58% (0.01%)	3.58% (0.00%)	1.88% (0.15%)	3.58% (0.00%)	3.58% (0.00%)
	(2.962)	(0.039)	(0.199)	(3.361)	(0.299)	(0.218)	(3.276)	(0.267)
6	3.96%	3.96%	3.28% (0.06%)	3.96% (0.00%)	3.96% (0.00%)	2.09% (0.06%)	3.96% (0.00%)	3.96% (0.00%)
	(2.940)	(0.043)	(0.201)	(3.399)	(0.244)	(0.291)	(3.268)	(0.227)
7	13.35%	13.35%	11.64% (0.08%)	13.50% (0.05%)	13.45% (0.05%)	10.07% (0.16%)	13.50% (0.05%)	13.52% (0.02%)
	(3.092)	(0.045)	(0.175)	(3.417)	(0.263)	(0.272)	(3.213)	(0.298)
8	15.17%	15.10%	11.97% (0.15%)	15.17% (0.00%)	15.10% (0.03%)	9.30% (0.14%)	15.17% (0.00%)	15.11% (0.01%)
	(2.885)	(0.038)	(0.161)	(3.363)	(0.251)	(0.271)	(3.232)	(0.277)
9	17.66%	17.64%	14.46% (0.17%)	17.67% (0.00%)	17.64% (0.02%)	12.07% (0.12%)	17.67% (0.01%)	17.64% (0.01%)
	(2.911)	(0.050)	(0.134)	(3.367)	(0.259)	(0.208)	(3.268)	(0.214)
10	8.30%	8.30%	6.85% (0.13%)	8.30% (0.00%)	8.30% (0.00%)	4.49% (0.17%)	8.30% (0.00%)	8.30% (0.01%)
	(2.923)	(0.055)	(0.156)	(3.393)	(0.262)	(0.275)	(3.252)	(0.227)
11	8.19%	8.19%	6.80% (0.16%)	8.19% (0.00%)	8.19% (0.00%)	4.43% (0.13%)	8.19% (0.01%)	8.19% (0.00%)
	(2.924)	(0.058)	(0.161)	(3.264)	(0.241)	(0.221)	(3.206)	(0.234)
12	11.58%	11.58%	10.35% (0.10%)	11.60% (0.04%)	11.61% (0.03%)	8.38% (0.10%)	11.59% (0.02%)	11.60% (0.00%)
	(2.975)	(0.052)	(0.183)	(3.251)	(0.268)	(0.258)	(3.214)	(0.242)
13	21.17%	18.48%	15.95% (0.11%)	21.19% (0.03%)	19.33% (0.07%)	10.95% (0.11%)	21.17% (0.00%)	18.48% (0.01%)
	(2.963)	(0.053)	(0.199)	(3.411)	(0.290)	(0.251)	(3.289)	(0.271)
14	19.63%	19.19%	15.22% (0.08%)	19.67% (0.05%)	19.40% (0.02%)	10.11% (0.13%)	19.63% (0.04%)	19.19% (0.05%)
	(2.989)	(0.045)	(0.181)	(3.252)	(0.284)	(0.279)	(3.266)	(0.226)
15	23.83%	20.17%	18.86% (0.16%)	23.85% (0.04%)	20.91% (0.16%)	14.29% (0.10%)	23.83% (0.02%)	20.51% (0.06%)
	(2.931)	(0.047)	(0.179)	(3.245)	(0.285)	(0.249)	(3.240)	(0.274)
16	11.80%	11.80%	9.86% (0.13%)	11.80% (0.00%)	11.80% (0.00%)	6.41% (0.06%)	11.80% (0.03%)	11.80% (0.03%)
	(2.938)	(0.049)	(0.169)	(3.273)	(0.279)	(0.230)	(3.217)	(0.264)
17	11.77%	11.77%	9.75% (0.12%)	11.77% (0.00%)	11.77% (0.00%)	6.50% (0.15%)	11.77% (0.01%)	11.77% (0.00%)
	(2.946)	(0.045)	(0.185)	(3.293)	(0.268)	(0.263)	(3.279)	(0.276)
18	12.26%	12.26%	10.17% (0.09%)	12.26% (0.00%)	12.26% (0.00%)	6.51% (0.13%)	12.26% (0.00%)	12.26% (0.00%)
	(2.974)	(0.054)	(0.193)	(3.283)	(0.282)	(0.278)	(3.208)	(0.264)

For GA, GA+DP1, GA+DP10, HS, HS+DP1, and HS+DP10, average values and standard deviations over 100 runs are shown. The computing time in seconds is given in the second line of each cell. For each case, the best result among compared eight ones is shown in bold type.

and Figure 4 show the comparison of the temporal and economic performances of DP1 (DP with a base unit 1 kWh), DP10 (DP with a base unit of 10 kWh), GA, GA+DP1 (the combined method of GA and DP with a base unit 1 kWh), and GA+DP10 (the combined method of GA and DP with a base unit 10 kWh) without demand charge. To compare the performance of the proposed method with other existing methods, we also performed the Harmony Search (HS) algorithm previously proposed in [49]. HS is a kind of metaheuristic algorithm and can be applied to optimization problems instead of GA. We similarly implemented HS, HS+DP1 (combined method of HS and DP with a base unit 1 kWh), and HS+DP10 (combined method of GA and DP with a base unit 10 kWh) using HS instead of GA. For GAs and HSs, average values and standard deviations over 100 runs were given. Economic performances were measured by calculating cost savings for each case. Cost savings are expressed as a percentage of the cost when there is no ESS. That is, the cost saving of Algorithm A is calculated by the formula, $100\times \left(Cos{t}_{NO-ESS}-Cos{t}_A\right)/Cos{t}_{NO-ESS}$, where $Cos{t}_A$ is the electrical energy cost incurred by Algorithm A. Computing time of a single run of each algorithm, which is expressed in seconds, is also provided in Table 3.

For the DP method, we conducted experiments with base units 1 kWh and 10 kWh. In Table 3, as the base unit increases, the economic performance decreases, but the time performance increases. In terms of the temporal performance, DP1 is slower than DP10. The combined method of GA and DP is affected by the time consumed, hence, GA+DP1 is slower than GA+DP10. GA is slower than DP10 but considerably faster than DP1. The economic performance of GA+DP1 is outstanding: the performance of DP1 is better than that of DP10, and the performance of GA+DP1 is better than that of GA+DP10. Consequently, the combination of GA and DP leads to better solutions.

HS performed slightly worse than GA overall, however, as in GA, it was the same that combining with DP produced better results than a standalone method. The performances of HS+DP1 and HS+DP10 are better than that of HS.

We conducted a $t$-test to compare the performances of GA+DP1 and GA+DP10. The results have a significance level of 1.2 $e^{-1}$, that is, the performance of GA+DP10 is significantly similar to that of GA+DP1, but GA+DP10 is faster than GA+DP1, which shows that GA+DP10 is preferable for practical application when compared with GA+DP1.

4.3. Comparison for the Case with Demand Charge

Table 4. Comparison of cost savings of the proposed methods with varying base units including a demand charge.

Case	DP1	DP10	GA	GA+DP1	GA+DP10	HS	HS+DP1	HS+DP10
1	5.87%	5.76%	6.71% (0.25%)	6.75% (0.19%)	6.74% (0.02%)	5.94% (0.15%)	6.73% (0.05%)	6.61% (0.05%)
	(2.924)	(0.052)	(0.192)	(3.201)	(0.257)	(0.224)	(3.212)	(0.281)
2	4.38%	4.33%	4.51% (0.10%)	4.62% (0.15%)	4.65% (0.04%)	3.66% (0.12%)	4.64% (0.06%)	4.61% (0.04%)
	(2.854)	(0.048)	(0.184)	(3.433)	(0.262)	(0.237)	(3.238)	(0.243)
3	3.60%	3.60%	4.05% (0.09%)	4.08% (0.17%)	4.12% (0.03%)	3.58% (0.08%)	4.26% (0.10%)	4.25% (0.07%)
	(2.994)	(0.047)	(0.164)	(3.452v	(0.273)	(0.242)	(3.228)	(0.201)
4	2.22%	2.21%	2.42% (0.12%)	2.75% (0.20%)	2.77% (0.00%)	1.84% (0.10%)	2.30% (0.06%)	2.29% (0.08%)
	(3.014)	(0.051)	(0.171)	(3.321)	(0.281)	(0.204)	(3.216)	(0.239)
5	2.21%	2.19%	2.39% (0.14%)	2.70% (0.15%)	2.72% (0.01%)	1.83% (0.12%)	2.29% (0.08%)	2.28% (0.10%)
	(2.962)	(0.039)	(0.199)	(3.361)	(0.299)	(0.218)	(3.276)	(0.267)
6	2.42%	2.40%	2.62% (0.14%)	2.95% (0.25%)	2.97% (0.00%)	1.99% (0.09%)	2.52% (0.07%)	2.49% (0.15%)
	(2.94)	(0.043)	(0.201)	(3.399)	(0.244)	(0.291)	(3.268)	(0.227)
7	2.50%	2.50%	13.54% (0.13%)	13.20% (0.27%)	13.43% (0.00%)	9.71% (0.12%)	10.12% (0.12%)	10.29% (0.02%)
	(3.092)	(0.045)	(0.175)	(3.417)	(0.263)	(0.272)	(3.213)	(0.298)
8	12.30%	12.26%	12.32% (0.11%)	12.69% (0.24%)	12.76% (0.05%)	11.14% (0.15%)	13.48% (0.13%)	13.39% (0.05%)
	(2.885)	(0.038)	(0.161)	(3.363)	(0.251)	(0.271)	(3.232)	(0.277)
9	6.21%	6.15%	14.81% (0.08%)	14.61% (0.20%)	14.60% (0.04%)	10.88% (0.10%)	11.18% (0.04%)	11.10% (0.14%)
	(2.911)	(0.05)	(0.134)	(3.367)	(0.259)	(0.208)	(3.268)	(0.214)
10	5.18%	5.18%	4.28% (0.09%)	5.70% (0.24%)	5.72% (0.00%)	3.13% (0.13%)	5.19% (0.05%)	5.18% (0.07%)
	(2.923)	(0.055)	(0.156)	(3.393)	(0.262)	(0.275)	(3.252)	(0.227)
11	5.12%	5.12%	4.18% (0.10%)	5.58% (0.21%)	5.63% (0.00%)	3.21% (0.12%)	5.12% (0.07%)	5.12% (0.13%)
	(2.924)	(0.058)	(0.161)	(3.264)	(0.241)	(0.221)	(3.206)	(0.234)
12	7.73%	7.73%	7.92% (0.13%)	8.74% (0.18%)	8.51% (0.05%)	6.95% (0.11%)	7.98% (0.05%)	7.94% (0.09%)
	(2.975)	(0.052)	(0.183)	(3.251)	(0.268)	(0.258)	(3.214)	(0.242)
13	10.01%	7.79%	11.09% (0.15%)	12.26% (0.19%)	12.28% (0.06%)	8.14% (0.10%)	11.4% (0.10%)	9.63% (0.12%)
	(2.963)	(0.053)	(0.199)	(3.411)	(0.29)	(0.251)	(3.289	(0.271)
14	9.74%	11.57%	10.32% (0.14%)	11.69% (0.15%)	12.60% (0.04%)	7.38% (0.11%)	10.19% (0.14%)	11.63% (0.10%)
	(2.989)	(0.045)	(0.181)	(3.252v	(0.284)	(0.279)	(3.266)	(0.226)
15	11.16%	8.20%	13.06% (0.12%)	13.82% (0.21%)	13.48% (0.12%)	11.06% (0.13%)	12.28% (0.10%)	11.73% (0.04%)
	(2.931)	(0.047)	(0.179)	(3.245)	(0.285)	(0.249)	(3.24)	(0.274)
16	7.20%	7.20%	6.17% (0.15%)	8.38% (0.24%)	7.87% (0.00%)	4.63% (0.06%)	7.20% (0.07%)	7.21% (0.06%)
	(2.938)	(0.049)	(0.169)	(3.273)	(0.279)	(0.23)	(3.217)	(0.264)
17	7.16%	7.16%	6.11% (0.11%)	8.39% (0.16%)	7.83% (0.00%)	4.58% (0.10%)	7.16% (0.08%)	7.17% (0.08%)
	(2.946)	(0.045)	(0.185)	(3.293)	(0.268)	(0.263)	(3.279)	(0.276)
18	7.45%	7.45%	6.33% (0.13%)	8.54% (0.18%)	8.30% (0.00%)	4.72% (0.09%)	7.48% (0.06%)	7.45% (0.10%)
	(2.974)	(0.054)	(0.193)	(3.283)	(0.282)	(0.278)	(3.208)	(0.264)

For GA, GA+DP1, GA+DP10, HS, HS+DP1 and HS+DP10, average values and standard deviations over 100 runs are shown. The computing time in seconds is given in the second line of each cell. For each case, the best result among compared eight ones is shown in bold type.

and Figure 5 show the experimental results for the case including demand charge. In this case, the proposed DP methods (DP1 and DP10) optimize only the charge by the TOU but not the demand charge, which often results in bad performances. However, the combined methods of GA and DP can perform better than the individual DP or GA methods. The combined methods of HS and DP also performed better than DP, however, their results were slightly worse than the combined methods of GA and DP. The combined methods of GA and DP (GA+DP1 and GA+DP10) performed better than the GA for almost all cases except two ones. In Table 4, there does not seem to be much difference between GA+DP1 and GA+DP10. Statistically, the p-value obtained from $t$-test was 7.3 $e^{-1}$, which showed that the performance of GA+DP1 had no significant differences from that of GA+DP10.

In terms of the temporal performance, DP10 is faster than DP1 because DP10 searches fewer solutions than DP1. Computing times of GA and HS are similar, though GA obtained better results than HS. For this case with demand charge, GA and HS are not only faster than DP1, but also showed better performance in terms of cost savings. The results by GA and HS can be improved by combining DP, as shown by the results of GA+DP1, GA+DP10, HS+DP1, and HS+DP10. However, they are slower than a standalone of GA or HS method. GA+DP10 takes shorter time than GA+DP1. In the above, we have shown that the performance of GA+DP1 has no significant differences from that of GA+DP10 by $t$-test, so we can conclude that GA+DP10 is the most practical method among compared eight ones.

Although the performances of DP1 and DP10 are poor for this case with demand charge, GA+DP1 and GA+DP10 show relatively good performances. This means that the performance of DP can be improved by combining GA, which helps to optimize the demand charge while DP only optimizes energy charge with TOU but not the demand charge.

4.4. Experiments with Various Sizes of Base Unit

In the experiments in Section 4.2 and Section 4.3, we have only experimented the combined methods of GA and DP with only two kinds of base units, 1 kWh and 10 kWh. In this subsection, we have investigated the performance of the combined methods of GA and DP with various sizes of base unit, 0.5 kWh, 1 kWh, 10 kWh, 20 kWh, and 50 kWh. Figure 6 shows those results.

For the case without demand charge (Figure 6a,b), GA+DP1 yields the best cost saving. GA+DP1 is even better than GA+DP0.5 though the results of the two methods have a significance level of 3.0 $e^{-1}$ in $t$-test. Usually, in the design of DP, the smaller the size of the base unit, the better the performance. However, when DP is combined with GA, it is not definitely true. In our experiments, if the base unit is smaller than 1 kWh, the combined method could not show the improved performance. So, if we consider only cost saving, the use of 1 kWh as base unit will be a reasonable choice. However, as mentioned in Section 4.2, the results of GA+DP1 and GA+DP10 have a significance level of 1.2 $e^{-1}$ in $t$-test, which shows that the performance of GA+DP10 is significantly similar to that of GA+DP1, and GA+DP10 is much faster than GA+DP1. Hence, GA+DP10 is preferable for practical application in terms of both cost saving and time.

In the results with demand charge (Figure 6c,d), GA+DP0.5 is slightly better than GA+DP1, and GA+DP1 is also slightly better than GA+DP10. However, the differences are very small. In this case with demand charge, the effect of GA is larger than that of DP, so the quality of solutions is not so sensitive of the size of base unit. Nevertheless, the cost saving of GA+DP50 is surely worse than that of GA+DP20. From these observations, we could conclude that 10 kWh or 20 kWh base unit size is the most practical when we apply the combined method of GA and DP in the case with demand charge, because they spend much less time than GA+DP1 obtaining similar quality of solutions to GA+DP1.

4.5. Experiments with Combined Methods of Improved GA and DP

In the experiments in Section 4.1 and Section 4.2, the population size and the number of generations of the proposed GA have been set to be a fixed number, 100 and 100,000. However, in those experiments, the average running times of GA (≈0.3 s) and GA+DP1 (≈3.3 s) have no significant difference in terms of 24 h. Even though we spend more time to running of GA, there will be no problem in temporal performance. So, we did additional experiments with an improved GA by increasing the number of population and generations. For this improved GA, the population size is set to be 200 and the number of generations is set to be 1,000,000. Increasing those numbers further is meaningless because there is no improvement in the quality of the solution.

Figure 7 shows the results with this improved GA (IGA). For the case without demand charge (Figure 7a,b), the performance of IGA was improved compared with GA and it spends more time than GA about 10 times. However, IGA is not better than GA+DP10 even though it spends more time. So in this case without demand charge, we could conclude that the combination of GA and DP is more effective than using a standalone GA. For the case with demand charge (Figure 7c,d), the results showed a different pattern. The performance of IGA was also better than that of GA and, moreover, it is even better than that of GA+DP1 while IGA consumed less time than GA+DP1. That is, the effect of DP was not so much in this case when compared to that of GA. However, IGA+DP1 and IGA+DP10 showed better performances than IGA. IGA still can be more improved by combining DP. In particular, IGA+DP10 spent not so much time when compared to IGA, so we can conclude that IGA+DP10 is the most practical among the compared methods in this case.

5. Conclusions

In this study, we proposed a combined method of GA and DP for ESS scheduling problems with and without demand charge. Because the temporal performance of DP degrades as the ESS capacity increases, we improved the performance of the combined method by adjusting the size of the base unit in DP.

Without demand charge, DP with a small base unit (1 kWh) showed good economic performance, but its temporal performance was lower than that of DP with a large base unit (10 kWh). The temporal performance of DP with a large base unit was excellent, but its economic performance did not match DP with a small base unit. GA also has a disadvantage of not producing good solutions. Therefore, we improved both the temporal and economic performances by combining GA and DP with a large base unit. Through experiments with various sizes of base units, we could observe that the combined method could not show further improved performance if the base unit is smaller than 1 kWh. Hence, the combined method of GA and DP with the base unit smaller than 1 kWh is not efficient. Moreover, the experimental results showed that GA+DP1 and GA+DP10 had significantly similar economic performances through t-test, and GA+DP10 is much faster than GA+DP1. Therefore, we could conclude that GA+DP10 was the most practical among the compared methods in terms of both economic and temporal performances in the case without demand charge.

When demand charge was included, we could observe that the economic performance of DP significantly degraded through experiments. By combining GA and DP, the advantages of both methods can be utilized, that is, that of GA, which optimizes the demand charge, and that of DP, which optimizes the energy charge. In this case, the combined method of GA and DP with a small base unit showed better economic performance than that with a large base unit. However, the $p$-value obtained from $t$-test between the two methods was larger than 0.05, which means that it is not sure that their performances differ. Actually, we could observe that the quality of solutions is not so sensitive of the size of base unit in the case with demand charge through experiments with various sizes of base units. The experimental results showed that the proposed combined method of GA and DP with a base unit 10 kWh or 20 kWh could not only save computing time but also find good solutions when compared with the same method with other base units.

Our study also has some limitations in that we did not consider the cost of battery cycling. For more practical applications, the advanced research considering battery cycling might be required as future work.