Optimal Energy Management for Microgrids with Combined Heat and Power (CHP) Generation, Energy Storages, and Renewable Energy Sources

Abstract

This paper studies an energy management problem for a typical grid-connected microgrid system that consists of renewable energy sources, Combined Heat and Power (CHP) co-generation, and energy storages to satisfy electricity and heat demand simultaneously. We formulate this problem into a stochastic non-convex optimization programming to achieve the minimum microgrid’s operating cost, which is difficult to solve due to its non-convexity and coupling feature of constraints. Existing approaches such as dynamic programming (DP) assume that all the system dynamics are known, which results in a high computational complexity and thus are not feasible in practice. The focus of this paper is on the design of a real-time energy management strategy for the optimal operation of microgrids with low computational complexity. Specifically, derived from a modified Lyapunov optimization technique, an online algorithm with random inputs (e.g., the charging/discharging of energy storage devices, power from the CHP system, the electricity from external power grid, and the renewables generation, etc.), which requires no statistic system information, is proposed. We provide an implementation of the proposed energy management algorithm and prove its optimality theoretically. Based on real-world data traces, extensive empirical evaluations are presented to verify the performance of our algorithm.

1. Introduction

Microgrids stand a good chance of becoming a future power grid paradigm that uses centralized power grids as well as local generated energy [1]. They can be operated with or without a grid connection. Microgrids usually consist of distributed renewable energy, decentralized energy storage devices (e.g., PHEVs), a local CHP System (e.g., gas-fired generators), and flexible loads.

With environmental concerns growing, a future power grid is expected to integrate more renewable energy (e.g., solar or wind) to reduce the discharge of greenhouse gas. For instance, the European Commission intends to include 20% renewables into the EU energy profile by 2020 [2], and California aims to get 33% of retail sales from renewables by 2020 [3]. As we know, the generation of renewable energy is intermittent and non-dispatchable. If we simply integrate large amounts of renewable energy, the system will encounter some reliability problems. Besides, renewable energy supply is a stochastic process, which brings a new dimension of uncertainty to energy management. Therefore, how to integrate the generation of renewables efficiently and ensure the reliability of our system simultaneously is of great importance for microgrids.

Energy storage devices are utilized to smooth energy fluctuations and reduce the system cost in a more environmentally friendly way by intelligent charging/discharging, which plays an important role in microgrids [4,5,6,7]. Apart from its power management capability, energy storage devices can act as a backup in microgrids when the external grid breaks down, which will reduce the negative effects with a quick response [8,9]. The distributed storage plays a significant role in the design and evolution of a power grid, and particularly increases additional design choices for reducing the operating cost of microgrids [10,11,12]. The hybird energy storage system is considered for primary frequency control using a dynamic droop method in an isolated microgrid power system [13]. Online energy management algorithms are developed to investigate the operating cost reduction for microgrids with an energy storage system [14].

Apart from renewables generation and energy storage devices, CHP systems are becoming very popular in the microgrids industry [15,16]. CHP systems can generate both electricity and thermal energy simultaneously, which can achieve a much higher energy efficiency than generating electricity and heat separately [17]. The characteristics of local generations and local consumptions of micrigrid make it more flexible in the utilization of renewable energy and CHP generation, which extends the adaptability of a traditional centralized grid. The power management strategy between different elements should be considered in order to design feasible control algorithms for microgrid systems [18]. Furthermore, with the augmentation of CHP generation technology, microgrids can often be much more economical than the traditional grid by using centralized grid supply and separate heat supply [19,20]. The integration analysis of hybrid energy storage system and novel CHP systems in residential scenarios are also investigated [21].

In this paper, we consider the grid connected microgrid. We aim to propose an intelligent scheduling action (e.g., charging/discharging of energy storage device, power drawn from centralized grid, power obtained from local generator, etc.) to achieve the operating minimum cost of microgrids while considering all the random inputs of the system. We first formulate the problem of achieving the minimum operating cost in microgrids as a stochastic non-convex programming. Considering that the dependence between power level of the battery pack and heat level of the water tank leads to this problem’s non-convexity, we study the relationship between them and convert it into stochastic convex optimization programming. Then we adopt the Lyapunov optimization [22] approach to design an online algorithm of some random system inputs (e.g., the charging/discharging of the energy storage devices, power from the local generator, the electricity from the power grid, and the renewable energy generation from different sources, etc.), which requires no statistic information of our system.

The contributions of this paper are summarized as follows:

• We formulate a stochastic non-convex programming for the online scheduling problem to minimize the microgrid’s cost, which captures the randomness in stochastic renewables, power and heat demands, charge level of energy storage, co-generation and physical constraints as well.
• To solve this stochastic non-convex optimization problem, we convert it into the subproblem with convex property. Then we design an online algorithm to reduce the operating cost of microgrids by using the Lyapunov optimization approach which relies on no future knowledge about the system inputs with stochastic distribution. In this way, we can get the optimal average cost.
• Through our evaluations by using practical data traces, we can see that by the proposed algorithm, we can achieve an approving empirical optimality ratio.

2. System Model and Problem Statement

The following components are typically included in our designed system: centralized power grid (supply power to the electricity load and charge the battery in an on-site way), large capacity battery (power energy storage), local co-generator (generate both heat and power energy simultaneously), external gas station (supply heat demand), thermal storage device (heat energy storage), and renewables generation (e.g., wind or solar). The system model is shown in Figure 1. For convenience of analysis, we assume that the system operates in discrete time with time slot $t\in \{0,1,2,\dots \}$. We then divide the time slots into frames of size T. Figure 2 gives the time structure of slots and frames. Let $T_{\mathrm{m}}$ denote the set of time slots in time frame m, i.e., $T_{\mathrm{m}}\triangleq \{\mathrm{m}T,\dots ,(\mathrm{m}+1)T-1\}$. This structure of time slot and time frame is defined to illustrate the time scales of the system operation for easy theoretical analysis. Therefore, we have two time scales in the system.

2.1. System Model

(1) Local co-generation (CHP): We assume that electricity and heat energy can be generated simultaneously by our local generator. Here we use an idealized model, we will investigate a more practical CHP model in our future work. ${\eta }_{\mathrm{ce}}$ and ${\eta }_{\mathrm{ch}}$ are the conversion efficiencies from fuel to the electricity and thermal energy, respectively. At each time slot t, the co-generator generates electricity and thermal energy, whose amounts are denoted as ${\eta }_{\mathrm{ce}}{P}_{\mathrm{c}}\left(t\right)$ and ${\eta }_{\mathrm{ch}}{P}_{\mathrm{c}}\left(t\right)$, respectively. The generated electricity can be used for power supply to directly satisfy the net power demand $(1-r\left(t\right)){\eta }_{\mathrm{ce}}{P}_{\mathrm{c}}\left(t\right)$ or be charged into the battery $r\left(t\right){\eta }_{\mathrm{ce}}{P}_{\mathrm{c}}\left(t\right)$. Similarly, the generated thermal energy ${\eta }_{\mathrm{ch}}{P}_{\mathrm{c}}\left(t\right)$ can be used for direct heat supply for users’ heat demand $(1-u\left(t\right)){\eta }_{\mathrm{ch}}{P}_{\mathrm{c}}\left(t\right)$ or be charged into the thermal tank $u\left(t\right){\eta }_{\mathrm{ch}}{P}_{\mathrm{c}}\left(t\right)$, respectively. $y\left(m_t\right)$ represents the on/off decision of the generator: $y\left(m_t\right)=1$ represents switching on and $y\left(m_t\right)=0$ denotes switching off in frame $m_t$, which $m_t=[t/T]+1$ is defined as the number of slots in a frame.

(2) Centralized power grid: We assume that the power grid and microgrid are connected. The power can be acquired from the centralized power grid in an on-demand manner to meet electricity demands. The system obtains power in the amount of $G_{\mathrm{l}}\left(t\right)$ for satisfying demands directly and the power in the amount of $G_{\mathrm{s}}\left(t\right)$ for charging the battery, respectively. $G_{\mathrm{l},\max }$ are defined as the upper bound of direct power supply from external power grid and $G_{\mathrm{s},\max }$ denotes the upper bound of charging power for the battery from the external power grid, respectively. Then we have $0\le G_{\mathrm{l}}\left(t\right)+G_{\mathrm{s}}\left(t\right)\le G_{\max }$ and $0\le G_{\mathrm{l}}\left(t\right)\le G_{\mathrm{l},\max }$, $0\le G_{\mathrm{s}}\left(t\right)\le G_{\mathrm{s},\max }$. Supposing that the power demand can be satisfied by power grid alone, we assume that $L_{\mathrm{e},\max }\le G_{\max }$ holds at any time slot, where $L_{\mathrm{e},\max }$ is the upper bound of $L_{\mathrm{e}}\left(t\right)$.

(3) External gas station: The heat energy from the external gas station, ${\eta }_{\mathrm{ah}}{P}_{\mathrm{a}}\left(t\right)$, can be used for direct heat supply and energy charging of the thermal tank. Using $v\left(t\right)$ to denote the fraction for charging, we denote the amount for heat supply and heat charging as $(1-v\left(t\right)){\eta }_{\mathrm{ah}}{P}_{\mathrm{a}}\left(t\right)$ and $v\left(t\right){\eta }_{\mathrm{ah}}{P}_{\mathrm{a}}\left(t\right)$, respectively. Under the online control algorithm that we will propose later, the heat demand can be satisfied with the energy from the co-generation and gas station while the total cost can be minimized in an intelligent way which schedules the energy properly.

(4) Renewable energy: Let $R\left(t\right)$ denote the renewable energy harvested at time t. In our model, the renewable energy is used as electricity supply for users first because it is free. If we have excess renewable energy when the power demand has been satisfied, we use this part of energy, which is called $R_{\mathrm{c}}\left(t\right)$ to charge into the battery. In addition, the amount of renewable energy harvested in a time slot is bounded, and thus we have $0\le R\left(t\right)\le R_{\max },\forall t\in T$. The excess renewable power that is charged to the battery cannot exceed the total amount of harvested renewable energy. Therefore, we have $0\le R_{\mathrm{c}}\left(t\right)\le {[-L_{\mathrm{e}}\left(t\right)]}^{+}$, where ${[-L_{\mathrm{e}}\left(t\right)]}^{+}=\max \{R\left(t\right)-L_{\mathrm{e}}\left(t\right),0\}$. Note that, although the system model we consider in this paper only involves the electricity renewable energy, heat renewable energy is applicable as well.

(5) Power and heat demands: In our microgrid system, the total demand includes the demand for power and heat. $L_{\mathrm{e}}\left(t\right)$ represents power demand at time slot t, which must be satisfied once requested. The net power demand ${\left[L_{\mathrm{e}}\left(t\right)\right]}^{+}$, which is the excess of power demand over renewable energy at time slot t, equals the subtraction of power demand and renewable energy, and can be expressed as ${\left[L_{\mathrm{e}}\left(t\right)\right]}^{+}=\max \{L_{\mathrm{e}}\left(t\right)-R\left(t\right),0\}$. Let $L_{\mathrm{e},\max }$ denote the maximum net power demand in a time slot, then we have $0\le {\left[L_{\mathrm{e}}\left(t\right)\right]}^{+}\le L_{\mathrm{e},\max }$. The power can be acquired from power grid, local co-generator as well as the battery, denoted as $G_l\left(t\right)$, $(1-r\left(t\right)){\eta }_{\mathrm{ce}}{P}_{\mathrm{c}}\left(t\right)$ and $D\left(t\right)$ respectively, to balance ${\left[L_{\mathrm{e}}\left(t\right)\right]}^{+}$. It can be presented as follows:

$${\left[L_{\mathrm{e}}\left(t\right)\right]}^{+}=G_{\mathrm{l}}\left(t\right)+D\left(t\right)+(1-r\left(t\right)){\eta }_{\mathrm{ce}}{P}_{\mathrm{c}}\left(t\right)y\left(m_t\right)$$

(1)

Similarly, the heat can be acquired from external natural gas station, co-generation as well as the thermal tank, denoted as $(1-v\left(t\right)){\eta }_{\mathrm{ah}}{P}_{\mathrm{a}}\left(t\right)$, $(1-u\left(t\right)){\eta }_{\mathrm{ch}}{P}_{\mathrm{c}}\left(t\right)$ and $W\left(t\right)$ (more details about $W\left(t\right)$ can be found in the thermal tank model) respectively, to balance the heat demand. Thus, at every time slot, we have:

$$L_{\mathrm{w}}\left(t\right)\le (1-v\left(t\right)){\eta }_{\mathrm{ah}}{P}_{\mathrm{a}}\left(t\right)+(1-u\left(t\right)){\eta }_{\mathrm{ch}}{P}_{\mathrm{c}}\left(t\right)y\left(m_t\right)+W\left(t\right)$$

(2)

Let $L_{\mathrm{w},\max }$ denote the maximum heat demand in a time slot. An additional constraint $L_{\mathrm{w},\max }\le {\eta }_{\mathrm{ah}}{P}_{\mathrm{a},\max }$ has to be added to assure the balance of heat demand and supply at any time slot, where $P_{\mathrm{a},\max }$ is the maximum heat output of the external gas station. Here ${\eta }_{\mathrm{ah}}$ are defined as the conversion efficiencies from gas to the thermal energy. Let parameter $r\left(t\right)$ denote the fraction of co-generated power that is used for charging. Then $(1-r(t\left)\right)$ denotes the fraction of co-generated power that is used for direct power supply. $u\left(t\right)$ is defined as the dispatch ratio from CHP to thermal tank, $v\left(t\right)$ denotes the dispatch ratio from thermal source to thermal tank. It should be noted that any stochastic information of the net power demands and heat demands is not required in our proposed algorithm. Here we use “≤” instead of “=” to insure mathematical rigorous. Actually, it could be “=” for both Equations (1) and (2) in our optimization problem. However, in the operation of the optimization problem, it should be consider the feasibility of the mathematical solution. In the algorithm design point of view, there is no difference for “=” and “≤” for Equation (2). We pointed out that if we change “≤” to “=” in Equation (2), the solution is the same. In fact, we can also change Equation (1) into “≤” and Equation (2) into “=” in the problem formulation. It has equivalent solutions for the two optimization problem.

2.2. Battery Model and Thermal Tank Model

(1) Battery model: The dynamics of battery’s state of charge (SOC) level $B\left(t\right)$ is given as follows:

$$B(t+1)=B\left(t\right)-{\eta }_{\mathrm{d}}D\left(t\right)+{\eta }_{\mathrm{c}}[R_{\mathrm{c}}\left(t\right)+G_{\mathrm{s}}\left(t\right)+r\left(\mathrm{t}\right){\eta }_{\mathrm{ce}}{P}_{\mathrm{c}}\left(t\right)y\left(m_t\right)]$$

(3)

where ${\eta }_{\mathrm{d}}$ stands for discharging efficiency of battery and ${\eta }_{\mathrm{c}}$ denotes the charging efficiency of it. We can find that the battery must satisfy constraints of capacity and charge/discharge in any slot t.

$$0\le B\left(t\right)\le B_{\max },0\le D\left(t\right)\le D_{\max },B\left(t\right)·D\left(t\right)=0$$

(4)

$$0\le G_{\mathrm{s}}\left(t\right)+r\left(t\right){\eta }_{\mathrm{ce}}{P}_{\mathrm{c}}\left(t\right)y\left(m_t\right)+R_{\mathrm{c}}\left(t\right)\le T{C}_{\mathrm{char}}$$

(5)

where $B_{\max }$ is the capacity of the battery and $D_{\max }$ is the maximum discharging power of the battery in each time slot and $T{C}_{\mathrm{char}}$ is the maximum charging power of the battery in each frame.

(2) Thermal tank model: We utilize a thermal tank to store the excess heat for later use. With the charging and discharging of tank at each time slot, the heat state evolves over time:

$$T(t+1)=T\left(t\right)-{\eta }_{\beta }W\left(t\right)+{\eta }_{\alpha }[u\left(t\right){\eta }_{\mathrm{ch}}{P}_{\mathrm{c}}\left(t\right)y\left(m_t\right)+v\left(t\right){\eta }_{\mathrm{ah}}{P}_{\mathrm{a}}\left(t\right)]$$

(6)

where $T\left(t\right)$ is the thermal tank’s heat energy state at time slot t. Because heat energy stored in the thermal tank can not exceed the capacity of thermal tank, we have:

$$0\le T\left(t\right)\le T_{\max },0\le W\left(t\right)\le W_{\max },T\left(t\right)·W\left(t\right)=0$$

(7)

$$0\le v\left(t\right){\eta }_{\mathrm{ah}}{P}_{\mathrm{a}}\left(t\right)+u\left(t\right){\eta }_{\mathrm{ch}}{P}_{\mathrm{c}}\left(t\right)y\left(m_t\right)\le T{h}_{\mathrm{char}}·T$$

(8)

Similarly, $T_{\max }$ is the upper bound of the thermal tank and $W_{\max }$ represents the discharging rate constraint of the thermal tank.

2.3. Problem Statement

System State and Constraint: According to the components described in our system, we define the system state as a state vector $Q_{\mathrm{t}}$:

$$Q_{\mathrm{t}}\triangleq [L_{\mathrm{e}}\left(t\right),L_{\mathrm{w}}\left(t\right),R\left(t\right),C\left(t\right),B\left(t\right),T\left(t\right)]$$

(9)

We assume that $Q_{\mathrm{t}}$ is an i.i.d. process over time. Although some of the elements in $Q_{\mathrm{t}}$ can be arbitrarily correlated, the control decisions at each time slot only depends on current system state $Q_{\mathrm{t}}$ without any future system information.

Through jointly scheduling the power and heat energy storage, centralized power grid, the renewables, and co-generation, our system can realize the goal of minimizing the long-term time-averaged operating cost. In particular, the control vector at time slot t isdefined by:

$$U_{\mathrm{t}}\triangleq [G_{\mathrm{l}}\left(t\right),G_{\mathrm{s}}\left(t\right),P_{\mathrm{c}}\left(t\right),P_{\mathrm{a}}\left(t\right),R_{\mathrm{c}}\left(t\right),r\left(t\right),u\left(t\right),v\left(t\right)]$$

(10)

The total cost of our system includes the cost of power acquired from external power grid, the fuel consumption of the co-generation, and natural gas for generating heat, switching and sunk cost:

$$f\left(t\right)=C\left(t\right)[G_{\mathrm{l}}\left(t\right)+G_{\mathrm{s}}\left(t\right)]+C_{\mathrm{f}}{P}_{\mathrm{c}}\left(t\right)y\left(m_t\right)+C_{\mathrm{g}}{P}_{\mathrm{a}}\left(t\right)+C_{\mathrm{m}}y\left(m_t\right)$$

(11)

We denote the real-time electricity price of power grid as $C\left(t\right)$, which is bounded by $C_{\min }$ and $C_{\max }$. $C_{\min }$ and $C_{\max }$ is the minimum and maximum electricity price. So we have $C_{\min }\le C\left(t\right)\le C_{\max }$. It should be noticed that although $C\left(t\right)$ can also be a stochastic process, the statistics will not be depended in our algorithm. In this paper, we set the fuel price $C_{\mathrm{f}}$ and the price of natural gas $C_{\mathrm{g}}$ to be constants at each time slot. Actually, our algorithm is also available in the case that the fuel price and natural gas price are not fixed since our algorithm is based on the current system state which can be known at each time slot.

So far we can formulate our first optimization problem as follows:

$$\begin{array}{cc}\mathrm{P}1:\hfill & \min \underset{T\to \infty }{\lim }\frac{1}{T}\sum _{i=0}^{T-1}\mathbb{E}\{f\left(t\right)\}\hfill \\ \hfill & \mathrm{subject}\mathrm{to}\hfill \end{array}$$

(12)

$$\begin{array}{c}{\left[L_{\mathrm{e}}\left(t\right)\right]}^{+}=G_{\mathrm{l}}\left(t\right)+D\left(t\right)+(1-r\left(t\right)){\eta }_{\mathrm{ce}}{P}_{\mathrm{c}}\left(t\right)y\left(m_t\right)\hfill \end{array}$$

(13)

$$\begin{array}{c}{L}_{\mathrm{w}}\left(t\right)\le (1-v\left(t\right)){\eta }_{\mathrm{ah}}{P}_{\mathrm{a}}\left(t\right)+(1-u\left(t\right)){\eta }_{\mathrm{ch}}{P}_{\mathrm{c}}\left(t\right)y\left(m_t\right)+W\left(t\right)\hfill \end{array}$$

(14)

$$\begin{array}{c}B(t+1)=B\left(t\right)-{\eta }_{\mathrm{d}}D\left(t\right)+{\eta }_{\mathrm{c}}[R_{\mathrm{c}}\left(t\right)+G_{\mathrm{s}}\left(t\right)+r\left(t\right){\eta }_{\mathrm{ce}}{P}_{\mathrm{c}}\left(t\right)y\left(m_t\right)]\hfill \end{array}$$

(15)

$$\begin{array}{c}T(t+1)=T\left(t\right)-{\eta }_{\beta }W\left(t\right)+{\eta }_{\alpha }[u\left(t\right){\eta }_{\mathrm{ch}}{P}_{\mathrm{c}}\left(t\right)y\left(m_t\right)+v\left(t\right){\eta }_{\mathrm{ah}}{P}_{\mathrm{a}}\left(t\right)]\hfill \end{array}$$

(16)

$$\begin{array}{c}0\le B\left(t\right)\le B_{\max },0\le T\left(t\right)\le T_{\max }\hfill \end{array}$$

(17)

$$\begin{array}{c}0\le G_{\mathrm{s}}\left(t\right)+r\left(t\right){\eta }_{\mathrm{ce}}{P}_{\mathrm{c}}\left(t\right)y\left(m_t\right)+R_{\mathrm{c}}\left(t\right)\le C_{\mathrm{char}}\hfill \end{array}$$

(18)

$$\begin{array}{c}0\le v\left(t\right){\eta }_{\mathrm{ah}}{P}_{\mathrm{a}}\left(t\right)+u\left(t\right){\eta }_{\mathrm{ch}}{P}_{\mathrm{c}}\left(t\right)y\left(m_t\right)\le T{h}_{\mathrm{char}}\hfill \end{array}$$

(19)

$$\begin{array}{c}0\le u\left(t\right)\le 1,0\le v\left(t\right)\le 1\hfill \end{array}$$

(20)

$$\begin{array}{c}0\le D\left(t\right)\le D_{\max },0\le W\left(t\right)\le W_{\max }\hfill \end{array}$$

(21)

$$\begin{array}{c}{G}_{\mathrm{l}}\left(t\right),G_{\mathrm{s}}\left(t\right),P_{\mathrm{c}}\left(t\right),P_{\mathrm{a}}\left(t\right),R_{\mathrm{c}}\left(t\right),r\left(t\right),u\left(t\right),v\left(t\right)\ge 0\hfill \end{array}$$

(22)

At the beginning of each frame, the local generator make a decision on choosing the on/off statement by solving a mixed-integer stochastic optimization program with constraints. We then jointly decide other components ($G_{\mathrm{l}}\left(t\right)$, $G_{\mathrm{s}}\left(t\right)$, $P_{\mathrm{c}}\left(t\right)$, $P_{\mathrm{a}}\left(t\right)$, $R_{\mathrm{c}}\left(t\right)$, $r\left(t\right)$, $u\left(t\right)$, $v\left(t\right)$) in each time slot.

Solving $\mathrm{P}1$ is challenging. In this paper, we aim to develop an online algorithm which requires no system statistics and is easy to implement.

3. The Co-Generation System Scheduling Algorithm

From the above, we know that $\mathrm{P}1$ is a challenge to solve by the current algorithm due to the non-convex optimization. However, we have found a feasible method to work out a convex optimization problem already. Therefore, in this section, we will change $\mathrm{P}1$ into a convex optimization problem. It is a real-time algorithm derived from the two-timescale Lyapunov optimization techniques [23].

3.1. Problem Relaxation

Stochastic optimization framework guarantees the balance of average energy consumption and average energy generation in the long term; however, it can not provide their hard bounds in any time slot. Thus, the problem above cannot be settled directly through stochastic optimization framework under those circumstances (17). To solve the problem, we try to take expectation on both sides of (15) and (16), which leads to P2 as follows:

$$\begin{array}{c}{\displaystyle \mathrm{P}2:\underset{U_{\mathrm{t}}}{\min }\underset{T\to \infty }{\lim }\frac{1}{T}\sum _{i=0}^{T-1}\mathbb{E}\{f\left(t\right)\}}\hfill \end{array}$$

(23)

$$\begin{array}{cc}{\displaystyle \mathrm{s}.\mathrm{t}.}\hfill & \overline{D\left(t\right)}={\eta }_{\mathrm{c}}[\overline{R_{\mathrm{c}}\left(t\right)}+\overline{G_{\mathrm{s}}\left(t\right)}+{\eta }_{\mathrm{ce}}\overline{r\left(t\right)P_{\mathrm{c}}\left(t\right)y\left(m_t\right)}]\hfill \end{array}$$

(24)

$$\begin{array}{c}\overline{W\left(t\right)}={\eta }_{\alpha }[{\eta }_{\mathrm{ch}}\overline{u\left(t\right)P_{\mathrm{c}}\left(t\right)y\left(m_t\right)}+{\eta }_{\mathrm{ah}}\overline{v\left(t\right)P_{\mathrm{a}}\left(t\right)}]\hfill \\ \left(13\right),\left(14\right),\left(18\right),\left(19\right),\left(20\right),\left(21\right),\left(22\right).\hfill \end{array}$$

(25)

After those operations, we finally obtain $\mathrm{P}2$, which fits the stochastic optimization framework. $\mathrm{P}2$ extends the limitation of Battery and Thermal tank storage. It no longer restricts the value of $B\left(t\right)$ and $T\left(t\right)$ in each time slot instead of restricting them in the whole process. Under the condition that the solutions must satisfy constraint (17) at each time slot, the framework is feasible to $\mathrm{P}1$. As long as we define these two constants suitably, solutions to $\mathrm{P}2$ can also be feasible solutions to $\mathrm{P}1$.

3.2. Online Algorithm

To simplify the following discussion, the virtual queues $E\left(t\right)$ and $X\left(t\right)$ are respectively defined for the battery and thermal tank as follows:

$$E\left(t\right)=B\left(t\right)-\theta$$

(26)

$$X\left(t\right)=T\left(t\right)-\epsilon$$

(27)

where $\theta$ and $\epsilon$ are two perturbation parameters, which are time-independent constants and will be specified later.

Then the queueing dynamics (15) and (16) can be transformed into:

$$E(t+1)=E\left(t\right)-{\eta }_{\mathrm{d}}D\left(t\right)+{\eta }_{\mathrm{c}}[R_{\mathrm{c}}\left(t\right)+G_{\mathrm{s}}\left(t\right)+{\eta }_{\mathrm{ce}}r\left(t\right)P_{\mathrm{c}}\left(t\right)y\left(m_t\right)]$$

(28)

$$X(t+1)=X\left(t\right)-{\eta }_{\beta }W\left(t\right)+{\eta }_{\alpha }[{\eta }_{\mathrm{ah}}v\left(t\right)P_{\mathrm{a}}\left(t\right)+{\eta }_{\mathrm{ch}}u\left(t\right)P_{\mathrm{c}}\left(t\right)y\left(m_t\right)]$$

(29)

In addition, the Lyapunov function is defined to be: $Q\left(t\right)=\frac{1}{2}{\left[E\left(t\right)\right]}^2+\frac{1}{2}{\left[X\left(t\right)\right]}^2$. Then the T-slot conditional Lyapunov drift can be defined as follows:

$$\mathsf{\Delta }\left(t\right)=E\{Q(t+T)-Q(t\left)\right|\left(E\right(t),X(t\left)\right)\}$$

(30)

Consider any $\tau \in [t,\dots ,t+T-1]$, squaring both sides of (28) and (29). Considering the result in one time slot after carrying out sub calculations, we can obtain:

$$\begin{array}{cc}Q(t+1)-Q\left(t\right)\hfill & =0.5\max \{{\eta }_{\mathrm{d}}^2{D}_{\max }^2,{\eta }_{\mathrm{c}}^2{[R_{\mathrm{c},\max }+G_{\mathrm{s},\max }+{\eta }_{\mathrm{ce}}{P}_{\mathrm{c},\max }y\left(m_t\right)]}^2\}\hfill \\ & -E\left(t\right)\{{\eta }_{\mathrm{d}}D\left(t\right)-{\eta }_{\mathrm{c}}[R_{\mathrm{c}}\left(t\right)+G_s\left(t\right)+r\left(t\right){\eta }_{\mathrm{ce}}{P}_{\mathrm{c}}\left(t\right)\mathrm{y}\left(m_t\right)]\}\hfill \\ & +0.5\max \{{\eta }_{\beta }^2{W}_{\max }^2,{\eta }_{\alpha }^2{[{\eta }_{\mathrm{ch}}{P}_{\mathrm{c},\max }y\left(m_t\right)+{\eta }_{\mathrm{ah}}{P}_{\mathrm{a},\max }]}^2]\}\hfill \\ & -X\left(t\right)\{{\eta }_{\beta }W\left(t\right)-{\eta }_{\alpha }[u\left(t\right){\eta }_{\mathrm{ch}}{P}_{\mathrm{c}}\left(t\right)y\left(m_t\right)+v\left(t\right){\eta }_{\mathrm{ah}}{P}_{\mathrm{a}}\left(t\right)]\}\hfill \end{array}$$

(31)

In each time slot, the CHP consume the fuel while the thermal source consumes the gas. The maximum amount of them are $P_{\mathrm{c},\max }$ and $P_{\mathrm{a},\max }$ separately. Similarly, $R_{\mathrm{c},\max }$ and $G_{\mathrm{s},\max }$ denote the maximum charging power from the renewable energy resource and the external power grid, respectively. We define B as: $B=0.5\max \{{\eta }_{\mathrm{d}}^2{D}_{\max }^2,{\eta }_{\mathrm{c}}^2{[R_{\mathrm{c},\max }+G_{\mathrm{s},\max }+{\eta }_{\mathrm{ce}}{P}_{\mathrm{c},\max }y\left(m_t\right)]}^2\}+0.5\max \{{\eta }_{\beta }^2{W}_{\max }^2,{\eta }_{\alpha }^2{[{\eta }_{\mathrm{ch}}{P}_{\mathrm{c},\max }y\left(m_t\right)+{\eta }_{\mathrm{ah}}{P}_{\mathrm{a},\max }]}^2]\}$.

Summing (31) over $\tau \in [t,\dots ,t+T-1]$ and taking the expectation conditional on $E\left(t\right)$ and $X\left(t\right)$ yields:

$$\begin{array}{cc}\mathsf{\Delta }\left(t\right)\le BT\hfill & -\mathbb{E}\{\sum _{\tau =t}^{t+T-1}E\left(\tau \right)[{\eta }_{\mathrm{d}}D\left(\tau \right)-{\eta }_{\mathrm{c}}(R_{\mathrm{c}}\left(\tau \right)+G_{\mathrm{s}}\left(\tau \right)+r\left(\tau \right){\eta }_{\mathrm{ce}}{P}_{\mathrm{c}}\left(\tau \right)y\left(m_{\tau }\right))]\}\hfill \\ & +\mathbb{E}\{\sum _{\tau =t}^{t+T-1}X\left(\tau \right)[{\eta }_{\beta }W\left(\tau \right)-{\eta }_{\alpha }[{\eta }_{\mathrm{ch}}u\left(\tau \right)P_{\mathrm{c}}\left(\tau \right)y\left(m_{\tau }\right)-{\eta }_{\mathrm{ah}}v\left(t\right)P_{\mathrm{a}}\left(\tau \right)]\}\hfill \end{array}$$

(32)

For the purpose of keeping $E\left(t\right)$ and $X\left(t\right)$ stable under the stochastic optimization framework, we should minimize the right-hand side of (32). Beyond that, the goal of our control algorithm is to minimize the system energy cost. Accordingly, we set parameter V to denote the tradeoff between energy storage and consumption and the drift-plus-penalty function is defined as follows:

$$\begin{array}{cc}\mathsf{\Delta }\left(t\right)+V\mathbb{E}\{f(t)\}\le BT\hfill & -\mathbb{E}\{\sum _{\tau =t}^{t+T-1}E\left(\tau \right)[{\eta }_{\mathrm{d}}D\left(\tau \right)-{\eta }_{\mathrm{c}}(R_{\mathrm{c}}\left(\tau \right)+G_{\mathrm{s}}\left(\tau \right)+r\left(\tau \right){\eta }_{\mathrm{ce}}{P}_{\mathrm{c}}\left(\tau \right)y\left(m_{\tau }\right))]\}\hfill \\ & +\mathbb{E}\{\sum _{\tau =t}^{t+T-1}X\left(\tau \right)[{\eta }_{\beta }W\left(\tau \right)-{\eta }_{\alpha }[{\eta }_{\mathrm{ch}}u\left(\tau \right)P_{\mathrm{c}}\left(\tau \right)y\left(m_{\tau }\right)-{\eta }_{\mathrm{ah}}v\left(t\right)P_{\mathrm{a}}\left(\tau \right)]\}\hfill \\ & +V\mathbb{E}\{C\left(t\right)[G_{\mathrm{l}}\left(t\right)+G_{\mathrm{s}}\left(t\right)+C_{\mathrm{g}}{P}_{\mathrm{a}}\left(t\right)]+C_{\mathrm{f}}{P}_{\mathrm{c}}\left(t\right)y\left(m_t\right)+C_{\mathrm{m}}y\left({\mathrm{m}}_{\mathrm{t}}\right)\}\hfill \end{array}$$

(33)

Replacing $G_{\mathrm{l}}\left(t\right)$ in (33) use $G_{\mathrm{l}}\left(t\right)={\left[L_{\mathrm{e}}\left(t\right)\right]}^{+}-D\left(t\right)-(1-r\left(t\right)){\eta }_{\mathrm{ce}}{P}_{\mathrm{c}}\left(t\right)y\left(m_t\right)$. In order to facilitate the algorithm, we conduct some manipulation and get the formula:

$$\begin{array}{cc}\mathsf{\Delta }\left(t\right)\hfill & +V\mathbb{E}\{f\left(t\right)\}\le BT+V\mathbb{E}\{C\left(t\right){\left[L_{\mathrm{e}}\left(t\right)\right]}^{+}|E\left(t\right)\}+V\mathbb{E}\{C_{\mathrm{m}}y\left(m_t\right)\}\hfill \\ & +\mathbb{E}\{E\left(t\right){\eta }_{\mathrm{c}}{R}_{\mathrm{c}}\left(t\right)|E\left(t\right)\}-\mathbb{E}\{D\left(t\right)[E\left(t\right){\eta }_{\mathrm{d}}+VC\left(t\right)]|E\left(t\right)\}-\mathbb{E}\{{\eta }_{\beta }W\left(t\right)X\left(t\right)|X\left(t\right)\}\hfill \\ & +\mathbb{E}\{G_{\mathrm{s}}\left(t\right)[{\eta }_{\mathrm{c}}E\left(t\right)+VC\left(t\right)]|E\left(t\right)\}\hfill \\ & +\mathbb{E}\{P_{\mathrm{c}}\left(t\right)y\left(m_t\right)[r\left(t\right){\eta }_{\mathrm{ce}}{\eta }_{\mathrm{c}}E\left(t\right)+{\eta }_{\alpha }{\eta }_{\mathrm{ch}}u\left(t\right)X\left(t\right)-(1-r\left(t\right)){\eta }_{\mathrm{ce}}VC\left(t\right)-{\eta }_{\mathrm{ce}}VC\left(t\right)+V{C}_{\mathrm{f}}]\}\hfill \\ & +\mathbb{E}\{P_{\mathrm{a}}\left(t\right)[{\eta }_{\alpha }{\eta }_{\mathrm{ah}}v\left(t\right)X\left(t\right)+V{C}_{\mathrm{g}}]|X\left(t\right)\}\hfill \end{array}$$

(34)

The main concept of our control algorithm is minimizing the right-hand side of (34). In other words, by observing the system inputs, i.e., $C\left(t\right)$, $E\left(t\right)$, $X\left(t\right)$, $L_{\mathrm{e}}\left(t\right)$ and $L_{\mathrm{w}}\left(t\right)$ at each time slot in a frame, then the values of $G_{\mathrm{l}}\left(t\right)$, $G_{\mathrm{s}}\left(t\right)$, $r\left(t\right)$, $P_{\mathrm{c}}\left(t\right)$, $P_{\mathrm{a}}\left(t\right)$, $R_{\mathrm{c}}\left(t\right)$, $D\left(t\right)$ can be determined.

Derived from the analysis above, an online algorithm can be developed by solving P3:

$$\begin{array}{cc}\mathrm{P}3:\hfill & \min G_{\mathrm{s}}\left(t\right)H_{\mathrm{s}}\left(t\right)+P_{\mathrm{c}}\left(t\right)H_{\mathrm{c}}\left(t\right)+P_{\mathrm{a}}\left(t\right)H_{\mathrm{a}}\left(t\right)-D\left(t\right)H_{\mathrm{d}}\left(t\right)-W\left(t\right)H_{\mathrm{w}}\left(t\right)+R_{\mathrm{c}}\left(t\right)E\left(t\right)\hfill \end{array}$$

(35)

$$\begin{array}{c}\mathrm{s}.\mathrm{t}.G_{\mathrm{l}}\left(t\right)+D\left(t\right)+(1-r\left(t\right)){\eta }_{\mathrm{ce}}{P}_{\mathrm{c}}\left(t\right)y\left(m_t\right)={\left[L_{\mathrm{e}}\left(t\right)\right]}^{+}\hfill \end{array}$$

(36)

$$\begin{array}{c}0\le G_{\mathrm{s}}\left(t\right)+r\left(t\right){\eta }_{\mathrm{ce}}{P}_{\mathrm{c}}\left(t\right)\mathrm{y}\left({\mathrm{m}}_{\mathrm{t}}\right)+{\mathrm{R}}_{\mathrm{c}}\left(\mathrm{t}\right)\le {\mathrm{C}}_{\mathrm{char}}\hfill \end{array}$$

(37)

$$\begin{array}{c}0\le D\left(t\right)\le D_{\max },0\le W\left(t\right)\le W_{\max }\hfill \end{array}$$

(38)

$$\begin{array}{c}(1-v\left(t\right)){\eta }_{\mathrm{ah}}{P}_{\mathrm{a}}\left(t\right)+W\left(t\right)+(1-u\left(t\right)){\eta }_{\mathrm{ch}}{P}_{\mathrm{c}}\left(t\right)y\left(m_t\right)\ge L_{\mathrm{w}}\left(t\right)\hfill \end{array}$$

(39)

$$\begin{array}{c}0\le u\left(t\right){\eta }_{\mathrm{ch}}{P}_{\mathrm{c}}\left(t\right)y\left(m_t\right)+v\left(t\right){\eta }_{\mathrm{ah}}{P}_{\mathrm{a}}\left(t\right)\le T{h}_{\mathrm{char}}\hfill \end{array}$$

(40)

$$\begin{array}{c}0\le r\left(t\right)\le 1,P_{\mathrm{c}}\left(t\right),G_{\mathrm{l}}\left(t\right),G_{\mathrm{s}}\left(t\right),P_{\mathrm{a}}\left(t\right)\ge 0\hfill \\ \\ \mathrm{Here}\hfill \end{array}$$

(41)

$$\begin{array}{c}{H}_{{\mathrm{R}}_{\mathrm{c}}}\left(t\right)={\eta }_{\mathrm{c}}E\left(t\right),H_{\mathrm{s}}\left(t\right)={\eta }_{\mathrm{c}}E\left(t\right)+VC\left(t\right)\hfill \end{array}$$

(42)

$$\begin{array}{c}{H}_{\mathrm{c}}\left(t\right)=r\left(t\right)H_{\mathrm{r}}\left(t\right)+u\left(t\right)H_{\mathrm{u}}\left(t\right)+H_{\mathrm{b}}\left(t\right)\hfill \end{array}$$

(43)

$$\begin{array}{c}{H}_{\mathrm{r}}\left(t\right)={\eta }_{\mathrm{c}}{\eta }_{\mathrm{ce}}E\left(t\right)y\left(m_t\right)+{\eta }_{\mathrm{ce}}VC\left(t\right)y\left({\mathrm{m}}_{\mathrm{t}}\right)\hfill \end{array}$$

(44)

$$\begin{array}{c}{H}_{\mathrm{u}}\left(t\right)={\eta }_{\mathrm{ch}}{\eta }_{\alpha }X\left(t\right)y\left(m_t\right),H_{\mathrm{v}}\left(t\right)={\eta }_{\alpha }{\eta }_{\mathrm{ah}}X\left(t\right)\hfill \end{array}$$

(45)

$$\begin{array}{c}{H}_{\mathrm{b}}\left(t\right)=V{C}_{\mathrm{f}}-{\eta }_{\mathrm{ce}}VC\left(t\right)y\left(m_t\right),H_{\mathrm{w}}\left(t\right)={\eta }_{\beta }X\left(t\right)\hfill \end{array}$$

(46)

$$\begin{array}{c}{H}_{\mathrm{a}}\left(t\right)=H_{\mathrm{v}}\left(t\right)v\left(t\right)+V{C}_{\mathrm{g}},H_{\mathrm{d}}\left(t\right)={\eta }_{\mathrm{d}}E\left(t\right)+VC\left(t\right)\hfill \\ \hfill \end{array}$$

(47)

Observing P3, we can find the problem function includes the term $P_{\mathrm{c}}\left(t\right)H_{\mathrm{c}}\left(t\right)$ and $P_{\mathrm{a}}\left(t\right)H_{\mathrm{a}}\left(t\right)$ with $r\left(t\right)$ and $u\left(t\right)$ in $H_{\mathrm{c}}\left(t\right)$ and $v\left(t\right)$ in $H_{\mathrm{a}}\left(t\right)$. It follows that the Hessian matrix of the function is not positive semi-definite, which makes $\mathrm{P}3$ a non-convex optimization problem and challenging to solve. However, with a further investigation of $\mathrm{P}3$, $D\left(t\right)$ and $W\left(t\right)$ can be decoupled from $H_{\mathrm{d}}\left(t\right)$ and $H_{\mathrm{w}}\left(t\right)$.

At first, we take the terms $D\left(t\right)H_{\mathrm{d}}\left(t\right)$ and $W\left(t\right)H_{\mathrm{w}}\left(t\right)$ into account. If $H_{\mathrm{d}}\left(t\right)<0$, it is clear that $D\left(t\right)=0$; otherwise, it can be easily obtained that $D\left(t\right)=\min \{D_{\max },L_{\mathrm{e}}\left(t\right)\}$. Similarly, if $H_{\mathrm{w}}\left(t\right)<0$, we have $W\left(t\right)=0$; otherwise, $W\left(t\right)=W_{\max }$. Consequently, we can only concentrate on the key part of (36) which is listed as follows:

$$\begin{array}{c}\min G_{\mathrm{s}}\left(t\right)H_{\mathrm{s}}\left(t\right)+P_{\mathrm{c}}\left(t\right)H_{\mathrm{c}}\left(t\right)+P_{\mathrm{a}}\left(t\right)H_{\mathrm{a}}\left(t\right)+R_{\mathrm{c}}\left(t\right)E\left(t\right)\hfill \\ \mathrm{s}.\mathrm{t}.\left(37\right),\left(38\right)\mathrm{and}\left(40\right)\hfill \end{array}$$

(48)

We discuss the solutions to minimize P3 when the local generator is on or off, respectively. We discuss the question under the circumstance that $y\left(m_t\right)=1$ first.

First, we assume that both (37) and (40) are inactive. Analyzing those equations we can easily see the linear relationship between $H_{\mathrm{a}}\left(t\right)$ and $v\left(t\right)$. Therefore, we can set $v\left(t\right)$ to be 0 or 1 to minimize $H_{\mathrm{a}}\left(t\right)$. Similarly, $r\left(t\right)$ and $u\left(t\right)$ can also be decided to be 0 or 1 in order to minimize $H_{\mathrm{c}}\left(t\right)$.

Secondly, supposing (37) to be active while (40) to be inactive, we have:

$$G_{\mathrm{s}}\left(t\right)=C_{\mathrm{char}}-r\left(t\right){\eta }_{\mathrm{ce}}{P}_{\mathrm{c}}\left(t\right)-R_{\mathrm{c}}\left(t\right)$$

(49)

Replacing $G_{\mathrm{s}}\left(t\right)$ using (49) in (48), now the problem change into:

$$\min R_{\mathrm{c}}\left(t\right)[E\left(t\right)-H_{\mathrm{s}}\left(t\right)]+H_{\mathrm{s}}\left(t\right)C_{\mathrm{char}}+P_{\mathrm{c}}\left(t\right)H_{\mathrm{u}}\left(t\right)u\left(t\right)+P_{\mathrm{c}}\left(t\right)H_{\mathrm{b}}\left(t\right)$$

(50)

Then the minimum of the above equation can be achieved by setting $u\left(t\right)$ to be 0 or 1.

Thirdly, supposing (37) to be inactive while (40) to be active, we have:

$$P_{\mathrm{a}}\left(t\right)={\displaystyle \frac{T{h}_{\mathrm{char}}-u\left(t\right){\eta }_{\mathrm{ch}}{P}_{\mathrm{c}}\left(t\right)}{{\eta }_{\mathrm{ah}}v\left(t\right)}}$$

(51)

and we replace $P_{\mathrm{a}}\left(t\right)$ using (51) in (48), then we have:

$$\min {\displaystyle \frac{V{C}_{\mathrm{g}}[T{h}_{\mathrm{char}}-u\left(t\right){\eta }_{\mathrm{ch}}{P}_{\mathrm{c}}\left(t\right)]}{{\eta }_{\mathrm{ah}}v\left(t\right)}}+G_{\mathrm{s}}\left(t\right)H_{\mathrm{s}}\left(t\right)+P_{\mathrm{c}}\left(t\right)r\left(t\right)H_{\mathrm{r}}\left(t\right)+P_{\mathrm{c}}\left(t\right)H_{\mathrm{b}}\left(t\right)+R_{\mathrm{c}}\left(t\right)E\left(t\right)$$

(52)

Since $T{h}_{\mathrm{char}}-u\left(t\right){\eta }_{\mathrm{ch}}{P}_{\mathrm{c}}\left(t\right)\ge 0$, we can set both $u\left(t\right)$ and $v\left(t\right)$ to 1 to minimize (52).

Finally, supposing (37) and (40) to be active, we can transform the problem into:

$$\begin{array}{c}\min {\displaystyle \frac{V{C}_{\mathrm{g}}[T{h}_{\mathrm{char}}-u\left(t\right){\eta }_{\mathrm{ch}}{P}_{\mathrm{c}}\left(t\right)]}{{\eta }_{\mathrm{ah}}v\left(t\right)}}+P_{\mathrm{c}}\left(t\right)H_{\mathrm{b}}\left(t\right)+T{h}_{\mathrm{char}}X\left(t\right){\eta }_{\alpha }+R_{\mathrm{c}}\left(t\right)E\left(t\right)\hfill \\ \mathrm{s}.\mathrm{t}.\left(37\right),\left(40\right),\left(41\right)\hfill \end{array}$$

(53)

Since $T_{\mathrm{char}}-{\eta }_{\mathrm{ch}}u\left(t\right)P_{\mathrm{c}}\left(t\right)\ge 0$, to minimize (53) we can set $v\left(t\right)=u\left(t\right)=1$.

Then we discuss the circumstance that $\mathrm{y}\left({\mathrm{m}}_{\mathrm{t}}\right)=0$.

Because $P_{\mathrm{c}}\left(t\right)$ is related to $\mathrm{y}\left({\mathrm{m}}_{\mathrm{t}}\right)$, so when $\mathrm{y}\left({\mathrm{m}}_{\mathrm{t}}\right)=0$, $P_{\mathrm{c}}\left(t\right)=0$. The problem is reduced to the equation as follows:

$$\min G_{\mathrm{s}}\left(t\right)H_{\mathrm{s}}\left(t\right)+P_{\mathrm{a}}\left(t\right)H_{\mathrm{a}}\left(t\right)+R_{\mathrm{c}}\left(t\right)E\left(t\right)$$

(54)

Replacing $R_{\mathrm{c}}\left(t\right)$ use (37). Setting $v\left(t\right)=0$ or 1 can get the minimum value of $H_{\mathrm{a}}\left(t\right)$. So the equation is more concise:

$$\min G_{\mathrm{s}}\left(t\right)[H_{\mathrm{s}}\left(t\right)-E\left(t\right)]+C_{\mathrm{char}}E\left(t\right)$$

(55)

From this, what we need to discuss is the value of $G_{\mathrm{s}}\left(t\right)$. If $({\eta }_{\mathrm{c}}-1)E\left(t\right)+VC\left(t\right)\ge 0$, to minimize (55), $G_{\mathrm{s}}\left(t\right)$ should be set to 0; If $({\eta }_{\mathrm{c}}-1)E\left(t\right)+VC\left(t\right)<0$, we have $G_{\mathrm{s}}\left(t\right)=G_{\mathrm{s},\max }$.

The above analysis display the minimization is four different circumstances.

4. Performance Analysis

Theorem 1.

Define θ and ε to be:

$$\begin{array}{c}\theta =\frac{V\max \{C_{\mathrm{e},\max },C_{\mathrm{f}}\}}{{\eta }_{\mathrm{c}}}+T\min \{{\eta }_{\mathrm{d}}{D}_{\max },L_{\mathrm{e},\max }\}\hfill \end{array}$$

(56)

$$\begin{array}{c}\epsilon =\frac{V{C}_{\mathrm{g}}}{{\eta }_{\alpha }{\eta }_{\mathrm{ah}}}+T{L}_{\mathrm{w},\max }\hfill \end{array}$$

(57)

Then concluding from the process of minimizing P3 we can obtain the result:

$$\begin{array}{c}0\le B\left(t\right)\le \theta +T{C}_{char},\forall t\in T\hfill \end{array}$$

(58)

$$\begin{array}{c}0\le T\left(t\right)\le \epsilon +T{h}_{\mathrm{char}}·T,\forall t\in T\hfill \end{array}$$

(59)

Proof. 

Using induction method, the upper and lower bounds of $B\left(t\right)$ and $T\left(t\right)$ can be proved under (56) and (57).

(1)

Firstly, we show the upper bounds. The main idea is to prove that the battery and thermal tank will not charging when there level exceed $\theta$ and $\epsilon$, respectively.

• Suppose $B\left(t\right)\le \theta$. Since the electricity charged in a time slot will not be more than $T{C}_{\mathrm{char}}$. Then $B(t+1)\le \theta +T{C}_{\mathrm{char}}$ holds, obviously.
• Suppose $B\left(t\right)>\theta$, i.e., $E\left(t\right)>0$. From (42), (44) and (47), it can be obtained that $H_{\mathrm{s}}\left(t\right)>0$, $H_{\mathrm{r}}\left(t\right)>0$, $H_{\mathrm{d}}\left(t\right)>0$ and $H_{{\mathrm{R}}_c}\left(t\right)>0$. According to $\mathrm{P}3$, we must set $R_{\mathrm{c}}\left(t\right)=0$, $G_{\mathrm{s}}\left(t\right)=0$, $r\left(t\right)=0$, $D\left(t\right)=\min \{{\eta }_{\mathrm{d}}{D}_{\max },L_{\mathrm{e}}\left(t\right)\}$ to minimize the problem function, i.e., the battery will not charge when its level rises over $\theta$. Accordingly, $B(t+1)\le B\left(t\right)$ holds when $B\left(t\right)>\theta$. With the conclusion when $B\left(t\right)\le \theta$, we can know that $B\left(t\right)\le \theta +T{C}_{\mathrm{char}},\forall t\in T$.
• Suppose $T\left(t\right)\le \epsilon$. Similar to the battery, the heat charged in a time slot will not exceed $T{C}_{\mathrm{char}}·T$. Hence, $T(t+1)\le \epsilon +T{h}_{\mathrm{char}}·T$ holds then.
• Suppose $T\left(t\right)>\epsilon$, i.e., $X\left(t\right)>0$. From (45) to (47), it can be obtained that $H_{\mathrm{u}}\left(t\right)>0$, $H_{\mathrm{v}}\left(t\right)>0$, $H_{\mathrm{w}}\left(t\right)>0$ and $H_{\mathrm{a}}\left(t\right)>0$. According to $\mathrm{P}3$, we must set $P_a\left(t\right)=0$, $v\left(t\right)=0$ and $W\left(t\right)=\min \{{\eta }_{\beta }{W}_{max},L_{\mathrm{w}}\left(t\right)\}$ to minimize the problem function, i.e., the thermal tank will not charge when its level rises over $\epsilon$. Accordingly, $T(t+1)\le T\left(t\right)$ holds when $T\left(t\right)>\epsilon$. With the conclusion when $T\left(t\right)\le \epsilon$, we know that $T\left(t\right)\le \epsilon +T{h}_{\mathrm{char}}·T,\forall t\in T$ .

The above proof presents the upper bounds of $B\left(t\right)$ and $T\left(t\right)$, i.e., the capacities of battery and thermal tank, respectively. To simplify the future investigation, we denote the capacities as $B_{\max }$ and $T_{\max }$, respectively. Since the capacities are functions of V, the value of V can be changed to make a tradeoff between energy storage and cost. In contrast, the value of V can be obtained with a given battery pack or thermal tank capacity.

(2)

Secondly, we show the lower bounds using (56) and (57). To keep $B\left(t\right)$ and $T\left(t\right)$ from being negative, we only need to prevent the battery and thermal tank from discharging when they can not afford, i.e., when $B\left(t\right)<T\min \{{\eta }_{\mathrm{d}}{D}_{\max },L_{\mathrm{e},\max }\}$ and $T\left(t\right)<T{L}_{\mathrm{w},\max }$

• Suppose $B\left(t\right)\ge T\min \{{\eta }_{\mathrm{d}}{D}_{\max },L_{\mathrm{e},\max }\}$. Certainly, it follows that $B(t+1)>0$.
• Suppose $0\le B\left(t\right)<T\min \{{\eta }_{\mathrm{d}}{D}_{\max },L_{\mathrm{e},\max }\}$ . From (42), (47) and (56) , it can be obtained that $H_{\mathrm{d}}\left(t\right)<0$, $H_{\mathrm{s}}\left(t\right)<0$. According to $\mathrm{P}3$, we have to set $D\left(t\right)=0,G_{\mathrm{s}}\left(t\right)\ge 0$, i.e., the battery will not discharge. Accordingly, $B\left(t\right)\le B(t+1)$ holds then. Consequently, we can conclude that $0\le B\left(t\right),\forall t\in T$.
• Suppose $T\left(t\right)\ge T{L}_{\mathrm{w},\max }$. Then $T(t+1)>0$ follows apparently.
• Suppose $0\le T\left(t\right)<L_{\mathrm{w},\max }$. From (46) and (57), it can be obtained that $H_{\mathrm{w}}\left(t\right)<0$.According to $\mathrm{P}3$, we have to set $W\left(t\right)=0$, i.e., the thermal tank will not discharge. Accordingly, $T\left(t\right)\le T(t+1)$ holds then. Finally, we can conclude that $0\le T\left(t\right),\forall t\in T$.

Theorem 1 shows that the battery and thermal tank both have finite capacities under the proposed algorithm, which means solutions of $\mathrm{P}3$ are feasible solutions to $\mathrm{P}1$ as well.  ☐

Theorem 2.

The gap between the optimal cost of P1 and the expected cost obtained by solving P3 is no more than $\frac{TB}{V}$, i.e.,

$$\underset{T\to \infty }{\lim }\frac{1}{T}\sum _{i=0}^{T-1}\mathbb{E}\{f^{{}^{\prime }}\left(t\right)\}\le P_1^{*}+\frac{TB}{V},$$

(60)

where $f^{{}^{\prime }}\left(t\right)$ is the energy cost at time slot t under the proposed algorithm, and $P_1^{*}$ is the optimal solution to the original problem P1.

5. Numerical Simulations

In this section, we use Matlab to evaluate the performance of the proposed algorithms by numerical simulations. We consider a hotel with battery and thermal tank as well as the CHP system. In our simulation, each time slot represents 15 min and each frame consists of 4 time slots (i.e., one hour). The parameter settings are partly listed in

Table 1. Parameter settings in numerical simulations.

Parameter	Value	Unit
${\eta }_{\mathrm{ah}}$	0.8
${\eta }_{\mathrm{c}}$	0.9
${\eta }_{\mathrm{d}}$	1.1
${\eta }_{\alpha }$	0.9
${\eta }_{\beta }$	1.1
$c_{\mathrm{f}}$	0.0035	$/kBtu
$c_{\mathrm{m}}$	0.1	$/h
$D_{\max }$	30	kWh/h
$C_{\mathrm{char}}$	20	kWh/h
$G_{\mathrm{l},\max }$	32	kWh/h
$G_{\mathrm{s},\max }$	32	kWh/h
$W_{\max }$	30	kBtu/h
$P_{\mathrm{c},\max }$	50	kBtu/h
$P_{\mathrm{a},\max }$	32	kBtu/h

and detailed in Section 5.1.

5.1. Simulation Setup

Centralized Power Grid: We obtain the electricity price data of power grid from [24]. The data trace is shown in Figure 3. The maximal supply power $G_{l,\max }$ and charging power $G_{s,\max }$ obtained from power grid are both set to be 32 kWh/h.

External Gas Station: We assume that the natural gas price $c_g$ varies across time and has a uniform distribution between (0.004, 0.010)$/kBtu. The maximal thermal output is set as $P_{a,\max }=32$ kBtu/h. The efficiency is set as ${\eta }_{\mathrm{ah}}=0.8$.

CHP System: The maximal thermal output of CHP system is set as $P_{c,\max }=50$ kBtu/h. The overall CHP efficiency is assumed to be 80%, and the electricity conversion efficiency is in the range of 30–40%. The fuel cost of CHP is set as $c_{\mathrm{f}}=0.0035$ $/kBtu. We set the minimal on/off period of CHP to be 1 h, i.e., 4 time slots. The sunk cost for maintaining the system in its active state is set as $c_m=0.1$$\$/\mathrm{h}$.

Harvested Wind Power: The harvested wind power data is obtained from [25]. The data path is shown in Figure 4.

Battery and Thermal Tank Model: We set the maximal charging and discharging rates of the battery as $C_{\mathrm{char}}=20$ kWh/h and $D_{\max }=30$ kWh/h, while the charging and discharging efficiency is set as ${\eta }_{\mathrm{c}}=0.9$ and ${\eta }_{\mathrm{d}}=1.1$, respectively. Similarly, the heat storing and releasing efficiency of the thermal tank is also set as ${\eta }_{ff}=0.9$ and ${\eta }_{fi}=1.1$. The maximal heat output is set as 30 kBtu/h.

Electricity and Heat Demand: We use the real demand data provided by California Commercial End-Use Survey (CEUS) [26] in our simulation. The data traces in 50 h (i.e., 200 time slots) are shown in Figure 5.

5.2. Results of the Simulation

With the parameters above, we simulate for 200 time slots and each time slot stands for 15 min. We let $V=5$.

Figure 6 shows the process where the CHP system adaptively makes on/off decisions in 200 time slots. We can see that the decisions are made every 4 time slots.

The sample paths in Figure 7 depict electricity and heat supplies in the first 24 h (i.e., 96 time slots). Both the electricity and heat demands can be satisfied with hybrid sources.

Figure 8 specifies the charging/discharging behavior of the battery and thermal tank. As shown in Figure 4 and Figure 5a, the electricity demand is larger than the harvested wind power at every time slot, there is no excessive renewable energy (wind power) left to charge into the battery. As a result, there is no wind power illustrated in Figure 7a. Actually, in our optimization problem and simulations, if there exists excessive wind power, it can be charged into the battery for future use under our designed algorithm.

Figure 9 shows the corresponding changes of battery level and thermal tank level in all the 200 time slots under $V=5$. We can see that the tank level remains almost stationary due to the uniform distribution of the gas price, while the battery level fluctuates in reaction to the electricity price. However, the capacities of both battery and thermal tank are bounded.

5.3. Performance vs. V and Charging/Discharging Efficiency

With further simulations under different values of ${\eta }_{\alpha }$, ${\eta }_{\beta }$, ${\eta }_{\mathrm{c}}$, ${\eta }_{\mathrm{d}}$ and V, we observe the impacts of the weight V and efficiency of charging/discharging.

As shown in Figure 10, the cost drops when charging and discharging get more efficient. Figure 11 shows that the capacity of battery and thermal tank is linear with V, which is also indicated in Theorem 1. The total cost curve shown in Figure 12, on the other hand, converges to the minimum with increasing V. Furthermore, by comparing with the situations where CHP is permanently on and off, the effectiveness of our adaptive on/off decision policy can be verified.

6. Conclusions

In this paper, we studied the operating cost minimization problem for microgrids with CHP generation, energy storages, and renewable energy resources by using the Lyapunov approach. We designed an algorithm LYP that can achieve near-optimal performance by adjusting the value of V. According to a large amount of empirical evaluations, the microgrid operating cost can be reduced significantly through such an integration of centralized grid, renewable energy, power storage device, and co-generation.