**3. Introducing the Proposed System**

As shown in Figure 1, the multi-agent combination, in which each agent performs its tasks to achieve the overall goal of the system, is called MCS. Generally, MCS is divided into three layers: upstream network, MG, and field, as shown in Figure 2. These three layers consist of eight agents. The agents are the upstream network, micro-grid, thermal, hydrogen, RB unit, renewable, storage, and load collector. The upstream grid agent is located in the first layer, which includes the natural gas grid and the electricity grid. This agent is used as an additional resource in case of a lack of energy production.

**Figure 1.** Structure of the proposed MEMG.

**Figure 2.** Architecture of the MCS and date exchange.

The micro-grid agent is located in the second layer, which is responsible for coordinating the production and consumption of electrical and thermal energy. This task is performed under the optimal performance of agents while observing the constraints.

The other six agents associated with the production or consumption of electrical and thermal energy and hydrogen are located in the field layer. The overall structure of the proposed method of this study is shown in Figure 3. As shown in Figure 3, in the first step, we select the data related to wind speed and energy demand, and apply them to the LSTM block as input data. In the second step, using the recursive neural network (LSTM) method, we predict the diagrams related to electrical energy data of the wind turbine output, energy demand, and energy price. In the third step, we use a mixed-integer linear programming method and optimize the total cost function, while meeting the existing constraints. This step is carried out according to the modeling of MEMG agents, which is discussed in the next section. Moreover, uncertainty data are modeled with the LSTM block. In the fourth step, the optimal performance of each agent is determined, while minimizing the objective function.

**Figure 3.** The overall structure of the proposed method.

### **4. Modeling Agents**

### *4.1. Upstream Network*

This agent must announce the hourly price of buying and selling electricity and natural gas, as well as the constraints on energy exchange in the network of the micro-grid operator.

$$\text{Price}\_{NET}(t) = \pm P\_{NET}(t) \,\text{C}\_{NET}(t) \Delta t \tag{1}$$

$$P\_{NET,\min} \le P\_{NET}(t) \le P\_{NET,\max} \tag{2}$$

### *4.2. Micro-Grid Agent*

This agent must transmit information about energy costs to field layer agents; it is also responsible for monitoring the optimal performance of field layer agents while observing the constraints imposed by the upstream network and reducing the operating costs of the micro-grid.

The electrical equilibrium equation is defined as follows:

$$P\_T(t) + P\_{WPP}(t) + P\_{WT}(t) + P\_{INV}(t) - P\_{REC}(t) \pm P\_{NET}(t) = P\_{ED}(t) \tag{3}$$

The *AC* power in the inverter is calculated using Equation (4):

$$P\_{INV,AC}(t) = P\_{Inv,DC}(t)\alpha\_{Inv} \tag{4}$$

Additionally, the *AC* power in the rectifier is obtained from Equation (5).

$$P\_{\text{Rec, }AC}(t) = \frac{P\_{\text{Rec, }DC}(t)}{\mathfrak{a}\_{\text{REC}}} \tag{5}$$

The thermal equilibrium equation is also defined as follows:

$$P\_{TT}(t) + P\_{TFC}(t) + P\_B(t) + P\_{TS}(t) = P\_{TD}(t) \tag{6}$$

The micro-grid agent checks the above equilibrium equations, and the system must operate in such a way that the above conditions are met.

The amounts of air pollutants emitted from the operation of micro-turbines, fuel cells, rubbish burning units, and the boiler in the micro-grid, in kg/kWh, are obtained from Equation (7):

$$Emission = \sum\_{t=1}^{24} \left\{ E\_T(t) + E\_{FC}(t) + E\_{WPP}(t) + E\_B(t) \right\} \tag{7}$$

Micro-grid performance must be optimized with the following constraints:

$$\frac{Emission}{\sum\_{t=1}^{24} P\_{ED}(t)} \le Emission\_{\text{max}}\tag{8}$$

where *Emissionmax* is the maximum value of the pollutants, and is equal to 0.66 kg/kWh. The objective function of total costs of the micro-grid is defined by the following equation:

$$\begin{array}{ll}\text{Obj. Function} = & \sum\_{t=1}^{24} \left\{ \mathbb{C}\_{f,\Upsilon}(t) + \mathbb{C}\_{\text{OM},\Upsilon}(t) + \mathbb{C}\_{\text{S},\Upsilon}(t) + \mathbb{C}\_{f \cdot F\mathcal{C}} + \mathbb{C}\_{\text{OM},F\mathcal{C}}(t) + \mathbb{C}\_{\text{S},\mathcal{F}\mathcal{C}}(t) \\ & + \mathbb{C}\_{f,\mathcal{WPP}}(t) + \mathbb{C}\_{\text{OM},\mathcal{WPP}}(t) + \mathbb{C}\_{\text{S},\mathcal{WPP}}(t) + \mathbb{C}\_{\text{OM},\mathcal{W}\mathcal{I}}(t) + \mathbb{C}\_{\text{OM},\Upsilon\mathcal{S}}(t) \\ & + \mathbb{C}\_{\text{OM},\mathcal{H}\mathcal{I}}(t) + \mathbb{C}\_{\text{OM},\mathcal{E}\mathcal{S}}(t) \end{array} \tag{9}$$

It should be noted that the electrical power of micro-turbine agents, fuel cells, rubbish burning units, electrical storage agents, and *PNet* are considered to be decision variables.

*4.3. Thermal Agent*

This agent consists of two parts: micro-turbine, and boiler. The equation of the electrical output power of micro-turbines is as follows:

$$P\_T(t) = \frac{a\_T HHV\_{gas}Cons\_T(t)}{\Delta t} \tag{10}$$

Additionally, the thermal output power of the micro-turbine is proportional to the electric power, which is as given in Equation (11):

$$P\_{TT}(t) = K\_{Th,T} P\_T(t) \tag{11}$$

The thermal output power of the boiler is also given as Equation (12).

$$P\_B(t) = \frac{a\_B HHV\_{\text{gas}}Cons\_B(t)}{\Delta t} \tag{12}$$

The costs of fuel, maintenance and repair, and switching on and off of the micro-turbine are also calculated by Equations (13)–(15), respectively.

$$\mathcal{L}\_{f,T}(t) = P\_T(t) \operatorname{Price}\_{\text{gas}} \Delta t \tag{13}$$

$$\mathbb{C}\_{OM,T}(t) = \mu\_T(t) P\_T(t) \text{Price}\_{OM,T} \Delta t \tag{14}$$

$$\mathcal{L}\_{\mathcal{S},T}(t) = \mathcal{S}\_T |\boldsymbol{\mu}\_T(t) - \boldsymbol{\mu}\_T(t-1)|\Delta t \tag{15}$$

The amounts of air pollutants produced by micro-turbines and boilers can be calculated through Equations (16) and (17), respectively.

$$E\_T(t) = \mu\_T(t) P\_T(t) E R\_T \Delta t \tag{16}$$

$$E\_B(t) = \mu\_B(t) P\_B(t) \, ER\_B \Delta t \tag{17}$$

### *4.4. Hydrogen Agent*

This agent includes *FC* and *HT*; it must announce the characteristics of the above two parts to the micro-grid agent. Electric and thermal output power in *FC* is calculated by Equations (18) and (19), respectively.

$$P\_{\rm FC}(t) = \frac{\mathfrak{a}\_{\rm FC}\mathfrak{a}\_{\rm ref}\,HHV\_{\rm methance}\mathbb{C}ons\_{\rm FC}(t)}{\Delta t} \tag{18}$$

$$P\_{TFC}(t) = K\_{Th,FC} P\_{FC}(t) \tag{19}$$

Costs related to fuel consumption, maintenance, and turning on and off of the *FC* are formulated as in Equations (13)–(15). Moreover, the amount of air pollutants produced by *FC* is similar to that given in Equation (16), according to the specifications of the *FC*.

The amount of hydrogen stored in the hydrogen tank is formulated as follows (20):

$$V\_{tank}(t) = V\_{tank}(t-1) + \Delta V\_{tank}(t) \tag{20}$$

$$
\Delta V\_{tank}(t) = \pm \frac{E\_{H\_2}(t)P\_{H\_2}}{HHV\_{H\_2}} \tag{21}
$$

where *PH*<sup>2</sup> is the density of hydrogen, which is equal to 0.085 g/L. The constant *HHVH*<sup>2</sup> is considered to be 142 MJ/Kg.

### *4.5. Rubbish Burning Agent*

The rubbish burning agent includes the RB power plant; it is also responsible for announcing the status and characteristics of the RB power plant to the micro-grid operator. Moreover, the source of waste supply for this agent is municipal solid waste. The electrical output power of this unit is calculated using Equation (10). Furthermore, the costs of fuel consumption, maintenance and repair, and turning on and off of the RB unit are formulated according to Equations (13)–(15). The amount of pollutants produced by the RB power plant is similar to that given by Equation (16).

### *4.6. Renewable Agent*

In recent years, artificial-intelligence-based methods have been known as a promising tool to model the different stochastic parameters, such as load demand, generation of renewable energy sources, and electric vehicle behavior [36,37]. In this paper, a method based on long short-term memory (LSTM) neural networks is used for modeling the stochastic parameters. The LSTM networks are very popular in time-series forecasting because they are robust against the vanishing gradient problem [38]. Interested readers are referred to [37,38] for more information about the LSTM network structure and its formulation.

### *4.7. Storage Agent*

The storage agent must report the status and characteristics of the electrical and thermal storage units to the micro-grid agent. In this section, the amount of electric charge stored by the system is calculated using Equation (22):

$$V\_{ES}(t) = V\_{ES}(t-1) + V\_{ES,Ch}(t) - V\_{ES,chh}(t) \tag{22}$$

The amount of heat stored by the system is also calculated with the same equation (Equation (22)). An equation similar to Equation (14) satisfies the maintenance and repair costs of the storage system.

### *4.8. Load Collector Agent*

As a renewable agent for modeling the uncertainty of the load controller agent, an LSTM neural network was used.

### *4.9. Agents' Connection*

According to Figure 2, the communication between agents takes place in six steps. Figure 2 shows the sequence of information exchange in the proposed MCS. In Figure 2, the numbers indicate the sequence of messages. It should also be noted that messages are sent on an hourly basis. The connections between agents in the system are such that in the first step, the upstream network agent announces information about energy purchase costs and constraints to the micro-grid. In the second step, the micro-grid agent requests the status of the agents from the field layer agents; then, in the third step, the field layer agents respond to the micro-grid request. In the fourth step, the micro-grid agent sends the status of the energy shortage and the purchase request to the upstream network agent in order to return the confirmation of the purchase or sale of electricity to the micro-grid in the fifth step. Finally, in the sixth step, the micro-grid agent sends instructions related to the performance of the field agents to each agent.

### *4.10. LSTM*

A recursive neural network (R-NN) is a modified version of conventional neural networks [39]. In deep R-NNs, the descriptive version of R-NNs, known as LSTM networks, can be used to solve the problem of gradient vanishing in hidden layers. In the mentioned LSTM, various operational gates are considered, as shown in Equations (23)–(27) [40].

$$\dot{\mathbf{u}}\_t = \sigma \left(\text{WiS}\_t^{(l-1)}\right) + \text{WhiS}\_{(t-1)} + b\dot{\mathbf{u}} \tag{23}$$

$$f\_t = \sigma \left( Wiq\varphi S\_t^{(l-1)} \right) + WhqS\_{(t-1)} + bf \tag{24}$$

$$c\_t = f\_t c\_{(t-1)} + i\_t \text{tanh} \left( W i \gamma S\_t^{(l-1)} \right) + W h \gamma S\_{(t-1)} + b \varepsilon \tag{25}$$

$$o\_t = \sigma \left(WioS\_t^{(l-1)}\right) + WloS\_{(t-1)} + bo \tag{26}$$

$$S\_t = \rho\_t \tanh(c\_t) \tag{27}$$

where

*Wi*, *Wiϕ*, *Wiγ* ∈ *Rr*×*nh Whi*, *Whϕ*, *Whγ* ∈ *Rnh*×*nh* and *bi*, *b f* , *bc*, *bo* ∈ *R*1×*nh* [41].

### **5. Linearization**

In this step, in order to reduce the computational costs and problem-solving time, we linearize the equations for modeling MEMG agents via the following methods:

• Linearizing by multiplying two binary variables *u*1, *u*<sup>2</sup> [42]:

$$z = \mathfrak{u}\_1 \times \mathfrak{u}\_2\tag{28}$$

So (28) will be linearized by (29).

$$z \le \mu\_1, \ z \le \mu\_2, \ z \ge \mu\_1 + \mu\_2 - 1 \tag{29}$$

• Linearizing by multiplying a binary variable *u*<sup>1</sup> and a continuous variable *x*<sup>1</sup> [43]:

$$z = \mu\_1 \times x\_1 \tag{30}$$

So (30) is linearized by the inequalities of (31).

$$z \le x\_{1\prime} \; z \le M \times u\_1 \; \prime \; z \ge x\_1 - M(1 - u\_1) \tag{31}$$

where *M* is a large constant;

• Linearizing quadratic cost function: To linearize quadratic cost function, we use the piecewise linear (P.W.L) method described in [44].

### **6. Simulation**

To validate the proposed method, we used the proposed MCS method in the described MEMG. All simulations were conducted with an Intel® Core (TM) i7-10810u CPU with a frequency of 1.61 GHz and with GAMS software.

### *6.1. Input Data*

In this research, information about uncertainties regarding wind turbine energy production as well as energy demand was predicted using the LSTM networks, as can be seen in the diagrams of Figures 4–7. The data from Ontario province in Canada were used as input data for the LSTM network based on [45,46]. Hourly data on wind speed, electricity prices, and energy demand over three years from 1 January 2018 to 30 December 2020 were investigated. It should be noted that the energy price data are for Ontario in Canada. Given that retail prices are commonly used for MGs, the Ontario market price data were scaled at an appropriate rate. The specifications of the micro-turbines, fuel cells, boilers, and the waste power plant are shown in Table 1 [47,48]. Moreover, the total cost in Equation (9) is minimized by considering the constraints in the system with GAMES software and a mixed-integer linear programming method. The nonlinear form of the total cost equation makes the calculations difficult. Therefore, once the nonlinear equation is minimized, the total cost equation is first linearized, and then the minimization is carried out. Furthermore, to compare the proposed method and the validation of this method, we used a conventional centralized approach to optimize the performance of agents in order to reduce the initial

costs of the MG. In the centralized method, uncertainties related to wind speed and the total energy demand are not considered, and the actual amount is not predicted.

**Table 1.** Specifications of the micro-turbine, fuel cell, boiler, and waste power plant.


**Figure 4.** Daily electric power generated by wind turbines.

**Figure 5.** Daily electricity prices.

**Figure 6.** Daily electricity load demand.

**Figure 7.** Daily thermal load demand.

### *6.2. Results*

The results of the amounts of energy production or consumption in each of the network agents are shown in Figures 8 and 9. To validate the proposed method, the optimization results in linear and nonlinear methods, as well as the common optimization method, are shown in Table 2, regardless of the uncertainties. As shown in Table 2, using the proposed method reduces MEMG operating costs by 34% compared to conventional centralized models. The USD 26.6 decrease is due to a reduction in the charge and discharge cycles (ES). According to the diagram in Figure 8, it is clear that the electrical energy exchanged between the MG and the upstream grid in one day is equal to 354.5 KW. Since power generation with an MG is always assumed to be cheaper than purchasing power from the upstream grid, micro-grids have reduced operating costs. On the other hand, according to the data in Table 1, the electrical energy produced in WPP is cheaper than the micro-turbines and FC units. According to Figure 8, it can be seen that the amount of electrical energy produced by the WPP is higher than the FC and micro-turbine units, which is also one of the reasons for reducing the cost of the MEMG. As shown in Figure 9, from points 2 to 6, the thermal energy produced by the FC and micro-turbine is more than the heat load, and the thermal storage system is charging. On the other hand, from points 9 to 13 and 16 to 22, since more heat load is generated and stored than thermal energy, the boiler responds to the heat load demand. Moreover, the use of the proposed method reduces the emission of pollutants by the micro-turbine, RB, FC, and boiler compared to the conventional centralized method. In addition, Table 2 shows that the use of the proposed method leads to a reduction in CPU optimization time. This reduction in time indicates a reduction in the computation in the proposed method. It is clear that by linearizing the equations related to MG agents, the simulation time decreases significantly due to the linearization of equations and the reduction in the complexity of the optimization calculations.

**Table 2.** Results of the proposed MCS-based method and centralized method.


**Figure 9.** Optimal management of thermal elements of the micro-grid.

### **7. Conclusions**

In this study, an applied method for optimal management of the performance of an MEMG is presented, considering the uncertainties associated with the prediction of daily demand. The agents of the energy system are at three decentralized levels, and are interrelated at these levels. The equations are formulated according to the relationships between agents at these three levels.

The proposed method was tested on an MEMG. The results indicate a reduction in operational network costs and the complexity of computations compared to centralized methods. On the other hand, linearization of equations was carried out. This linearization can reduce the computational complexity of the proposed method.

Therefore, energy management systems with an MCS-based modeling approach are a suitable solution for optimal energy management and reducing the demand of microconsumers (urban buildings) from upstream networks, electricity, and natural gas networks, reducing greenhouse gas emissions. In future work, the design of an MG should be considered so that the MG can make operational decisions that affect the market price.

**Author Contributions:** Conceptualization, S.S.; methodology, M.F. and S.S.; software, M.F.; validation, M.K. and T.S.; formal analysis, S.S. and A.N.; investigation, M.K.; resources, A.N.; data curation, M.F. and S.S.; writing—original draft preparation, M.F.; writing—review and editing, A.N. and M.K.; visualization, S.S.; supervision, A.N. and T.S.; funding acquisition, A.N. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** Publicly available datasets were analyzed in this study. This data can be found here: [http://www.ieso.ca/power-data] (8 January 2022), [http://climate.weather.gc.ca/ historicaldata/searchhistoricdatae.html] (8 January 2022).

**Conflicts of Interest:** The authors declare no conflict of interest.
