Decision Support for Negotiations among Microgrids Using a Multiagent Architecture ^†

Abstract

This paper presents a decision support model for negotiation portfolio optimization considering the participation of players in local markets (at the microgrid level) and in external markets, namely in regional markets, wholesale negotiations and negotiations of bilateral agreements. A local internal market model for microgrids is defined, and the connection between interconnected microgrids is based on nodal pricing to enable negotiations between nearby microgrids. The market environment considering the local market setting and the interaction between integrated microgrids is modeled using a multi-agent approach. Several multi-agent systems are used to model the electricity market environment, the interaction between small players at a microgrid scale, and to accommodate the decision support features. The integration of the proposed models in this multi-agent society and interaction between these distinct specific multi-agent systems enables modeling the system as a whole and thus testing and validating the impact of the method in the outcomes of the involved players. Results show that considering the several negotiation opportunities as complementary and making use of the most appropriate markets depending on the expected prices at each moment allows players to achieve more profitable results.

1. Introduction

Microgrids are distributed power networks that are electrically connected to the power grid at the distribution level and have clear electrical boundaries. Thus microgrids are seen from the system viewpoint as single controllable entities acting as aggregated generators or loads [1]. The lower operating costs of the grid drive the microgrid central controller (MGCC) to cooperate with other interconnected microgrids [2]. Reaching efficient synchronization between the grid and microgrids is crucial for short-term operation of integrated microgrids.

It is expected that in future power grids influenced with the deployment of smart grid technologies, the main building blocks of the network are microgrids that are interconnected with each other, as well as the upstream network. For instance, the electricity grid of a district can be seen as a microgrid. The concept of transactive energy allows the transaction of energy among these networks. Transactive energy is used by the GridWise Architecture Council’s Framework to refer to a combination of economic and control techniques used to balance the generation and consumption within a power grid while considering the grid constraints [3,4].

1.1. Objectives

This paper aims at analyzing the interaction between the local entities of a microgrid and the coordination among integrated microgrids as a complementary transaction opportunity to the wholesale market. In order to potentiate the implementation of microgrids, we need the models that clearly define and characterize the behavior of participants in the market environment, where players search for short-term and long-term financial benefits. The operation of inter-connected microgrids is represented as a transactive energy system, which encompasses agents [5]. The transactive energy system framework for interconnected microgrids allows the transaction of energy between neighbor microgrids and between these microgrids and other players in the power system. The transactive signals between the agents existing in the interconnected microgrids are economic value signals, such as price or incentives.

The evolution of the power system towards a transactive energy system, making use of the already made investment in smart grids and microgrids [6], forces the involved players to adapt to the new reality and seek new ways to potentiate their investments in such a competitive and constantly changing environment. Decision support and simulation solutions become increasingly important in this domain [7], with optimization models and multi-agent systems (MAS) as some of the most widely needed approaches.

1.2. Related Work

MAS are suitable for modeling the interactions between players with heterogeneous characteristics and capabilities. For this reason, MAS have become promising solutions to study and explore different types market models and associated negotiation strategies. E-commerce is a timely example in which much relevant work has been proposed. For example, in [8], a stable multilateral automated negotiation model is proposed to enable the matching process in a two-sided e-marketplace. With this purpose, this approach proposes two optimization models, one with the goal of reaching an effective balance among the proposal value of issues and the other maximizing social welfare. Several negotiation policies, customer’s tactics and coalition tools are proposed in [9] as services in e-marketplaces. MAS are applied as means to model the e-market architecture. The work presented in [10] studies how negotiation efficiency is affected by the varying bidding strategies of a costumer. With this purpose, this work proposes several competitive bidding strategies for many-to-many supplier negotiation process. In [11] a constraint model for negotiations is proposed, directed to the resolution of scheduling and supply planning problems. These problems are modeled through inter-agent fuzzy constraints.

The developments in MAS approaches have created the basis for a suitable modeling of microgrids’ operation in short-term markets [1,12]. These approaches represent loosely connected networks of interacting distributed intelligent hardware and software agents [1,13]. MAS approaches are also very useful to model smart grids, as they enable representing the decision-making process of different entities. The MAS approach proposed in this work is used to model neighbor microgrids operation in a multi-market setting.

Different MAS approach are proposed in [2] to deal with microgrids’ centralized control and management. A laboratorial case study is experimented in [14] to test the practical implementation of integrated microgrids. This work validates control and reliability models for DG and energy storage systems. The short-term operation of a microgrid containing DGs and storage units is modeled in [15], and it has been noted that the storage units are more profitable when the MGCC plans to increase the share of intermittent units. Microgrids attain more benefits from resource sharing among electrically interconnected microgrids [12]. A hierarchical control model is used in [1,12] to manage the optimal scheduling of generation and the energy transactions between integrated microgrids and the power grid. A market model dealing with DR exchange based on MAS technology is proposed in [16].

The aggregate performance of microgrids is optimized with the control algorithms proposed in [17]. This work aims at achieving the optimal scheduling of both energy demand and thermal comfort. Two scheduling layers are developed for the optimal operation of interconnected microgrids in [18]. In the proposed hierarchical energy management system, operation scheduling within the microgrid is done by local energy management system and a community energy management system is responsible for the optimal planning of the interconnected microgrid. Coordination among the microgrids is the responsibility of this top layer.

Work done in [5] addresses the microgrid operation, considering the integration with the upstream network under the transactive energy concept. Homes with advanced energy management systems can participate in transactive energy system, namely by receiving price signals and power profile forecasts with nodes at higher levels [19,20] discusses the importance of integration DSO, aggregators into wholesale market and bulk power using a transactive energy framework to face the operational and business challenges arisen from variability of DERs in distribution systems. The operational challenges of integrating DSO, DGs and microgrids with AC and/or DC distribution systems has been addressed in [5]. The hourly interactions between microgrids is defined with a coalitional game theory model in [21]. The optimal coalition is obtained by reaching an equilibrium state. The main assumption in this model is the existence of cooperative microgrids. The optimality in this problem is achieved by minimizing the power loss and avoiding interaction with the upstream distribution network.

In [22] authors have proposed an approach that considers three different levels of negotiation. The first level refers to the internal dispatch of each microgrid. The two remaining levels refer to the external negotiations with players outside the microgrid. These players refer to neighbor microgrid and electricity market agents. The external market is a distribution system market, which is operated by the DSO. In this previous work, the three levels are considered as sequential and complementary, i.e., firstly each microgrid performs its internal dispatch, then neighbor microgrids negotiate directly with each other in order to look for advantageous deals before participating in the external market to fulfil their consumption needs or sell the surplus generation. This approach as proven to be advantageous for the microgrids as it enables them to profit from the participation in the three negotiation levels.

1.3. Contributions

In this paper, a decision support model is introduced with the objective of enabling players to optimize their negotiations by selecting best market opportunities to participate in each moment, considering the expected outcomes in each of the markets. This model is extended from the work presented in [23], which addresses the optimization of wholesale electricity market negotiations. The new model introduced in this paper is now able to consider negotiations at the local level as complementary to those at the wholesale level. Thus, the several negotiation levels are no longer considered as sequential as in [22], rather as complementary and/or alternative depending on the expected negotiation prices at each time. In this way, players may choose to negotiate the full amount in a certain market if it is expected to generate a more profitable outcome in a certain moment, or, on the contrary, negotiate partial amounts in different negotiation opportunities if this is expected to be more economically advantageous. The internal operation of the microgrids is now approached as a local (internal) microgrid market, and nodal pricing in distribution network (microgrid) is used as a basis for interaction between interconnected microgrids. Nodal pricing in the distribution level is an effective means to manage the consumption. It can produce the value signals that are used in transactive energy systems for agents’ negotiation. These signals show the dynamically varying needs of the agents [24]. In [25], it is concluded that the DG units have higher revenues under nodal pricing and contribute more to reduced line losses. The method used to derive nodal prices in a distribution network is similar to obtaining them at the transmission level [25].

Moreover, the modeling of microgrids interaction with the market is conducted with a MAS approach. In specific, the decision support model for market participation portfolio is integrated in Adaptive Decision Support for Electricity Markets Negotiations (AiD-EM) [26], a decision support MAS for electricity market negotiations. Multi-Agent Simulator for Electricity Markets (MASCEM) [27] is used to study and simulate the actual market transactions. Finally, Multi-Agent Smart Grid Platform (MASGriP) [28], is used to simulate the players in a microgrid/smart grid environment. The integration of the proposed models in these MAS and the communication between the different systems enables simulating the entire system from the small players inside the microgrids to the large players in the wholesale market, while using the required decision support capabilities to evaluate the impact of their decisions on their different types of interactions. Figure 1 shows a general overview of the proposed approach.

Figure 1 shows that the proposed approach can be applied to multiple combinations of electricity market models, which the objective of optimizing a certain player’s participation in these complementary market opportunities. The considered market models include the microgrids’ internal (local) market model, which is proposed in Section 2.1; the negotiation among microgrid, presented in Section 2.2; and several market models that are typically part of the wholesale market, such as the day-ahead spot market, bilateral contract negotiations and intraday markets, as described in Section 2.3. The historic log of prices established in each of the different markets is used by the proposed model to forecast or estimate the expected price to be achieved in each of the markets depending on the negotiated amount, as explained in Section 3. The expected prices in each market are used by optimization model proposed in Section 3.1 to optimize the support player’s participation in the several markets. The optimization model results in the set of volumes that the player should sell and/or buy in each of the considered market opportunities. These recommendations are used by the player to, later on, perform the actual negotiations in each market. The simulation of the different market models is performed using a multi-agent approach, which is described in Section 3.2. The decision support components are also integrated in the MAS model, enabling the market participating players to benefit from this aid to improve their outcomes from market participation.

In summary, the main original contributions of this work are:

• The definition of a local market setting for internal microgrids management. This local market model is extended from the preliminary work initially presented in [22]. The extension from the model of [22] includes the improvement of the optimal power flow formulation and the detail of the market model formulation;
• The introduction of a negotiation portfolio optimization model for decision support in negotiations in local and wholesale markets;
• The modeling of the integrated microgrids in the market environment using a MAS approach.

After this introductory section, Section 2 focuses on the operation of interconnected microgrids, including the specification of the considered internal (local) microgrid market and an overview of the complementary external negotiation opportunities. Section 3 presents the proposed negotiation portfolio model, which includes the mathematical formulation of the decision support model, and a description of the integration of the proposed models in the MAS simulation framework. Section 4 presents the case studies that evaluate the advantages of using the proposed model. Finally, Section 5 presents the most relevant conclusions of this work.

2. Operation of Interconnected Microgrids

Microgrids facilitate the integration and management of DG units by providing the means to guarantee a reliable operation of the small portion of the grid. Energy storage units can also be integrated in microgrids to ease the management of large amounts of variable DG. Microgrids are usually connected to the main upstream grid in a point of common coupling (PCC). It is thereby seen from the main system as a controllable independent subsystem [29]. These operation characteristics are why the future power system is usually seen as a group of interconnected microgrids, where each microgrid can be limited to a district. Thus, under this scheme it is unlikely that large blackouts can threaten the power grid as they are doing now. On the other hand, the small size of microgrids makes the operation more challenging, with physical implications on the power system performance [30].

The MGCCs are at the top level of microgrids’ control and management structure and coordinates the operation between the commercial aggregators (CAs), virtual power players (VPPs) and the utility grid. MGCC are responsible for optimizing the microgrid operation, regarding voltage and frequency levels. MGCC decisions are crucial both when the microgrid is connected to the main grid, as also when the microgrid is operating in isolated mode [31]. In the proposed model, the responsible administrative entity for demand side management is the CAs. They can offer elastic demand bids or flexibility to the MGCC. CAs in a microgrid aggregate the demand of end-users and have the flexibility to accept DR commands from the MGCC. The market participation and operation of neighbor microgrids is designed by means of three negotiation models: (i) an internal market considering the bids of the players located inside the local network, with regard to the network constraints; (ii) the negotiations among neighbor microgrids; and (iii) the participation in wholesale electricity markets.

2.1. Microgrids’ Internal (Local) Market

In the internal market of each microgrid the MGCC clears the market while satisfying the network constraints. This market is for the VPPs and CAs to submit their offers and bids with regard to the internal market particular rules and policies. The CAs submit price-responsive demand bids at each bus for price elastic loads and inelastic demand bids for the loads that accept energy at any cost. The supply offer and demand bid curves at the buses with more than one serving VPPs or CAs are aggregated before the market clearing procedure. Load and generation dispatch for the involved players is calculated. From the market clearing results the LMPs at the different buses.

Congestion management is formulated as maximization of social welfare subject to power balance and line congestion constraints [32]. The objective of the MGCC for internal market clearing would be to maximize the social welfare in the microgrid Equation (1), subject to the power flow constraints [33]. Maximizing the social welfare causes the maximization of consumers’ benefits and minimization of suppliers’ generation cost [32]. In (1), $C_b$ is the aggregated generation cost function of the VPPs at each bus and $B_b(·)$ is the aggregated consumption benefit function of the CAs at each bus. The expected cost and benefit function of the interconnected microgrids at the joint buses are also aggregated with these functions. $p_b^{Load,t}$ is the aggregated load at buses and $p_b^{DG,t}$ is the aggregated of the DGs at bus b. Equations (2) and (3) show how these variables are calculated. $p_g^t$ is the active power generation of DG g and $p_l^t$ is the active demand required by load l [34]:

$$\mathrm{Maximize}{\displaystyle \sum _{t\in T}{\displaystyle \sum _{b\in B_i}{B}_b(p_b^{Load,t})}-C_b(p_b^{DG,t})}$$

(1)

$$p_b^{DG,t}={\displaystyle \sum _{g\in {\Omega }_g^b}{p}_g^t}\forall t\in T,\forall b\in B_i$$

(2)

$$p_b^{Load,t}={\displaystyle \sum _{l\in {\Omega }_l^b}{p}_l^t}\forall t\in T,\forall b\in B_i$$

(3)

The power balance constraint required by the DC-Optimal Power Flow (OPF) at all buses of the microgrid is shown in Equation (4). Constraint in Equation (5) enforces the lower and upper bus voltage limitations. Generation of each DG unit and the consumption of each load is limited by Equations (6) and (7).

$$p_b^{DG,t}-p_b^{Load,t}={\displaystyle \sum _{c\in B_i}\frac{1}{x_{bc}}\left({\delta }_b^t-{\delta }_c^t\right)}\forall t\in T,\forall b\in B_i$$

(4)

$${\delta }_b^{Min}\le {\delta }_b^t\le {\delta }_b^{Max}\forall t\in T,\forall b\in B_i$$

(5)

$$P_g^{Min}\le p_g^t\le P_g^{Max}\forall t\in T,\forall g$$

(6)

$$P_l^{Min}\le p_l^t\le P_l^{Max}\forall t\in T,\forall l$$

(7)

Congestion of the lines have a critical impact on determining the price of electricity. The operator has to clear the market at the bus level when there is congestion in the system [35]. The market clearing price at the bus level is known as LMPs. LMPs can be calculated using either DC Optimal Power Flow (OPF) or full AC OPF [36]. Despite the high precision of the AC OPF model in calculating the economic signals, its linear approximation, i.e., the DC OPF model is generally preferred to that by the market operators due to the convergence issues of the AC OPF [36]. LMP is the marginal increase in the total system cost due to one unit increase in the active power consumption at each bus [36]. The hourly LMP for the real power at each bus is the Lagrange multiplier associated with the real power balance constraints at that bus in the OPF model [33,35]. It is the dual variable for the equality constraint at that bus [35]. The lossless DC OPF is widely used by the market operators to provide economic signals for market participants [36].

2.2. Negotiation among MGCCs

Neighbour microgrids can also negotiate with each other, helping to share resources locally, and also by seeking opportunities for deals at more attractive prices than the wholesale market. These negotiations mean that each MGCC should analyse the market to forecast the expected market prices for the following time periods. This value can be used as reference for the analysis of the possible deals to be negotiated. When offers from nearby MGCCs are more attractive than the predicted market prices, the MGCCs may decide to transact some power among them, obtaining better deals than they would have if negotiated the entire amount in the market. In addition to the market analysis, MGCCs should also apply negotiation strategies to try and reach advantageous deals with their neighbours.

2.3. Wholesale Electricity Market

The power and energy sector has been changing significantly. Currently, most countries have their own market or participate jointly with other countries in a regional markets [37]. Each electricity market is typically formed by several market models, most commonly: day-ahead spot, intraday or balancing market and bilateral contracts model. The differences between them lie under their rules, objectives and how they operate. The day-ahead spot market aims to negotiate energy for the following day, the intraday/balancing market allows players to negotiate almost in real-time in order to adjust their needs and compensate possible forecast errors. Both these market types are usually operated by means of double auctions, where buyers and sellers submit their purchase and sale bids, and from which results a unique market price for each negotiation period, which set the price for all the transactions made in that market in the corresponding period. Negotiation by means of bilateral contracts consists on trading power directly with other entities, usually for a large time horizon [38].

The spot market is where most of the energy is usually transacted. It also promotes competitiveness, as it is more likely to be able to sell in this market when the sale bids are lower, which gets more buyers to compete in the bidding. This process contributes to lowering the selling prices through competitiveness.

3. Proposed Negotiation Portfolio Optimization Model

The proposed decision support methodology aims at providing the best investment profile for a market player, considering different market opportunities, including negotiations in local markets. The amount of power that each supported player should negotiate in each available market type in order to maximize its profits, takes into account the prices that are expected to be achieved in each market, in different contexts. The expected prices in each market at each time period are used by an optimization process that has the objective of maximizing the outcomes of players when participating in the market, through the optimization of players’ market participation portfolios. This model is presented in Section 3.1.

The decision support model is integrated in a multi-agent framework composed by different MAS, which are specific for distinct purposes, namely electricity market negotiations, microgrid and smart grid modelling, and decision support for players’ negotiations, as presented in Section 3.2.

3.1. Mathematical Formulation

Considering the expected available power of a market player for each period of each day, the amount to be negotiated in each market type is optimized to get the maximum income that can be achieved. The inputs are:

• the weekday, referred as d in Equation (6);
• the number of days, Nday;
• the negotiation period, referred as p;
• the number of periods, Nper;
• a boolean variable for each distinct market or negotiation platform, indicating if this player can enter it to sell: $Asel{l}_{M1\dots NumM}$;
• a boolean variable for each session of the balancing market, indicating if this player is allowed to buy in each of them: $Abu{y}_{S1\dots NumS}$;
• M1, M2, …, NumM are the considered markets;
• S1, S2, …, NumS are the considered balancing market sessions;

The outputs are:

• $Spo{w}_{M1\dots NumM}$ representing the amount of power to sell in each market;
• $Bpo{w}_{S1\dots NumS}$ representing the amount of power to buy in each session of the balancing market;

In this formulation ps_M,d,p is the expected price for the selling of power, and pb_S,d,p the expected price for buying. The formulation of the proposed model for multiple markets participation is presented in (8):

$$\begin{gathered} f\left(Spo{w}_{M\dots NumS},Bpo{w}_{S1\dots NumS}\right)=Max\left[\begin{array}{c}{\displaystyle \sum _{M=M1}^{NumM}}\left(Spo{w}_{M,d,p}\times p{s}_{M,d,p}\times Asel{l}_M\right)-\\ {\displaystyle \sum _{S=S1}^{NumS}}\left(Bpo{w}_S\times p{s}_{S,d,p}\times Abu{y}_S\right)\end{array}\right] \\ \forall d\in Nday,\forall p\in Nper,Asel{l}_M\in \left\{0,1\right\},Abuy\in \left\{0,1\right\} \\ p{s}_{M,d,p}=Value\left(d,p,Spo{w}_M,M\right) \\ p{s}_{S,d,p}=Value\left(d,p,Bpo{w}_S,S\right) \end{gathered}$$

(8)

The Value function returns the expected value of the power for each particular period of each day, and for each market. Market prices forecasting is performed using an artificial neural network [39]. In some markets, the price also depends on the power amount to trade. When a player tries to establish a bilateral contract, the deals may be highly dependent on the amount of power that is being negotiated. The same fact is verified in other markets, even if not in such a clear way. So, this prediction takes that in consideration too, by applying fuzzy logic on the absolute amount of the power, to classify it in one of the categories defined by a clustering mechanism, which groups the ranges of amounts that present similar prices in each market [40]. The correspondent price is obtained through the Data matrix, which stores all the prices. The value function is expressed in (9):

$$Value\left(day,per,Pow,Market\right)=Data\left(fuzzy\left(pow\right),day,per,Market\right)$$

(9)

This formulation has some constraints that are dependent on the individual characteristics and requirements of each particular market. Other constraints that must be taken into consideration are the ones imposed by the complex conditions that each player can present. These constraints are formulated depending on the set of conditions that the player presents, that also depend on each market that it enters.

The main constraint, which is applied to every situation, is expressed in (10), to impose that the total power reserved to be sold in the set of all markets is never higher than the total expected production (TEP) of the player, plus the bought power along all sessions of the balancing market:

$${\displaystyle \sum _{M=M1}^{NumM}Spo{w}_M}\le TEP+{\displaystyle \sum _{S=S1}^{NumS}Bpo{w}_S}$$

(10)

Variables Abuy and Asell are Boolean variables that indicate if the player is “allowed to buy or sell” in each session of each market, i.e., if the participation as a seller or as a buyer is permitted. A reference example is the balancing market, where a player that is registered as seller, is able to buy, and vice-versa; also, in bilateral contracts, and in local market negotiations, there is no restriction, meaning that any player is able to buy or sell power. However, there is no restriction that obligates a player to do so; it just indicates if the player is able to do it if desired.

This optimization process allows us to:

• Play with the possibility of negotiating in different market opportunities depending on the expected prices at each moment, considering the negotiation amount;
• Play with the possibility of negotiating with neighbor players in search for advantageous deals, thus avoiding the need to negotiate solely in regional or wholesale markets;
• Play with the possibility to negotiate with different players in the bilateral contracts, and so having the chance to get higher or lower prices, depending on the circumstances;
• Play with the chance to wait for the later sessions of the balancing market to provide higher amounts of energy, if it is expected for the price to go up;
• Play with the possibility for sellers to buy and buyers to sell in the balancing market, to get good business opportunities: using arbitrage opportunities, buying extra energy when the prices are expected to be lower, and then selling it later when the prices go up; or if the prices show the opposite tendency, offer more energy than the player actually expects to produce, to get greater profit, and then buy that difference in the expected lower prices opportunities.

3.2. Multi-Agent Architecture

3.2.1. AiD-EM

The proposed negotiation portfolio optimization model is integrated in AiD-EM [23], a multi-agent decision support system that enables electricity market players to use the available data in an intelligent and adaptive way in order to cope with the multiple challenges that arise from electricity markets’ participation.

AiD-EM uses real market data, data derived from past and current simulations, and external sources data (e.g., weather conditions such as wind speed, solar intensity and temperature; or raw materials prices, among other) to support the decision making process. Decisions are modelled specifically for each different market negotiation type, namely the negotiation of bilateral and forward contracts, and participation in auction based markets, such as the day-ahead spot market and balancing markets. The multi-agent approach of AiD-EM facilitates the interactions between the different components and also the communication with external agents, such as the market players themselves, which make use of the decision support. Figure 2 presents the multi-agent architecture of AiD-EM, including the portfolio optimization model.

As shown by Figure 2, the AiD-EM manager agent is the main entity of the system, detaining the responsibility of providing the connection with the MASCEM electricity market simulator through the direct interaction with the supported market player(s). When several market players require the decision support of AiD-EM simultaneously, multiple AiD-EM manager agent instances are created, so that each supported market player has its own manager agent, with the sole responsibility of handling the player’s decision making process. For this, the AiD-EM manager agent executes the portfolio optimization methodology, presented in Section 3.1, in order to decide whether and when the supported market player should participate in each market type. Once the objectives for each market participation are defined, Adaptive Learning Strategic Bidding System (ALBidS) [26] is used to support the negotiations in auction based markets, e.g., spot, balancing and intraday markets. Complementarily, when the negotiation by means of bilateral negotiations is also envisaged, the Decision Support for Energy Contracts Negotiation (DECON) system is used [41].

The connection to MASCEM enables testing and validating the developed decision support methodologies under realistic simulation conditions, taking advantage on the enhanced simulation capabilities of MASCEM and on the interactions between the involved players.

3.2.2. MASCEM

MASCEM [42] is a MAS-based simulator for electricity markets. MASCEM supports the most common market models: namely the day-ahead spot market, intraday markets, bilateral contracts, forward markets and ancillary services. MASCEM also supports the definition of alternative specifications for the market mechanisms, e.g., the definition of block and flexible orders and specification of complex conditions. MASCEM includes the exact market models of several real market operators, namely those of the Iberian electricity market—MIBEL, central European market—EPEX, and northern European market—Nord Pool.

Simulation scenarios in MASCEM are automatically defined, using the Realistic Scenario Generator (RealScen) [43]. RealScen gathers real data online, comprising market data, including submitted proposals; accepted proposals and established market prices; proposals details; execution of physical bilateral contracts; statement outages, accumulated by unit type and technology; among others. Simulation scenarios are thereby created by combining the real market data with data from simulations. These scenarios may represent what has occurred exactly in a certain market in a specific day, or they may concern different configurations, aiming at experimenting the impact of players’ behaviour under alternative defined scenarios.

3.2.3. MASGriP

MASGriP enables simulating the microgrid, smart grid and involved players [28]. MASGriP combines fully simulated agents and agents that interact with real hardware. Simulations with this thereby may include both real infrastructures and simulated agents, enabling the experimentation of a large set of situations and models (Energy Resource Management (ERM) algorithms, demand response (DR) models, negotiation procedures, among many other) in a realistic simulation environment [44].

MASGriP’s simulated agents reflect the real world, including some operators, such as the distribution system operator (DSO) and the independent system operator (ISO). However, the majority of players represents energy resources, such as several types of consumers (e.g., industrial, commercial, residential), different types of producers (e.g., wind farms, solar plants, cogeneration units), EVs with vehicle-to-grid capabilities, among others. Aggregators present an important role in the future power system management and operation. Some examples of the considered aggregators are: virtual power players (VPPs) [45], which can aggregate any other resource, including other aggregators; curtailment service providers (CSPs) [46], which aggregate consumers that participate in DR programs; smart grid and microgrid operators, which manage the players that are contained in a specific area. These players introduce a higher level of complexity in the system management.

MASGriP’s interaction with MASCEM and with AiD-EM creates an enhanced modelling and simulation framework that facilitates the study of a large diversity of scenarios, providing the means for analysing the interactions between different players of very distinct natures, and assessing the impact of alternative types of management and negotiation.

4. Case Studies

Using the integrated multi-agent society, composed by the three MAS, some simulations are executed with the aim of validating the proposed methodology and assessing the impact of using the proposed model for decision support of migrogrid negotiations in a competitive market environment. The first case study shows the detailed results achieved in a specific negotiation period, in order to facilitate the understanding of the benefits that methodology actually brings for the negotiating players. The second case study shows the comparison of the results achieved using the proposed methodology against those achieved in previous work [22].

4.1. Case Study 1

This case shows an illustrative example of the results that can be achieved using the proposed methodology, considering the expected negotiation prices in a single negotiation period.

In order to define a realistic scenario, five market models are considered, thereby enabling the supported market player to sell and buy in all of them. The considered markets are the day-ahead spot market, bilateral contract negotiations, the intraday (or balancing) market, and a microgrid level local market. The balancing market is divided into different sessions. In the day-ahead spot market the player (acting as seller) is only allowed to sell electricity, while in the other market types the player can either buy or sell depending on the expected prices. Limits have also been imposed on the possible amount of negotiation in each market. In this case, it is only possible to buy up to 1 MWh in each market in each period of negotiation, which makes a total of 4 MWh purchased. It is possible to sell power on any market, and it can be transacted as a whole or in instalments. The player has 1 MWh of own production for sale.

In this problem, it has also been imposed that in each session of the balancing market, the player can only either sell or buy in each period. In bilateral contracts and in local market negotiation, it is possible to both sell and purchase in the same period (by negotiating with different players).

Since the optimization requires real market data, so that it can be used to support players’ decisions in a realistic environment, it is necessary that the electricity prices are provided. The real electricity market prices data, concerning the day-ahead spot market, the intraday market, and bilateral contracts have been extracted from website of the Iberian electricity market operator—MIBEL [47]. Local market prices are based on the results of previous studies [45].

Figure 3 shows the variation of electricity prices in the different considered market types, depending on the traded amount. This representation allows explaining how the optimization model behaves.

As can be seen from Figure 3, some markets (day-ahead spot and balancing market sessions) present a unique market price for this hour, regardless of the amount that the supported player negotiates. On the other hand, in bilateral contracts and local markets, prices are variable with the negotiated amount. The Smart-Grid (local market) is assumed as a bilateral contract based market among local energy community players; for this reason, the variation of prices depending on the negotiated amount is also visible, similarly to regular bilateral contract negotiations. The expected result is that the optimization allocates the sales to markets and amounts where the price is higher and purchases when the price is lower.

Table 1. Scheduling of sales and purchases in the different markets.

	Spot Market	Bilateral	Balancing 1	Balancing 2	Local Market
Sales (MWh)	1.478	1.150	0	0	0.846
Purchases (MWh)	0	0.475	1.000	1.000	0

shows the scheduling of sales and purchases made in the different markets, as result of the proposed optimization process.

By matching Table 1 and Figure 3 it is possible to explain the functioning of the proposed methodology for the considered period. As expected, the model presents a solution with the purchased energy in the cheapest markets and sales in the most profitable. As the total energy that can be bought in each market is 1 MWh, the maximum amount is bought in the balancing sessions (lower prices), and also a purchase of 0.475 MWh in bilateral contracts. The sale is set to the local markets in 0.846 MWh, 1.150 MWh in bilateral contracts and 1.478 MWh in the spot market.

4.2. Case Study 2

This case study demonstrates the advantages of the proposed approach by comparing its results with those of previous work [22]. Three levels of negotiation are simulated (internal/local microgrid market, negotiations among neighbor microgrids, and participation in the external market); and the results are validated with real electricity markets data. Real data of the Iberian Electricity Market Operator (MIBEL) [47], namely for the day 19 July 2013 is used in this case. In the previous work, the three negotiation levels are approached as sequential, i.e., firstly the local microgrid market takes place, then the neighbor microgrids negotiate with each other, and finally the microgrids participate in the external market in order to fulfill their needs or sell surplus power. On the other hand, in this work the proposed decision support model is used to optimize the negotiation amount that should be invested in each market opportunity, with the goal of improving the negotiation results of the players. The microgrids of the test system (Figure 4) are assumed to have control, measurement and sensing devices with two-way communication structure where the smart operation is guaranteed with the dynamic responding of the agents. It is assumed that the whole system is operating at 25 kV. The loads and generation units are distributed among the three MGCCs, as shown in

Table 2. Overview of generation units in the considered integrated microgrids.

Generators	Min Real Power Output (PU)	Max Real Power Output (PU)	Bus	MGCC
gen1	0.00	0.20	1	1
gen2	0.01	0.23	4	1
gen3	0.00	0.24	6	1
gen4	0.06	0.15	7	1
gen5	0.01	0.29	8	1
gen6	0.00	0.20	9	2
gen7	0.01	0.23	10	2
gen8	0.00	0.24	11	2
gen9	0.06	0.15	12	2
gen10	0.01	0.29	13	2
gen11	0.00	0.30	14	2
gen12	0.00	0.40	15	2
gen13	0.00	0.15	16	3
gen14	0.01	0.20	17	3
gen15	0.00	0.10	18	3
gen16	0.06	0.12	19	3
gen17	0.01	0.14	20	3

.

From Table 2 it is visible that MGCC 2 is the microgrid that contains the larger amount of generation. In the clearing of the local market, MGCC consider the expected demand and supply bids from the grid and from the microgrids connected at the joint buses. The MGCC determines the LMPs at the different buses based on these expected bids and on the bids placed by the players located within the microgrid. The LMP values in the connection buses of the three considered microgrids, resulting from the internal microgrid market are presented in Figure 5.

From Figure 5 it is visible that the LMPs of the connection buses of the three microgrids present rather variable values. The LMPs represent the minimum sale price or the maximum purchase price when negotiating with the connected microgrids.

After the microgrids’ internal dispatch each microgrid may have surplus generation power, which may be sold, or a lack of supply that needs to be purchased. The situation regarding the three considered microgrids in this case study is:

• MGCC 1: Has power to sell in hours 1 to 7, 23 and 24;
• MGCC 2: Has extra generation in all hours of the day;
• MGCC 3: Sells from hours 1 to 7, and needs to buys on the remaining hours of the simulated day.

Considering these results from the internal microgrid markets, in the previous work, the microgrids negotiate with each other in order to achieve advantageous deals. Figure 6 presents the profits achieved in the second level of negotiation.

From Figure 6 it is visible that the three MGCCs were able to negotiate some of the available amount directly with each other in some hours of the day. MGCC 2 sells in some hours of the day, and MGCC 1 and MGCC 3 mostly buy (thus the negative profits). The transactions set in this level are, however, not enough to fulfil all the needs from the three MGCC, therefore the microgrids need to participate in the third negotiation level—the external market. Figure 7 presents the profits achieved by the three microgrids in the external market.

From Figure 7 once can see that the three microgrids were able to negotiate the remainder of the power to fulfil their needs. MGCC 2 mostly sells in the hours that it has not been able to sell before, in the second negotiation level. MGCC 1 and MGCC 3 sell and buy to complement the agreements from level 2.

On the other hand, when using the proposed approach, the three negotiation levels are no longer seen as sequential, rather, the several market opportunities are considered as both complementary and alternative depending on the expected prices in each market, i.e., the proposed methodology suggests in which markets should the players negotiate at each time. Following the rules of the MIBEL market, the considered market is composed by the following market types [47]:

• Day-ahead spot market;
• Intraday (or balancing) market;
• Negotiation of bilateral contracts.

Adding the local negotiations between neighbour microgrids to these market opportunities, and using the expected market prices from the considered simulation day, the results shown in Figure 8 are achieved for the negotiations of MGCC 1.

Figure 8 shows that, similarly to the previous study, when using the proposed methodology as decision support to plan the negotiations, MGCC 1 still considers the negotiation with the neighbour microgrids as advantageous in some periods of the considered simulation day, namely to purchase some amount of power. The day-ahead wholesale market is the main market where sales are scheduled. Bilateral contracts are also used in the first periods of the day to purchase some extra amount of power at lower prices, which can be sold in the day-ahead spot market at higher prices, thus increasing the profits of MGCC 1. The same occurs with MGCC 2, as can be seen by Figure 9.

As can be seen from Figure 9, some opportunities of achieving power at low prices are used by MGCC 2 to purchase (mainly through bilateral contracts), so that higher profits can be achieved by selling that amount plus the generation surplus amount of this microgrid, mainly in the day-ahead spot market, and during some periods in the intraday market as well. The local negotiation with neighbour MGCC is used as complementary selling place during some hours of the day. The negotiation with neighbour MGCC is the main investment of MGCC 3, as displayed by Figure 10.

As discussed previously, MGCC 3 is the microgrid that needs the larger amount of power from external players to the microgrid. For this reason, three distinct opportunities are used during the day to purchase the required amount of power at the best prices, namely bilateral contracts, the negotiation with neighbor MGCCs, and the intraday market. The day-ahead spot market is, similarly to the other MGCCs, the chosen marketplace to sell (due to the higher expected prices).

Figure 11 presents the comparison of the profits that are achieved by each of the three MGCCs in the total of the 24 h of the considered simulation day, when using the proposed methodology, and when not using it (from the previous study [22]).

From Figure 11 it is visible that using the proposed methodology allows the three MGCCs to increase their profits. MGCC 1 had negative profits (costs) when not using the proposed methodology (a value of −€73.98), which is turned into profit when using the proposed methodology to choose the negotiation opportunities that are most suitable at each moment (profit of approximately €60.39). MGCC 2, due to the larger amount of generation in this microgrid, was able to generate a fair amount of profits even when not using the proposed approach. However, when using the proposed method, when choose the negotiation opportunities appropriately, the profits increase from €2890.1 to €3799.74. Finally, MGCC 3 is not able to turn the costs into profits, as MGCC 1, due to its greater need to purchase in order to fulfil the needs of its loads; but it still is able to decrease the costs by making a proper scheduling of its purchases and sales. The costs are reduced from −€362.44 to −€278.68.

5. Conclusions

This paper proposes a decision support methodology to help electricity market players in optimizing their negotiation investments. The proposed model considers the complementarity between different electricity markets, contrarily to the traditional approach of sequential of alternative market opportunities. A local market model for internal microgrids negotiation is presented and extended from previous work. Negotiation among different microgrids is also considered as complementary market opportunity to wholesale market models. The modeling and simulation of the diverse market models is performed using a MAS approach and the proposed negotiation portfolio optimization model for decision support in negotiations in local and wholesale markets is also integrated in the MAS model in order to facilitate the decision-making process.

Results show that by using the proposed methodology, the different market negotiation opportunities can be used as complementary, providing microgrid operators with the chance of looking for the better deals. By using the available market opportunities in an optimized way, taking into account the expected transaction prices in each market at each moment, players are able to increase their profits or decrease their costs when compared to approaches in which the markets are considered as sequential or alternative to each other.

The promising results achieved by the proposed approach suggest that further improvements should be experimented. One possible research path is the exploration of alternative local market models that can be implemented in a complementary way to current wholesale market models. Another interesting research direction is the exploration of ways for players (especially small players) to aggregate and form coalitions that may be advantageous for them to increase market power and be able to participate in markets in each they cannot participate due to their limited negotiation volume or network constraints.