3.2.2. MAS for Electricity Markets

MAS of the electricity markets concern market players and markets modeling, strategic bidding and decision support [37]. Multi-agent-based simulation of the electricity markets usually combines with artificial intelligence techniques and game theories and is not only simulation platforms but also provides opportunities for the scenario comparison, future evolution study and sensitive analysis [38].

Several studies have applied MASs to model and simulate electricity markets [14]. For instance, Li et al. [39] discuss the potential for developing Open Source Software (OSS) for power market research. The Agent-based Modelling of Electricity Systems (AMES) is an agent-based OSS laboratory, specifically designed for the experimental study of reconstructed wholesale power markets. The AMES

simulation includes an independent system operator, load-serving entities, and generation companies distributed across the transmission grid.

Another electricity market model is the Electricity Market Complex Adaptive System (EMCAS) model [40] utilized by Koritarov [1]. The model is used to capture and investigate the complex interactions between the physical infrastructures (generation, transmission, and distribution) and the economic behavior of market participants [41]. Furthermore, the model applies an agent-based approach where agents' strategies are based on learning and adaption. This approach enables simulations in different time periods, from real-time to decades including both pools and bilateral contract markets. This approach also makes it possible to see the evolution of an electricity market over time and stakeholders' reaction towards changes in economy, finance, and regulation. The study describes two methods of how the agents learn: observation-based and exploration-based learning. In observation-based learning, the learning process is based on a structured process of past market performance evaluation, future market status prediction, and investigation of other agents' actions. Agents decide either to keep or adjust their current market strategy or use a new strategy. Agents based on exploration-based learning explore new market strategies, and these strategies are simulated in a simulation tool. The results are observed, and the strategies are either accepted or rejected based on the results and the agents' goals.

Praca et al. [42] develop the Multi-Agent Simulation of Competitive Electricity Markets (MASCEM) [43]. The model is developed to study the behavior and evolution of an electricity market. The MASCEM is a modeling and simulation tool aiming to study the operation of complex and competitive electricity markets [44]. The agents in the system represent the market entities, such as generators and customers. The MASCEM allows agents to establish their own decision rules and adapt their strategies as the simulation progresses based on previous events. As a decision-supporting tool, the simulator includes different possibilities regarding electricity market negotiations [45,46]. The MASCEM is a flexible tool which makes it easy for users to define models including strategies, types of agents and market types. For example, this flexibility is utilized by Santos et al. [3,47,48] for modeling and simulating the EPEX (central European electricity market) and Nord Pool spot market (Scandinavian electricity market). The MASCEM can also be used for modeling and simulation of other electricity markets such as MIBEL (the Iberian electricity market), GME (the Italian electricity market), and even markets outside Europe [48].

## 3.2.3. MAS for Demand-Side and Building Systems

MAS provides a flexible and reliable solution to manage and optimal loads at demand-side with the consideration of energy cost minimization and user's comfort maximizations [49,50]. MAS has been applied in automated building management systems (BMS) for energy-related building research [16,51–53].

The automated BMS research in energy-related building systems mainly focuses on control mechanisms of building loads and investigate possibilities and potentials of energy efficiency and flexibility in buildings [54,55], and especially much equipment in buildings can be controlled and deliver demand flexibility, e.g., lighting and HVAC, and can respond to the grid signals [56]. Although complex control systems are important in building systems, these processes need to be optimal, flexible, and automated.

Multi-agent-based modeling techniques have been used to integrate real-time intelligent decision-making in building control. For instance, an indoor environment that actively supports its inhabitants can be created with these techniques [57]. These modeling techniques also include unpredictable user-behavior, fluctuating weather conditions, and grid imbalances [52,58]. For instance, the study by Anvari-Moghaddam et al. [52] demonstrates how MAS is used to optimize management strategies for a building through computer simulations in combination with third-party software such as MATLAB and GAMS. Hence, studies show that energy consumption can be reduced without compromising the inhabitants' comfort level in residential buildings.

In the study [52], a smart grid is simulated with several residential buildings, conventional and RES. The residential buildings include underfloor heating, heat pumps, and energy storages. The simulation incorporates meteorological data for the examined location together with technical data, to estimate the power production from RES. The simulation result shows that it is possible to reduce domestic energy consumption and meet the system's objectives and constraints at the same time. However, the study does not take fault-tolerant and uncertainty handling capabilities into account.

The study by Zeiler and Boxem [16] analyses how smart grid and building optimization can work together and presents an ontology of a software system which acts as a bridge between BMS and a smart grid. Several experiments are conducted in this study to test a HVAC system in a building environment, including the interaction with a smart grid. The study also includes the dynamic behavior of the occupants towards the systems in combination with an overall goal of energy efficiency. The study finds that different elements depend on each other, e.g., changes in required heating affect the available energy. The automated equipment, controlled and managed by the building, responds to demand response requests from the grid to balance the grid condition [59]. The experiment also shows that the comfort level increases while the energy consumption decreases in their MAS modeling.

Meanwhile, the study by Mousavi et al. [53] includes the unpredictable nature of the business process in an office building in a simple model with only a few devices to control. This study does not include a response to the grid conditions. Instead, the study investigates an energy automatic model for office buildings to reduce energy consumption and increase the indoor comfort level. The model is a MAS with the ontology based on the standard IEC 61499 (automation system standard) [60]. The goal of this study is to optimize the energy consumption in an office building where the ontology provides the communication logic and allows agents in the model to share knowledge and data [61]. In the MAS model, agents communicate and collaborate towards a common goal. The method has been applied to an office meeting room, where meeting activities and equipment can be automatically controlled, including measurements of energy consumption. Based on the data gathered as a result of the simulation, the study shows that it is possible to reduce 50 % of the room's monthly energy consumption by controlling the operation and preparation of the room automatically. The duration of the meeting room simulation is 20 working days (1 working month). The simulated BMS automatically acknowledges the meeting schedules and needs for shading, screen, and blackboard usage, etc. The business process is combined with automated processes to overcome the inefficient use of energy in buildings and lower the number of system failures.
