Multi-Agent Systems Applications in Energy Optimization Problems: A State-of-the-Art Review

Abstract

This article reviews the state-of-the-art developments in Multi-Agent Systems (MASs) and their application to energy optimization problems. This methodology and related tools have contributed to changes in various paradigms used in energy optimization. Behavior and interactions between agents are key elements that must be understood in order to model energy optimization solutions that are robust, scalable and context-aware. The concept of MAS is introduced in this paper and it is compared with traditional approaches in the development of energy optimization solutions. The different types of agent-based architectures are described, the role played by the environment is analysed and we look at how MAS recognizes the characteristics of the environment to adapt to it. Moreover, it is discussed how MAS can be used as tools that simulate the results of different actions aimed at reducing energy consumption. Then, we look at MAS as a tool that makes it easy to model and simulate certain behaviors. This modeling and simulation is easily extrapolated to the energy field, and can even evolve further within this field by using the Internet of Things (IoT) paradigm. Therefore, we can argue that MAS is a widespread approach in the field of energy optimization and that it is commonly used due to its capacity for the communication, coordination, cooperation of agents and the robustness that this methodology gives in assigning different tasks to agents. Finally, this article considers how MASs can be used for various purposes, from capturing sensor data to decision-making. We propose some research perspectives on the development of electrical optimization solutions through their development using MASs. In conclusion, we argue that researchers in the field of energy optimization should use multi-agent systems at those junctures where it is necessary to model energy efficiency solutions that involve a wide range of factors, as well as context independence that they can achieve through the addition of new agents or agent organizations, enabling the development of energy-efficient solutions for smart cities and intelligent buildings.

1. Introduction

An agent is a computer system situated in some environment that is capable of autonomous action in this environment in order to meet its design objectives. Autonomy is a difficult concept to pin down precisely, but we mean it simply in the sense that the system should be able to act without the direct intervention of humans (or other agents), and should have control over its own actions and internal state [1]. A multi-agent system (MAS) is designed to meet a number of objectives according to a set of rules and regulations. An MAS is a system that integrates a set of agents that interact, communicate and coordinate themselves to achieve the established objectives. Self-coordination refers to the way in which the agents that make up the system cooperate to reach the objective of consuming less resources. This is where communication comes into play, and it is an essential element of MAS. These basic principles of communication and self-organisation are maintained from their first definitions. One of these first definitions of agent can be found in the proposal of Wooldridge [2], “an agent is a computer system that is situated in some environment, and that is capable of autonomous action in this environment in order to meet its design objectives”.

Later definitions include the grouping of agents for achieving objectives and their adaptation to the environment, as in the definition proposed by Ferber [3] “An agent can be a physical or virtual entity that can act, perceive its environment (in a partial way) and communicate with others, is autonomous and has skills to achieve its goals and tendencies. An MAS consists of an environment, objects and agents (the agents being the only ones to act), relations between all the entities, a set of operations that can be performed by the entities and the changes of the universe in time and due to these actions”.

We can summarize that agents must be able to interact, negotiate and coordinate to achieve the common goal. We can define agents as intelligent entities with social skills (interaction, collaboration, communication, coordination, competence, negotiation, intelligence) that encapsulate a functionality to solve a problem, Wooldridge and Jennings [4], and Jennings et al. [5].

The use of an agent as a concept has been widely used in social sciences, since it is one of the alternatives to mathematical models to model different behaviors, so that they can be simulated. Orcutt [6] noted that the macroeconomic models that had been proposed up to that point could not provide relevant information about the influence of government policies on the evolution of micro-level entities (households and companies). The objective of Orcutt was to model micro-level entities within a model that would allow their simulation, making their results more relevant and useful for the study of the dynamics of social systems. For this, the basic principle of microsimulation is to concretely integrate the micro level by means of rules, whether deterministic or stochastic, that are applied in the attributes of the individual units, which leads to the modeling of changes in their status and behavior. However, in 1968, the concept of agents as a paradigm of artificial intelligence (AI) was not yet developed and we would have to wait a few years [7].

In the field of energy optimization in buildings, this ability to simulate social behaviors is a very useful tool to know whether to achieve the marked objective (reduce energy consumption) according to the set of rules (economic parameters cannot be jumped, non-invasive methods have to be used, etc.) does not affect user comfort (lighting, heating, cooling and ventilation within the values desired by the user).

However, the MAS have evolved in such a way that they not only serve model situations. The skills of the agents have been perfected as the concept and development of the MAS has evolved. In the work developed by Corchado and Laza [8], one of the solutions on which the community of investigators of agents was working was presented, in order to provide the learning capacity to the belief–desire–intention (BDI) agent architectures (deliberative). For this, they used Case-based reasoning (CBR) systems to help agents solve problems, facilitating their adaptation to changes in the environment and identifying new possible solutions. From the adoption of learning techniques by the agents, different works emerged, as presented by Di Mascio et al. [9], where an agent-based approach was developed for children between seven and eleven years of age with hearing difficulties, with the aim of improving their reading ability, thus improving their school performance—thanks to the ability of agents to learn about the situation of each child and provide different games and educational activities that improve reading. De la Prieta et al. [10] presents an agent-based architecture that uses the advantages provided by Cloud Computing platforms to deal with the open issues on the Learning Object paradigm.

The granting of learning capabilities to agents has allowed MAS to reason about different possibilities (different configurations of lighting, heating, cooling or ventilation in an Intelligence Building (IB), for example), autonomous decision-making, learning new situations and errors from the past (a temperature configuration has been made that allows the maximum energetic optimization according to the set of rules, but the user has decided to increase the temperature a little), programming of tasks and forecasting of future situations.

These new characteristics and abilities have enabled MAS to be used in very different areas and contexts such as drug identification systems for intoxicated drivers [11], obtaining genes that present different levels of expression indicating a disease [12,13,14] or its application or the classification of faces according to gender and age [15].

In this section, we have presented the concept of agents, their evolution in organized systems and how they have evolved to have the ability to adapt to the context and environment, learning and decision-making by modelling agents with AI techniques. Having made this introduction, it is necessary to know the state of this technique applied to energy optimization, to optimize energy in Intelligent Buildings and to conduct a study about the advantages, disadvantages and shortcomings that this methodology presents.

The rest of the article is structured as follows: Section 2 presents an overview about MAS. Section 3 describes the state of the art of energy optimization in IB with MAS. Section 4 presents a proposal based on MAS to achieve energy efficiency in intelligent buildings. Finally, the conclusions section explains how MAS is a methodology that allows us to address the problem of energy efficiency in intelligent buildings.

2. Overview of Multi-Agent Systems

2.1. MAS Concept

The paradigm of multiple agent systems (MAS) provides a wide range of possibilities in the field of building control. The MAS emerged as a response to distributed artificial intelligence problems. Many of these distributed intelligence problems have a series of characteristics very similar to current problems regardless of the area. A very concise definition of the term MAS is the one proposed by Poole and Mackworth [16]: AI is about practical reasoning: reasoning in order to do something. A coupling of perception, reasoning, and acting comprises an agent. An agent acts in an environment.

As mentioned in Section 1, agents are characterised by their skills such as autonomy, flexibility, location, communication and distribution [1]. These skills organize the needs and skills perceived by agents to solve problems through the use of distributed artificial intelligence techniques.

The autonomy, the distribution and the opening present some important advantages to improve and evolve the current architecture of the Building Management System (BMS). This allows for, in distributed problems such as energy optimization in IB, the multi-agent architecture can be decomposed so that all the processes are managed in a more intuitive way and with less maintenance.

MAS makes it possible to emulate the actions and interactions of human organizations through the decomposition of problems. Jennings and Bussmann [17] argue that this allows the development of a more natural mechanism for problem solving, an attractive resource when current systems are built with high complexity and equipment is increasing. An organizational structure of agents imitates human interaction by providing agents with specific tasks and sufficient autonomy to perform those tasks successfully. The main objective is to decompose a problem in such a way that each agent acts autonomously and interacts with other agents. In supervisory control systems, on the other hand, the hierarchy is decentralized.

An example of this decomposition of the problem is that which occurs when a group of agents that monitor the comfort temperature of the user interacts with the heating, ventilation, and air conditioning (HVAC) system that heats or cools. In turn, the controlling agent of the HVAC system interacts with a service agent that collects past situations (case-based reasoning systems) to help configure the ideal temperature for the residents. This agent can be found in a Cloud system which allows for storing previous information, or obtaining future information such as weather forecasts. Thus, the decentralization of this type of problems offers us with many possibilities for structuring the solution.

2.2. MAS Technology

This subsection provides an overview of the main technologies employed in the development of multi-agent systems.

2.2.1. MAS Communications

The Foundation for Intelligent Physical Agents (FIPA) [18] was created with the aim of enabling agent communication for allowing agents to find each other, send and receive comprehensible messages between each other. The objective of FIPA was to make agent based systems more advanced. One of the first developments provided by FIPA was a set of standards and specifications that facilitated agent operations between different agent middleware. FIPA-Directory Facilitator (DF) is a centralized registry of entries that associates the descriptions of each service with the ID or ID’s of agent or groups of agents. FIPA proposed as standard communication language FIPA-Agent Communication Language (ACL). ACL acts to standardize a set of communication tools that have specific meanings and protocols between agents. ACL has origins in another communication protocol: Knowledge Query and Manipulation Language (KQML). KQML was a language and protocol for communication between software agents and knowledge-based systems developed by Defense Advanced Research Projects Agency (DARPA) and which was replaced by FIPA-ACL.

2.2.2. MAS Frameworks

To facilitate the design and modelling of agents, their behavior, the addition of AI techniques, and the development of communication protocols. Soon, different frameworks appeared to facilitate the development of proposals based on MAS.

One of the frameworks par excellence is JADE [19,20], a middleware that has been in use for more than a decade. This framework has a wide reference documentation that allows the development of agents in a very simple way for users with a moderate level of experience in Java programming. One of the advantages that JADE provided was its compatibility with FIPA in terms of communication between platforms, so this gives the possibility of using it together with other MAS middleware that shares compliance with FIPA. This was an advance over previous frameworks such as ZEUS [21]. JADE has broad support and implementation throughout the research community and even in the industry.

Python Agent DEvelopment framework (PADE) is a framework for the development, execution, and management of multi-agent systems in distributed computer environments. This framework allows for developing MAS in Python. This framework is oriented at objects, uses FIPA-ACL standard messages, uses the FIPA communication protocol, and is multi-platform so that it can be installed and used in embedded hardware running distinct operating systems such as Linux, Raspberry Pi, BeagleBone Black, as well as Windows.

Open Agent Architecture (OAA) is a framework for integrating a community of heterogeneous software agents in a distributed environment [22]. The OAA is useful for building complex systems in which there are many heterogeneous components, and in which flexibility and extensibility is important. It is certainly true that if you have a frozen system, this system could be built without using an agent architecture; however, if you want a system that is adaptable, and can easily be extended (incrementally), delegation-based agent architectures offer a new paradigm of solutions.

PANGEA is an MAS framework focused on the development of virtual organizations of agents [23,24]. PANGEA is better than other multiagent platforms because it provides a number of agents that encapsulate elementary functionality, such as data access, service discovery [25] or rule control, and supports standard interagent communication protocols such as FIPA-ACL [26] PANGEA incorporates agents that manage security at the system level, unlike other types of systems in which it is necessary to develop these types of measures so that the information collected is really the one that is transmitted and analyzed [27].

2.2.3. Methodologies for MAS Development

Because the MAS were conceived as a paradigm to respond to complex problems, the development of an MAS architecture is not easy. There are several methods that facilitate this trivial task. One might think that traditional object-oriented (OO) approaches can be suitable for this purpose. Agents employ some of the mechanisms and philosophies used by objects. In fact, many software developers strongly advocate composing agents from objects—building the infrastructure for agent-based systems on top of the kind of support systems used for OO software systems [28]. However, in MAS, an additional layer of software components may be naturally expressed as objects and collections of objects and use the communication, cooperation and coordination capabilities together with the characteristics of the agents such as the belief–desire–intention software model.

MASs publish a description of the services they provide in a page’s registry (Directory Facilitator in JADE framework) and are capable of communicating by means of shared messages on a ontology. In addition, mobility can be provided so that a running agent may migrate to a different device preserving its execution state. MAS specializes the oriented programming methodology. Agents’ architecture are easy to design and implement, when it must deal with computational intensive systems that may need to scale to cope with highly variable number of tasks, while OO is all about encapsulating state and associated behaviour together. In energy works, the use of states and understanding of the context (agent programming) is necessary so that the software entities (agents) react based on the behaviors, actions or beliefs with which they have been modelled. The most used methodologies in the main works with MAS according to the literature are GAIA [29], MaSe [30] and Tropos [31].

GAIA is a general purpose methodology based only an analysis (Decomposition of the organization in sub-organizations, Environment Model, Preliminary Roles Model, Preliminary Interaction Model, Rules of Organization) and design (Organizational structure: Topology and Control structure, Final Roles Model, Final Interaction Model, Agent Model, Service Model) phases [32,33]. GAIA has the ability to implement from the simplest solution to those with a high degree of complexity, as well as the analysis, design and release of the same. At GAIA, we understand that the objective of the analysis is to understand the system and its structure without referencing any aspect of implementation. This methodology only seeks to specify how a partnership of agents collaborates to achieve the objectives of the system, and what is required of each one to achieve the latter [34,35].

The main criticism that can be made of GAIA is that it remains at too high a level of abstraction. GAIA does not provide the tools for development and deployment that other methodologies currently incorporate. The MaSE methodology focuses on capturing goals, applying use cases, refining roles, developing agent classes, building conversations, assembling agents and finally designing systems. This approach is based more on the objectives. The Tropos methodology focuses more on popular belief, desires and architectural intentions and the definition of how agents create plans and goals.

2.2.4. MAS Ontologies

Being a complex paradigm, the MAS has ontologies that allow the formal definition of types, properties, and relationships between entities that really or fundamentally exist for a domain of discussion in particular. A well-defined ontology is necessary, beyond the definition made by the FIPA-Directory Facilitator (FIPA-DL). Several specifications have emerged in ontologies; however, none has gained wide acceptance. Web Ontology Language (OWL) is one of the most widely used standards in MAS.

The need to use ontologies is due to the inherent complexity of the applications developed in the context of Multi-Agent Systems, which causes the following difficulties: (i) Abundance of communication between agents; (ii) Interoperability of systems and platforms and (iii) Semantic problems.

For agents to communicate with each other, they must share the same language, vocabulary and protocols. By following the recommendations of the FIPA standard, JADE already provides a certain degree of coincidence in the use of FIPA communicative acts and their SL (Semantic Language) content language, which determine how messages are exchanged by agents. However, it is necessary to define specific ontologies, with their own vocabulary and semantics of the content of the messages exchanged by the agents.

2.3. Ubicomp with MAS

When dealing with the problems of energy optimization in IB, we are basically referring to the concept of Ubiquitous Computing (Ubicomp), so that a computer system is in the environment of people in a way that facilitates daily tasks but without being perceived. What is proposed with the MAS is basically that, to provide a system that is responsible for performing energy optimization tasks without being annoying to the user, even reducing the interaction of one with the other by incorporating learning techniques.

There was a great challenge for the incorporation of learning techniques, techniques of massive data analysis, etc. in the MAS. However, the most important challenge is the development of an MAS architecture that allowed its incorporation in almost any IB.

This challenge poses the design of a standard communication protocol with HVAC, for example. Until then, each HVAC system had its own commercial communication protocol, although a communication protocol could be developed, this was not valid for another HVAC from another manufacturer. This is one of the advantages of the MAS in the ability to encapsulate communication protocols. The different types can be applied according to the technology.

Currently for communication with HVAC systems, there are three open protocols that were quickly made with the leadership of the standard protocols for communication with HVAC: ModBus, LonWorks and BACnet. These protocols have a set of characteristics which make them suitable for different uses and applications [36]. According to a survey conducted on the Building Operating Management website in 2011 [37], 30% of respondents had at least one application using Modbus, 40% for Lon-Works and 62% for BACNet.

3. MAS and Energy Optimization: A New Approach

An optimization problem consists of finding the best solution, according to a series of criteria, within a set of possible solutions. Optimization algorithms are general procedures, which solve a problem if they produce feasible solutions when applied to the context of that problem. Optimization algorithms are exact if they find the optimal solution or heuristic if the solution is not necessarily optimal. Within the problems of optimization of energy, the ideal is to obtain an optimal solution; however, in those problems in which there is a human factor, it is necessary to apply heuristic methods to adapt it to the needs or characteristics that the user desires. To apply these methods, it is necessary to: (i) Identify the problem; (ii) Define and present the problem; (iii) Explore the viable strategies; (iv) Advance the strategies; and (v) Achieve solutions and evaluate the effects of the application of those solutions. Due to their characteristics, MAS have been recently used as promising heuristic techniques for solving problems whose domains are distributed, complex and heterogeneous. In this way, several solutions provide optimization within a set of rules that may vary.

On the other hand, the IBs are characterized for being buildings equipped with automation systems and control of lighting, heating, cooling or ventilation. It is necessary to control these factors as they are key in the optimization problem and some influence others. The optimization problem must meet its objective while satisfying the demands regarding aspects such as comfort, safety, connectivity, energy efficiency, operations and maintenance, complying with the regulations and current legislation. Intelligent Buildings cover all types of types, homes, shops, offices, industries, sports facilities, etc. In addition to multi-agent systems, other proposals have been made to optimize energy consumption such as linear programming [38,39], predictive models [40], decision models [41,42] embedded systems [43] and gradient models [44].

The following subsections review extensively the state-of-the-art proposals of energy optimization in IB using MAS. It presents in a profound way a review of the first works using MAS, evolution, characteristics, advantages and disadvantages within the four main focuses of work: simulation of communication, management of wireless sensor networks (WSN) for efficient optimization decision-making, demand response problems and energy optimization through the use of gamification techniques in MAS.

3.1. Demand Response with MAS

Demand response (DR) is a change in the way energy is consumed by the client of an electricity company, as energy demand is better adjusted to the offer. DR is a way to respond to cases of maximum energy demand so that users can be restricted from accessing all or part of their network energy consumption when they are asked to do so. This restriction of electric service in periods of broad demand provides a broad economic benefit to users who stop using it in these periods. The DR approach arose from the difficulty of storing electrical energy, which is why utility companies have traditionally matched demand and supply by slowing the production pace of their power plants, taking energy from other power stations or importing energy from other power companies.

There are limits that can be achieved from the supply side because some units can take a long time to come up to full power, some units can be very expensive to operate, and demand can be greater than the capacity of all the available power plants put together. DR seeks to adjust the energy demand instead of adjusting the supply. On the customer’s side, the elimination of network energy consumption can be done by restricting the consumption of energy-consuming equipment, and this does not prevent the client from having energy generators or the use of energy storage batteries. These actions of cutting or restricting the supply are taken at times of maximum demand, failure in the electrical network, maintenance of any installation, etc. To take these actions, it is necessary to know multiple factors and their parameters both on the side of the supplier (energy to be supplied, facilities, network, etc.) and the client (energy requested, installation, possibility of restricting the service, etc.).

Because of these characteristics, since they began to develop solutions in DR problems, proposals that used MAS as a methodology to focus them soon appeared. One of the first works was the one proposed by Praça et al. [45] in which the authors present a multi-agent based approach that simulates an electricity simulation environment including the most important aspects of the electricity markets restructuring process. This platform allowed for evaluating decisions within the electricity markets before making them and there were two different approaches: an auction and a direct negotiation market. In 2003, Praça et al. [46] developed MASCEM, a multi-agent simulation system in which agents represent market entities such as generators, consumers and market operators, and new entrants such as merchants.

It implies an evolution with respect to the previously presented works. Since they introduce a novelty that changes the focus within the DR works using MAS, the agents can establish their own objectives and decision rules. In 2011, Vale et al. [47] begin to use agents with basic learning techniques in the simulation process. However, until now, the proposed works lack reinforcement learning techniques and the inclusion of clustering to group periods that present similar tendencies.

In this work, it proposes, for future versions, the inclusion of a metalearner that combines the outcome of different predictive models to create the final result.

Table 1. Review of Demand Response’s work using Multi-Agent System (MAS): Part I.

Publication	Functionality	Agent Platform	Advantages	Shortcomings
Praça et al. [45] (2001)	This paper introduces a multi-agent simulation system for competitive electricity markets. This model considers auction based markets based on the rules from the Iberian Electricity Market (MIBEL) market operator. The considered agents are sellers, buyers and the market operator, which receives the bids from the negotiators, and sets the market price.	OAA	Very simple model, based on MIBEL market	Limited to simple auction based markets and basic seller and buyer agents
Praça et al. [46] (2003)	This work presents the first version of the MASCEM simulator, which includes auction based market and negotiation by means of bilateral contracts. Besides seller and buyer agents, which are the negotiating players, MASCEM includes also a market operator and system operator agents.	OAA	Based on MIBEL market, combined simulation of auction market and bilateral contracts	Limited to basic seller and buyer agents, limited scalability with OAA
Vale et al. [47] (2011)	This paper presents the 2011 version of the MASCEM simulator, which includes advanced decision support using machine learning for players’ negotiations. New types of players, such as aggregators are also present, as well as the models that support these aggregations’ formation and management.	OAA	Advanced decision support, introduction of aggregators, context-aware system	Limited scalability with OAA, limited to a few European market models, limited real-time access to real data
Oliveira et al. [48] (2012)	This paper introduces MASGriP—A Multi-Agent Smart Grid Simulation Platform, a smart grid simulator that incorporates agents that represent small players, such as different types of consumers, generators, electric vehicles, energy storage systems, etc. This system supports several energy resource management models, as well as diverse demand response programs.	JADE	Multiple entities, energy resource management models, demand response models	No interaction with the market, limited range of supported scenarios, limited use of real data
Gomes et al. [49] (2014)	This paper explores the simulation of smart grids, with focus on the assessment of different types of demand response programs. This system is connected to real physical resources, thus enabling to experiment with the impact of the simulations on real devices.	JADE	Connection to physical resources, variety of demand response programs, intelligent demand response management	Limited number of participating entities, no interaction with the market, limited range of supported scenarios

and

Table 2. Review of Demand Response’s work using MAS: Part II.

Publication	Functionality	Agent Platform	Advantages	Shortcomings
Santos et al. [50] (2016)	This paper presents the restructured version of the MASCEM simulator, which includes the migration of the agent platform to JADE, the introduction of ontologies to support communications between agents and enable the connection with external systems, including systems that simulate the smart grid.	JADE	Supports interoperability with external systems, fast execution times, good scalability	Limited to a few European market models, limited to the wholesale market
Faria et al. [51] (2016)	This paper addresses the definition of novel dynamic demand response programs. The introduced models are experimented by means of a multi-agent system that enables the interaction of these programs with different entities, different energy management models, physical resources, and real time simulation.	JADE	Connection to physical resources, new demand response programs, real-time simulation	No interaction with the wholesale market, limited range of supported scenarios
Gazafroudi et al. [52] (2017)	Optimize residential energy based on Time of Use (ToC) demand response program.	Proprietary MAS	Considering uncertainty of PV power generation based on a novel interval optimization method.	The proposed energy management system does not work in the real-time process. Control unit is not considered in the energy scheduler agent of the smart home.
Gazafroudi et al. [53] (2017)	Optimize residential energy based on Time of Use (ToC) demand response program and moving windows algorithm.	JADE	Residential energy is rescheduled every hour to update its managed energy. considering uncertainty of PV power generation based on a novel interval optimization method	Uncertainty of electricity price is not seen in this model. Control unit is not considered in the energy scheduler agent of the smart home
Gazafroudi et al. [54] (2017)	Optimize residential energy based on Time of Use (ToC) demand response program and flexibility of energy storage systems.	Proprietary MAS	Considering uncertainty of PV power generation based on a novel interval optimization method. Energy scheduler agent is modelled completely that consists of the prediction engine, energy scheduler, and control unit.	The proposed energy management system is not adaptive.

show the most active works in DR using MAS. In this table, the publication of each work is shown, the year of publication, MAS functionality, programming language, framework or platform used for the development of the MAS, advantages that the work provided with respect to the works developed and the disadvantages or shortcomings. From the review of this tables, it can be observed the work of demand response, which focuses on modelling and simulating the behaviour of users before certain actions of energy suppliers. Although Demand response is not focused directly and exclusively on energy optimization in the home, it is necessary to observe how this concept can model user behavior and transfer it to an energy management system in homes or buildings. Simulation of user behavior or how it affects the increase in the price of light among others.

==layoutwidth=297mm,layoutheight=210 mm, left=2.7cm,right=2.7cm,top=1.8cm,bottom=1.5cm, includehead,includefoot [LO,RE]0cm [RO,LE]0cm

==

3.2. Human Behavior Simulation in Intelligent Buildings

The field of simulation has been from the beginning one of the great focuses for which the MAS have been used. Therefore, within the area of energy optimization, they have been used to simulate Demand Response problems, actions of the administrators, user actions, consequences, etc. This involves the simulation of the relationship between the energy supplier and the customer, more specifically the making of decisions that provide energy and economic savings to the customers in their home and the supplier company the best way to manage energy avoiding energy peaks. In this case of Human behavior simulation in intelligent buildings, user behavior is simulated exclusively. These simulations focus on the presence in the home, and the decisions they make in it, such as the interaction with appliances, heating, lighting systems or electrical devices, and look for relationships as if users increase temperature, increase energy consumption or if a window is open, increase heating consumption in winters, thus obtaining useful information for development. In these types of simulations, it is also necessary to simulate the behavior of the inhabitants taking into account all the external factors that affect the decision-making process, such as the time zone in which the inhabitants live at home, the price of energy, climatological factors, etc.

The possibility of modelling the characteristics of any building was one of the incentives that also motivated its use. One of the first works was the one developed by Davidsson and Boman [55], which simulated how energy consumption is affected using a temperature and light optimizing management system in office buildings.

However, this work used several simplifications in the use of the system. The outdoor temperature was constant and set at 10 ${}^{\circ }$C and the preferred temperature for all users was 22 ${}^{\circ }$C. The system has a timer that began to raise the temperature at 7 a.m. to reach the desired temperature if there is a person in the building. Thus, the work simplified most of the factors and fixed its value. In 2002, for Davidsson’s study [56], each person was modelled by separate agents, which allowed for evaluating different behaviors for each inhabitant of the building. The MAS automatically detected in which room each person was in real time and adapted the conditions of the room according to the preferences of each person. This provided an advance over the previous work, since the preferences were different for each person; however, they still had many factors with pre-set values. These researchers evolved the proposal made in works such as [55,56,57,58] to a commercialised product. The purpose for the multiagent system is to dynamically control a system so that the load of the system is below certain threshold values without reduction of quality of service, and, by doing this, to avoid the usage of top load production sources and to reduce energy consumption. The fundamental idea behind the system is that a large number of small local decisions taken all in all have a great impact on the overall system performance [59]. This led to the launch of the NODA AB (https://noda.se/), which was a start-up from the use of MAS in managing utility power and heating.

As the integration of learning capabilities in agents evolved, they began to develop MAS that they used in simulation work, as well as DR problems. One of the first works that introduces agents with learning capacity is the developer by Hagras et al. [60], in which presented multi-agent architecture for IB adaptation and control using hierarchical fuzzy-genetic agents for learning resident comfort habits and adapting to them while an energy optimization is done. The next big jump that occurred in this field was the introduction of some means that allowed to control the parameters that were being set as the outside temperature. In this sense, Sandhu et al. [61] performed an MAS that allowed the simulation of the deployment of a Wireless Sensor Network (WSN) that allowed for monitoring these values and through an MAS to control the lighting factors through the WSN itself. This also required addressing the management of several interrelated areas, including system design, decision-making, machine learning, topology management, message management, etc. all through the MAS. However, the MAS only allowed for simulating all the aspects related to the management of data from the WSN, and aspects such as the exhaustive evaluation of the factors related to the user were left out, and also aspects related to privacy.

Wang et al. [62] introduces a Particle Swarm Optimization (PSO) algorithm to optimize the set points of the control system during system operations. MAS is used to minimize the main conflict in smart and energy-efficient buildings in terms of power and users’ comfort. Yang and Wang [63] introduces the interaction between the user and the building. The architecture has an agent specially designed to allow the interactions between the occupants and the environment by learning the behaviors of the occupants. The personal agent observes user’s actions on adjusting his/her preferences towards the environment and the information of outdoor temperature from the local agent as inputs for training. The user’s preferred temperature can be obtained through multiple ways: a simpler one is to ask a user directly for the preferred set point of temperature; another one is to observe whether the user has taken any action to adjust the set point for an HVAC system in a stable thermal environment: if they do not take any action, the current temperature is considered a comfortable temperature for recording. For less frequent users in the building, the personal agent sets the temperature according to the common learning preferences.

Currently, the work focuses on knowing the factors that have a greater impact on energy consumption, so that it is not necessary to simulate a wide range of factors that make the problem difficult and may not obtain adequate results. Darakdjian et al. [64] use the Principal Component Analysis (PCA) technique to efficiently identify the main parameters that influence energy needs. The PCA technique together with MAS was more commonly used in the classification of facial faces according to their gender and age [15]. PCA was found to be an efficient way to identify the main parameters that influence energy needs, which is a fundamental stage for improving models, optimizing building design and carrying out energy performance guarantees. Correlation matrices display information that is difficult to investigate, but contain quantitative data that provide information that is complementary to the PCA data.

In

Table 3. A human behavior simulation in Intelligent Buildings works review: Part I.

Publication	Functionality	Agent Platform	Advantages	Shortcomings
Davidsson and Boman [55] (2001)	MAS that monitors and controls an office building in order to provide services such as change the temperature and turn on/off the light.	April Language	One of the first works that uses an MAS to fit an office to the preferences of its user.	This work is a simulation, in which a fixed consumption is assumed for an outdoor temperature set at 10 ${}^{\circ }$C.
Hagras et al. [60] (2003)	A hierarchical fuzzy genetic multi-embedded agent architecture that utilises sensory information to learn to perform tasks related to user comfort, energy conservation, and safety. The agents are coordinated in a fuzzy way according to deliberative plans. The fuzzy rules related to the room resident comfort are learnt and adapted online using our patented fuzzy-genetic techniques.	Proprietary MAS	A novel online learning, adaptation and control algorithm based on a double hierarchical fuzzy-genetic system is presented. It adapted in real time, online accommodating new data and rules, as they become known to the system.	The system requires interaction with the user to learn. The system was only tested under very limited conditions in terms of possible changes in building occupancy, user preferences, etc.
Sandhu et al. [61] (2004)	The MAS consists of wireless sensor nodes located throughout the physical environment for purposes of sensing (light, temperature, and occupancy), actuation, and communication.	Proprietary MAS	MAS that introduces a sensor fusion and validation techniques for decision-making in WSN-based lighting control.	The authors point out the need to include machine learning techniques, adaptation to topology and the inclusion of privacy techniques.
Macal and North [65] (2005)	Introduce Agent-based modelling and simulation (ABMS) approach. to modelling complex systems composed of interacting, autonomous ‘agents’. Agents have behaviours, often described by simple rules, and interactions with other agents, which in turn influence their behaviours.	-	The different kinds of implementation approaches and environments have various strengths and weakness depending on the modelling questions of interest.	Although the work is extensive in the benefits of using MAS as a tool to model situations, it does not specify the application of this methodology.
Wang et al. [62] (2010)	A multi-agent intelligent system is developed for energy and comfort management by controlling the building temperature, illumination and ventilation.	Proprietary MAS	The MAS introduce a Particle Swarm Optimization (PSO) to optimize the set points of the control system during system operations. It can be integrated with Supervisory Control And Data Acquisition (SCADA).	It is a simulation that does not have any method or technique of learning user behavior.

and

Table 4. A human behavior simulation in Intelligent Buildings works review: Part II.

Publication	Functionality	Agent Platform	Advantages	Shortcomings
Klein et al. [66] (2012)	Multi-agent comfort and energy system (MACES) to model alternative management and control of building systems and occupants. MACES specifically improves upon previous multi-agent systems as it coordinates both building system devices and building occupants through direct changes to occupant meeting schedules using multi-objective Markov Decision Problems (MDP).	OpenGL	Energy management via data fusion and analysis for real-time monitoring, prediction, and control of energy management systems in modern buildings.	It lacks policies generated in the real environments. This will allow the evaluation of policies with actual energy and comfort results.
Zhao et al. [67] (2013)	MAS that use a CPS for increasing the energy utilization efficiency by decision-making control methodology. A detailed description of the physical aspect of CPS-enabled BEMS, the CEBEMS.	Proprietary MAS	There are significant reductions of thermal power generation, and at the same time the electrical power generation is met without constraints, showing an improved performance achieved.	Thermal demand of the chosen building is relatively constant compared with common commercial office building.
Sharafi and ELMekkawy [68] (2014)	A novel MAS approach is proposed for optimal design of HRES including various generators and storage devices. The $ϵ$-constraint method which is a non Pareto-based search technique has been applied to minimize simultaneously the total cost of the system, unmet load, and fuel emission. The idea of this approach is to minimize the total cost while CO${}_2$ emission and unmet load are considered as constraint bound by permissible levels.	Proprietary MAS	PSO simulation based technique has been used to handle the developed multi-objective optimization problem.	A detailed load analysis must take into consideration the heat and electricity loads separately in order to recommend an optimum configuration.
Saba et al. [69] (2017)	An MAS architecture is proposed for the modeling and the simulation with the objective of ensuring energy economies in a habitat. The work done clearly demonstrates the benefits of using artificial intelligence and modern information and communication technologies for energy management.	Proprietary MAS	MAS model that allows the incorporation of different IA techniques.	It is only a contribution, an implementation must be carried out to evaluate the proposed solution.
Darakdjian et al. [64] (2018)	This work demonstrates that data mining on building performance simulation results can identify inputs, including occupant behaviour, which should be carefully defined, with a view to guaranteeing energy performance.	Proprietary MAS	Occupant behaviour models were integrated into the MAS in order to co-simulate BPS EnergyPlus with the platform.	They identify the factors that present the greatest energy consumption in the building as opening windows, but do not make recommendations.

, it is observed how the use of the MAS employees for the simulation has evolved as the initial gaps in obtaining environmental data were resolved. Once sensor technology was used, an advance was made in user behavior learning techniques to achieve a balance between energy efficiency and user comfort. Each new advance towards more complex simulation problems when a great amount of parameters are intervening in this sense begins to incorporate techniques like PCA to reduce the number of factors to only those that show relevance in this problem. Even so, these dimensionality reduction techniques still do not achieve the desired results in terms of effectiveness, and therefore research is still underway in their application [64].

3.3. WSN Management MAS for Optimization Decision-Making

This area began once, instead of performing simulations of energy optimization problems, obtaining the values of these parameters was done thanks to the deployment of a sensor network. In this aspect, an MAS performs the management of data capture processes, communication mechanisms and data analysis by deploying different agents that take care of all the functionality required by a Wireless Sensor Network (WSN).

The first works were multi-agent monitoring systems and were very basic and practically focused on modeling the Proportional integral-derivative (PID) process [70]. Some, such as the one developed by Sharpless et al. [71], used an MAS to simulate in a real building how a set of sensors would act. The sensor information was used to control heating, lighting, appliances, security and alarms). However, this work lacked dynamic behavior on the part of the agents, setting the rules of behavior of the users of the building.

Therefore, the next step in the progress should be made in the MAS focused on consensus decision-making between the optimization of energy in an IB and the comfort of its users. The parameters previously set in the simulation MAS as the climatic conditions or the presence of users can now be acquired through the deployment of sensors, and the MAS through the sensor network can obtain their values in real time. Quickly, the deployment and management of these networks began to be simulated. First, Sandhu et al. [61] performed an MAS that allowed the simulation of a WSN deployment that would allow for monitoring these values and controlling the lighting factors through an MAS.

Although there already existed a wide range of research works focused on the management of information from sensors and how to manage it online through an MAS based platform, it was necessary to know precisely the most relevant information to be monitored. One of the works that addresses this problem more clearly is the one carried out by Nguyen and Aiello (2013), which makes an important classification of the most relevant elements to be monitored: occupation, preferences, occupation prediction and detailed activities [72]. The occupation in real time in the rooms is the information that is taken as the basis of multiple works. Therefore, in [73], lighting is controlled according to occupancy; in [74], the temperature is automatically reduced when there is no occupation; and, in [75,76], both are combined, controlling lighting and air conditioning depending on the occupancy of the meeting rooms. The previous works have allowed us to know exclusively how the inclusion of user preferences greatly enriches the knowledge in terms of occupation to improve the energy efficiency that these systems obtain.

While it was going to be managed in a more optimized way, the use of sensors to capture data that really provided useful information on where to direct the design of MAS to obtain a reduction in energy consumption, it began to apply techniques of gamification. Some of the pioneering works in the use of gamification techniques were those proposed by Tuyls et al. [77,78] in which they used Reinforcement Learning (RL) to model the learning of the agents. These RL algorithms are sensitive to the correct choice of parameters so it is necessary to make a correct choice of parameter settings. The gamification techniques helped the MAS for the correct choice of these parameters. However, what is really intended with the incorporation of gamification techniques in the optimization problem is to model this problem as a game, where the inhabitants of an IB obtain a benefit if they realized a lower consumption of energy and are punished if they perform a excessive consumption of energy. Some of the works that have recently addressed these objectives are those made by Garcia et al. In [79], he develops Context-Aware Framework for Collaborative Learning Applications (CAFCLA) framework that allows for integrating multiple technologies , which provide both context-awareness and social computing, in which the theory of games is used to encourage users to reduce energy consumption in their workplace and through social interactions and competitiveness accelerate the achievement of good results and changes in behavior that favor energy savings. Each user of the office in which the case study was developed should minimize their consumption of light, electricity and consumption of the HVAC system. One of the actions that was rewarded was to climb up the stairs instead of the elevator. In [80], an evolution of the CAFCLA framework was made to be used in homes, so that energy saving benefits such as those presented in the office building could be obtained. However, this work was a theoretical study in which the modifications that the framework had to have for its adaptation to real homes were proposed. It is already in [81], where the use of the CAFCLA framework as a recommendation system in homes is evaluated. One of the points where the system evolved most was the integration of a Real-Time Localization System in to Wireless Sensor Networks, making it possible to develop applications that work under the umbrella of Social Computing. This system obtained an average energy saving of 17% and in some cases up to 22%, while until then the leading jobs obtained savings between 10% and 15%. In

Table 5. Wireless Sensor Network (WSN) management MAS for optimization decision-making: Part I.

Publication	Functionality	Agent Platform	Advantages	Shortcomings
Spears et al. [82] (2004)	An MAS system to build sensor network systems of autonomous distributed mobile sensing agents to analysis of aggregate sensor systems.	Proprietary MAS	The agents sense and react to virtual forces, which are motivated by natural physics laws. Thus, physicomimetics is founded upon solid scientific principles.	MAS systems whose simulations are based on factors whose values do not vary over time.
Tuyls and Nowé [77] (2005)	This paper surveys the basics of reinforcement learning (RL) and game theory in MAS. Introduce an overview on the fundamentals of RL. Summarize the most important aspects of evolutionary game theory and discuss the state of the art of multi-agent reinforcement learning and the mathematical connection with evolutionary game theory.	Proprietary MAS	Introduce reinforcement learning and evolutionary game theory approach.	Overcoming problems of incomplete information and large state spaces in MASs (for instance, sensor webs) are still difficult.
Nguyen and Aiello [72] (2013)	This work made a great contribution to the work related to user behaviour. It did a deeper search on factors that affect the energy consumption, the methodologies to analyze these variables in order to get an energy saving potential.	-	Authors devise new metrics to compare the existing studies. Through a survey, they determine the most valuable activities and behaviours and their impact on energy saving potential for each of the three main subsystems, i.e., HVAC, light, and plug loads.	When this study was carried out, the switching on and off of lights, heating systems, etc. was not domotized. It is pointed out that it is necessary to incorporate sensors to perform certain actions according to user behavior.
Konstantakopoulos et al. [83] (2014)	We describe a social game that we designed for encouraging energy efficient behavior amongst building occupants with the aim of reducing overall energy consumption in the building. The social game consists of occupants logging their vote for the lighting setting in the office and they win points based on how energy efficient their vote is compared to other occupants.	Proprietary MAS	One of the first works that uses game theory to motivate users to change their behaviour towards better energy efficiency habits.	The social game was simulated, which allows us to obtain relevant results of change towards more efficient habits but should be carried out in a real environment using real user behaviour and real incentives.

and

Table 6. WSN management MAS for optimization decision-making: Part II.

Publication	Functionality	Agent Platform	Advantages	Shortcomings
Reka and Ramesh [84] (2016)	An MAS solution for evaluating different situations and to take decisions to improve the current state of the system. Agents mediate the trade-off between energy savings and occupant comfort, which for non-residential buildings directly correlates with productivity. The agents reach their goal through a process of negotiation based on energy and comfort costs.	JADE	Introduce a negotiation process modelling by an MAS in order to get a consensus between user and comfort.	Parameters in terms of climatic conditions affecting the building and comfort preferences are created artificially.
García et al. [79] (2017)	A virtual organization of agents for the development of context-aware serious games that spark a behavioural change towards more energy efficient habits.	PANGEA	The main novelties are the use of WSN and RTLS in a public building, to deploy an individual and at the same time, collective game between users. The management of this game through the CAFCLA framework and a virtual organization of agents, to improve game development and deployment, and the integration of technologies to enable higher customization and collective interaction.	The work shows how some aspects such as the optimization in the use of energy in the job site has not been maintained as expected, since the level of savings are lower compared to those that were obtained during the performance of the experimentation.
García et al. [81] (2017)	The work presented in this paper deals with the benefits of monitoring, precise localization of home residents and real-time context-awareness, which offer possibilities for the development of systems and applications that foster energy efficiency. Those technologies also present a great potential for the development of Social Computing tools which would promote energy saving and good energy consumption habits. The main feature of this type of solutions is the collection of data through sensors.	JADE	This work presents the CAFCLA framework (Context-Aware Framework for Collaborative Learning Applications) which combines WSN and Social Computing to develop activities for users rapidly and effectively, helping them to change their energy consumption habits. The framework, designed from the perspective of Social Computing, is used as the basis for the implementation of applications that include context-aware, localization, and social functionalities.	The accuracy of the system makes its implementation in homes very expensive for users. In this regard, it is necessary to improve the accuracy of localization using more standard devices, such as mobile phones.
Sangi et al. [85] (2017)	An agent-based control of building energy systems in the modelling language Modelica is developed, implemented and finally validated in a case study. This library allows "plug-and-play" implementation of MAS into any model of a building energy system in the Modelica environment, thus allowing the investigation in agent-based building energy system control through dynamic simulation.	JADE	The results show that the MAS is able to maintain a system variable within margin while reducing the effort in accordance to an optimization goal (in this case fuel price).	The functionality of the system was validated in the simulation. However, the MAS can also be used to control real-life applications because the agents communicate to each other beyond the border of the simulation software framework as they use a UDP/IP protocol to communicate via an Ethernet network.
González-Briones et al. [86] (2018)	The system uses a combination of information from the habits and preferences of inhabitants and the variables that influence energy expenditure to achieve energy savings.	JADE	Acquisition of information related to the inhabitants in a non-intrusive way thanks to the use of sensors or a CBR system, conjunction of the current indoor and outdoor temperature together with the forecast temperature to prevent temperature jumps of the HVAC system that make the energy consumption increase strongly.	This system does not take into account the comfort preferences of users (lighting and heating).

, a review is made about the evolution of the MAS focused on the taking of data from sensors, the analysis of this information collected, and the decision-making, so that an energy optimization is obtained in IB.

==layoutwidth=297mm,layoutheight=210 mm, left=2.7cm,right=2.7cm,top=1.8cm,bottom=1.5cm, includehead,includefoot [LO,RE]0cm [RO,LE]0cm

==

4. An Energy Optimization MAS Proposal

In Section 3, a review was made of the application of multi-agent systems in the three main areas of energy optimization: Demand Response, Human Behavior simulation and WSN management for optimization decision-making. Although they may seem three completely independent areas, the MAS are perfectly suited for the development of solutions in each of them due to the ability to communicate, cooperate and adapt to the environment of the agents. It can be seen how the capabilities to simulate contexts has had a wide acceptance in the community of researchers focused on the development of proposals in the field of energy efficiency. It can be observed how there are common points in the three areas, and how the evolution in each one of them fed back to the rest, as when the MAS adopted learning techniques.

The review of the main research works, developed to obtain a reduction of the energy consumption by the IB users, has revealed a series of possibilities for the adoption of measures that have provided great results such as the simulation of the behavior of the users (mediating techniques of learning and knowledge), the management of the information of the context in which they act (through sensor networks) and rewarding good behaviors (social computing and gamification techniques).

4.1. Architecture Design Concept

Therefore, in this section, we present an MAS-based architecture that allows the simulation of measures that obtain an energy optimization. For this, the synergies of the previous technologies will be used with the aim of providing the user with a lower energy consumption without sacrificing comfort, all in a non-invasive way. The system should focus on three aspects: (i) knowledge and learning; (ii) analysis, adaptation and communication with the environment; and (iii) decision-making.

Knowledge and learning. The agents that grant the knowledge capacity must focus on two aspects: knowledge of the user and knowledge of the environment. Knowing the user means knowing all the aspects of the user that affect the consumption of energy, such as tastes in terms of comfort (temperature), preferences, habits, and timetables in the building to model actions. As for knowing the environment, it is necessary to know the exterior temperature of the building, solar incidence, weather forecast, etc.

Analysis, adaptation and communication with the environment. The analysis of the previous data will provide information that the MAS must use to be adapted to the environment in which it is deployed (this allows the system to be context-aware). The self-adaptation capacity of the MAS allows that, regardless of the context, the system behaves in a deterministic way and allows the user to control and detect external consumption factors that influence energy consumption, without having to make complex configurations. By having agents with adaptability, agents can be developed that implement standard communication protocols allowing communication with the HVAC system (the system that generates the most energy consumption in a building), electrical appliances and lighting. The system completely independently interacts with these devices, through the knowledge obtained from the user, as well as other external factors that influence consumption. To communicate with the environment, send and receive information from it, it is necessary to deploy a WSN. The deployment of a WSN that can be deployed in a multi-storey office building presents problems that in a domestic installation do not appear as the ability to observe dynamic scenes of a set of sensors by controlling their configuration, changes in status or sequencing and programming of assigned resources. The system will manage these types of problems by deploying agents that use Information Fusion (IF) techniques. They are in charge of the association, correlation and combination of data or information from individual or multiple sources in order to obtain more precise data and estimates. This facilitates the management of the WSN user since it will not have to perform complex configurations.

Decision-making. It is the point that will equip the MAS with greater intelligence to achieve the goal of optimizing energy with the highest possible result. The deployment of agents that are able to work with the information acquired in previous stages to communicate with the HVAC system (reduce or increase temperature), appliances (on and off), lighting (on, off or attenuation). There is not a variety of algorithms in this aspect, so it must be the researchers themselves who develop algorithms that combine optimization and comfort. For this, they must give greater weight to the factors they consider appropriate. However, thanks to the review of the state of the art that this article has presented, the system will have an agent that applied the PCA technique that was already used by Darakdjian et al. [64] to efficiently identify the factors that have the greatest incidence in the energetic consumption.

4.2. Technical Development of the Proposal MAS

For the conception of technical development, the GAIA approach has been used, since, as it has been seen in Section 2.2.3, this methodology focuses on the analysis and design of software systems based on intelligent agents. JADE framework has been used due to the Java knowledge of the authors, and to the wide research community that supports it, facilitating the solution of the problems that arise in the development. As we have learned from the main jobs that use MAS, it is very important to decompose the problem into subproblems. In our case, these subproblems are those previously indicated: knowledge and learning, analysis, adaptation and communication with the environment and decision-making, for which five organizational structures have been created, called layers. Each of these layers formed by different agents encloses the functionality of the system. This organizational structure is made according to the affinity of the activities that each of them performs, as shown in Figure 1.

• Simulation Layer: Layer that allows us to simulate the behavior of the system in an IB. This layer allows for establishing the terms according to which the optimization of energy will be performed, using the Energy Optimization/Comfort agent, if the energy optimization or user comfort predominates. Context Agent collects the characteristics of the environment (IB) in which the system has been deployed. PCA agent allows us to set a series of parameters on which the system will base its optimization decisions, and this agent obtains the parameters that most influence to improve the results. Weka’s Java Principal Components class was used for the development of the PCA algorithm. This layer has agents that implement AI techniques as the agent that implements a case-based reasoning system (CBR). This system allows us to collect previous optimization cases that allow us to obtain an optimization of the consumption (Retrieve step), as the context may have undergone variations, the case is modified to adapt it to the problem with the current characteristics (Reuse step), a revision is produced that prevents actions in the workflow (Revise step), and once it has been used with the correct results, the case is stored for later uses (Retain step). The simulation agent allows for executing the simulations by coordinating the rest of the agents of the system. It also offers the possibility of knowing the consumption data and seeing that it is collected in real time by each sensor of the WSN.
• Manage Workflow Layer: this agent is in charge of coordinating the rest of the agents that make up the system, establishing the communication between different parties to allow the correct order for the activity of each agent. The inclusion of this agent responds to the need to automate actions that respond to frequent events.
• WSN Data Acquisition Layer: this layer is responsible for collecting data from the environment. The agents of this layer obtain external temperature, interior or presence data, and it also includes the IF agent. The Data Acquisition Agent allows you to set the frequency of data collection and establish communication with the sensors through a middleware. In the present development, BACnet has been chosen for its perfect adaptability to HVAC systems.
• Knowledge Information Layer: this layer that has the knowledge required to model the optimization activities. This layer obtains data from the WSN Data Acquisition Layer, regarding temperature and presence, but, in this layer, values are obtained about the work calendar (it allows for identifying a pattern in the behaviour of the user). User Behavior Analysis Agent analyzes the behavior patterns of the IB users, establishing a schedule for each person (time of departure from the building, time of entry, periods of stay). It has an agent that obtains the weather forecast so that the system knows the difference between indoor and outdoor temperature that affects the comfort of the user. Gamification Agent is an agent that together with User Behavior Agent will allow for knowing the amount of energy that each user spends, and is focused on pressing good behaviors and bad ones (it will prevent the ignition of certain devices through the WSN).
• Action Layer: this layer makes decisions based on the information received from the other layers. It is formed by the agents that configure the HVAC system (heating or cooling), lighting (ignition, shutdown or attenuation of the light) and appliances configuring agent (ignition or shutdown of appliances) and configuration of consumption mode (of the appliances that have it). Figure 2 presents an exchange of data between the agents of this layer in the proposed MAS, developed with JADE.

The purpose of the article is to review the state of the art about multi-agent systems that provide energy optimization in homes. However, for more details about the implementation of the proposed architecture and the algorithms that allow energy consumption optimization to be achieved, the authors refer to the article where it is detailed in depth [86].

4.3. Results

The purpose of the proposal is to validate the coupling of different technologies used in the three energy optimization areas reviewed in the state of the art. The multi-agent system integrates different methodologies in simulation concepts, data management from WSN and learning behaviour patterns. The MAS has been used to obtain the energy consumption data that would be obtained if the MAS were to be deployed in an IB—72.46 m${}^2$ in a household simulation, shown in Figure 3, in which a family of four people in the household (two parents and two kids) lives. This figure shows how the WSN sensors are displayed in the house.

The experiment consisted of two phases. In the first phase, called the baseline period, the proposed MAS has monitored the electrical consumption, learned each person’s timetables at home and recorded the outdoor temperature (${}^{\circ }$C), indoor temperature (${}^{\circ }$C) and lighting on the dwelling (lm) of each day to generate the cases used by the CBR. The second phase, called the simulation period, has consisted of recreating the conditions of the month of the baseline period in order to evaluate the efficiency of the proposal MAS. In the second phase, the conditions of the first phase have been recreated so that the system, under the same conditions, makes the optimization decisions made by the Decision Making Agent.

The Decision Making Agent makes decisions according to the users’ comfort knowledge. In the first stage, it has known under which climatic conditions the users have modified the temperature of the HVAC system. The agent that implements the CBR system has been able to predict the presence or absence of people in the home and, based on this, adjust the temperature of the HVAC system. This temperature setting consists of lowering the temperature when there is no one in the house and increasing it progressively to have the value that the inhabitants want when they return home—switching off appliances when there are no people in the house (except the washing machine and microwave) by using data from the displayed sensors (presence, time spent in the home) together with data provided by calendar DB agent. This agent connects to databases from which it obtains that days are non-working or non-academic. The cooperation between these agents makes it possible to know which days the inhabitants may or may not spend more time on the floor. Thanks to these variables, the agent that implements the CBR will obtain cases that allow for making decisions such as the connection or disconnection of appliances and the temperature adjustment of the HVAC system. This information as seen in the work that has been part of the state-of-the-art review is not included in the energy management systems proposed so far in the literature. PCA agent has not been required by the system because very few dimensions of the problem (factors) have been addressed.

The architecture has the required mechanisms to obtain the environment data through a WSN. However, both weather and presence data have been pre-established in the case study. Decision-making has also been simulated, although the architecture is designed to provide communication with the HVAC system by means of the standard BACnet protocol, which will allow the increase, decrease, and turning on or off of this system. The connection of the MAS with the devices has been simulated using the IoT device, Cloogy [87], which allows the connection or disconnection of the devices.

One of the innovations introduced by this proposal, in addition to the conjunction of several technologies, is the incorporation of the agent that implements the CBR system that allows learning behavior patterns, along with the other variables measured. This translates into energy savings through knowledge of floor schedules and learning the schedules for programming intelligent thermostats [88]. The incorporation of the learning capacity of MAS agents allows a deeper analysis of the data, and performs it automatically. Before carrying out a new analysis, the system checks whether the data collected are the same as those collected for any of the analyses previously carried out. If so, that case is recovered and the actions are carried out automatically.

All this has allowed us to verify the effectiveness of our system by simulating the case study. In the baseline period, we obtained a consumption of 26.19 Wh. Recreating the same climatic conditions and user behaviour (same entry, exit and residence time), four simulations were carried out, in which MAS achieved a maximum saving rate of 20.58% in the fourth simulation regarding the consumption of the Baseline period (26.19 Wh). The CBR in each simulation stores more cases that allow you to better adjust the temperature of the HVAC system and make decisions for switching on and off devices connected to the grid. The results of each simulation are shown in

Table 7. Results of energy consumption in the different periods of the experiment.

	Baseline Period (Wh)	Simulation Period (Wh)	Difference (Wh)	Savings (%)
Simulation 1	26.19	22.33	3.86	14.73
Simulation 3	26.19	22.09	4.10	15.65
Simulation 2	26.19	22.01	4.18	15.96
Simulation 4	26.19	21.85	4.34	16.57

.

As shown in

Table 8. Results of the Student’s t-test and Levene’s test performed in the four simulations. Difference of means (electrical consumption in kWh) and variances between the baseline period and the evaluation period of the four simulations

	Baseline Period		Evaluation Period
	Mean (kWh)	Stdr. Deviation (kWh)	Mean (kWh)	Stdr. Deviation (kWh)	t	Sig. (2-Tailed)	F	Sig.
Simulation 1	26.19	1.56	22.33	1.83	14.660	0.000	10.632	0.001
Simulation 2	26.19	1.56	22.09	1.60	16.776	0.000	4.854	0.029
Simulation 3	26.19	1.56	22.01	1.85	15.763	0.000	9.999	0.002
Simulation 4	26.19	1.56	21.85	1.64	17.509	0.000	5.510	0.020

, the energy consumption is notably lower in the case study. We can see how the p-value is always under 0.05, which ensures that making decisions to reduce the temperature (if we are using heating) or turn off devices or appliances when the user is not at home provides a reduction in energy consumption.

The results obtained by the proposed architecture are comparable to the results obtained in other works such as [86] (where 41% of energy optimization was obtained, but the algorithm was specialized in the behavior pattern of the inhabitants of the house) and [42] (where the system presented obtained a reduction of 17.15% but was focused on obtaining an agreement between the users and the system to satisfy the preferences of the users and at the same time perform the process of energy optimization).

5. Conclusions

This paper has reviewed the state of the art of the multi-agent systems used to optimise energy consumption. Within this area of energy optimization, MAS has been used mainly in three areas: demand response, human behaviour simulation in intelligent buildings and WSN management for optimization decision-making. These three areas have evolved as MAS have evolved. MAS were used to simulate problems in these areas.

In the area of Demand Response, MAS modelled the entities involved in the process (energy grid and user) and how they interacted as the energy demand was best adjusted according to supply. MAS simulated the restriction of network energy consumption and how it affected customers and what benefit it produced. The MAS were developed by carrying out simulations in which a greater number of parameters of electricity market (price), supplier (energy to be supplied, installations, network, etc.) and the customer himself (energy requested, installation, possibility of restricting the service, etc.) were taken into account. Negotiation models started to be included between the electricity supplier and the customer. In the area of Human behaviour simulation in intelligent buildings, it simulated how to reduce the consumption of electricity in the home, depending on the optimization actions to be carried out. This allowed as in DR to simulate aspects of negotiation between the degree of energetic optimization and user comfort so that the electrical consumption is reduced without reducing the user’s comfort sensation or there is just a little reduction in user’s comfort.

In these simulations, they took into account how external factors such as the hours of the inhabitants of the house, the price of energy, climatic factors, etc. were affected. The work carried out showed very promising results, and soon MORE focused on translating these simulations into real case studies began to be developed, and the factors instead of being simulated began to be acquired through the deployment of WSN. Once MASs incorporated learning-capable agents, MASs gained greater acceptance to simulate or monitor and control real-world case studies, as they could learn user behavior. This learning together with the intrinsic characteristics of the agents in the concept of self-organization and self-adaptation allow the system to make decisions of energetic optimization in a non-invasive way for the user, and the user does not perceive any change in their habits.

As for the multi-agent overview of the state of the art, there is a lack of automation of these systems that has been solved in recent years with the emergence of standard protocols such as Building Automation and Control Networks (BacNET) and Digital Addressable Lighting Interface (DALI). There is a detailed study about the factors that influence the energy consumption, both climatic and the habits of the users. There are no systems that incorporate learning techniques to learn from users’ habits and respond to them with decisions that produce more efficient optimization. There are no systems in place that allow comfort preferences to be negotiated in ways that are more comfortable and more economical for the user (by reducing energy consumption). It should also be noted that there is a lack of research focused on optimising an office building or a household at a global level. As for the methodology used for the development of the MAS of the state-of-the-art works, the vast majority of GAIA has been the selected methodology.

An MAS has been developed to verify that a multi-agent architecture can be adapted to any context. The MAS developed allows for generating different solutions, which are applied in a simulated way, which allows knowing if these decisions are adequate for optimizing the energy consumption. These decisions allow the MAS to identify which factors allow for greater energy optimization and must be taken into account. The use of an MAS in this energy management problem has been fundamental, since it has provided us with an easy way to simulate the monitoring of the devices with the highest energy consumption of the building (HVAC system, appliances and lighting). It allowed for knowing how users interacted with the devices and what external factors influenced consumption. The simulations carried out in the case of use have given us very encouraging results for the complete development of MAS in terms of WSN management that we would need to deploy in a real case. In the case study, an average reduction of 20.58% has been obtained thanks to the proposed MAS. The results could be improved by including gamification techniques, which will be introduced in the next version of the MAS. In future versions, algorithms will be developed to make more complex optimization decisions that allow greater savings, not only the reduction or increase in temperature according to the presence of people in the home.

To conclude, MASs are going through a constant evolution and, thanks to their multiple characteristics, they are a very suitable approach to modelling systems in the field of energy optimization. They allow the simulation of different conditions, in different contexts and under uncontrolled conditions. If the simulations show adequate results, a complete MAS can be designed that manages a WSN and allows translating the actions taken in the simulation to a real context, providing very similar results.