• Ontology-based agent communication design

According to FIPA [107], semantic MAS interaction can be specified with three dimensions: 1) Internal agent behavior: action selection and execution; 2) External (agent) interaction to exchange: a) content of the interaction including both information and tasks; b) context of the Interaction and its relation to an agent organization; 3) System, or platform, services: message transport, discovery, action execution, management, and inter-platform interaction.

The FIPA (agent interaction) model (often referred to as the FIPA-ACL) is an Agent Interaction Protocol Suite (AIPS). The AIPS contains several distinct semantic protocols for agent communication including interaction process, communicative acts, content logic, and content ontologies (shown in Figure 7) [107].

**Figure 7.** Foundation for the Intelligent Physical Agent (FIPA) specifies multi-agent systems (MAS) interaction using specifications for an Agent Interaction Protocol Suite (AIPS) and MAS platform [107].

The design of internal agent behavior and interaction in a MAS mainly concerns the agent communication models as in the majority of the selected literature. The design of agent communication usually includes messages (message content) and message exchange (protocol). Messages and protocol are usually described in the UML diagrams as class diagrams and sequence diagrams, e.g., in [20], the communication sequence and communication parameters are introduced. The content of a message comprises two parts: content language (provides the syntax or grammar of the content) and ontology (consists of the semantics or lexicon of a message) [91]. The ontology-based agent communication model can be shown in Figure 8.

MAS developers usually use JADE to create agents because JADE agents communicate by exchanging message in compliance with the FIPA ACL. The FIPA Semantic Language (FIPA-SL) is popularly adopted as the standard content language [123]. In FIPA-SL, an ontology comprises a list of concepts, predicates, and actions specific to the domain of communication. However, the structures of ontologies in the selected literature are different. For instance, the ontology in [124] is defined in the form of EBNF and includes seven parts (policy, modality, trigger, subject, behavior, target, and constraint), and the ontology in [20] contains four parts (ID, type, parameter, and value).

When designing a MAS, developers usually introduce the syntax and semantics of the domain ontologies and application-specific ontologies applied in the MAS and describe the purposes and functions of the ontologies. For instance, [124] applies a policy ontology in their MAS. In [124], the policy ontology regulates behaviors of agents including application activity, authorization activity, monitoring activity, requesting-monitoring activity, discovery activity, and negotiation activity. This research designs a policy engine within each agent who is the subject of obligation policies or the target of authorization policies and the policy engine interprets and enforces the policy when the policy is enabled.

**Figure 8.** The illustration of the ontology-based agent communication model [125].

The FIPA agent standards focus on specifying protocols for external interaction and platform services rather than on the internal agent behavior [111]. It is because the internal agent behavior is are often problem-specific or application specific, and not easily accessible and observable. In the FIPA Ontology Service, an ontology agent is recommended to provide a number of ontology-related services for solving the problem of using multiple ontologies [91].

However, this solution is difficult to be implemented due to challenges of the system integration including between-ontology mapping, translation mappings, etc. Therefore, [91] recommends defining a common upper ontology that represents the general concepts used in the domain of power system. Meanwhile, related common standards in a domain can serve as a foundation for an upper ontology, e.g., The power systems Common Information Model (CIM) [126]. The upper ontology for the MAS interoperability of the electricity markets and demand side is well discussed by Santos et al. [3,47,48].

#### **4. Discussion**

The literature shows that there is an increase in MAS application in energy domain since the distributed nature of MAS allows the energy system design to deal with complex systems [127]. In a MAS, complete knowledge about the system is not required, but each agent in the system acts autonomously toward some predefined objectives to optimize the system performance [128]. Therefore, agents have possibilities to represent different market participants, network components, or systems [9]. The agents' individual goals decide the agents' behaviors, e.g., either cooperate or compete with other agents [127]. The behavior of the overall system is a result of the agents' behaviors.

MAS is not necessarily a simulation tool, but simulations may be important for the study of the energy domain, e.g., scenario comparisons, evolution studies, and sensitivity analyses. Several MAS studies are found in the literature dedicated to the energy domain. For instance, the study of Koritarov [1] demonstrates the application of the EMCAS in electricity markets. The model enables the investigation of the physical infrastructure and the economic behaviors of the market participants. The study of Li et al. [39] demonstrates the AMES simulation for the wholesale operations and market participates strategies. In the building sector, the simulations of electricity consumption in an office building are simulated by Mousavi et al. [53]. The study considers the unpredictable nature of business processes. Meanwhile, the research by Zeiler and Boxem [16] simulate the grid conditions in their study of building control.

All these simulations aim to solve problems in specific domains and are limited to an existing system (do not allow for connections to external systems) or do not take advantage of the formal exchange of knowledge. It is possible to solve problems that cover more complex domains if these systems can communicate and exchange knowledge with each other.

The combination of different systems can simulate a complex system such as the energy system. In such a system, stakeholders work together, interact, and negotiate with each other, while the demand and supply of resources need to be managed. The heterogeneity among these systems makes the interoperability complex, and the system may have different domains, concepts definitions, programming languages, etc. In order for the MAS to be able to communicate with each other and overcome their individual limitations, a mechanism for communication is important. This mechanism should allow information and knowledge sharing. At the same time, the system should be flexible to deal with several processes. Therefore, a communication standard should be defined, ensuring that agents in the system use terms with the same meanings [129].

The FIPA is the de facto standard for agent development [9]. FIPA provides different interoperability standards, e.g., the standard agent communication language (FIPA-ACL), which make it possible to integrate different MASs [130]. However, it does not mean that agents belonging to different MASs can share any useful information if the MASs use different ontologies. The ACL provides a framework for the communication standardization between agents, but the standard only defines the structure of messages and interactions. Therefore, agents speak the same language but do not share the same vocabulary.

In an ACL, the content of messages must be understood by agents for the messages to be meaningful. Catterson et al. [91] describe it as " ... the structure and meaning of the content are in a format expected by the receiving agent so it can decode the sender's intentions". Agents exchange information to achieve their goals and therefore must apply the same language to interact with each other. But it also needs a common representation of concepts for agents, which ontology can provide.

The ontology describes the concepts and the relations among agents and therefore must be a part of each agents' knowledge base [131]. Ontology is described as a form of knowledge representation of the world or some parts of it and "provides a shared vocabulary, which can be used to model a domain that is, the type of objects, and/or concepts that exist, and their properties and relations" [132]. Meanwhile, Luncean et al. [131] states that "An ontology is used to represent knowledge that is shared between different entities. It provides terms and vocabulary used to represent knowledge so that both sender and receiver can understand" Several ontologies already exist in the energy field. In [16,52], the main goal of ontologies is to support the interactions between energy management of buildings and the smart grid.

It is important to mention that the design of an ontology itself does not contribute to energy savings or energy-neutral building environments. However, it brings several benefits to the design of the software process of a MAS. First, it gives a deeper insight into the modeled domain and system functionality. Secondly, it reflects upon the data types and required communication between agents. These factors are useful when concepts are shared between different teams and systems, e.g., when different domains need to be connected to a smart grid.

However, MASs in the energy domain are developed with their own ontology, which cannot be directly integrated into other systems. A standard to solve the problem of multiple ontologies would lower the cost and human effort when different systems need to be connected. In the literature [91], several solutions for MAS integration are investigated. The FIPA ontology services the integration of existing MASs by introducing an ontology agent. This agent provides ontology-related services, e.g., translating expressions between ontologies and identifying a common ontology to two agents [16]. However, ontology designers still need to identify the similarities and differences between ontologies manually to translate the ontologies. This likely introduces more complexity and potential errors.

An upper ontology, as discussed in [21,103] could be an alternative to represent the general concepts of the domain. Such ontologies provide the framework in which the low-level ontologies can work. The upper ontology allows communication between different systems and each system with separated low-level ontologies. An upper ontology can be defined through multi-layered architecture or smaller reusable modules. The development and maintenance of MAS are easier and more efficient by composing a large-scale ontology out of smaller ones. This makes the ontologies simpler to modify, e.g., if legislation changes. The independent parts of an ontology must be well defined and separated. Thus, it is possible to reuse the parts in similar applications. The layered architecture also makes the ontology easier to be extended for other application domains and not just the intended domain [91].

An upper ontology for the energy sector can serve as an open standard that can assist the development of multi-agent solutions. It should not be a standard for all applications, but a tool from which the low-level ontologies can be extracted. Upper ontologies for the electricity domain are found in the literature, but the integration with the entire energy sector is still missing. This integration is necessary to fully understand and control the energy sector because the energy sector becomes more complex and consists of multiple hybrid systems.

The literature reviewed in this study presents different energy domains and includes different agents, data, and terms. This heterogeneity hinders the full adoption of these MASs and ontologies in a real scenario. Hence, there is a need for developing a unified ontology that represents all energy domains and provides a common terminology. In the literature, business models are separated from the MASs in the energy domain. For a deeper understanding of the domain and related agents, business models should be considered as part of MASs.

The combination of MASs, ontologies, and business models will enable simulations of the energy sector for exploring the interplay of policy, economy, and technology. Furthermore, a standardization of communication between agent will provide better knowledge- and data exchange between agent and domains. However, better simulation tools which can be used for scenario comparison, prediction of future evolution and sensitivity analysis are important, and it will make simulations easier to predict future events, identify unmet needs and act deliberated to changes in the energy sector.
