*Proceedings* **An Intelligent and Collaborative Multiagent System in a 3D Environment †**

#### **Alejandro Rodríguez-Arias** *∗* **, Bertha Guijarro-Berdiñas and Noelia Sánchez-Maroño**

Campus de Elviña s/n, Universidade da Coruña, CITIC, 15071 A Coruña, Spain; berta.guijarro@udc.es (B.G.-B.); noelia.sanchez@udc.es (N.S.-M.)

**\*** Correspondence: alejandro.rodriguez.arias@udc.es

† Presented at the 3rd XoveTIC Conference, A Coruña, Spain, 8–9 October 2020.

Published: 21 August 2020

**Abstract:** Multiagent systems (MASs) allow facing complex, heterogeneous, distributed problems difficult to solve by only one software agent. The world of video games provides problems and suitable environments for the use of MAS. In the field of games, Unity is one of the most used engines and allows the development of intelligent agents in virtual environments. However, although Unity allows working in multiagent environments, it does not provide functionalities to facilitate the development of MAS. The aim of this work is to create a multiagent system in Unity. For this purpose, a predator–prey problem was designed in which the agents must cooperate to arrest a thief driven by a human player. To solve this cooperative problem, it is required to create the representation of the environment and the agents in 3D; to equip the agents with vision, contact, and sound sensors to perceive the environment; to implement the agents' behaviors; and, finally but not less important, to build a communication system between agents that allows negotiation, collaboration, and cooperation between them to create a complex, role-based chasing strategy.

**Keywords:** intelligent agents; multiagent systems; unity; agent communication language

#### **1. Introduction**

Multiagent systems (MASs) are an artificial intelligence discipline that allows facing complex, heterogeneous, and distributed problems that cannot be solved by a single software agent. A MAS is a society of homogeneous or heterogeneous intelligent agents specialized in solving part of a problem who cooperate or compete for resources to achieve their goals [1]. In a MAS, each agent acts autonomously. Therefore, agents need to be able to negotiate, cooperate, and coordinate to achieve their goals [2]. This is only possible if some agent communication language (ACL) has been developed.

The world of video games provides problems and suitable environments for the use of MAS. Unity is one of the most used video game engines and allows the development of intelligent agents in virtual 2D/3D environments [3]. The Unity Machine Learning (ML) Agents project enables games and simulations run in Unity to be used as intelligent agent training environments using machine learning algorithms, enabling developers and researchers to work with Unity and machine learning [4]. Even though both Unity and ML Agents allow working in environments with multiple agents, none provide functionalities to work with MAS, that is, there are no tools for agents to communicate and, consequently, they cannot carry out tasks in a coordinated way.

The objective of this work is to create a multiagent system in Unity. To test our development, we use a predator–prey problem where the MAS, in the role of security guards, must use a collaborative strategy that allows it to catch a human player, in the role of a thief, who has stolen a treasure and escapes using the exit door in a virtual 3D environment.

#### **2. New Functionalities for the Development of Multiagent Systems in Unity**

For solving problems, an agent must be able to perceive the environment and act in consequence. In the case of complex problems in which the actions of several agents are necessary, communication actions are also mandatory to permit coordination and/or negotiation between agents. Therefore, it is necessary to build a communication system in Unity that allows this communication.

#### *2.1. Single Agents*

In our system, each agent has been endowed with three sensors that allows them to perceive the environment they are in (in our example, mainly walls, doors, and the player). These three sensors are: (1) a vision sensor that allows the agents to recognize any object in their front view cone; (2) a sound sensor that allows the agents to hear, through a nearby radio, the noise of the steps generated by the player; (3) a contact sensor that allows the agents to identify the objects that they collide with.

These agents can perform three different actions: (1) move freely around the stage using the A\* algorithm to find the optimal route to their objective; (2) open/close doors to go through them; (3) capture the player once an agent collides with him.

#### *2.2. The Agent Communication System*

A communication system between agents is based on three key points: (1) the structure of messages; (2) a system for sending, creating, and receiving messages; and (3) a system for managing conversations between agents.

For the structure of the messages, it is necessary to use an agent communication language (ACL). An ACL is a standard that provides high-level methods for an agent to exchange information. The ACL itself defines the types and meanings of the messages that agents can exchange. Most ACLs are based on speech act theory [5], which means that messages are communicative acts.

As Unity lacks a communication language between agents, we developed a plugin based on the ACL proposed by the Foundation for Intelligent Physical Agents (FIPA) [6,7], reducing the number of communicative acts and parameters available in FIPA-ACL to those essential for communication and developing a collaborative strategy complex enough to solve problems.

#### *2.3. Behavior*

In our system, the agents' behavior varies over time. Initially, everyone is assigned a limited area where they patrol. When one detects the player, the agents form a team and each one assumes a role according to their current position with respect to the player, other agents, and the player's objective.

An agent can take on different roles: a *goalkeeper*, who patrols near the player's target; *defense*, attempting to block accessible rooms from the player's current position; an *attacker*, who will attempt to block exits from the player's current room; and a *pursuer*, who will chase the player across the scenario.

Agents with the same role form a team with randomly assigned leader who helps coordinate negotiations on the objective of each agent in their role (for example, which door access will be blocked). Thanks to the designed communication system, each agent maintains negotiations with other agents with the same role and shares information with all of the agents in the system to try to use the most effective strategy at all times.

An example of how the system works can be seen in https://youtu.be/CEw7XkMlKfc

#### **3. Conclusions**

In this work we have provided Unity and Unity ML Agents with the necessary facilities to develop a MAS, that is: (a) a communication system; (b) an ACL based on FIPA to understand the interchanged messages; (c) basic sensors to allow agents to perceive the environment. These functionalities allow the resolution of problems that require collaborative solutions between agents, as demonstrated in the predator–prey problem implemented.

**Author Contributions:** These authors contributed equally to this work.

**Funding:** This work has been supported in part by the Xunta de Galicia (Grants ED431C 2018/34 and ED431G/01), the European Union's H2020 SMARTEES project (Grant 763912), and the European Union FEDER funds.

#### **References**


c 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

#### *Proceedings*  **Comparison of Image Compressions: Analog Transformations** <sup>P</sup> **†**

#### **Jose Balsa**

<sup>P</sup> CITIC Research Center, Universidade da Coruña (University of A Coruña), 15071 A Coruña, Spain; j.balsa@udc.es

† Presented at the 3rd XoveTIC Conference, A Coruña, Spain, 8–9 October 2020.

Published: 21 August 2020

**Abstract:** A comparison between the four most used transforms, the discrete Fourier transform (DFT), discrete cosine transform (DCT), the Walsh–Hadamard transform (WHT) and the Haarwavelet transform (DWT), for the transmission of analog images, varying their compression and comparing their quality, is presented. Additionally, performance tests are done for different levels of white Gaussian additive noise.

**Keywords:** analog image transformation; analog image compression; analog image quality

#### **1. Introduction**

Digitized image coding systems employ reversible mathematical transformations. These transformations change values and function domains in order to rearrange information in a way that condenses information important to human vision [1]. In the new domain, it is possible to filter out relevant information and discard information that is irrelevant or of lesser importance for image quality [2]. Both digital and analog systems use the same transformations in source coding. Some examples of digital systems that employ these transformations are JPEG, M-JPEG, JPEG2000, MPEG-1, 2, 3 and 4, DV and HDV, among others. Although digital systems after transformation and filtering make use of digital lossless compression techniques, such as Huffman.

In this work, we aim to make a comparison of the most commonly used transformations in stateof-the-art image compression systems. Typically, the transformations used to compress analog images work either on the entire image or on regions of the image. The transforms whose performance will be analyzed with compression and analog noise are the discrete Fourier transform (DFT), discrete cosine transform (DCT), the Walsh–Hadamard transform (WHT) and the Haarwavelet transform (DWT) [3,4].

These transformations can be applied to the whole image or to parts of the image. The decision will depend on the performance of the transformation, the computational load, the desired compression level and the impact of the noise on the coefficients of the transformed domain. In this work, we intend to make an analysis of the performance of the above-mentioned transformations by varying the characteristics listed.

#### **2. System Description**

The system proposed and implemented in MATLAB performs the four transformations, DFT, DCT, WHT and the Haar-based wavelet (DWT), simultaneously, with the same images, the same compression ratio and additive white Gaussian noise (AWGN) with the same intensity. The purpose of the system is to test and compare the performance of the four transformations applied to images as a function of the compression ratio and noise.

The operation of the DFT, DCT and WHT transforms are similar. These transforms change the domain of the image symbols to concentrate the relevant information into certain coefficients, allowing for compression by setting a transmission threshold. These transforms are applied to the entire image or to smaller divisions of the image, usually in squares. The selection of the size of the block where it will be applied is also relevant to the quality of the image [1], and we will take it into account in the results.

The Haar-based wavelet [4] is a transformation that condenses the important information of an image onto certain symbols. The reverse transformation allows the recovery of the exact image. For the compression, a threshold is set from which the symbols are approximated by zero and it is these zeros over the total symbols that allow the calculation of the compression ratio.

Therefore, the system created allows a comparison of the four transformations as a function of image quality, using the structural similarity index measure (SSIM) [5], and varying the compression ratio and noise intensity. A bank of 14 images with different characteristics is used.

#### **3. Results and Conclusions**

The results shown are for a block size of 16 × 16 for the DFT, DCT and WHT transformations. In the case of the Haar-based wavelet transformation, it is performed on the whole image. In addition, a filling with zeros is performed if the multiplicity is not met, as in the case of WHT and the DWT. For this purpose, it is necessary to do the reverse transform at the reception and then remove the image positions that were filled with zeros.

Figure 1a shows the result for when there is practically no noise; in this case, we can see how the DWT has the best behavior, followed by the DCT. It should be noted that the DFT is sluggish and does not achieve the maximum quality of 1 of SSIM. This is due to the fact that when we do the DFT transform we get twice as many symbols as the rest of the transforms because they are in the real and imaginary domain, so this coding is considerably penalized when comparing the number of symbols transmitted.

**Figure 1.** The relationship between compression rate and image quality (SSIM) is shown for two levels of signal-noise rate (SNR): (**a**) 100 dB additive white Gaussian noise (AWGN); (**b**) 15 dB AWGN.

Figure 1b shows the result for the above situation, but with an AWGN noise level of 15 dB. This time, a better performance of the DCT is shown in general. It can be noticed that the DFT, when there is no compression, obtains better results than the rest of the transformations, but we must emphasize that we are in a very small range of SSIM values, so its relevance is small.

As a conclusion, we can say that, among the most known transformations for images, those that present a greater performance are the DCT and the wavelet. In view of the results obtained, the DCT presents a higher quality of SSIM for noise levels below 20 dB AWGN and the wavelet for higher levels of 20 dB. With this, we can say that it is interesting to use the wavelets for digital systems because the noise does not affect the transformed values. However, for analog systems, where noise

#### *Proceedings* **2020**, *54*, 37

is introduced into the transformed symbols, the DCT is the best choice because, in view of the results, it offers a better balance between noise-free and noisy transmissions. Finally, we can say that the WHT and the DFT offer the worst results for transmissions with and without noise.

#### **References**


© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
