Basic Characteristics of a Macroscopic Measure for Detecting Abnormal Changes in a Multiagent System

Abstract

Multiagent application systems must deal with various changes in both the system and the system environment at runtime. Generally, such changes have undesirable negative effects on the system. To manage and control the system, it is important to observe and detect negative effects using an appropriate observation function of the system’s behavior. This paper focuses on the design of this function and proposes a new macroscopic measure with which to observe behavioral characteristics of a runtime multiagent system. The proposed measure is designed as the variance of fluctuation of a macroscopic activity factor of the whole system, based on theoretical analysis of the macroscopic behavioral model of a multiagent system. Experiments are conducted to investigate basic characteristics of the proposed measure, using a test bed system. The results of experiments show that the proposed measure reacts quickly and increases drastically in response to abnormal changes in the system. Hence, the proposed measure is considered a measure that can be used to detect undesirable changes in a multiagent system.

1. Introduction

A multiagent system (MAS) consists of agents who operate cooperatively as a team and handle problem-solving tasks according to their unique features relating to autonomy, reactivity, and social ability. At the runtime of an MAS, the system’s behavior fluctuates according to various changes in both the system and system environment. Because these changes generally have undesirable negative effects on the system, the system has to deal with the changes to maintain its behavior and performance. It is desirable to detect negative effects quickly using appropriate monitoring functions of the system’s behavior before fatal changes to the system occur. To develop monitoring functions for various information systems, measures for observing the behavior of systems have been investigated. Moreover, quantitative and qualitative measures such as the quality of service (QoS) have been developed for various information systems aiming to realize user-oriented services. However, it appears difficult to use these measures to detect abnormal situations for a system because observed measures such as the QoS fluctuate immediately after various changes within the system itself. Useful methods for detecting abnormal situations according to the system’s changes before the QoS degrades have yet to been established. Hence, the development of a new measure and/or measurement method that predicts the occurrence of abnormal changes is an important problem to solve to avoid unrecoverable situations of a system.

Focusing on the problem above, this paper proposes a new macroscopic measure defined using the macroscopic activity factor of an MAS according to theoretical analysis of the behavioral model of the MAS. The proposed measure is defined as the variance in fluctuation of the activity factor of the MAS. The analysis of the behavioral model reveals that the proposed measure increases drastically when the activity factor degrades. Therefore, this paper assumes a working hypothesis that clues that can reveal undesirable changes in the system are unusual increases in the proposed measure. Basic characteristics of the proposed measure are then investigated in experiments using a test bed system of a hierarchical MAS. The experiments show that the proposed measure reacts quickly in response to a system’s abnormal changes in terms of a drastic rise in the observed values as suggested by the working hypothesis above.

The remainder of the paper is organized as follows: Section 2 explains the background and motivation of the study. Section 3 defines a behavioral model of the MAS from a macroscopic viewpoint and proposes a new macroscopic measure using this model. The proposed measure is defined as the variance of fluctuation of the macroscopic activity factor of the MAS. Section 4 designs a measurement function for a test bed system using the proposed measure, and realizes a test bed system using a repository-based multiagent framework. The results of experiments conducted using the test bed system are presented to demonstrate basic characteristics of the proposed measure. Finally, Section 5 concludes the paper.

2. Background and Motivation

It is difficult to presume the nondeterministic behavior of an MAS at runtime because the properties and functions of agents are affected directly or indirectly by various changes in the system’s runtime conditions and environment. These changes occasionally have undesirable effects and can even cause catastrophic damage to the runtime system. It is thus necessary to develop effective methods and functions to maintain the system’s behavioral characteristics in response to such unusual situations.

Studies, such as those on resilient systems, have required a system to be proactive and able to anticipate, monitor and respond to environmental change or unexpected emerging threats. To realize such capabilities, Balchanos et al. [1] proposed a set of problem-dependent metrics that capture a resilient system’s ability to perform essential functions of a naval cooling network. The risks faced by MASs, which can cause serious breakdowns in systems’ operations, have been studied in contexts such as the trust management of MASs [2]. Self-protecting software systems have been studied as a class of autonomic systems capable of detecting and mitigating security threats at runtime [3]. Kaindl et al. [4] proposed a description model called self-representation for agents that represents both the software and hardware of mechatronic components in flexible automation systems. Moreover, Rahimian et al. [5] proposed a method of preserving the structural controllability of multiagent networks in the event of simultaneous failures using different quantitative measures of reliability. The above studies provided concepts and methods related to the design of new measures, which is the aim of the present paper.

Qualitative/quantitative measures or indexes have been studied in many technical domains, and schemes on the QoS [6] are adopted in various information systems including MASs. For instance, a multiagent application system, which operates adaptively in response to changes in both the user’s requests and the system environment, has been realized using problem-oriented QoS parameters of multimedia communication services [7,8]. Furthermore, Gutierrez et al. [9,10] studied an interesting method of detecting undesirable patterns of communication in MASs based on metrics for the observation of the performance of agents. The proposed metrics are defined as microscopic measures, which are calculated using various parameters representing agents’ activities. These metrics are different from the QoS but relate strongly with the QoS of an MAS, and they are used to detect undesirable behavioral patterns of an MAS and thus assist designers of an MAS. The results of measurement are used to find out undesirable pattern of communication, which should be improved using the detailed design information of the system. On the other hand, our measure is defined as a macroscopic index based on the macroscopic activity factor of the whole system. The purpose of the proposed measure is to make the system’s users aware of changes of the behavioral situations of the whole system as soon as possible. Moreover, the macroscopic activity factor can be observed by a relatively simple method, which does not utilize the microscopic metrics like QoS parameters of the system’s components.

Furthermore, it should be noted that undesirable changes in QoS parameters are observed only after the system has been altered. It is difficult to sense occurrences of undesirable changes in the system before the system falls into abnormal situations using QoS schemes only. Therefore, in our previous work [11], an index named the margin of potential capability (MoC) was designed to observe the degree of maneuverability of QoS parameters of an agent-based application system. Using the predefined threshold values of the MoC, degradations of maneuverability of the system can be detected and recovery processes relating to a system’s behavioral properties can be activated before the system goes down. However, the MoC is calculated using the observed QoS parameters, and the system thus cannot predict the occurrence of changes in the system using the MoC. If we want to avoid fatal situations like a system going down, we have to devise a new measure that is similar to leading indicators in the field of economics to predict undesirable changes of the system.

We thus introduce a macroscopic behavioral model of the MAS in this paper, and design a new observable measure based on theoretical analysis using this model. There have been studies on the design of metrics that are used in model-based methods to detect and analyze abnormal parts and/or faults in MASs. For instance, Franch [12] investigated a framework for the analysis of predictability of agent-oriented models written in the i* language using indicators to define metrics that measure model properties. Moreover, Mani et al. [13] proposed a testing method that extracts potential deadlock information from the design models of an MAS. This method allows the testing of a multiagent manufacturing system for deadlocks while the system is under development. The models of above papers are defined to describe the functional specification of an MAS and used in the design stage of an MAS. These models are useful in the design and implementation of MASs. In contrast, our behavioral model provides theoretical foundation of the design of a macroscopic measure to detect abnormal changes of an MAS, but does not aim to support the development of various MASs.

Against the background discussed above, this paper proposes a new macroscopic measure based on both the analysis of the theoretical behavioral model and experiments conducted using the test bed system of the MAS.

3. Method of Detecting Changes in a Behavioral Property of an MAS

3.1. Definition of the Macroscopic State Transition Equation of an MAS

To express the macroscopic behavior of an MAS, the structure and functions of the system are modeled using a macroscopic state transition equation. Let the number of agents in a system be n. An agent A_i in the system has its own behavioral state x_i, which represents the degree of activity of A_i at time t. When A_i works cooperatively with another agent A_j with a behavioral state x_j, behavioral states of A_i are affected by actions of A_j directly and/or indirectly, and vice versa. When there are many tasks in the system, many agents become active to process tasks. For instance, if A_j works actively and its behavioral state is high, the possibility of increasing the degree of activity of A_i increases. Hence, in this model, a behavioral state of A_j at time t is determined according to various effects of other agents. The degree of positive effects of A_j, with respect to A_i at time t, is specified by a parameter w_ij, called the cooperation coefficient, as shown in Figure 1.

Figure 1. Model of a multiagent system.

Figure 1. Model of a multiagent system.

To simplify the behavioral model, let a behavioral state x be a binary variable. The behavioral state $x_i^{\prime }$ of A_i at time t + 1 is defined using a step function:

$$\begin{gathered} x_i^{\prime }= stp(u_i) \\ {u}_i={{\displaystyle \sum }}_{j=1}^n{w}_{ij}\cdot x_j-h_i, 0\le w_{ij}\le 1 \\ {x}_i^{\prime }=\{\begin{array}{c}1, u_i>0\\ 0, u_i\le 0\end{array} \end{gathered}$$

(1)

Here, $h_i$ is called the threshold coefficient. This parameter specifies the degree of degradation of activity of an agent, due to the runtime situation. For instance, conflicts relating to resources of agents on a single platform and overheads of processing and communication of agents on distributed platforms can be considered causes of degradation. If the threshold coefficient of an agent is small, then the agent has enough room to work more actively.

Next, the macroscopic behavior of the MAS is defined using behavioral states of the system. Because an agent is modeled as a threshold element, the MAS can be viewed as a network of threshold elements like a neural network. Hence, the macroscopic behavioral model of the MAS is defined employing a similar method of artificial neural network modeling [14].

A macroscopic state of the MAS at time t is given as a set of behavioral states of agents and represented as $X=[x_1, x_2,\dots , x_n]$. Let Z be the degree of activity at time t. The degree of activity Z’ at time t + 1 is given by the function:

$$Act\left(X\right)= \frac{1}{n}・{\displaystyle \sum }_{i=1}^n{x}_i^{\prime }$$

(2)

Let the variables u, w, and h be Gaussian variables. The ensemble average of $u_i$ (i.e., $<u_i>$) and its variance ${\text{σ}}_u^2$ are given as:

$$<u_i>={{\displaystyle \sum }}_j<w_{ij}>\cdot x_j - <h_i>$$

Hence:

$$\overline{u}= n\bullet \overline{w}\bullet Z-\overline{h}$$

(3)

where $\overline{u}=<u_i>,\overline{w}=<w_{ij}>,\overline{h}=<h_i>$ and ${\text{σ}}_u^2=n\cdot {\text{σ}}_w^2+{\text{σ}}_h^2$.

On the other hand, the probability density function of a Gaussian variable u is given by:

$$\frac{1}{\sqrt{2\text{π} }\bullet {\text{σ}}_u}\cdot \exp \left\{-\frac{{(u-\overline{u})}^2}{2{\text{σ}}_u^2} \right\}$$

Hence, the probability of $x_i^{\prime }$ = 1 is calculated as:

$$p=Prob\left\{u_i>0\right\}={{\displaystyle \int }}_0^{\mathrm{\infty }}\frac{1}{\sqrt{2\text{π} }\bullet {\text{σ}}_u}\bullet \exp \left\{-\frac{{\left(u-\overline{u}\right)}^2}{2{\text{σ}}_u^2} \right\}du ={{\displaystyle \int }}_{-\raisebox{1ex}{$\overline{u}$}\!\left/ \!\raisebox{-1ex}{$\text{σ}$}\right.}^{\mathrm{\infty }}\frac{1}{\sqrt{2\pi }}\cdot \exp \left\{-\frac{v^2}{2}\right\} dv$$

(4)

The ensemble average of $x_i^{\prime },$ $<x_i^{\prime }>,$ is given by:

$$< x_i^{\prime }> =p\cdot 1+\left(1-p\right)\cdot 0= p$$

Using the error function, the ensemble average $<x_i^{\prime }>$ is expressed as:

$$\text{Φ}\left(u\right)={{\displaystyle \int }}_{-\mathrm{\infty }}^u\frac{1}{\sqrt{2\text{π}}}\cdot \exp \left\{-\frac{v^2}{2}\right\} dv$$

(5)

$$<x_i^{\prime }>= \text{Φ}\left(\raisebox{1ex}{$\overline{u}$}\!\left/ \!\raisebox{-1ex}{${\text{σ}}_u$}\right.\right)$$

(6)

The variance of the degree of activity, ${\text{σ}}_u^2$, is given as:

$$Z^{\prime }= <x_i^{\prime }>$$

$${\text{σ}}_u^2= <{\left| Act\left(X\right) - <x_i^{\prime }>\right|}^2> = \frac{1}{n^2}\cdot {\displaystyle \sum }_{i}Var(x_i^{\prime })$$

(7)

When n is large, $Act\left(X\right)$ can be approximated by $<x_i^{\prime }>$, because $Var(x_i^{\prime })$ can be considered zero. Hence, $Z^{\prime }$ is expressed as:

$$Z^{\prime }= \text{Φ}\left(Z\right)= \text{Φ}\left(C\cdot Z-\text{Θ}\right)$$

(8)

Here, $C=\raisebox{1ex}{$n\cdot \overline{w}$}\!\left/ \!\raisebox{-1ex}{${\text{σ}}_u$}\right.$ is the macroscopic cooperation coefficient and $\text{Θ}=\raisebox{1ex}{$\overline{h}$}\!\left/ \!\raisebox{-1ex}{${\text{σ}}_u$}\right.$ is the macroscopic threshold coefficient. Formula (8) is the macroscopic state transition equation of the MAS, and the system’s characteristics can be manipulated by the system’s parameters C and Θ. Moreover, the macroscopic state transition function has hysteresis characteristics due to an error function. Hence, in the runtime system, it is expected that unstable states of the activity factor will emerge according to changes in the system’s parameters. An image of unstable states S_a and S_b with respect to different sets of parameters can be depicted as in Figure 2. Because an unstable state will jump to another state discontinuously, the behavioral characteristics of the system will also change suddenly at the unstable state.

Figure 2. Image of unstable state of activity factor.

Figure 2. Image of unstable state of activity factor.

3.2. Analysis of the Macroscopic Behavior of an MAS

To define a new measure that can be used to observe the behavior of the MAS as a whole, the macroscopic behavioral model of the MAS is constructed and analyzed using a master equation [15,16].

Let N be the number of agents of an MAS, $x_i$ be the state of an agent and $h_i$ be the threshold coefficient. The agent is in an active state if $x_i\geqq h_i$ and in an inactive state if $x_i<h_i$. Let N₁ be the number of active agents and N₂ be the number of inactive agents. The activity factor $z$ at time t is then defined as:

$$z= \raisebox{1ex}{$\left(N_1-N_2\right)$}\!\left/ \!\raisebox{-1ex}{$N$}\right. \text{and }N=N_1+N_2$$

(9)

An agent becomes active or inactive along with a process of the system’s behavior. Such a process is modeled as a stochastic birth–death process and represented by the following master equation. The following transitions of behavioral states of agents can occur for an initial state $(N_1, N_2)$ at time t:

$$(N_1,N_2)\to (N_1+1,N_2-1):\text{ an agent becomes active}$$

(10)

$$(N_1,N_2)\to (N_1-1,N_2+1):\text{ an agent becomes inactive}$$

(11)

Let $W_{+,-}$ and $W_{-,+}$ be the ratios of transitions defined as:

$$W_{+,-}\left(N_1, N_2 \to N_1+1, N_2-1\right)=N_2\cdot {\text{π}}_1(z)$$

(12)

$$W_{-,+}\left(N_1, N_2 \to N_1-1, N_2+1\right)=N_1\cdot {\text{π}}_2(z)$$

(13)

Here, ${\text{π}}_1(z)$ is the transition rate for an inactive agent becoming active, and ${\pi }_2(z)$ is the transition rate for an active agent becoming inactive.

Let $P (N_1, N_2, t)$ be the probability density of finding a state $(N_1, N_2)$ at time t. Considering Equations (10)–(13), the change of $P (N_1, N_2, t)$ at time t is defined as:

$$\begin{gathered} P (N_1, N_2, t+1)= [1- N_2\bullet {\pi }_1\left(z\right)- N_1\bullet {\text{π}}_2\left(z\right) ]\bullet P( N_1, N_2, t) \\ + (N_2+1)\bullet {\text{π}}_1\left(z\right)\cdot P( N_1-1, N_2+1, t) \\ + (N_1+1)\bullet {\text{π}}_2\left(z\right)\cdot P( N_1+1, N_2-1, t) \end{gathered}$$

(14)

As shown in Figure 3, first term of Equation (14) represents the effects that $(N_1, N_2)$ changes to adjacent states $(N_1+1, N_2-1)$ and $(N_1-1, N_2+1)$, at time t. On the other hand, second and third terms represent the effects that $(N_1+1, N_2-1)$ and $(N_1-1, N_2+1)$ change to $(N_1, N_2)$.

Equation (14) is rewritten as

$$\begin{gathered} \raisebox{1ex}{$[P (N_1, N_2, t+1)-P (N_1, N_2, t)]$}\!\left/ \!\raisebox{-1ex}{$t$}\right. \\ \cong \raisebox{1ex}{$\partial P(N_1, N_2, t)$}\!\left/ \!\raisebox{-1ex}{$\partial t$}\right. \\ = -[ N_2\bullet {\text{π}}_1\left(z\right)+ N_1\bullet {\text{π}}_2\left(z\right) ]\bullet P( N_1, N_2, t) \\ + (N_2+1)\bullet {\text{π}}_1\left(z\right)\cdot P( N_1-1, N_2+1, t) \\ + (N_1+1)\bullet {\text{π}}_2\left(z\right)\cdot P( N_1+1) \end{gathered}$$

(15)

Figure 3. Transitions between state $(N_1, N_2)$ and its adjacent states at time t.

Figure 3. Transitions between state $(N_1, N_2)$ and its adjacent states at time t.

Using Equation (15), we define the master equation as:

$$\begin{gathered} \raisebox{1ex}{$\partial P(N_1, N_2, t)$}\!\left/ \!\raisebox{-1ex}{$\partial t$}\right.=-[ N_2\bullet {\text{π}}_1\left(z\right)+ N_1\bullet {\text{π}}_2\left(z\right) ]\bullet P( N_1, N_2, t) \\ + (N_2+1)\bullet {\text{π}}_1\left(z-\text{ε}\right)\cdot P( N_1-1, N_2+1, t) \\ + (N_1+1)\bullet {\text{π}}_2\left(z+\text{ε}\right)\cdot P( N_1+1, N_2-1, t) \end{gathered}$$

(16)

Here, $\text{ε}=\raisebox{1ex}{$2$}\!\left/ \!\raisebox{-1ex}{$N$}\right.$ and we assume $\text{ε} \ll 1$.

Considering the relations:

$$N_1=\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$\text{ε}$}\right.\cdot (1+z) \text{and} N_2=\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$\text{ε}$}\right.\cdot (1-z)$$

which are derived from Equation (9), the master Equation (16) is rewritten as:

$$\begin{gathered} \raisebox{1ex}{$\partial P\left(z, t\right)$}\!\left/ \!\raisebox{-1ex}{$\partial t$}\right.=-\left[ \raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$\text{ε}$}\right.\cdot \left(1-z\right)\bullet {\text{π}}_1\left(z\right)+\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$\text{ε}$}\right.\cdot \left(1+z\right)\bullet {\text{π}}_2\left(z\right)\right]\bullet P\left(z, t\right) \\ + \raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$\text{ε}$}\right.\cdot \left(1-z+\text{ε}\right)\bullet {\text{π}}_1\left(z-\text{ε}\right)\cdot P\left(z-\text{ε}, t\right) \\ + \raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$\text{ε}$}\right.\cdot (1+z+\text{ε})\bullet {\text{π}}_2\left(z+\text{ε}\right)\cdot P(z+\text{ε}, t) \end{gathered}$$

(17)

Using a well-known procedure [17], the stochastic Equation (17) is equivalently transformed to the Fokker–Planck equation with respect to the fluctuation of activity factor, ξ, (see Appendix A):

$$\begin{gathered} \raisebox{1ex}{$\partial P\left(\text{ξ},t\right)$}\!\left/ \!\raisebox{-1ex}{$\partial t$}\right.= - \frac{\partial }{\partial z_0}\bullet \left[\left(1-z_0\right)\bullet {\text{π}}_1\left(z_0\right)-\left(1+z_0\right)\bullet {\text{π}}_2\left(z_0\right)\right]\bullet \frac{\partial \left[\text{ξ}\bullet P\left(\text{ξ},t\right)\right]}{\partial \text{ξ}} \\ + \frac{1}{2}\bullet [(1-z_0)\bullet {\text{π}}_1\left(z_0\right)+(1+z_0)\bullet {\text{π}}_2\left(z_0\right)]\bullet \frac{{\partial }^{2}P(\text{ξ},t)}{\partial {\text{ξ}}^2} \end{gathered}$$

(18)

Here, $z_0$ is the steady state of activity factor z, and the fluctuation ξ is defined as:

$$\text{ξ}=\frac{1}{\sqrt{\text{ε}}}\cdot (z-z_0)$$

(19)

Moreover, the evolution of z₀ is given as (see Appendix A):

$$\raisebox{1ex}{$\partial z_0$}\!\left/ \!\raisebox{-1ex}{$\partial t$}\right.= (1-z_0)\bullet {\text{π}}_1\left(z_0\right)- (1+z_0)\bullet {\text{π}}_2\left(z_0\right)$$

(20)

On the other hand, the evolution of the variance of the fluctuation of the activity factor is derived using Equation (18) (see Appendix B) as:

$$\begin{gathered} \raisebox{1ex}{$\partial {\mathrm{\sigma }}_{\text{ξ}}^2$}\!\left/ \!\raisebox{-1ex}{$\partial t$}\right.=2\bullet \frac{\partial \left[\left(1-z_0\right)\bullet {\text{π}}_1\left(z_0\right)-\left(1+z_0\right)\bullet {\text{π}}_2\left(z_0\right)\right]}{\partial z_0}\bullet {\mathrm{\sigma }}_{\text{ξ}}^2 \\ + [(1-z_0)\bullet {\text{π}}_1\left(z_0\right)+(1+z_0)\bullet {\text{π}}_2\left(z_0\right)] \end{gathered}$$

(21)

Using the results above, the characteristics of steady states of the activity factor are analyzed in the next section.

3.3. Macroscopic Measure Used to Observe the Behavioral Property of an MAS

In the steady state of the system, z₀, the condition:

$$\frac{\partial z_0}{\partial t}=\frac{\partial {\mathrm{\sigma }}_{\text{ξ}}^2}{\partial t}=0$$

is satisfied. Under this situation, the behavioral properties of an MAS are stable, and the system works smoothly. Using Equations (20) and (21), $z_0$ and ${\mathrm{\sigma }}_{\text{ξ}}^2$ are calculated as:

$$z_0=\raisebox{1ex}{$[ {\text{π}}_1\left(z_0\right)- {\text{π}}_2\left(z_0\right) ]$}\!\left/ \!\raisebox{-1ex}{$[{\text{ π}}_1\left(z_0\right)+{\text{π}}_2\left(z_0\right) ]$}\right.$$

(22)

$${\mathrm{\sigma }}_{\text{ξ}}^2=\raisebox{1ex}{$-[(1-z_0)\bullet {\text{π}}_1\left(z_0\right)+(1+z_0)\bullet {\text{π}}_2\left(z_0\right)]$}\!\left/ \!\raisebox{-1ex}{$2\bullet \frac{\partial [(1-z_0)\bullet {\text{π}}_1\left(z_0\right)-(1+z_0)\bullet {\text{π}}_2\left(z_0\right)]}{\partial z_0}$}\right.$$

(23)

Moreover, we assume that ${\text{π}}_2\left(z_0\right)=1-{\text{π}}_1\left(z_0\right)$ and ${\text{π}}_1\left(z_0\right)=\text{π}\left(z_0\right)$, and $z_0$, $\text{π}\left(z_0\right)$ and ${\mathrm{\sigma }}_{\text{ξ}}^2$ are calculated as:

$$z_0=2\bullet \text{π}\left(z_0\right)-1$$

(24)

$$\text{π}\left(z_0\right)=\raisebox{1ex}{$\left(z_0+1\right)$}\!\left/ \!\raisebox{-1ex}{$2$}\right.$$

(25)

$$\begin{gathered} {\mathrm{\sigma }}_{\text{ξ}}^2=\raisebox{1ex}{$-\left[\left(1-z_0\right)\bullet \text{π}\left(z_0\right)+\left(1+z_0\right)\bullet \text{π}\left(z_0\right)\right]$}\!\left/ \!\raisebox{-1ex}{$2\bullet \frac{\partial \left[\left(1-z_0\right)\bullet \text{π}\left(z_0\right)-\left(1+z_0\right)\bullet \text{π}\left(z_0\right)\right]}{\partial z_0}$}\right. \\ =\raisebox{1ex}{$-[1+z_0-2\bullet z_0\bullet \text{π}\left(z_0\right)]$}\!\left/ \!\raisebox{-1ex}{$2・(2\bullet \frac{\partial \text{π}\left(z_0\right)}{\partial z_0}-1)$}\right. \end{gathered}$$

(26)

When unusual changes occur in the runtime MAS, the system’s behavior becomes unstable temporary or permanently, and ${\mathrm{\sigma }}_{\text{ξ}}^2$ fluctuates according to changes of activities of the MAS. It is obvious that ${\mathrm{\sigma }}_{\text{ξ}}^2\to \mathrm{\infty }$ when $\raisebox{1ex}{$\partial \text{π}\left(z_0\right)$}\!\left/ \!\raisebox{-1ex}{$\partial z_0$}\right.\to \raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$2$}\right.$, that is, there exists a state where the variance of fluctuation is divergent.

As explained in Section 3.1, the macroscopic state transition function has hysteresis characteristics. Hence, an image of transitions of z₀ and ${\mathrm{\sigma }}_{\text{ξ}}^2$, with respect to the parameter Θ, can be depicted as in Figure 4. The variance of fluctuation gradually increases and finally diverges as the steady state approaches the point of discontinuous transition. Such a feature shows the possibility that the variance of fluctuation of the activity factor can be used as an observable measure of behavioral characteristics of the system. Furthermore, when an unusual increase in the variance of fluctuation of an activity factor is observed, it can be said that the system is in an unstable state and the system’s state is going to transit to another state in the near future, through abnormal changes of the system. Therefore, the variance of fluctuation of an activity factor of an MAS is proposed as a new measure for observing the behavior of a system, in this paper. The proposed measure is taken as a measurement function of an MAS. The specific measurement function of the test bed system is designed and implemented in the next section.

Figure 4. Image of transitions of z₀ and ${\mathrm{\sigma }}_{\text{ξ}}^2$ with respect to the parameter Θ.

Figure 4. Image of transitions of z₀ and ${\mathrm{\sigma }}_{\text{ξ}}^2$ with respect to the parameter Θ.

4. Experiments and Evaluation

4.1. Test Bed System and the Environment of Experiments

Experiments are conducted to test and validate the proposed measure using a test bed system. An important objective of the experiments is to explore basic properties and usefulness of the proposed measure using a real MAS in a real computer environment. The test bed system, which simulates typical behavior of multiagent applications such as a distributed monitoring system [18] and agent-based microgrid [19], developed in previous studies, has been realized as a simple hierarchical MAS using a repository-based multiagent framework [20,21].

The test bed system consists of one manager agent and many worker agents as shown in Figure 5. The manager agent generates a task, selects a worker agent, which is in the inactive state because it has no task to be processed, and assigns the task by sending an assignment message to the selected worker agent. In the test bed system, a task is designed as a pseudo task to make the selected worker agent active. If a task is assigned successfully, the manager agent considers that the selected worker agent is in the active state until receiving a finish message from the worker agent. From the viewpoint of the worker agent, the worker agent can accept and process one task at a time. The task process functions of worker agents are the same. The selected worker agent becomes active by accepting the assigned task, and stays active during the processing of the assigned task. When the assigned task is finished, the worker agent sends a finish message to the manager agent and becomes inactive again. Moreover, we introduce a random delay time before sending a finish message, to simulate the task process overhead of a worker agent. In this sense, the performance of each worker agent is different from each other.

Figure 5. Structure of the test bed system.

Figure 5. Structure of the test bed system.

The assigned task has information on the elapsed time, which specifies the required time to complete the task. An elapsed time of a task is set as a random value distributed around the specified average value. As mentioned above, we assign a random delay time of task process to each worker agent. Hence, the total processing time of the assigned task, called the duration time, is the sum of the elapsed time and delay time.

Figure 6. Configuration of the measurement environment.

Figure 6. Configuration of the measurement environment.

All agents in the test bed system are designed and implemented as software agents using the repository-based agent framework and its interactive design environment (version: IDEA_1.3.1) [20,21]. The agent framework supports the development of multiagent applications, on both a single platform and distributed platforms. In the experiments, the test bed system is realized as a system running on a single computer (CPU: Core-i5 3.0 GHz, Memory: 4.0 GB, OS: Windows 7). For this architecture, each agent is directly and adversely affected by other agents’ workloads and minimizes the overhead of measurement throughout the system environment. The agent framework allows the measurement of the behavioral properties of agents. This facility collects data of runtime agents, such as the task processing time, the duration time, the messages processed by agents and the number of activated agents, and accumulates time series data as external log files, as shown in Figure 6. Moreover, the analysis module inspects behavioral situations of the system based on the proposed measure and the results are to be used by maintenance operations to be realized in future work. The function of the analysis module is discussed in Section 4.5.

4.2. Implementation of the Measurement Function

The proposed measure is defined according to the active agents in the test bed system. As shown in Figure 5, N is the total number of agents of the test bed system, and $n_t$ is the number of active worker agents at time $T$. According to the definition of the activity factor in Equation (9), the normalized activity factor $z_t\in \left[0,1\right]$ at time T is calculated in each unit time $∆T$ and is specified by the measurement facility of the agent framework:

$$z_t=\frac{1}{2\bullet N}\bullet \left(2\bullet n_t-N+1\right)$$

(27)

The time series of the normalized activity factors, {$z_1$, $z_2$, $\cdots$, $z_j$, $\cdots$, $z_t$} is then given at time T. Here, $z_j$ is the jth element at time $T_j=T-(t-j)\times ∆T$. Next, the fluctuation of the activity factor is calculated as:

$${\text{ξ}}_t=z_t-\frac{1}{L}\bullet {\displaystyle \sum }_{i=0}^{L-1}{z}_{t-i}$$

(28)

The variance of fluctuation of the activity factor, $v({\text{ξ}}_t)$, is also calculated using part of a time series, {${\text{ξ}}_{t-M+1}$, $\cdots$, ${\text{ξ}}_{t-1}$, ${\text{ξ}}_t$ }, as:

$$v({\text{ξ}}_t)=\frac{1}{M}\bullet {\displaystyle \sum }_{i=0}^{M-1}{\text{ξ}}_{t-i}^2-{\left(\frac{1}{M}\bullet {\displaystyle \sum }_{i=0}^{M-1}{\text{ξ}}_{t-i}\right)}^2$$

(29)

The smoothed data are useful for the detection of abnormal changes observed in the time series of $var({\text{ξ}}_t)$. Hence, the moving average of the variance of fluctuation is calculated using part of a time series, {$v({\text{ξ}}_{t-M+1})$, $\cdots$, $v({\text{ξ}}_{t-1})$, $v({\text{ξ}}_t)$}, as:

$$Var({\text{ξ}}_t)=v({\text{ξ}}_t)-\frac{1}{M}\bullet {\displaystyle \sum }_{i=0}^{M-1}v({\text{ξ}}_{t-i})$$

(30)

A new measurement function of the proposed measure is implemented using these equations as the design specification in the test bed system. In the implementation, the unit time $∆T$ is set to 2 s, and the parameters in above equations are set as L = 20 and M = 20.

4.3. Experiment 1 and Evaluation

4.3.1. Setting of Experiment 1

The behavior of the test bed system in normal situations, in which the system does not change, is observed to study basic characteristics of the proposed measure. The structure and function of the normal test bed system do not change during the measurement in Experiment 1. The system consists of one manager agent and 150 worker agents. The elapsed time of a task is set as a random value selected between 18 and 22 s, and the average elapsed time is 20 s. The delay time of each task is also set as a random value, selected between 0 and 10 s, and the average delay time is 5 s. At the beginning of the experiment, to make worker agents active, the manager agent sends initialization messages to all worker agents, one by one, at intervals of 0.1 s, and the task assignment and processing begin. The log data of runtime agents are collected and accumulated in intervals of 0.01 s, until a time of 5760 s. Using the log data, the normalized activity factor and the variance of fluctuation are calculated in intervals of 2 s.

Figure 7. Results of measurements of the normal test bed system.

Figure 7. Results of measurements of the normal test bed system.

4.3.2. Evaluation of Experiment 1

The results of measurement (i.e., time series data of both the system’s activity factor and the variance of fluctuation of the activity factor) of a normal system are depicted in Figure 7a,b. In general, the amplitude distribution and power spectrum of the low-frequency region are useful for the observation of the features of time series data. Hence, three parts of the time series data specified as Section 1, Section 2 and Section 3 in Figure 7a, are selected and analyzed. Here, Section 1 is the steady-state region immediately after the transient region of initialization, while Section 2 and Section 3 are steady-state regions long after system initialization. We can specify these sections freely as non-overlapped sections in the steady-state region (600–5000 s). The length of each section is 1024 s. The periods of Section 1, Section 2 and Section 3 are set to 1000–2024, 2288–3312 and 3736–4760 s, respectively. The measurement results are shown in Figure 7c,d.

The temporal change in the proposed measure is no more than 0.02, indicating that the system behavior is stable; i.e., the temporal transition of the system’s activity factor remains in a stable range of 0.6–0.7 after 1000 s. The amplitude distributions of activity factors in the three sections have almost the same average values, about 0.65, and the power spectra show almost the same features. The results of experiment 1 show that the behavioral property of the system is stable when the system does not change.

4.4. Experiment 2 and Evaluation

4.4.1. Setting of Experiment 2

The purpose of Experiment 2 is to evaluate the proposed measure using a system that changes dynamically at runtime. This experiment simulates the unusual situations in which a system’s components halt gradually because of trouble in the system.

The setting of Experiment 2 is basically the same as that of experiment 1. However, the structure of the test bed system is changed at runtime by removing worker agents. Some worker agents are selected randomly and removed from the system at the time points given in Table 1. The system’s activity factor, in general, is worsened by the negative effects of structural and functional changes of the system. For the test bed system, the proposed measure can reveal the deterioration of macroscopic activity of the system but cannot classify causes of the deterioration. Therefore, structural changes of the system are adopted as a cause of deterioration.

Table 1. Setting of changes in the test bed system.

Time Point [s]	Number of Worker Agents to be Removed	Number of Agents of the System
600	1	150
1100	2	148
1600	4	144
2100	8	136
2600	16	120
3100	32	88

Table 1. Setting of changes in the test bed system.

Time Point [s]	Number of Worker Agents to be Removed	Number of Agents of the System
600	1	150
1100	2	148
1600	4	144
2100	8	136
2600	16	120
3100	32	88

4.4.2. Evaluation of Experiment 2

The temporal transitions of both the system’s activity factor and the variance of fluctuation of the system’s activity factor are shown in Figure 8a,b. The time points at which the worker agents are removed are indicated by dashed lines in Figure 8a,b. The amplitude distributions and the power spectra are calculated with respect to the sections specified for Experiment 1, and the results are shown in Figure 8c,d. However, in Experiment 2, Section 1, Section 2 and Section 3 correspond to regions of small changes, large changes, and no changes, respectively.

Figure 8. Results of measurements of the changed test bed system.

Figure 8. Results of measurements of the changed test bed system.

Interestingly, the temporal feature of the activity factor is similar to that in Experiment 1 until Section 2. The averages of activity factors in Section 1 and Section 2 fluctuate around 0.6 and the activity factor degrades a little according to changes in the system. The disturbance of the activity factor remains small until a discontinuous change at 3100 s. It thus appears difficult to detect abnormal situations for the system by observing only the transition of the activity factor, even when there are actual structural changes.

However, using the proposed measure, it becomes possible to detect abnormal situations according to an unusual increase in the proposed measure, in the early stages of abnormal situations of the system. The temporal feature of the variance of fluctuation of the activity factor is clearly different from that in Experiment 1. The variance of fluctuation becomes large gradually, according to changes in the system. At around the time point where the worker agents are removed, the proposed measure varies violently, and increases. The amplitude of the proposed measure becomes two or three times that for the unchanged system. As expected from theoretical analysis of the behavioral model in Section 3, an unusual increase in the proposed measure is observed in response to abnormal changes in the system.

Meanwhile, the variance of fluctuation becomes small after 2700 s. At this point, about 21% of the worker agents have been removed from the system, and it seems that the system’s behavioral characteristics transited to unusual states because of undesirable changes of the system. Obviously, the peak shifts can be observed in the amplitude distributions of Section 2 as shown in Figure 8c. Specifically, the single-peak distribution in Section 1 shifts to a double-peak distribution in Section 2, and then returns to a single-peak distribution in Section 3. Moreover, the features of the power spectrum of Section 2 resemble those of Section 1 whereas the power spectrum of Section 3 has different features compared with the other spectra.

The above result suggests that the behavioral characteristics of the initial system in Section 1 transformed to other characteristics via unstable states in Section 2. To inspect unstable behavioral situations in Section 2, the activity factor of the manager agent, who is responsible for controlling the whole system, is observed at the time corresponding to measurements in Experiments 1 and 2. The activity factor of the manager agent is a macroscopic measure defined as the rate of inference processing time per unit time. In the experiment, the unit time is set to 2 s.

The result of measurement of the manager agent is shown in Figure 9. In the normal system, the manager agent works at almost 100% to control the whole system. The behavior of the manager agent is stable and the variance of fluctuation of the activity factor is always very small, as shown in Figure 9a1,a2. By contrast, in the changed system, the behavior of the manager agent is stable until the large change in the system at around 2600 s. However, immediately after this point, the behavior of the manager agent changes suddenly. The variance of fluctuation of the activity factor increases rapidly, and fluctuates violently, as shown in Figure 9b1. Moreover, as shown in Figure 9b2, there is a peak shift of the amplitude distribution in Section 2, and the activity of the manager agent reduces to about 60% of that of the normal system.

The negative effects of eliminating worker agents are observed immediately after the first change in Figure 8b. As the worker agents are removed step by step, the rate of responses of the worker agents decreases and it becomes difficult for the manager agent to find suitable worker agents to accept the assignment of new tasks. The idle time of the manager agent varies rapidly, depending on the fluctuating situation of the worker agents, and the average idle time also increases. The behavior of the manager agent becomes unstable states around 2600 s, as shown in Figure 9b1. The unstable states of both the manager agent and the whole system continue until the end of measurement, and the capability of the whole system also reduces. This result shows that the original behavioral characteristics of the test bed system have shifted to other characteristics at around 2600 s.

In Experiment 2, no point of divergence of the variance of fluctuation is observed because such a point corresponds to an unstable state in which the system cannot behave stably. However, it can be said that the basic feature of the proposed measure is confirmed by the results of experiments in this paper. When there are abnormal changes in the system, unusual changes of the fluctuation of the system’s activity factor emerge and the variance of fluctuation increases rapidly. Hence, the proposed measure (i.e., the variance of fluctuation of the activity factor) can be considered a new observable measure with which to monitor the behavior of an MAS.

Figure 9. Results of measurements of the manager agent.

Figure 9. Results of measurements of the manager agent.

4.5. Considerations of the Observations of the Runtime System

The results of the above experiments reveal that the proposed measure responds to undesirable changes in an MAS. Hence, a mechanism that provides an alarm in the event of changes in a runtime system can be realized using the proposed measure. In this section, an idea for the design of such an alarm mechanism is discussed.

As shown in Figure 6, the alarm mechanism will be realized as part of the function of the analysis module, which can work in on-line or off-line mode. In the off-line mode adopted in the experiments of this paper, various analyses of time series data can be conducted using all data. However, the alarm cannot be raised at the runtime of the system. In the test bed system, the measurement facility of the agent framework provides the log data files, which hold raw data of agent behavioral properties accumulated in intervals of 2 s. Hence, in the experiments discussed above, we analyzed the time series of the proposed measure of the whole observation period in intervals of 2 s, as shown in Figure 7 and Figure 8.

The purpose of the on-line mode is to generate alarms quickly in the event of changes in the runtime system using some of time series data of agent behavioral properties. We can calculate the proposed measure in each fixed time period specified in advance of the measurement, and we can obtain a discrete time series of the proposed measure. Moreover, using alarm generation mechanisms, we can detect a time point at which there is a discontinuous change of the measure and issue an alarm at this point. The following simple procedure is an example of alarm generation mechanisms.

[Alarm generation procedure at discrete time point p]

-

At the discrete time point p, the result of observation is given by a time series of observed values; i.e., {$v_0$, $v_1$, $\cdots$, $v_{p-1}$, $v_p$}. Here, $v_0=0$.

-

The interval of the adjoining observed values, $d_p$, is given by $d_p\leftarrow v_p-v_{p-1}$.

-

The maximum value of $d_t$ ($p\ge t\ge 0$) is $d_{max}$.

-

The predefined threshold is $H$ ($H\ge 0$).

-

The score at time point p, $S_p$, is calculated in this procedure.

-

At the initial time point $p=0$, $S_0\leftarrow 0$, $d_0\leftarrow 0$, and $d_{max}\leftarrow 0$.

-

if $d_p\ge H$

• then {if $d_p>10\times d_{max}$ then {$S_p\leftarrow 0$; $d_{max}\leftarrow 0$; “Generate Alarm”};

  • if $d_p>d_{max}$ then $d_{max}\leftarrow d_p$;
  • if $S_{p-1}>0$ then $S_p\leftarrow S_{p-1}+1$ else $S_p\leftarrow 1$ }
• else {if $\left|d_p\right|\ge H$

  •    then {$S_p\leftarrow 0$; $d_{max}\leftarrow 0$;

    •      if $S_{p-1}>0$ then “Generate Alarm”}
  •    else $S_p\leftarrow S_{p-1}$ };

The above procedure generates an alarm if a peak of change is detected or $d_p$ has a sudden large change exceeding 10 times $d_{max}$. Because $S_p$ is reset after the alarm is generated, this procedure can generate alarms many times during a long observation of the runtime system.

Figure 10 shows the result of an experiment for the above procedure using some of the data of experiments 1 and 2. The discrete time series of the proposed measure are depicted by circles. In this experiment, we set the fixed observation period to 200 s, and the proposed measure is calculated every 200 s using the short time series of data with length of 40 s. The predefined threshold is set to 0.01.

In the case of the normal test bed system, the observed measure is stable and stays under 0.02, as shown in Figure 10a. No alarm is generated. In the case of the changed system, two alarms are generated in response to discontinuous changes in the observed measure as shown in Figure 10b. The upward arrows show the time points of $d_p\ge H$ and the downward arrows show the time points of $\left|d_p\right|\ge H$. In this case, the alarms are generated in response to large discontinuous changes in the proposed measure.

Figure 10. Experiment on an observation in on-line mode.

Figure 10. Experiment on an observation in on-line mode.

5. Conclusions

A macroscopic measure that responds to abnormal changes in the behavioral property of a multiagent system was proposed in this paper. The proposed measure was designed as the variance of fluctuation in a system’s activity factor according to theoretical analysis of the behavioral model of a multiagent system defined in this paper. To verify basic characteristics of the proposed measure, experiments were conducted using a test bed system based on a repository-based multiagent framework. The results of experiments showed that the proposed measure undergoes an unusual increase in response to abnormal changes in the system. Hence, the proposed measure is considered a measure that can be used to detect undesirable changes in a multiagent system. Further studies on the design of easy-to-use measures for various multiagent applications and the development of a practical measurement environment of runtime systems remain as future work.