Multi-Agent Systems and Complex Networks: Review and Applications in Systems Engineering

Abstract

Systems engineering is an ubiquitous discipline of Engineering overlapping industrial, chemical, mechanical, manufacturing, control, software, electrical, and civil engineering. It provides tools for dealing with the complexity and dynamics related to the optimisation of physical, natural, and virtual systems management. This paper presents a review of how multi-agent systems and complex networks theory are brought together to address systems engineering and management problems. The review also encompasses current and future research directions both for theoretical fundamentals and applications in the industry. This is made by considering trends such as mesoscale, multiscale, and multilayer networks along with the state-of-art analysis on network dynamics and intelligent networks. Critical and smart infrastructure, manufacturing processes, and supply chain networks are instances of research topics for which this literature review is highly relevant.

1. Introduction

Systems engineering is an amalgamation of engineering disciplines for the design, control, and overall management of the life cycle of engineered systems from an interdisciplinary point of view [1]. This involves various levels of abstraction of a system in which the interconnections between the parts (at each abstraction level) are often represented by a complex network [2,3]. Each component of the system then works towards individual and collective objectives to optimise local and general performance objectives. This decision-making process of each component can be modelled by multi-agent systems (MAS) [4,5]. Thus, both complex networks and MAS are of main importance for systems engineering and management. This paper presents the essentials of complex networks and MAS for control and optimisation in systems engineering. This is made through a theoretical overview and literature review of both approaches, introducing them separately ahead of discussing how they can be combined.

A complex network is mathematical abstraction of a real system in the form of a graph. As a difference from graphs, complex networks usually take non-regular topologies to better represent such real-world systems from which may also inherit another features. Examples of complex networks are utility networks [6], social networks [7], chemical reactions [8], or molecular networks [9], among many others. In many cases, networks may vary their properties and functionality depending on internal evolution of their properties or by reacting to exogenous interactions. For instance, in a telecommunication system network, nodes (e.g., routers and switches) may suffer over time a degradation in their properties that may lead them to diminish their performance. However, just the daily variation on traffic demand also has an impact on such nodes performance. Unexpected, external intervention such as cyber-attacks or extreme weather conditions may affect the properties, even the topology of a telecommunication system.

Intelligent distributed systems are capable of modelling how different parts of the network might work individually and collectively [10]. Ultimately, this is used as support for any decision-making system aiming to achieve better overall system functioning and its consequent performance. Intelligent distributed systems encompass autonomous learning units that can be associated with the nodes of a complex network. These intelligent network nodes are able to act independently and also interact with other nodes to pursue both individual (local) and general system-level targets. The necessary communication between individuals can be represented by links of the complex network connecting their nodes. It is natural to understand such nodes as intelligent agents within a MAS framework. Each agent separately obeying simple rules but working together with other agents makes it possible to approach complex engineering challenges. Intelligent distributed systems have been proposed in a broad range of engineered complex systems [11,12]. Precisely, the review proposed herein focuses only on engineered systems and manufacturing processes, given the wide spectrum of the approached topic. Out of this review fall, then, topics related to social systems analysis such as migration models [13,14] or online social media [15,16]. Similarly, natural systems are out of this review scope. This is the case of computational chemistry [17,18], biology [19,20], and metabolic networks [21,22].

Automatic optimisation and control in an engineering system is associated with a near real-time data acquisition and an optimal decision-making. The aim is to maximise the quality and performance of the outcome while minimising the overall costs of the process. The combination of complex networks and MAS provides an integrated framework for systems optimisation and control. The success of this framework is mainly based on its high applicability together with the relative simplicity of the approach. Part of the challenges and research directions in complex networks are coming from the investigation further of their structure at several dimensions or network layers and from the variety of resolution levels (mesoscale networks) in which a network can be analysed. These research topics are complement of those related to network-flow dynamics [23], time-evolving networks [24], and smart systems [25] for which MAS have emerged in their research.

This paper is a literature review of complex networks and MAS. The bibliographic search is approached using a list of academic databases and search engines such as Google Scholar, MDPI, Science Direct, SpringerLink, arXiv, IEEE Xplore, and Scopus. Section 2 and Section 3 encompass a literature review for basic, theoretical essentials on complex networks and MAS, respectively. As a convention, the candidate articles to be part of the review should contain word combinations in the title, keywords, or abstract related to “graph theory” or “complex networks” or “multi-agent systems” or “agent-based modelling” and their variations. The paper continues with Section 4 introducing real-world systems engineering examples for which control and optimisation processes based on complex networks and MAS have been shown to be essential for their operation, management, and protection. The above keywords are subsequently crossed (“and” operator) with others such as “engineering systems” or “critical infrastructure” or “engineering processes” or “manufacturing” or “supply chain” and their variations. Section 5 closes the paper with an open discussion about the systems engineering applications presented in the paper and future research directions on applied complex networks and MAS.

2. Complex Networks

A mathematical graph, $\mathcal{G}$, is formally defined as an ordered pair $(\mathcal{V},\xi )$, where $\mathcal{V}$ is a finite nonempty set of vertices or nodes and $\xi$ is the set of edges or links between such nodes $\xi \subseteq \left\{\right(u,v\left)\right|u,v\in \xi \}$. Complex networks are graphs in which vertices represent physical or virtual items and edges represent the interaction between them. In this paper we refer as nodes and links to complex networks elements; the same elements are, respectively, referred as vertices and edges within the graph theory framework.

2.1. Graph Theory: Basic Concepts

Graph theory is the subject of mathematics specifically dedicated to the study of graphs [26]. In order to approach further analysis for graphs, there is a need to represent them as matrices. A common way to do it is by defining the adjacency matrix, $\mathcal{A}$. In case of undirected graphs, $\mathcal{A}$ is symmetric and its elements have values $a_{ij}=a_{ji}=w_{ij}$ if nodes i and j are directly connected and $a_{ij}=a_{ji}=0$ otherwise. A directed graph is such that the edges are directed from one vertex to another. In such case, the matrix representing a directed graph is the so-called (oriented) incidence matrix. The sign of the matrix elements distinguishes between outgoing and incoming edges to each node. A common criterion is to give a positive sign for outgoing edges and a negative sign for incoming edges; however, the opposite sign convention is also in use.

Since the physical and performing characteristics of every link may vary, it can be considered to work with weighted graphs and their respective adjacency matrices defined by $w_{ij}>0$. It can be understood that the unweighted graph is such that $w_{ij}=1$ for all i and j. The adjacency matrix for directed graphs does not need to necessary be symmetric. In an undirected graph, it is defined the degree of a vertex as the total number of vertices directly connected to it (adjacent vertices). The concept of node degree of directed graphs is split between outdegree (number of outgoing edges) and indegree (number of incoming edges). If the degree of a vertex is 0, then it is a singleton or isolated vertex.

Note that all the properties and models for graphs and complex networks developed herein consider, by default, simple graphs. That is, there are not considered parallel edges connecting 2 vertices or any loop connecting a vertex to itself. These characteristics define the so-called multigraphs which are out of the scope of the current paper.

2.2. Complex Networks Models

Complex networks are instances of real-world graphs. They include examples such as the Internet [27], social networks [28], supply networks [29], metabolic networks [30], and critical infrastructures [31], among other engineered systems.

Table 1. Elements related to nodes and links for a variety of engineering systems.

	Nodes	Links	Topology	Flow	Quality of Service	Challenges
Electricity	transformers, users	power lines	quasi-planar, radial	electric load	continue service	peak demand
	substations	cables	small-world, core-p.	Ohm’s, Kirchhoff	meeting demand	energy cuts
Gas	transformers, users	transmission lines	quasi-planar	liquid gas	composition	safety
	gas stations	pipelines, valves	small-world, core-p.	Floyd algorithm	meeting demand	energy cuts
Water	tanks, users	pump stations	quasi-planar	water	drinking water	low pressure
	reservoirs	pipes, valves	core-periphery	hydraulic laws	quantity, pressure	contamination
Transportation	cities	railways	quasi-planar, radial	vehicles, commodities	safety	accidents
	stations	roads, streets	qcore-periphery	circulation rules	synchronisation	delays
Telecom.	computers, routers	cables	quasi-planar, radial	voice, data, video	quality	disruption
	peripheral devices	wireless	core-periphery	many-to-many	speed	ubiquity

shows examples of engineering systems and their network characteristics.

In theory, graphs can take any topology. However, most of them are analysed either by random graphs [32] or by a completely regular distribution. This is not the case of complex networks, where underlying mechanisms provide the network of neither random nor regular structures but follow some distinctive patterns (see

Table 2. Some general properties (randomness, mean path length, and clustering coefficient) of main theoretical graph structures: Summary of parts of Albert and Barabási [59].

Graph Structure	Randomness	Mean Path	Clustering Coeff.
Regular graph	Ordered	Long	High
Small world	Medium	Medium	High
Scale free	Medium-high	Short	Medium
Random graph	Disordered	Short	Low

). Some of these structures are the following:

• Small-world network [33]: The paths between two randomly chosen network nodes is relatively short (usually scales with the logarithm of the total number of nodes). Therefore, in a small-world network, nodes that are not directly neighbours of one another are connected by passing just through a small number of other nodes in between.
• Scale-free network [34]: In a random graph, the node degree distribution for all the nodes in the network follows a Poisson law. However, in complex networks, this distribution often is a heavier right-skewed one and it is better described by a power-law distribution function following the relation $f\left(x\right)=x^{-k}$, where x represents the node degree and $k>0$ the law parameter. Such distribution indicates that some nodes have many more connections than the rest of the network. As the power-law function is invariant with respect to the scale, networks with node degree following this distribution are named “scale-free networks”.
• Planar or quasi-planar networks [35]: A planar graph is such that there are not crosses between edges. That is, the edges intersect only at their endpoints. These types of graphs are naturally sparse as they have the same order for the number of edges as for the number of vertices. Planarity or near-planarity propriety can be taken into account to simplify the network analysis of real-world applications. These include street network representation [36,37], road networks [38], water distribution systems [39], data networks [40], and general network flow problems [41].
• Community structure [42]: This structure happens when subsets of nodes within node–node connections are dense but between which are less dense. Communities in a social network straightforwardly extend to applications in biology [43], ecology [44], engineering [45], and industry [46], among others. The property of modularity [47] is often used for detecting community structures. Modularity measures the strength of the division of a network into modules (clusters or communities). This is defined by the fraction of the edges that fall within the given groups minus the expected fraction if edges were distributed at random. The value of the modularity lies in the range $[0,1]$.
• Core-periphery structure: These are structures in networks that present a set of densely connected nodes (core) and a set of sparsely connected nodes (periphery) [48]. Although the most widely studied network structure is that based on the concept of community, core-periphery networks have also emerged as structures of high interest on complex networks modelling [49].

2.3. Complex Networks Measures

In addition to the common structures of complex networks, there are other network properties and descriptors. This is the case of the centrality measures that are widely used for describing the network connectivity. The following are the most common centrality measures:

• Degree centrality [50]: This is defined as the number of links incident upon a node. That is, a node with higher degree centrality will be supplied easier by any item flowing through the network.
• Betweenness [51]: This is a measure of the relative number of shortest paths from all vertices to all others that pass through a node.
• Closeness [52]: This measures the average distance between the network nodes. The information that this measure provides is on the density of nodes that exists in a network together with an idea of how well each node is connected with the network in terms of geodesic distance.
• Eigenvector centrality [53]: This assigns relative scores to all nodes in the network based on the concept that connections to high-scoring nodes contribute more to the score of the node in question than equal connections to low-scoring nodes. Google’s PageRank [54] and Katz centrality [55] are variations of this concept.

All these measures can also be defined at the link level in addition to the nodes. Thus, the previous definitions can straightforwardly be extended to measures such as edge-betweenness, edge-closeness, and so on.

Additional statistical measures are as follows:

• Transitivity or clustering [56]: This property is based on counting the number of triangles in the network—two linked nodes each of them linked to other third node in common. This is ultimately a measure quantifying how network nodes tend to cluster together.
• Degree distribution [57]: A network node degree is the number of links connecting with that node. This has associated the following measures:

  –

  Degree density: This measure is regarding how strong the vertices of a graph are connected.

  –

  Degree-correlation measures: This is computed between nodes of different nature or function for the network (degree assortativity) [58].

2.4. Percolation and Node Ranking in Complex Networks

In a context of risk analysis and resilience assessment of engineered systems, it is worth mentioning how to develop complex network-based measures useful as surrogate indices or estimations of the network performance [60]. Usually, these measures are based on the so-called percolation analysis. Percolation analysis measures the consequence of nodes/links removal from the network with respect to how the typical length of a path connecting pairs of nodes increases, eventually leading to a disconnected network (infinite distance). Network resilience is a measure on how network performance indices may vary after removal of such nodes and/or links out of the network. From a more general perspective, percolation may be understood as a methodology for ranking nodes in complex networks. This subsection presents as well alternative solutions relying on concepts of diversity in connectivity (vitality) and other solutions based on computational Epidemiology processes.

2.4.1. Percolation Analysis in Complex Networks

In Stauffer and Aharony [61], percolation theory is presented as a method to analyse cascading failures in networks. Percolation models several types of network failures ranging from a single node disruption to a scenario in which a critical fraction of the network components have failed [62]. Within a complex networks framework, these failures are modelled by removal of the associated node/link elements. A fully operative network can become into nonfunctional and disconnected network as increasing the percolation of its components. Given the analogies between percolation theory and cascading failures, percolation has been widely used for risk analysis and resilience assessment [63].

Monte Carlo (MC) methods are key for percolation in complex networks [64]. MC methods generate random processes aiding approaching complex or large-scale problems [65]. They can be understood as a sampling mechanism that assigns a probability per node to be removed or to remain in the network configuration [66]. In this way, it is possible to simulate several random disruption scenarios and to check the global consequences at removing a series of links or nodes. Note that removing a node consequently removes its connection links. Li et al. [67] applied percolation theory to modelling bottlenecks in transportation networks. Carvalho et al. [68] found it also suitable for the resilience assessment of gas networks. Percolation analysis was also investigated in water networks by the works of Torres et al. [69] and also by [70]. Chen et al. [71] used percolation analysis to approach cascading models for cyber-physical power systems.

2.4.2. Node Ranking in Complex Networks

MC methods are also key for node ranking and prioritisation of assets to further management and rehabilitation plans. For instance, Hui [72] proposed MC methods as a criterion for prioritising network assets in order to approach a reliability ranking for maintenance issues. The following bullet points gather some main features of node raking in networks.

• Percolation centrality is a way to better assess the network nodes importance. This measure enhances the purely centrality-based measures with node information with respect to the percolation state [73], making it to vary with the network dynamics of the propagation processes [74].
• Vital nodes are defined by their topological role in the network as well as by their function and performance within the whole system [75,76]. These functions range from network synchronisation [77] to information spreading [78].
• An alternative for locating sensitive nodes to trigger cascading failures comes by borrowing models from epidemiology [79,80,81]. How virus spread through a network has a direct parallelism to the way failures can happen at infrastructures [82]. Epidemiology models have been already adapted to aerospace infrastructure [83], transportation networks [84], and urban water networks [85], among others.

2.5. Evolving and Multilayer Complex Networks

The recent advances in real-time monitoring of engineering systems are among the main reasons why evolving complex networks should be considered further [86]. The network assets status, their properties, and even their existence vary over time in response to exogenous variables and given the dynamic nature of the network flow. Barrat et al. [87] pointed out that these variations need to be considered when modelling engineering systems through changing topology complex networks.

Understanding and modelling evolving networks have enabled the development of a wide and diverse range of ranking algorithms that take the temporal dimension into account [74]. To approach this challenge, Kim and Anderson [88] presented the temporal node centrality concept. This directly extends the well-known centrality metrics by representing the dynamic case through a static network with directed flows. An alternative methodology is based on identifying network hubs and on describing how they change over accumulation-time intervals [89]. Shekhtman et al. [90] showed that dynamic complex networks are suitable to consider failures and recovery time of nested networks configurations representing power grids, transportation systems, and communication networks.

Evolving complex networks can be understood as a special case of multilayer networks. A multilayer network is a network with more than one dimension. This is often approached as an adjacency multidimensional array (tensor) of which dimension can be reduced by constraining the network space or by applying operators for flattening the tensor into a matrix [91,92]. It is possible to analyse multilayer complex networks by generalising main network descriptors such as those on degree centrality, clustering coefficients, eigenvector centrality, and modularity [93,94]. Diffusion dynamics [95], failure spread processes [96,97], percolation analysis [98,99], and MAS simulations [100] have also been developed for multilayer networks modelling. Milanović et al. [101] showed how multilayer networks aid to model interconnected critical infrastructures. These systems’ performance depends on a hierarchy of their parts that should work synchronised. These are the physical system, a hardware and software system aiding management and control of the physical assets, and an organisational system in which there is carried out the interrelationship between various infrastructures and/or elements of the same system.

3. Multi-Agent Systems

Multi-agent Systems have been in the research arena for at least 40 years now. Their foundations date back to around 1980 when these systems were identified as a branch of the distributed artificial intelligence (DAI) field [102,103]. For the purpose of this review, we distinguish between multi-agent system (MAS) and agent-based modelling and simulation (ABMS).

• MAS is the sub-field of DAI, originated as an approach to tackle complex problems, with a distributed nature, by splitting work among cooperative computing units (agents) that plan, reason, and communicate with each other to execute their part of the solution [102]. The essence of a MAS is its ability to enable solutions beyond the individual capabilities of each participating agent [104]. Hence, the role of agents as part of a society and the mechanisms for coordination and cooperation with others are fundamental characteristics of any MAS [5].
• ABMS is the approach for representing repeated interactions of agents within a social system [105,106]. From this perspective, a multi-agent system is simply a network of dynamic entities called agents [107]. ABMS focuses on providing tools for observing and analysing the individual and collective behaviour of agents in a simulated environment. Different sciences and engineering disciplines have benefited from ABMS by representing humans [108], animals [109], financial traders [110], machines [111], and other active entities. ABMS is used as a tool to explore self-organisation and emergent behaviours and also to evaluate MAS theories, architectures, protocols, etc. at a macro level, that otherwise would be costly, time-consuming, or even impossible to achieve.

3.1. Agents and their Properties

Despite the progress made in last decades, there is no agreement about what an agent is and what its essential properties are. Some authors identify actions as distinctive characteristics of agents [112]. Hence, agents are action triggers with a wide spectrum of complexity in the process that lead to every action. Other researchers use the notion of agency to distinguish between two types of agents: those that exhibit properties attributed to hardware and software systems, i.e., autonomy, social ability, reactivity, and pro-activeness, and those that exhibit properties normally attributed to humans, e.g., based mental or emotional notions such as knowledge [113]. From this analysis, agents are regarded as computer systems that perform autonomous actions within the environment they are situated in pursuit of meeting their objectives [5]. Agents can also be seen as intentional systems, with representations of the mental attitudes such as belief, desires, and intentions [114]. From an ABMS perspective, agents are autonomous, not necessarily computing, entities that are proactive and interact with others [115]. The agent’s behaviour is defined by a set of simple rules to respond to local events in a certain environment, hence leading to the emergence of a system behaviour as opposite to predefined rules for the overall system behaviour [116].

The key properties of agents are presented in

Table 3. Summary of main agent properties.

Criteria	Property	Description	Relevance
Location	Situatedness	Agent is situated within and is a part of an environment [117].	Essential
	Mobility	Able to travel across networks [104] and transport itself among different machines [117]	Optional
Abilities	Autonomy	Different views. Absolute: the ability to manipulate its own capabilities [118]. Relative to other entities, subjects or functions, ability to perform an action with independence of others [119], e.g., operate with no human guidance [104].	Essential
	Perception	Ability to perceive environment through sensors, with a perception referring to an instant input and a perception sequence to the complete history [120].	Essential
	Communication	Agents communicate with other agents, even people [117]	Essential
	Adaptation	Agent learns, i.e., uses previous experience to change environment [117]	Optional
Behaviour	Reactivity	Agent responds in timely fashion to changes in environment [117,121]	Essential
	Pro-activeness	Agents have a purpose (goal) beyond acting in response to environment [117]; they take the initiative to satisfy these goals [121]. Agents follow a deliberation process that includes reasoning, planning, negotiating, and coordinating with other agents [104].	Optional
	Rationality	Agents are expected to choose actions that maximise their expected performance [120]	Optional
	Social	Agents have dynamic interactions with others that influence their behaviour [106].	Essential

; we group them according to criteria where these come from. We also provide a relevance assessment indicating if these are mainly found in literature as essential or optional properties.

As consequence of the different interpretations of the agent notion, multiple authors, mainly from an MAS perspective, have come out with classifications. Authors of Reference [104] present a multidimensional scheme where, based on agent’s properties, tasks, and structure, they propose 7 types: collaborative, interface, mobile, information, reactive, hybrid, heterogeneous, and smart agents. Another classification [117] incorporates novel agent’s properties such as flexibility (lack of scripts on agent’s actions) and character (believable personality and emotional state). To date, reactive and deliberative (goal-oriented) agents have been widely used to differentiate key behaviour and properties of agents. It is also worth noting from Franklin’s classification [117] the identification of biological and robotic agents in addition to the computational ones.

3.2. Multi-Agent Models

Multi-agent models include representation for the individual agents, their interaction, and the environment [106]. The relevant agent definition depends on what they are representing, e.g., humans, machines, particles, organisms, or computing systems. The key components of multi-agent models found in literature are illustrated in Figure 1 and described below.

The agent’s capabilities are influenced by the adopted notion of agent (see Section 3.1) but commonly include communication with other agents, a mechanism for sensing the environment in order to capture the state of the properties of interest and a mechanism to act on or to influence others and the environment. The knowledge model covers relevant information for the agent to operate, including its own state as well as that of the environment and other agents [106]. In ABMS, knowledge is usually simplified and modelled as set of properties and values or as a state-machine [122,123]. More complex knowledge representations have been proposed, mainly from a MAS perspective, for example, using the fuzzy cognitive model in Reference [124] or ontologies in References [125,126,127]. The approach for representing agent’s knowledge is tied to the decision-making process. The decision-making approach uses agent’s current knowledge to trigger actions given its capabilities and, hence, drive individual agent behaviour. A simple approach to decision-making in ABMS is based on the generation of random numbers [128]. Bringing more rationality to the process is possible by using predefined condition/action rules where a state-machine captures also the conditions for the transitions among them [123].

The work on intentional systems [129] has influenced more complex models that try to mimic the way humans make decisions. In the belief-desire-intention (BDI) model, the information perceived, by the agent as facts, are the beliefs, and the desires and intentions represent a pool of future states the agent might reach, with the difference that an agent is only committed to working towards the intentions. Other cognitive, conceptual, and mathematical models for modelling human decision-making in ABMS are reviewed in Reference [130]. From a MAS perspective, the decision-making models have been widely studied in the context of a more complex reasoning process and they are the distinctive feature of the agent architectures. The three main classes of architectures include reactive, deliberative, and hybrid [131]. For example, the BDI models are a key reference for building deliberative architectures [132].

The interaction models drive the collective system behaviour and enable communication between agents. On top of this communication, the coordination model enables the management of interdependencies between agent’s activities [133]. The interactions between agents might happen spontaneously within the environment (e.g., in case of agents making decisions randomly) or agents might try to achieve goals rationally which requires interactions to follow a defined model. In the later case, different forms of coordination have been explored as it is an essential condition for complex collective behaviours including conflict-resolution, cooperation, organisation, collective learning, planning, control, and optimisation.

The coordination between agents can be based on direct or indirect communication. In the first case, extensive work has been done around definition of languages for communication between agents, for example, using different types of messages according to the purpose and categories of the speech acts theory [134,135]. Indirect coordination is possible when agents observe updates on the environment state produced by other agents; for example, authors of Reference [136] present a coordination model inspired in the ant colony behaviour that does not require direct agent communication. One of the main problems in coordination is consensus, i.e., agents agreement over a certain value of interest, depending on their states [137]. Consensus can be achieved, for example, by specifying rules of information exchange [138].

Different techniques for achieving coordination have been proposed. Authors of References [139,140] present a review of different approaches for coordination that include organisational structures (defined a priori by agent’s responsibilities, capabilities, connectivity, and control flow), contracting (to accomplish a set of tasks), planning (centralised or distributed plans that drive their behaviour/operation), and negotiation (seek agreement with others). In MAS literature, there is no clear distinction between negotiation and consensus. Other nature-inspired models for coordination include stigmergy (indirect coordination through, for instance, the environment), chemical coordination, physical coordination, and biochemical coordination [141]. Animal organisation has inspired models for addressing motion of agents. For example, swarms have enabled distinction of different group of agents as introduced by [107] and presented in

Table 4. Swarms and formations; differences according to Reference [107].

	Swarm	Formation
Structure	Low	High
Quantity of agents	High	Low
Motion dynamics	Uncertainty	Deterministic

. Figure 2 identifies some of the most common coordination models found in literature.

The environment provides the space where agents interact; it imposes constraints for their operation (e.g., boundaries) and provides resources they can consume [106]. The environment is also the place where relevant events under study happen so the agents can perceive them and consider them in their operation. Some examples of environments in ABMS include simulated geographical location [142], living organisms such as animals [143], the financial markets [144], and a product-manufacturing shop floor [111]. In MAS, the environment is usually a software platform that offers services such as communication, life-cycle management, or advertisement of agent’s services, for instance, JADE [145] or EVE [146].

3.3. Agent-Based Complex Networks

Agent-based complex networks is a research topic directly related to evolving complex networks. This proposes a new management framework where each system’s element cooperates with others towards their own individual targets, also achieving a global solution. Agent-based systems suit well at dealing with the nowadays ubiquity of sensors, smart-meters and, in general, cyber-physical systems. Thus, agent-based systems are of major importance for monitoring and controlling engineered systems. They are also straightforwardly related to the distributed information and intelligence behind to manage the Internet of Things of assets placed in a network [147,148]. Agent-based solutions have shown to be suitable for smart-grids [149,150], transportation [151,152], water distribution systems [153], and telecommunication infrastructure [154]. The works of Cardellini et al. [155], Setola et al. [156], and Iturriza et al. [157] showed how MAS are suitable to model network interdependence.

In a more-than-ever interconnected world of monitoring and controlling engineered systems, there is the emergence of cyber-attacks which are, today, an important concern for system processes functioning [158,159]. Cyber-attacks typically interfere with the supervisory control and data acquisition (SCADA) systems. In normal conditions, SCADA is ready for leading industrial automated control of systems at near real time. However, cyber-attacks target those systems misleading them and even blocking their readings while they are disguised as normal commands [160]. This directly affects the natural system performance. In gas transmission [161,162], SCADA system controls and monitors moisture, quantity, pressure, and temperature of the network of pipelines. In water distribution systems, cyber-attacks can be considered to control unexpected scenarios that can potentially produce shortages and can reduce the water quality for public consumption [163,164]. In the case of smart-grids, cyber-attacks can cause damage at connecting physical assets [165,166]. In transportation, cyber-attacks might be in the way vehicles dynamics and monitoring is collected and analysed [167]. In telecommunication systems the cyber-attacks may directly affect the network topology by line-addition, line-removal, and line-switching [168]. Also in telecommunications and mobile networks, cyber-attacks can directly inject false data in the network, spread malware, send spam, or collect information for illegal purposes [169]. Self-organised networks [170,171] and virtual network functions [172,173] aid to get an early detection and better mitigation of cyber-attacks in mobile networks [174,175].

4. Control and Optimisation of Complex Networks and Multi-Agent Systems

This section presents how MAS and complex networks have been used to address a wide set of control and optimisation problems arising in systems engineering. We particularly focus on representative applications related to manufacturing processes and critical infrastructure management. Thanks to them, we show how complex networks and MAS are able to model, control, and optimise engineering systems of main importance.

4.1. Complex Networks for Control and Optimisation

Controlling complex networks attempts to guarantee that such networks can reach a targeted performance. This is through monitoring how their state evolves on time, under a range of scenarios, and what actions are better placed to reach the aimed control. Control is also achieved through MAS, as it is stated below. However, network dynamics and topology play a key role in the success of such control process [176]. This can also be checked in the paper of Ding et al. [177], where the authors select network key-nodes to be connected to external controllers. Such key nodes can be selected by their relative importance on propagating errors/information or by their connectivity. Overall, this is a particular case of the so-called landmark nodes [178]. Landmark nodes are special nodes in the network, typically aiding to speed up internal computations such as centrality measures. This is of interest in case of dealing with large-scale networks or with near real-time operations [179,180]. For instance, in the work of Giudicciani et al. [181] landmark nodes are selected as nodes in the boundary of network communities (nodes with links connecting nodes of other community). This set of landmark nodes shows to be suitable to get faster computation of the shortest paths in addition to improving the overall management of complex networks related to critical infrastructures.

Control is naturally related to optimisation tasks. To this end, there are several works on network dynamics, topology, and design to optimise resilience [182], recovery [183], connectivity [184], performance [185], and even network control [186]. Overall, complex networks couple with MAS in many optimisation processes. Complex networks have shown to be essential to assign agents (or agents of different breeds) to network nodes depending on their importance. Still, MAS for optimisation gain the space, distance, and neighbourhood notions thanks to the complex network topology. To this end, a network of agents (likely different to the complex network aimed to be controlled or optimised) may also be necessary for an optimal activity between agents, so providing an enhanced response.

4.2. MAS Architectures for Control and Optimisation

The control architecture determines, among others, the components of the system, the responsibilities, and interactions [187]. The control architecture is tailored for each system and might include specific domain functions or entities. However, there are some common abstractions and approaches that can be reused across different problems and domains. This encompasses terminology, structure, a standard template of components and their relationships, and even examples [188]. There are three main distributed control architectural approaches: hierarchical, heterarchical, and holonic [189]. These three are briefly introduced in the following bullet points:

• Hierarchical architectures imply components or functions are structured along two or more levels with the upper levels having broader view and influence over the lower levels [190]. They also have a command/respond communication across the levels with decision-making in the higher levels, whereas the modified hierarchical architecture shifts the command-based communication to a coordination approach where subordinates might interact with each other within a level in order to complete some of their tasks, without requiring constant instructions from a higher level [189]. Hierarchical architectures are usually rigid and lack flexibility to adapt to changes and disturbances [187].
• Heterarchical architectures lack a direct controlling component; instead, the supervision is spread across the system and cooperatively carried out [190,191]. The key aspect is that functions are allocated in distributed entities that make decisions with a local perspective. These are autonomous entities and use communication protocols to cooperate with other peers without a central coordinator. The horizontal distribution nature of the functions implies there is no consideration of a global view, which prevents autonomous entities from reaching global optimum goals and incorporates unpredictability to the system.
• Holonic architectures intend to overcome disadvantages of hierarchical and heterarchical approaches by offering a hybrid solution [188,189,192]. In a holonic architecture, the system is structured around “holons” (entities that are both a sub-whole from an interior perspective and a part from a system-wide point of view [193]) that can be arranged in different forms according to concrete system requirements. The generic holonic form combines distributed and centralised optimisation by enabling holons to react timely to disturbances and by considering updated local views when making decisions while operating under the view of a central coordinator holon.

MAS organisation may result in control as shown above and also in optimisation processes. This optimisation takes place thanks to distributed, local objectives of agents (or aggregation of agents) that negotiate/coordinate/cooperate with other agents (or aggregation) towards a global optimum [194].

4.3. Representative Applications

This subsection presents a number of representative applications of complex networks and agent-based control tackling key challenges in different systems engineering domains. We give special emphasis to application requirements and key architectural aspects with designs that have led to concrete implementations either of real systems or prototypes.

4.3.1. Supply Chain and Manufacturing Networks

Control of manufacturing processes is a challenging work-stream that has obtained attention from researches from both MAS and complex networks. The production processes incorporate heterogeneity of functions, goods, workflows, work products (i.e., orders), and resources that are usually constrained and require efficient utilisation while ensuring the quality of the end products. Several solutions have been proposed to manage and control production of goods in shop floors [192,195,196,197,198,199,200]. The key control requirements cover scheduling, simultaneous processing or orders, quality assurance, real-time customisation, and context-aware servicing and maintenance. These requirements can only be met by flexible and agile factories able to reconfigure and adapt to changes, even at late stages of the manufacturing process. The need of a supervision function aligned with this dynamics is implicit to these requirements.

• Complex networks: Supply chain and manufacturing processes have a proper research avenue within a network science framework [201]. To this end, simplistic chain models can be approached by complex systems allowing, for instance, a deeper interpretation of the relationship between different supply actors [202]. The paper of Hearnshaw et al. [203] is a pivotal work on supply chain network theory where complex network developments are shown to be a useful working environment. There also are specific applications, as it is the analysis of supply chains for the aerospace industry [29]. Complex networks make, then, possible to extract useful information such as nonlinear pathways between firms, geographic locations and industrial-sectors communities, and connectivity hub firms.
• MAS: In the last two decades, distributed approaches have gained attention, becoming a solid alternative to monolithic architectures [204,205]. Together, holonic manufacturing systems (HMS) and multi-agent systems help to overcome the limitations of the centralised approaches such as lack of flexibility, agility, dynamics and reconfiguration features [198]. In HMS, the scheduling is intended to be realised from the cooperative interaction of holons while ensuring that global factory concerns are addressed, sometimes, with some degree of central coordination [192]. The agent notion has been used as both a solution domain abstraction and its corresponding software. When used as software building blocks, agents complement broader engineering concepts such as holon [206], intelligent products [207,208], and self-service assets [199].

4.3.2. Electricity Power Grids

Electrical power is essential for basic services such as providing lighting, heating, cooling, and refrigeration in the built environment. Computers, mobile phones, and other domestic appliances use electrical power. Electricity is generated at power plants and moves through a complex system called the “grid”. The grid is made of electricity substations, transformers, and power lines that connect electricity load from source providers to users.

• Complex networks: Electrical power grids can be considered complex networks [209]. The nodes are power plants and distributing and transmission substations [210]. The links are the power lines which may have different voltage (see Table 1). At the urban level, grids often become a higher inter-connectivity system aiming to get reliability in the supply. Smart grids [211] enhance the traditional electricity supply by adding digital technology which allows utilities and customers to receive information from and to communicate with the grid. This benefits optimal energy generation, electricity and meter-reading cost, and reliability in case of interruptions and blackouts [212]. The work of Das et al. [213] shows how topological, physical, and electrical features of a power grid provide complementary information. As a consequence, all of such features should be considered further to better address near real-time challenges in power grids.
• MAS: Smart grids have been widely supported by SCADA systems. However, other distributed approaches have been used to overcome limitations of traditional supervisory control systems [214]. Particularly, multi-agent systems have become an enabler of distributed control for the power systems, providing some of the supervision functions without a hierarchical or central supervisor. The requirements for control in power systems include reliability, economic efficiency, and capacity to support from individuals to large industrial customers. Some of the problems addressed include market operations, time-sensitive control, service restoration, and system evolution/flexibility. The authors of References [215,216] present a solution to improve, in real time, the energy market performance with a large number of multiple production and consumption units, each one with different objectives. Based on an auction model, agents use a price vector to gather the quantity of bids for a particular energy demand, and this enables them to decide on the supplier considering the global desire [216]. A similar approach is presented in Reference [217], where a distributed architecture with a single control layer and multiple distributed agents is organised around a set of central facilities.

4.3.3. Transportation Systems

A transportation system is a spatial network which permits either vehicular movement, flow of people, or products supply. The transportation system comprises transport infrastructure, vehicles, and equipment. This should also be considered the transportation assets operation and service to the end-user. This is of main importance for society development and well-being as millions of citizens worldwide use transportation systems on a daily basis.

• Complex networks: Instances of transportation systems are such important infrastructures as roads and streets, railways, and airline networks. All of them are organised in network patterns [218,219] (see Table 1). For instance, by considering urban streets and roads, it is possible to take segments of these routes as links. The intersections and ends are considered as nodes [220,221,222]. Another example is the associated network to a city underground transit as shown in Figure 3, where the nodes represent the metro-stations and the links are the train lines connecting such stations. Some of the most common issues in transportation and communication systems are related to link and node congestion [223] in a network which often have a scale-free topology.
• MAS: Smart transportation systems [224] also named intelligent transport systems aim to achieve traffic efficiency by minimising their associated issues such as traffic congestion [225]. Having real-time data of the network status, it is possible to release traffic alert messages and public safety messages. Associated with the concept of smart transportation systems comes the idea of smart, resilient, and energy efficient cities. Baronti et al. [226] proposed an integration of energy storage systems for the smart transportation and the smart grid. A distributed hierarchical approach for control of automated highways is presented in Reference [227]. The architecture is based on a hierarchical control where supervision functions are distributed among four layers. Two layers (network and link) are in the roadside, and two other layers (coordination and regulation) are in the vehicle.

4.3.4. Water Distribution Systems

Water supply services are naturally related to food availability, health, and hygiene. However, they are also key for energy, manufacturing, and other fundamental services. Drinking water comes from reservoirs from which water flows through pressurised pipes to tanks (to storage water for daily supply operations in small urban areas) and demand points. The risks in water distribution systems are associated with pipe bursts, contamination events, and lack of resources during drought periods that are usually of high demand. These events can cause socioeconomic losses but also directly affect the citizen health and well-being.

• Complex networks: Water distribution systems can be considered as complex networks where nodes are water sources and demand points and links are water pipes, valves, and pump stations [228] (see Table 1). The works of Herrera et al. [229] and di Nardo et al. [230] present instances on how complex network analyses provide useful approaches for the operation and management of water distribution systems. Features and positions of valves and pumps have special relevance for the global water network performance. Smart water networks extend the traditional water distribution system elements by including assets such as sensors and flow-meters, providing information of the network performance. Among other advantages, Candelieri et al. [231] highlighted that the cyber-physical water system is key for more efficient water distribution network management, hydraulic performance, and optimised network protection [164].
• MAS: Water networks are heterogeneous, ranging from ad hoc farm irrigation systems to critical water infrastructures. The authors of Reference [232] use agent-based modelling to sample large size water supply networks graphs and, then, to propose their further division into district metered areas for management purposes. Similar work is found in References [233,234]. From an asset management point of view, it is worth mentioning the work of Ayala-Cabrera et al. [235] since the authors use an agent-based system to locate and classify buried pipes. Authors of Reference [236] propose an architecture based on a SCADA system that uses model predictive controller (MPC) [237] techniques for controlling an automatic water canal. A key contribution of this architecture is to extend the standard SCADA system with capabilities for complex mathematical processing via a dynamic data exchange (DDE) protocol.

4.3.5. Gas Transmission

Gas supply is essential for heating, hygiene, and cooking food. The gas is transported by high-pressure transmission pipelines from the production facility to the entry point (gate station) of the distribution network. The transmission systems are made by high-pressure pipelines, compressor stations, and storage facilities among other elements. From the gate station, natural gas moves into distribution lines towards users at home. The distribution network consists of smaller distribution pipes which typically supply urban areas. Each distribution system is connected to the higher pressure transmission system at a pressure reduction station.

• Complex networks: Pipelines for gas transmission can be considered a complex network [238]. Gas pipelines and compressor stations are network-links, and underground storage systems and gas stations are network nodes (see Table 1). Smart gas grids are controlled near real time to meet the time-varying gas demand and to interact with the electrical power smart grid [239]. In this regard, Bliek et al. [240] pictured an ideal smart gas grid as the one that is able to communicate with the smart electric power grid for an improved energy distribution. Brown et al. [241] also pointed out the smart grid capacity to transport nonconventional gases such as biogas or syngas.
• MAS: The authors of Reference [242] introduce a holonic architecture for the control of continuous production complexes. They work with the case of oil production, where various complex processes take place involving extraction, transportation, treatment, and delivery of oil and secondary products (e.g., gas). Holons are linked to production groups that are a specialisation of production units (PU) representing oil wells or flow stations [188]. The PUs aggregate orders, resources, and a component of process supervision and control. The supervisory layer carries out standard functions of measurement, identify state changes on discrete process, and update the state. Holons are implemented with temporary agent programs (TAP) that negotiate to accept mission assignments. A supervisor agent determines the production method to apply for achieving the missions and requests for external resources to other PUs.

An example of the interdependence between the electric power network and the natural gas system is shown in Figure 4. Several elements for the gas transmission such as PUs and pipeline compressors directly rely on the electricity supply.

5. Discussion and Research Directions

Advances in complex network analysis have boosted their ability to represent more realistic examples of real cases. For instance, there have been important developments on weighted networks [244] and networks with time-varying characteristics and topology [245]. Multiscale complex networks [246] and networks of networks [247] have been recently a very active research topic. Along with the essentials, the current paper already introduced highly advanced methods in complex networks, MAS, and their combination. However, there are some methods and technologies of particular interest, having the potential of becoming main topics in future research. This section highlights important insights and research avenues.

5.1. Discussion

This paper presents a comprehensive review of complex networks and MAS. Both are of main importance for control and optimisation in systems engineering. Therefore, one of the main contributions of this review is the emphasis on real-world systems such as critical infrastructures, manufacturing processes, and supply chain networks. In all cases, complex networks and MAS have shown to be essential for their control and optimisation. In a world where society and systems are more connected than ever, it highlights the importance to analyse how such systems are related between themselves and how their optimal management provide further socioeconomical benefits. This is the case of interconnected infrastructures such as electricity and gas, or telecommunications and transport. Still, there is a emergence of research directly focused on hybrid systems development. In contrast to systems interconnection, hybrid systems naturally ensemble two or more systems into a new one that cannot be understood without the others. This is the case of electricity and transport networks [248], both directly converging for the management of electric bus fleets in metropolitan areas [249]. Another example is the novel concept for water distribution systems integrated into urban smart grids for energy recovery and production [250]. On top of this, any engineering system (single, interconnected, or hybrid) may be considered to have several levels of domain. That is, a physical system (already having several levels if it is an interconnected or a hybrid system) is often coupled with a control system (hardware–software system that can be designed as MAS and organised as a network). Another way to understand additional domain levels of a system is by including the level related to the workforce networkas well as the customers network. We can also combine all of the previously mentioned possibilities. In all the cases, the literature review presented herein represents the fundamentals for the optimal management of these systems, their optimal control and optimisation, and their coordination approaches. The following section shows the foreseen work directions in complex networks and MAS in conjunction with emerging technologies.

5.2. Future Methodology Developments

Applied complex networks to systems engineering should come with developments at multiscale, dynamic, and multidimensional systems. On top of this, MAS will play an essential role providing these networks intelligence in their network flow, evolution, and protection (self-healing, resilient design, and so on). Both complex networks and MAS should be integrated in a process in which distributed, networked agents agree in a common objective for optimal systems control and decision making. There is, then, a need for a consensual dynamics in complex networks, working with the streams of time series varying over time [251] and how the agents can reach a consensus at near real time is a research avenue regarding the methodology development as well as the applications. The rest of the section describes other main research challenges for complex networks and MAS in systems engineering.

5.2.1. Complex Networks

• Time series in networks: Temporal networks might well be understood as the study on how network topology and features vary over time. Some approaches use multilayer networks to represent as many layers as time units capturing the network status variations through snapshots at each time. This research framework being relatively new, it should be developed further by conducting proper analysis on the streams of time series data associated to complex networks, that is, through the analysis of the temporal evolution of such streams and how it has an impact on the very network structure and performance. As a consequence, future research will be about statistics and inference in dynamic graphs [252]. The challenge may be extended further to the more general framework of machine learning in networks.
• Graph convolutional neural networks: Convolutional neural networks (CNN) have been mainly focused so far on image analysis. A series of convolutional filters and pooling layers are imposed over the matrix representing such images. The process ends with a layer where the actual learning and the image approached, for instance, is classified. However, there is an emergence of the so-called geometric deep learning in which the CNN input is a manifold or graph-structured data [253]. In the case of CNN over graphs, the input can be the adjacency or the Laplacian matrices associated to such a graph. Then, the learning is similar to the developed for study images since the image input is also a matrix. There is ample room for research on graph-CNN on the analysis of (evolving) complex networks representing engineering systems [254].

5.2.2. Multi-Agent Systems

• Big data and calibration of ABMS: The validity of agent-based models is given by the real data and theories it uses to base behaviour of agents in the model. Thanks to the wide spreading of sensors, we are experiencing an increased ability to capture huge amounts of data related to physical properties from living/active entities. These data can set the basis for modelling, validation, and calibration of agent-based models across different scientific disciplines. A key challenge is how to process these data effectively while providing feedback to the model. The use of supervised, unsupervised, and semi-supervised learning techniques can produce new methods for model calibration, for example, by enabling classification and comparison of key features of the model within a particular observation window or by easing the definition of baselines for predicted behaviours across the model [255,256].
• Breaking down learning phase with MAS: This future research highlights the problem of agent breeds learning to teach other agents within an MAS environment. The initial efforts have been via reinforcement learning [257] in which each agent takes the role of student or teacher, requesting and providing advice, respectively, at the appropriate moments looking forward an improved overall system performance [258]. There is a number of further challenges coming from this approach to deal with complex domains for real-world applications.

5.3. Future Applications in Systems Engineering

Advances on complex networks and MAS consequently lead to advances further in their applications for systems engineering. There are foreseen some key applications as a potential breakthrough in systems engineering briefly introduced in the following bullet points.

• Cyber-physical systems (CPSs): CPSs are representations of physical, spatially distributed systems into a network of sensors and actuators leading to a suitable monitor and control of the system processes. The final aim is to reach optimal decision-making over the system for their optimal management. There are emerging challenges for CPSs to address further in which MAS and complex networks will play a significant role. For instance, in industrial systems such as smart electric grids, there is a need of decentralised, adaptive CPSs framework towards their operative automation and optimal performance [259]. Other systems may also benefit from the use of MAS and complex networks over their CPSs, from smart manufacturing and logistics [260] to food supply chain systems [261].
• Digital twins (DT): DTs are a digital replica of physical assets and engineering systems taking into account their internal functioning and external processes that may affect their performance. Working with digital twins, it is possible to test in advance systems performance under any regular or anomalous scenario to aid the decision-making process and to accurately foresee further systems issues. One of the key challenges for future digital twins is on increasing their dimensionality and complexity. This will make necessary to create a new generation of systems engineering modelling systems relationships and interdependencies at large scale. To accomplish this challenge, it will be necessary to combine a set of relevant methods such as complex networks, MAS and visualisation processes, among others [262].
• Blockchain technologies: Blockchain is a global ledger that records transactions on a chain of blocks. Within a complex network framework, these blocks can be expressed as network nodes while transactions are network links representing the exchanges between nodes. Future research will be based on temporal and dynamic complex networks to model and analyse blockchain technologies. Still, as the information flow is passing through the network, agent-based systems will also have a key role in synchronisation and network control tasks [263]. Furthermore, blockchain technologies offer a truly decentralised environment where new network dynamics and coordination models are emerging facilitating distribution artificial intelligent processes [264].