Simulation of Smart Factory Processes Applying Multi-Agent-Systems—A Knowledge Management Perspective

Abstract

The implementation of Industry 4.0 and smart factory concepts changes the ways of manufacturing and production and requires the combination and interaction of different technologies and systems. The need for rapid implementation is steadily increasing as customers demand individualized products which are only possible if the production unit is smart and flexible. However, an existing factory cannot be transformed easily into a smart factory, especially not during operational mode. Therefore, designers and engineers require solutions which help to simulate the aspired change beforehand, thus running realistic pre-tests without disturbing operations and production. New product lines may also be tested beforehand. Data and the deduced knowledge are key factors of the said transformation. One idea for simulation is applying artificial intelligence, in this case the method of multi-agent-systems (MAS), to simulate the inter-dependencies of different production units based on individually configured orders. Once the smart factory is running additional machine learning methods for feedback data of the different machine units may be applied for generating knowledge for improvement of processes and decision making. This paper describes the necessary interaction of manufacturing and knowledge-based solutions before showing an MAS use case implementation of a production line using Anylogic.

1. Introduction

Industry 4.0, defined as the fourth industrial revolution, leads to changes in the way companies produce their goods and provide customer services. A motor for this change is digitization and change of customer expectations. Key characteristics of a smart production environment are flexibility, intelligent maintenance, IoT for building cyber–physical systems (CPS), interconnected production environments, self-organized adaptive logistics or customer individual engineering [1]. Today customers oftentimes demand extensive options to configure or customize their orders [2], to create individual products no one else in their vicinity owns or to fulfill specific requirements [3]. Competition on the market changes from competition between best products towards competition between best business models leading to hybrid offerings consisting of products and combined (smart) services [1].

The trend of individualization leads to changes in the way the singular manufacturing steps, production processes and overall organization of a factory works as changes must be implemented in the ordering process, supply chain and the production shop floor leading to a more complex and interconnected production environment [4]. For detecting the consequences, but also possibilities and innovative potential of Industry 4.0, Müller, Buliga and Voigt (2020) conducted a survey comparing SMEs and large companies targeting their “exploratory and exploitative innovation strategies” due to Industry 4.0 [5]. They emphasize the necessity of adapting technology as well as business models [5]. Essential are the changes of processes towards digital interconnected processes to realize a network between the involved machines [1]. A key factor for success of a smart factory are decentrality and addressing of singular objects inside the smart environment for autonomous processes and decision making [4].

One aspect of modern manufacturing processes in the sense of CPPS are the generation and consideration of data. On a low level this data allows monitoring of manufacturing steps or involved machines, while on a higher level it might be used for predicting future maintenance activities, optimizing production planning or decision making. At this stage computer science algorithms are getting involved, ranging from simple statistics towards machine learning or even more complex artificial intelligence (AI) algorithms. Lang et al., e.g., published a recent practical example for a knowledge-based approach for improving additive manufacturing [6], while Mirfar, Kadivar and Azarhoushang (2020) analyzed acoustic sounds of emission sensors with the help of machine learning in the form of neural networks for improving grinding processes [7].

Another concrete and widely discussed AI method applicable in industry 4.0 are multi-agent-systems (MAS). The idea of using MAS for industrial application purposes has not emerged recently but has already been in focus previous of the emergence of the concept of smart factories. One example for such an approach would be the one of Lou, Ong and Nee, who discussed in 2009 an use case of applying “agent-based distributed scheduling for virtual job shops” leading towards more flexibility of production, planning and customization [8]. Applying MAS in industry 4.0 offers the possibility to transfer the reality or future design of a smart factory into a simulation model where different manufacturing steps, production machines or systems may be represented in form of depending agents which influence each other. With the help of key performance indicators, enriching the simulation, it is possible to measure, e.g., time, material or safety parameters. This approach may be combined with the digital twin concept, where each physical component has got a counterpart in form of a cyber-model. Gorodetsky et al. (2019) sketch an exemplary lifecycle of an autonomous CPS while applying multi-agent systems consisting of 10 different steps, such as planning and scheduling, forecasting, or learning from experience [9]. Overall, “modeling enables the analysis of materials, production line and product performance… with the aim of improving and optimizing the overall operations and reducing setup costs, errors or machine downtimes” [2]. Gunal even calls simulation the “enabler of Industry 4.0” [3].

The given paper analyzes how the application of MAS, for the planning of a production line, may be used and how the generated data may be transferred into knowledge for smart process improvement. Therefore, a use case of a picture frame production line has been designed. In this scenario customers may configure the type of material, size of the frame and color. The simulation applies agents to represent the different manufacturing steps and their dependencies. It has been executed and evaluated with the software tool AnyLogic [10], defining different KPIs as well as comparing a normal to a manipulated production process.

The structure of the paper covers a summary of the transformation towards a smart factory from a data and knowledge management perspective, before the concept and applicability of MAS as a knowledge-based method for simulation purposes and data analytics in a smart factory is being discussed. In Section 3 the basic concept of the use case will be introduced, before the implementation results will be discussed. At the end there will be a conclusion and outlook for future works.

2. Materials And Methods

This section builds the theoretic background of approaches for a knowledge-driven manufacturing in modern cyber–physical production systems and emphasizes the importance of data for this change in manufacturing from a computer science point of view. In this context, the human–machine as well as machine–machine interaction will be discussed as well as the applicability of multi-agent-systems (MAS) for simulation and adaptation in future smart factories. MAS are an artificial intelligence method providing potentials for planning and optimizing manufacturing processes, therefore offering a possibility of smart manufacturing. The method will be applied later on for the use case implementation.

2.1. Knowledge-Driven Manufacturing

2.1.1. Merging of Physical and Digital Manufacturing

Industry 4.0 motivated a transformation process of “regular” factories into smart factories based on digitization from singular production units, over whole shop floors towards globally interconnected production lines. Smart factories, factories of future, or in general smart production environments apply building blocks such as sensor-based modules and systems, ubiquitous computing, IPv6, Internet of Things (IoT) and cloud computing to establish a cyber–physical (production) system (CP(P)S) infrastructure [11]. CPPS, or single CPS represent a merging of physical production equipment and digital/cyber systems, which leads to changes of production processes, human–machine-communication and emerging machine-to-machine-communication [11]. Gerhard (2017) pronounces the two central aspects of CPPS, one where CPS are integrated into products as well as being building blocks for restructuring the production environment itself. The modern product lifecycle management has to adapt to this development [12], as well as the trend of integrating or involving AI applications or machines, working with the help of artificial intelligence algorithms [13].

Karnouskos et al. (2019) identified the three characteristics of autonomy, integrability and convertibility as the main CPPS pillars [14]. Due to the interconnection, automation and integration of machines and other objects and devices towards CPS, data is generated continuously up to the amount of big data, which in consequence needs to be processed and analyzed to generate engineering value [15]. Core elements for the establishment of CPS are IoT devices, which generate continuous data streams, applicable for different forms of data analysis [16]. A concept called edge computing provides the ability to execute data storage and analysis near or embedded into singular smart objects (e.g., machines on the shop floor), while fog computing allows to gather data from different objects and transfer them, e.g., into a cloud storage for further analysis or archiving [13]. Identification and communication technologies such as RFID, GPS, WLAN or further industrial communication protocols are used to create the necessary inter-connectivity inside the smart factory and to identify the singular objects explicitly.

If an object such as, e.g., a material transport carrier may not be identified explicitly it is not able to receive information for its next destination which in consequence prevents automation. To realize a continuous identified stream of feedback data a “network of interconnected physical and virtual nodes, where each node relates to an arbitrary number of sensors and actuators or external systems” needs to be established [17]. This leads to an inherent characteristic of “interconnected twin cybernetics digital system[s]” [18] or “digital twin-based CPPS” [9], whose gathered and analyzed data allows prediction and supports decision making [18]. In general, the digital twin concept combines “a physical entity, a virtual counterpart, and the data connection in between” [19]. This way, the digital twin ties the physical and virtual world based on the physical product, its virtual counterpart and the data exchange in between. This data may be used for analysis, simulation or real-time updates [20]. According to the IIC “a digital twin is a formal digital representation of some asset, process or system that captures attributes and behaviors of that entity suitable for communication, storage, interpretation or processing within a certain context” [21]. A digital twin may cover physical-based, analytical or visual models and data as well as time-series, transactions or general computations and their respective associated data [21]. Campos-Ferreira et al. (2019) declare that the “Digital Twin is one of the most promising technologies for system simulation and monitoring” [22]. Recent approaches even see a direct possibility for applying blockchain technologies for implementing a digital twin concept and to secure the data generated from the model [23]. In consequence, it might be resumed, that data and virtual models are the core components of the digital twin concept, ergo in combination with data analysis methods some core components of smart manufacturing [24].

This is especially beneficial for the product lifecycle management as product feedback data may be generated implicitly due to analyzing, e.g., generated IoT sensor data in the virtual space. The digital twin, as the virtual representation of the physical model, and the analyzed data allow real-time reflection, interaction and convergence as well as self-evolution due to updating data and continuous improvement [20]. This way a two-way feedback flow in the form of a physical-virtual-connection and virtual-physical-connection may be established [19].

2.1.2. From Data to Knowledge

In general, smart factories and the implementation of CPPS as well as usage of IoT devices lead to the generation of big data as discussed before. The generated data offers an organization different possibilities: (a) Ignore the data, which would be inefficient and a lost opportunity to generate feedback knowledge from production processes, (b) implement ad-hoc analysis and visualization of data streams based on real-time-processing, to, e.g., display dashboards about current processes or difficulties inside the smart factory, (c) implement and visualize long-term analysis and reports based on in-depth analysis. In case of (c) this would offer benefits for production planning and optimization for the next production sequence; (b) is required to analyze the current state but to also automate the production line and adjust it based on limited or blocked resources or other current production problems.

From a knowledge management point of view, there are some central aspects or research questions to consider. For example, it has to be analyzed which available models and methods may be used or be adapted for this new working environment, named, e.g., “digitized knowledge society” leading to knowledge 4.0 and work 4.0 [25]. The organizational change and the required lifelong learning need to be considered in all knowledge management activities of the smart factory [4]. Knowledge engineers need to be involved to support the transformation process from a traditional factory into a smart factory and to be the bridging element between different involved work groups and disciplines and their respective knowledge of the production process. In consequence, holistic views of the different processes may be modeled, and their inter-dependencies and variations will create a basis for the aspired digital changes and transformation towards a CPPS.

Next to the promising digital twin concept, discussed in the previous subsection, another big area of research (e.g., [26]) is how to integrate knowledge-based methods such as machine learning algorithms, data mining or other methods leading to artificial intelligence and how they might be applied for (a) planning a smart factory or production process, e.g., in form of a simulation of the future realization and (b) for analyzing the data created inside the CPPS of machines and humans, leading to improvement and decision support. The “smartness” or “intelligence” of a smart factory may only be realized if the different production units and involved components constantly deliver data, which needs to be set in context, to be analyzed and to create a knowledge feedback loop. This feedback knowledge will again help to improve the representation of, e.g., the digital twin, the production process, decision making, re-configuration of the CPPS and monitoring of the production process.

The challenge of the different aspects mentioned above is how to start a transformation of a factory into a smart factory if the production process is not allowed to be stopped in between for a longer period, and there is no plan to build up an entirely new production plant from scratch. Furthermore, existing shop floors will soon consist of physical and cyber objects that need to interact with each other to create the aspired CPPS. Artificial intelligence offers ways to simulate processes based on changing or randomly generated parameters, thus offering a viable alternative to pre-design, simulate and test different variants of a future solution. Lee et al. (2018) ventured a broader look into the “Industrial AI eco-system” of the future to meet the needs of self-awareness, self-comparability, self-prediction, self-optimization and resilience. In this context they emphasize that the Industrial AI needs to follow a systematic learning approach to yield a real benefit for Industry 4.0 [27].

For the realization of data analysis “each node represents the source of information collected from the given environment”, which in consequence leads to a decision process where further steps for analyzing, filtering or constraining of data and information throughout the network need to be taken [17]. The things or sensors gather observation data in the real-production world and establish a web of data, which needs to be transformed into individual information and subsequently knowledge for proposing solutions or decisions [16]. Fei et al. (2019) proclaim, “data stream analytics [as] one of the core components of CPS” and evaluate different machine learning (ML) algorithms regarding their applicability for analyzing CPS data streams [18]. Those algorithms originated from computer science and artificial intelligence indicate the merging of cyber and engineering/manufacturing world. In this context, Burggräf, Wagner and Weißer (2020) conducted a literature review in which they approached the future of production environments from a problem-solving perspective to analyze which knowledge-based approaches might be applicable or promising. This indicates another trend of a merging of the cyber and physical world as well as the worlds of computer science and manufacturing. The authors focus their literature review on publications discussing problem solving in physical product development with the help of either machine learning, artificial intelligence, expert systems or case-based reasoning. As a result they create a superposition about the different knowledge-based methods and their interrelation [26].

Finally, the information technology systems (e.g., ERP, MES, PLM,…) supporting smart factory processes require more flexibility [2,4] as well as more data, which needs to be connected between different systems, to be analyzed, documented and visualized for the employees fulfilling different roles inside the organization. The interconnection between the different systems to follow the chain of an individual order or product lifecycle is especially important as the specifics of the product lead to specific configurations of the shop floor. If subsequent orders are differentiating fundamentally, the production line needs to be “smart” enough to allow easy reconfiguration as otherwise the manual tasks and time to execute this reconfiguration will lead to additional production time and costs. An alternative, already as part of lean production (e.g., [11]) established method, is to optimize the order of the production queue in such a way that orders with similar configuration settings are processed after each other, essentially covering an optimization problem to save time for reconfiguration. Overall, in Industry 4.0 there will be changes transforming the former automation pyramid towards a service-oriented network which has to be integrated horizontally and vertically into an organization, for setting up a smart production environment while realizing the necessary communication channels [4].

In general, the established infrastructure and change of processes is targeted towards smart manufacturing (e.g., [24]), where data and knowledge are mainly used for planning of actions as well as self-optimization [28]. This leads to advantages such as “real-time monitoring, simulation and prediction of manufacturing operations”, which are evaluated as “vital to improve the production efficiency and flexibility” [9]. Those decisions may be made by human or machine, in a human–machine-collaboration or due to machine–machine-communication. Job scheduling and dynamic processing factors need to be considered in this context as well [29]. In addition, an improvement of tackling the problem of flow shop scheduling might be approached with the help of an evolutionary recommendation approach consisting of “collaborative filtering”, a “ratings converter and some “recommendation assessment metrics [30].

Overall, data leading to knowledge are the core driving factors inside a smart factory as otherwise an automation, re-scheduling, monitoring, optimization or learning of production processes would not be possible. The given paper therefore, focuses this knowledge-based perspective instead of the manufacturing one, with the purpose of simulating manufacturing processes in a fictional production plant.

2.2. Human–Machine and Machine–Machine Interaction

With the changing technology and digitization of factories the interaction of employees with their machines and, in consequence, the way of executing their work changes. Gorecky et al. (2017) visualize the adapted production strategy and role of human workers in industry 4.0 by showcasing the process steps of data gathering in CPPS/CPS, automated aggregation towards information and knowledge, and the following human interpretation and intervention (if necessary), which indicates the supervising role of the worker in a smart production environment [31]. To execute this role, the human worker is now interconnected with CPS with the help of multi-modal human–machine interfaces [4]. This interaction may be divided into two directions of human–machine-cooperation (depending on the reference also called human–machine-interaction) and human–machine-collaboration. Whereas the first one focuses on the execution of assigned tasks in cooperation or interaction between the employee and a (smart) machine, the second one targets the exchange of information which previously happened mostly between the involved humans [32,33].

While the IT world has established the research area of human-computer-interaction, the interaction of human and smart device (e.g., IoT-device), smart machine or even autonomous robots bring on a new complexity as the device, machine or robot now possesses somehow cognitive abilities and a certain intelligence. According to Seeber et al. (2019) an AI “would always base its decision-making on optimizing its objectives, rather than incorporating social or emotional factors” [34], that’s why a human working with a smart or even AI machine needs to consider and predict its likely behavior. The aspects of cooperation and collaboration require a different machine handling and a more comprehensive overview of the machine environment and the related or even automated processes. The responsibility for supervising, managing and controlling different, perhaps not even local machines, will increase and will require different skills from the workers as well as different digital tools and dashboards [31]. The way an employee interacts with other employees and machines is expected to differ greatly between a “traditional” and a smart factory. In the smart factory the machine or robot may be a collaborative peer to the employee or replace certain standard tasks while the employee takes on the more controlling or supervising role [25]. Security aspects such as (1) a security stop of a robot or machine, in case the human employee is inside a certain working area, (2) manual mode of machine control, (3) speed and distance control or (4) performance and power limitations must be considered, especially for the execution of collaborative tasks [35]. For both, human and machine it needs to be clear who is responsible of executing a certain task in an automated process loop of a smart factory and what happens when either of them takes an unexpected action leading to a previously undefined situation for the machine or dangers for the human. How is the machine able to react and how may it learn for future similar situations? Respectively, how does the human learn and anticipate the possible actions of the machine? Working in a smart environment with expected collaboration and cooperation with a smart machine requires not only additional knowledge but also extended context-aware competences detecting dependencies and relations to the organizational context, processes or interaction [33].

Kaivo-oja et al. (2020) analyzed the interaction between man and machine in industry 4.0 and the benefits of creating digital twin representations of both to support future knowledge management. While beforehand oftentimes only the machine had a digital equivalent, the authors also envision the digital twin of the worker to create a holistic view of the process. They propose a “Personal Digital Twin”, which may contain regular updates of the workers “best experience”. The target process of building synergies is the “physical-to-digital-to-physical” loop, indicating the necessary steps of gathering data from the physical process, its digital processing and improving of the involved digital twins as well as analysis based on, e.g., AI, before the results allow real-world decision making [36].

David, Lobov and Lanz (2018) on the other hand discuss the applicability of the digital twin model in the context of education, learning and competence building to prepare workers for the future of flexible manufacturing systems [37].

For the given work, both the human–machine-cooperation and human–machine-collaboration are both interesting aspects, as the application of agent-based-systems allows different benefits, the central one being the modeling of different machines and human worker roles in form of different agents and their behavior, leading to pre-simulations of the interaction between machine and human in the given factory context. Later on, results and observations from the real-environment may be used for updates, refinement and learning.

2.3. Application of Multi-Agent-Systems for Smart Factory Simulation and Adaptation

Multi-Agent-Systems (MAS) are a method from the field of artificial intelligence (AI) used for simulating real-world-problems with the help of differently acting agents in a specific environment. Agents in general are viewed as a reliable method to implement service-oriented architectures and the necessary exchange of information [4]. The efficiency and ability for decision-making of a MAS depends significantly on data or “data-driven technologies” [29]. A well-known definition of Woolridge [38] declares an agent as “a computer system that is situated in some environment, and that is capable of autonomous action in this environment in order to meet its delegated objectives”. It is not a new method or concept, but its inherent characteristics of agility and adaptability [29], as well as reactiveness, proactiveness and social abilities [38], allow MAS to be applied in the context of smart environments, either for scheduling or simulation purposes [29]. This holds especially true as MAS support individual or autonomous information gathering and decision making [29], as well as flexibility and dynamic reconfiguration [39]. Therefore, Beierle and Kern-Isberner (2014) declare it as one of the paradigms which essentially contributed towards the further development of AI in recent years [40], while Botti et al. (2019) see MAS as “most suitable candidates for the design and development of distributed and intelligent applications in complex and dynamic environments” [41]. Soic et al. (2020) stress the ability of MAS for “modeling … high-level cognitive processes [while] making use of all the available information” [17].

The applicability of MAS in smart factories or more specific CPPS has been discussed for some time in the manufacturing community, e.g., [13,28,39,42]. In the following, a few examples and different approaches of recent years shall be discussed to indicate the development trends and application possibilities of MAS in CPPS:

The main reason why MAS is more frequently used today, is that for simulating purposes computing power is being required, which has not been available at the time the method has been published. In addition, there are more complex and human-centered problems which require simulation activities [41]. In general, a MAS consists of several “agents” which represent real-world-entities. Those agents are able to observe their environment, meaning they are perceptive to outside signals while having their own agenda of acting towards a desired goal. Therefore, the agent acts in a somehow predefined but still independent way. The agent receives values and beliefs for defining his behavior, similar to the behavior of humans with different upbringing, goals and intents. This way, the agent has some beliefs, desires and intentions he wants to fulfill inside the application domain or environment. Actions are triggered based on different input criteria such as time, activation of certain states or execution of processes. Logic or rules enhance the execution of reasoning tasks [38,40,43,44].

To provide a more likeliness towards human interaction, “aspects such as reactivity, proactivity, and sociability” need to be considered [41]. Woolridge (2009) indicated them already as key characteristics of MAS [38]. According to Dorri, Kanhere, and Jurdak (2018) there are two types of agents, one communicative type of agent and one situation-based agent [43]: The communicative type actively interacts with other agents inside the environment, thus he is able to send messages towards them. The agent reacting on a certain situation scans his environment and executes certain actions based on his perception. Overall the agents might work together or against each other. The detail level of the environment may be between individual agents, a group of agents or different groups or types of agents. This way, different machines, CPS configurations or social interaction between humans or human and machine might be simulated. For the detailed use case in the next section the MAS will be used to simulate the behavior of the different production steps and units inside the production plant. At this stage of the implementation the human worker is not yet part of the MAS scenario, but needs to be considered in the future.

Next to the agents, the environment where they act needs to be defined and specified. Beierle and Kern-Isberner (2014) name different criterion for defining a MAS environment such as if the environment is accessible, deterministic, static, dynamic, if it covers discrete or continuous states or if it relates to certain situations or episodes [40]. As the smart factory aims for dynamic reactions and processes, this criterion would be a key factor for the MAS implementation of a CPPS. MAS-Tools, such as Anylogic [10], allow combined modeling, implementing and simulating of the MAS environment as well as the agent behavior.

3. Use Case

In this section the chosen use case scenario will be introduced as well as the necessary steps for designing the MAS simulation will be discussed from a knowledge management and process perspective.

3.1. Scenario Description

The use case scenario for the simulation of the smart factory production process covers the configuration, ordering, production and logistics of individualized picture frames of different sizes and materials. The customer may therefore configure the size, type of material and type of color of each frame individually. The different types of material, such as glass, wood, metal, PVC, require the application of different production processes and tools. The logistic processes of materials or parts as well as finished goods have to be considered as well. The shop floor consists of machines for cutting material, grinding with specific tools based on the material, painting (if desired and possible) as well as assembly. Not all of the steps can or have to be executed for each material, e.g., the material glass does not require the painting job. The painting itself may again happen individually based on the chosen colors and depending on the type of material, as there are different types of color for painting different frame materials. Figure 1 summarizes the smart factory areas consisting of logistics, storage and the central shop floor. The generated data is indicated in form of an internal and outside feedback loop applying data analytics.

For the scenario RFID-chips are used as information carriers for the different materials or parts as they receive the order information and necessary steps of the production workflow and carry them through the whole production process. They may identify themselves at special RFID-reader-gates which are installed at the beginning of the production line or be updated at each production step. For automizing the process, data about the order configuration are provided on the RFID-chip. This step may also be done at the beginning of the production line, before the material may be transferred with the help of a driver-less or automated transport system or any other kind of robot. As the production line itself is aware which machines and assembly stations are occupied, the material is delivered to a fitting open station or must wait for an open slot. Once the material is at the desired location a robot may pick the material from the carrier and start the production or assembly step. For the whole production line this means that the station is blocked for other parts. After one production step is finished, a carrier robot or transport vehicle moves the parts to the next destination. Once the whole frame is produced and assembled it will be transported to a high-bay storage until a charge is ready for delivery. Due to the data generated via the RFID-communication the ERP system and PLM systems are able to track the availability of working material and necessary re-orders as well as finished products.

The core of the smart factory use case is the shop floor, where four different production shops are located (see Figure 2) which may be used in sequence or be skipped if a shop process is not necessary for the chosen material (see Figure 3), e.g., when the material for the frame does not need any grinding or painting.

3.2. Involved Systems, Machines and Tasks

Before it is possible to define the specific agents, an overview about the different roles and tasks inside the simulation environment needs to be generated.

Table 1. Multi-Agent-Systems (MAS) approaches in manufacturing.

Year	Authors	Approach
2017	Büth et al. [45]	Introduction of a combined simulation approach applying agent-based simulation as well as discrete-event simulation towards an industrial context. The discrete-event part offers elements such as time and events, passive entities and triggered state changes, while the agent-based part supports the modeling of the environment, active entities and specific triggers based on a specific situation.
2017	Musli et al. [46]	Analyzed and compared the possibilities of using architecture-based, multi-agent based and self-organizing based approaches for self-adaptation in CPPS environments. Their systematic study indicates that MAS are applicable at least at three layers of the CPS technology stack.
2018	Dorri, Kanhere and Jurdak [43]	Description of different application areas for MAS in form of a survey, one of them being smart grids, indicating that MAS is seen as a viable method for simulating smart environments.
2018	Rocha et al. [47]	Implementation of a use case where a rule based approach is build on top of a MAS, to have rules support “agents to coordinate the execution, correctly verify the quality issues and if necessary trigger some recovery mechanism and strategy.”
2018	Suganuma et al. [48]	The authors targeted the challenge of creating flexibility between edge and cloud computing for IoT environments with the help of a multi-agent platform. This approach should lead to more adaptability with regards to the environment as well as the user orientation, targeting factors such as “processing load and task assignment” between edge and cloud.
2019	Filz, Herrmann and Thiede [49]	Investigation of the application of an agent-based simulation for material supply chains inside a “matrix-structured manufacturing system”. Simulation factors such as required new materials per station, transport of produced goods, usage of automated guided vehicles or locations for supplies needed to be considered leading to a simulation of uncertain and dynamic material supply strategies. The authors stress the agent-based simulation “under consideration of uncertainties” to reach flexibility and dynamics. On top of this the authors envision a module to analyze the simulated data for improving the material planning process.
2019	Cruz Salazar et al. [50]	Proceeded to conduct a systematic review how MAS patterns might be used for “enabling the migration to CPPS” . The authors predicted a rapid applicability of MAS in case fixed design patterns might be usable.
2019	Karnouskos et al. [14]	Identification of four main challenges of necessary “patterns, interfaces, metrics and distributed intelligence” to apply MAS in the context of CPPS. Furthermore, they discussed the necessity of overthinking the classical automation pyramid towards MAS, due to the cyber–physical inter-connectivity of CPS entities and their coverage of different layers of said pyramid.
2019	Gorodetsky et al. [9]	According to the authors MAS “tend to become a natural part of CPS for operational management”. Therefore, they see a direct correlation of developing CP-MAS which are able to execute tasks inside the CPS autonomous and which support the concept of digital twins, thus providing a connectivity of the different agents with real-world and cyber-world objects for simulations or re-configurations inside the smart environment.
2019	Pires et al. [51]	Concerning the digitial twin concept, the authors created a comparison of associated technologies for the required phases of modeling, data acquisition, analytics and processing, which allocates the “agent-based simulation” within the modeling phase. In this context, they name Anylogic [10] as one possible software solution for realization. Furthermore, the authors envision advantages of the application of digital twins for “process control, process monitoring, predictive maintenance, operator training, product development” in the manufacturing sector as well as being beneficial in “real-time monitoring, decision-support based on real data, simulation/optimization of various operating scenarios and reduction of costs by saving resources”.
2020	Pantoja et al. [52]	Focus on the application of MAS in the area of ubiquitous computing and embedded systems instead of a whole smart factory. Their idea sees MAS as a means to create and manage ambient intelligence “in open environments based on IoT”.

summarizes the identified roles and tasks inside the production process. In a real-world scenario this identification could take place via different knowledge management approaches executed by the knowledge engineer. Examples are (1) interviews with the domain experts, in this case, e.g., the production workers or engineers, (2) a silent observation or shadowing of the manufacturing steps and involved workers, machines and processes, (3) analysis of existing process descriptions or production plans, (4) rounds of discussions with different experts involved in the production planning as well as execution, (5) analysis of training plans of new employees inside the production plant, or (6) a combination of the aforementioned approaches. If there are already running log files of the production machines, even a process mining would be possible as this would automatically detect variances or process execution alternatives (7). After the knowledge engineer has an overview of the environment, as well as the actors and process alternatives it is important to structure, those results into roles and tasks to later on model the necessary agents for the MAS. In addition, their beliefs and behavior have to be considered to create a realistic representation of a real-world entity (e.g., worker, machine). The system part of the MAS has to be modeled based on detected requirements of the production environment such as dependencies or limitations.

Limitations when creating a simulation are of course the complexity of a real-world production unit and the involved entities as well as their different behavior. If the goal is to conduct decisions based on the simulation, it needs to be decided how complex and realistic the simulation needs to be to make a well-founded decision. MAS are able to represent complex scenarios but this of course requires the knowledge engineer to analyze the environment in deep and to start a time consuming creation of the possible different agents and their interaction or dependencies with each other. In this case, the knowledge engineer and the involved stakeholders need to come to an agreement how detailed such a MAS simulation should be and where it is possible to make some more general assumptions for the behavior of an agent without losing the credibility of the simulation results or real-world representation. The following

Table 2. Roles and tasks in the production process.

No.	Roles	Role Description	Involved Tasks
1	Production coordinator	Represents employees coordinating and producing frames. Might be differed into specific production employees.	Submit configuration data, gather material for production task, cutting job, grinding job, painting job, assembly job
2	Order configurator	Represents the order system for configuring an individualized frame and deducing the material needed for the specific frame.	Configure product, order product, submit configuration data
3	Customer	Represents the individual customers ordering different kinds of products	Configure product, order product
4	Material	Represents different types of material needed for producing the aspired frames.	Gather material for production task, cutting job, grinding job, painting job, assembly job
5	Driver-less transport system or automated guided vehicle	Represents a vehicle operating inside the smart factory, which transports the necessary material to and from the different storages and machines.	Gather material for production task, cutting job, grinding job, painting job, assembly job, put final product into storage
6	Autonomous storage system	Represents a technical logistic system for storing goods into storage systems autonomously.	Gather material for production task, put final product into storage
7	Cutting Machine	Represents the cutting machine(s) and executed cutting jobs.	Cutting job
8	Grinding Machine	Represents the grinding machine(s) and executed grinding jobs.	Grinding job
9	Painting Machine	Represents the painting machine(s) and executed painting jobs.	Painting job
10	Assembly Machine	Represents the Assembly station and executed assembly jobs.	Assembly job
11	Final product	Represents the final product in form of an individualized picture frame.	Configure product, order product, put final product into storage
12	Truck	Represents the vehicle for transporting new material to the smart factory.	Material delivery
13	Shipping Truck	Represents the vehicle for transporting final products from the smart factory to the customer.	Product delivery

,

Table 3. Agents and their behavior, part 1: Order production process.

No.	Agent	Belief	Desire	Intention
				Agent Initialization	Agent Action	Agent Removal
1	Main	Definition of the current state based on a state transition diagram (see Figure 1)	Simulation of the entire production process	Coordination of the initialization of the different agents	-	-
2	Order configurator	Defined by choice of variables: woodColor, woodNoColor, aluColor, aluNoColor, pvcColor or pcvNoColor	Simulation of the order process of the customer	Random choice of the variables mentioned under section beliefs	-	-
3	Material	Defined by choice of variables: isWood, isAlu, isPVC, isGlass, isAttechment	Simulation of the production material of wood profiles, alumina profiles, PVC profiles, glass surfaces, cartonnage for packaging or other support material	Random choice of the variables mentioned under section beliefs	-	-

and

Table 4. Agents and their behavior, part 2: Machines and vehicles.

No.	Agent	Belief	Desire	Intention
				Agent Initialization	Agent Action	Agent Removal
4	Automated guided vehicle	Definition of a resource pool-Differentiation between status reserved or free	Simulation of material transport inside the factory	Agent reserves a free resource for simulation of the transport	Transport inside the factory	Agent releases resource → end of transport
5	Cutting Machine	Definition of a ResourcePool-Differentiation between status reserved or free	Simulation of cutting process	Agent reserves free resource → start of cutting process	Agent reserves resource for defined time interval → cutting time.	Agent releases resource → end of cutting process.
6	Grinding Machine	Definition of a ResourcePool-Differentiation between status reserved or free	Simulation of grinding process	Agent reserves free resource → start of grinding process	Agent reserves resource for defined time interval → grinding time.	Agent releases resource → end of grinding process.
7	Painting Machine	Definition of a ResourcePool-Differentiation between status reserved or free	Simulation of painting process	Agent reserves free resource → start of painting process	Agent reserves resource for defined time interval → painting time.	Agent releases resource → end of the painting process.
8	Assembly Machine	Definition of a ResourcePool-Differentiation between status reserved or free	Simulation of assembly process	Agent reserves free resource → start of assembly process	Agent reserves resource for defined time interval → assembly time.	Agent releases resource → end of the assembly process.
9	Truck	Agent (loaded/unloaded) arrives continuously in one-hour time intervals.	Simulation of material arrival and unloading of the truck	Start material delivery	Material unloading	Stop material delivery
10	Shipping Truck	Agent (loaded/unloaded) arrives continuously in one-hour time intervals.	Simulation of product delivery	Start product delivery	Truck loading	End of product delivery

specify the roles, tasks, agents as well as their behavior for the given use case scenario. For the scenario the number of different agents has been limited to make the process and implementation easier to understand, while still representing all important aspects of the production process:

Based on the aforementioned roles a set of agents has been defined. The following table summarizes the individual agents, their beliefs, desires and intentions. The basic scenario has been simplified a little bit, in leaving the customer and storage processes of the autonomous storage system out at the beginning.

Another set of agents specifies the involved machines and vehicles:

4. Results

In the results section the implemented use case in Anylogic [10] will be introduced, before the results will be discussed based on a number of KPIs.

4.1. Implementation

The MAS requires process implementations to simulate the interaction between the different agents and their time-related inter-dependencies. That is the reason why the knowledge engineer has to detect and analyze the production processes beforehand, as otherwise the behavior and interaction of the agents would not represent the real-world. For the given scenario the starting point is the order process. In Figure 4 it is assumed to have an order of a wooden frame with a chosen color.

The logical order of producing a wooden frame requires the cutting of the wood, grinding of edges, additional painting and assembly of the frame via different steps. At the end the frame is wrapped up in some cardboard box for transport purposes. For the process implementation there needs to be a ResourcePool which is connected to different nodes representing machine positions. The capacity of each resource is limited, similar to the real-world limitations of available material. Figure 5 indicates an example of the implementation of the CuttingMachine-process.

To implement this process the Anylogic Process Modeling Library has been utilized [10]. The cutting process for the agent CuttingMachine is implemented with the help of

• a seize block for reserving a machine instance.
• a rackpick block for picking the necessary material from a material pool and to transport it to the machine. The specifications about availability of material and mobility are additional simulation parameters.
• a delay block to simulate the production time
• a release block to simulate the end of the production process and to release the reserved machine resource and make it available in the ResourcePool

At the beginning and at the end there are queue blocks for simulating the delay or waiting time for new material or if the machine is currently blocked due to a previous order. The position of the agent in the simulation may always be defined with a pre- and post-position. The GrindingMachine and the PaintingMachine are implemented in a similar way, with slight differences due to the fact that not all materials require grinding and painting.

The AssemblingMachine in Figure 6 is implemented differently as there needs to be different combine and delay blocks for simulating the different assembly steps and necessary time. This way the arriving agents and their actions are coordinated and consolidated into a new agent whose result is a finished picture frame.

Figure 7 summarizes the logical stations of the whole production line including the different production areas of the shop floor.

4.2. Discussion

The evaluation of the MAS simulation detailed above has been executed with the help of time measure blocks integrated into the MAS. The MAS allows different simulation and in consequence evaluation criteria based on discrete events or continuous events over time:

While the results above indicate a small number of KPIs, MAS allow different simulation results of the same scenario. This is mainly due to the varying factors such as configuration of agent behavior, application of random parameters, artificial speed-up of time-intervals to compare different production times, on-purpose removal or delay of agents to indicate a delay of failure of real-world entities, such as production material or production units or re-configuration of paths or processes the agents take inside the simulation. The knowledge engineer and the production engineer might work together to create different realistic simulation scenarios and compare their effectiveness for a real-world implementation or modification of the production-plant or production-process. From a knowledge management perspective this simulation might be combined with other knowledge-based approaches using CPS related data for, e.g., further enriching the simulation model or optimizing workflows. While the given use case has a reduced complexity, it is apparent that in a real-world production unit there are a lot more dependencies which make the effort of creating a simulation model worthwhile.

Table 5. Simulation results.

No.	Result
Number of good parts, bad parts and rejection rate	An essential criterion is to count all correct produced goods as well as those classified as bad parts. To vary the simulation a range of good parts in percent has to be defined. Due to the application of a random number generator the rate of simulated good parts might be higher or lower. In consequence, a rejection rate might be simulated with the help of a relation between the number of bad parts and the sum of overall produced parts. The rejection rate shall be lower than a varying threshold. Both evaluation steps might be combined in a result dashboard visualizing the number of produced parts, the evaluated good parts as well as bad parts and the resulting rejection rate. A continuous evaluation of a target number of good parts, e.g., 100, might be used to detect the necessary time (e.g., lead time) to reach this number.
Lead time, clock rate and cycle time	The lead time and its prediction is an essential question in production as indicated in the recent systematic literature review of Burggräf et al. (2020). The resultes indicate the discussion of numerous methods from the fields of operations research and machine learning, such as artificial neural networks [53]. For the given scenario the lead time of an incoming order at time t1 and an outgoing product at time t2 might be evaluated with the help of a time measure start block and end block respectively based on the involved incoming and outgoing agents. Incorporating different time measure blocks for all six production lines allows it to detect the number of produced parts at any time tn. The simulation uses twelve time measurement blocks of which a starting and ending block are added per production line respectively. These added blocks are visualized in Figure 8. This evaluation might be combined with the previous number of good parts and number of bad parts. For the calculations, time measure functions need to be programmed. These functions analyze the start time and end time, number of agents, and distribution of agents per time. Based on these calculations it is, e.g., possible to visualize and compare the number of agents on the x-axis and the average residence time of each agent in minutes on the y-axis for the production of wood, alu and pvc frames. Both the clock rate of the production steps and cycle time might be visualized in one diagram, see Figure 9. The cycle time for the execution of a task is calculated based on the number of all incoming and outgoing agent objects per clock rate for all six production lines. Reflecting on the production planning an ideal cycle time might be deduced.
Normal and manipulated process	Another evaluation of these criteria is a comparison between a normal and artificially manipulated production process (e.g., due to missing production material). The exemplary results show for the chosen parameters of the normal simulation an average lead time of 0.2-time units and in average 76.48 agents circulating the system per time unit. In comparison, the results for the manipulated simulation indicate an (expected) prolonged average lead time for each agent of 0.3 time units while constantly increasing. Consequently, the number of agents circulating the system decreases to about 61.57 agents per time unit, meaning the productivity decreases. The low processing time at the beginning of this simulation is due to the delays in material supply and unproductive machines. In the manipulated process a first saddle point of the curve is due to a first failure of a grinding machine. From there on more failures appear, in consequence repair times are blocking resources and the lead time curve constantly increases. Instead, for the normal running process the average lead time moves to a constant value per time unit. In comparison of the two simulations over 5 time units, the manipulated factory is still able to produce goods, but the output is about 20 percent less than within the normal running one. While this is only exemplary, the small simulation model allows the integration of other influencing or negative factors and dependencies appearing in the real world, e.g., time for expert to appear onsite, no fitting spare parts for the problem available, other delays in logistic processes, unplanned maintenance, etc..
Idle time	The idle time is especially interesting for production planning as this KPI might be used for detecting unproductive time intervals. The calculation of idle times requires functions for each production step which indicate if the observed production step (e.g., cutting of glass) is idle or not. Based on these an overall idle time might be calculated and visualized as well as detailed results for each production step.
Overall equipment effectiveness (oee)	The oee indicates the availability of production machines, the performance and quality of production. As this is a self defined, combined KPI it is necessary to first calculate the goodParts, the ideal cycle time and the planned production time, leading to the overall equipment effectiveness $$oee=\frac{goodParts·idealCycleTime}{plannedProductionTime}·100$$ Interesting for the oee is, e.g., at which time interval 50 or 100 percent effectiveness could be reached and which simulation factors might need to be improved to decrease the required time to reach these thresholds.

summarizes the simulation results.

5. Conclusions

The given paper discusses the applicability of multi-agent-systems in smart factory environments and showcases a use case scenario of applying MAS for planning, simulating and evaluating a smart production line and the associated logistical processes with the help of the simulation software Anylogic [10]. Smart factories and the digitization of production environments are current trends which will transform the way of producing (individualized and personalized) goods and organizing production processes. While the transformation will surely happen in the years to come, the effort of digitizing an existing production environment might be tremendous. The literature study above indicates that the AI method of MAS is a solution to simulate certain dependencies, processes and possible failures beforehand as well as support decision-making. The quality of the simulation depends on the input as well as level of detail, in this case the accuracy of the simulation model and the simulation parameters. From a knowledge management perspective, it is recommended to involve a knowledge engineer who talks with and extracts knowledge from the different involved working groups of the process, such as engineer, production or logistics staff. In addition, he might use existing documentation (e.g., process models) or observe the target processes beforehand.

Afterwards, this knowledge must be formalized and used for creating the agents representing real entities of the production process, their personal and machine characteristics and behavior (normal and abnormal) as well as dependencies. While there are more steps necessary to create a consistent and realistic simulation as well as refine it after testing, the aforementioned steps already indicate the required dialogue between different disciplines and expenditure of time. On the other hand, this work and time spend beforehand saves the time of realizing a perhaps ineffective digitization measure.

The simulation scenario shown in the given use case resembles a shop floor consisting of different elements such as storage units as well as production machines and assembly stations. The industry 4.0 paradigm of having a semi-automated process for producing individualized goods is met with the possibility of configuring different picture frames using different materials and colors. The evaluation integrated in the realization part compared and simulated different KPIs in form of a normal (expected) and manipulated (delayed) process. While the number of KPIs might surely be increased, the examples already indicate the possibilities for automatically counting, analyzing, comparing and visualizing the results of the simulation for the respective target group. In case of a real use case this step would directly indicate where the production process might be ineffective, where goods are delayed or how the storage capacities might be optimized. While operational research and optimization is certainly no new discipline, the application of MAS for the emerging field of industry 4.0 and the inter-dependencies of the aspired cyber–physical production systems creates a field of work where their applicability suggests direct benefits.

For extending the given work it would of course be possible to integrate more machines, dependencies and processes into the simulation model and to extend the evaluation part of the work. The application inside a real production with a similar production process would of course enhance the model and allow for direct deduction of optimization potentials, decision-making or general digitization planning. Real data from the CPS might be analyzed with the help of data or process mining and be used to enrich the model or to detect additional rules or dependencies. A combination with the concept of digital twins should lead to an even more realistic version of the simulation model as well as feed this model with real data.