Integration and Evaluation of Intra-Logistics Processes in Flexible Production Systems Based on OEE Metrics, with the Use of Computer Modelling and Simulation of AGVs

Abstract

The article presents the problems connected with the performance evaluation of a flexible production system in the context of designing and integrating production and logistics subsystems. The goal of the performed analysis was to determine the parameters that have the most significant influence on the productivity of the whole system. The possibilities of using automated machine tools, automatic transport vehicles, as well as automated storage systems were pointed out. Moreover, the exemplary models are described, and the framework of simulation research related to the conceptual design of new production systems are indicated. In order to evaluate the system’s productivity, the use of Overall Equipment Efficiency (OEE) metrics was proposed, which is typically used for stationary resources such as machines. This paper aims to prove the hypothesis that the OEE metric can also be used for transport facilities such as Automated Guided Vehicles (AGVs). The developed models include the parameters regarding availability and failure of AGVs as well as production efficiency and quality, which allows the more accurate mapping of manufacturing processes. As the result, the Overall Factory Efficiency (OFE) and Overall Transport Efficiency (OTE) metrics were obtained. The obtained outcomes can be directly related to similar production systems that belong to World Class Manufacturing (WCM) or World Class Logistics (WCL), leading to the in-depth planning of such systems and their further improvement in the context of the Industry 4.0.

1. Introduction

One of the key factors, which determine the level of competitiveness of manufacturing plants, is the ability to achieve flexible production and delivery of goods in accordance with customer requirements. Therefore, logistics and Supply Chain Management play an important role in market competition [1]. Logistic operations are carried out in two areas, internal and external to the organization. Therefore, the term “Intra-logistics” describes the organisation and realisation of internal material flow and logistic technologies as well as the goods transhipment in the industry, using technical components, partial and full systems and services [2].

In connection with the flexible production, there are also increased requirements regarding the flexibility and reliability of internal transport systems, associated with the production of short series of various products and thus requiring more transport operations [3]. In response to these needs, the Automated Guided Vehicles (AGVs) are used a lot more widely, which enables full automation of transport operations and can handle various transport routes using only computer navigation systems [4,5,6].

In order to evaluate the system’s productivity, the use of Overall Equipment Efficiency (OEE) metrics has been proposed that is usually used for stationary resources, like machines. The aim of the paper was to prove the hypothesis that the OEE metric can also be used for transport facilities such as AGVs, and there is a dependence between machine and transport effectiveness.

To investigate the influence of the intra-logistic system over the performance of the manufacturing system we have developed a conceptual model of Flexible Manufacturing System (FMS) with an automated transport system with AGVs. The model of the FMS was built with the use of FlexSim software and the OEE metric was used for system integration and description of the availability, reliability and performance parameters for machines and vehicles. Then the Discrete Event Simulation (DES) method was used for performing the series of experiments and an analysis of results is presented in form of Overall Factory Efficiency (OFE) and Overall Transport Efficiency (OTE). Our previous works [7,8,9] show that the computer simulation of the detailed model of the production line with machines, operators and robots with reliability parameters allows better representation and understanding of a real production process which is important for early design and enables front-end planning.

The AGV parameters play an important role for FMS performance; therefore, literature review about FMS, AGV and logistics issues, are presented in the next section. The subsequent sections of the paper include description of the problem, modelling and simulation experiments, results analysis and discussion and final conclusions.

2. State-of-the-Art

Many researchers in logistics have examined the influence of high-performance logistics practices on organizational performance [1,10,11]. In an attempt to drive performance improvements, managers often struggle with multiple, seemingly conflicting, objectives [1]. Logistics management is faced with a tough choice: either strive for efficiency; or strive for effectiveness. Some recent logistics research has suggested that these two performance objectives are mutually exclusive [12]. Performance measures are essential for effective management of any organization. Performance measurement provides a needed assessment of service and cost aspects of logistics execution in the supply chain. Specifically, there is little guidance regarding where a specific measure should be used and, more pointedly, where the use of the measure would be less appropriate.

Fugate et al. [13], have presented the model of logistics performance with the concept of simultaneous pursuit of efficiency, effectiveness and differentiation. However, most companies’ priorities change over time due to market and competitive dynamics. In light of this business reality, enterprises and managers must be able to identify and select new or different measures consistent with evolving organizational priorities [14].

Muthiah and Huang [15] reviewed and categorized various productivity improvement methods and productivity metrics. For example, Overall Equipment Efficiency (OEE is an established technique in World Class Manufacturing (WCM). It is used as a key performance indicator (KPI) in conjunction with lean manufacturing efforts to provide a quantifiable measurement of success. There are a few examples of the performance evaluation of manufacturing systems with the use of the OEE metric, including [16,17], but without considering the efficiency of the transport subsystem.

Muñoz-Villamizar et al. [18], have used OEE to evaluate the effectiveness of urban freight transportation systems and a framework for Overall Transport Efficiency (OTE) based on OEE factors was proposed by Dalmolen et al. [19]. McCalion [20] ask the question: is OEE relevant to logistics management and Automated Guided Vehicle (AGV) operations?

Hayes [21] suggests that the OEE can be used for eliminating the ripple effect caused by stopped vehicles and along with Six Sigma [22] can be used as a measure for World Class Logistic (WCL).

Comparing WCL with WCM, they have a lot in common. The common area is related to intra-logistic in manufacturing systems. Intra-logistic performance has a great influence over the manufacturing performance, because of inter-operational breaks which have a great impact on materials flow in flexible manufacturing system [23].

The literature review only shows a few publications on the design methodology of AGV systems and most of the them use simple KPIs as metrics [24,25,26,27,28,29]. At the time of preparing this paper, no publication was found concerning the detailed assessment of the impact of the AGVs system on the manufacturing process effectiveness that includes the OEE metric. Therefore, the studies have been undertaken in order to elaborate this problem in terms of transport and production effectiveness and to strengthen the logistics potential of the organization.

2.1. Issues Related to FMS and AGV

The flexible manufacturing system (FMS) is a fully automated production system that interconnects machines and workstations with the logistics equipment, where the entire manufacturing process is coordinated by the digital control systems such as Computer Numerical Control (CNC) or Programmable Logic Controller (PLC). Such flexible, automated manufacturing systems are intended for tasks of large typological diversity, high complexity, ensuring on-time delivery and minimal manufacturing costs, while production is unpredictable, being organized in small batches, with frequent changes [30].

The FMS has been studied over the last couple of decades and the researchers have found a variety of problems, which can be distributed in three major categories: workshop design, transportation network design and scheduling problems [31,32]. Different methods were used to solve them, including mathematical (linear, constraints, stochastic) programming, combinatorial optimization, Petri nets and scenario analysis, but computer simulation, especially Discrete Event Simulation (DES), is the most universal and widely used one [33], e.g., for the design of manufacturing systems [27], efficiency [9] and stability analysis [34] of production systems and the design of warehouse transportation systems with Automated Guided Vehicles (AGVs) [35,36,37].

The AGVs are classified as service robots for professional purposes in manufacturing environments and broad review of AGV is presented in [6,28].

Modern AGV vehicles are characterized by precision of operation, speed of movement and high reliability. They can have various equipment to perform numerous transport tasks, such as transporting pallets and containers, pulling trailers with cargo, lifting with a forklift or manipulating details using an integrated robot arm. Examples of AGV carriages are shown in Figure 1.

In comparison to the other solutions of transport systems, AGV vehicles show many advantages including [5,6]:

• they do not require an operator’s service, which allows reducing the labour costs,
• increased work efficiency—it can work 24 h a day,
• high positioning precision—less material losses during transport,
• high security—the replacement of the operator reduces the number of accidents at work, safety systems reduce the risk of collision,
• flexibility of use—the ability to program the route according to the requirements of the process, easy route change and system expansion.

Typical features of AGV are related to the following parameters, as [5]:

• weight and size,
• load capacity (from a few kgs to several tons),
• driving speed 1–2 m/s,
• drive power,
• navigation method, positioning accuracy,
• time of loading and unloading,
• battery capacity,
• working time, battery charging time.

AGV uses electric drive and efficient batteries, however, it requires periodic recharging. Depending on the battery capacity and load, a typical work cycle includes 8–16 h of work and 4–8 h of battery charging, which takes place completely automatically. Some solutions for recharging the battery during short interruptions in the work of AGV (Opportunity Battery Charging) can be found. There are also solutions based on manual or automatic replacement of a depleted battery with a new fully charged battery (Battery Swap). This action takes about 10 min and allows to take full advantage of AGV’s working time but requires a more advanced service system and additional battery packs.

The design of a transport system based on AGVs requires an advanced navigation system and appropriate delineation of transport routes and reloading points. Based on the technique, various navigation systems are used, such as [37]:

• photo-optical—with a passive lead line,
• inductive—with an active lead line,
• without a lead line—autonomous navigation with different location methods: incremental, infrared, ultrasonic, laser, gyroscopic, satellite (GPS).

Various methods can be used to design AGV systems, including mathematical programming methods, heuristic methods, Petri nets and computer simulations. These methods are used in order to improve the transport network in terms of criteria, for minimizing the length of transport routes, maximizing the production flow, scheduling transport tasks, number and location of transhipment points, parking zones and others [28,39].

Transport routes can be one- or two-way. Due to the reduction of the risk of collision, one-way roads in the form of closed loops, which enable cyclical transport operations, are the most commonly used [22]. In this case, it is easier to develop traffic control algorithms than in the case of two-way traffic, which requires additional passing and parking zones. Therefore, during the design of the AGV system, the most frequently used zones are defined including specific segments of the route (Segmented Flow Configuration) and individual transport loops (Single Loops) in a given segment. The advantages of such a solution are related to [40]:

• All AGV vehicles move in the same direction, which practically excludes collisions,
• system control is simplified due to the lack of alternative routes.

In turn, some drawbacks are connected with [35]:

• Small fault tolerance, in the event of failure of one vehicle, the others usually cannot pass it by,
• if the vehicle passes the given transfer point, it cannot turn back, but it must cross the entire loop once again to reach it again,
• vehicles hold each other, which may lead to blockages of the system (deadlock).

When designing the AGV transport system, the most important problem is determining the number of vehicles needed to achieve the required production volume or the minimum number of vehicles required to obtain the optimal production volume [36,40].

There are a lot of situations in which the AGV system may stall because of a deadlock. A variety of deadlock-detecting algorithms are available in literature [41], but these methods work mainly for manufacturing system where the network layout is simple and uses only a small number of AGVs. The paper [42] discusses the development of an efficient strategy for predicting and avoiding the deadlocks in a large scale AGV systems. The integration of the scheduling of production and transport tasks tends to also be problematic because of computational complexity [43,44]. In initial papers, the transportation times between machines have not been considered. Their authors claimed that because transportation times are very small in comparison with the processing times, they are negligible [45]. On the other hand, in recent decades, the more researchers have been attracted by some issues that the transportation times were considerable and ignoring them can have impacts on the solution of scheduling problems.

2.2. Evaluation of FMS and AGV

The performance of the AGV logistics system can have a great influence over the performance of the whole FMS system; therefore, a performance evaluation should be conducted. The key performance indicators (KPI) of the production system include [16,46]:

• Production throughput,
• time of the production process (Manufacturing Lead Time),
• average waiting time for transport,
• length of queues in storage buffers,
• work in progress (WIP),
• downtime of workstations,
• delayed execution of production orders,
• OEE—Overall Equipment Effectiveness,
• OTE—Overall Throughput Effectiveness.

Work efficiency and the use of the means of production can be expressed by using the OEE metric that depends on three factors: availability, performance and quality [16].

$$\mathrm{OEE}=\left(\mathrm{Availability}\right)\times \left(\mathrm{Performance}\right)\times \left(\mathrm{Quality}\right)$$

(1)

Availability is the ratio of the time spent on the realization of a task to the scheduled time. Availability is reduced by disruptions at work and machine failures.

$$\mathrm{Availability}=\frac{\mathrm{available}\mathrm{work}\mathrm{time}-\mathrm{failure}\mathrm{time}}{\mathrm{scheduled}\mathrm{time}}$$

(2)

Machinery failures may cause severe disturbances in production processes and the loss of availability. Inherent availability can be calculated with Formula (3).

$$\mathrm{Availability}=\frac{\mathrm{MTBF}}{\mathrm{MTBF}+\mathrm{MTTR}}$$

(3)

where:

• MTBF—Mean Time Between Failures,
• MTTR—Mean Time To Repair.

The OEE metric was developed for single component maintenance. In the case of complex systems including serial or parallel subsystems the availability is changed. For the series system to be available, each subsystem should be available. For the parallel system to be available, whichever subsystem should be available.

Performance is the ratio of the time to complete a task under ideal conditions compared to the realization in real conditions; or the ratio of the products obtained in reality, to the number of possible products to obtain under ideal conditions. Performance is reduced (loss of working speed) by the occurrence of any disturbances, e.g., human errors.

$$\mathrm{Performance}=\frac{\mathrm{ideal}\mathrm{cycle}\mathrm{time}}{\mathrm{real}\mathrm{cycle}\mathrm{time}}=\frac{\mathrm{real}\mathrm{number}\mathrm{of}\mathrm{products}}{\mathrm{ideal}\mathrm{number}\mathrm{of}\mathrm{products}}$$

(4)

Quality is expressed by the ratio of the number of good products and the total number of products.

$$\mathrm{Quality}=\frac{\mathrm{number}\mathrm{of}\mathrm{good}\mathrm{quality}\mathrm{products}}{\mathrm{total}\mathrm{number}\mathrm{of}\mathrm{products}}$$

(5)

To compare the influence of the AGV logistic system over the manufacturing system, we will consider different OEE factors. However, according to the lean manufacturing paradigm, the flow of production through bottlenecks is the most important, therefore some equipment should be fully utilized whilst other equipment does not require full utilization. The literature review [16,46] indicates that OEE metrics are lacking at complex manufacturing systems and the factory level. In order to address this gap, an overall throughput effectiveness metric can be used [47]. It measures the factory-level performance and can also be used for performing factory-level diagnostics such as bottleneck detection and identifying hidden capacity. It also accounts for subsystems processing multiple products. Any factory layout can be modelled using a combination of the predefined subsystems (serial, parallel), which allows a determination of the Overall Factory Effectiveness (OFE). Note that the OEE equation can be further simplified as [46,47]:

$$\mathrm{OEE}=\frac{\mathrm{Actual}\mathrm{throughput}\left(\mathrm{units}\right)\mathrm{from}\mathrm{equipment}\mathrm{in}\mathrm{total}\mathrm{time}}{\mathrm{Theoretical}\mathrm{throughput}\left(\mathrm{units}\right)\mathrm{from}\mathrm{equipment}\mathrm{in}\mathrm{total}\mathrm{time}}$$

(6)

By extending this definition to the factory level, we have Overall Factory Effectiveness (OFE):

$$\mathrm{OFE}=\frac{\mathrm{Actual}\mathrm{throughput}\left(\mathrm{units}\right)\mathrm{from}\mathrm{factory}\mathrm{in}\mathrm{total}\mathrm{time}}{\mathrm{Theoretical}\mathrm{throughput}\left(\mathrm{units}\right)\mathrm{from}\mathrm{factory}\mathrm{in}\mathrm{total}\mathrm{time}}$$

(7)

Similarly, the Overall Transport Effectiveness (OTE) can be defined:

$$\mathrm{OTE}=\frac{\mathrm{Actual}\mathrm{throughput}\left(\mathrm{units}\right)\mathrm{from}\mathrm{vehicle}\mathrm{in}\mathrm{total}\mathrm{time}}{\mathrm{Theoretical}\mathrm{throughput}\left(\mathrm{units}\right)\mathrm{from}\mathrm{vehicle}\mathrm{in}\mathrm{total}\mathrm{time}}$$

(8)

3. Description of the Problem—Materials and Methods

Let us consider a production system with eight automated machine tools, such as a CNC machining centre, which performs a two-stage process of machining a family of typical machine parts, like sleeves or discs of different sizes.

The machines are arranged as shown in Figure 2, which allows the series-parallel flow of production.

The system also includes an Automatic Washing Station and Inspection Station as well as a Storage System with an automatic rack stacker. Randomly generated various production orders are delivered to the system, differing in the duration of the operation (from 2.5 to 15.2 min). As a means of product transportation devices, several AGV vehicles are used, which will move along the planned transport routes. We assume that the manufacturing process meets the lean manufacturing, i.e., the flow of a single product and minimal buffers capacity to limit production in progress.

When designing a production system, we strive to achieve maximum production efficiency and, in particular, maximum utilization of machines and devices constituting bottlenecks in the manufacturing process. Other machines and devices will usually be used to a lesser extent, but they are necessary for the production process. By introducing changes to the model, we can analyse the formation of bottlenecks in the production system and their impact on the production volume. This allows, i.e., to determine the required storage capacity and capacity of the transport system. Particularly, the most interested issue is the impact of the number of AGV transport resources on the production efficiency of the entire system. For this purpose, a simulation model was developed in the FlexSim 2018 environment, shown in Figure 3.

Initially, the reference system consisted only of machines, without taking transport into account. It represents the manufacturing system in ideal conditions. At the next step, transport-related constraints were added to the model. The layout takes into account the dimensions of individual model objects and the distance between them. According to the recommendations given in the literature, the model uses unidirectional transport routes forming three main loops. Several control points have been introduced to designate places of loading and unloading as well as parking spaces. For the most used intersections, control zones were used to reduce the risk of collisions and blockages.

Typical parameters of AGV were assumed, including the speed of 2 m/s and a loading/unloading time of 30 s. A FIFO (First In First Out) control strategy was applied. In addition, the warehouse system was expanded by adding components such as the high storage warehouse with an automated storage retrieval system (ASRS).

4. Results of the Simulation Experiments

The developed model was used to conduct a series of simulation experiments. In subsequent experiments, the number of AGV vehicles from 0 to 8 was changed. The simulation time of 24 h was assumed as the time of automatic maintenance of the entire system. A random generation of production orders was assumed according to the exponential distribution with the expected value of 100 s. As a result of a lack of data and the need for simplification, the retooling of the system, charging of AGV batteries and the failure of machines and vehicles was omitted. As part of each experiment, 30 simulation runs were carried out. The results are presented in

Table 1. The results of simulation experiments (Average production completed P_avg, in [pieces] for 24 h of work, 30 simulation runs in each experiment, 95% confidence level).

Number of AGVs NAGV	Minimum Production Pmin [Pieces]	Lower Limit of 95% Confidence Interval [Pieces]	Average production Pavg [Pieces]	Upper Limit of 95% Confidence Interval [Pieces]	Maximum Production Pmax [Pieces]	Standard Deviation Σ
0	548	559.0	561	563	573	6.3
1	182	183.66	184.7	184.87	187	1.62
2	333	335.97	336.7	337.43	341	1.95
3	408	412.1	413.4	414.7	422	3.6
4	419	428.6	430.3	432	438	4.5
5	434	439.8	441.5	443.3	450	4.7
6	435	443.0	444.7	446.4	453	4.6
7	432	443.8	445.6	447.4	456	4.9
8	436	443.6	445.7	447.8	458	5.6

and Figure 4, respectively. Due to the stochastic parameters of the model, the production value P_avg obtained in the experiment is a random variable with a distribution close to normal.

The first row in the Table 1, where the number of AGV amount is 0, represents the reference system consisting only of machines and not taking transport into account (transport time equal to zero).

Figure 4 presents the relationship between average production P_avg and the number of AGVs used for transport in the form of a box-and-whisker plot.

The box represents a 95% confidence interval, which means the average production volume is within this range with a probability of 95%, whilst, whiskers represent the maximum and minimum value of production obtained in a given experiment (there is a small spread of results, therefore some of the boxes in the chart are very small).

In the Figure 4, it can be seen that initially the increase of the AGV number results in a rapid increase of obtained production volume. On the other hand, increasing the AGVs number above 5 units results in a slight increase in production, as more vehicles are used to a lesser extent. A similar phenomenon is described in the literature [36] as the effect of the mutual blocking of AGVs.

To eliminate them, it would be necessary to use multi-lane transport routes or passages. The excessive increase in the number of AGVs is in turn associated with high costs and brings little effects and a relatively small increase in production efficiency [34] with a drop in the effectiveness of AGV being used.

The comparison with the results obtained in the ideal conditions (AGV = 0, P_avg = 561 pc.) shows a great difference with the other results. For example, for 5 AGVs system, there was a P_avg = 441.5 pc., average machine utilization of about 70% and average AGVs utilization of about 50% (from the range of 28–71% for AGV1 and AGV5). The increase of the AGV number caused very little increase in machine utilization and a considerable decrease in the utilization of the additional AGVs.

That problem requires a more detailed investigation, but the traditional metrics for measuring productivity as throughput or utilization rate are not very helpful for identifying the problems and underlying improvements needed to increase productivity. In this situation, a more rigorously defined productivity metric is needed [44]. Therefore, in this case, OEE metrics can be used, which take into account equipment availability, breakdowns, performance (reduced speed, idling) and quality (good and bad quality products).

Second Experiment

A more detailed model of the FMS system was developed taking into account the quality, availability and reliability of AGVs and battery charging.

AGVs have very advanced design and are considered very reliable, but there are certainly few publications about AGV reliability, compared to publications about machine reliability including [48,49]. With the use of fault tree analysis, a reliability block diagram and a hazard decision tree of AGV components, reliability evaluation of the failure rate λ [1/h] was estimated to be 0.003 [48] and 0.0014 [49]. That can also be described with the Mean Time Between Failures (MTBF) as reciprocal of the failure rate λ. Basing on the λ values, we have the range of MTBF = 300 ÷ 700 h and we have assumed an average of MTBF_agv = 500 h for modelling the reliability of AGV. We have also assumed Mean Time To Repair (MTTR_agv) = 8 h. The reliability of CNC machine tools was omitted because its reliability should be much better than that of AGVs, and we will concentrate on the failure effect of the AGVs. The machines are working parallelly; therefore, the effect of machine failures would be very small. On the other hand, another random factor could hinder the analysis of results.

The AGV can work 24 h per day but sometimes battery loading is required. We have assumed a working schedule for 6 AGVs with a 4 h pause for battery loading. It means that AGVs charge the batteries alternately, and in each moment 5 AGVs are working and one is charging the battery. In the case of malfunction, the AGV is automatically moving to the parking place for maintenance or should be manually removed to prevent blockage.

The scenario includes continuous work of the FMS for 3 shifts per day and 5 days per week. As a result of the long-time effect of AGVs failures, long-time simulations were performed, including work for 24, 120, 500 and 1500 h. The experiment’s results without and with reliability parameters of AGVs are presented in

Table 2. The results of simulation experiments (Average production completed Pavg, in [pieces] for 6 AGVs with battery charging, without failures, 30 simulation runs in each experiment, 95% confidence level).

Time [Hours]	Minimum Production Pmin [pc.]	Lower Limit of 95% Confidence Interval [pc.]	Average Production Pavg [pc.]	Upper Limit of 95% Confidence Interval [pc.]	Maximum Production Pmax [pc.]	Standard Deviation σ	Average Throughput [pc./Hour]
24	593	611	614.2	617.5	634	8.7	25.59
120	3094	3131.8	3139.9	3148.1	3188	21.8	26.17
500	13,061	13,116.7	13,128.3	13,139.9	13,188	31.1	26.26
1500	39,288	39,387	39,406	39,425	39,534	51	26.27

and

Table 3. The results of simulation experiments (Average production completed Pavg, in [pieces] for 6 AGVs with battery charging, with AGVs failures, MTBF = 500 h, MTTR = 8 h, 30 simulation runs in each experiment, 95% confidence level).

Time [Hours]	Minimum Production Pmin [pc.]	Lower Limit of 95% Confidence Interval [pc.]	Average Production Pavg [pc.]	Upper Limit of 95% Confidence Interval [pc.]	Maximum Production Pmax [pc.]	Standard Deviation σ	Average Throughput [pc./Hour]
24	549	607.3	613.2	618.3	634	14.8	25.55
120	2810	3091	3116	3141	3174	68	25.97
500	12,327	12,965	13,030	13,094	13,162	173	26.06
1500	38,402	39,044	39,151	39,258	39,463	287	26.10

, respectively. (The raw data are included in the Supplementary Materials).

An analysis of the previous model showed that blockage of the machines sometimes occurs; therefore, small loading/unloading buffers with a capacity of one piece were added to machines in order to improve the production flow. Quality parameters were defined as 99.9% of good products according to the OEE quality factor.

Comparing the results from Table 2 and Table 3, a small but significant effect of AGVs failures on production can be seen (a decrease of about 0.7%). For a more detailed analysis, the OFE metrics can be used. Since the model was built based on the OEE components and contains parameters of availability, performance and quality, the production value from the simulation P_avg can be directly used to calculate the OFE indicator [25,41].

$$\mathrm{OFE}=\frac{{\mathrm{P}}_{\mathrm{avg}}}{{\mathrm{P}}_{\mathrm{limit}}}$$

(9)

The value P_limit represents the theoretically available maximal production in ideal conditions. For the average machining time of t_m = 530 s, the limit is equal to 6.79 pc./hour for one machine and 652 pc./24 h for the whole machining system (P_limit = 27.17 pc./hour).

The juxtaposition of the OFE indexes is included in

Table 4. The Overall Factory Effectiveness (OFE) metrics for model of 6 AGVs without failures OFE₂ and with failures OFE₃ and OFE₁ from the previous experiment.

Time [Hours]	Plimit	Pavg1	OFE1	Pavg2	OFE2	Pavg3	OFE3
24	652	444.7	0.68206	614.2	0.94203	613.2	0.94049
120	3260	2252.3	0.69089	3139.9	0.96316	3116	0.95583
500	13,583.33	9407.6	0.69258	13,128.3	0.96650	13,030	0.95926
1500	40,750	28,230.7	0.69278	39,406	0.96702	39,151	0.96076

.

The differences between OFE₂ and OFE₃ are related to the warmup of the system in a short time and to the effect of AGV failures in a long time. This result is consistent with assumed reliability parameters and inherent availability (Equation (3)) of AGV and parallel system. As there is a small probability of simultaneous failure of all AGVs, the effect is connected with the loss of performance including loss of speed during loading/unloading, waiting for transport and blocking.

5. Discussion

The question is—which KPI should be used for an evaluation of the whole manufacturing system and the transport subsystem?

The key factor is the production flow through the machines, as there is the bottleneck, which is related with the utilization of the machines and transport vehicles.

The relationship between average utilization of machines and AGVs and the number of AGVs used for 24 h of simulation is shown in the Figure 5. With the increasing number of AGVs, the utilization of machines is increasing, and at the same time the utilization of vehicles is decreasing. These two performance goals are mutually exclusive. The maximum average machine utilization was about 95% compared to about 53% for 6 AGVs (from the range 46–62%).

We propose an analysis of the effectiveness of the transport system by the Overall Transport Effectiveness (OTE) metric that can be determined on the basis of the number of transport operations carried out and the theoretical planned limit of transport operations per vehicle. There are 4 transport operations for each product in the production flow, therefore, the production of P_limit = 652 products requires AGV_limit = 2608 transport operations. One AGV can make about 782 transport operations during 24 h. As the AGVs are working parallelly, then theoretically the 3.3 AGVs should be enough, but if there are more vehicles the blocking can occur more frequently. Therefore, the overall transport effectiveness per vehicle was also calculated and is presented in the

Table 5. OTE—Overall Transport Effectiveness and OFE—Overall Factory Effectiveness (24 h of simulation, 30 simulation runs in each experiment, without failures and battery charging).

Nr of AGVs Nagv	Plimit [Pc.]	AGVlimit (4 × Plimit/Nagv) [Pc.]	Pavg [pc.]	Finished Transport Operation [pc.]	Average Transport Oper./AGV [pc.]	OTE	OFE
1	652	2608	186.7	781.9	781.9	0.29981	0.28635
2	652	1304	347.23	1428.2	714.1	0.54762	0.53256
3	652	869.3	516.7	2107.5	702.5	0.80812	0.79249
4	652	652.0	606.0	2461.4	615.4	0.94379	0.92945
5	652	521.6	614.0	2493.0	498.6	0.95590	0.94172
6	652	434.7	615.0	2497.0	416.2	0.95737	0.94325
7	652	372.6	614.0	2494.4	356.3	0.95637	0.94172
8	652	326.0	611.7	2484.5	310.6	0.95265	0.93819

.

There is a close relationship between the number of achieved products and the number of required transport operations. However, there is a small difference in the values of OFE and OTE, because of work in progress and related transport operations. The value of OTE is depended on the number of AGVs. The maximal value of OTE = 95.737% (±0.25%) was achieved for 6 AGVs. In the case of battery charging, the value of OTE = 95.621% was achieved. The failures have decreased the value by about 0.27% to OTE = 95.353%.

However, for longer simulation time, the system is more stable and a maximal value of OFE = 96.076% and OTE = 96.194% was achieved for the longest simulation time of 1500 h.

It should be noted that there can be different versions of the OTE metric due to the scope of the data taken into account. The main difference in this version is that only planned transport operations are taken into consideration (not all possible working time as in utilization rate).

According to principles of lean manufacturing, an unnecessary movement of people, information or materials wastes time and increases costs. Any unnecessary transport of raw materials in the plant is a waste, and thus should be reduced.

Any non-critical resource such as AGV should be “utilized”, such that the bottleneck is never starved for work and all work that is processed by the bottleneck is of high quality. Otherwise, additional activation of these resources just generates excess work-in-process and additional costs. This condition will be met if the OTE is greater than or equal to OEE (OFE).

OTE ≥ OEE (OFE)

(10)

Therefore, the hypothesis that there is dependence between plant effectiveness and transport effectiveness, which can be expressed by the use of the OEE metric, has been proven.

In the case of other logistics systems (e.g., transport of multiple products, different routes with returns), the difference in value between OFE and OTE may be greater. These problems and industrial implementation of the proposed methodology in the context of the digital twin for Industry 4.0, will be taken into account in further research.

6. Conclusions

Due to the complexity of AGV systems, they cause many decision problems, which are difficult to solve. The paper has presented an example of the Flexible Manufacturing System solution with the AGV transport system and discusses some issues related to the design and simulation of such systems. The stage of initial system design optimization is very important, and computer simulation enables the relatively easy elaboration and testing of various variants of manufacturing and logistics systems. On the other hand, excessive simplifications may be applied at the modelling stage, which will make the simulation not reflect the production system properly. It should be noted that detailed modelling is very labour intensive and requires the involvement of experienced specialists. Therefore, choosing what parameters are used in the modelling process and which metric is used to evaluate the model is very important. In order to make the simulation more accurate and to evaluate the system’s productivity, the use of Overall Equipment Effectiveness (OEE) metrics was proposed.

The results obtained from the presented simulations show that the OEE metrics can be used for the modelling and productivity evaluation of manufacturing and logistics systems, with the generalization of Overall Factory Effectiveness (OFE) and Overall Transport Effectiveness (OTE). The use of OEE factors also allows to compare the results obtained from different manufacturing systems. In the real world, most of manufacturing companies have OEE scores closer to 60%, but there are many of them with OEE scores lower than 45%, and a small number of world-class companies that have the OEE value higher than 85% [50]. According to that, the results of simulation can be also used to analyse the costs involved in the implementation of a given project and at the stage of in-depth design of the production system.