**5. Discussion**

The question is—which KPI should be used for an evaluation of the whole manufacturing system and the transport subsystem?

The key factor is the production flow through the machines, as there is the bottleneck, which is related with the utilization of the machines and transport vehicles.

The relationship between average utilization of machines and AGVs and the number of AGVs used for 24 h of simulation is shown in the Figure 5. With the increasing number of AGVs, the utilization of machines is increasing, and at the same time the utilization of vehicles is decreasing. These two performance goals are mutually exclusive. The maximum average machine utilization was about 95% compared to about 53% for 6 AGVs (from the range 46–62%).

**Figure 5.** The relationship between average utilization of machines and AGVs and the number of AGV used for 24 h of simulation.

We propose an analysis of the effectiveness of the transport system by the Overall Transport Effectiveness (OTE) metric that can be determined on the basis of the number of transport operations carried out and the theoretical planned limit of transport operations per vehicle. There are 4 transport operations for each product in the production flow, therefore, the production of Plimit = 652 products requires AGVlimit = 2608 transport operations. One AGV can make about 782 transport operations during 24 h. As the AGVs are working parallelly, then theoretically the 3.3 AGVs should be enough, but if there are more vehicles the blocking can occur more frequently. Therefore, the overall transport effectiveness per vehicle was also calculated and is presented in the Table 5.


**Table 5.** OTE—Overall Transport Effectiveness and OFE—Overall Factory Effectiveness (24 h of simulation, 30 simulation runs in each experiment, without failures and battery charging).

There is a close relationship between the number of achieved products and the number of required transport operations. However, there is a small difference in the values of OFE and OTE, because of work in progress and related transport operations. The value of OTE is depended on the number of AGVs. The maximal value of OTE = 95.737% (±0.25%) was achieved for 6 AGVs. In the case of battery charging, the value of OTE = 95.621% was achieved. The failures have decreased the value by about 0.27% to OTE = 95.353%.

However, for longer simulation time, the system is more stable and a maximal value of OFE = 96.076% and OTE = 96.194% was achieved for the longest simulation time of 1500 h.

It should be noted that there can be different versions of the OTE metric due to the scope of the data taken into account. The main difference in this version is that only planned transport operations are taken into consideration (not all possible working time as in utilization rate).

According to principles of lean manufacturing, an unnecessary movement of people, information or materials wastes time and increases costs. Any unnecessary transport of raw materials in the plant is a waste, and thus should be reduced.

Any non-critical resource such as AGV should be "utilized", such that the bottleneck is never starved for work and all work that is processed by the bottleneck is of high quality. Otherwise, additional activation of these resources just generates excess work-in-process and additional costs. This condition will be met if the OTE is greater than or equal to OEE (OFE).

$$\text{OTE} \geq \text{OEE} \text{ (OFE)}\tag{10}$$

Therefore, the hypothesis that there is dependence between plant effectiveness and transport effectiveness, which can be expressed by the use of the OEE metric, has been proven.

In the case of other logistics systems (e.g., transport of multiple products, different routes with returns), the difference in value between OFE and OTE may be greater. These problems and industrial implementation of the proposed methodology in the context of the digital twin for Industry 4.0, will be taken into account in further research.
