Next Article in Journal
Adsorption of Zinc(II) Ion by Spent and Raw Agaricus bisporus in Aqueous Solution
Previous Article in Journal
Sustainable Electricity Production Using Avocado Waste
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

A Comprehensive Analysis of Sensitivity in Simulation Models for Enhanced System Understanding and Optimisation

1
Department of Industrial Engineering, Faculty of Mechanical Engineering, University of Žilina, Univerzitná 8215/1, 010 26 Žilina, Slovakia
2
Department of Plastics, Rubber and Fibers, Faculty of Chemical and Food Technology, Slovak University of Technology in Bratislava, Radlinského 9, 812 37 Bratislava, Slovakia
3
Department of Industrial and Digital Engineering, Faculty of Mechanical Engineering, Technical University of Košice, Park Komenského 9, 040 01 Košice, Slovakia
*
Author to whom correspondence should be addressed.
Processes 2024, 12(4), 716; https://doi.org/10.3390/pr12040716
Submission received: 26 February 2024 / Revised: 22 March 2024 / Accepted: 29 March 2024 / Published: 1 April 2024
(This article belongs to the Topic Modern Technologies and Manufacturing Systems, 2nd Volume)

Abstract

:
This article delves into sensitivity analysis within simulation models of real systems, focusing on the impact of variability in independent input factors (x) on dependent system outputs (y). It discusses linear and nonlinear regression to analyse and represent relationships between input factors and system responses. This study encompasses three sensitivity analysis areas: factor screening, local sensitivity analysis, and global sensitivity analysis, highlighting their roles in understanding the significance of factors in simulation models. The practical application of sensitivity analysis becomes clear through a case study in a manufacturing system. The case study utilises the Simio simulation system to investigate the impact of input factors on production lead time and work in process (WIP). The analysis uses regression to quantify the impact of seven factors, showcasing the most significant ones with tornado charts and emphasising the application of sensitivity analysis to optimise system responses.

1. Introduction

Sensitivity analysis (SA) in simulation models has emerged as an indispensable tool for researchers and practitioners across various disciplines, from engineering to socioeconomics [1,2,3,4]. This analytical method aids in understanding how changes in model inputs affect outputs, revealing the robustness and reliability of models under different conditions [5]. SA plays a crucial role in enhancing the transparency model and guiding decision-making processes by identifying critical factors that influence system behaviour. Given the significance of sensitivity analysis in uncovering the dynamics between input variables and their resultant effects on outputs, it stands as a fundamental tool for enhancing model accuracy and predictive power [6,7]. The significance of SA extends beyond enhancing model transparency; it is instrumental in the decision-making process by identifying key factors that influence system behaviour. As a cornerstone for improving model accuracy and predictive capability, SA is crucial in areas demanding precise model outcomes, such as climate change forecasting, financial risk assessment, and healthcare planning [8,9,10]. Integrating artificial intelligence (AI) and big data has markedly advanced SA capabilities, enabling more comprehensive and dynamic analyses of complex systems [11,12,13,14]. To provide a clearer understanding of the current state of the art and to identify existing research gaps, we have undertaken a detailed categorisation of the literature based on the objectives of their research. This categorisation helps to highlight the multifaceted applications of SA and its developing role in addressing the contemporary challenges in the following paragraphs.
Manufacturing efficiency and quality improvement: A significant body of research, including studies conducted within a Polish automotive manufacturing facility [15] and investigations into ergonomic workplace design [16], underscores SA’s critical role in enhancing operational efficiency and product quality. These studies not only show SA’s utility in optimising manufacturing processes but also its potential in strategic planning and resource allocation. The application of SA in evaluating lean production practices [17] further illustrates its capacity to refine production strategies, presenting a relatively unexplored avenue for enhancing efficiency and sustainability in manufacturing. The authors in [18] addressed manufacturing documentation for high-variety products, showing a gap in the literature regarding SA’s role in managing documentation and information flow in complex manufacturing settings. Following the exploration of SA in various manufacturing contexts, recent studies have further broadened the application of SA in the field.
Integration with Industry 4.0: The literature also highlights SA’s relevance in Industry 4.0, particularly in sustainable manufacturing practices and logistics optimisation [19,20,21]. Research in this area showcases SA’s capacity for assessing the sustainability implications of various production methodologies and its applicability in optimising logistics and supply chain operations. This focus on Industry 4.0 introduces novel application areas for SA, emphasising its importance in the development of smart factories and positing sustainability goals.
Advanced computational techniques: The integration of SA with advanced computational techniques, such as machine learning for environmental modelling [22] and the application of Bayesian optimisation in manufacturing processes [23], represents a significant advancement in the field. The development of a new protocol for conducting SA in agent-based models (ABMs) [24] addresses the complexities of analysing models with both parametric and non-parametric elements. A simulation-based analysis is demonstrated in [25] that is focused on the availability of manufacturing equipment, offering insights into optimising maintenance systems for better operational efficiency.
These advancements highlight the ongoing evolution of SA as a critical tool for research and practice, leveraging new computational techniques and methodologies to provide valuable insights into complex systems. Recent advancements in sensitivity analysis (SA) and simulation models reflect the ongoing evolution of SA as a critical tool for research and practice, leveraging new computational techniques and methodologies to provide valuable insights into complex systems. Our article contributes to this developing field by applying sensitivity analysis to assess the impact of input variability on system outputs, employing both linear and nonlinear regression. To highlight the comparative advantages and limitations inherent in each method (see Table 1), we juxtapose this approach with state-of-the-art AI-driven algorithms.
This comparison reveals that, while linear and nonlinear regression methods offer simplicity and moderate accuracy with lower data requirements, they lack the flexibility and adaptability of AI-driven algorithms. AI algorithms, although requiring more substantial data and computational resources, excel in handling complex, nonlinear patterns, offering significantly higher accuracy. This is clear in the works of [26,27], where AI-driven optimisation leads to substantial improvements in engineering solutions and renewable energy technologies. Similarly, refs. [28,29] illustrate the broad applicability and effectiveness of AI in enhancing design, optimisation processes, and decision-making strategies in various industrial contexts. Integrating AI technologies in sensitivity analysis not only enhances the robustness and reliability of simulation models under different conditions but also supports strategic planning and policymaking, advancing decision-making processes to additional levels of effectiveness [30]. Through a practical case study in manufacturing, our research showcases how SA can pinpoint areas for system enhancement, contributing both to SA’s theoretical framework and its practical utility. The findings encourage further exploration and application of AI-driven methods in sensitivity analysis, promising new insights and advancements in various disciplines. Our article contributes to the field of sensitivity analysis (SA) within simulation models by addressing a notable gap in the current literature: the application and interpretation of SA, specifically within production systems. While researchers widely recognise sensitivity analysis as a critical tool for enhancing model accuracy and facilitating decision making in various domains, they have given limited attention to applying it to the nuanced dynamics of production systems and interpreting simulation results in these contexts. We advance the understanding of SA by focusing on both linear and nonlinear regression to explore the impact of input variability on system outputs. This methodological approach is not novel per se, but our work stands out by applying these techniques to a detailed case study in a manufacturing system. Here, the practical application of SA—using the Simio simulation software—demonstrates how it can effectively identify key areas for improvement in production processes, a domain that has been overlooked in existing studies. Our research distinguishes itself by elucidating how to interpret the outcomes of sensitivity analyses in the realm of manufacturing simulations. This aspect is critical, as understanding the implications of SA results directly impacts the ability to make informed decisions for system enhancement. Through a methodical examination of the factors influencing system outputs, our study sheds light on the significance of each variable, guiding the optimisation of production processes based on empirical evidence. In summary, our article stands out for its focus on the under-explored area of production systems within the domain of sensitivity analysis in simulation models. By providing a clear methodology for applying and interpreting SA in this context, our research offers valuable contributions to the theoretical framework of SA and its practical utility in improving manufacturing system performance.

2. Materials and Methods

Simulation models of real systems usually represent complex, nonlinear, and interdependent processes. To perform an in-depth analysis of the relationship of independent input factor (x) to dependent outputs—system responses (y)—we need to investigate the influence of variability x on the variance of response values y. Sensitivity analysis is a procedure that makes it possible to analyse the relationship between the variance of independent input variables (x) and the variance of dependent output variables (y). Linear or nonlinear regression can analyse and represent dependencies between input factors and output responses (parameters). When using them, it is advisable to select a minimum number of replications higher than the number of investigated factors. Thus, sensitivity analysis helps to determine how an input factor affects the output response, using linear regression methods most often. A graph that allows the dependence of individual factors and their response to be represented is also called a tornado chart.
According to [31], in principle, three different areas of sensitivity analysis can be distinguished:
  • Factor screening (i.e., sorting and arranging factors according to their significance): Factor screening allows the qualitative impact of input variables (factors) on one output parameter (one output variable) to be investigated. Thus, it does not examine quantitative characteristics. It is mainly used to “sift” factors, meaning to exclude those factors that do not have a significant impact on the parameter under study. For screening, it is possible to use, for example, the well-known Ishikawa diagram.
  • Local sensitivity analysis: this analysis allows the influence of selected factors for a certain functional value of an output variable (local optimum) to be investigated. Using local sensitivity analysis, we investigated what effects small changes in factor values have on the output parameters. It is mainly used to test the stability (robustness) of the system for a selected combination of factors.
  • Global sensitivity analysis: This analysis studies the influence of factor level variation on output parameters across the entire factor definition. Global sensitivity analysis enables researchers to better understand the significance and importance of the factors used in a simulation model and how they compare with each other.
Sensitivity analysis plays a crucial role in systematically searching for simulation response responses to extreme values of model input factors or radical changes in the structure of the model used. In detail, the paper discusses sensitivity analysis [31].
Figure 1 shows the principle of global sensitivity analysis.
For example, in a typical sensitivity analysis task, we determine what happens if a customer increases their orders by 100% or reduces their orders to 30%. What impact will the change in management rules have on performing the system, etc.?
For this type of analysis, regression analysis, also known as Analysis of Variance (ANOVA), is used in the planning of experiments. This type of analysis is based on the application of the metamodel, which we created as a model from the responses of the simulation model. Thus, a metamodel is an approximation of a simulation model (more precisely, its input/output transformation relationships) and is sometimes referred to as a response surface. Such a metamodel usually takes the following form:
  • Grade one polynomial—includes only major effects around the mean.
  • Polynomial of the first degree extended by interactions between pairs of factors (interactions of two factors).
  • A polynomial of the second degree, which also includes quadratic effects.
The metamodel uses the simulation model as a black box, and what it represents are its inputs and outputs and the relationship between them (transformation function).
The output of planning experiments is to determine the order of factors and their levels, based on significance. By identifying the most significant factors, we can create a metamodel from them that more easily explains how the simulation transforms a group of input factors into the output response of the modelled system. Such a metamodel is then used to predict the responses of other combinations and levels of investigated factors, which is very convenient (simulation experiments are costly). Combinations of factors and their levels represent variants of the solution of the modelled system. The metamodel can also find a combination of factor levels that optimises the response of the system (minimum, maximum).
Scatter plots are often used as a graphical comparison of the effect of simulation responses, in which values of one factor are plotted on the x-axis (for example, the intensity of input x) and the simulation response is plotted on the y-axis (for example, the average throughput time y). Such a graph shows the I/O transformation of the simulation model (black box), i.e., it represents the statistical dependence between the factor and the output parameter. Figure 2 shows an example of a scatter diagram.
The scattering diagram clearly shows whether a factor has no effect on response (Figure 2a), has a negative effect (Figure 2b) on response, or has a positive effect (Figure 2c) on response. Figure 2b shows a weak negative dependence between a factor and the corresponding response. Figure 2c shows a strong dependence between a factor and the observed response.
Such a diagram allows for further analyses; for example, an adjustment curve can be found for the obtained data and mathematically represents the dependence. In further analysis, it is possible to combine several such diagrams and look for dependencies of several factors (combinations). Based on the above, we can utilise the regression metamodel to approximate the input/output (I/O) transformation of the simulation model. We generate outputs to which we apply regression analysis. The issue of metamodels has been dealt with, for example, in works [32,33,34].
Sensitivity analysis of the response measures how the system’s responses change as a result of variations in the levels of input factors. This analysis uses linear regression, which analyses the relationship between each factor and each response. Here, the requirement is that the minimum number of replications in this case must be higher than the number of input factors.
Among the frequently used graphs to show the sensitivity of responses to input factors, researchers use well-known pie or bar charts. Their disadvantage is that they allow you to display only the percentage response sensitivity for input factors and their levels in the respective experiment. They do not provide information about the strength of dependence and do not support the creation of descriptive models (estimation of model regression coefficient values).
Sensitivity analysis helps to identify how changes in input variables can significantly affect the outputs of a simulation model. This is particularly valuable in industrial engineering, where optimising system performance and making informed decisions based on model predictions is essential. The following algorithm (Figure 3) provides a step-by-step guide to performing sensitivity analysis, enabling us to explore and understand the impact of various factors on model outcomes.
  • The critical initial step is to define the goal and scope of the simulation model, which dictates the processes and interactions to be modelled and the decisions the model’s outcomes aim to support. In this article, the goal is to use sensitivity analysis to assess how variability in independent input factors affects dependent output values in simulation models of real systems, specifically focusing on production systems. The scope includes a triple area of sensitivity analysis—factor screening, local sensitivity analysis, and global sensitivity analysis—aimed at optimising system response, such as production lead times and work in process (WIP).
  • To identify input and output variables, one must determine which factors (inputs) to examine and which output variables (system responses) are applicable to the model’s objectives. In this article, input variables include arrival times and processing times for various types of semi-finished products and workstations (e.g., Processing_Time1, Arrival1). Output variables represent key performance indicators of the system, such as throughput time and work in process (WIP).
  • Selecting an appropriate sensitivity analysis method depends on the model characteristics and available data. Methods vary in their ability to handle different types of models and analysis objectives. In this article, linear and nonlinear regression are used to analyse the relationships between inputs and outputs, complemented by global sensitivity analysis for a comprehensive assessment of factor impacts across their entire definition range.
  • This article outlines the use of a comprehensive simulation model to study the impact of varying input factors on significant system outputs. It emphasises the importance of systematically exploring the input space by selecting a robust experimental design, such as factorial designs or Design of Experiments (DoE). This involves defining specific scenarios or sets of input variables to simulate, ensuring a broad and representative sample of the model’s operational range. For each experimental setup, we performed multiple simulation runs to account for the stochastic variability inherent in the system. This article suggests a higher number of replications than the number of investigated factors to ensure statistical significance. This thorough approach helps in accurately capturing the system’s behaviour under various conditions. We collected data from simulation runs and used selected statistical methods, such as linear or nonlinear regression analysis and Analysis of Variance (ANOVA), to subject them to sensitivity analysis. This article highlights the creation of metamodels or surrogate models as an efficient way to approximate the relationship between input variables and system outputs, facilitating the identification of significant factors with no exhaustive simulation.
  • After conducting the sensitivity analysis, a thorough analysis and interpretation of the data follows. This step entails quantifying the impact of individual input variables on output values, identifying the most influential factors, and using graphical representations (e.g., tornado charts, scatter plots) for intuitive understanding. The interpretation should focus on the practical implications of the findings for system optimisation and decision making. This phase is crucial for improving transparency in model-based decision making and providing guidance for future research by highlighting areas that need additional data collection or model refinement.

3. Results

This section presents a practical example of applying sensitivity analysis (SA) in a simulated manufacturing environment. Using the Simio simulation system, this part of the paper will illustrate how SA can identify key leverage points for system improvement, particularly focusing on production lead time and work in process (WIP). We will quantify the influence of various input factors and showcase their relative importance through tornado charts and other statistical methods. This real-world application underscores the value of SA in making informed decisions for optimising manufacturing systems. Figure 4 depicts the simulation model of the production system.
As seen in Figure 3, the production system uses two types of products (P1 and P2) that enter the input buffers in front of the first workplace. The workplaces of the production system subsequently process semi-finished products based on production processes. After completion of processing, the products leave the production system. Thus, we have seven factors (two inputs and five workplaces). As a response, we chose throughput time ( y 1 ) and work-in-progress WIP ( y 2 ).
The mean time between arrivals of type 1 semi-finished products P1 of the first type (Arrival1) has an exponential distribution with the parameter Exponential (10), and the mean time between arrivals of P2 blanks of the second type (Arrival2) also has exponential distributions with the parameter Exponential (12) (see Table 2).
Each generated random variable has a defined, independent stream of random numbers (seven streams of random numbers) of its own.
Figure 5a (Arrival1) and Figure 5b (Arrival2) display histograms of random variables. A selection of 10 thousand values formed each histogram.
Table 3 provides the parameters for individual workstations, which determine the operating (machining) times. These times are random quantities that follow a triangular distribution.
Figure 6 displays histograms of random quantities of machining times at individual workplaces. Each histogram comprises a selection of 10,000 values.
Given the aim of this study, we will not consider equipment failures in the production system, nor will we consider handling and transport times.
Figure 7 shows a schematic simulation model of the production system created in the Simio simulation system, where seven input factors define the following: Arrival1: mean time between arrivals of semi-finished P1; Arrival2: mean time between arrivals of semi-finished P2; Processing_Time: machine working time for 1 to 5. Within the experiment, we define two responses, throughput time and WIP, and investigate their sensitivity to the input factor.
We chose the simulation time to be 24 h. We defined 100 replications (simulation runs) in the simulation scenario.
Figure 8 shows the results of the analysis of the sensitivity of the examined factors to the throughput time. The tornado chart displays all factors, and it becomes clear from the graph that three factors, namely Processing_Time1, Arrival1, and Arrival2, have a decisive influence. The tornado chart shows that the throughput time ( y 1 ) is most sensitive to the Processing_Time1 factor, followed by the Arrival1 factor and finally the Arrival2 factor.
We can express the dependence of the output response and seven input factors using a simple additive linear regression model.
y 1 = β 1 x 1 + β 2 x 2 + β 3 x 3 + β 4 x 4 + β 5 x 5 + β 6 x 6 + β 7 x 7
As seen in Figure 7, the value of the regression coefficients will be: β 1 = + 47.879 , β 2 = 22.873 ,   β 3 = 12.417 , β 4 = + 0.766 , β 5 = + 0.273 , β 6 = 0.051 , β 7 = + 0.041 .
Here, the regression model would take the following form:
y 1 = 47.879 x 1 22.873 x 2 12.417 x 3 + 0.766 x 4 + 0.273 x 5 0.051 x 6 + 0.041 x 7
As seen from Figure 7, the first three factors have the most significant influence on response, with a weight of up to 98.73% (see the bar graph below).
The practical significance of the values of the regression coefficients is the following:
  • The throughput time is positively correlated with the machining time of products at workplace 1.
  • The throughput time is negatively correlated with the mean time between the arrival of semi-finished P1 in the system (Arrival1).
  • The throughput time is negatively correlated with the mean time between the arrival of semi-finished P2 in the system (Arrival2).
  • Throughput time is positively correlated with machining time at workplace 5.
  • Throughput time is positively correlated with machining time at workplace 2.
  • Throughput time is negatively correlated with machining time at workplace 4.
  • Throughput time is positively correlated with machining time at workplace 3.
It follows from the above that the unit increase in the machining time value ( x 1 ) will cause an increase in the throughput time by 47.879 min. Unit growth of the mean time between arrivals for Arrival1 ( x 2 ) will cause a decrease in throughput time of 22.873 min. Unit growth of the mean between arrivals for Arrival2 will cause a decrease in throughput time of 12.417 min. The same principle mentioned above applies to the other factors. Because of their low importance, we will not dwell on them in more detail below.
From the tornado graph in Figure 8, in the analysed production system, the relationship between machining time at workplace 1 and throughput time is positive. Thus, the higher the value of the processing time, the longer the running time will be. The relationship between the mean time between the arrival of semi-finished products for Arrival1 and the intermediate production time is negative, i.e., the shorter the value of Arrival1, the longer the intermediate period will be.
The tornado chart also provides us with additional information. The size of the tornado chart’s columns determines the significance of the factor. As seen in Figure 7, the most significant is the Processing_Time1 factor, followed by the Arrival1 factor, and the least significant is the Arrival2 factor. Other factors are insignificant.
Analogous conclusions can also be made for the dependence of the factors in question and the state of WIP (the number of products that are simultaneously present in the production system). It is clear from Figure 9 that the greater the machining time, the greater the production WIP (positive, direct dependence). The shorter the times between the arrival of semi-finished products in the production system, the higher the production WIP will be (negative, indirect dependence).
In the sensitivity analysis of the production system from Figure 4, the results of the regression analysis (sensitivity) between the WIP state y 2   and the seven input factors are displayed in more detail in Figure 9 as a tornado graph.
Figure 9 shows a tornado chart for WIP.
We determine the values of the βi coefficients directly by subtracting them from the tornado graph. As seen in Figure 9, the value of the regression coefficients will be: β 1 = 620.386 , β 2 = 462.270 ,   β 3 = 270.179 , β 4 = + 10.852 , β 5 = 5.848 , β 6 = 5.233 , and β 7 = 0.518
Here, the regression model would take the following form:
y 2 = 620.386 x 1 462.270 x 2 270.179 x 3 + 10.852 x 4 5.848 x 5 + 5.233 x 6 0.518 x 7
The practical significance of the values of the regression coefficients is analogous to the above case of the analysis of the intermediate period:
  • WIP is positively correlated with workplace machining time 1.
  • WIP is negatively correlated with the mean time between request 1 arriving in the system (Arrival1).
  • WIP is negatively correlated with the mean time between the arrival of request 2 in the system (Arrival2).
  • WIP is positively correlated with workplace machining time 5.
  • WIP is negatively correlated with workplace machining time 4.
  • WIP is positively correlated with workplace machining time 2.
  • WIP is negatively correlated with machining time at workplace 3.
It follows from the above that the unit increase in the value of processing time ( x 1 ) will result in an increase in WIP by 620.386 pcs. The unit growth of the median period between the arrivals of semi-finished P1 ( x 2 ) will result in a decrease in WIP by 462.270 pcs. The unit growth of the mean between the arrivals of semi-finished P2 ( x 3 ) will result in a decrease in WIP by 270.179 pcs.
From the tornado graph in Figure 9, it is clear that in the analysed production system the relationship between processing time and WIP is positive. Thus, the higher the value of the processing time, the higher the WIP will be. The relationship between the mean time between the arrival of semi-finished products 1 (Arrival1) and the WIP is negative, i.e., the lower the value of Arrival1, the higher the WIP will be. The relationship between the mean time between the arrival of semi-finished P2 (Arrival2) and the WIP is negative, i.e., the lower the value of Arrival2, the higher the WIP will be.
As seen in Figure 9, the most significant is the Processing_Time1 factor, followed by the Arrival1 factor and the Arrival2 factor. Other factors have very little effect on response.

4. Discussion and Conclusions

The quantification of the influence of factors on production efficiency is presented in Figure 10 and Figure 11. These figures present the results of the sensitivity analysis as pie and bar graphs, depicting the percentage influence of each factor on the respective responses. Specifically, for work in progress, Processing_Time1 impacts up to 45.11%, Arrival1 contributes 33.61%, and Arrival2 affects 19.65%. Collectively, these three factors account for a significant 98.37% influence, underscoring the critical importance of time management and material flow in optimising production efficiency. This dominance of specific factors emphasises the necessity for managers to consider not only their direct impacts but also the potential for synergies and conflicts when devising process improvements.
Similarly, these factors’ influence on the length of the running period shows a clear hierarchy: Processing_Time1 at 56.82%, Arrival1 at 27.12%, and Arrival2 at 14.73%, cumulatively accounting for 98.67% impact. This information, confirming the substantial effects of processing and arrival times on production timelines, provides a roadmap for targeted interventions. It suggests the value of implementing predictive scheduling algorithms and fine-tuning Just-In-Time (JIT) delivery systems to mitigate the variability of these crucial factors. Integrating analytical tools like queueing theory could further dissect these influences, offering a mathematical basis for observed effects and supporting the design of more resilient production systems. Figure 10 depicts the sensitivity analysis results for throughput time and WIP in a pie graph format, offering an intuitive breakdown of each factor’s contribution. This visual representation enhances understanding and strategically guides managerial focus towards optimising process timing and material flow.
Figure 11, conversely, details the percentage influence of individual factors on work in progress and throughput time in a bar diagram format. This serves as a direct guide for prioritising process optimisation efforts, highlighting that minor adjustments in key areas can significantly improve overall efficiency.
Figure 12 combines a tornado chart for throughput time with a bar diagram for both throughput time and WIP. This combined graphical representation provides a comprehensive overview of the factors’ impacts, enabling decision-makers to visualise the relative importance of each variable at a glance. Such a synthesis of information enables decision-makers to have a balanced understanding of various factors’ influences, supporting a more nuanced management approach to production systems.
In our analysis, we utilised a simple additive linear regression model, which allows for a certain degree of prediction accuracy between independent and dependent variables. Although this approach provides a solid foundation for understanding fundamental relationships within the data, it is important to highlight that there also exists a nonlinear regression model, which may better capture the more complex relationships between variables (see Table 4).
The fundamental difference between linear and nonlinear regression models lies in the assumption about the relationships between variables. While the linear model operates under the assumption of direct proportionality between independent and dependent variables, the nonlinear model opens up the space for analysing more complex, potentially nonlinear relationships that may exist in real data.
The choice between these two models should depend on the specifics of the data and the goals of the analysis. Although the linear model provided us with the necessary perspective for our specific conclusions, we recognise the potential value of nonlinear modelling in situations where the linear approach could be overly simplistic.
In addressing the inclusion of factor screening and local sensitivity analysis, it is acknowledged that while our study primarily showcased global sensitivity analysis, the roles of the former methodologies were integral yet under-discussed. Factor screening would have efficiently identified Processing_Time1, Arrival1, and Arrival2 as pivotal, optimising our focus and resource allocation for in-depth analysis. Local sensitivity analysis could offer detailed insights into the system’s response to minor deviations from these factors’ baseline values, enabling precise operational adjustments. Future studies will aim to fully integrate this tripartite approach, enhancing the precision of optimisation strategies within manufacturing systems and providing a more nuanced understanding of factor impacts.
This study contributes to the field by elucidating the paramount importance of processing times and material arrival schedules in production efficiency realms. By leveraging sensitivity analysis, we identified specific areas where focused improvements can lead to significant enhancements in productivity and competitive edge, offering insights for manufacturing entities aiming to elevate their operational efficiency. This marks a pivotal step forward in applying sensitivity analysis for production system optimisation, paving the way for more informed and effective management strategies and extending the applicability of these findings to diverse manufacturing contexts to explain sensitivity analysis’s generalizability and potential as a foundational tool in industrial engineering.

Author Contributions

Conceptualization, P.G.; methodology, M.G.; software, Š.M.; formal analysis, L.M.; investigation, Š.M.; resources, M.M.; data curation, J.T.; writing—review and editing, P.G. and M.G.; visualization, H.K.; supervision, L.M.; project administration, P.G.; funding acquisition, P.G. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by Slovak Research and Development Agency, grant number APVV-21-0308, and grant agency VEGA under the contract No. 1/0150/24.

Data Availability Statement

The data presented in this study are available on request from the corresponding author. Due to company privacy, the full text of the data is not publicly available.

Acknowledgments

This work was supported by the Slovak Research and Development Agency under the contract no. APVV-21-0308. This article was created with the support of VEGA project: VEGA 1/0150/24.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Buckova, M.; Krajcovic, M.; Edl, M. Computer Simulation and Optimisation of Transport Distances of Order Picking Processes. In Proceedings of the 12th International Scientific Conference of Young Scientists on Sustainable, Modern and Safe Transport, High Tatras, Slovak Republic, 31 May–2 June 2017; Bujnak, J., Guagliano, M., Eds.; Procedia Engineering. Elsevier Science Bv: Amsterdam, The Netherlands, 2017; Volume 192, pp. 69–74. [Google Scholar] [CrossRef]
  2. Hnilica, R.; Jankovsky, M.; Dado, M.; Messingerova, V. Experimental Evaluation of Combined Effects of Risk Factors in Work Environment. In Proceedings of the 12th International Scientific Conference Engineering for Rural Development, Jelgava, Latvia, 23–24 May 2013; Osadcuks, V., Ed.; Engineering for Rural Development. Latvia Univ Agriculture, Faculty Engineering, Inst Mechanics: Jelgava, Latvia, 2013; pp. 577–583. [Google Scholar]
  3. Sujova, E.; Cierna, H.; Bambura, R. Simulation Model of Production as Tool for Industry 4.0. Implementation into Practice. In Proceedings of the 18th International Scientific Conference Engineering for Rural Development, Jelgava, Latvia, 22–24 May 2019; Malinovska, L., Osadcuks, V., Eds.; Engineering for Rural Development. Latvia Univ Agriculture, Faculty Engineering, Inst Mechanics: Jelgava, Latvia, 2019; pp. 1192–1197. [Google Scholar] [CrossRef]
  4. Wozniak, W.; Sasiadek, M.; Stryjski, R.; Mielniczuk, J.; Wojnarowski, T. An Algorithmic Concept for Optimising the Number of Handling Operations in an Intermodal Terminal Node. In Proceedings of the Vision 2020: Innovation Management, Development Sustainability, and Competitive Economic Growth, Seville, Spain, 9–10 November 2016; Soliman, K.S., Ed.; Int Business Information Management Assoc-Ibima: Norristown, PA, USA, 2016; Volume I–VII, pp. 1490–1500. [Google Scholar]
  5. Krajcovic, M.; Plinta, D. Comprehensive Approach to the Inventory Control System Improvement. Manag. Prod. Eng. Rev. 2012, 3, 34–44. [Google Scholar] [CrossRef]
  6. Janik, S.; Szabo, P.; Mlkva, M.; Marecek-Kolibisky, M. Effective Data Utilization in the Context of Industry 4.0 Technology Integration. Appl. Sci. 2022, 12, 10517. [Google Scholar] [CrossRef]
  7. Gola, A.; Nieoczym, A. Application of OEE Coefficient for Manufacturing Lines Reliability Improvement. In Proceedings of the 2017 International Conference on Management Science and Management Innovation (MSMI 2017), Suzhou, China, 23–25 June 2017; Wee, H.M., Yuan, C., Eds.; AEBMR-Advances in Economics Business and Management Research. Atlantis Press: Paris, France, 2017; Volume 31, pp. 189–194. [Google Scholar]
  8. Buckova, M.; Krajcovic, M.; Plinta, D. Use of Dynamic Simulation in Warehouse Designing. In Intelligent Systems in Production Engineering and Maintenance; Burduk, A., Chlebus, E., Nowakowski, T., Tubis, A., Eds.; Advances in Intelligent Systems and Computing; Springer International Publishing Ag: Cham, Switzerland, 2019; Volume 835, pp. 485–498. [Google Scholar] [CrossRef]
  9. Stofkova, J.; Poliakova, A.; Stofkova, K.R.; Malega, P.; Krejnus, M.; Binasova, V.; Daneshjo, N. Digital Skills as a Significant Factor of Human Resources Development. Sustainability 2022, 14, 13117. [Google Scholar] [CrossRef]
  10. Szabo, P.; Mlkva, M.; Markova, P.; Samakova, J.; Janik, S. Change of Competences in the Context of Industry 4.0 Implementation. Appl. Sci. 2023, 13, 8547. [Google Scholar] [CrossRef]
  11. Meciarova, J.; Dado, M. Computer Application for Decision-Making Support in Manufacturing Technology. In Proceedings of the Annals of DAAAM for 2008 & Proceedings of the 19th International DAAAM Symposium, Trnava, Slovakia, 22–25 October 2008; Katalinc, B., Ed.; Annals of DAAAM and Proceedings. Daaam Int Vienna: Wien, NY, USA, 2008; pp. 839–840. [Google Scholar]
  12. Nielsen, I.; Dang, Q.-V.; Nielsen, P.; Pawlewski, P. Scheduling of Mobile Robots with Preemptive Tasks. In Proceedings of the Distributed Computing and Artificial Intelligence, 11th International Conference, Salamanca, Spain, 4–6 June 2014; Omatu, S., Bersini, H., Corchado, J.M., Rodriguez, S., Pawlewski, P., Bucciarelli, E., Eds.; Advances in Intelligent Systems and Computing. Springer: Berlin, Germany, 2014; Volume 290, pp. 19–27. [Google Scholar] [CrossRef]
  13. Hwang, D.; Byun, D.W.; Talat Odman, M. An Automatic Differentiation Technique for Sensitivity Analysis of Numerical Advection Schemes in Air Quality Models. Atmos. Environ. 1997, 31, 879–888. [Google Scholar] [CrossRef]
  14. Razavi, S.; Gupta, H.V. A New Framework for Comprehensive, Robust, and Efficient Global Sensitivity Analysis: 1. Theory. Water Resour. Res. 2016, 52, 423–439. [Google Scholar] [CrossRef]
  15. Wozniak, W.; Nawrocki, W.; Stryjski, R.; Jakubowski, J. Identification and Reduction of Product Defects in Mass Production at Toyota Motor Manufacturing, Poland. In Proceedings of the Vision 2020: Sustainable Economic Development, Innovation Management, and Global Growth, Madrid, Spain, 8–9 November 2017; Soliman, K.S., Ed.; Int Business Information Management Assoc-Ibima: Norristown, PA, USA, 2017; Volume I–IX, pp. 4774–4782. [Google Scholar]
  16. Broum, T.; Gorner, T.; Kleinova, J.; Simon, M. Increasing the Value of Ergonomic Design of Workplace in Compliance with Limit Costs. In Proceedings of the 8th International Conference of DAAAM Baltic Industrial Engineering, Tallinn, Estonia, 19–21 April 2012; Otto, T., Ed.; Tallinn Univ Technology: Taillinn, Estonia, 2012; Volume 1–2, pp. 413–418. [Google Scholar]
  17. Sasiadek, M.; Basl, J. Lean Production in Practice. In Proceedings of the Innovation Vision 2020: From Regional Development Sustainability to Global Economic Growth, Amsterdam, The Netherlands, 7–8 May 2015; Soliman, K.S., Ed.; Int Business Information Management Assoc-Ibima: Norristown, PA, USA, 2015; Volume I–VI, pp. 699–705. [Google Scholar]
  18. Mleczko, J.; Dulina, L. Manufacturing Documentation for the High-Variety Products. Manag. Prod. Eng. Rev. 2014, 5, 53–61. [Google Scholar] [CrossRef]
  19. Micieta, B.; Binasova, V.; Lieskovsky, R.; Krajcovic, M.; Dulina, L. Product Segmentation and Sustainability in Customized Assembly with Respect to the Basic Elements of Industry 4.0. Sustainability 2019, 11, 6057. [Google Scholar] [CrossRef]
  20. Grzegorz, B.; Izabela, N.; Arkadiusz, G.; Zbigniew, B. Reference Model of Milk-Run Traffic Systems Prototyping. Int. J. Prod. Res. 2021, 59, 4495–4512. [Google Scholar] [CrossRef]
  21. Malaga, M.; Ulrych, Z. STEM Concept in the Era of Industry 4.0. In Proceedings of the Vision 2025: Education Excellence and Management of Innovations through Sustainable Economic Competitive Advantage, Madrid, Spain, 13–14 November 2019; Soliman, K.S., Ed.; Int Business Information Management Assoc-Ibima: Norristown, PA, USA, 2019; pp. 9669–9678. [Google Scholar]
  22. Razavi, S.; Jakeman, A.; Saltelli, A.; Prieur, C.; Iooss, B.; Borgonovo, E.; Plischke, E.; Lo Piano, S.; Iwanaga, T.; Becker, W.; et al. The Future of Sensitivity Analysis: An Essential Discipline for Systems Modeling and Policy Support. Environ. Model. Softw. 2021, 137, 104954. [Google Scholar] [CrossRef]
  23. Gunn, B.; Hettiarachchi, I.T.; Johnstone, M.; Le, V.; Creighton, D.; Preston, L. Improving High Value Manufacturing with Simulation-Based Bayesian Optimisation. In Proceedings of the 2022 IEEE International Systems Conference (SysCon), Montreal, QC, Canada, 25–28 April 2022; pp. 1–8. [Google Scholar] [CrossRef]
  24. Borgonovo, E.; Pangallo, M.; Rivkin, J.; Rizzo, L.; Siggelkow, N. Sensitivity Analysis of Agent-Based Models: A New Protocol. Comput. Math Organ Theory 2022, 28, 52–94. [Google Scholar] [CrossRef]
  25. Pokorádi, L. Simulation-Based Sensitivity Analysis of Manufacturing Equipment Availability. Polytech. Univ. Bucharest. Sci. Bull. Ser. D Mech. Eng. 2016, 78, 67–78. [Google Scholar]
  26. Wang, T.; Wei, Q.; Xiong, W.; Wang, Q.; Fang, J.; Wang, X.; Liu, G.; Jin, C.; Wang, J. Current Status and Prospects of Artificial Intelligence Technology Application in Oil and Gas Field Development. ACS Omega 2024, 9, 3173–3183. [Google Scholar] [CrossRef] [PubMed]
  27. Liu, Y. Applying Artificial Intelligence for Spiral Blade Pitch Distance Optimisation Installed on the Tubes Containing Nanofluid Embedded in PCM-Filled Solar Panel: A Two-Phase Simulation Model. Eng. Anal. Bound. Elem. 2024, 158, 321–331. [Google Scholar] [CrossRef]
  28. Mahouti, P.; Belen, A.; Tari, O.; Belen, M.A.; Karahan, S.; Koziel, S. Data-Driven Surrogate-Assisted Optimisation of Metamaterial-Based Filtenna Using Deep Learning. Electronics 2023, 12, 1584. [Google Scholar] [CrossRef]
  29. Wang, L.; Yao, Y.; Luo, X.; Daniel Adenutsi, C.; Zhao, G.; Lai, F. A Critical Review on Intelligent Optimisation Algorithms and Surrogate Models for Conventional and Unconventional Reservoir Production Optimisation. Fuel 2023, 350, 128826. [Google Scholar] [CrossRef]
  30. Black, E.; Brandão, M.; Cocarascu, O.; De Keijzer, B.; Du, Y.; Long, D.; Luck, M.; McBurney, P.; Meroño-Peñuela, A.; Miles, S.; et al. Reasoning and Interaction for Social Artificial Intelligence. AIC 2022, 35, 309–325. [Google Scholar] [CrossRef]
  31. Siebertz, K.; van Bebber, D.; Hochkirchen, T. Statistische Versuchsplanung—Design of Experiments (DoE); Springer: Berlin, Germany, 2010; p. 341. ISBN 978-3-642-05492-1. [Google Scholar]
  32. Gregor, M.; Hromada, J.; Furtáková, S.; Gregor, M.; Grznár, P. Simulačné Meta-Modelovanie; Slovenské Centrum Productivity: Žilina, Slovakia, 2021; p. 160. ISBN 978-80-89333-23-3. [Google Scholar]
  33. Stavropoulos, P.; Papacharalampopoulos, A.; Sabatakakis, K.; Mourtzis, D. Metamodeling of Manufacturing Processes and Automation Workflows towards Designing and Operating Digital Twins. Appl. Sci. 2023, 13, 1945. [Google Scholar] [CrossRef]
  34. Gregor, M.; Palajová, S.; Gregor, M. Simulation Metamodeling of Manufacturing Systems with the Use of Artificial Neural Networks. In Proceedings of the 14th International Conference on Modern Information Technology in the Innovation Processes of the Industrial Enterprises, MITIP 2012, Budapest, Hungary, 24–26 October 2012; Hungarian Academy of Science: Budapest, Hungary, 2012; pp. 178–190, ISBN 978-963-311-373-8. [Google Scholar]
Figure 1. The principle of global sensitivity analysis.
Figure 1. The principle of global sensitivity analysis.
Processes 12 00716 g001
Figure 2. Example of a scatter diagram: (a) there is a scatter of points with a horizontal trend line indicating no correlation; (b) the points are negatively correlated as the trend line slopes downward; (c) there is a positive correlation shown by the upward sloping trend line.
Figure 2. Example of a scatter diagram: (a) there is a scatter of points with a horizontal trend line indicating no correlation; (b) the points are negatively correlated as the trend line slopes downward; (c) there is a positive correlation shown by the upward sloping trend line.
Processes 12 00716 g002
Figure 3. Stepwise algorithm for sensitivity analysis.
Figure 3. Stepwise algorithm for sensitivity analysis.
Processes 12 00716 g003
Figure 4. Simulation model of the production system.
Figure 4. Simulation model of the production system.
Processes 12 00716 g004
Figure 5. Histogram of random quantity for Arrival1: (a) description of what is contained in the first panel; (b) description of what is contained in the second panel.
Figure 5. Histogram of random quantity for Arrival1: (a) description of what is contained in the first panel; (b) description of what is contained in the second panel.
Processes 12 00716 g005
Figure 6. Histograms of random quantities for Processing_Time.
Figure 6. Histograms of random quantities for Processing_Time.
Processes 12 00716 g006
Figure 7. Schematic representation of the simulation model of the production system.
Figure 7. Schematic representation of the simulation model of the production system.
Processes 12 00716 g007
Figure 8. Tornado sensitivity analysis graph for throughput time.
Figure 8. Tornado sensitivity analysis graph for throughput time.
Processes 12 00716 g008
Figure 9. Tornado chart for production in progress.
Figure 9. Tornado chart for production in progress.
Processes 12 00716 g009
Figure 10. Pie graph sensitivity analysis for throughput time and WIP.
Figure 10. Pie graph sensitivity analysis for throughput time and WIP.
Processes 12 00716 g010
Figure 11. Sensitivity analysis bar diagram for throughput time and WIP.
Figure 11. Sensitivity analysis bar diagram for throughput time and WIP.
Processes 12 00716 g011
Figure 12. Combined chart form.
Figure 12. Combined chart form.
Processes 12 00716 g012
Table 1. Comparison of linear and nonlinear regression with AI-driven algorithms.
Table 1. Comparison of linear and nonlinear regression with AI-driven algorithms.
FeatureLinear RegressionNonlinear RegressionAI-Driven Algorithms
ComplexityLowMediumHigh
FlexibilityLowMediumHigh
Adaptability to nonlinear patternsLowHighHigh
Computational costLowMediumVariable
Data requirementsLowMediumHigh
InterpretabilityHighMediumLow
AccuracyModerateHighVery high
Table 2. Mean time between arrivals of semi-finished products in the system.
Table 2. Mean time between arrivals of semi-finished products in the system.
InputProcessing Time (min)Random Number Stream
Source1Exponential (10)1
Source2Exponential (12)2
Table 3. Parameters of workplace service times.
Table 3. Parameters of workplace service times.
WorkplaceVariableProcessing Time (min)
M 1Processing_Time1Triangular (8, 10, 12)
M 2Processing_Time2Triangular (7, 9, 13)
M 3Processing_Time3Triangular (7, 10, 14)
M 4Processing_Time4Triangular (8, 10.5, 12.4)
M 5Processing_Time5Triangular (7.7, 9.6, 14.2)
Table 4. Comparison of linear regression model and nonlinear regression model.
Table 4. Comparison of linear regression model and nonlinear regression model.
FeatureLinear Regression ModelNonlinear Regression Model
AssumptionsRequires a linear relationship between variables.Does not require a linear relationship, suitable for modelling more complex relationships.
ComplexitySimpler and easier to interpret.More complex in implementation and requires more advanced analysis techniques.
FlexibilityLimited to linear relationships.Allows for flexible modelling of various relationships.
InterpretabilityDirect interpretation of coefficients.Interpretation can be more complicated due to the forms of relationships.
SuitabilityExcellent for simple relationships and quick analysis.Preferred when examining systems with complex interactions.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Grznár, P.; Gregor, M.; Mozol, Š.; Mozolová, L.; Krump, H.; Mizerák, M.; Trojan, J. A Comprehensive Analysis of Sensitivity in Simulation Models for Enhanced System Understanding and Optimisation. Processes 2024, 12, 716. https://doi.org/10.3390/pr12040716

AMA Style

Grznár P, Gregor M, Mozol Š, Mozolová L, Krump H, Mizerák M, Trojan J. A Comprehensive Analysis of Sensitivity in Simulation Models for Enhanced System Understanding and Optimisation. Processes. 2024; 12(4):716. https://doi.org/10.3390/pr12040716

Chicago/Turabian Style

Grznár, Patrik, Milan Gregor, Štefan Mozol, Lucia Mozolová, Henrich Krump, Marek Mizerák, and Jozef Trojan. 2024. "A Comprehensive Analysis of Sensitivity in Simulation Models for Enhanced System Understanding and Optimisation" Processes 12, no. 4: 716. https://doi.org/10.3390/pr12040716

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop