A Comprehensive Analysis of Sensitivity in Simulation Models for Enhanced System Understanding and Optimisation
Abstract
:1. Introduction
2. Materials and Methods
- Factor screening (i.e., sorting and arranging factors according to their significance): Factor screening allows the qualitative impact of input variables (factors) on one output parameter (one output variable) to be investigated. Thus, it does not examine quantitative characteristics. It is mainly used to “sift” factors, meaning to exclude those factors that do not have a significant impact on the parameter under study. For screening, it is possible to use, for example, the well-known Ishikawa diagram.
- Local sensitivity analysis: this analysis allows the influence of selected factors for a certain functional value of an output variable (local optimum) to be investigated. Using local sensitivity analysis, we investigated what effects small changes in factor values have on the output parameters. It is mainly used to test the stability (robustness) of the system for a selected combination of factors.
- Global sensitivity analysis: This analysis studies the influence of factor level variation on output parameters across the entire factor definition. Global sensitivity analysis enables researchers to better understand the significance and importance of the factors used in a simulation model and how they compare with each other.
- Grade one polynomial—includes only major effects around the mean.
- Polynomial of the first degree extended by interactions between pairs of factors (interactions of two factors).
- A polynomial of the second degree, which also includes quadratic effects.
- The critical initial step is to define the goal and scope of the simulation model, which dictates the processes and interactions to be modelled and the decisions the model’s outcomes aim to support. In this article, the goal is to use sensitivity analysis to assess how variability in independent input factors affects dependent output values in simulation models of real systems, specifically focusing on production systems. The scope includes a triple area of sensitivity analysis—factor screening, local sensitivity analysis, and global sensitivity analysis—aimed at optimising system response, such as production lead times and work in process (WIP).
- To identify input and output variables, one must determine which factors (inputs) to examine and which output variables (system responses) are applicable to the model’s objectives. In this article, input variables include arrival times and processing times for various types of semi-finished products and workstations (e.g., Processing_Time1, Arrival1). Output variables represent key performance indicators of the system, such as throughput time and work in process (WIP).
- Selecting an appropriate sensitivity analysis method depends on the model characteristics and available data. Methods vary in their ability to handle different types of models and analysis objectives. In this article, linear and nonlinear regression are used to analyse the relationships between inputs and outputs, complemented by global sensitivity analysis for a comprehensive assessment of factor impacts across their entire definition range.
- This article outlines the use of a comprehensive simulation model to study the impact of varying input factors on significant system outputs. It emphasises the importance of systematically exploring the input space by selecting a robust experimental design, such as factorial designs or Design of Experiments (DoE). This involves defining specific scenarios or sets of input variables to simulate, ensuring a broad and representative sample of the model’s operational range. For each experimental setup, we performed multiple simulation runs to account for the stochastic variability inherent in the system. This article suggests a higher number of replications than the number of investigated factors to ensure statistical significance. This thorough approach helps in accurately capturing the system’s behaviour under various conditions. We collected data from simulation runs and used selected statistical methods, such as linear or nonlinear regression analysis and Analysis of Variance (ANOVA), to subject them to sensitivity analysis. This article highlights the creation of metamodels or surrogate models as an efficient way to approximate the relationship between input variables and system outputs, facilitating the identification of significant factors with no exhaustive simulation.
- After conducting the sensitivity analysis, a thorough analysis and interpretation of the data follows. This step entails quantifying the impact of individual input variables on output values, identifying the most influential factors, and using graphical representations (e.g., tornado charts, scatter plots) for intuitive understanding. The interpretation should focus on the practical implications of the findings for system optimisation and decision making. This phase is crucial for improving transparency in model-based decision making and providing guidance for future research by highlighting areas that need additional data collection or model refinement.
3. Results
- The throughput time is positively correlated with the machining time of products at workplace 1.
- The throughput time is negatively correlated with the mean time between the arrival of semi-finished P1 in the system (Arrival1).
- The throughput time is negatively correlated with the mean time between the arrival of semi-finished P2 in the system (Arrival2).
- Throughput time is positively correlated with machining time at workplace 5.
- Throughput time is positively correlated with machining time at workplace 2.
- Throughput time is negatively correlated with machining time at workplace 4.
- Throughput time is positively correlated with machining time at workplace 3.
- WIP is positively correlated with workplace machining time 1.
- WIP is negatively correlated with the mean time between request 1 arriving in the system (Arrival1).
- WIP is negatively correlated with the mean time between the arrival of request 2 in the system (Arrival2).
- WIP is positively correlated with workplace machining time 5.
- WIP is negatively correlated with workplace machining time 4.
- WIP is positively correlated with workplace machining time 2.
- WIP is negatively correlated with machining time at workplace 3.
4. Discussion and Conclusions
Author Contributions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Buckova, M.; Krajcovic, M.; Edl, M. Computer Simulation and Optimisation of Transport Distances of Order Picking Processes. In Proceedings of the 12th International Scientific Conference of Young Scientists on Sustainable, Modern and Safe Transport, High Tatras, Slovak Republic, 31 May–2 June 2017; Bujnak, J., Guagliano, M., Eds.; Procedia Engineering. Elsevier Science Bv: Amsterdam, The Netherlands, 2017; Volume 192, pp. 69–74. [Google Scholar] [CrossRef]
- Hnilica, R.; Jankovsky, M.; Dado, M.; Messingerova, V. Experimental Evaluation of Combined Effects of Risk Factors in Work Environment. In Proceedings of the 12th International Scientific Conference Engineering for Rural Development, Jelgava, Latvia, 23–24 May 2013; Osadcuks, V., Ed.; Engineering for Rural Development. Latvia Univ Agriculture, Faculty Engineering, Inst Mechanics: Jelgava, Latvia, 2013; pp. 577–583. [Google Scholar]
- Sujova, E.; Cierna, H.; Bambura, R. Simulation Model of Production as Tool for Industry 4.0. Implementation into Practice. In Proceedings of the 18th International Scientific Conference Engineering for Rural Development, Jelgava, Latvia, 22–24 May 2019; Malinovska, L., Osadcuks, V., Eds.; Engineering for Rural Development. Latvia Univ Agriculture, Faculty Engineering, Inst Mechanics: Jelgava, Latvia, 2019; pp. 1192–1197. [Google Scholar] [CrossRef]
- Wozniak, W.; Sasiadek, M.; Stryjski, R.; Mielniczuk, J.; Wojnarowski, T. An Algorithmic Concept for Optimising the Number of Handling Operations in an Intermodal Terminal Node. In Proceedings of the Vision 2020: Innovation Management, Development Sustainability, and Competitive Economic Growth, Seville, Spain, 9–10 November 2016; Soliman, K.S., Ed.; Int Business Information Management Assoc-Ibima: Norristown, PA, USA, 2016; Volume I–VII, pp. 1490–1500. [Google Scholar]
- Krajcovic, M.; Plinta, D. Comprehensive Approach to the Inventory Control System Improvement. Manag. Prod. Eng. Rev. 2012, 3, 34–44. [Google Scholar] [CrossRef]
- Janik, S.; Szabo, P.; Mlkva, M.; Marecek-Kolibisky, M. Effective Data Utilization in the Context of Industry 4.0 Technology Integration. Appl. Sci. 2022, 12, 10517. [Google Scholar] [CrossRef]
- Gola, A.; Nieoczym, A. Application of OEE Coefficient for Manufacturing Lines Reliability Improvement. In Proceedings of the 2017 International Conference on Management Science and Management Innovation (MSMI 2017), Suzhou, China, 23–25 June 2017; Wee, H.M., Yuan, C., Eds.; AEBMR-Advances in Economics Business and Management Research. Atlantis Press: Paris, France, 2017; Volume 31, pp. 189–194. [Google Scholar]
- Buckova, M.; Krajcovic, M.; Plinta, D. Use of Dynamic Simulation in Warehouse Designing. In Intelligent Systems in Production Engineering and Maintenance; Burduk, A., Chlebus, E., Nowakowski, T., Tubis, A., Eds.; Advances in Intelligent Systems and Computing; Springer International Publishing Ag: Cham, Switzerland, 2019; Volume 835, pp. 485–498. [Google Scholar] [CrossRef]
- Stofkova, J.; Poliakova, A.; Stofkova, K.R.; Malega, P.; Krejnus, M.; Binasova, V.; Daneshjo, N. Digital Skills as a Significant Factor of Human Resources Development. Sustainability 2022, 14, 13117. [Google Scholar] [CrossRef]
- Szabo, P.; Mlkva, M.; Markova, P.; Samakova, J.; Janik, S. Change of Competences in the Context of Industry 4.0 Implementation. Appl. Sci. 2023, 13, 8547. [Google Scholar] [CrossRef]
- Meciarova, J.; Dado, M. Computer Application for Decision-Making Support in Manufacturing Technology. In Proceedings of the Annals of DAAAM for 2008 & Proceedings of the 19th International DAAAM Symposium, Trnava, Slovakia, 22–25 October 2008; Katalinc, B., Ed.; Annals of DAAAM and Proceedings. Daaam Int Vienna: Wien, NY, USA, 2008; pp. 839–840. [Google Scholar]
- Nielsen, I.; Dang, Q.-V.; Nielsen, P.; Pawlewski, P. Scheduling of Mobile Robots with Preemptive Tasks. In Proceedings of the Distributed Computing and Artificial Intelligence, 11th International Conference, Salamanca, Spain, 4–6 June 2014; Omatu, S., Bersini, H., Corchado, J.M., Rodriguez, S., Pawlewski, P., Bucciarelli, E., Eds.; Advances in Intelligent Systems and Computing. Springer: Berlin, Germany, 2014; Volume 290, pp. 19–27. [Google Scholar] [CrossRef]
- Hwang, D.; Byun, D.W.; Talat Odman, M. An Automatic Differentiation Technique for Sensitivity Analysis of Numerical Advection Schemes in Air Quality Models. Atmos. Environ. 1997, 31, 879–888. [Google Scholar] [CrossRef]
- Razavi, S.; Gupta, H.V. A New Framework for Comprehensive, Robust, and Efficient Global Sensitivity Analysis: 1. Theory. Water Resour. Res. 2016, 52, 423–439. [Google Scholar] [CrossRef]
- Wozniak, W.; Nawrocki, W.; Stryjski, R.; Jakubowski, J. Identification and Reduction of Product Defects in Mass Production at Toyota Motor Manufacturing, Poland. In Proceedings of the Vision 2020: Sustainable Economic Development, Innovation Management, and Global Growth, Madrid, Spain, 8–9 November 2017; Soliman, K.S., Ed.; Int Business Information Management Assoc-Ibima: Norristown, PA, USA, 2017; Volume I–IX, pp. 4774–4782. [Google Scholar]
- Broum, T.; Gorner, T.; Kleinova, J.; Simon, M. Increasing the Value of Ergonomic Design of Workplace in Compliance with Limit Costs. In Proceedings of the 8th International Conference of DAAAM Baltic Industrial Engineering, Tallinn, Estonia, 19–21 April 2012; Otto, T., Ed.; Tallinn Univ Technology: Taillinn, Estonia, 2012; Volume 1–2, pp. 413–418. [Google Scholar]
- Sasiadek, M.; Basl, J. Lean Production in Practice. In Proceedings of the Innovation Vision 2020: From Regional Development Sustainability to Global Economic Growth, Amsterdam, The Netherlands, 7–8 May 2015; Soliman, K.S., Ed.; Int Business Information Management Assoc-Ibima: Norristown, PA, USA, 2015; Volume I–VI, pp. 699–705. [Google Scholar]
- Mleczko, J.; Dulina, L. Manufacturing Documentation for the High-Variety Products. Manag. Prod. Eng. Rev. 2014, 5, 53–61. [Google Scholar] [CrossRef]
- Micieta, B.; Binasova, V.; Lieskovsky, R.; Krajcovic, M.; Dulina, L. Product Segmentation and Sustainability in Customized Assembly with Respect to the Basic Elements of Industry 4.0. Sustainability 2019, 11, 6057. [Google Scholar] [CrossRef]
- Grzegorz, B.; Izabela, N.; Arkadiusz, G.; Zbigniew, B. Reference Model of Milk-Run Traffic Systems Prototyping. Int. J. Prod. Res. 2021, 59, 4495–4512. [Google Scholar] [CrossRef]
- Malaga, M.; Ulrych, Z. STEM Concept in the Era of Industry 4.0. In Proceedings of the Vision 2025: Education Excellence and Management of Innovations through Sustainable Economic Competitive Advantage, Madrid, Spain, 13–14 November 2019; Soliman, K.S., Ed.; Int Business Information Management Assoc-Ibima: Norristown, PA, USA, 2019; pp. 9669–9678. [Google Scholar]
- Razavi, S.; Jakeman, A.; Saltelli, A.; Prieur, C.; Iooss, B.; Borgonovo, E.; Plischke, E.; Lo Piano, S.; Iwanaga, T.; Becker, W.; et al. The Future of Sensitivity Analysis: An Essential Discipline for Systems Modeling and Policy Support. Environ. Model. Softw. 2021, 137, 104954. [Google Scholar] [CrossRef]
- Gunn, B.; Hettiarachchi, I.T.; Johnstone, M.; Le, V.; Creighton, D.; Preston, L. Improving High Value Manufacturing with Simulation-Based Bayesian Optimisation. In Proceedings of the 2022 IEEE International Systems Conference (SysCon), Montreal, QC, Canada, 25–28 April 2022; pp. 1–8. [Google Scholar] [CrossRef]
- Borgonovo, E.; Pangallo, M.; Rivkin, J.; Rizzo, L.; Siggelkow, N. Sensitivity Analysis of Agent-Based Models: A New Protocol. Comput. Math Organ Theory 2022, 28, 52–94. [Google Scholar] [CrossRef]
- Pokorádi, L. Simulation-Based Sensitivity Analysis of Manufacturing Equipment Availability. Polytech. Univ. Bucharest. Sci. Bull. Ser. D Mech. Eng. 2016, 78, 67–78. [Google Scholar]
- Wang, T.; Wei, Q.; Xiong, W.; Wang, Q.; Fang, J.; Wang, X.; Liu, G.; Jin, C.; Wang, J. Current Status and Prospects of Artificial Intelligence Technology Application in Oil and Gas Field Development. ACS Omega 2024, 9, 3173–3183. [Google Scholar] [CrossRef] [PubMed]
- Liu, Y. Applying Artificial Intelligence for Spiral Blade Pitch Distance Optimisation Installed on the Tubes Containing Nanofluid Embedded in PCM-Filled Solar Panel: A Two-Phase Simulation Model. Eng. Anal. Bound. Elem. 2024, 158, 321–331. [Google Scholar] [CrossRef]
- Mahouti, P.; Belen, A.; Tari, O.; Belen, M.A.; Karahan, S.; Koziel, S. Data-Driven Surrogate-Assisted Optimisation of Metamaterial-Based Filtenna Using Deep Learning. Electronics 2023, 12, 1584. [Google Scholar] [CrossRef]
- Wang, L.; Yao, Y.; Luo, X.; Daniel Adenutsi, C.; Zhao, G.; Lai, F. A Critical Review on Intelligent Optimisation Algorithms and Surrogate Models for Conventional and Unconventional Reservoir Production Optimisation. Fuel 2023, 350, 128826. [Google Scholar] [CrossRef]
- Black, E.; Brandão, M.; Cocarascu, O.; De Keijzer, B.; Du, Y.; Long, D.; Luck, M.; McBurney, P.; Meroño-Peñuela, A.; Miles, S.; et al. Reasoning and Interaction for Social Artificial Intelligence. AIC 2022, 35, 309–325. [Google Scholar] [CrossRef]
- Siebertz, K.; van Bebber, D.; Hochkirchen, T. Statistische Versuchsplanung—Design of Experiments (DoE); Springer: Berlin, Germany, 2010; p. 341. ISBN 978-3-642-05492-1. [Google Scholar]
- Gregor, M.; Hromada, J.; Furtáková, S.; Gregor, M.; Grznár, P. Simulačné Meta-Modelovanie; Slovenské Centrum Productivity: Žilina, Slovakia, 2021; p. 160. ISBN 978-80-89333-23-3. [Google Scholar]
- Stavropoulos, P.; Papacharalampopoulos, A.; Sabatakakis, K.; Mourtzis, D. Metamodeling of Manufacturing Processes and Automation Workflows towards Designing and Operating Digital Twins. Appl. Sci. 2023, 13, 1945. [Google Scholar] [CrossRef]
- Gregor, M.; Palajová, S.; Gregor, M. Simulation Metamodeling of Manufacturing Systems with the Use of Artificial Neural Networks. In Proceedings of the 14th International Conference on Modern Information Technology in the Innovation Processes of the Industrial Enterprises, MITIP 2012, Budapest, Hungary, 24–26 October 2012; Hungarian Academy of Science: Budapest, Hungary, 2012; pp. 178–190, ISBN 978-963-311-373-8. [Google Scholar]
Feature | Linear Regression | Nonlinear Regression | AI-Driven Algorithms |
---|---|---|---|
Complexity | Low | Medium | High |
Flexibility | Low | Medium | High |
Adaptability to nonlinear patterns | Low | High | High |
Computational cost | Low | Medium | Variable |
Data requirements | Low | Medium | High |
Interpretability | High | Medium | Low |
Accuracy | Moderate | High | Very high |
Input | Processing Time (min) | Random Number Stream |
---|---|---|
Source1 | Exponential (10) | 1 |
Source2 | Exponential (12) | 2 |
Workplace | Variable | Processing Time (min) |
---|---|---|
M 1 | Processing_Time1 | Triangular (8, 10, 12) |
M 2 | Processing_Time2 | Triangular (7, 9, 13) |
M 3 | Processing_Time3 | Triangular (7, 10, 14) |
M 4 | Processing_Time4 | Triangular (8, 10.5, 12.4) |
M 5 | Processing_Time5 | Triangular (7.7, 9.6, 14.2) |
Feature | Linear Regression Model | Nonlinear Regression Model |
---|---|---|
Assumptions | Requires a linear relationship between variables. | Does not require a linear relationship, suitable for modelling more complex relationships. |
Complexity | Simpler and easier to interpret. | More complex in implementation and requires more advanced analysis techniques. |
Flexibility | Limited to linear relationships. | Allows for flexible modelling of various relationships. |
Interpretability | Direct interpretation of coefficients. | Interpretation can be more complicated due to the forms of relationships. |
Suitability | Excellent for simple relationships and quick analysis. | Preferred when examining systems with complex interactions. |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Grznár, P.; Gregor, M.; Mozol, Š.; Mozolová, L.; Krump, H.; Mizerák, M.; Trojan, J. A Comprehensive Analysis of Sensitivity in Simulation Models for Enhanced System Understanding and Optimisation. Processes 2024, 12, 716. https://doi.org/10.3390/pr12040716
Grznár P, Gregor M, Mozol Š, Mozolová L, Krump H, Mizerák M, Trojan J. A Comprehensive Analysis of Sensitivity in Simulation Models for Enhanced System Understanding and Optimisation. Processes. 2024; 12(4):716. https://doi.org/10.3390/pr12040716
Chicago/Turabian StyleGrznár, Patrik, Milan Gregor, Štefan Mozol, Lucia Mozolová, Henrich Krump, Marek Mizerák, and Jozef Trojan. 2024. "A Comprehensive Analysis of Sensitivity in Simulation Models for Enhanced System Understanding and Optimisation" Processes 12, no. 4: 716. https://doi.org/10.3390/pr12040716
APA StyleGrznár, P., Gregor, M., Mozol, Š., Mozolová, L., Krump, H., Mizerák, M., & Trojan, J. (2024). A Comprehensive Analysis of Sensitivity in Simulation Models for Enhanced System Understanding and Optimisation. Processes, 12(4), 716. https://doi.org/10.3390/pr12040716