Safety Risks Analysis: Moderating Effect of Risk Level on Mitigation Measures Using PLS-SEM Technique
Round 1
Reviewer 1 Report
Although the author has made many efforts to complete this paper, it is not suitable for publication. There is severe logical flaw in this paper. The paper defines risk as the outcome of a mixture of likelihood and severity, indicating that the authors consider likelihood and severity to be the essential properties of risk. However, the theoretical model constructed in the paper views likelihood and severity as moderating, mediating, and exogenous variables, denying the basic assumption that likelihood and severity are essential properties of risk. The authors need to reorganize the logical relationships of risk, likelihood, severity, and mitigation measures, construct appropriate theoretical models, and collect and analyze data. In addition, the abstract, introduction, literature review, and research methodology of this paper also need to be improved.
Author Response
Please see the attachment.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
This article "Safety Risks Analyses: Effects of Likelihood and Severity Using PLS-SEM Technique" is written on a current topic and corresponds to the profile of the special issue. It may be published after a number of changes and improvements.
The authors have done a lot of work to study this problem. The PLS-SEM technique can serve as a fruitful and accessible tool for SE researchers to gain the necessary knowledge to develop sound and useful solutions.
The title of the work should be corrected and improved.
The purpose and objectives of the article are defined as follows: (a) determining the likelihood and severity of common fatal construction accidents; (b) identify factors that cause safety risks; (c) investigate practical measures taken by contractors to mitigate or manage these safety risks; and (d) study the impact of likelihood and severity on safety mitigation measures implemented using the PLS-SEM method.
The goal and objectives should be redefined based on the work completed. Let's say, in our opinion, the objective (a) is poorly formulated and its implementation remains doubtful; the objective (b) is implemented in an unclear way, in terms of the outcome of the study; the objective (c) is only declared, but not implemented.
In the Abstract, the authors note: “When likelihood and severity are acting as mediators, moderators, or as exogenous latent constructs to MM, the path coefficient between safety risks and mitigation measures and the total effect on MM are reduced, and the effect size is reduced from large to medium effect size.” It goes on to talk about practical measures taken at construction sites to mitigate safety risks, but it is not clear what the practicality of these measures is. The authors do not say a word about this. Since the authors only explore the practical measures of third-party contractors, it can be concluded that the article is purely descriptive in nature and the contribution of the article to the topic under study can be assessed as low. The abstract does not give an idea of what, in fact, is the novelty of the article. Is it in the processing of personal data using software? It must be said that the article (from beginning to end) suffers from excessive descriptions of the model and a lack of scientific interpretation of the results. The authors do not fully understand the results of the analysis and experience great difficulties in interpreting them, which greatly diminishes the scientific significance of the article. This shortcoming is especially characteristic of Conclusions.
The Introduction is inadequate. Since the essence of the work is data processing by computer software, the Introduction should focus on the latest advances in data processing using the PLS-SEM method, in particular, the SmartPLS 4 program. Material about the research hypothesis should be moved to the “Methodology” section as appropriate subsections on research materials. Alternatively, this material could be categorized and placed immediately after the Literature Review.
The PLS-SEM method is not widely known. Compared to the Literature Review, the Methodology section should be expanded and detailed more carefully.
In the article, descriptive analysis was used. This method led to the conclusion that a fall accident has become the most likely and most serious safety risk and that "likelihood and severity are inherent characteristics of safety risks." The scoring scale helps to rank 18 indicators (then 6 were excluded) that measure safety risk. The authors note that their "study adds new insight on likelihood and severity being twin characteristics of safety risks." However, this is somewhat overestimated, since the article proposes to implement a systematic risk management process, but does not say what measures can be used to implement it (for example, how the authors want to reduce the “severity” indicator). Quantification of risk parameters is not exhaustive. The measures given by the authors in the Literature Review are standard measures in any production in accordance with labor safety standards adopted as a guide not only in developed industrialized countries, but also in many developing countries (for example, ISO 45001:2018 “Occupational health and safety management systems - Requirements with guidance for use.”) Of course, using information about measures to assess safety risk is quite legitimate, but this needs a clear explanation of what the authors wanted to do and what actually happened. After reading the article, readers are left wondering about the novelty of the article, do the authors really believe that the more tables there are, the more accurately they will calculate the safety risk? It looks like it really is. This is one of the fundamental shortcomings of this work.
The design of the study involves the use of the SmartPLS 4 software.
PLS-SEM is one way to analyze latent variables and the relationships between them. It focuses on predicting statistical models whose hypotheses suggest causal explanations. This method is useful when the purpose of the study is to predict and identify key (“driver”) parameters; when researchers have complex models that include many different constructs, metrics, and relationships. Therefore, the choice of this method is quite justified for the purposes of the analysis.
Of particular interest is the ability of PLS-SEM to identify latent variables that cannot be measured directly. To measure a "hidden" construct, the researcher defines a set of observable variables that represent that construct. A set of variables (i.e. measurement tool, external model) is applied to rate respondents' statements on a Likert scale. The scheme of the method is presented below:
fig.docx
The figure shows an example of an abstract PLS-SEM model. Constructs A, X and Y are measured reflectively; B is measured formatively. Let's pay attention to the presence of the B element (formatively measured) in the model. However, in the author's final model (Figure 4.5), as we can see, the corresponding formative element is missing, which most likely distorts the interpretation of the latent construct, since one part of the schema understanding is missing.
It is impossible for the reader of the article to find a clear definition of when the measurement model should be reflective or formative. Of course, the authors adhere to the model as a result of conceptualization. The question arises, how to correctly determine the model? Does the model include reflective or formative measures? It is clear that the process of definition depends on constructs that, in their essence, as the authors think, are not reflexive or formative. In other words, for authors, modeling depends on how the nature of the construct is defined. But how right is this? The article does not discuss this. Even if the terms "A" and "B" are understood as various algorithms used to generate "hidden" indicators of variables, then in this case, doubts about the correctness of the model still remain. And that's why.
The combined model calculated by the authors using PLS-SEM is an approximation of the constructs. This may lead to Type I errors (false positives) as the effect may be significant although it may not be observed in the population. Also for this reason, a 95% confidence interval and a power level of 80% are usually recommended, which the authors do not always respect (300 copies of questionnaires prepared but a total of 83 completed questionnaires were received).
In another case, it is considered that the elements must have at least 50% (or 0.5) of the total dispersion. Therefore, AVE values must be greater than 0.50, as the construct is expected to represent more than 50% of the dispersion of its elements. Then the indicator reliability with outer loadings should be at least 0.70 (from 0.641 to 0.915 for the authors).
If some special conditions are not met, then in the case of small sample sizes, Cronbach's alpha will underestimate the reliability of the internal consistency of the model. The systematic error of the authors' general method is a well-known phenomenon in applied statistics: it occurs when the response variation is determined by the measurement instrument.
In addition, HTMT thresholds are very conservative and have a high probability of false rejection of discriminant validity (i.e., Type II error). Discriminant validity using the Heterotrait-Monotrait/Fornell and Larcker criteria, requiring all correlations between constructs to be less than the smallest of the AVE square root values, is outdated and out of date. Accordingly, Tables 11 and 12 are scientifically obsolete arguments. They can be removed.
The strength of the path between the two latent variables is an indicator of convergent validity. A value above 0.70 is recommended.
Standard p thresholds are 0.05 and 0.01. The authors need to explain the non-standard of their meanings.
Condition Index (CI). The authors correctly test for critical levels of collinearity in formative measurement models and tolerance using this index. In cases with high collinearity, researchers should carefully consider the element's theoretical contribution to the design before rejecting it.
Reflective and formative model evaluations. To better understand the results of the study, authors should check internal consistency, reliability, convergent and discriminant validity, as well as content and convergent validity, collinearity, significance, and relevance of formative elements.
As is known, the PLS method was developed to correct estimates of reflectively measured constructs via a reliability factor (i.e., it evaluates scores of indicators, intra-construct correlations, and path coefficients). But there is such a problem as the limitation of the accuracy of the prediction caused by the sampling restrictions, mainly dependent on the choice of who will be the participants in the survey. Sampling deficiencies can lead to biased results that undermine generalizability requirements. The authors make ex post adjustments to the sample weights in model evaluation and weight demographic characteristics. Thus, it turns out that if the distribution of the educational level of the target group is known ex ante, and the sampling procedure cannot determine the exact representative group, then this task is adjusted using a computer program. However, in our opinion, a statistical study that has not received an official ethical license (certificate) confirming reliable and reliable procedures for collecting and processing information (what and how was done, what was eventually established) cannot guarantee the credibility of the adaptation and validation of the questionnaire in order to assess the risk of bias in the conducted survey.
Consideration of the methodological aspect of the study should be preceded by an assessment of the compliance of the design of the study with the goal. Assessing the methodological quality of a statistical survey involves determining the extent to which the methods of its conducting prevent the occurrence of systematic and random errors, leading to a decrease in the reliability and bias of its results. The latter, in turn, can lead to wrong decisions.
Although PLS-SEM does not make any assumptions about the normality of the distribution of the data, an analysis of the distribution of the data should nevertheless be carried out because excessive non-normality can damage the estimation of the significance of the parameters. A distribution with a mean greater than 1 or less than -1 can be considered highly skewed. Such occurrences may indicate inaccurate data and may skew the model. It is not clear what confirms the author's model. Authors should discuss when the skewness of a measured construct is due to the nature of the construct or due to an erroneous measurement process.
The final conceptual model of the authors needs to be tested and further refined using current techniques and modern interpretation.
The Discussion material should be moved to the Results, and the Discussion section itself should be rewritten from the point of view of interpreting the research results. What is not consistent with the modeling of the authors, in what direction the PLS-SEM method can be improved in relation to the research topic, what difficulties exist in understanding the research results, etc. In the article, it is not clear whom the authors are arguing with, because there is no actual dispute in the content of the Discussion.
The Conclusions section should be shortened to 1 page. In it, the authors should correlate the results of the study with the purpose and objectives of the study. In its present form, the Conclusions provide a simple description of the model and results. It is not clear what the contribution of the authors to the topic under consideration is.
The article defines risk as follows: risk increases when the likelihood of an incident increases or the severity of an injury increases. The higher the probability of occurrence of an accident and the more serious the accident, the higher the level of risk. Therefore, these dual risk characteristics, working hand in hand, will determine the level of risk of an accident, which is determined by the relationship "level of risk = probability x severity".
This definition is based on probability. The concept of risk includes events (triggering events, scenarios, A), consequences (outcomes, C) and probabilities (P). Severity is a way of characterizing consequences that refers to intensity, size, expansion, volume, and other potential measures of magnitude, and affects what people value. Losses and gains, expressed in terms of money or death toll, for example, are ways of determining the severity of the consequences.
Indeed, the indicators obtained by the authors can be used both for quantitative risk assessment and for risk management and decision-making on projects in the field of safety (as already mentioned, the task turned out to be unsolved). However, this method has a number of disadvantages. With a larger number of indicators (factors), there are no formulas for calculations or specially developed software, which will not give a complete picture of the risks associated with a safety gap. Furthermore, probability is just a tool used to represent or express uncertainties. It follows that risk analysis should not be limited to characteristics A, C, P. The importance of uncertainty should be emphasized (i.e. will events occur and what will be the consequences?). In this case, the risk is characterized by uncertainty of the result, actions and events.
The authors note, that “uncertainty and severity are intrinsic/inherent properties of safety risks. Uncertainty and severity of factors causing safety risks may act as exogenous latent variables to the safety risks, as mediators or moderators in the path between safety risks and mitigation measures, or as exogenous latent variables to the MM construct, causing the practical measures implemented on construction sites to mitigate or manage the impacts of safety risks to be less effective. The effect of likelihood and severity is higher when they are acting as exogenous latent variables to MM construct, because the reduction in path coefficient and hence the total effect is higher. Therefore, mitigation measures taken by the contractors must always take into account the types of factors causing safety risks, as well as the uncertainty or likelihood and severity of these factors. According to [7], the likelihood of incidents and their severity could be reduced by conducting effective pre-job safety analyses.”
The authors determine the probability of accidents in construction, based on the assumption that there must be a certain accident load on the construction site (especially for fatal cases); however, in real life, an accident can also occur at a lower level of stress (under factors that the authors excluded from their study). As a result, the assigned probability did not reflect this uncertainty.
The authors refer to probability and severity as a tool for describing uncertainties. All this leads the authors to conceptual difficulties that are incompatible with the everyday use of risk in most practical cases. As an example, consider the number of occupational accidents in construction in Malaysia from 2015-2021. In this case, the uncertainty is quite small, since the number of deaths varies quite little from year to year. Thus, following this definition of risk, we must conclude that the risk is low, even though the number of deaths is in the hundreds per year. It is clear that this definition of risk fails to capture the essential aspect, the measurement of consequences (intensity, size, distribution, etc. of consequences).
Thus, the author's model will not give a complete picture of the risks associated with a safety breach. It is incompatible with the daily use of risk in most practical cases. The title of the work could be: "Risk Modeling Using PLS-SEM Technique."
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Reviewer 3 Report
The methodology is correct, however the paper is very general and looks like a simple statistical application. Authors write inaccurate things about subjects that are not their own.
For example for "road accident they write "Road accident is one of the safety risks frequently happened in the construction industry". First of all, road accidents are the third cause of death for young people, so there is a problem with industries but it is marginal. So the authors should revise this part as well for the other sectors, otherwise the paper appears.
Furthermore, in the various sectors there are already methods to evaluate the safety risk, for example for road accidents there are different methods, one is through the safety index (see DOI 10.1016/j.aap.2022.106858) or more general aspects see doi: 10.1016/ j.trpro.2016.05.392
Authors should elaborate on the introduction and literature review
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Reviewer 4 Report
Dear authors, please take note of my comments (attached)
Comments for author File: Comments.pdf
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Round 2
Reviewer 1 Report
After the author's revision, the quality of the paper has been greatly improved. Issues in the original version have been resolved and the current version is suitable for publication.
Reviewer 2 Report
This article corresponds to the profile of the special issue. It may be published.
Reviewer 4 Report
Dear authors, I see that all my comments have been taken into consideration, so the paper may be accepted
