Sustainability Performance Evaluation of Faceshield Bracket Manufacturing by Using the Analytic Hierarchy Process

Round 1

Reviewer 1 Report

 

Although hundreds of publications have been published using the AHP method, this article is interesting. It should be noted that the chosen application example was addressed in a complex way.

Evaluation data was collected from designers, machine operators, literature review, and experts. As many as 38 sets of indicators are used. Indicator selection methods were chosen and the general methodology used by the authors would be useful to other researchers. Therefore, more attention could have been paid to their description.

The article fits well with the theme of the journal. As for the problem to be solved, the latest sources of literature were used. However, the author cites very old MCDM methods developed 30-40 years ago. Many new and widely used MCDM methods (VIKOR, COPRAS, ARAS, WASPAS, EDAS, CODAS, MOORA, MULTIMOORA, etc.) have been developed recently. Detailed review articles have been written about them. There are articles to review the application of MCDM methods specifically to address sustainability issues.

These new sources of literature are not analyzed. It is proposed to rewrite and extend Section 3.5.1 and justify why the methods were chosen in the context of the present.

It is necessary to explain in detail why the AHP method was used in this study. There are also more new methods suitable and widely used for this type of research (BWM, SWARA, PIPRECIA, etc.).

 

A

Author Response

Dear Reviewer,

We highly appreciate the time and efforts you exerted to help us improve the quality of our manuscript. Thus, attached herewith please find the replies and comments from authors for the remarks you made. We strongly believe that we have tried our best to address all of your remarks.

Thank you,

Regards,

 

Reply to reviewers' comments

 

Remarks by reviewer #1

 

1. English language and style are fine but minor spell check is required.

 

2. Evaluation data were collected from designers, machine operators, literature reviews, and experts. As many as 38 sets of indicators are used. Indicator selection methods were chosen and the general methodology used by the authors would be useful to other researchers. Therefore, more attention could have been paid to their description.

 

3. The article fits well with the theme of the journal. As for the problem to be solved, the latest sources of literature were used. However, the author cites very old MCDM methods developed 30-40 years ago. Many new and widely used MCDM methods (VIKOR, COPRAS, ARAS, WASPAS, EDAS, CODAS, MOORA, MULTIMOORA, etc.) have been developed recently. Detailed review articles have been written about them. There are articles to review the application of MCDM methods specifically to address sustainability issues.

 

4. These new sources of literature are not analyzed. It is proposed to rewrite and extend Section 3.5.1 and justify why the methods were chosen in the context of the present.

 

5. It is necessary to explain in detail why the AHP method was used in this study. There are also more new methods suitable and widely used for this type of research (BWM, SWARA, PIPRECIA, etc.).

 

 

 

 

 

 

 

 

 

Replies for remarks by Reviewer #1

 

1. English language and style

 

• The authors revisited the entire manuscript and amended possible errors in the English language, style homogeneity, and spellcheck.

 

2. Data collection and evaluation

 

• As mentioned in the manuscript, the authors tried to undergo extensive research on design requirements (associated with the selected application), and gather relevant information which can be translated into relevant inputs required by the methodology applied for this study.

 

• The authors of this manuscript have a prior publication entitled “Sustainability performance indicators for additive manufacturing: a literature review based on product life cycle studies” where a total of 68 indicators are identified for sustainability performance evaluation. However, for this specific study and application, the authors selected 38 of these 68 indicators for sustainability performance by considering approaches of indicator selection mentioned in the manuscript. An example is demonstrated by considering one Sub-subcriteria (SSC) from each sustainability dimension on how this methodology is selected is also included in the manuscript to show how deep the authors made considerations of indicators selection as a series issue in the manuscript. As required by the AHP method, this is included in the manuscript as an ‘approach’, section 3.5.1 of the manuscript, and the complete judgement on existing alternatives is reported in Table 4.

 

3. Utilization of AHP as a MCDM method for this study?

 

• This section summarizes replies for remarks 3, 4, and 5 by the first reviewer. Yes, the authors fully agree with the remarks of the reviewer that many recent methods could be further reviewed and included to extend section 3.5.1. We have accepted the remark and included a comparison with a few more methods since it is difficult to include all recent methods that can be counted in more than 20s. Moreover, since there are many similarities, the authors believe that it is necessary to mention why we selected AHP for this study, with some references. Major reasons can be summarized mentioned as;

 

• AHP is one of the most powerful, easy to structure and solve complex problems which can be helpful in inconsistency verifications.

 

• Like the many available MCDM methods, AHP is fully capable of helping authors to easily identify problems, possible solutions, evaluate solutions and choose the best alternative.

 

• As mentioned in section 3.5.1 where a brief description of the approach is given, the strength and weaknesses of selected methods are assessed before selecting AHP as the MCDM method for the selected application. The most important criterion that helps researchers to adapt approaches is how performance is aggregated. It is mentioned that most methods can be categorized into one of the three operational approaches; the single criterion synthesis approach, synthesis outranking approach, and interactive local judgement approach with trial and error iterations.

 

• The concept of MCDM is an old concept and various methods have been evolving since several years ago. It can be assessed (authors tried to assess for the past five years) that there are many recent active studies and publications where AHP is still one of the major methods, sometimes by integrating it with other MCDM methods.

 

• As far as the method follows a scientific and logical approach, and based on the past experiences of researchers, we believe that the researchers can choose methods with reasons. Thus, we used the single criterion of synthesis approach, where AHP is one of the methods, to evacuate any incomparability, preferences are introduced and aggregated into a single function and then optimized.

 

Based on the reviewer’s feedback, the authors extended section 3.5.1 of the manuscript by adding the following three paragraphs. 

... Added to Section 3.5.1...

Nowadays, decision-makers have Various MCDM methods to choose in decisions on problems that involve multiple criteria. For example, the following methods have been developed in different periods having their features and objectives. SMART (1986), REGIME (1986), ORESTE (1980), VIKOR (1998), PROMETHEE I-II-III (1986), QUALIFLEX (1975), SIR (2000), EVAMIX (1982), ARAS (2010), Taxonomy (1763), MOORA (2004), COPRAS (1994), WASPAS (2012), SWARA (2010), DEMATEL (1971), MACBETH (1990), ANP (1996), MAUT (1976), IDOCRIW (2016), TODIM (1992), EDAS (2015), PAMSSEM I & II (1996), ELECTRE I-II-III (1990), EXPROM I & II (1991), MABAC (2015), CRITIC (1995), and KEMIRA (2014). Each method helps decision-makers to select the best alternatives among the different alternatives and has its features based on the independent and dependent attributes or criteria and whether or not the qualitative attributes should be converted into the quantitative attributes (Alinezhad and Khalili, 2019). 

Since there are significant similarities between the various MCDM methods, it is difficult to compare AHP with all existing MCDM methods. However, comparisons of AHP with two of the recently introduced methods, the Best worst method (BWM) and a Step-wise Weight Assessment Ratio Analysis (SWARA), are reported here. The BWM method helps decision-makers to increase the consistency and uniformity of MCDM methods and enough capability to determine the optimum results (even though various decision makers’ forward different opinions) in a decision making problem. Optimum results are acceptable by all decision-makers. Furthermore, both AHP and SWARA use pairwise comparison to express the relative comparisons to express the relative significance of the elements in a hierarchy, which is called the comparative importance of average value in the SWARA method. For the same number of criteria, the AHP method requires a significantly greater number of pairwise comparisons as compared to SWARA methods.

A large number of criteria may create possible inconsistency of performed comparisons. However, AHP’s capability for consistency verification procedure is an advantage over the SWARA method, which doesn’t have this capability (Stanujkic et al., 2015). Predefined scales (a nine-point scale) is used by the AHP method as reported in Table 3. However, the SWARA method has greater freedom to express judgements than the AHP method where a nine-point scale is used to make judgements (Saaty and Vargas, 2012). SWARA is developed recently where the relative significance and initial priority of the independent attributes are determined according to the opinion of the decision-maker, and then the relative weight of each attribute is determined before the priority and ranking of the attributes are done (Alinezhad and Khalili, 2019). Thus, AHP is selected as an MCDM methodology for this study for the fact that it is one of the most powerful, easy to structure and solve the defined problem, and can do inconsistency verifications even if the judgements are made based on the predefined Saaty’s scales.    

 

Additional remarks

The following remark is not mentioned by reviewer #1 but since the authors enhanced the quality of figures 3 to 8 based on remarks from other reviewers, it is included here for information.

 

4. Readability of Figures

 

• The authors revisited all figures especially figures 3 to 8 to make them as readable as possible. Now, the figures were inserted as editable and readable. The authors verified the readability by making soft and hardcopy prints in different qualities and also made judgements between working groups and colleagues.

 

• The reviewer also mentioned that there was a problem with understanding Figure 3. Figure 3 is only intended to illustrate the structuring of the problem from two perspectives. The first focuses on the hierarchy of criteria, sub-criteria, and sub-sub-criteria to simplify the overall evaluation. The hierarchical representation includes the main objective of the problem, the three criteria, the eleven sub-criteria, the thirty-eight sub-sub-criteria. The second shows that the three alternatives to be compared are related to all sub-sub-criteria. Thus all the sub-sub-criteria participate in the evaluation of the performance for all the alternatives.

 

References

Alinezhad, A., Khalili, J., 2019. New Methods and Applications in Multiple Attribute Decision Making (MADM), International Series in Operations Research & Management Science. Springer International Publishing, Cham. https://doi.org/10.1007/978-3-030-15009-9

Saaty, T.L., Vargas, L.G., 2012. Models, methods, concepts & applications of the analytic hierarchy process, 2. ed. ed, International series in operations research & management science. Springer, New York.

Stanujkic, D., Karabasevic, D., Zavadskas, E.K., 2015. A framework for the Selection of a packaging design based on the SWARA method. Eng. Econ. 26, 181–187. https://doi.org/10.5755/j01.ee.26.2.8820

 

Author Response File: Author Response.docx

Reviewer 2 Report

A pragmatically conceived research model with reserves in the triangulation of research tools and academic applicability, nevertheless interesting especially for the application sphere with a complex concept of analysis. For some experts, it represents an interesting source of information and, in any case, stimuli for discourse in the researched field of science. 

Author Response

Dear Reviewer,

We highly appreciate the time and efforts you exerted to help us improve the quality of our manuscript. Your comments concerning the model and its applicability are positive and highly encouraging. For your information, based on remarks from the other reviewers, we have updated section 3.5.1 of our manuscript and the quality of the figures.

Thank you,

Regards,

Reviewer 3 Report

The authors aim to evaluate the sustainability performance of medical product manufacturing using multicriteria decision-making methodology (MCDM). They used the analytic hierarchy process (AHP) as an example of MCDM to perform the evaluation. 
I have a number of concerns regarding this manuscript.
1) I found no sufficient reasoning on why the AHP is chosen. There should be a comparison of different MCDM methods to find the best evaluation criteria, including weighted sum method, weighted product method, Technique for Order Performance by Similarity to an Ideal Solution (TOPSIS), or others. The current straightforward analysis is too simple to meet the quality requirement of the Sustainability journal.
2) Figure 3 is really hard to read and to understand, it should be improved and redrawn. This is not the proper way to draw the hierarchical framework
3) Figures 4,5,6,7, and 8 are too small and hardly readable.
  

Author Response

Dear Reviewer,

We highly appreciate the time and efforts you exerted to help us improve the quality of our manuscript. Thus, attached herewith please find the replies and comments from authors for the remarks you made. We strongly believe that we have tried our best to address all of your remarks.

Thank you,

Regards,

 

Reply to reviewers' comments

 

Remarks by reviewer #3

 

1. Extensive editing of English language and style required.

 

2. I found no sufficient reasoning on why the AHP is chosen. There should be a comparison of different MCDM methods to find the best evaluation criteria, including weighted sum method, weighted product method, Technique for Order Performance by Similarity to an Ideal Solution (TOPSIS), or others. The current straightforward analysis is too simple to meet the quality requirement of the Sustainability journal.

 

3. Figure 3 is really hard to read and to understand, it should be improved and redrawn. This is not the proper way to draw the hierarchical framework.

 

4. Figures 4,5,6,7, and 8 are too small and hardly readable.

Replies for remarks by Reviewer #3

 

1. English language and style

 

• The authors revisited the entire manuscript and amended possible errors in the English language, style homogeneity, and spellcheck.

 

2. Utilization of AHP as a MCDM method for this study?

 

• This section summarizes replies for remark 2 by the 3^rd Yes, the authors fully agree with the remarks of the reviewer that many recent methods could be further reviewed and included to extend section 3.5.1. We have accepted the remark and included a comparison with a few more methods since it is difficult to include all recent methods that can be counted in more than 20s. Moreover, since there are many similarities, the authors believe that it is necessary to mention why we selected AHP for this study, with some references. Major reasons can be summarized mentioned as;

 

• AHP is one of the most powerful, easy to structure and solve complex problems which can be helpful in inconsistency verifications.

 

• Like the many available MCDM methods, AHP is fully capable of helping authors to easily identify problems, possible solutions, evaluate solutions and choose the best alternative.

 

• As mentioned in section 3.5.1 where a brief description of the approach is given, the strength and weaknesses of selected methods are assessed before selecting AHP as the MCDM method for the selected application. The most important criterion that helps researchers to adapt approaches is how performance is aggregated. It is mentioned that most methods can be categorized into one of the three operational approaches; the single criterion synthesis approach, synthesis outranking approach, and interactive local judgement approach with trial and error iterations.

 

• The concept of MCDM is an old concept and various methods have been evolving since several years ago. It can be assessed (authors tried to assess for the past five years) that there are many recent active studies and publications where AHP is still one of the major methods, sometimes by integrating it with other MCDM methods.

 

• As far as the method follows a scientific and logical approach, and based on the past experiences of researchers, we believe that the researchers can choose methods with reasons. Thus, we used the single criterion of synthesis approach, where AHP is one of the methods, to evacuate any incomparability, preferences are introduced and aggregated into a single function and then optimized.

 

Based on the reviewer’s feedback, the authors extended section 3.5.1 of the manuscript by adding the following three paragraphs.  

... added to Section 3.5.1...

Nowadays, decision-makers have Various MCDM methods to choose in decisions on problems that involve multiple criteria. For example, the following methods have been developed in different periods having their features and objectives. SMART (1986), REGIME (1986), ORESTE (1980), VIKOR (1998), PROMETHEE I-II-III (1986), QUALIFLEX (1975), SIR (2000), EVAMIX (1982), ARAS (2010), Taxonomy (1763), MOORA (2004), COPRAS (1994), WASPAS (2012), SWARA (2010), DEMATEL (1971), MACBETH (1990), ANP (1996), MAUT (1976), IDOCRIW (2016), TODIM (1992), EDAS (2015), PAMSSEM I & II (1996), ELECTRE I-II-III (1990), EXPROM I & II (1991), MABAC (2015), CRITIC (1995), and KEMIRA (2014). Each method helps decision-makers to select the best alternatives among the different alternatives and has its features based on the independent and dependent attributes or criteria and whether or not the qualitative attributes should be converted into the quantitative attributes (Alinezhad and Khalili, 2019). 

Since there are significant similarities between the various MCDM methods, it is difficult to compare AHP with all existing MCDM methods. However, comparisons of AHP with two of the recently introduced methods, the Best worst method (BWM) and a Step-wise Weight Assessment Ratio Analysis (SWARA), are reported here. The BWM method helps decision-makers to increase the consistency and uniformity of MCDM methods and enough capability to determine the optimum results (even though various decision makers’ forward different opinions) in a decision making problem. Optimum results are acceptable by all decision-makers. Furthermore, both AHP and SWARA use pairwise comparison to express the relative comparisons to express the relative significance of the elements in a hierarchy, which is called the comparative importance of average value in the SWARA method. For the same number of criteria, the AHP method requires a significantly greater number of pairwise comparisons as compared to SWARA methods.

A large number of criteria may create possible inconsistency of performed comparisons. However, AHP’s capability for consistency verification procedure is an advantage over the SWARA method, which doesn’t have this capability (Stanujkic et al., 2015). Predefined scales (a nine-point scale) is used by the AHP method as reported in Table 3. However, the SWARA method has greater freedom to express judgements than the AHP method where a nine-point scale is used to make judgements (Saaty and Vargas, 2012). SWARA is developed recently where the relative significance and initial priority of the independent attributes are determined according to the opinion of the decision-maker, and then the relative weight of each attribute is determined before the priority and ranking of the attributes are done (Alinezhad and Khalili, 2019). Thus, AHP is selected as an MCDM methodology for this study for the fact that it is one of the most powerful, easy to structure and solve the defined problem, and can do inconsistency verifications even if the judgements are made based on the predefined Saaty’s scales.    

...end...

 

3. Readability and the issue of understanding of Figures

 

• Based on the 3^rd and 4^th remarks by reviewer 3, the authors revisited all figures especially figures 3 to 8 to make them as readable as possible. Now, the figures were inserted as editable and readable. The authors verified the readability by making soft and hardcopy prints in different qualities and also made judgements between working groups and colleagues.

 

• The reviewer also mentioned that there was a problem with understanding Figure 3. Figure 3 is only intended to illustrate the structuring of the problem from two perspectives. The first focuses on the hierarchy of criteria, sub-criteria, and sub-sub-criteria to simplify the overall evaluation. The hierarchical representation includes the main objective of the problem, the three criteria, the eleven sub-criteria, the thirty-eight sub-sub-criteria. The second shows that the three alternatives to be compared are related to all sub-sub-criteria. Thus all the sub-sub-criteria participate in the evaluation of the performance for all the alternatives.

 

References

Alinezhad, A., Khalili, J., 2019. New Methods and Applications in Multiple Attribute Decision Making (MADM), International Series in Operations Research & Management Science. Springer International Publishing, Cham. https://doi.org/10.1007/978-3-030-15009-9

Saaty, T.L., Vargas, L.G., 2012. Models, methods, concepts & applications of the analytic hierarchy process, 2. ed. ed, International series in operations research & management science. Springer, New York.

Stanujkic, D., Karabasevic, D., Zavadskas, E.K., 2015. A framework for the Selection of a packaging design based on the SWARA method. Eng. Econ. 26, 181–187. https://doi.org/10.5755/j01.ee.26.2.8820

 

Author Response File: Author Response.docx

Round 2

Reviewer 1 Report

Accept in present form

Reviewer 3 Report

OK