ELECTRE-TRI Multicriteria Approach for Measuring Performance of Rural Co-Operatives in Southwest Paraná, Brazil^†

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The manuscript, really interesting and promising according to its aims, study object, and methodology, being this last suitable.

However, it lacks of different aspects which must be analysed in more detail:

- Firstly, according to the scientific background, incorporating some references about the relevance of the third sector, and cooperatives to local and rural development; i.e. Olmedo, L., & O'Shaughnessy, M. (2022). Community‐based social enterprises as actors for neo‐endogenous rural development: a multi‐stakeholder approach☆. Rural Sociology, 87(4), 1191-1218.

- Secondly, it´s not clear the established categorization made by the cooperatives, the three levels of performance based on the principles of solidarity economy.

- Thirdly, and more importantly, it is needed more information/causes explaining the results and the data; why did some dimensions not achieve good evaluation? More context and geographical information would probably help to improve the reasons behind these results.

Author Response

Dear Reviewer

Thank you very much for your comments and contributions.

We have listed and responded to your comments below.

- Comment 1: Firstly, according to the scientific background, incorporating some references about the relevance of the third sector, and cooperatives to local and rural development; i.e. Olmedo, L., & O'Shaughnessy, M. (2022). Community‐based social enterprises as actors for neo‐endogenous rural development: a multi‐stakeholder approach☆. Rural Sociology, 87(4), 1191-1218.

Response 1: We incorporated the reference Olmedo, L., & O'Shaughnessy, M. (2022). Community‐based social enterprises as actors for neo‐endogenous rural development: a multi‐stakeholder approach. Rural Sociology, 87(4), 1191-1218, in the paper introduction. How you can see in line 97.

 

- Comment 2: Secondly, it´s not clear the established categorization made by the cooperatives, the three levels of performance based on the principles of solidarity economy.

Response 2: We clarified this by including sentence in line 325.

 

- Comment 3: Thirdly, and more importantly, it is needed more information/causes explaining the results and the data; why did some dimensions not achieve good evaluation? More context and geographical information would probably help to improve the reasons behind these results.

Response 3: We included sentence in line 408.

 

We hope we have addressed all of your concerns.

With the best regards

The authors

Reviewer 2 Report

Comments and Suggestions for Authors

In the article "ELECTRE-TRI MULTICRITERIA APPROACH TO MEASURE PERFORMANCE IN RURAL SOLIDARITY ECONOMY ORGANIZATIONS", the authors perform an MCDA for the agriculture co-operatives in Brazil, highlighting its significance to sustainability. The manuscript is well written, with all the appropriate information in each chapter and the results are clear. However, my main concern, considering the research methodology, is this: 

In line 270, it is mentioned that "Based on the answers to the questionnaire, an evaluation matrix of the alternatives in each criterion was constructed, showing the performance of each co-op in relation to the 271 defined decision criteria". 

From this sentence it can be concluded that the performance of each co-operative was based on their own answers, which could indicate a certain degree of bias. The authors should state that clearly in the manuscript. Therefore, I suggest minor revision. 

 

Author Response

Dear Reviewer

Thank you very much for your comments and contributions.

We have listed and responded to your comments below.

- Comment:  In line 270, it is mentioned that "Based on the answers to the questionnaire, an evaluation matrix of the alternatives in each criterion was constructed, showing the performance of each co-op in relation to the 271 defined decision criteria". 

From this sentence it can be concluded that the performance of each co-operative was based on their own answers, which could indicate a certain degree of bias. The authors should state that clearly in the manuscript. Therefore, I suggest minor revision. 

 

Response: We included sentence, in line 158 to clarify this, where we explain the methodology.

 

We hope to satisfy you with this.

 

With the best regards

 

The authors

Reviewer 3 Report

Comments and Suggestions for Authors

With great interest, I read the reviewed article. The article aligns with its title, providing a detailed analysis and demonstration of the performance evaluation model, as promised. The author justifies the importance of the topic by highlighting its significance for sustainable development, emphasizing the social and economic benefits of the solidarity economy organizations, and indicating the need and innovativeness of the developed performance evaluation model.

The author defined the objective of the article as developing and applying a performance evaluation model for solidarity economy enterprises and achieved this objective. A detailed model was developed, applied in practice, analyzed, and conclusions were presented, which demonstrates the full realization of the intended goal.

The article makes a significant contribution to the field of the solidarity economy and introduces an innovative model for evaluating its performance. However, to fully exploit its potential, certain corrections and additions are necessary, which would significantly increase the value and utility of the reviewed work.

The data collection process described by the author included the selection of cooperatives, the development of a questionnaire, the collection of responses from cooperative presidents, and the analysis of the data using specialized multi-criteria analysis software. This systematic and detailed data collection process provides a solid foundation for evaluating the performance of solidarity economy enterprises. However, I see a significant problem here. The data comes from 2018. Six years is a long enough period to significantly change the conditions for data collection, and thus the obtained results.

The author did not address the issue of data aging in the article. In the results and conclusions section, the author discusses the results based on the collected data and presents conclusions about the usefulness and stability of the model. However, there is no reference to the currency of the data and possible changes that may have occurred since their collection. In the future research section, the author suggests the possibility of applying the model to other solidarity economy enterprises and adapting the model to the specific conditions of other cases. This indicates the author's awareness that the model can be used in various contexts, but does not directly address the need to update the data.

In the context of scientific research, especially those concerning evaluation models, the currency of data can be crucial for the validity and usefulness of the results. The lack of reference to this aspect is considered a weakness of the article, as the results obtained based on 2018 data may no longer reflect the current situation of the studied cooperatives. For a more comprehensive assessment of the model's usefulness and its results, it is necessary for the author to address the issue of data aging and suggest regular updates to the collected information so that the model can be used continuously and provide current and accurate results.

Besides the outdated data, I see two more aspects that I would like the author to explain. The model created by the author has great applicability potential (as stated by the author in the article). I believe a detailed analysis of the economic costs and benefits associated with the implementation of the proposed model should be conducted. It is worth referring to operational, administrative costs of implementing the model and its maintenance costs. It is also necessary to indicate economic benefits such as improved operational efficiency, increased cooperative revenues, cost reduction, or potential long-term benefits such as increased financial stability, better market reputation, and increased ability to attract investments.

The author can also conduct a detailed cost-benefit analysis (CBA) of the model's implementation. CBA allows for comparing the expected costs with the anticipated benefits, which can help in assessing the feasibility and efficiency of the model. Examples of model implementations in different cooperatives can provide valuable data for conducting CBA. Case studies can show the actual costs and benefits and enable conclusions about the model's feasibility.

The article should also include a risk and uncertainty analysis and implementation recommendations (including implementation strategies and monitoring and evaluation).

Another issue I would like the author to address is the number of evaluation criteria used in the research. Although 36 criteria provide a solid foundation, some important aspects of the cooperative's activities may have been omitted, which could affect the comprehensiveness of the performance evaluation. Several examples of such potential omissions can be pointed out:

- The criteria do not directly include the cooperatives' innovativeness or their ability to adapt to changing market and technological conditions. Innovativeness is a crucial element for long-term success and sustainable development.

- The criteria do not directly assess customer or beneficiary satisfaction with the cooperative's activities. Customer opinions and satisfaction are key for maintaining business relationships and long-term success.

- The criteria do not include relationships with stakeholders (e.g., suppliers, business partners, local communities), which are an important factor influencing the cooperative's activities.

- Human resource management criteria could be more detailed, including aspects such as employee satisfaction, work-life balance, and recruitment and retention policies.

- A detailed evaluation of operational process efficiency, including supply chain management, logistics, and production, is not explicitly included in the criteria.

- Social innovation management aspects, which can contribute to solving social problems in new and creative ways, may be an important element of the cooperative's activities but are not directly included in the presented criteria.

And finally, two more comments:

First: the abstract contains information about the results, but more detailed conclusions about the significance of these results could be added. The abstract could include conclusions regarding the model's impact on the cooperatives' management practices or long-term benefits for farmers.

Second: literature citations should be adjusted to the formal standards of the journal.

Author Response

Dear Reviewer

Thank you very much for your comments and contributions.

We have listed and responded to your comments below.

 

Comment 1. The author did not address the issue of data aging in the article. In the results and conclusions section, the author discusses the results based on the collected data and presents conclusions about the usefulness and stability of the model. However, there is no reference to the currency of the data and possible changes that may have occurred since their collection. In the future research section, the author suggests the possibility of applying the model to other solidarity economy enterprises and adapting the model to the specific conditions of other cases. This indicates the author's awareness that the model can be used in various contexts, but does not directly address the need to update the data.

In the context of scientific research, especially those concerning evaluation models, the currency of data can be crucial for the validity and usefulness of the results. The lack of reference to this aspect is considered a weakness of the article, as the results obtained based on 2018 data may no longer reflect the current situation of the studied cooperatives. For a more comprehensive assessment of the model's usefulness and its results, it is necessary for the author to address the issue of data aging and suggest regular updates to the collected information so that the model can be used continuously and provide current and accurate results.

 

Response 1: We included sentences, see in line 467 and in line 472.

 

Comment 2. Besides the outdated data, I see two more aspects that I would like the author to explain. The model created by the author has great applicability potential (as stated by the author in the article). I believe a detailed analysis of the economic costs and benefits associated with the implementation of the proposed model should be conducted. It is worth referring to operational, administrative costs of implementing the model and its maintenance costs. It is also necessary to indicate economic benefits such as improved operational efficiency, increased cooperative revenues, cost reduction, or potential long-term benefits such as increased financial stability, better market reputation, and increased ability to attract investments.

The author can also conduct a detailed cost-benefit analysis (CBA) of the model's implementation. CBA allows for comparing the expected costs with the anticipated benefits, which can help in assessing the feasibility and efficiency of the model. Examples of model implementations in different cooperatives can provide valuable data for conducting CBA. Case studies can show the actual costs and benefits and enable conclusions about the model's feasibility.

The article should also include a risk and uncertainty analysis and implementation recommendations (including implementation strategies and monitoring and evaluation).

Response 2:

In the second and third paragraphs of Conclusions, we comment on the benefits of applying the model.

A more in-depth analysis at this time is unfortunately not feasible, as it would require a new stage of research, which is not possible in the time made available by the journal for the adjustments suggested by the reviewers.

 

Comment 3. Another issue I would like the author to address is the number of evaluation criteria used in the research. Although 36 criteria provide a solid foundation, some important aspects of the cooperative's activities may have been omitted, which could affect the comprehensiveness of the performance evaluation. Several examples of such potential omissions can be pointed out:

- The criteria do not directly include the cooperatives' innovativeness or their ability to adapt to changing market and technological conditions. Innovativeness is a crucial element for long-term success and sustainable development.

- The criteria do not directly assess customer or beneficiary satisfaction with the cooperative's activities. Customer opinions and satisfaction are key for maintaining business relationships and long-term success.

- The criteria do not include relationships with stakeholders (e.g., suppliers, business partners, local communities), which are an important factor influencing the cooperative's activities.

- Human resource management criteria could be more detailed, including aspects such as employee satisfaction, work-life balance, and recruitment and retention policies.

- A detailed evaluation of operational process efficiency, including supply chain management, logistics, and production, is not explicitly included in the criteria.

- Social innovation management aspects, which can contribute to solving social problems in new and creative ways, may be an important element of the cooperative's activities but are not directly included in the presented criteria.

Response 3: The model and criteria were based on the principles of the solidarity economy, so these topics highlighted by the reviewer, which are indeed very relevant, are not in the model. Because they are small cooperatives, some of the criteria mentioned are not even considered by them.

However, in Conclusions we insert in line 473.

 

Comment 4. And finally, two more comments:

First: the abstract contains information about the results, but more detailed conclusions about the significance of these results could be added. The abstract could include conclusions regarding the model's impact on the cooperatives' management practices or long-term benefits for farmers.

Response 4: Done. You can see in abstract.

Second: literature citations should be adjusted to the formal standards of the journal.

Response4: Done.

 

We really hope to satisfy you with this.

 

With the best regards

 

The authors

Reviewer 4 Report

Comments and Suggestions for Authors

I find this work interesting and believe that more publications on cooperatives are needed. However, I have some concerns and recommendations to upgrade the manuscript to be published:

1) Cooperatives must be in the title instead of Solidarity Economy Organizations. Cooperatives must be included as a keyword.

2) Please clarify that a cooperative is one (important) type of social organization, and explain that there are other types of organizations within the solidarity economy. Support your arguments with references such as:

Castilla-Polo, F., & Sánchez-Hernández, M. I. (2020). Cooperatives and sustainable development: A multilevel approach based on intangible assets. Sustainability, 12(10), 4099.

Monzon, J. L., & Chaves, R. (2008). The European Social Economy: concept and dimensions of the third sector. Annals of public and cooperative economics, 79(3‐4), 549-577.

Sánchez-Hernández, M. I., & Castilla-Polo, F. (2021). Intellectual capital as a predictor of cooperative prominence through human capital in the Spanish agrifood industry. Journal of Intellectual Capital, 22(6), 1126-1146.  

3) Please actualize the information in the first paragraph of the manuscript (FAO, 2012) as we are in 2024. In general terms, all references must be updated. Authors should include citations from the recent years, specifically from 2019 to 2024

4) Both MCDA and ELECTRE-TRI should be explained in more detail. For those unfamiliar with these methods, it is crucial to understand what is being done and why. Additionally, the reasons for selecting the criteria in Table 1 rather than others must be explained, with sources cited and the choices well-argued. Figure 1 cannot be presented as it is, appearing as outputs from the authors' computer and in Portuguese; the reader cannot understand this figure at all.

5) Finally, I think it might be interesting to include the questionnaire so that the article can be useful to other researchers and gain more citations.

I hope my comments help improve this work and that it will be published soon. Good work

 

Author Response

Dear Reviewer

Thank you very much for your comments and contributions.

We have listed and responded to your comments below.

Comment 1 - Cooperatives must be in the title instead of Solidarity Economy Organizations. Cooperatives must be included as a keyword

Response 1: Done

 

Comment 2 - Please clarify that a cooperative is one (important) type of social organization, and explain that there are other types of organizations within the solidarity economy. Support your arguments with references such as:

Castilla-Polo, F., & Sánchez-Hernández, M. I. (2020). Cooperatives and sustainable development: A multilevel approach based on intangible assets. Sustainability, 12(10), 4099.

Monzon, J. L., & Chaves, R. (2008). The European Social Economy: concept and dimensions of the third sector. Annals of public and cooperative economics, 79(3‐4), 549-577.

Sánchez-Hernández, M. I., & Castilla-Polo, F. (2021). Intellectual capital as a predictor of cooperative prominence through human capital in the Spanish agrifood industry. Journal of Intellectual Capital, 22(6), 1126-1146.  

Response 2: We incorporated this references in the paper. You can see in line 60.

 

Comment 3 - Please actualize the information in the first paragraph of the manuscript (FAO, 2012) as we are in 2024. In general terms, all references must be updated. Authors should include citations from the recent years, specifically from 2019 to 2024.

Response 3: We used the current information. You can see this in the first paragraph of Introduction section.

 

Comment 4 - Both MCDA and ELECTRE-TRI should be explained in more detail. For those unfamiliar with these methods, it is crucial to understand what is being done and why. Additionally, the reasons for selecting the criteria in Table 1 rather than others must be explained, with sources cited and the choices well-argued. Figure 1 cannot be presented as it is, appearing as outputs from the authors' computer and in Portuguese; the reader cannot understand this figure at all.

Response 4:  We have included more detail both MCDA and ELECTRE-TRI, but the space in the paper is limited. See line 158.

About the Figure: We corrected it

 

Comment 5 - Finally, I think it might be interesting to include the questionnaire so that the article can be useful to other researchers and gain more citation.

Response 5: Done

 

We really hope to satisfy you with this.

 

With the best regards

 

The authors

Reviewer 5 Report

Comments and Suggestions for Authors

The paper highlights the crucial role of a satisfactory quality of life in rural areas in achieving sustainable development. The authors show that solidarity economy organisations are catalysts in promoting economic, social, and environmental sustainability. In this vein, the paper proposes a performance evaluation model to support decision-making in social economy enterprises. The authors use the Southwest region of Parana State, Southern Brazil, as a case study to test the applicability of the performance evaluation model. The paper is interesting but requires some revisions before being published in Economies.

1. The Southwest region of Parana State, Southern Brazil, should be incorporated in the paper's title.
2. The authors should provide a historical/theoretical perspective of the solidarity economy to illustrate its conjunction with capitalist development.
3. The authors rightly point out that the solidarity economy is associated with sustainable development (lines 38-39). The authors have to add a brief discussion on sustainable development to engage the readers. Moreover, it is highly recommended to include references to the following two papers: 'Broad strokes towards a grand theory in the analysis of sustainable development: a return to the classical political economy', New Political Economy, 27(5), pp. 866-878, and 'History, Knowledge, and Sustainable Economic Development: The Contribution of John Stuart Mill’s Grand Stage Theory', Sustainability, 13 (3).
4. The paper needs a separate section to present its theoretical and policy implications.
5. What are the limitations of this research paper? For instance, what are the potential challenges in the model's application?   

Comments on the Quality of English Language

Moderate editing of the English language is required.

Author Response

Dear Reviewer

Thank you very much for your comments and contributions.

We have listed and responded to your comments below.

Comment 1 - The Southwest region of Parana State, Southern Brazil, should be incorporated in the paper's title.

Response 1: Done

 

Comment 2 - The authors should provide a historical/theoretical perspective of the solidarity economy to illustrate its conjunction with capitalist development.

Response 2: Unfortunately, provide this perspective at this time is not feasible, as it would require a new stage of research, which is it impossible is impossible with the time allocated of adjustments suggested by the reviewers.  Furthermore, it would stray a little from the focus of the article. And, the space in the paper is limited.

 

Comment 3 - The authors rightly point out that the solidarity economy is associated with sustainable development (lines 38-39). The authors have to add a brief discussion on sustainable development to engage the readers. Moreover, it is highly recommended to include references to the following two papers: 'Broad strokes towards a grand theory in the analysis of sustainable development: a return to the classical political economy', New Political Economy, 27(5), pp. 866-878, and 'History, Knowledge, and Sustainable Economic Development: The Contribution of John Stuart Mill’s Grand Stage Theory', Sustainability, 13 (3).’

Comment 4: The paper needs a separate section to present its theoretical and policy implications.

Response 3 and 4: Response: We really sorry, but it is impossible. The space and the time are limited. We considered it is very important, although, we believe it is not feasible at this time.  Furthermore, it conflicts with the focus of the paper.

 

Comment 5 - What are the limitations of this research paper? For instance, what are the potential challenges in the model's application?   

Response 5: We improve the Conclusion section. You can see in the paper.

 

The English revisions was not possible due to the deadline. But, if the article is accepted and it is necessary to do the revision, we will commit to doing so.

 

We really hope to satisfy you with this.

With the best regards

The authors

Round 2

Reviewer 4 Report

Comments and Suggestions for Authors

Thank you for improving the manuscript following my recommendations. I continue thinking that Figure 1 has not a good presentation. It is not necessary to show 6 times the screenshot of the computer with the information in the left side of each screenshot and a lot of space in white. Good luck in this second round. 

Author Response

Dear Reviewer

Thank you very much!

It is really difficult to improve the presentation of Figure 1 because its purpose is to present the result presented by the software. This is how the software presents it. However, we have tried, once again, to improve the figure. We hope to meet your expectations.

With the best regards

The authors

Reviewer 5 Report

Comments and Suggestions for Authors

I would like to thank the authors for their effort to address my previous comments. However, I believe that, at least, a brief discussion of sustainable development is necessary to put their analysis into a broader perspective. 

Comments on the Quality of English Language

Moderate editing of the English language is required.

Author Response

Dear Reviewer

Thank you very much!

We incorporated your suggestion in the paper. You can see in line 443.

We hope we have addressed all of your concerns. 

With the best regards

 

The authors

Round 3

Reviewer 5 Report

Comments and Suggestions for Authors

 

The authors addressed my previous comments and improved their manuscript.

Comments on the Quality of English Language

Minor editing of the English language is required.