The Economic Benefits of Supporting Private Social Enterprise at the Nexus of Water and Agriculture: A Social Rate of Return Analysis of the Securing Water for Food Grand Challenge for Development

Round 1

Reviewer 1 Report

This paper presents an evaluation of a large USAID funded initiative – Securing Water For Food. It uses a social rate of return (EROR) approach to assess the cost and benefits of this initiative and compares the results with EROR for alternative investments, such as foreign aid portfolios. I like the idea of the paper as it is informative to provide quantitative assessments of the returns to aid. However, I find the paper not very well structured, and it contains many ‘loose ends’ as concept are not explained well and abbreviations not well explained. I also strongly feel a sensitivity analysis is missing, which should be added.

 

Major comments

• Line 88-203. In this section you present lots of details on the various projects, how data was collected to evaluate the projects, information on the Nigeria project, which acts as benchmark, etc. This is rather overwhelming for an introduction. I recommend you move this to at least two subsections in Materials and methods, which deal with (a) the details of the SWFF and (b) the Nigeria project. Section a can be combined with Section 3.4 where you summarize the innovations. Having read the full paper, it is not clear to me why you add an extensive description of the Nigeria project. In the main part of the paper you refer to it once (line 396) as a benchmark but that seems to be it. I would remove the detailed description of this project.
• Line 107: The SROR for Nigeria is introduced here. This is somewhat confusing as you do not clearly state that this is an alternative project, which is used to compare with the SWFF. Please make this clear.
• Line 229-246: The explanation is not always clear. On line 226, you introduce I but this only comes back in eq 2 (I think). I seems to be the number of innovations. And in eq2 why is C denoted with INVESTOR and then with INVESTER ADMIN?. I also would remove the BCR and SROR examples as one can assume readers know the basic equations and do not require a fictive example.
• Line 309: Costs of DIV’s portfolio. As mentioned before DIV is not explained in the text, so I do not understand this header.
• I would merge the section on NPV and MIRR with that on BCR and SROR as these concepts are related/more or less the same and then follow with the background section on the innovations (3.4, also see comment above). In this way your methods are grouped together, followed with Data. However, the link between the BCR, SROR on the one hand and the NPV and MIRR on the other hand is not clear. In the end, you only seem to calculate NPV/MIRR so why explain the other approaches? What is the preferred interest rate On line 305 you use 10% for the SROR but in line 374 you use 12%. This seems inconsistent. Why are they different? Table2, why does ICU Peru have three MIRR values?
• The results section is not always clear. What is the difference between incremental and whole economy NPV? This is not explained. On Line 393, you show the average MIRR is 42% but on line 413 you mention the SROR is 42%. However as mentioned above the SROR approach is not exactly the same as the MIRR approach. Hence, what did you calculate the SROR or the MIRR?
• Sensitivity analysis. The calculation of the MIRR and NPV strongly depends on the costs and benefits data as well as the interest rate. As you mentioned in Line139-155, there is no baseline study for the SWFF and data on costs and benefits were collected retroactively. Hence, this information is subject to large uncertainties. I would highly recommend adding a sensitivity analysis in which several parameters are varied to calculate the range of possible outcomes. The normal approach would be to undertake a Monte Carlo analysis in which parameters are sampled over a realistic range. Moreover as the high MIRR of the portfolio are determined by only a few very large projects (e.g. the one in Ghana), the sensitivity analysis would show how sensitive the portfolio outcome is to uncertainties in a handful of key projects.

 

Minor comments

• Line 12: “investments and significant and exceeds”. This sentence does not flow, perhaps remove second “and”?
• Line 38: “compared” maybe better to use “analyze” as this is what you mean I think.
• Line 42: What is the DIV program, first use of abbreviation?
• Line 51: Comma after “template”.
• Line 68 (SROR) and line 71 (SWFF), you already introduced these abbreviations in previous sections so better to be consistent and use them throughout.
• Line 168. It is a bit odd to mention the name of the team lead in the a scientific paper.
• Line 188: Lots of details, better to add to an annex.

Eq 2 and further. The layout of the equations is not consistent. Please center them and add an 

Author Response

Nigeria Markets II detailed explanation deleted

Details of SWFF innovation moved to Section 3.4, Section 3.4 renamed Innovator Selection to Innovator and Data Selection

Line 107, it is stated that SWFF and the Nigeria Markets II project are fundamentally different and only the numerical format of the Nigeria Markets II SROR was used for SWFF

BCR fictive example deleted.

Investor admin in equation 2 changed to investor

i in eq 1 and 2 described as being each individual innovation in the BCR portfolio. 

Line 309: Cost of DIV's Portfolio replaced with Cost of SWFF Portfolio

Line 305: Discount rate standardized to 12% as this rate is where finance rate of capital equals reinvestment rate of capital in SWFF MIRR

Table 2: Explanation of 3 different MIRRs for ICU Peru explained

NPV, MIRR and BCOR/SROR sections merged, sections renumbered accordingly.

Line 413: SROR changed to MIRR

Difference between incremental NPV and whole economy NPV explained

Sensitivity analysis and Monte Carlo simulation text and tables added in end of Results section. Baseline NPV is weighted according to the number of households reached per innovator. 

Reviewer 2 Report

The bibliographic sources are few, the authors do not prove the study of the subject approached in the specialized literature

Author Response

Bibliographic sources have been increased from 11 to 17, with more in-text citations and larger body of evidence to prove subject matter. 

Reviewer 3 Report

The subject of the article was the assessment of the economic benefits of supporting private social enterprises at the interface between water and agriculture, and the analysis of water security for food. The subject of the article is important and is a challenge for the development of the modern world and countries, especially those countries that have problems with access to water.
The issues discussed in the article are very interesting and important. The authors analyze the rate of return on financing social enterprises and make a comparative assessment of the socio-economic benefits. The results are supported by extensive research.
However, the article needs significant improvements:
1. In the abstract (10-19), the authors clearly presented the aim of the study and the assumed hypothesis, and included the research method and results. However, regarding the method used, it is worth describing it in more detail, specifying the time and place of the test. One can also consider a more precise formulation of the assumed research hypothesis.
2. The introduction (23-203) to the article needs to be improved. This part of the article is not clear and is not properly structured. first of all, the introduction is too long. The authors described the SWFF program in a very broad manner, thanks to which they could conduct the study. However, such a comprehensive description should be considerably shortened and included in the methodological part. The introduction, as the name itself says, should introduce the reader to the whole article, it should contain the essence of the analyzed issue described in the article (economic benefits of support, sustainable agriculture; foreign aid; social enterprise; international development; foreign direct investment; rate of return; water protection) e.t.c.). The content of the introduction should be supported by a thorough literature review.
3. After the introduction to the article, the authors should present the literature review that is missing from this article. The subject of the article is very interesting and currently extensively analyzed in many magazines, reports and books. The subject and content of the article contain many important keywords about which the authors should make a thorough literature review.
4. In the methodological chapter, the authors presented in detail the research assumptions, indicators, calculations, etc. However, this chapter should also contain information about materials, data sources, research objects. Some of the issues obtained in this chapter should be presented by the authors in the part with the results.
5. The graphic and tabular part is clearly presented.
6. The method used is suitable for testing the hypothesis / hypotheses and achieving the goal.
7. The results section (380-408) is too short and laconic. In order to improve the quality of the article, authors should reformat the methodological part and the results, e.g. the results described in the methodological part should be included in the results chapter.
8. The chapter "Discussion" also needs improvement. While the authors rightly explained why the results of the study resulted in such and not other results, there was no discussion of the results in the context of the theory on which the study is based, no dialogue between the results presented in the article and the results obtained by other researchers.
9. The conclusions of the study (455-474) are presented very briefly, but clearly.
10. The literature review (479-502) needs to be supplemented. In addition to item 11, the latest publications are from 2016, and we are now in 2022. The library databases contain a lot of current literature from well-known researchers dealing with issues similar to those in this article.

Author Response

1. "Through using an extensive benefit cost ratio, we obtain the Marginal Internal Rate of Return whereby the discounted value of future benefits is equal to the reinvestment cost of capital for the SWFF portfolio. This allows the impact of SWFF to be comparable to other investments and serve as a benchmark for future analyses" added to abstract. 
2. Introduction shortened by deleting detailed Nigeria Markets II survey (lines 113-130). Lines 149-187 taken to subsection 3.4 Innovator and Data Seleciton. 
3. Four in-text citations added to the main text, with more sources in the discussion section elaborating on the theory and practice of foreign aid, its inconsistences, and how it can be optimized. Bibliographic references expanded from 11 to 17 sources.
4. Research methods, methods of data collection, and other details on methodology taken from Lines 149-187 and moved into Methods section. Lines 188-203 from introduction moved to Materials and Methods section under line 211.  
7. Sensitivity analysis with baseline portfolio NPV and NPV changes from currency value, input, and wage constraints added to results section along with Monte Carlo simulation to test sensitivity to said constraints.  
8. Discussion section lengthened with 2 in-text citations. See 3. 
10. See 3. Sources added from 2016-2022. 

Round 2

Reviewer 1 Report

The authors revised the first draft considerable and I appreciate that they added a Monte Carlo analysis. I recommend the authors to undertake a thorough final editing as (a) many of the equations turn out wrong because of a problem with the font (interesting to see little birds instead of numbers/symbols in my version of the pdf and also equation numbering is missing, and (b) I detected missing spaces in between sentences somewhere.

Author Response

There was a formatting error when transferring some formulas and equations from Microsoft Word online to Microsoft Office. Images of original formulas are pasted over the equations with odd symbols, they are renumbered, and centered. 

Reviewer 3 Report

The authors of the article took into account too few items of literature, the subject of the article and the results of the study are not supported by a sufficient literature review, only 17.

Author Response

References changed from 17 to 27 and in text citations increased from 9 to 16. 

Literature review expanded much more into Discussion section with much more evidence given to support results and prove statements regarding foreign aid effectiveness.