An Approach to Chance Constrained Problems Based on Huge Data Sets Using Weighted Stratified Sampling and Adaptive Differential Evolution†
Round 1
Reviewer 1 Report
The paper can be accepted for publication.
Author Response
I deeply appreciate your acceptance decision.
Reviewer 2 Report
In this revised manuscript, the author discard the previous DEDM optimization in favor of ADEP, an Adaptive Differential Evolution with pruning technique. Furthermore, the authors claims a mistake in the previous version and update the results. It is not very clear to me how this change affects the results since everything in figures seems unchanged [ Fig 3 to 11]. Additionally, it's not very clear why the author had to swap to ADEP which shows little novelties and minor improvements in running times since the average times between ADEP and ADE differ for about 0.1sec (is this unit of measurement correct?) . In this regard:
-Deviation Standard should be reported for running times in order to properly compare the results.
Also, I see tiny average running times and it's really hard to spot the advantage of one method over another and I wonder if would be valuable to try on harder problems that required more complexity for the fitness evaluation. For example on real dataset, which would also increase the overall strength of the paper.
Author Response
I deeply appreciate your comments for improving my paper.
Author Response File: 
Author Response.pdf
This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.
Round 1
Reviewer 1 Report
Lines 29-32 mention CCP with Monte Carlo simulation. however, not every CCP problem uses Monte Carlo simulation. please be more specific about the ideas you wanted to express here.
The paper should contain more elaborated literature review to discuss earlier models and the contributions of the proposed models.
Why y_n in Eq. 7 is called indicator function with y_n \in {0,1}? only one \epsilon^n can satisfy the condition? also, indicator function is 1(.), not y.
please avoid contractions (can't)
Section 3.2.1 mentions bins or strata. indeed, strata refer to specific parameters of population (e.g., age category), whereas bins are more general. It is unclear whether the bins are ordered?
unclear term "exemplar sample". perhaps this is just a subsample.
"Algorithm of Proposed DEDM" should read The Proposed Algorithm of DEDM
the title of the paper mentions big data, yet there are no examples of the big data presented in the paper. please revise.
there is no discussion in the paper. thus, Section 8 should be Conclusions.
Reviewer 2 Report
The authors purport to use a new weighted sampling approach to solve probabilistic optimization problems.
It is questionable if their approach been clearly explained. The unnecessary use of mathematical formulations to explain simple concepts, insufficiently detailed in the text, coupled with a gratuitous use of greek symbols of very doubtful utility, make the formulation particularly turgid. The overall writing style is poor and should definitely be revised. There are nonetheless more pressing problems with the paper.
a) The method is given without any demonstration over a known data set. From what I can understand the data has been simulated (e.g., text above Eq. 33, on page 10) There are too many problems with this approach, but we can list two a) typically variables in watershed models are not independent (which is an assumption of the model, see lines 73-77 of page 2); b) Normal distributions are never used in watershed models except for the most simplistic cases - typically an extreme value distribution like a Weibull or Gompertz are used - this would obviously make the full sampling approach used (WSS) questionable just by itself
b) There is little variety of even artificially created problems. The second test case is very similar to the first one and even on this simulated data there seems no difference between SRS and WSS when the problems is just slightly bigger
c) There is insufficient discussion of the results in table 4. And no comparison whatsoever to any other methods. The bottom of page 14 (lines 258-272) are enigmatic and no justification what soever for the sample sizes taken , sufficiency or correction levels. Finally the reader reads that "From Table 4, we can confirm the availability of the proposed method" Whatever this may mean.
d) On tables 5 and 6 authors change parameters of their evolutionary approach, but no comparison whatsoever with any other method is given. Furthermore no statistical assessment whatsoever is given of the quality of the solutions, rather than the simplest casuistic values
In essence. The authors propose a modification of evolutionary learning to address difficult optimization problems, while at the same time proposing a new sampling method. They do not compare their approach to anything else, use two simulated unrealistic data for test beds and no statistics whatsoever are presented to justify their approach. This does not mean that the method does not work, but rather that its poor presentation makes it unfit for publication as it is
Reviewer 3 Report
The authors investigate the problem of chance constrained optimization in a Big Data context, where typically the available dataset can not be entirely employed for computational reasons.
Indeed, they proposed an efficient method in order to estimate the probabilities constraints involved in the optimization problem without considering the full dataset, by following a data reduction method inspired by stratified sampling techniques. Additionally, the authors proposed an optimization method based on Differential Evolution heuristic for solving the chance constrained problem, introducing a technique called Directed Mutation which aims at improve the cardinal DE.
In the manuscript, the novelties introduced are extensively described from a formal viewpoint along with a comprehensive theoretical background that improve the quality of the presentation. Furthermore, the tests proposed on the two cases of study regarding the Flood Control are very well described and formally formulated in a comprehensive manner. Regarding the experimental section, the authors could improve the overall quality by providing a deeper explanation of the results obtained. The major weakness that slightly undermine the paper, regards to the strength of the conclusion drawn for the DEDM: I would have expected comparisons between the state-of-the art methods and the proposed DEDM approach and major numerical results supporting the reduction of running time achieved with the pruning technique.
In particular:
-it could be very kind to discuss the statement in line 237-238 (appears also for the second case in line 271-272) that claims the difficult to obtain a feasible solution for smaller sufficiency levels. Is this a general results? Is it dependent on the sampling employed? Is it dependent to the number N of samples considered?
-How do you choose the number of sample N used for SRS and WSS in the experimental section? Did you follow the method described on Section 3.1.2? If yes, δ and ε how are determined?
-In line 278 εh seems never defined on the text.
-In line 293, you claim that running times are improved with pruning techniques by only consider the rate of discarded vectors. This could be reinforced reporting the average running times, with and without pruning.
