Weather Radar Echo Extrapolation with Dynamic Weight Loss

Round 1

Reviewer 1 Report

This paper presents a dynamic weight loss function during deep learning modeling to improve the predictive performance of weather radar echo extrapolation. The results show that dynamic weight loss is effective in reducing the accumulation of errors over time and improves the predictive performance of currently popular deep learning models. Therefore, it is a relatively innovative paper. There are several review comments as follows:

1.     L10, precipitation nowcasting is a crucial instrument, “instrument” is not suitable;

2.     L206, 216 and L229, Using different symbols to represent Hadamard product and element-wise multiplication, however, what is the difference between the two operations?

3.     L218, f denotes the activation function. f has already been used to represent the forgetting gate in formula 2, so here it can be directly written as tanh.

4.     L275-277，The error of each subsequent frame will gradually accumulate over time, which will lead to blurring and loss of accuracy of the image. In fact, blur is mainly not caused by error accumulation, but by the smoothing effect of algorithms such as CNN itself.

5.     L372, this section is not necessary.

6.     L636 and 637, two therefore are too close.

This paper is fluent in English.

Author Response

Please see the attachment

Author Response File: Author Response.docx

Reviewer 2 Report

The authors have presented a new loss function for deep learning in the context of radar image nowcasting. The dynamic weight loss function takes into account the rainfall intensity, weighting heavy rainfall more highly so that predictions that miss heavy (severe) rainfall will be penalised more. Additionally the weight changes with the forecast length. Four deep learning models are tested using four loss functions. The results are evaluated using standard precipitation metrics.

Overall the paper is quite clearly written, the experiment is sensibly designed and described, and the analysis of results makes sense. 

Comments:

L97-L104. This is a very long sentence. Break it into smaller pieces to be easier to read.

L109. Check order of year and author.

L129-130. Not clear

L134-137. This is your key point reasoning why you suggest the new weight loss method. It could be explained more clearly. Maybe also start a new paragraph to emphasize this point.

L178 The radar data are (according to DOI link) CAPPI at 3 km (above sea level?). Do you regrid from PPI (polar) coordinates or are the data regridded to the 256x256 km coordinates previously?

L199. FC-LSTM was not previously mentioned or defined (I assume it's Fully Connected?). Not sure if this acronym needs to be defined properly.

L204. C_t and O_t need to be capitalised to be consistent with the equation.

Eq 2, 3. 'b' is undefined

Eq 13. Should one of the y be ŷ? Also for Eq 14.

L478-486. Excessively long sentence and L484-485 particularly do not make sense. Please rewrite this for clarity.

Figures 2-5. Consider reordering the panels so that CSI and FSC are in separate columns.

Figures 2-5. It is strange that FSC and CSI increase with time step, since objectively skill should not increase. This suggests that some artifact is coming into play which biases the results. At a guess, this might be because the actual rainfall band is becoming more diffuse with widely-distributed moderate intensity rainfall, while the models also tend to devolve towards more smooth rainfall fields, as in Figures 7 and 8. However, further investigation would be warranted to understand why the skill indices increase, since these are only two examples given in Figures 7 and 8. Breaking down the validation metrics into individual cases that show clear improvement or degradation in skill might be useful to understand this.

Is the learning efficiency improved? Does the DLW function alter the learning time?

The nowcasting precipitation process that I am most familiar with from its use at various operational meteorological centres is STEPS (e.g. https://doi.org/10.1002/wrcr.20536). I'm not sure that your description of existing radar nowcasting methods captures the method used within STEPS to extrapolate radar data. However, I would leave it to the authors to determine its relevance to their work.

Some minor issues in the text need correcting; I have not listed all typographic errors, so the authors should check carefully, including the equations.

Author Response

Please see the attachment

Author Response File: Author Response.docx

Reviewer 3 Report

Weather Radar Echo Extrapolation with Dynamic Weight Loss:

·        Add some of the most important quantitative results to the Abstract.

·        In the last paragraph of the Introduction, the authors should mention the weak point of former works (identification of the gaps) and describe the novelties of the current investigation to justify that the paper deserves to be published in this journal.

·        Discuss the most important reasons for the variations of the trends in MAE scores for ConvLSTM, ConvGRU, PredRNN, and PredRNN++ using different loss methods.

·        “The experimental results show that the proposed dynamic weight loss outperforms current commonly used loss methods and can improve the forecasting performance of current radar echo extrapolation models, so the experimental results validate its effectiveness.”. Explain.

·        Focus on the advantages/disadvantages of the proposed method concerning the obtained results.

·        What are the strategies/recommendations to reduce uncertainties in this study?

·        At the end of the manuscript, explain the implications and future works considering the outputs of the current study.

  The quality of the language needs to be improved for grammatical style and word use.

Author Response

Please see the attachment

Author Response File: Author Response.docx

Round 2

Reviewer 2 Report

The authors are commended for their efforts to improve the readability of the manuscript, which addresses the majority of my comments regarding the original manuscript. I thank the authors for their careful responses to each comment.

 

On two points I am unsatisfied with the authors' response.

1.  Re Comment 9 for Eq 13. Your response does not clarify why the text defines y and ŷ on line 348, but the equation shows only y.

 

2. Comment 12 is the main reason why major revisions to the paper were recommended. The authors have not demonstrated their effort to understand why the CSI increases, which seemed unusual enough to warrant investigation. As I understand the authors' argument, the skill might improve later in the nowcast because the model is placing a greater importance on skill later in the nowcast. However, I don't think this necessarily means that the prediction accuracy would improve with respect to its own earlier predictions, although it is expected to improve compared with the MSE/MAE loss functions.

My recommendation was that the authors evaluate relationships between the CSI and the behaviour of the true reflectivity, to determine if a tendency to shift from convective to stratiform rainfall (high spatial variability to low spatial variability) is a common behaviour in their training and evaluation datasets. Does the forecasting problem become easier as the rain evolves? Or is the model tendency towards more diffuse rainfall being unduly benefited by a bias in the evolution of observed rainfall? This could be easily tested by characterising the rainfall in the training and validation datasets and looking for correlations with the skill metrics.
Alternatively, the authors could consider using the Equitable Threat Score instead of CSI, which may be less biased in some circumstances. Additionally, estimation of confidence intervals for the skill metrics would show if the behaviour of the CSI curves is significant or not. This may satisfactorily resolve this query.

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 3 Report

I appreciate the authors addressing the comments. The manuscript can be accepted in its current form. Congrats!

Author Response

Please see the attachment.

Author Response File: Author Response.docx