Validation of the AROME, ALADIN and WRF Meteorological Models for Flood Forecasting in Morocco

Round 1

Reviewer 1 Report

This paper provides insights to implement robust flood forecasting approaches in Morocco. The study evaluates the model performances of three meteorological models included AROME, ALADIN and WRF, to forecast flood events. Moreover, the paper compares the two different forecasting methods. The first method relies on a standard hydrological model based on the SCS-CN infiltration method. The second method is based on a linear regression approach linking three meteorological parameters. Finally, three different soil moisture products are compared to estimate the initial soil moisture conditions before flood events for both methods. In general, this is a well-executed and well-written study. The topic of this paper is relevant to the scope of the Water.  However, to my knowledge, the hydrological method such as SWAT model, has similar application ability in flood forecasting just like the models used by the authors in this paper. So, why do the authors not use or compare the SWAT model with others?

Comments for author File: Comments.pdf

Author Response

Reviewer 1:

In general, this is a well-executed and well-written study. The topic of this paper is relevant to the scope of the Water.

We would like to thank you for this positive feedback.

However, to my knowledge, the hydrological method such as SWAT model, has similar application ability in flood forecasting just like the models used by the authors in this paper. So, why do the authors not use or compare the SWAT model with others?

We found that the suggestion of using SWAT model is quite interesting. But in the context of our study, we wanted a parsimonious model that has been already tested in the study area. The SWAT model is also using, in its most common version, the SCS-CN production scheme, but it is more intended to run continuously at the daily time step than hourly and event-based (see: https://doi.org/10.5194/hess-22-5001-2018). Since the HEC-HMS is an industry-standard for civil engineers, this this the reason why we choose the model, since the majority of the water department’s engineers work with this model.

Reviewer 2 Report

General comments:

   I enjoyed reviewing the manuscript of “Validation of AROME, ALADIN and WRF meteorological models for flood forecasting in South Morocco”. The authors started with introducing the importance of studying models for flood forecasting in South Morocco as well as clearly introducing the importance and novelty of this work. In general, this paper provided useful information on evaluating three meteorological models to provide hydrological forecasts in South Morocco. I recommend the manuscript for publication in water after minor revision. Below are some detailed comments and suggestions.

Introduction

Lines 84-86. “The considered NWP models are the AROME and AlADIN meteorological models that are implemented operationally by the Meteorological office of Morocco and the WRF model.”

First of all, the full name of AROME, ALADIN, and WRF models can be listed when first appear in the paper. Secondly, the authors may consider discussing a little bit about previous studies regarding to these three models in the introduction section, so the background of these three models can be better understood.

Equation (6)-(8)

The unit of parameters may be listed along with the definition of parameters.

Author Response

Reviewer 2:

I enjoyed reviewing the manuscript of “Validation of AROME, ALADIN and WRF meteorological models for flood forecasting in South Morocco”. The authors started with introducing the importance of studying models for flood forecasting in South Morocco as well as clearly introducing the importance and novelty of this work. In general, this paper provided useful information on evaluating three meteorological models to provide hydrological forecasts in South Morocco. I recommend the manuscript for publication in water after minor revision.

We appreciate your positive feedback, Thank you.

Lines 84-86. “The considered NWP models are the AROME and AlADIN meteorological models that are implemented operationally by the Meteorological office of Morocco and the WRF model.”

First of all, the full name of AROME, ALADIN, and WRF models can be listed when first appear in the paper.

We added the full name of AROME, ALADIN and WRF model in the introduction section, from the Line: 85 to 88

Secondly, the authors may consider discussing a little bit about previous studies regarding to these three models in the introduction section, so the background of these three models can be better understood.

This paper is the first evaluation of the ALADIN, AROME and WRF models for flood forecasting in Morocco. No studies have been conducted in Morocco to evaluate these forecasts for flood modelling in Morocco (to our knowledge). We added the configuration of these models in Morocco (Hdidou et al. 2020) in the Meteorological models section and also the use of these models in France (Demargne et al 2019, Sahlaoui et al 2020, Fourrié et al 2019).

Previous research of the application of WRF model when dealing with heavy precipitation and flash-flooding has been added in the introduction. From line 89 to 94

Equation (6)-(8): The unit of parameters may be listed along with the definition of parameters.

We added the units, Thank you.

Reviewer 3 Report

In this study the authors deal with the interesting topic of choosing both the best model (ALADIN vs. AROME vs. WRF) and the best method (coupled meteorological-hydrological models vs. regression approach) in order to develop a flash flood early warning system in south Morocco. This study is based on a systematic evaluation approach: first, the 3 different models are evaluated by using (1) comparison with rain fall data and (2) a contingency method; then the 2 methods are evaluated by using 3 different kind of soil moisture datasets. I found the article interesting and worthy of publication after the following the following three major corrections:

As it stands I found that the article is difficult to read because of its structure. I would recommend the authors to simplify and clarify their article by restructuring their sections 2 to 4.

For example, first presenting the models then the methods, then the events selected then the datasets used to evaluate the rainfall as well as use for the contingency table, then the datasets used to evaluate the regression method … This would truly help the reader better understand which data is used for what and how it is used …

In addition, all the location names (high Atlas, Rheraya, Ourika, etc.) mentioned in the text should appeared in a figure (Figure 1) including (1) the country borders as actually presented but also with the name of the different countries appearing, and (2) the zoomed area of study which is not necessarily well known for all the atmospheric community …

The ALADIN model should not be included in the study. Indeed, both AROME and WRF models are non-hydrostatic and convection-permitting models at 2.5 km resolution. Their inter-comparison is thus quite fair. However, the ALADIN model is a hydrostatic model at 10km resolution and by design will always provide less accurate results than the other two models. In this sense I do not see the value of showing in this article what is already common knowledge … Instead, I think the difference in terms of the physics used, the vertical structure used, the numerical schemes used, etc. …  in AROME and WRF should be discussed in depth in order to explain the differences in results. This then could open the door to a better understanding of the model performances depending of their configuration. For the moment the different type of the selected events is not discussed. Are all these events resulting from the same king of synoptic and hydrological conditions? I found that maybe this should also be investigated and discussed as the performance of the models may vary depending on it.

Author Response

Reviewer 3:

In this study the authors deal with the interesting topic of choosing both the best model (ALADIN vs. AROME vs. WRF) and the best method (coupled meteorological-hydrological models vs. regression approach) in order to develop a flash flood early warning system in south Morocco. This study is based on a systematic evaluation approach: first, the 3 different models are evaluated by using (1) comparison with rain fall data and (2) a contingency method; then the 2 methods are evaluated by using 3 different kind of soil moisture datasets. I found the article interesting and worthy of publication after the following three major corrections

Thank you for your positive feedback

As it stands I found that the article is difficult to read because of its structure. I would recommend the authors to simplify and clarify their article by restructuring their sections 2 to 4.

For example, first presenting the models then the methods, then the events selected then the datasets used to evaluate the rainfall as well as use for the contingency table, then the datasets used to evaluate the regression method … This would truly help the reader better understand which data is used for what and how it is used …

Thank you for your suggestion. It is indeed possible to present first the methods and then the data, or vice versa. However in our case the data available is a strong motivation for the methods proposed, therefore we kept the data section before methods. Yet we changed the structure by merging sections relevant to datasets, including soil moisture, into one homogeneous section.

 

In addition, all the location names (high Atlas, Rheraya, Ourika, etc.) mentioned in the text should appeared in a figure (Figure 1) including (1) the country borders as actually presented but also with the name of the different countries appearing, and (2) the zoomed area of study which is not necessarily well known for all the atmospheric community

We edited the Figure. Thank you.

The ALADIN model should not be included in the study. Indeed, both AROME and WRF models are non-hydrostatic and convection-permitting models at 2.5 km resolution. Their inter-comparison is thus quite fair. However, the ALADIN model is a hydrostatic model at 10km resolution and by design will always provide less accurate results than the other two models. In this sense I do not see the value of showing in this article what is already common knowledge

We definitely agree with you, but the ALADIN model is still operational in Morocco by the DMN (Directorate of National Meteorology) and we wanted to compare every model used by the DMN over Morocco, since no study exists comparing the used NWP models in morocco, this result is a proof that ALADIN is not accurate as AROME and WRF are. In addition, other countries of the ALADIN-HIRLAM consortium are still using only ALADIN for their forecasting applications. This included countries such as Algeria and Tunisia, so we think it is important to show the added benefit of convection-permitting models in North Africa, since there is no study that previously addressed this topic in the region.

in AROME and WRF should be discussed in depth in order to explain the differences in results. This then could open the door to a better understanding of the model performances depending of their configuration

The WRF model configuration is based on AROME’s characteristics, which make the comparison possible.

We have improved the description of all three NWP models in the introduction and in section 3 to provide the reader with additional information regarding model dynamics and physical parameterizations. The idea of unquestionably identifying specific causes for the observed differences in model performance is appealing but beyond the scope of this study, as WRF and AROME/ARPEGE are built upon different assumptions, numerical approximations, parameterization suites, vertical grid-spacings and even different horizontal grid structures. Since the three NWP models are robust and well-tested, differences in performance arise from the synergism of these factors, preventing the identification of the causes modulating the final outputs.

For the moment the different type of the selected events is not discussed. Are all these events resulting from the same king of synoptic and hydrological conditions?

I found that maybe this should also be investigated and discussed as the performance of the models may vary depending on it.

The selected events are from the same rain and discharge gauges, yet it must be stressed that data scarcity in an issue. Therefore it is difficult to compare in detail the synoptic conditions for the events since the data sample is quite low. The performances of the models are surely depending on the magnitude of the flood event but also to their spatial distribution. In addition the hydrological forecast is very sensitive to antecedent soil moisture conditions.

This is an analysis we would like to perform subsequently, when more data will be recorded in the upcoming years, to provide a detailed analysis of synoptic conditions associates with heavy rainfall events in this region. We added this element in the discussion.

In addition, we added the types of weather that are responsible of floods in our basins from line 137 to 142.

Reviewer 4 Report

Summary: the authors present a test of flash flood forecasting methods for two basins in Morocco. Their methods comparisons include combinations of three different NWP models (ALADIN, AROME, and WRF), two streamflow estimation methods (HEC-HMS and regression) and three different soil moisture products (in situ, ERA5, and ESA-CCI). This is useful work and I believe that the paper can be published after some revision. I do not request any new analysis, but I have several clarifying questions for the authors, some of which might lead to new analysis:

1. For reasons that the authors explain, the NWP experiments are conducted in reanalysis rather than forecast mode. This is fine for comparing models, but the authors should be clear that this means their results are not for true forecasts, but for NWP simulations informed by reanalysis boundary conditions. This could be addressed in the methods and/or discussion. On the point of boundary conditions, what dataset was used for lateral boundary conditions?

2. NWP evaluation: the authors conclude that WRF and AROME offer better performance than ALADIN. What is the basis for this conclusion, beyond the % bias result (and the associated hit rate for an absolute threshold)? If the NWP models suffer from consistent, systematic bias (which is not uncommon), couldn't the outputs be subjected to a simple bias correction? I would be interested to see how well the models perform in terms of espatial distribution of precipitation in each basin (satellite data could be used for comparison if in situ observations are inadequate) and in the contingency table and RMSE for precipitation and streamflow once a basic bias adjustment is performed.

3. NWP computational cost: operationally, running real-time ensembles of high resolution non-hydrostatic NWP models can be computationally prohibitive. What is the relative computational cost of the three modeling methods used, and what are the implications of those costs for operationalizing a flash flood forecast system?

4. What steps were made to optimize each NWP system for this region? I am not very familiar with ALADIN or AROME, but I know that WRF offers multiple physics options. It is standard practice to evaluate different sets of physics settings when applying WRF in a new context. Were such tests performed, or did the authors have prior knowledge of optimal physics settings? And which physics settings were used in the simulations presented in the paper?

5. As a major motivation for this study is operational flood warning, it will be necessary to have a system that can be run in real time. What is the data latency on the three soil moisture products tested in the regression model? Presumably only a product that has near zero latency, or very slow variability, would be useful in this context. It would be useful if the authors could comment on latency requirements and current latency reality for these products.

6. Were the soil moisture products only used for the regression? The use of the term "initial conditions" made me think that these products were used to initialize NWP, or at least HEC-HMS, but it seems from the methods that this was not the case. Please clarify. Also, if these products were not used to initialize NWP models and HEC-HMS, then what initial conditions did those models use?

7. It appears that the authors make no use of runoff predicted by the land surface model component of their NWP models. Again, I do not know what ALADIN or AROME offer, but WRF certainly includes prediction of surface and subsurface runoff (though routing is still needed, and is not included in standard WRF). Is there a reason that these predictions were passed over in favor of using HEC-HMS to generate runoff, or why the authors didn't consider regression strategies that make use of the NWP runoff fields?

Minor typos:

line 114: "monitored," not "motorized."

line 152-154: should be deleted.

 

Author Response

Reviewer 4:

The authors present a test of flash flood forecasting methods for two basins in Morocco. Their methods comparisons include combinations of three different NWP models (ALADIN, AROME, and WRF), two stream flow estimation methods (HEC-HMS and regression) and three different soil moisture products (in situ, ERA5, and ESA-CCI). This is useful work and I believe that the paper can be published after some revision

Thank you for this positive feedback

For reasons that the authors explain, the NWP experiments are conducted in reanalysis rather than forecast mode. This is fine for comparing models, but the authors should be clear that this means their results are not for true forecasts, but for NWP simulations informed by reanalysis boundary conditions. This could be addressed in the methods and/or discussion. On the point of boundary conditions, what dataset was used for lateral boundary conditions?

In fact the models have been run in conditions that mimic the forecast mode, since no data assimilation has been performed during the event. We modified the text to better clarify this point, since the use of the word “reanalysis” was indeed misleading.

The boundary conditions used for ALADIN and AROME is the ARPÈGE model (Line:139 for AROME,  Line: 140 for ALADIN and Line: for WRF). WRF model uses initial and lateral boundary conditions provided by the operational deterministic ECMWF forecasts. Further details have been provided in section 3.

NWP evaluation: the authors conclude that WRF and AROME offer better performance than ALADIN. What is the basis for this conclusion, beyond the % bias result (and the associated hit rate for an absolute threshold)?

The conclusion that we made is based on the result of the contingency table and different skill scores calculated to compare the NWP models. We confirmed our conclusion by comparing the maximum discharge forecasted using ALADIN model with the observed maximum discharge. The characteristics of ALADIN model make him less accurate than the other two models.

If the NWP models suffer from consistent, systematic bias (which is not uncommon), couldn't the outputs be subjected to a simple bias correction?

This is true. But to detect a possible systematic bias, there is a need for a larger sample of events. We would like to remind that monitoring networks are much less developed in Morocco and the data that would be required to implement a robust bias correction method does currently not exists.

I would be interested to see how well the models perform in terms of espatial distribution of precipitation in each basin (satellite data could be used for comparison if in situ observations are inadequate) and in the contingency table and RMSE for precipitation and stream flow once a basic bias adjustment is performed.

We first analyzed the spatial distribution of rainfall in the basin, with observed precipitation, ALADIN, AROME and WRF simulated precipitation. However the results were not conclusive, due to the small number of events, and the spatial variability of rainfall in the different simulations. This is why we did not add these plots in the manuscript. The use of basin-average precipitation is more robust in this context. We would like also to stress that there is small number of rain gauges and the mountainous nature of the basin makes it difficult to assess properly the spatial variability of rainfall. Also satellite products are not reliable in these basins, as shown by preliminary analysis of GPM rainfall (which is available at 10km resolution only.)

NWP computational cost: operationally, running real-time ensembles of high resolution non-hydrostatic NWP models can be computationally prohibitive. What is the relative computational cost of the three modeling methods used, and what are the implications of those costs for operationalizing a flash flood forecast system?

The ALADIN and AROME models are already running at the National Department of Meteorology of Morocco. The department is running the models every day, but they are not yet providing ensemble simulations. The method used in the present work, the hydrological model and the regression, are very low in terms of computational costs and could be implemented easily by the DMN of Morocco.

What steps were made to optimize each NWP system for this region? I am not very familiar with ALADIN or AROME, but I know that WRF offers multiple physics options. It is standard practice to evaluate different sets of physics settings when applying WRF in a new context. Were such tests performed, or did the authors have prior knowledge of optimal physics settings? And which physics settings were used in the simulations presented in the paper?

No test has been performed to optimize the WRF model configuration over Morocco. The authors have used the same WRF model configuration that it is routinely run operationally by the Meteorology Group at the University of the Balearic Islands (http://meteo.uib.es/wrf) in a neighboring geographical context. This configuration is based on previous research on similar study cases (i.e. heavy precipitation and flash-flooding over complex orography in the semi-arid Mediterranean region). The WRF model set-up was selected after evaluating multiple physics configurations. These particular physics settings exhibited a reliable performance over multiple case studies in the region (e.g. Ravazzani et al. (2016) and Amengual et al. (2017)). Further details and clarifications in this topic have been included in section 3.  

Refs.:

Ravazzani, A. Amengual, A. Ceppi, V. Homar, R. Romero, G. Lombardia, and M. Mancini, 2016: Potentialities of ensemble strategies for flood forecasting over the Milano urban area. J. Hydrol., 539, 237–253

As a major motivation for this study is operational flood warning, it will be necessary to have a system that can be run in real time. What is the data latency on the three soil moisture products tested in the regression model? Presumably only a product that has near zero latency, or very slow variability, would be useful in this context. It would be useful if the authors could comment on latency requirements and current latency reality for these products.

Thank you for this point which is important in the development of a flood forecasting system. The in-situ measurements are useful for the operational flood warning system and could be available almost in real time since the stations are using GSM networks for data transfer. We evaluate the product with different soil moisture products in order to estimate the soil moisture in a large scale and possibly usable in ungauged basins (a paper is under review in the Remote Sensing Journal on this aspect). The latency time for the ERA5 is 5 day but the in-situ measurements are the main data for operational flood warning system tested here.

We added the latency time for each soil moisture product section (ESA-CCI : section 4.1 and ERA5: section 4.2).

Were the soil moisture products only used for the regression? The use of the term "initial conditions" made me think that these products were used to initialize NWP, or at least HEC-HMS, but it seems from the methods that this was not the case. Please clarify. Also, if these products were not used to initialize NWP models and HEC-HMS, then what initial conditions did those models use?

The initial conditions are the soil moisture condition, provided by three soil moisture products (ERA5, in-situ and ESA-CCI) used into the HEC-HMS and regression models. We mean here by initial condition the antecedent soil moisture level in the hydrological approaches.

It appears that the authors make no use of runoff predicted by the land surface model component of their NWP models. Again, I do not know what ALADIN or AROME offer, but WRF certainly includes prediction of surface and subsurface runoff (though routing is still needed, and is not included in standard WRF). Is there a reason that these predictions were passed over in favor of using HEC-HMS to generate runoff, or why the authors didn't consider regression strategies that make use of the NWP runoff fields?

Yes we agree with you, the WRF model offer a land surface model (WRF-hydro) that can be used as an additional component to the WRF model. But unfortunately, the ALADIN and AROME have not this specificity and they are using a simplified ISBA land surface scheme that is not able to reproduce flash floods. Vincendon et al. (doi:10.1016/j.jhydrol.2010.04.012) introduced a modified version of this land surface scheme, but this component is not straightforward to implement in particular in the context of data scarcity. This is the reason why parsimonious approach using off-ling hydrological models or statistical models should be preferred in this context.

line 114: "monitored," not "motorized."

Thank you.

line 152-154: should be deleted.

Thank you.

Round 2

Reviewer 3 Report

Overall, the authors have addressed my concerns even though I still believe that more in-depth analysis of the flash flood events and the model physics would provide meaningful clues on the behavior of the different models and how to better set-up flash flood early warning systems. I nevertheless understand the problems faced by the authors (e.g. scarcity of data) and consider the presented study worth of publication.

However, in terms of the article structure, I would still recommend to include the description of the models in section 2 with the presentation of the study area and the data.

Author Response

Thank you very much for the positive comments. Since the authors reviewers and the editor did not request a change in the article structure, we kept it. We think it is more relevant to separate the data and method section.

Best regards Yves Tramblay

Reviewer 4 Report

The authors have addressed my concerns. I am happy to recommend the paper for publication.

Author Response

Thank you