*3.3. Model Perfroamnce with 'Good Parameter Sets'*

Figure 7 presents the selected five global parameters (CN2, GW\_DELAY, ALPHA\_BF (Base flow Alpha Factor), SOL\_AWC (Available Water Capacity in the Soil layer), and ESCO, details of parameters are in Table S1 in Supplementary Materials) and corresponding model performance for the last iteration of Hkamti station taking as an example station to discuss the variation of model performances with 'good parameter set'. *Remote Sens.* **2020**, *12*, x FOR PEER REVIEW 17 of 26 and R2) of streamflow and evapotranspiration vary within a narrow range (Figure 7), implying 'good parameter sets' can reproduce similar model results.

**Figure 7.** Box plots of selected global parameters of best 100 simulations for each calibration approach (single variable: Q and ET separately and multivariable: Q + ET) at Hkamti station (**top panel**) and the corresponding range of performance indicator values of streamflow (blue) and ET (red) (**bottom panel**). In each plot, the boxes are limited to 25th and 75th percentiles of sample and thick line shows the median value. Whiskers are extended to 1.5 times inter‐quartile range to the top and bottom of the boxes. **Figure 7.** Box plots of selected global parameters of best 100 simulations for each calibration approach (single variable: Q and ET separately and multivariable: Q + ET) at Hkamti station (**top panel**) and the corresponding range of performance indicator values of streamflow (blue) and ET (red) (**bottom panel**). In each plot, the boxes are limited to 25th and 75th percentiles of sample and thick line shows the median value. Whiskers are extended to 1.5 times inter-quartile range to the top and bottom of the boxes.

*3.4. Parameter Sensitivity and Uncertainty*  The most sensitive model parameters for streamflow and evapotranspiration were found to be different. Figure 8 shows the five most sensitive model parameters of each variable (streamflow and evapotranspiration) with respective MNSE values. GW\_DELAY, RCHRG\_DP (Deep aquifer Percolation fraction), ALPHA\_BF, SLSOIL (Slope length for lateral subsurface flow), and LAT\_TIME (Lateral flow travel time) are identified as the five most sensitive parameters for streamflow, whereas same parameters for evapotranspiration are SOL\_BD (Moist bulk density of soil layer), SOL\_Z (Depth from soil surface to bottom of layer), ESCO, EPCO, and SOL\_AWC (refer Table S1 for definitions of parameters). Variations of MNSE for evapotranspiration‐based calibration was found to be less than that for streamflow based calibration. In the multivariable calibration, weighted average MNSE In single variable calibration, four parameters show high variability for calibration with streamflow than with evapotranspiration, only ALPHA\_BF shows high variability for calibration with evapotranspiration. Among these five parameters, CN2, GW\_DELAY and ALPHA\_BF are more sensitive to streamflow, whereas SOL\_AWC and ESCO are more sensitive to evapotranspiration. In contrast, model performances show low variability except for performances of streamflow under calibration only with evapotranspiration (Figure 7). This implies that a range of parameter sets (100 sets considered here) can produce similar model performances of streamflow and evapotranspiration when those variables are calibrated separately. Therefore, these parameter ranges are very likely to show a similar level of performance with a dataset not used in calibration (i.e., validation dataset), unless inter/ intra-annual variability of the new dataset is considerably higher than calibration dataset.

values show less variation than individual streamflow and evapotranspiration calibrations. A considerable variation of parameter values are shown for ALPHA\_BF and SOL\_BD in evapotranspiration based model calibration. Results also indicate that, except SLSOL (subsurface lateral flow) and SOL\_Z (depth of soil layer from surface), final parameter ranges in the last iteration In multivariable calibration, all the selected five global parameters show high variabilities (based on 100 'good parameter sets' considered) compared to the variabilities associated with the single-variable calibration (Figure 7). These best performed simulations have resulted from different combinations of

Except for SLSOL, all the above identified sensitive parameters of streamflow are mostly related to the base flow (or groundwater flow) SLSOL parameter is related to the subsurface lateral flow. In calibration parameter sets (100 'good parameter sets'), in which parameter value vary within a high range. Similar to single variable calibration, model performances (NSE, PBIAS, and R<sup>2</sup> ) of streamflow and evapotranspiration vary within a narrow range (Figure 7), implying 'good parameter sets' can reproduce similar model results.

#### *3.4. Parameter Sensitivity and Uncertainty*

The most sensitive model parameters for streamflow and evapotranspiration were found to be different. Figure 8 shows the five most sensitive model parameters of each variable (streamflow and evapotranspiration) with respective MNSE values. GW\_DELAY, RCHRG\_DP (Deep aquifer Percolation fraction), ALPHA\_BF, SLSOIL (Slope length for lateral subsurface flow), and LAT\_TIME (Lateral flow travel time) are identified as the five most sensitive parameters for streamflow, whereas same parameters for evapotranspiration are SOL\_BD (Moist bulk density of soil layer), SOL\_Z (Depth from soil surface to bottom of layer), ESCO, EPCO, and SOL\_AWC (refer Table S1 for definitions of parameters). Variations of MNSE for evapotranspiration-based calibration was found to be less than that for streamflow based calibration. In the multivariable calibration, weighted average MNSE values show less variation than individual streamflow and evapotranspiration calibrations. A considerable variation of parameter values are shown for ALPHA\_BF and SOL\_BD in evapotranspiration based model calibration. Results also indicate that, except SLSOL (subsurface lateral flow) and SOL\_Z (depth of soil layer from surface), final parameter ranges in the last iteration of multivariable calibration are within the ranges of single variable calibration. The above two parameters directly correspond to changes in soil water balance.

Except for SLSOL, all the above identified sensitive parameters of streamflow are mostly related to the base flow (or groundwater flow) SLSOL parameter is related to the subsurface lateral flow. In model calibration with evapotranspiration alone, all the sensitive parameters are related to soil properties except for EPCO, which is the plant uptake compensation factor [61]. Parameter values resulting in a reasonable MNSE (> 0.5) are scattered within their initial ranges (for initial ranges, please refer to Table S1 in Supplementary Material). This scattered parameter values indicate that there are more than one combination of parameter values that can reproduce similar outputs.

For the multivariable calibration, each of the model parameter uncertainty range was examined in terms of normalized uncertainty scores (Equation (2)) that range from 0 to 100 (Figure 9). Uncertainty scores presented here were calculated from the last iteration of multivariable calibration. The analysis shows that uncertainty scores vary greatly among different parameters and stations for the same parameter. Parameters which are most sensitive to streamflow show higher uncertainty than the parameters sensitive to evapotranspiration except for SOL\_BD. However, among the five most sensitive parameters for streamflow, SLSOIL and LAT\_TIME show considerably less uncertainty at all the stations. In contrast, only ESCO, one of the most five sensitive parameters for evapotranspiration, shows considerably less uncertainty. Furthermore, uncertainty ranges for SOL\_AWC are minimal at Homalin and Kalewa stations, whereas their maximum variations are found at Hkamti and Monywa stations.

model calibration with evapotranspiration alone, all the sensitive parameters are related to soil properties except for EPCO, which is the plant uptake compensation factor [61]. Parameter values resulting in a reasonable MNSE (> 0.5) are scattered within their initial ranges (for initial ranges,

there are more than one combination of parameter values that can reproduce similar outputs.

**Figure 8.** Parameter values or absolute changes versus objective function MNSE for each iteration with 2000 simulations for calibration with a single variable; blue dots represent calibration based on streamflow, red dots represent calibration based on evapotranspiration, and black dots for multivariable calibration. Y‐axis is a MNSE value ranging from 0 to 1 and the X‐axis represents the changes of parameter value or absolute change of a corresponding parameter. 'v\_' denotes **Figure 8.** Parameter values or absolute changes versus objective function MNSE for each iteration with 2000 simulations for calibration with a single variable; blue dots represent calibration based on streamflow, red dots represent calibration based on evapotranspiration, and black dots for multivariable calibration. Y-axis is a MNSE value ranging from 0 to 1 and the X-axis represents the changes of parameter value or absolute change of a corresponding parameter. 'v\_' denotes replacement of the existing parameter value (e.g., v\_GW\_DELAY.gw), 'a\_' denotes adding a fix value to existing value (e.g., a\_SOL\_BD.sol). The first five parameters correspond to streamflow and the last five to evapotranspiration.

found at Hkamti and Monywa stations.

five to evapotranspiration.

replacement of the existing parameter value (e.g., v\_GW\_DELAY.gw), 'a\_' denotes adding a fix value to existing value (e.g., a\_SOL\_BD.sol). The first five parameters correspond to streamflow and the last

For the multivariable calibration, each of the model parameter uncertainty range was examined in terms of normalized uncertainty scores (Equation (2)) that range from 0 to 100 (Figure 9). Uncertainty scores presented here were calculated from the last iteration of multivariable calibration. The analysis shows that uncertainty scores vary greatly among different parameters and stations for the same parameter. Parameters which are most sensitive to streamflow show higher uncertainty than the parameters sensitive to evapotranspiration except for SOL\_BD. However, among the five most sensitive parameters for streamflow, SLSOIL and LAT\_TIME show considerably less uncertainty at all the stations. In contrast, only ESCO, one of the most five sensitive parameters for evapotranspiration, shows considerably less uncertainty. Furthermore, uncertainty ranges for

**Figure 9.** Normalized uncertainty scores of selected parameters out of 22 parameters used. Each box plot contains parameter values corresponding to 2000 simulations for the last iteration of **Figure 9.** Normalized uncertainty scores of selected parameters out of 22 parameters used. Each box plot contains parameter values corresponding to 2000 simulations for the last iteration of multivariable calibration.

#### multivariable calibration. **4. Discussion**

Results presented in this study are another application among a limited number of studies that have used GLEAM ET data to support hydrological model calibration in the data-scarce Chindwin River Basin in Myanmar. In general, under the three calibration approaches used, streamflow shows the best and worst performance in calibration only with streamflow and evapotranspiration, respectively, and vice versa in the calibration performance for evapotranspiration. Both variables show slightly lower performance in multivariable calibration than the individual calibrations with streamflow and evapotranspiration. In some of the sub-basins (e.g., sub-basins 2 to 4), simulated ET does not properly represent the GLEAM-ET during dry seasons (Figure 6b) under all three calibration approaches. In contrast, sub-basins 6, 8, and 9 overestimate ET during high flow periods (June to September) for the calibrations with streamflow and multivariable calibration. These differences can be attributed to the precipitation over the basins. Similar findings were obtained in a study on the Karkheh River Basin in Iran by Rientjes et al. (2013) [43]. They found that streamflow and ET obtain good performances when the variables are calibrated separately. However, for model calibration with both streamflow and ET,

simulated ETs of most of the sub-basins show good agreement with SEBS-ET (satellite-based surface energy balance system – actual evapotranspiration). Immerzeel and Droogers (2008) [40] also could not find any improvement in streamflow simulation under the model calibration with satellite-based evapotranspiration from time series of MODIS images of the Upper Bhima catchment in south India. López et al. (2017) [42] also found that model calibration with GLEAM ET and soil moisture data shows a reasonable streamflow estimation (with NSE values varying from 0.5 to 0.75). However, better model performance was obtained when it was calibrated with in-situ streamflow data.

Tobin and Bennett (2017) [44] concluded that although there is no performance improvement in streamflow simulation, GLEAM ET data can be used to constrain the evapotranspiration parameters in the SWAT applications. They considered 16 major SWAT parameters, in which five are attributable to evapotranspiration (ESCO, EPCO, CANMX (Maximum canopy storage), GW\_REVAP (Groundwater "revap" coefficient), and REVAPMN (Threshold water depth in shallow aquifer for "revap" or percolation to occur), refer to Table S1 for details). Here, ESCO and CANMX were identified as the most sensitive parameters for model calibration. Immerzeel and Droogers (2008) [40] found that actual ET is more sensitive to the groundwater (GW\_REVAP) and meteorological (monthly rainfall increment—-RFINC) parameters than soil (SOL\_AWC) and land-use parameters (maximum plant leaf area index—-BLAI). In our analysis, SOL\_BD, SOL\_Z, ESCO, EPCO, and SOL\_AWC are found to be the most sensitive parameters for evapotranspiration. The analysis of 'good parameter sets' and resultant model performances imply that many combinations of model parameters can reproduce the streamflow and evapotranspiration comparable to same obtained from 'the best parameter set'. Furthermore, Winsemius et al. (2009) [1] suggested that satellite-based ET and soil moisture information are required in hydrological model calibration to reduce parameter uncertainties and constrain model parameters within physically realistic ranges.

Even though we discussed only the parameter uncertainty, there are other sources of uncertainty in hydrological modelling, such as model input data, model structure, and observed data used for calibration [75,76]. Precipitation, the primary input data used in this analysis was derived from interpolated gauge data of six stations, which are not well distributed over the basin. Therefore, we believe that precipitation data contributes to prediction uncertainty. In terms of observed streamflow data, Harmel et al. (2009) [78] stated that the typical uncertainty in measured streamflow varies by ±7 − 23%. Di Baldassarre and Montanari (2009) [79] stated that the average error of measured streamflow data was 25.6% for the Po River, Italy. With respect to ET, the accuracy of remote sensing-based products may vary significantly over the different regions due to climatic variability, topography, and land cover [47]. For example, Trambauer et al. (2014) [3] have compared eight different ET products over the African continent and summarized that some products show good consistency among them in some areas and diverge in other areas of the continent. Another analysis by Tobin and Bennett (2017) [44] showed that simulated ET matches with GLEAM-TMPA (GLEAM - Tropical Rainfall Measurement (TRMM) Mission Multi-satellite Precipitation Analysis) better than GLEAM-CMORPH (GLEAM - Climate Prediction Center MORPHing technique). Therefore, these RS-based ET products may possess considerable biases. Thus, further information is required to analyze and discuss the uncertainty in model outputs.

#### **5. Conclusions**

This study evaluated the use of measured streamflow and RS-based ET data to calibrate a hydrological model of the Chindwin Basin, Myanmar. The model was developed with SWAT and two calibration approaches were tested: single variable and multiple variable calibration, and identified and discussed 'good parameter sets', which can produce similar or comparable results that obtained from 'the best parameter set'. RS-based ET data used in the model calibration were obtained from the Global Land Evaporation: Amsterdam Model (GLEAM ET), and measured streamflow were obtained at four gauging stations in the basin.

In general, this study indicates that GLEAM ET data, together with streamflow data, offers good potential for hydrological model calibration in the study region as the simulation results show a good performance for streamflow (with an NSE > 0.85 on monthly time series), while maintaining a reasonable performance for evapotranspiration (with an NSE > 0.61). Moreover, the results of single variable calibration with GLEAM ET indicate that even in the absence of streamflow data (i.e., ungauged basin), the model would have produced streamflow NSE values of more than 0.69 for three out of four stations with PBIAS varying between −2.6% and −23%. The only exception to the above behavior was found at the Hkamti station with NSE of 0.16 and PBIAS of −22%. It is noted that the Hkamti station is the uppermost station of the four and hence its calibration is independent of the other three stations. In contrast, the other three stations (Homalin, Kalewa, and Monywa) are all downstream of Hkamti and in series. Thus, their calibration results are influenced by the calibration results of up-stream station(s).

Results also indicate that there can be many different sets of parameter values ('good parameter sets') can produce similar results, which can be obtained from 'the best parameter set'. The analysis of calibration parameters suggests that the parameter sensitivity and their values change among different calibration set-ups, and uncertainty ranges of parameters may vary among both different parameters and stations.

This study provides valuable insights on hydrological modelling, in the context of using remote-sensing based and multiple data sources for model calibration, which are particularly useful for data poor basins. There is, however, room for further research on multivariable calibration in distributed hydrological modelling with the use of globally available datasets such as evapotranspiration, snow cover, and soil moisture data together with traditionally used streamflow data. Such a calibration approach could lead to better representation of hydrological responses of particularly, ungauged basins, as this approach would enable reasonable parameter estimation across the basin.

**Supplementary Materials:** The following are available online at http://www.mdpi.com/2072-4292/12/22/3768/s1. Table S1: Hydrological parameters used in model calibration.

**Author Contributions:** T.A.J.G.S. carried out the design, model simulations, analysis of results, and drafting the article. S.M. provided the guidance on methodology and comments on the manuscript. R.R. provided critical review for the manuscript. All authors have read and agreed to the published version of the manuscript.

**Funding:** This study is a part of TAJGS's Ph.D research, which is supported by the EPP Myanmar project and Netherlands Fellowship Programme (NFP).

**Acknowledgments:** TAJGS is supported by the EPP Myanmar project and Netherlands Fellowship Programme (NFP). RR is supported by the AXA Research Fund and Deltares Strategic Research Programme "Coastal and Offshore Engineering.". The SWAT simulations were carried out on the Dutch national e-infrastructure with the support of SURF Cooperative.

**Conflicts of Interest:** The authors declare no conflict of interest.

## **References**


**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

MDPI St. Alban-Anlage 66 4052 Basel Switzerland Tel. +41 61 683 77 34 Fax +41 61 302 89 18 www.mdpi.com

*Remote Sensing* Editorial Office E-mail: remotesensing@mdpi.com www.mdpi.com/journal/remotesensing

MDPI St. Alban-Anlage 66 4052 Basel Switzerland

Tel: +41 61 683 77 34 Fax: +41 61 302 89 18

www.mdpi.com ISBN 978-3-0365-2700-0