Evaluation of Convective Environments in the NARCliM Regional Climate Modeling System for Australia

Abstract

Severe thunderstorms lead to multiple hazards including torrential precipitation, flash flood, hail, lightning, and wind gusts. The meso- to micro-scale nature of thunderstorms impose great challenges from understanding individual storm dynamics, storm climatology as well as projecting their future activities. High-resolution regional climate models can resolve the convective environments better than global models. Australia, especially the east and southeast parts of the continent, is a global hot spot for severe thunderstorms. This study evaluates the simulated convective environments from the New South Wales (NSW) and Australian Regional Climate Modelling (NARCliM) project based on the parameters of CAPE, CIN, 0–6-km vertical wind shear and storm severity. The ensemble regional downscaling is compared against the fifth-generation European Centre for Medium-range Weather Forecast Reanalysis (ERA5). The results show that although there are apparent biases (generally positive for CAPE and negative for CIN, and slightly overestimated vertical wind shear) in the downscaled storm parameters, the climatology of measures of storm severity over land, including their spatial patterns and seasonality, agree well with ERA5. These results have strong implication on the application of the climate projection to assess future convective environments in the region.

1. Introduction

Severe thunderstorms lead to multiple hazards, including torrential precipitation, flash floods, hail, lightning, gusts and tornadoes. These hazards not only lead to some of the largest financial losses (e.g., [1] for Australia), but often they are also life threatening (e.g., [2,3]). Being meso-scale and micro-scale weather systems, severe thunderstorms and their associated impacts are often not well observed and possess low predictability. Therefore, establishing reliable thunderstorm climatology and understanding its connection to natural climate variability is challenging [4,5,6]. For example, based on about four decades of reliable observations of convective environments, various trends for different categories of thunderstorms have been reported globally [7]. From an observational perspective, one effective way is to link local thunderstorm activity with large-scale factors of climate variability, such as the El Niño Southern Oscillation among the others [8].

Severe thunderstorms commonly occur in the Australian region [9], in which the favorable environments for convective storms (instability, moisture and lifting mechanisms) [10] occur. The Australian Bureau of Meteorology (BoM) maintains an official database of thunderstorm activity based on in-situ observations and reports. There are basically two categories of thunderstorms: severe thunderstorms are associated with one or more of extreme rainfall leading to flash flood, hail greater than 2 cm, wind gusts in excess of 90 km h⁻¹ and tornadoes, while significant severe thunderstorms produce one or more of hails with size 5 cm or greater, wind gusts exceeding 120 km h⁻¹ and tornadoes exceeding the Fujita F2 intensity [9]. Spatially, severe thunderstorm activity is concentrated in northern Australia (N Australia hereafter), eastern and southeastern Australia (E and SE Australia hereafter), because these regions have access to moisture and convective instability from the tropics, as well as vertical wind shear (VWS) within the lower atmosphere according to the ingredients-based methodology of studying thunderstorms [11]. The climatological distribution of lightning agrees with that of thunderstorm activity [12,13]. On the other hand, hail occurrence is concentrated more at E and SE Australia [14], while tornadoes also generate over southwestern Australia [9].

The small-scale nature of thunderstorms means that they are not explicitly resolved in global climate models (GCMs), and thus studying the climate variability and change in thunderstorms based on GCMs is not directly possible. Although as discussed above there are linkages between thunderstorm activity and large-scale climate variability, the inadequately simulated local convective activities in GCMs do not allow us to establish robust statistical relationships and/or perform process-oriented diagnoses [15]. To bridge this research gap, high-resolution regional models with convection-permitting or convection-resolving ability are necessary [16]. There have not been many modeling studies of thunderstorms in Australia, some of them focused on high-impact hailstorm cases. For example, Buckley et al. [17] simulated the 1999 Sydney hailstorm with a 1-km resolution model. In a climate modeling setting with downscaling to 1 km, Leslie et al. [18] estimated future hailstorm activity in the Sydney basin. More recently, Hartigan et al. [19] simulated a tornadic storm near Sydney using a 500-m resolution WRF model. While these are all case studies, [20] examined the performance of two climate model in simulating the thunderstorm environments in Australia. It was found that the climate models explained at least 75% in the spatial variance of thunderstorm environments, however, one of the models did not resolve the annual cycle nor the diurnal cycle well. Thus, evaluation of some more recent modeling systems would provide us information on their skills in reproducing the thunderstorm climatology, such as the focuses of activity at N, E and SE Australia especially during summer as aforementioned. Ideally, simulations from high-resolution regional models would cover a long period for evaluation studies and in an ensemble setting such that uncertainties in model physics can be reduced.

This study evaluates the simulated convective environments in the New South Wales (NSW) and Australian Regional Climate Modelling (NARCliM) system. The objective is to identify the biases in the simulations in terms of the major ingredients for thunderstorm development. Based on this evaluation, projection of future convective storm activity in the Australian region can then be robustly performed. The structure of this paper is as follows. Section 2 briefly describes the NARCliM system including the two generations of it applied in this study. Section 3 documents the reference reanalysis dataset, the storm parameters under consideration and the evaluation metrics. In Section 4, we present the evaluation results on the convective environments in NARCliM in different aspects. Since we mainly present results from the ensemble means, we briefly discuss the individual contributions from the GCMs and regional climate models (RCMs) in Section 5. An overall summary with further discussion is found in Section 6.

2. The NARCliM Regional Downscaling System

This study uses downscaled climate simulations from the NARCliM project, which was designed to provide plausible future climate conditions for SE Australia [21]. The NARCliM project has delivered two generations of regional climate data (NARCliM1.0 and NARCliM1.5) based on two generations of the Intergovernmental Panel for Climate Change (IPCC) Climate Modelling Intercomparison Project (CMIP3 and CMIP5).

NARCliM1.0 and NARCliM1.5 (hereafter N1.0 and N1.5) are available at two resolutions: 50-kilometre for the CORDEX Australasia domain and 10-kilometre for the SE Australia NARCliM domain) (Figure 1). Both generations have a period of reanalysis-driven simulations and a common period of GCM-driven simulations. The two NARCliM datasets thus form a good basis to evaluate the simulated convective environment by reanalysis and to compare the impacts from using different GCMs to drive the RCMs.

N1.0 is an ensemble of 12 GCM/RCM combinations [21]. The four GCMs (MIROC3.2, ECHAM5, CGCM3.1 and CSIRO-MK3.0) were selected from the CMIP3 ensemble based on their model performance over Australasia and independence of their errors, and to span the full range of potential future changes in precipitation and temperature over SE Australia. Each of these four GCMs was dynamically downscaled using three selected RCMs (R1, R2 and R3), which consist of distinct combinations of physics schemes of the Weather Research and Forecasting (WRF) V3.3 model. The detailed physics schemes, including convection, planetary boundary layer and land surface, can be found in [20] and we have also listed them as

Table A1. The model configuration for the three independent RCMs. Here, the planetary boundary layer schemes include the Yonsei University (YSU) and the Mellor-Yamada-Janjic (MYJ) schemes. The cumulus physics schemes include the Kain-Fritsch (KF) and the Betts-Miller-Janiac (BMJ) parameterization. The radiation schemes include the Dudhia shortwave scheme, Rapid Radiative Transfer Model (RRTM) longwave scheme and the Community Atmospheric Model (CAM) radiation scheme. The cloud microphysics applies the double-moment five-class (WDM5) scheme.

NARCliM Ensemble Member	Planetary Boundary Layer Physics	Cumulus Physics	Surface Layer Physics	Cloud Microphysics	Shortwave/Longwave Radiation Physics
R1	MYJ	KF	Eta similarity	WDM 5 class	Dudhia/RRTM
R2	MYJ	BMJ	Eta similarity	WDM 5 class	Dudhia/RRTM
R3	YSU	KF	MM5 similarity	WDM 5 class	CAM/CAM

in Appendix A. All RCM simulations were run for three 20-year periods: the recent past (1990–2009), near future (2020–2039) and far future (2060–2079), at 10-km resolution over Southeast Australia, embedded within the 50-km resolution CORDEX domain, under the SRES A2 scenario [22] which reflects the ‘business-as-usual’ scenario in CMIP3. The three selected RCMs were also run by using the National Centers for Environmental Prediction (NCEP) reanalysis dataset [23] for 1950–2009 to assess the capability of RCMs to simulate the observed regional climate.

N1.5 consists of three CMIP5 GCMs (ACCESS1.0, ACCESS1.3 and CanESM2) downscaled using two WRF RCMs used in N1.0 (R1 and R2). under two future emission scenarios [24]. CMIP5 uses emissions scenarios called ‘representative concentration pathways’ (RCPs) and N1.5 uses both RCP4.5 and RCP8.5. These scenarios reflect a medium level of mitigation and a business-as-usual approach, respectively. RCP8.5 is most comparable to the SRES A2 scenario used in N1.0. Each N1.5 simulation was run from 1950 to 2100 continuously using the WRF V3.6 model. This was a newer version of WRF in N1.5 (WRF3.6) as compared to N1.0, however, the same RCM physics configuration has been applied [24]. During the development of N1.0 and N1.5, the latest version of the WRF model has been applied and due to consistency, the model version has been kept the same during the production period of the ensemble systems. The 150-year GCM driven simulations are also accompanied by ERA-Interim reanalysis [25] forced simulations for 1979–2013. N1.5 is the latest downscaled climate simulations available in Australasia. In the latest generation of NARCliM, named N2.0, CMIP6 GCMs are being downscaled and also accompanied by a set of ERA5-driven historical simulations (see Section 6.2). Therefore, N2.0 would represent improvements in data and model resolution, quality and model physics. However, due to limited availability of three-dimensional data from GCMs, computation of convective indices directly from them is not ideal. Dynamical downscaling provides finer and more accurate data than the driving GCMs.

Since its first delivery in 2014, several studies have evaluated the N1.0 historical simulations [26,27] and indicated that N1.0 can simulate the observed climate well, even if most of the simulations have wet and cold biases. Further, evaluation results for N1.5 indicated that the new generation performed substantially better than N1.0 in capturing seasonal pattens and magnitudes of precipitation, while its skills for simulating maximum and minimum temperatures were similar to that of N1.0 [24,28]. Because both N1.0 and N1.5 run on the same domains, they provide a complementary and expanded dataset. Users can conveniently use either or both the N1.0 and N1.5 dataset without consideration of the impacts from the change in the boundary and/or regional model physics.

3. Evaluation Methodology

3.1. Convective Parameters

Thunderstorms are a local convection event that depends on the thermodynamic profile of the troposphere. That is, thunderstorm development potential is determined by the vertical variation of the air temperature and moisture (measured by the dew point temperature). The initial development of a thunderstorm is related to the altitude of the lifting condensation level (LCL) and the level of free convection (LFC) [29]. The convection energy in the thunderstorm comes from the convective available potential energy (CAPE) stored in the atmosphere [30]. On the other hand, an air parcel must overcome the negative energy, which is between the LCL and LFC, by an external lifting mechanism to become buoyant. Such negative energy is named the convective inhibition number (CIN) [31]. Thus, thunderstorm development is under the competing effect between CAPE and CIN.

The values of CAPE and CIN are sensitive to the choice of air parcel to lift [32]. Air parcel may be surface-based or based on average temperature and humidity within the mixed layer, which would result in a value of CAPE named the mixed-layer CAPE (MLCAPE). MLCAPE has been quite extensively applied in thunderstorm studies (e.g., [33]). The mixed-layer depth of 100 hPa (~1000 m) is usually used, however, in Australian studies a thinner layer of 50 hPa (~500 m) is often applied [34,35,36] due to shallower moisture reservoirs. There is also the most unstable CAPE (MUCAPE) that identifies the most unstable parcel. Algorithms computing the MUCAPE usually identifies the level with the maximum equivalent potential temperature in the first 3 km above ground, and then lift a thin layer (e.g., 500 m) centered on that level (Software platforms such as the National Center for Atmospheric Research (NCAR) Command Language (NCL) and the WRF-Python have implemented this algorithm based on [37,38], in which the term Maximum CAPE (MCAPE) was used. NCAR’s latest GeoCAT software, https://geocat.ucar.edu/ (accessed on 19 February 2023) on the other hand, implements the surface-based CAPE following the Python package MetPy. The NCL algorithm was applied in this study.). For the purpose of matching with the reference reanalysis dataset (Section 3.2) and for identifying the maximum impacts in our later analysis of climate projection, MUCAPE and the associated condensation levels and CIN are the main thermodynamic indices examined in this study.

The local wind circulation is also important to thunderstorm development. Studies have identified the difference of the winds between surface and the mid-troposphere (i.e., the vertical wind shear, hereafter VWS) as critical. This is because VWS enables a slightly slantwise convection configuration, followed by a better inflow of moist air to support the convection and better development of the hydrometeors (rain, graupel, hail) in the storm cloud. In thunderstorm studies, it is most common to examine the VWS magnitude (based on both zonal and meridional components) between surface and 6 km (termed S06; [34,35,39]) as a storm index, although winds at a lower altitude (e.g., between surface and 1 or 3 km; [36]) are also considered in some studies. There are 29 (unstaggered) vertical levels in N1.0 and N1.5, and linear interpolation (Algorithm is based on the NCL wrf_user_interp3d subroutine.) of the zonal and meridional winds from the model levels to 6 km is performed.

Among the storm parameters mentioned above, studies have widely identified the combined influence from MUCAPE (representing the thermodynamic environment) and S06 (representing the kinematic environment) as determinants of storm severity [40,41], which measures rain intensity, wind speed and in the case of hail production the hail size. A discriminant has been developed that combines these two parameters in the form of MUCAPE·S06^1.67 (termed TSTORM). In the space spanned by MUCAPE and S06, a threshold of TSTORM is thus a discriminant line separating the less and more convective environments. Values of the threshold and the exponent of S06 have been developed for thunderstorms in different regions of the world based on observed storm data [34]. After extensive testing based on the Australian thunderstorms in the past and suitability to compute the parameters from typical resolutions of climate models, [34,35] determined that the threshold of TSTORM = 25,000 (68,000) is able to roughly distinguish the environment for developing severe thunderstorm (significant severe thunderstorm) according to the BoM definition (Severe Thunderstorms, at http://www.bom.gov.au/weather-services/severe-weather-knowledge-centre/severethunder.shtml, Bureau of Meteorology, Australian Government (last accessed on 19 February 2023)) we discussed in the introduction.

Certainly, there are many other environmental indicators of convective storm development (Thompson, R.L., Explanation of SPC Severe Weather Parameters, Storm Prediction Center, National Oceanic and Atmospheric Administration, U.S., at https://www.spc.noaa.gov/exper/mesoanalysis/help/begin.html (last accessed on 19 February 2023)). For example, even only based on radiosondes observations, tens of parameters under the categories of parcel, moisture and temperature parameters can be computed [33]. There are also some other indices that consider some unique aspects of storm, such as moisture flux convergence [42] and storm relative helicity (e.g., [39,43]). In this study, we have analyzed the most common ones in the literature of climatology and climate change studies (e.g, [16,20,44]). For precipitation including the extremes, to which thunderstorms contribute substantially, evaluation has been performed in [26] for N1.0 and [24,28] for N1.5.

The N1.0 and N1.5 model outputs are all 3-hourly for most of the 3-dimensional variables. The convective parameters we examine here possess clear diurnal variation. In order to examine the largest potential of thunderstorm development, we identify the daily maximum of MUCAPE before its seasonal and annual mean are generated. For CIN, we also identify its daily maximum to analyze the largest inhibition factor. When evaluating S06 only its daily mean is considered, while for TSTORM the daily maximum of the combined storm severity factor is computed.

3.2. Reanalysis Dataset

Although upper-level observations (i.e., radiosonde soundings) are the baseline references for model validation of the convective parameters, they are highly inhomogeneous in both space and time, and thus unable to evaluate the spatial patterns of the simulations. Thus, the fifth generation of atmospheric reanalysis from the European Centre for Medium Weather Forecast (ERA5; [45]) is used as the observational dataset for evaluation. The first phase of ERA5 started from 1979 up to the present, which has well covered the evaluation period (1990–2009) common to both N1.0 and N1.5.

During the time of performing this study, we have not obtained access to the model-level dataset of ERA5, which would allow us to compute the convective parameters using the same algorithms as have been applied to NARCliM. For this reason, we used MUCAPE and CIN parameters provided by ERA5 in their single-level catalogue. The dataset is available from the Copernicus Climate Data Services (See https://cds.climate.copernicus.eu/cdsapp#!/dataset/reanalysis-era5-single-levels?tab=overview (last accessed on 19 February 2023)). The horizontal resolution is 0.25° × 0.25° in latitude and longitude, and the temporal resolution is hourly.

Note that in ERA5, CAPE is calculated by considering parcels of air departing at different model levels below the 350-hPa level. Then the maximum CAPE produced by the different parcels is the value retained. Thus, the search for the most unstable parcel is within a larger depth in ERA5 than the algorithm we applied to NARCliM. However, unless there are elevated lifting mechanisms higher than 3 km, the two algorithms should produce similar MUCAPE value. For computation of the S06, both the surface and pressure-level winds are needed. At the surface, the 10-m zonal and meridional winds are available from ERA5. The upper-level winds are available on 37 pressure levels. A vertical interpolation is performed to obtain the winds at 6 km in altitude.

The above ERA5 variables, which are on a global latitude/longitude grid, are regridded using the bilinear algorithm in the Climate Data Operator to the CORDEX and NARCliM domains of N1.0 and N1.5 for comparison. Daily maximum and minimum values of the parameters are also extracted as for the corresponding NARCliM variables.

3.3. Evaluation Metrics

For the variables available in the ERA5 (MUCAPE, CIN, S06, TSTORM), bias analysis of the N1.0 and N1.5 is performed. In addition, we also compute the root-mean-square error (RMSE) and pattern correlation (PCorr) between ERA5 and simulations. Since thunderstorm development over land is the main concern, we compute these error measures for the land grid points only. We also examine the climatology of LCL and LFC as simulated in the two generations of NARCliM. Other than the average convective environments in the simulations, the extremes in the distributions of MUCAPE and TSTORM would determine the occurrence of impactful thunderstorms. Therefore, the extreme values of the convective parameters are also compared with those in the reanalysis dataset.

While CIN represents the inhibition effect to thunderstorm development, the relative occurrence frequency of high values of CIN with respect to those of MUCAPE would be informative in the general convective environment in a region and the quality of the NARCliM simulations. Accordingly, the co-distributions of MUCAPE and CIN are analyzed. On the other hand, MUCAPE and S06 contribute cooperatively to the storm severity and thus their relative occurrence frequency would also determine the convection characteristics of a region. Thus, the MUCAPE-S06 co-distributions of the NARCliM simulations are also evaluated against ERA5.

Given that there are determined thresholds of TSTORM that have been demonstrated to indicate the potential of severe thunderstorm (and the significant category) occurrence well, it is important to identify how many thunderstorm days have been simulated in N1.0 and N1.5 versus the statistics from ERA5. Such information is indicative of whether the NARCliM simulations are too strong or weak in the convective environment.

Climatologically, most of the severe thunderstorms and associated impacts occur during summer in the Australian region. Earlier studies such as [44] indicated that the convective season may be lengthened in the future warm climate. Thus, we examine the seasonal cycle of the identified thunderstorm days in N1.0 and N1.5. The biases against the seasonal cycle in ERA5 would form a baseline when the NARCliM ensembles are applied to project thunderstorm activities under climate change. Although it is not expected that the CMIP3 and CMIP5 models would simulate the interannual variability of convection activities well [46], we also evaluate the interannual variability of the thunderstorm days in N1.0 and N1.5 and examine whether there are similar trends as in the reanalysis.

The interannual variability score (IVS; [47]) has also been applied to evaluate the agreement of a variable with respect to the reference. The IVS is defined as

$$IVS={\left(\frac{{STD}_m}{{STD}_o}-\frac{{STD}_o}{{STD}_m}\right)}^2$$

(1)

where STD stands for standard deviation of a yearly time series and the subscript m/o refers to model and observation, respectively. In other words, an ideal time series would have IVS zero. Given that the index is always positive if not zero, the larger the IVS the model’s interannual variability deviates more from observation. To apply IVS on our convective parameters, the domain means are first computed and then the yearly time series in the reference period are applied.

The evaluation period is 1990–2009, which is the common period for N1.0 and N1.5. In the following, mainly the evaluation of the N1.0 and N1.5 ensemble means are presented. Individual GCMs and RCMs have contributed with different degrees to the ensemble means. Such within-ensemble variability will be briefly discussed in Section 5, with support from the associated appendix. In the following evaluation results, the annual climatology and bias are first presented followed by brief discussion on seasonal variations (September–November or SON, December–February or DJF, March–May or MAM, June–August or JJA). For discussion on the convective environment, extremes and storm days, we focus on summer season (DJF) in which most thunderstorms occur.

4. Evaluation Results

4.1. Climatology and Bias Analysis

Since LCL and LFC are not available from ERA5 for comparison, only the simulated climatology from N1.0 and N1.5 are discussed here. The climatological mean of the annual LCL and LFC simulated in NARCliM have their highest values over the Australian continent and decrease towards the coasts and ocean surface (Figure A1 and Figure A2). Such spatial distribution is related to the hot and dry surface conditions over land. In other words, the lower LCL and LFC values at the coastal regions are more conductive for thunderstorm development. In the CORDEX domain (d01 hereafter), the noticeable difference between the two ensembles is that N1.5 simulates higher LCL and LFC over western Australia, resulting in a two-peak pattern in N1.5. Even for the peak to the east, that in N1.0 has lower value of LCL and LFC. Moreover, the peaks of LFC are to the south of those of LCL. Such difference between N1.0 and N1.5 may be due to the hotter and dryer GCMs for driving the RCMs in N1.5 compared to those in N1.0 [21].

There is small seasonal variation of LCL over land in terms of both magnitude and spatial pattern (not shown). During spring (SON) and summer (DJF), the LCL over the Southern Ocean in both ensemble means are higher than in the other seasons. Comparatively, LFC has larger seasonal variation. The LFC is generally lower during autumn (MAM) and winter (JJA). During autumn, the spatial pattern is similar to that of the annual mean that has two peaks over the Australian continent. During winter, the high LFC values shift to north and northwest Australia in both ensembles.

The observed annual climatology of MUCAPE in d01 has the highest values over the tropical ocean surface and the coasts, while decreasing towards the latitudes to the south and have low values in the Southern Ocean (There are a few spots of high CAPE values in ERA5 over central and southern Australia. These high values have been identified in the original data from ERA5 before the regridding was performed. They may be due to the lakes in South Australia that have been resolved in ERA5’s landuse, however this must be further confirmed. These peaks are thus ignored in the remaining discussion.) (Figure 2a). N1.0 and N1.5 both simulate more extensive areas of high MUCAPE over the tropical ocean and higher peak values. Thus, the largest positive biases are over the tropical ocean and remain positive throughout the domain. N1.0 simulates slightly higher MUCAPE over the Southern Ocean than ERA5, while N1.5 is higher than N1.0 in the same region (Figure 2b–e). However, if the focus is over land, the overall pattern in both N1.0 and N1.5 are close to that in the reanalysis and also the biases are much smaller than those over the oceans.

Examining the individual seasons, the bias patterns over the CORDEX domain during SON, DJF and MAM are similar to that annually (Figure A3, Figure A4 and Figure A5), except that some negative biases over the north coast of Australia are found especially in spring and summer. During JJA, MUCAPE values are relatively small over land in ERA5 and the NARCliM ensembles and thus the larger biases are mostly over the tropical ocean (Figure A6).

In d02, the high MUCAPE values over the coasts and ocean at E Australia are clearer than those in d01, especially during SON and DJF (Figure A15). Both N1.0 and N1.5 overestimate in this region, with the biases the highest over Queensland and northern NSW and decrease towards the south (Figure A16). The distributions of MUCAPE in N1.0 and N1.5 are similar, except that the N1.5 simulated even higher values on the northern coast of NSW. During JJA, the highest MUCAPE values are off the coast of SE Australia, which is consistent with that in ERA5.

In the ERA5, the annual climatology of CIN (Note that positive values of CIN here means negative buoyancy.) in d01 has the largest values over the ocean surface northwest of Australia (Figure 3a), which may be due to the surface dryness in that region. Global analysis of CIN distribution also has a maximum in that region [48]. Most of the inland regions also have high CIN values, while the coasts generally have smaller CIN. The lowest CIN values reside at the Southern Ocean near the domain border. The N1.0 and N1.5 have simulated the maximum to the west, however, with the peak more to the south and thus over the northwest there are negative biases. The CIN peak in N1.5 is also stronger than that of N1.0. Over the inland regions, both N1.0 and N1.5 simulate too low CIN values (Figure 3b–e). To a certain extent this may be due to that ERA5 has searched for the most unstable parcel beyond 3 km that was implemented in the algorithm we applied to the NARCliM simulations. Another possibility is that the ERA5, with 137 model levels (with about 20 of them in the first km), has better resolved the lower atmosphere than the RCMs that have only 29 model levels (with about 8 of them in the first km). Over the ocean surface to the south and north, N1.0 and N1.5 have similar pattern and low magnitude.

The climatology and bias patterns in CIN during SON, DJF, and MAM are similar to those annually (Figure A7, Figure A8 and Figure A9). The maximum values of CIN are still to the west and northwest in both ensembles, while there is general underestimation by NARCliM. During JJA, the large CIN values shift to north Australia in ERA5, and N1.0 and N1.5 have simulated the shift well, but still with negative biases (Figure A10).

Focusing over SE Australia in d02, large CIN values from ERA5 are also found at the southeast coast besides the inland values, especially during SON and DJF (Figure A17). The N1.0 and N1.5 have successfully simulated the locations of these high CIN values, albeit with the same underestimates in the inland regions, which are clearly shown in the bias patterns (Figure A18), especially during summer. Nevertheless, with the appropriate CIN maximum over southeast NSW the NARCliM ensembles have simulated the convection inhibition effect over there realistically with only moderate biases. During JJA, the overall CIN values decrease over the southeast land regions, and the N1.0 and N1.5 have simulated the pattern.

The kinematic factor of S06 has a meridional climatology distribution in ERA5 (Figure 4a), revealing smaller VWS in the tropics where the easterly trades flow and higher VWS in the extra-tropics where there are the westerlies. The N1.0 and N1.5 basically have reproduced this pattern. However, N1.0 has too high S06 over the Southern Ocean and southern part of the continent, especially the southeast (Figure 4b,d). On the other hand, S06 in N1.5 is slightly too low over the Southern Ocean compared with ERA5, contrarily too high over central Australia (Figure 4c,e). High VWS is related to the jets in the westerlies. In other words, N1.5 has likely simulated the channels of the westerly jets at lower latitudes than those in ERA5.

The S06 during SON, DJF and MAM has a similar pattern as that annually, but weaker in magnitude during spring (Figure A11, Figure A12 and Figure A13). The bias patterns of N1.0 and N1.5 in these three seasons are also similar to the annual pattern, and the bias magnitudes are the smallest during spring. During JJA, the large-S06 region is a zonal band across central Australia (Figure A14), and thus the largest biases are also in that region.

In d02, the meridional pattern of S06 during SON and DJF in ERA5 over SE Australia is similar to that in d01 (Figure A19). N1.0 has simulated too large S06 at the south coast in these two seasons (Figure A20). On the contrary, both N1.0 and N1.5 have simulated too high S06 over Queensland during JJA. In other word, they have overestimated that zonal band of high S06 values.

We have also examined the climatology and bias of the two ensembles driven by the NCEP and ERA-Interim reanalysis, respectively (i.e., the NNRP and ERAI ensemble) over the CORDEX domain. Both ensembles have smaller biases for MUCAPE than those driven by GCMs (Figure 5). The bias patterns for CIN are also similar to those driven by GCMs and also slightly smaller in magnitude (Figure 6). For S06, the NNRP have similar biases to the GCM-driven N1.0, while the biases in ERAI are quite smaller than the GCM-driven N1.5 mean, which is attributable to that the reference ERA5 originates from the same family of model as in ERA-Interim (Figure 7). These two sets of reanalysis-driven ensemble show that part of the biases in N1.0 and N1.5 came from the GCMs, which will further be discussed in Section 5.

Statistics of the measures mean bias, RMSE and PCorr for both the reanalysis-driven and GCM-driven N1.0 and N1.5 ensembles have been documented in

Table A2. Mean annual bias, RMSE and PCorr for the indices CIN, MUCAPE and S06 simulated in the CORDEX (d01) domain by the reanalysis-driven and GCM-driven N1.0 and N1.5 ensemble mean. Unit: J kg⁻¹ for CIN and MUCAPE and m s⁻¹ for S06. The PCorr values are all significant at the 0.01 confidence level.

	CIN			MUCAPE			S06
	Bias	RMSE	PCorr	Bias	RMSE	PCorr	Bias	RMSE	PCorr
N1.0-NNRP	−343.5	367.5	0.90	−341.4	585.7	0.82	−9.9	10.5	0.98
N1.0-GCM	−271.7	291.5	0.88	248.8	337.6	0.91	2.7	3.0	0.99
N1.5-ERAI	−331.7	355.1	0.85	−236.6	452.8	0.95	−7.7	8.2	0.97
N1.5-GCM	−256.2	272.4	0.87	216.8	297.4	0.94	3.0	3.4	0.95

and

Table A3. As in Table A1 except for the NARCliM (d02) domain. The PCorr values are all significant at the 0.01 confidence level.

	CIN			MUCAPE			S06
	Bias	RMSE	PCorr	Bias	RMSE	PCorr	Bias	RMSE	PCorr
N1.0-NNRP	−358.3	369.3	0.94	−46.7	62.5	0.75	−11.0	11.0	0.38
N1.0-GCM	−293.6	302.7	0.92	180.2	201.6	0.64	3.3	3.4	0.46
N1.5-ERAI	−347.1	358.0	0.85	1.4	41.5	0.67	−10.0	10.0	0.30
N1.5-GCM	−285.8	293.5	0.94	186.9	204.6	0.77	2.5	2.8	0.25

for the two domains, respectively, for reference. It can be seen from these tables that N1.5 ensemble mean is comparable to or slightly better than that of N1.0 in these measures. In addition, the GCM-driven ensembles are even better than the reanalysis-driven ensembles in some of the measiures. These statistical measures for individual models in N1.0 and N1.5 can also be found in [49].

4.2. Convective Environments

The relationship between MUCAPE and CIN form the thermodynamic environment for convection [16]. Here we analyze how frequent the environment occurs for given values of MUCAPE and CIN. Because the main concern is thunderstorm development over land, we include only the grid points over land in the analysis. Within a season, all the daily values of MUCAPE and CIN are collected at the land grid points, and then a two-dimensional histogram is generated.

For d01, ERA5 has the most frequent environment occurring with MUCAPE between 1000–2000 J kg⁻¹ and CIN between 100–200 J kg⁻¹. However, due to quite large biases of CIN in N1.0 and N1.5, the ensembles deviate substantially from ERA5 in the frequency distributions (not shown). The results from d02 still show that the variability in the MUCAPE-CIN space of ERA5 is much larger than those in N1.0 and N1.5 (Figure A21). The NARCliM ensembles have simulated an environment with smaller spread in MUCAPE and CIN compared with ERA5. There is also a diagonal relationship in the NARCliM ensembles, i.e., when the environment has larger MUCAPE the corresponding CIN also increases. In contrast, most of the environments in ERA5 also have MUCAPE values within about 500 J kg⁻¹. However, there are more environments with this range of MUCAPE but possess higher CIN values.

On the other hand, thunderstorm development depends on the thermodynamic and kinematic environment, i.e., the combined influence from MUCAPE and S06 as represented in TSTORM [50]. Examination of the two-dimensional histograms in the MUCAPE-S06 space during summer shows that given the biases in MUCAPE and CIN, N1.0 and N1.5 have simulated similar spatiotemporal distribution of the convective environment (Figure 8). In d01, the ERA5 has many environments with low MUCAPE values when the S06 is about 5–10 m s⁻¹. Then there are environments with higher MUCAPE about 1000 J kg⁻¹ up to 2000 J kg⁻¹ with S06 around 5 m s⁻¹. The N1.0 and N1.5 distributions are essentially similar to that in ERA5, but with a general shift towards higher S06 values. Many environments in N1.0 are in the MUCAPE range (1000–2000 J kg⁻¹), while in N1.5 more environments are in the low-MUCAPE regime combined with high S06.

Comparison with the two TSTORM discriminant lines for severe and significant severe thunderstorms in Figure 8 shows that most of the environments in ERA5 are below the severe threshold and seldom surpass the significant severe threshold. Mainly due to the availability of higher S06 in N1.0 and N1.5, more environments are over the severe threshold and a number also higher than the significant severe threshold. Note also that the N1.0 environment has a large population with MUCAPE higher than that in ERA5, which has also driven the exceedance of the severe thunderstorm threshold.

The situation in d02 is similar to that in d01. The spatiotemporal distributions in N1.0 and N1.5 are similar to that in ERA5. There are only a small number of environments over the severe threshold in ERA5, and again due to the higher S06 values, more environments in N1.0 and N1.5 may develop severe thunderstorms, with an appreciable number reaching the significant severe threshold. Also noted is that the spread towards high MUCAPE values in the ERA5 environments is larger than that in the two NARCliM ensembles.

4.3. Extremes

It is known that even reanalysis datasets and convection-permitting models cannot reproduce the extreme part of the distribution in CAPE when compared with station observations [51]. The distributions of CAPE from reanalysis datasets and numerical models are much narrower than that derived from ground truth. Since we have not utilized station observations in this study, we compare the extreme values of MUCAPE and CIN from ERA5 and the two NARCliM ensembles.

The 95th percentile of MUCAPE within the reference period (1990–2009) has similar distributions to the climatology from ERA5 as well as N1.0 and N1.5 (Figure 9). In d01, the highest values are still over the tropical coasts and ocean surface, with values above 5000 J kg⁻¹. The N1.0 and N1.5 have their extreme values at about the same locations, albeit with the maximum MUCAPE values about 1000 J kg⁻¹ lower. In d02, the highest values of nearly 2000 J kg⁻¹ from ERA5 are over E and SE Australia, as in the climatology. In N1.0 and N1.5, it can be seen that a maximum has been simulated east of Queensland. N1.5 has also simulated a maximum northeast of NSW coast. Other than these maxima, the extreme values over E and SE Australia from these two ensembles underestimate ERA5.

The substantial underestimation in CIN by the NARCliM ensembles also apply to the 95th percentiles (Figure 10). In ERA5, the extreme CIN values have extended from northwest Australian coast (which is climatologically also high) to northern Australia. The maximum over the southeast coast is also clear. The N1.0 and N1.5 have their extreme values also over northwest Australia; however, the magnitude is much lower. The two ensembles’ extreme values over southeast Australia have a similar spatial extent to that in ERA5 but are also low in magnitude, which can be seen in the d02 domains. One of the possible reasons for the higher extreme values of CIN in ERA5 is that, as has been discussed in the methodology section, identification of the most-unstable parcel is within a thicker layer than the 3 km in the algorithm we applied on the NARCliM ensembles. Thus, ERA5 may have more high condensation levels and accordingly more extreme CIN values.

For S06, the 95th percentile values in N1.0 are still generally higher than those in N1.5 as in the mean (Figure 11). In d01, the N1.0 has overall lower values than those in ERA5 over the continent and the Southern Ocean. However, the maxima over Victoria and Tasmania have been reproduced. The maximum over Victoria in d02 is also clear. The pattern of extreme values in N1.5 is similar to that in N1.0 for both domains, and in d02 there is also a maximum over Victoria, although weaker in magnitude.

4.4. Storm Days

For the two thresholds of the TSTORM parameter measuring the potential for severe thunderstorm and significant severe thunderstorm, we have examined the average number of storm days during summer over the reference period [52]. In ERA5, the highest severe thunderstorm days concentrate at N Australia, and extending along the northeast and southeast coast, the latter can be clearly identified in d02 (Figure 12). This is consistent with the observed spatial distribution of thunderstorms based on the BoM storm archive [9], with storms concentrating at southeast Queensland, east NSW and SW Australia. Observed lightning activities also concentrate at E and NW Australia.

Given our previous discussion of the convective environments in N1.0 and N1.5 following Figure 8, the two ensembles have simulated larger areas exceeding the severe thunderstorm threshold. Both N1.0 and N1.5 have high storm days (e.g., most of summer) covering the entire tropical Australia and more inland regions of Queensland and NSW. N1.0 also has high storm days (e.g., half of summer) over large part of Western Australia, and comparatively N1.5 has less extent. In both ensembles, southwest coast and Southern Australia have the lowest storm days.

For storm days exceeding the significant severe thunderstorm threshold, ERA5 focuses on some of the tropical coasts and with a small peak over southeast coat of NSW (Figure 13). Instead, the N1.0 and N1.5 have high storm days covering most of the tropical coasts and those of Queensland and NSW. N1.5 has also simulated high storm days more inland into Queensland and northeast NSW compared with N1.0.

With reference to the observed and simulated MUCAPE-S06 environments (Figure 8), the overestimation of the storm days with respect to both thresholds in N1.0 and N1.5 is attributable more to that in S06, rather than in MUCAPE. However, N1.0 does possess more high-MUCAPE events than N1.5 in the CORDEX domain.

4.5. Seasonal Variation and Interannual Variability

The characteristics of the convective parameters in the other seasons (i.e., MAM, JJA, SON) are described here. As expected, the seasonal variation of MUCAPE is clear with a minimum during winter. The spatial pattern in the Australian region remains similar in all seasons, during winter the magnitude is low over the entire continent. The CIN also has clear seasonal variation. During spring to autumn, the largest variation is the magnitude of the maximum CIN over the west and northwest coast of Australia (Figure A7, Figure A8 and Figure A9). During winter, CIN over the entire Australian region is much smaller than in the other seasons (Figure A10). Compared to MUCAPE and CIN, the LCL and LFC has smaller season variation.

On the other hand, the S06 has the largest seasonal variation in the latitudes with largest VWS. From spring to summer, large VWS originates from the southern latitudes (Figure A11 and Figure A12). Progressing from autumn to winter, the regions with large VWS shift northward and in winter they form a zonal band between 20°–30° S (Figure A13 and Figure A14). Thus, during these two seasons S06 values in d02 are relatively low. The storm severity TSTORM, as a combination of MUCAPE and S06, largely follows the spatial distribution of MUCAPE. For example, the maximum of TSTORM in d01 is also over the ocean surface northwest of Australia and the coasts there. In d02, when S06 has its seasonal peak over E and SE Australia matching with that of MUCAPE, there is a clear maximum TSTORM. Even during winter when all the TSTORM values over land are quite small, there is still a weak maximum over the east and southeast coasts.

When evaluated by ERA5, the biases in the thermodynamic convective parameters considered here essentially the same sign across the seasons. Overall, the N1.0 and N1.5 simulate with positive biases in MUCAPE and negative biases in CIN, with seasonal variation in the bias magnitude. In terms of the spatial patterns of biases, they are not highly dependent on individual ensemble members. However, some dependency on GCM and RCM will be pointed out in Section 5. The N1.0 and N1.5 also have more regions in which the kinematic factor of S06 has been overestimated in all seasons. There are some specific regions with negative biases, such as over the Southern Ocean. Some model members that have negative biases of S06 in some regions would have a persistent sign of bias in most of the seasons.

Driven by MUCAPE and S06, N1.0 and N1.5 also overestimate TSTORM in most of the regions. In the d01 domain, individual models underestimate TSTORM over the tropical northwest during spring and summer. While in the d02 domain, negative biases also appear over southeast NSW during spring and summer. When we apply the two thresholds of severe and significant severe thunderstorm to TSTORM, the resulting seasonal variation of storm potential is apparent (Figure 14 and Figure 15). In these figures, the variability of estimate for each season refers to that within the 20 years of the reference period. It can be seen that for severe thunderstorm days, the N1.0 and N1.5 have similar seasonal variation as well as variability within the reference period. The two ensembles overestimate the storm days for all seasons, especially spring and summer. On the other hand, the N1.0 does not overestimate the significant severe thunderstorm days too much. Variability within the reference period is also smaller than that for the lower storm threshold. N1.5 estimates higher number of significant severe thunderstorm days than N1.0, especially during spring to autumn, and thus overestimates ERA5 even for this high category of storm threshold. It can also be seen that the variances in the NARCliM ensembles are much larger than those in ERA5 for most of the seasons, which are attributed to different contributions from the GCMs and RCMs to the ensemble means.

The interannual variability of the identified storm days in Figure 14 and Figure 15 have also been analyzed. It can be seen that for severe thunderstorm days, the interannual variability in N1.0 and N1.5 is much larger than that in ERA5 (Figure A22). Although trend analysis is not a focus in this study, the NARCliM ensembles have simulated an increasing trend of storm days during spring, while the simulated quite stable number of storm days during summer is consistent with that in ERA5 although with overestimation. During autumn and winter, both N1.5 (and N1.0 during JJA) simulate an apparent increase in the last decade of the time series. For N1.5 it is not sure whether this is related to the application of projection data (since CMIP5 historical simulations end in 2005). Further analysis is required to clarify this, similarly for the increase in N1.0 during the last few years during JJA. For the significant severe thunderstorm threshold, both N1.0 and N1.5 simulate a much lower number of storm days and thus are closer to that identified from ERA5 for most of the years (Figure A23). The simulated trends during spring are similar to that in ERA5. Again N1.5 has overestimated after 2005 during DJF, which may be due to application of projection data. During autumn and winter, the simulated storm days in N1.0 and N1.5 basically agree with that in ERA5.

Another perspective to examine the simulated interannual variability in NARCliM is the IVS (Equation (1)). Here we examine the IVS of the TSTORM parameter (

Table 1. Interannual variability score (IVS) of the domain-mean TSTORM simulated by N1.0 and N1.5 ensemble mean in the two domains. Zero IVS is the perfect match between model and reference data.

	SON	DJF	MAM	JJA
N1.0 (d01)	0.07	0.14	0.59	0.50
N1.5 (d01)	0.16	0.01	0.02	0.39
N1.0 (d02)	0.02	0.25	1.20	1.26
N1.5 (d02)	0.01	0.01	1.99	0.70

). Comparison of the scores show that N1.5 performs better during summer in both domains, which is the major thunderstorm season. However, there is no definite conclusion on the interannual variability matching with ERA5 for N1.0 and N1.5, since one ensemble would outperform the other in one domain but may not in the other domain.

5. Contributions from GCMs and RCMs

In a regional climate modeling system such as NARCliM, the RCMs with their own model configurations and physics downscale the information provided by the GCMs, which again have their own model configurations and physics. Thus, in the downscaled products there are relative contributions from the GCMs and RCMs with respect to different variables. In the following, such relative contributions are briefly discussed for the convective parameters in this study. More specifically, in N1.0 three physics packages of WRF3.3 was used (named R1, R2 and R3). In N1.5, the same model physics packages of R1 and R2 were applied, except that the RCM has been updated to WRF3.6 [53].

For the kinematic factor of S06, the driving GCMs seem to have dominated the patterns of VWS in the RCMs. For example, it can be seen from examining the individual ensemble members of S06 during summer that the bias pattern has been governed by the driving GCMs (Figure A24). For example, in d01 the RCMs driven by the GCM CCCMA3.1 have the same location of maximum bias. Similarly, the models driven by the GCM CSIRO-MK3.0 have the same negative bias over Southern Ocean. In N1.5, the GCM family of ACCCESS1.0 and ACCESS1.3 have similar bias pattern, in contrast to that driven by the GCM CanESM2. In other words, the RCMs have not modified the large-scale to regional scale circulation substantially, and thus the VWS patterns are not very different under the same GCM.

On the other hand, the thermodynamic convective parameters are more dependent on the RCM model physics. For MUCAPE, which is the major factor in the storm severity, it can be seen that the bias pattern in d01 can still be explained by the driving GCM to some extent (Figure A25). For example, there are the characteristic large biases over the tropical ocean in the models driven by the GCM CCCMA3.1 and ECHAM5. In addition, the biases are large at northeast Australia for models driven by the GCM CSIRO-MK3.0 and to some extent also GCM MIROC3.2. However, the variability under different RCMs is much larger than that in S06, such as the different signs of bias in northern Australia from R1 and R2 driven by one of the GCMs in N1.5.

Such contributions from the RCMs can also be identified in d02 of simulated MUCAPE. In this domain, the negative biases over the southeast coast are clear from GCMs CSIRO-MK3.0 and MIROC3.2 (Figure A26). However, it is also apparent that RCM R1 has simulated larger biases than R2 and R3 (for N1.0). Therefore, modification of the physics, including that affecting the thermodynamics of convection, is critical to the quality of the simulated thunderstorm environment. The benefit of NARCliM is that the large ensemble of the combined N1.0 plus N1.5 is able to reduce the uncertainty arising from such sensitivity to the model physics.

6. Summary and Discussion

6.1. Discussion

This study has performed an evaluation of the convective environments in NARCliM based on the reanalysis dataset ERA5. However, as shown in studies such as [29,47] reanalysis datasets likely underestimate CAPE, low-level moisture and VWS, and cannot capture the extreme part of the CAPE distribution. Such biases in the reanalysis datasets would influence the comparison results we have presented. In order to understand the results in this study better, the ERA5 convective environments should be evaluated by the station radiosondes observations [54]. In a recent study, Varga and Breuer [55] evaluated convective parameters derived from ERA5 by station sounding observations in the European region. They found that while ERA5 highly correlated with the sounding data, there are indeed systematic biases in parameters such as CAPE, CIN and vertical wind shear.

In terms of spatial resolution, the ERA5 is about 31 km in grid size that is one of the best among the reanalysis datasets. However, this is still much coarser than the NARCliM-domain simulations (10 km). The high-resolution simulations in NARCliM can resolve the small variability in air temperature, moisture and VWS better. Accordingly, the biases of the storm parameters identified in NARCliM domain should be attributable to such difference in spatial resolution. Nevertheless, the 10-km resolution in the N1.0 and N1.5 inner-domain simulations are still not convection permitting (e.g., compared with [16]), which represents a limitation for these ensembles in reproducing realistic thunderstorm structures. In regard to this limitation, the new generation of NARCliM (N2.0), which applies two versions of WRF V4.1.2, will has a spatial resolution of 4 km and resolve convection better. Five CMIP6 GCMs have been selected for downscaling based on multiple climate change scenarios (including Shared Socioeconomic Pathways SSP126, SSP245 and SSP370), and thus N2.0 will produce a high-resolution large ensemble system.

In any regional downscaling system, the relative contributions from the GCMs (via boundary conditions) versus those from RCMs (via model physics) is an important issue. Our preliminary analysis in Section 5 indicates that kinematic factors such as S06 are more inherited from GCMs, while thermodynamic factors such as MUCAPE are more inherited from RCMs. In other words, these indices determining the convective environments would have different bias characteristics and uncertainties associated with GCMs and RCMs. For example, in terms of bias characteristics the biases of S06 would be correlated with those in the GCMs. For the biases in the thermodynamic parameters such as MUCAPE and CIN, the responsible factors are complex. They may be due to the land and sea surface temperatures and moisture from the GCMs driving the RCMs, and also due to the internal physics of the RCMs. From our discussion in Section 5, it is quite clear that some model physics have generated larger biases than the other ensemble members. Further insights would be obtained if the analysis is extended to examine the full multi-model ensemble spreads in the indices, and compare with the GCM-only spreads (for specific RCM) and RCM-only spread (for specific GCM).

The application of the terminology “bias” here is only with respect to difference from the reference reanalysis dataset. Broadly considered, the “differences” we have identified represent the added value from the RCMs that have resolved the convection processes better than the driving GCMs. In other words, some of these “biases” may be considered benefits that have been introduced by the downscaling process.

The application of the evaluation period 1990–2009 here is due to the availability of N1.0 only for this historical period. In fact, continuous simulations from 1950 to 2005 in N1.5, which are driven by the selected CMIP5 GCMs, are available. This long period of simulations is a valuable resource for investigating the climate variability of convective environments, such as the association with different modes of variability. Thus, the quality of the simulated interannual variability in the N1.5 ensemble is a potential topic for future study.

Climate change would likely impact the future convective environments in the Australian region [44,56], as has been studied for other parts of the world [16,48,52]. The large ensemble of N1.0 plus N1.5 here, with the benefits of downscaling, is a great candidate to apply for assessing the future convective environments. However, given the biases in the convective parameters we have identified, application of pre-determined discriminants to measure severe thunderstorm potential (such as those we applied on TSTORM) may not be appropriate for climate projection. Strategies should be developed for assessing the future convective environments, such as through examination of the changes in the extreme value distributions of the convective parameters and then estimating the future probabilities of occurrence of different categories of thunderstorms [56]. Equally important is through a physical understanding of how the convective parameters change in a warmed climate, such as the effects of increased surface temperature, changes in low-level moisture, equilibrium level and circulation changes [48]. Accordingly, we will extend this study by developing a scheme to project the future convective environments in the Australian region with consideration of the biases identified here and improving our understanding of what changes in the physical environment are responsible for those in future thunderstorm activities we expect.

6.2. Summary

The convective environments in the two generations (N1.0 and N1.5) of the NARCliM regional climate ensemble modeling system for the Australian region have been evaluated based on the ERA5 reference reanalysis dataset. The convective parameters in the evaluation include the maximum unstable CAPE (MUCAPE), convective inhibition (CIN), vertical wind shear (S06) and storm severity (TSTORM). The climatology of the condensation levels (LCL and LFC) in NARCliM have also been examined. N1.0 and N1.5 have respectively downscaled selected CMIP3 and CMIP5 GCMs to 50 km (10 km) in the outer (inner) domain, and thus have much better resolved convective processes.

The evaluation results of annual climatology, convective environment in the peak thunderstorm season of summer (DJF) are the focus with additional discussion on seasonal variations. Biases in both ensembles have been identified in the NARCliM simulations especially for the thermodynamic parameters, which are overall overestimation of MUCAPE and underestimation of CIN. However, the spatial distributions in both domains are reproduced well by the two ensemble means. The kinematic factor of S06 is well simulated by N1.0 and N1.5 in terms of the locations of high VWS and its magnitude. It is found that the biases in S06 mostly originate from the driving GCMs, while those in MUCAPE and CIN are modulated more substantially by the physics in individual RCMs. When the extremes (95th percentile) of these parameters are examined, the spatial patterns, especially the locations of the peak values, match between the NARCliM ensembles and ERA5 well.

The simulated convective environments have been examined, with focus on land areas over southeast Australia, through the MUCAPE-CIN and MUCAPE-S06 relationships. In other words, the occurrence frequency of the environment with specific pairwise values of these convective parameters are analyzed. Given the biases identified in MUCAPE and CIN, the MUCAPE-CIN relationship is also biased. However, the regime in which most environments occur during summer is similar between N1.0/N1.5 and ERA5. Under the MUCAPE-S06 space, the regimes in which the environment most frequently occurs also match well between N1.0/N1.5 and ERA5, and this applies to both the Australian continent (outer domain d01) and southeast Australia (inner domain d02). The regimes in the NARCliM ensembles have shifted toward higher MUCAPE and slightly higher S06, which result in higher estimated storm days with respect to the severe thunderstorm and significant severe thunderstorm threshold for TSTORM. Under the severe thunderstorm category, the spatial distributions of storm days in N1.0 and N1.5 have the same pattern as in ERA5, but with extended areas in tropical Australia, coastal Queensland and NSW. Under the significant severe thunderstorm category, the distributions of storm days agree well with ERA5 with slight overestimation along the Queensland and NSW coasts.

The MUCAPE and CIN have clear seasonal variation with their minima occurring during winter. For S06, the largest seasonal variation is in the latitudes with high VWS, such as the shift to the lower latitudes during winter. In general, the sign of the biases in N1.0 and N1.5 for these convective parameters remain the same in all the seasons. That is, overestimated MUCAPE and underestimated CIN, and there are more regions with overestimated S06. This explains the fact that when the TSTORM is used to estimate the storm days, the numbers are in N1.0/N1.5 are higher than that in ERA5 in all seasons, except during winter when the significant severe thunderstorm days are all very low. However, the seasonal cycle has been well simulated. When the interannual variability of the estimated storm days within the 20-year reference period is examined, variability is much larger in the NARCliM simulations compared with ERA5. Further analyses are necessary to explain the apparent trends in the simulations. The measure of IVS does indicate that during summer N1.5 has better matching of the interannual standard deviation with the reanalysis, but N1.0 outperforms in some other seasons and specific domain.

Our evaluation results indicate that regional climate simulations such as NARCliM can capture the convective storm environment in the Australian region in certain aspects. The biases over land in the indices we evaluated from ERA5 are less than those over the oceans. Importantly, the spatial distributions of the indices have been simulated well (e.g., with respect to the PCorr in Table A1 and Table A2). Seasonal variation in the simulations agree with that in ERA5, given that the biases in the indices still exist. Although interannual variability in NARCliM does not match well with ERA5, the evaluation period is short and we have only analyzed certain aspects of interannual variability such as the IVS. Overall, these results suggest that regional climate simulations, with appropriate bias adjustment, can be used to undertake future climate projections of storm environment, which will provide evidences for long-term planning of resources and infrastructure to minimize damage of extreme events in future.