Demonstration of a Nowcasting Service for High Ice Water Content (HIWC) Conditions

Abstract

Areas of high ice water content (HIWC) within cloud systems may cause power loss events and engine damage in jet aircraft due to ice crystal icing (ICI). The Algorithm for Prediction of HIWC Areas (ALPHA) was developed to identify these regions and enable provision of guidance to airlines. ALPHA combines numerical weather prediction model data, satellite data, and radar data (where available), and applies fuzzy logic to identify the likely presence of HIWC. In a collaboration between the U.S. National Center for Atmospheric Research, Australian Bureau of Meteorology, U.S. Federal Aviation Administration, and Australian airlines, a trial of ALPHA was conducted for an area across Indonesia, Papua New Guinea (PNG), and northern Australia, a region with frequent deep convection and a relatively high incidence of ICI events. ALPHA was adapted to ingest data from the Australian Community Climate and Earth System Simulator model and the Japanese Himawari-8 geostationary meteorological satellite. Radar data was not used. The HIWC product was made available to stakeholder groups for evaluation. Independent validation of the HIWC product was undertaken by comparing it with retrieved profiles of ice water content (IWC) from the cloud profiling radar on the NASA polar-orbiting CloudSat satellite. Conduct of the ALPHA trial and results from validation of the HIWC product provides confidence in the potential utility for flight planning, maintaining situational awareness, and flight monitoring.

1. Introduction

Deep convective cloud systems in the tropics and sub-tropics often contain regions with a high mass concentration of ice particles at high altitudes. For jet aircraft, these regions of high ice water content (HIWC) may cause engine power loss events and engine damage due to ice crystal icing (ICI) [1,2,3,4]. The ice particles may also accumulate in air data probes, causing anomalies in the output that can impact operations of the flight management system and maintaining flight control [5,6]. These impacts are a function of the concentration and path length for the aircraft. The characteristics of these HIWC regions can vary significantly and the associated conditions may appear benign to pilots with no significant radar echoes due to small ice particle size, little or no lightning or turbulence, and no airframe icing. Bravin and Strapp (2019) [4] analysed 174 ICI engine events (including power loss, damage, and high vibration) recorded in a Boeing event database. They show there is a wide geographical distribution of events across the tropics and sub-tropics, both oceanic and continental, with maxima over North America, South America, Africa, and the Southeast Asia/maritime continent region. It has also been shown that engine events can result from prolonged exposure to moderate values of ice water content (IWC) (0.5 < IWC < 1.0 g m⁻³) as well as short exposure times at high values (≥1.0 g m⁻³) of IWC. In a more recent study, Rugg et al. (2021) [7] used 11 years of IWC retrievals from CloudSat and CALIOP satellite data to provide a first estimate of the global distribution of HIWC frequency based on a threshold of at least 1.0 g m⁻³. These results also show maxima in South America, Africa, and the Southeast Asia/maritime continent region.

In recent years, there has been an international collaboration between the European High Altitude Ice Crystal (HAIC) Project and the USA-led High Ice Water Content (HIWC) project directed at characterising the ice-crystal environment, better understanding the fundamental cloud physics in deep convection, understanding critical engineering issues associated with ICI in jet engines, developing new certification standards for engines and air data probes, and developing strategies to mitigate the risks to international aviation [8,9,10]. The work included flight campaigns with instrumented research aircraft out of Darwin, Australia, in 2014; Cayenne, French Guiana, in 2015; and Fort Lauderdale, Florida, USA, in 2015. Additional research flights were conducted from Fort Lauderdale, Florida, Palmdale, California, and Kona, Hawaii in 2018. In situ observations of IWC were collected during these flights. The work also included development of nowcasting techniques to identify potential areas of HIWC and enable the provision of guidance to better support flight operations and manage a tactical response [11,12,13,14].

The Algorithm for Prediction of HIWC Areas (ALPHA) [14] (hereafter referred to as H2020) was developed by the U.S. National Center for Atmospheric Research (NCAR) with funding support from the U.S. Federal Aviation Administration (FAA). This application uses a combination of numerical weather prediction (NWP) model data, satellite data, and radar data (where available), to identify the likely presence of HIWC with data collected during the HAIC-HIWC field campaigns used in the development. A key objective with this development is the provision of near-real-time guidance based on meteorological information that has wide coverage, is routinely available, and has appropriate spatial and temporal resolution.

Following development of ALPHA, the Australian Bureau of Meteorology (hereafter referred to as the Bureau) collaborated with NCAR, FAA, and Australian-based airlines to trial ALPHA for an area across Indonesia, Papua New Guinea (PNG), and northern Australia. The application was adapted to ingest data from the Bureau’s Australian Community Climate and Earth Simulation System (ACCESS) model [15] and the Japan Meteorological Agency (JMA) Himawari-8 (H8) geostationary meteorological satellite [16]. Available radar data coverage across the trial domain is very limited; thus, it is not used.

Arrangements were made for the dissemination and visualisation of the trial HIWC product from ALPHA in the Bureau National Operations Centre and the Australian airlines Qantas and Virgin Australia. Initially, the trial objectives included evaluation by airline operations personnel (including pilots, flight planning, and flight operations) and consideration of the product utility for maintaining situational awareness and flight monitoring, input to further development of the HIWC product to address user needs, and input to the development of standards and recommended practices proposed for HIWC guidance for the international aviation industry. These objectives were severely impacted by the COVID-19 pandemic in early 2020. With access to airline operations personnel and their feedback on product utility no longer available, the trial objectives were re-scoped to focus on performance evaluation and potential utility of the product for maintaining situational awareness and flight monitoring. Validation of the HIWC product was undertaken by comparing it with data from the NASA polar-orbiting CloudSat satellite, from which vertical profiles of IWC can be retrieved.

In Section 2, we describe the HIWC product, including the input meteorological data and associated processing. Section 3 reviews validation of ALPHA during its initial development and outlines the methodology for independent validation made possible for this trial with improved retrievals of IWC from the CloudSat satellite. Section 4 presents details on the operational setup for the ALPHA trial. This included determination of the trial domain, adaptation of ALPHA to ingest and process the ACCESS model and satellite data from the Bureau, and arrangements for dissemination of the HIWC product to stakeholders. Results from the validation are presented in Section 5, with a detailed case study for an event on 7 January 2020 and an overall performance evaluation comparing the HIWC product with profiles of IWC for 397 daytime CloudSat passes during the observation period. In Section 6, we discuss the outcomes of the ALPHA trial and the implications for development and provision of operational advisories for HIWC.

2. HIWC Product Description

Atmospheric conditions associated with ICI engine events span a range of temperatures, flight altitudes, and convective storm types. Analyses have shown that common attributes make the identification of HIWC conditions potentially feasible using routinely available meteorological data [3,17]. Using observations of meteorological conditions associated with these events along with research aircraft measurements of IWC, ALPHA was developed to diagnose areas of suspected HIWC conditions. Lacking a rigorous physical understanding of the processes which lead to hazardous concentrations of ice crystals, a machine learning approach was employed to estimate the potential for HIWC conditions.

Fuzzy logic provides a framework for the optimal combination of a set of variables to solve decision-making problems where there is uncertainty, with applications in a wide range of areas [18]. As discussed in H2020 [14], ALPHA uses a set of critical meteorological variables derived from satellite, NWP model, and radar (where available) data together with fuzzy logic methods to identify areas with a likely presence of HIWC. The satellite-based component of the algorithm locates high, cold cloud tops with a large cloud optical depth (COD). NWP model fields are used to identify areas of potential convection using updrafts, convective precipitation, and condensate fields. Model temperature profiles bound the vertical extent of glaciated cloud conditions. The radar component of ALPHA detects active updrafts, high radar reflectivity, and the height of the convective clouds. The “three-input” version of ALPHA is based on satellite, model, and radar data (where available), and the “two-input” version is based on satellite and model data only. For this trial, the “two-input” version of ALPHA was used due to the limited availability of radar data across the domain. The lack of radar data in the “two-input” version means it is not possible to provide information on the vertical structure of HIWC that is possible in the “three-input” version. In situ measurements of IWC by research aircraft during field campaigns were used in training the algorithm with a machine learning technique. A significant challenge is how well the algorithm delineates the boundaries of HIWC areas due to spatial and temporal variability in the associated cloud characteristics. Hence, there will be increased uncertainty around the edges of HIWC features.

Fuzzy logic membership functions are probabilistic or heuristic relationships between a single variable and the presence of HIWC. When applied to an input field it is transformed into an interest field with values in the interval [0,1] forming a member of a fuzzy set. The interest fields for a set of variables can then be aggregated using a weighted sum that is optimised to generate an interest field for each data type. The interest fields for the data types are further aggregated with a weighted sum to create a 3-dimensional diagnosis of HIWC Potential with values in the range [0,1], where 0 represents minimum likelihood of HIWC and 1 represents maximum likelihood. The complete set of membership function parameters and optimised weights for the variables used in ALPHA are presented in H2020 [14] at Tables 2 and 3.

To illustrate, the in situ observations of IWC from research flights together with associated satellite observations and derived variables were used to derive fuzzy membership functions for the daytime satellite interest field. The function parameters are presented in

Table 1. Membership function parameters and weights for the daytime satellite interest field in the two-input version of ALPHA (adapted from H2020 [14]).

Variable	Sign	Inflection Points		Weight (%)
Cloud optical depth (COD)	Positive	16.2	129.2	61.2
Cloud top temperature (CTT) (K)	Negative	196.5	196.5	31.9
Brightness temperature at 10.8 µm (BT10) (K)	Negative	214.95	225.25	6.9

, with graphical formats shown in Appendix A (Figure A1). Figure A1 also shows the distribution for the in situ observations of IWC associated with each of the variables. They show higher values for the COD, and colder cloud top temperatures are associated with higher IWC.

Table 1 shows that the transformed COD is equal to 0 for values less than 16.2; for values greater than or equal to 129.2 it is 1, and there is a linear interpolation for intermediate values. For CTT and the 10.8 µm brightness temperature (BT10), the sign is negative, so the transformed variable is equal to 1 for values less than the lower threshold and 0 for values greater than or equal to the upper threshold.

Given the large number of data types and associated variables, a machine learning technique, known as particle swarm optimisation (PSO), was applied to tune ALPHA, and this is detailed in H2020 [14]. This multi-parameter optimisation technique was adapted to objectively select specific input variables and exclude others, define piecewise linear membership functions with associated inflection points, and determine weighting factors for optimal aggregation of the interest fields for the input variables and the intermediate data types [14]. The in situ measurements of IWC collected during field campaigns and other associated observations were used to guide the PSO process.

The ALPHA product is a three-dimensional grid of the uncalibrated potential for clouds to contain high concentrations of ice particles (more than 0.5 g m⁻³). Values of the HIWC Potential lie in the range 0–1, with higher values indicating an increased probability of encountering ice water content above the 0.5 g m⁻³ threshold. As the “two-input” version of ALPHA was used, there is no variation in the vertical structure for HIWC Potential, apart from the upper and lower bounds. The product is currently an analysis at the time of observation, and a short-term nowcast is not yet provided.

3. HIWC Product Validation Methods

3.1. Prior Validation of ALPHA Product

Development of the ALPHA application was based on data collected during the HAIC-HIWC flight campaigns. This included bulk IWC measurements using an isokinetic total water content (TWC) probe (IKP2) developed for the campaigns, with measured values reaching 5 g m⁻³ [9,10]. These data were provided as 5 s moving averages. They were further averaged over 25 s intervals to better match the spatial resolution of the input data for ALPHA. The 25 s data were sampled in time to ensure temporal independence and partitioned into two subsets, with the first used for the training of the algorithm and the second reserved for verification. The verification subset was used to evaluate the performance of ALPHA by comparing HIWC Potential with the in situ IWC measurements [14].

Various statistical metrics were used to characterise ALPHA’s skill for detecting HIWC conditions. Figure 1 (adapted from H2020 [14]) shows the relationship between observed HIWC conditions and HIWC Potential estimated by ALPHA for assumed HIWC thresholds of 0.5 g m⁻³ and 1.0 g m⁻³. The two thresholds were assessed, as there is not a generally agreed threshold for defining HIWC. Good correlation is obtained for HIWC Potential values up to approximately 0.65. At higher values, the relationship is less robust, probably due to small sample sizes in the data set with high Potential values.

By assigning threshold IWC values of 0.5 g m⁻³ and 1.0 g m⁻³ to define HIWC conditions, the authors calculated probability of detection (POD), probability of false detection (POFD), and false alarm ratio (FAR) statistics for specified values of HIWC Potential. With an IWC threshold of 0.5 g m⁻³ and an HIWC Potential value of 0.25, the POD, POFD, and FAR values were 86%, 29%, and 51%, respectively.

Table 2. ALPHA performance statistics showing probability of detection (POD), probability of false detection (POFD), and false alarm ratio (FAR) based on an IWC threshold of 1.0 g m⁻³. Derived from H2020 [14].

HIWC Potential Threshold	POD	POFD	FAR
0.2	95%	42%	77%
0.3	85%	30%	73%
0.4	63%	18%	68%

gives these statistics for an IWC threshold of 1.0 g m⁻³ and various values of HIWC Potential. By varying the HIWC Potential threshold, receiver operating characteristic (ROC) curves were constructed from these statistics. The resulting area under the ROC curve (AUC) for the entire verification data set was 0.85, while for in-cloud data only the AUC was 0.69.

Results indicated the application has skill in detecting HIWC conditions, albeit with a significant false alarm ratio. Monte Carlo simulations were used to assess uncertainties in the performance of the application [14].

3.2. Validation of ALPHA with CloudSat IWC Retrievals

The exercise described in this paper provided an opportunity to evaluate ALPHA using a different source of IWC data. The HIWC Potential product was compared with matched data from the NASA polar-orbiting CloudSat satellite that carries a 94 GHz nadir-looking cloud profiling radar (CPR) [19]. The radar has a narrow beam with a surface footprint of around 1.4 km diameter and a vertical resolution of 500 m. These data enable the retrieval of a vertical cross section of IWC within clouds the satellite passes over, based on a statistical relationship between the radar reflectivity and the IWC [20]. The along-track resolution of the vertical cross section data is around 1.06 km.

In early studies with the CloudSat radar data the IWC in ice clouds was estimated from aircraft-based observations of the particle size distribution (PSD) together with a prescribed statistical relationship between ice-particle mass and diameter, m(D) [20]. There was considerable uncertainty in the radar reflectivity Z–IWC relationship, arising from uncertainty in the PSD–m(D) relationship and from non-Rayleigh scattering effects that become important at higher values for Z.

As discussed in Section 3.1, bulk IWC measurements were collected during the HAIC-HIWC flight campaign out of Darwin. The research aircraft also carried the 95 GHz RASTA (Radar Aéroporté et Sol de Télédétection Atmosphérique) multibeam cloud radar with three non-collinear antennas looking up and three non-collinear antennas looking down [20]. The radar reflectivity from the nadir and zenith antennas nearest to the aircraft were linearly interpolated to produce a time series of Z₉₅ that could be compared with the IKP2 data. This unique dataset enabled development of a statistical relationship between the RASTA radar data and measured IWC. Protat et al. (2016) [20] used a power-law functional form of the IWC–Z₉₅ relationship to fit the distribution, as follows:

$$\mathrm{IWC}\left(\mathrm{g} \mathrm{m}{¯}^3\right)=0.108 {\mathrm{Z}}_{\mathrm{e}}^{0.770 }$$

where ${\mathrm{Z}}_{\mathrm{e}}={10}^{{\mathrm{Z}}_{95}/10.0}$ is the radar reflectivity Z₉₅ expressed in linear units (mm⁶ M⁻³). The radar data above and below the aircraft then allowed retrieval of the vertical distribution of IWC in and around tropical deep convective cores observed during the flight campaign.

Previous quantitative comparisons of the 95 GHz RASTA data with the CloudSat spaceborne 94 GHz cloud radar for altitudes above the freezing level (FZL) showed the two radar reflectivities agreed to within 1 dB [21]. The CloudSat radar reflectivities also agreed statistically within 0.4 dB with cloud radar data from five different ground-based sites. These results give confidence that the RASTA cloud radar is well-calibrated. They also provide confidence that the statistical relationship between RASTA radar data and IWC can be applied to the CloudSat cloud profiling radar to enable retrieval of a vertical cross section of IWC within clouds that the satellite passes over.

The CloudSat satellite is on a sun-synchronous orbit and passes across the ALPHA trial domain 2–3 times during the day on the ascending orbit. The local solar time at the equator for each overpass is around 1410 local time (LT). No radar data is available for the night-time descending pass following a battery malfunction on the CloudSat spacecraft in 2011, so the validation only applies to daylight hours. CloudSat data for 397 daytime passes in the period 1 October 2019–31 March 2020 were retrieved and processed to provide vertical cross sections of the IWC.

4. Operational Implementation

For this trial, the latest two-input version of ALPHA (v2.4) was adapted to ingest data from the Bureau’s regional version of the Australian Community Climate Earth System Simulator model (ACCESS-R) [15,22] and the JMA H8 satellite [16].

Intensive operations were conducted in the period October 2019–March 2020 inclusive, with the following objectives:

• To provide aviation industry stakeholders, including Bureau forecasters and airlines, with the HIWC product for evaluation.
• To inform further improvements to ALPHA.
• To inform a decision on provision of an operational HIWC nowcasting product.
• To progress the development of standards and recommended practices for the provision of HIWC guidance required by the international aviation industry.

The Bureau’s regional ACCESS-R model is nested in the global ACCESS-G model and has a horizontal domain that covers the Australian mainland and surrounding areas, extending well beyond the ALPHA trial domain. The horizontal resolution is 0.11° × 0.11°, and there are 70 levels in the vertical with a top around 80 km. The model runs on a six-hour cycle and data files were routinely generated for ingestion by ALPHA with the relevant forecast fields provided at hourly time steps. The ACCESS-R fields used in ALPHA and associated fuzzy logic membership function parameters and weights are presented at Table 2 in H2020 [14].

The H8 satellite data is received every 10 min and is pre-processed to generate data files containing observed and derived parameters for ingestion by ALPHA. The spatial resolution of these data is interpolated to 0.02 degrees. A different blend of data fields, fuzzy logic membership functions, and weights are used for daytime and night-time hours (H2020, Table 2 [14]). The derived satellite parameters were generated using the Geostationary Cloud Algorithm Testbed (GEOCAT) application developed at NOAA NESDIS (http://cimss.ssec.wisc.edu/csppgeo/geocat.html, accessed on 15 April 2023) which is a software framework for running science algorithms. The derived parameters used in ALPHA included cloud top temperature (CTT) and cloud top height (CTH) [23], as well as daytime cloud optical depth (COD) [24]. The COD field has significant spatial variability at small scales and is spatially averaged within ALPHA using a 10 km × 10 km window prior to processing by the fuzzy logic.

The intermediate model and satellite interest fields are blended with the respective weights presented at Table 3 in H2020 [14]. It is noted the NWP model interest field has a small weight, so the HIWC Potential in this version of ALPHA is largely determined by the satellite interest field. The model-based temperature profiles are used to constrain the HIWC Potential to layers with sufficiently cold temperatures.

The area of coverage for the HIWC product is 5° N–20° S, 100–160° E, with a horizontal resolution of 0.02 degrees, and from FL200 to FL450 at 14 levels (Figure 2). It is updated every 10 min immediately following receipt of the H8 satellite data and associated processing. There is a latency of approximately 25 min from the satellite scan start time until the product is available for users. This is due to the time required for completion of the full satellite scan from north to south, receipt and pre-processing of these data in the Bureau (around 16 min), plus processing by ALPHA and dissemination of the product (9 min). It is noted that the full H8 scan takes nearly 10 min to complete from north to south, and the local scan time at the equator is around 5 min later than the nominal scan start time. Hence, for areas near the equator, the latency in receipt of the HIWC product is, in fact, around 20 min after the scan time across that area.

To facilitate secure and efficient collaboration between the Bureau and NCAR, ALPHA was installed on an Amazon Web Services (AWS) Elastic Cloud Compute (EC2) platform, configured as a virtual private cloud. This provided a secure environment that was accessible by both Bureau and NCAR scientists, enabled the installation, configuration, and maintenance of ALPHA, uploading of input data, processing of these data by the application, downloading of output products for dissemination to user groups, and archiving of the output.

Planning meetings were held with airline representatives, including pilots, flight operations personnel, and the airline weather forecasting personnel. It was agreed that the HIWC product should be made available to the meteorologists in the airline flight operations area. Arrangements were made to deliver the HIWC product to user systems as a netCDF file that was automatically ingested and made available as an overlay with other products, such as infrared satellite imagery on operational workstations. This is illustrated in Figure 3. For the display systems, an RGBA color map for the product was developed to highlight increasing values of the HIWC Potential. Values of HIWC Potential less than 0.2 are not considered significant, and the alpha (A) value in the color map was set to zero, or transparent, so they were not displayed. Education material was prepared, and training sessions were held for personnel.

5. Results

5.1. Case Study 7 January 2020

We examined a maritime mesoscale convective system located north of the Northern Territory, Australia, on 7 January 2020, in the area centred around 9° S, 135° E (Figure 3). This long-lived convective system was associated with a low-pressure system that moved slowly west and eventually developed into Tropical Cyclone Claudia on 11 January. Figure 4 shows the HIWC product generated by ALPHA from the 0500 UTC, 7 January 2020, H8 scan (local scan time centred around 05:05:30 UTC) with the yellow through red areas in Figure 4 corresponding to the black areas in the centre area of Figure 3. Figure 4 also shows the ascending CloudSat track across the sector, 12° S to 6° S. The overpass is centred at 05:00:12 UTC at 9° S, and the duration across the sector was 98 s. The overpass intersected three areas of elevated HIWC Potential around 11.3° S, 10.0° S, and 7.5° S.

The vertical cross section of radar reflectivity above the FZL from CloudSat (Figure 5a) and retrieved IWC (Figure 5b) shows three areas with elevated reflectivity around 11.3° S, 10.0° S, and 7.5° S, with differences in the horizontal and vertical structure for each. The area around 11.3° S shows that the cloud top extends to 16 km with a maximum radar reflectivity of 9–12 dBZ and associated IWC around 1.0 g m⁻³ at an altitude around 9 km. The area around 10.0° S shows deep convection with a relatively high radar reflectivity extending to an altitude above 17 km and an IWC of 2 g m⁻³ extending to 16 km. At lower altitudes there are three separate convective cores, including a new convective core near 10.2° S and altitude 7 km with a radar reflectivity approaching 24 dBZ and an associated IWC of 6 g m⁻³. The area around 7.5° S shows a broader stratiform region with a cloud top to 16 km, a relatively high radar reflectivity, and an associated IWC of 1–2 g m⁻³ extending to 10 km.

In comparing the CloudSat data with the HIWC Potential and associated H8 fields, care is required to ensure correct alignment of the CloudSat data, which has a narrow footprint of 1.4 km, with the satellite data and HIWC Potential, particularly in areas near the edge of cloud features where gradients can be high. This includes accounting for cloud movement, growth or decay due to time differences between the scans, parallax errors, and possible navigation errors in the H8 geostationary satellite data. This is particularly relevant for the area around 11.3° S where the CloudSat passed over the western edge of the cloud feature at 04:59:34 UTC. The local scan time for the 0500 UTC H8 scan is centred around 05:05:38 UTC at 11° S, 6 min later than the CloudSat overpass. This cloud feature was moving west at 8.3 m s⁻¹, giving a movement of 3.0 km in the 6 min. Local changes in the cloud structure around this time were not significant. There are also parallax errors in the H8 satellite data that increase with the off-nadir angle and cloud height and result in an apparent displacement of cloud features away from the satellite nadir point relative to the true location. The H8 satellite is at a longitude of 140.7° E, and for a 16 km cloud at 135.7° E the parallax error will be around 1.5 km to the west.

For the calculation of HIWC Potential during daylight hours in the “two-input” version of the ALPHA algorithm, a weighting of 97.6% is applied to the satellite data, and 2.4% is applied to the model data. For the satellite data type, the weightings for aggregation of the input variables are 61.2% for COD, 31.9% for CTT, and 6.9% for BT10 (Table 1).

To allow for the time difference and parallax errors, the COD, CTT, BT10, and HIWC Potential were retrieved from the 0500 UTC H8 satellite data and HIWC product, on a track displaced 0.05 degrees west of the CloudSat track and without any spatial averaging. These fields show elevated values for COD (Figure 6) and cold CTT (Figure 7) in the segments indicating elevated HIWC Potential. The BT10 also showed cold temperatures in these areas, but it is highly correlated with the CTT and is, as such, not presented here. The spatial variability in the COD field is particularly notable in comparison to the CTT field, which is much smoother. To illustrate the impact of the track correction, Figure A2 in Appendix A shows the cross sections of COD and CTT retrieved from the H8 satellite data along the uncorrected and corrected CloudSat track. The most notable difference is the presence of an additional peak in the uncorrected COD cross section in the area around 10.9° S. A close examination of the COD field in Figure 6a shows a maximum in this area which is around 0.1° across. There is no evidence of this in the CloudSat data, and it is believed that the satellite passed to the west of this feature. The COD cross sections in Figure A2 show the maximum optical depth around 11.3° S is lower in the corrected track, which will give a lower value in the corresponding interest field. In other areas, the differences between the corrected and uncorrected cross sections for COD and CTT are relatively small, and there will be little difference in the interest fields.

Comparison of the column maximum IWC from the CloudSat data with the corresponding HIWC Potential field from ALPHA (Figure 8) shows high values of the potential correlate well with ice water concentrations of 2 g m⁻³ and higher for the areas around 10° S and 7.5° S. The satellite data showed high values for the COD and cold CTTs in these areas. For the area around 11.3° S, the HIWC Potential of 0.8 from ALPHA implies an IWC comparable with the areas at 10° S and 7.5° S, but the CloudSat data shows that the column maximum IWC was only in the range 0.5–1.0 g m⁻³. Although the COD is significantly less in this area (Figure 6), the cloud temperatures (CTT and BT10) are cold, and it is the weighted sum of the interest fields that gives the HIWC Potential and determines the areal extent of higher values. Figure A1 implies it is almost certain that the IWC would be higher a short distance east of the track where the COD is higher and there are colder cloud temperatures.

The membership functions for the CTT and BT10 have sharp changes on the inflection points (Table 1), and if the temperature is colder than these thresholds, the HIWC Potential increases sharply. A more gradual change in the membership function across the respective thresholds may be appropriate. It must also be acknowledged there are limits in how well visible and infrared satellite data will capture the internal characteristics of clouds.

5.2. ALPHA Performance

For a more comprehensive evaluation of the HIWC product, the retrieved maximum IWC (maxIWC) from 397 daytime CloudSat passes in the period from 1 October 2019 to 31 March 2020 are aligned with the ALPHA data in location and time and processed to generate performance metrics. No corrections were made for time differences or parallax errors in this analysis, as automating this would be overly complex. The column maximum IWC from the CloudSat data are compared against the HIWC Potential for 30 km averages. The latter are computed by averaging 26 CloudSat column maxima along the flight track and the mean HIWC Potential of a 13 × 13 pixel neighbourhood around the mean CloudSat location. The 30 km average is based on the distance an aircraft would travel in 2 min at typical cruise speeds, and a previous study that showed this period is sufficient to induce anomalies in output from air data probes [6].

For the 30 km averages, Figure 9 shows a box plot of the maxIWC from the CloudSat data as a function of the column maximum HIWC Potential. Each box includes all observations within a 0.05 window for the HIWC Potential and the interquartile range (25th to 75th percentile) for the maxIWC. These are plotted at the centre of the HIWC Potential interval. The number of data points are shown for each class, and a rapid decrease is evident with increasing HIWC Potential. This should be expected given that at any specific time deep convection will generally occupy a small fraction of the domain and there will be large areas with little or no cloud.

Figure 9 shows a general increase in the median IWC as the HIWC Potential increases up to a value of 0.6. For values of Potential above 0.6, there is more variability in the associated column maximum IWC, and it is unclear if this is due to the much lower number of observations and associated sampling uncertainty, there are limits in the relationship between HIWC Potential and IWC at higher values, or there is some combination of these factors. The whiskers show scatter in the distribution, and some component of this will be a result of time differences between the H8 and CloudSat scans and parallax errors, as discussed in Section 5.1. It is believed this variability mainly occurs near the edge of clouds, as highlighted by H2020 and shown in Section 5.1.

Performance statistics for a maxIWC threshold of 1.0 g m⁻³ and several values of the HIWC Potential are shown in

Table 3. ALPHA performance statistics showing probability of detection (POD), probability of false detection (POFD), and false alarm ratio (FAR) for an IWC threshold of 1.0 g m⁻³.

HIWC Potential Threshold	POD	POFD	FAR	CSI
0.1	94%	6%	78%	21%
0.15	83%	3%	64%	33%
0.2	66%	1%	52%	39%
0.3	38%	0.3%	33%	32%
0.4	19%	0.1%	27%	18%

. Across the domain, only 1.7% of all observations showed a maxIWC ≥ 1.0 g m⁻³, confirming that only a small fraction of the cloud field has elevated IWC. These performance metrics differ from the results of H2020 that are shown in Table 1, but there are significant differences in the sampling. Results presented by H2020 are based on in situ measurements of IWC from aircraft data during field campaigns, and the authors note the data are biased toward high values of IWC as this was the objective of the campaigns. The results presented in this study are based on estimates of the column maximum IWC from 397 daytime passes of CloudSat, with 2–3 passes across the domain each day, in the 6 month period of October 2019 to March 2020 inclusive. It is also noted the ALPHA HIWC Potential was the two-input version based on satellite and NWP data only, as the operational availability of radar data across the domain is very limited.

Additional insight into the performance of ALPHA can be gained if we represent the joint probability distribution of forecasts and observations as P(Y_i,O_j), where Y_i represents the forecasts (HIWC Potential) and O_j represents the binary observation O₁ [maxIWC ≥ 1.0 g m⁻³] and O₂ [maxIWC < 1.0 g m⁻³]. The joint probability distribution can be factored in two ways [25] with the first being the calibration-refinement factorisation, as follows:

P(Y_i,O_j) = P(O_j|Y_i) P(Y_i);   i = 1, …, I; j = 1, 2,

where P(O_j|Y_i) is the conditional probability of observations given the forecast and P(Y_i) is the unconditional distribution of forecasts. The conditional probability of maxIWC ≥ 1.0 g m⁻³ for given HIWC Potential, P(O₁|Y_i), can then be presented as a function of the forecast Y_i in a calibration diagram (Figure 10). For a well-calibrated forecast, the observed probability of maxIWC ≥ 1.0 g m⁻³ would lie along the diagonal. Figure 10 shows there is a consistent increase in the probability that the maxIWC ≥ 1.0 g m⁻³ as the potential increases up to 0.6, demonstrating good resolution. There is an over-forecasting bias for values less than 0.15, and for values in the range 0.15 to 0.6 there is an under-forecasting bias where the probability that maxIWC ≥ 1.0 g m⁻³ is higher than expected for a well-calibrated forecast. For Potential values in the range 0.5 to 0.6 there is an 80−90% probability that the observed maxIWC ≥ 1.0 g m⁻³. The increased variability for potential values above 0.6 reflects the uncertainties noted in Figure 9 and indicates a review of the membership functions in the fuzzy logic might be appropriate, as discussed in Section 5.1.

The second form for the joint distribution is the likelihood-base rate factorisation, as follows:

P(Y_i,O_j) = P(Y_i|O_j) P(O_j);   i = 1, …, I; j = 1, 2,

where P(Y_i|O_j) is the distribution of conditional forecasts given the observation, and P(O_j) is the unconditional probability of HIWC observations P(O₁) and null events P(O₂). Two curves showing the probability of HIWC Potential for cases where maxIWC ≥ 1.0 g m⁻³ [P(Y_i|O₁)] and cases where maxIWC < 1.0 g m⁻³ [P(Y_i|O₂)] can then be presented as a function of the HIWC Potential (Figure 11). This shows that although there is overlap at low values, the HIWC Potential provides good discrimination of events (where maxIWC ≥ 1.0 g m⁻³) from null events (where maxIWC < 1.0 g m⁻³).

In further evaluating ALPHA, our primary interest was in areas where some cloud was present, so we only considered data points where the column maximum IWC from CloudSat was greater than 0.1 g m⁻³. This was around 8% of all the data.

Following H2020, relative operating characteristic (ROC) curves were generated from these data for three IWC thresholds, namely 0.5, 1.0, and 1.5 g m⁻³. For each IWC threshold, this was carried out by varying the threshold value of HIWC Potential that defines the likely presence of HIWC and calculating the probability of detection (POD) and probability of false detection (POFD) for the retrieved IWC [26,27]. Figure 12 shows the POD as a function of POFD for each of the IWC thresholds, namely 0.5, 1.0, and 1.5 g m⁻³, and the associated performance metric, “area under the curve (AUC)” (0.70, 0.82, and 0.86, respectively).

The optimal HIWC Potential threshold from the ROC curves is the value that minimises the distance to the top left corner, and this is 0.20. Applying this threshold for the HIWC Potential, contingency tables can be calculated for the 0.5, 1.0, and 1.5 g m⁻³ thresholds. For a potential threshold of 0.20 and IWC threshold of 1.0 g m⁻³, this gives a POD of 67%, POFD of 19%, false alarm ratio (FAR) of 53%, frequency of hits (FOH) of 47%, and critical success index (CSI) of 38%. From a user perspective, the FOH is of interest, as this is the conditional probability that the observed column maximum IWC is greater than 1.0 g m⁻³, given that the HIWC Potential is 0.20 or greater.

6. Discussion

In this paper, the reliable provision of a near-real-time, automated, and frequently updated HIWC product to stakeholders was demonstrated through a trial conducted in the period from 1 October 2019 to 31 March 2020. The trial was a collaboration between the Bureau, NCAR, FAA, and Australian-based airlines. The area of coverage extended across Indonesia, PNG, and northern Australia, a region with frequent deep convection and a relatively high incidence of ICI events. For this trial, the “two-input” version of ALPHA, based on satellite and NWP data only, was used. Apart from upper and lower bounds, this provides no information on the vertical structure of the HIWC Potential.

ALPHA was adapted to ingest data from the Bureau’s ACCESS model and the Japan Meteorological Agency Himawari-8 satellite. The output HIWC product was updated every 10 min following receipt and pre-processing of the satellite data and processing by ALPHA. There was a latency of approximately 25 min from the satellite scan start time until the product was available to users. In practice, the satellite scans across the trial domain around 5 min after the start time, so the latency is around 20 min.

The HIWC product was disseminated to stakeholders as a netCDF file and processed to enable overlay on other products, such as infrared satellite imagery and lightning data, in operational workstations. For display systems, an RGBA color map was developed to highlight increasing values of HIWC Potential. A period of intensive operations was conducted from 1 October 2019 to 31 March 2020.

To validate the HIWC product, we compared it with IWC profiles retrieved from CloudSat data, aligning the vertical cross section of IWC with the ALPHA data in location and time. Data from 397 ascending daytime passes across the domain in the operational period were used. Quantitative validation of the night-time HIWC product was not possible, as night-time radar data is not available from CloudSat following a battery failure in 2011. The CloudSat radar has a small footprint and care is required to ensure correct alignment with the H8 satellite data and HIWC Potential field, particularly near the edges of cloud features where gradients can be high, to account for time differences between scans, parallax errors, and navigation errors.

A case study from 0500 UTC, 7 January 2020, presents a detailed comparison of the HIWC Potential field with the CloudSat vertical cross section of IWC for a mesoscale convective system centred around 9° S, 135° E, north of the Northern Territory, Australia. The CloudSat radar intersected three areas where the ALPHA product indicated elevated HIWC Potential. These areas were centred around 11.3° S, 10.0° S, and 7.5° S. The vertical cross section of radar reflectivity from CloudSat shows that these three areas have elevated radar reflectivity and IWC with differences in the horizontal and vertical structure for each.

Comparison of the column maximum IWC from CloudSat with the corresponding HIWC Potential from ALPHA shows that high values of the potential correlate well with ice water concentrations of 2 g m⁻³ and higher for the areas around 10° S and 7.5° S. The satellite data also showed high values for the COD and cold temperatures (CTT and BT10) in these areas, consistent with higher IWCs.

For the area around 11.3° S, the HIWC Potential is comparable with the areas at 10° S and 7.5° S, but the CloudSat data shows the column maximum IWC was only in the range 0.5–1.0 g m⁻³. This exceeds the 0.5 g m⁻³ IWC threshold used in the development of ALPHA but is not consistent with higher values of HIWC Potential, indicating higher values for column maximum IWC. For this area the CloudSat satellite intersected the edge of the cloud feature, and although the COD is significantly lower on this flight segment, the CTT and BT10 are very cold. It is the blend of these fields that determines the areal extent of the high HIWC Potential. It is almost certain that the maximum IWC would be higher a short distance east of the CloudSat track where the COD is higher and there are colder cloud top temperatures. The fuzzy logic membership functions for the CTT and BT10 have sharp changes at the inflection points, and if the temperature is colder than these thresholds the HIWC Potential increases sharply. A more gradual change in the membership function across the inflection points should be investigated.

Although there was no CloudSat radar data for the night-time descending pass, subjective monitoring of the HIWC product showed that coherent structures were maintained across the day-night transition periods, albeit with some changes in the HIWC Potential. This also showed that convection can be quite dynamic but that local areas of elevated HIWC can persist for many hours.

As discussed by H2020, there are some limitations with ALPHA. The satellite retrievals largely represent conditions near the cloud top, the NWP model data currently does not have the spatial resolution or required skill to capture convective processes adequately, and radar coverage can be limited. For the “two-input” version of ALPHA, information on the vertical structure of IWC is limited to the lower and upper bounds, whereas there can be significant variation as illustrated in the case study from 7 January 2020. Notwithstanding this limitation, ALPHA has widespread coverage with good spatial and temporal resolution and is available in near real-time. Conduct of the ALPHA trial and validation of the HIWC product against CloudSat data provides confidence in the potential utility for providing guidance on possible ICI and the path length for flight planning, maintaining situational awareness, and flight monitoring.

The project objectives relating to the evaluation of the utility of the HIWC product by users were severely impacted by the COVID-19 pandemic, as it resulted in a sudden and severe reduction in airline operations during this trial and an inability to progress this activity. However, a 2 month study was undertaken in 2022 in the USA to evaluate the utility of the HIWC product for supporting flight operations across the CONUS [28]. The HIWC product was made available to a selected group of 17 people comprising meteorologists (airline and national weather service), flight dispatch personnel, and pilots. Participants were asked to consider the product’s utility for providing situational awareness and enabling the avoidance of HIWC conditions. The evaluation team and subject matter experts conducted interviews with each participant and asked them to complete questionnaires on the utility of the product in their decision-making during operations. Results showed that the participants found the product to be understandable and easy to use, and that they used it as supplementary information to other products that are used routinely. It also became apparent that there was limited understanding of the frequency and impact of ICI events among the group, and the absence of a formal requirement for HIWC guidance diminished the perceived need for an operational nowcasting product. Education is required to ensure a better understanding of the impact of ICI events. There is a need to consider other product formats that might be required for display in flight operations areas, on the flight deck and other potential user groups. Subject to user requirements operational procedures will also need to be developed. Good user engagement will be required to progress with these objectives.

In recent years engine manufacturers have implemented modifications in newer aircraft to mitigate the risk of ICI and to meet upgraded engine certification standards. Engine monitoring data are routinely recorded, and these should provide a valuable source of information for further validation and improvement of ICI nowcasting products. It is also noted there are currently no ICI detectors certified for commercial aircraft, and that confirmation of an ICI event, including anomalous air data probe readings, requires a detailed post-event analysis, for which resources are limited. There is a belief the true number of ICI events, including anomalous air data probe readings, is significantly higher than currently reported, and, as such, the development and certification of an ICI detector for commercial aircraft should be a priority. Establishing international arrangements whereby these data can be shared for the further development of HIWC guidance products would be of great value.

Under the ICAO Global Air Navigation Plan (GANP) there are moves towards a data-centric model for air traffic management (ATM) products, including weather data, on a range of time and space scales to support more automation. Provision of an automated near real-time, gridded HIWC product with a relatively high temporal and spatial resolution as demonstrated in this trial is consistent with this objective. This would enable the provision of a guidance product to flight operations groups, as well as to the flight deck.