Clutter Elimination Algorithm for Non-Precipitation Echo of Radar Data Considering Meteorological and Observational Properties in Polarimetric Measurements

Abstract

Non-precipitation echoes due to ground and sea clutter, chaff, anomalous propagation, biological targets, and interference in weather radar observations are major issues causing a decline in the accuracy of meteorological and hydrological applications based on radar data. Statistically based quality control techniques using polarimetric variables have improved the accuracy of radar echo classification, however their performance is affected by attenuation, nonuniform beam filling, and hydrometeor diversity as well as terrain blockage, beam broadening, and noise correction issues due to the quality degradation of polarimetric measurements. To address this, a new quality control algorithm, named clutter elimination algorithm for non-precipitation echo of radar data (CLEANER), was designed by employing independent feature parameters and variable classification conditions with spatial and temporal observation environments to adapt to these meteorological artifacts and observational limitations. CLEANER was applied to several precipitation cases with various non-precipitation echoes, showing improved performance compared with results from the fuzzy logic-based quality control algorithm in terms of non-precipitation echo removal as well as in precipitation echo conservation. In addition, CLEANER shows better computational efficiency and robustness, as well as an excellent expandability for different radar networks.

1. Introduction

Non-precipitation echoes (NPEs), such as ground and sea clutter (GC and SC, respectively), chaff (CF), biological target (BI), clutter by anomalous propagation (AP) and interference (IF) are frequently observed with precipitation echo (PRE); the intensity of NPE and its observed area tend to change depending on the surrounding environment in time and space in weather radar observations. These NPEs affect the accuracy of radar-based products for meteorological and hydrological applications, especially the accumulation rainfall amounts, depending on the performance of the quality control (QC) technique. Dual-polarization radar has led to remarkable achievements in terms of the classification of radar signals [1,2,3,4], as well as in microphysical and quantitative applications such as rainfall estimation [5,6,7] and data assimilation [8,9].

Most radar echo classification algorithms have been developed based on a probabilistic approach using supervised learning for polarimetric variables (reflectivity, $Z_H$; differential reflectivity, $Z_{DR}$; differential phase shift, ${\Phi }_{DP}$; and correlation coefficient, ${\rho }_{HV}$) and their spatial fluctuations to discriminate between PRE and NPE in real time [10,11,12,13]. These dynamic filtering algorithms using neural networks or fuzzy logic classification have shown generally good performance in the classification of a typical type of PRE and NPE. However, these statistically-based QC techniques may fail to identify severe NPE areas, such as GC overlapped with PRE, abnormal SC and AP, due to super refraction or ducting because of the distributions of their feature parameters for severe NPE do not partially follow that for usual NPE. Furthermore, the quality of polarimetric measurements changes over time due to beam blockage, attenuation, beam broadening with a range from radar, nonuniform beam filling (NBF), hydrometeor diversity within a sample volume, and noise correction issues at low signal-to-noise ratio (SNR) [14,15,16,17,18]. These statistical properties of polarimetric variables and physical limitations of radar observation may finally result in the performance degradation of statistically based QC techniques. Ref. [19] developed a physically-based QC scheme based on meteorological principles using $Z_H$, ${\rho }_{HV}$ and atmospheric thermodynamic fields to conserve atypical PRE, which showed a better skill score than that of the neural network QC algorithm.

This study aims to develop an operational QC algorithm for South Korean weather radar networks that is minimally affected by the quality of polarimetric variables, observational properties and environment. The new QC technique was named clutter elimination algorithm for non-precipitation echo of radar data (CLEANER), which has several dynamic filters designed by considering various radar artifacts to improve the classification performance between severe NPE and atypical PRE. Section 2 introduces the concept and characteristics of CLEANER with operational radar networks. The methodology is described in Section 3, and CLEANER is compared with the fuzzy logic-based QC (FQC) algorithm in Section 4. Conclusions and remarks are summarized in Section 5.

2. CLEANER

Weather radar observations in Korea are significantly contaminated by GC, SC and CF, and affected by beam blockage and seasonal variation in the hydrometeor because the Korean peninsula, located in the middle latitudes, has mountainous terrain up to 70% (Figure 1) and frequent air force flight training. In addition, AP by super refraction and ducting at coastal radar sites are frequently caused by the inflow of moist air from the sea and large daily temperature changes [20] that often remain even after applying a statistically based QC algorithm. The weather radar center (WRC) of the Korea meteorological administration (KMA) has operated ten Doppler weather radars, which were all upgraded to dual-polarization radar from 2014 to 2019. The volume scan (VCP), with nine elevation angles, and a long-range scan (LNG) with one low-elevation angle, are performed every 5 min with the radar observation characteristics listed in

Table 1. Characteristics of the S-band dual-polarization radars operated by the KMA.

Parameters	Values
	VCP	LNG
# of site	10
Dual polarization	Simultaneous H and V
Band	S-band
Beam width	0.95°
Pulse width	2 μs	4.5 μs
PRF	620–1200 Hz with elev.	306 Hz
Observational range	240 km	480 km
Range resolution	250 m
Elevation angles	9 (−0.2 to 15°)	1 (0.0°)
Scan duration	224 s	40 s

. As the other three departments of the Korean government have operated their own polarimetric radars with different observation characteristics for different purposes (e.g., five S-band by the flood control office, six C-band by the Air Force and one C-band by the Navy), WRC has made efforts to provide high-quality weather information from polarimetric radars by developing QC techniques that can encompass all the different radar networks.

CLEANER, which began to be developed as a sweep-based radar QC algorithm in 2017, has been continuously improved to classify all types of NPEs from PREs that are reflected from all types of precipitation particles more clearly, even if the PRE is affected by a melting layer or nonuniform beam filling, except for range-folding echo. CLEANER has been applied to the operational system of the Korean weather radar networks since 2019. The followings are the characteristics of CLEANER: (1) employs new feature parameters that has different properties with generally-used feature parameters based on statistical characteristics; (2) basically applies individual classification conditions and constraints for each feature parameter; and (3) customizes each condition spatially and temporally in real time by considering the quality of polarimetric variables, observational properties and environment. New feature parameters were based on meteorological characteristics between polarimetric variables, and each classification condition; constraints were determined to minimize misidentification for PRE. For the development of CLEANER, Korean operational radar data collected from 2016 to 2020 were analyzed to comprehend the meteorological and observational characteristics of polarimetric variables for PRE and NPE that were distinguished using polarimetric observations and their statistical quality, clustered-echo shapes, moving tendency of the echo, vertical structure of the echo and comparisons with other radar data, surface observations, and satellite data at the same time.

CLEANER consists of four main processes applied sequentially for real-time operation and two sub-steps to automatically reproduce the GC static map and beam blockage mask with a renewed radar scan strategy (Figure 2). The main procedure includes (1) the identification of GC using traditional radar moments (unfiltered reflectivity, $Z_H{}^U$; clutter-filtered reflectivity, $Z_H{}^F$; and radial velocity, $V_r$), (2) real-time offset determination and unfolding of ${\Phi }_{DP}$, (3) classification of NPE and (4) residual NPE removal. This sequential flow considers the state of Korean operational radar data, in which the dual-polarization measurements in GC areas are mostly lost by a clutter filter in signal processing (except for KMA radar systems after April 2020), especially when there is no PRE. The sub-steps provide a static GC map and simulated beam blockage fraction (PBB) corresponding to each radar scan strategy, which are used as feature parameter and correction factors in the first and third steps of the main procedure, respectively. The details of the CLEANER methodology are described in Section 3.

3. Methods

3.1. Identification of Ground Clutters

GCs returned from surface targets, such as mountains, bridges and ships, have mostly high reflectivity, which rapidly decreases at its boundary. GCs can be suppressed by clutter filters, such as the infinite impulse response (IIR) or Gaussian model adaptive processing (GMAP) filters in signal processing but may be partially retained when they have a radial velocity due to moving targets (e.g., vegetation, ships and vehicles). Furthermore, GC overlapping with PRE above a moderate rainfall rate is difficult to suppress completely by a clutter filter; these filters weaken the intensity of PRE signals and make polarimetric measurements abnormal in an area with near-zero Doppler velocity. These misclassifications cause significant errors in the quantitative precipitation estimation, especially in accumulative rainfall.

Generally, techniques to detect GC using radar moments use static GC maps or dynamic feature parameters. The static map can correctly implement the usual GC area, but cannot respond to the variations in the clutter areas and their intensity due to super refraction or the renewal of radar scan strategy. The dynamic feature parameters are more adaptive for identifying the GC. However, the radial texture ($TXT$) (i.e., the standard deviation within a given range interval in a radial direction) of the polarimetric variables generally used in GC identification has lower performance in the center of larger-clustered clutter than that of the range interval of $TXT$ and weak GC with low spatial fluctuation; it may also misidentify some PRE in contact with GC or in the boundary of a strong convective storm. Hence, this study designed a new GC identification module using $Z_H{}^U$, $Z_H{}^F$, and $V_r$ for optimization with the Korean radar systems, even for a single polarization Doppler radar system. The GC identification module defined new feature parameters and merged dynamic and static identification techniques to complement the weaknesses of each technique for operational applications.

3.1.1. Feature Parameters for Ground Clutter

New dynamic feature parameters were developed based on the effects due to clutter filtering and the “towering” shape of GC with higher reflectivity than that of the surroundings ($i$) in the $Z_H{}^U$ field. Herein, $\Delta Z_{Filter}$ is defined as the difference between $Z_H{}^U$ and the $Z_H{}^F$, and $Tower$ and $\Delta Tower$ are defined as the deviation with lower reflectivity within a given range interval and as the difference between $Tower$s of $Z_H{}^U$ and $Z_H{}^F$, respectively, as follows:

$$\Delta Z_{Filter}\left(i\right)=Z_H{}^U\left(i\right)-Z_H{}^F\left(i\right),$$

(1)

$$Towe{r}^U\left(i\right)=\sqrt{\sum {\left[Z_H{}^U\left(i\right)-Z_H{}^U\left(i+j\right)\right]}^2/n}\mathrm{if}\left[Z_H{}^U\left(i\right)-Z_H{}^U\left(i+j\right)\right]>5\mathrm{dB},$$

(2)

$$Towe{r}^F\left(i\right)=\sqrt{\sum {\left[Z_H{}^F\left(i\right)-Z_H{}^F\left(i+j\right)\right]}^2/n}\mathrm{if}\left[Z_H{}^F\left(i\right)-Z_H{}^F\left(i+j\right)\right]>5\mathrm{dB},$$

(3)

$$\mathrm{and}\Delta Tower\left(i\right)=Towe{r}^U\left(i\right)-Towe{r}^F\left(i\right),$$

(4)

where $i$ and $j$ indicate the index number in each radial, and $j$ changes within a given range interval that decrease from 10 to 3 km with the elevation angle to consider the range scale of GC clusters. $Tower$ is calculated with a tolerance of 5 dB to consider a statistical fluctuation and $n$ is the number of available data within the range interval. In addition, $Tower$ with a 2 km range interval is used to identify small-scale clutters, such as ships and bridges.

The static GC map is an assistive feature parameter for larger-clustered GC than the range interval of $Tower$ or low-reflectivity GC that is indistinguishable in the dynamic feature parameters. The static map is defined with two fields: GC detection frequency ($GCF$) and GC mean reflectivity ($GCZ$). These are constructed using $Z_H{}^U$ observations for greater reflectivity than 20 dBZ during an hour on a clear day and reproduced when the radar scan strategy (i.e., elevation angles or spatial resolution) is changed. This short counting time interval enables the rapid updating of the static map in response to the renewal of the scan strategy in operation. Herein, the clear day is the day when there are few echoes, except for GC, and it is discernible with few echoes in the $Z_H{}^F$ field.

3.1.2. Characteristics of the Feature Parameters and Identification for Ground Clutter

Figure 3 shows an example of $Z_H{}^U$, $Z_H{}^F$, and $V_r$ and the feature parameters for GC identification for the super refraction case accompanied by PRE from the west. $Tower$-based parameters are effective in the GC boundary and a smaller size clutter than the range interval of $Tower$; these are ineffective in the center of bigger clustered-clutter than the range interval of $Tower$ (yellow box in Figure 3) and weak GC with a low spatial fluctuation and low Doppler filter effect (white box in Figure 3). $\Delta Z_{Filter}$ is a good classifier in stationary GC with no or weak precipitation regardless of the spatial size of GC, which may incorrectly identify some precipitation areas near zero $V_r$ as clutter (purple box in Figure 3). $GCF$ works well in most GC regardless of intensity; however, it cannot identify GC by super refraction in the far range (red box in Figure 3).

GC echoes are dynamically identified from PRE or other NPE if any of the feature parameters independently exceeds each threshold value in

Table 2. Feature parameters and identification conditions for ground clutter.

Parameters		Constraints	Thresholds [dBZ]
$Towe{r}^U$	$\mathrm{interval}$ ~10 km	$Z_H{}^U>15\mathrm{dBZ}$	$>45.0+0.1Z_H{}^U$
	$\mathrm{interval}$ = 2 km	$Z_H{}^U>30\mathrm{dBZ}$	$>30.0+2.0\varphi$
		$\varphi <0.3°$ and $Z_H{}^U>55\mathrm{dBZ}$	$>12.5+0.1Z_H{}^U$
$\Delta Tower$		$Z_H{}^U>15\mathrm{dBZ}$	$>5.0$
$\Delta Z_{Filter}$		$Z_H{}^U>15\mathrm{dBZ}$	$>20.0$
		$\varphi <3°$ and $Z_H{}^U>25\mathrm{dBZ}$	$>20.0-{\left(Z_H{}^U/20.0\right)}^{2.3}-\left|V_r\right|$ (min = 7.5, max = 15.0)
$GCZ$		$\varphi <5°$ and $Z_H{}^U>15\mathrm{dBZ}$ & $h<2\mathrm{km}$ and $GCF>90\%$	$>Z_H{}^U-5.0$

to compensate for the weakness of each feature parameter. Each threshold value was empirically determined to a level that minimizes the misidentification for PRE. The static GC map (Figure 3g,h) specifies regular GC areas when $GCF$ exceeds 90%, and adaptively identifies GC echoes using the threshold value adjusted by the difference between $GCZ$ and $Z_H{}^U$ at a low elevation angle ($\varphi$) and low altitude ($h$). This adaptive condition for the static map can identify large-clustered GC areas from no or weaker PRE, and does not classify weak GC when strong PRE is overlapped.

3.2. Processes for Differential Phase Shift

3.2.1. Offset Determination in Real Time

The offset of ${\Phi }_{DP}$ can be changed by a radar system check for maintenance. System maintenance is usually performed on clear days, such that it is difficult to properly set the offset of ${\Phi }_{DP}$ (${\Phi }_{DP}{}^{OFF}$) for PRE. In addition, the offset value often changes after initially observing the precipitation echo by radar operators when setting an improperly high or low offset. Thus, ${\Phi }_{DP}{}^{OFF}$ real-time determination is required prior to the quantitative use of ${\Phi }_{DP}$ measurements.

This study initially determined the ${\Phi }_{DP}{}^{OFF}$ of each volume scan from the continuous PRE area. The ${\Phi }_{DP}{}^{OFF}$ is calculated by averaging the offset values of each elevation angle. The offset values of each elevation angle are calculated by averaging the ${\Phi }_{DP}$ values extracted from radially continuous PRE. Here, the continuous PRE is determined with high ${\rho }_{HV}$ and low $TXT$ values from polarimetric variables before meeting the convective cell (>35 dBZ) within a 20 km range of the PRE path for each radial. If a continuous PRE area does not exist, the ${\Phi }_{DP}{}^{OFF}$ is extracted. Second, from the AP echo area that has soft spatial textures of polarimetric fields, we searched for a minimum value of the averaged ${\Phi }_{DP}$ values within a 20 km range interval at elevations lower than 2.5°. The AP area is limited with the following conditions to exclude other NPEs such as SC, BI, CF, and interference: (1) 0 < $Z_H$ < 30 dBZ; (2) |$V_r$| > 5 ms⁻¹; (3) 0 < $Z_{DR }$< 5.0 dB; (4) 0.5 < ${\rho }_{HV }$< 0.92; (5) $TX{T}_{{\Phi }_{DP }}$< 30°; (6) $TX{T}_{Z_{DR}}$< 4 dB; and (7) $TX{T}_{{\rho }_{HV }}$< 0.3. This study defined $TX{T}_X$ of the polarimetric variable ($X$) as the radial root mean square differences. $TXT$s of the polarimetric variables are calculated as follows:

$$TX{T}_X\left(i\right)=\sqrt{{{\displaystyle \sum }}_j{\left[X\left(i\right)-\mathrm{X}\left(i+j\right)\right]}^2/n},$$

(5)

where $i$ and $j$ indicate the index number in each radial, $j$ changes within a given range interval and $n$ is the number of available data within the range interval. The range intervals adaptively increased up to 1.75 km with increasing reflectivity (minimum is 3 gates) to reduce NPE-neighboring contamination because weaker precipitation echoes have a higher probability of contact with NPE and vice versa.

Figure 4 shows the time series of the Φ_DP offset and the areal ratio of the PRE and NPE. The offset from PRE is highly stable (red line in Figure 4) whereas the offset from NPE (yellow line in Figure 4) is similar to that from PRE. Herein, numerous far-range GCs were detected accompanying high AP areal ratios and gradually decreased as precipitation approached the radar site. This indicates that the detection number of a far-range GC can provide some guidelines for the AP-dominant observation situation.

3.2.2. Unfolding for Multi- and Inverse-Folding

The ${\Phi }_{DP}$ measurements were produced from 0° to 360° (or 180° with data-stored resolution) as phase-based variables. When strong convective cells lie in radial directions, the ${\Phi }_{DP}$ rapidly increases over 360°, and the folding occurs, even in S-band radar (Figure 5a). In contrast, when the ${\Phi }_{DP}$ offset value is near zero, ${\Phi }_{DP}$ folding occurs inversely in the boundary of the PRE (Figure 5c).

The ${\Phi }_{DP}$ unfolding technique that uses the radial texture of ${\Phi }_{DP}$ as a folding determinant parameter may be miscalculated in the area where folding and unfolding occurs alternately. This study developed an ${\Phi }_{DP}$ unfolding technique based on the difference between the observed ${\Phi }_{DP}$ (${\Phi }_{DP}{}^{OBS}$) and reference ${\Phi }_{DP}$ (${\Phi }_{DP}{}^{REF}$) by following seven steps with each radial. The ${\Phi }_{DP}{}^{REF}$ is the ${\Phi }_{DP}$ of the prePRE bin, where the PRE bin is updated during each radial unfolding using random error ($E\left({\Phi }_{DP}\right)$) within a 1.5 km range interval.

• We set the maximal-valid increment considering the maximal specific differential phase shift with radar wavelength and also set the valid fluctuation amount every bin with data quality affected by the wavelength and total-valid ${\Phi }_{DP}$ difference ($\Delta {\Phi }_{DP}{}^{REF-OFF}={\Phi }_{DP}{}^{REF}-{\Phi }_{DP}{}^{OFF}$) due to PRE.
• Set initial ${\Phi }_{DP}{}^{REF}$ to ${\Phi }_{DP}{}^{OFF}$
• Add the folding amount ($\Delta folding,360°\mathrm{or}180°$) to ${\Phi }_{DP}{}^{OBS}$ if the difference between ${\Phi }_{DP}{}^{OBS}$ and ${\Phi }_{DP}{}^{REF}$($\Delta {\Phi }_{DP}{}^{OBS-REF}$) is smaller than–75% of $\Delta folding$
• Subtract $\Delta folding$ to ${\Phi }_{DP}{}^{OBS}$ if $\Delta {\Phi }_{DP}{}^{OBS-REF}$ is larger than 75% of $\Delta folding$
• Save step 3 or 4 as unfolded ${\Phi }_{DP}$(${\Phi }_{DP}{}^{UNF}$) if the absolute value of the difference between ${\Phi }_{DP}{}^{UNF}$ and ${\Phi }_{DP}{}^{REF}$($\left|\Delta {\Phi }_{DP}{}^{UNF-REF}\right|$) is smaller than $\left|\Delta {\Phi }_{DP}{}^{OBS-REF}\right|$
• Update ${\Phi }_{DP}{}^{REF}$ with the averaged value between ${\Phi }_{DP}{}^{REF}$ and processed ${\Phi }_{DP}$ by unfolding if the bin is PRE according to the following conditions: (i) ${\Phi }_{DP}$ larger than ${\Phi }_{DP}{}^{REF}$ and ${\Phi }_{DP}{}^{OFF}$; (ii) difference between ${\Phi }_{DP}$ and ${\Phi }_{DP}{}^{REF}$ is larger than –50% of $E\left({\Phi }_{DP}\right)$ and smaller than (valid increment $+E\left({\Phi }_{DP}\right)$); and (iii) $E\left({\Phi }_{DP}\right)$ is smaller than the valid variation.
• Repeat step 3 $n$ times until ${\Phi }_{DP}{}^{REF}$ is smaller than 75% of ($\Delta folding\times n+{\Phi }_{DP}{}^{OFF}$) to unfold the multi-folding (for a C- or X-band radar system).

Figure 5d–f shows the ${\Phi }_{DP}$ fields after applying the unfolding procedure developed in this study. The left panel shows the folding case due to the convective precipitation system with hail in the S-band radar, middle panel shows the multi-folding three times in the C-band radar, and right panel shows the inverse-folding due to near-zero offset in the winter season. All types of ${\Phi }_{DP}$ folding were completely unfolded, and this unfolding algorithm showed a 99% success rate for S-, C- and X-band radar systems in long-term applications. Through this unfolding, the ${\Phi }_{DP}$ can be used to classify NPE as a feature parameter both quantitatively and statistically.

3.3. Classification of Precipitation and Non-Precipitation Echoes

Figure 6a–f (and Figure 5d) show an example of polarimetric measurements and their $TXT$s of a precipitation case with hail and NBF. The hail particles with low values of ${\rho }_{HV}$ and $Z_{DR}$ were present near the radar site and NBF area with low values of ${\rho }_{HV}$ and $Z_{DR}$; high $TXT$s for the polarimetric variables were observed along with the hail and convective cells at 10–15° azimuthal direction, yielding ${\Phi }_{DP}$ in folding. The hydrometeors changed to snow and ice crystals above a height of 2.5 km (approximately 180 km range). SC was observed within a 40 km range, and weak AP were also observed over 90–300° azimuthal directions. The SC is well identified from PRE in polarimetric fields, but some AP areas in the south direction are difficult with respect to classification from PRE because the $TXT$s and ${\rho }_{HV}$ values in the AP area are not different from hail, NBF and ice crystals.

This study defined new feature parameters based on the consistency between polarimetric variables, considering their meteorological distributions, which were used to completely classify PRE-NPE with $TXT$s and ${\rho }_{HV}$. This Section describes the feature parameters, their characteristics and classification conditions for NPE. The observed $Z_H$ was corrected with wet-radome attenuation for each elevation angle as reported in [21], partial beam blockage fraction ($BK$) simulated with a digital elevation model (DEM), and path-integrated attenuation ($PIA$) estimated PRE-NPE classification from $Z_H$ as reported in [22].

3.3.1. Set of Maximum Reflectivity for Non-Precipitation Echo

NPEs, especially AP, which is difficult to classify from PRE, are strongly observed at low observational elevation angles. Figure 7 shows the two different distributions of the maximum-observed reflectivity at each elevation angle ($\varphi$) after GC removal for mostly strong AP and hail-PRE cases. In contrast to the PRE case with hail, the maximum reflectivity over 70 dBZ at low elevation in the strong AP case due to ducting rapidly decreases to 30 dBZ with an increasing elevation angle. Therefore, setting the maximum reflectivity of the NPEs ($Z_{MAX}{}^{NPE}$) is effective in preventing the misclassification of the PREs, especially with an increase in the observational elevation angle. The $Z_{MAX}{}^{NPE}$ can be applied as a function of $\varphi$ or as a table of discontinuous values for each $\varphi$. CLEANER empirically sets $Z_{MAX}{}^{NPE}$ as a function of $\varphi$ (dashed line in Figure 7) as follows:

$$Z_{MAX}{}^{NPE}\left(\varphi \right)=30.0+\frac{70.0}{{1.5}^{\varphi +1.0}}.$$

(6)

This excludes the possibility of misidentification of the BB and hail area at high elevation angles (blue shaded area in Figure 7).

3.3.2. Feature Parameters and Their Characteristics for Non-Precipitation Echo

The radial $TXT$s of the polarimetric variables within a specific range interval (e.g., usually 1–2 km) have good performance for SC and CF because they mostly have high spatial fluctuations within the range interval of the $TXT$. However, $TXT$s may have low performance in strong SC by ducting where the spatial size of the fluctuation is valid on a larger spatial scale than the range interval of the $TXT$; in the precipitation boundary, they may be contaminated by the fluctuations of neighboring NPEs as the range interval is larger. In this section, $TX{T}_X$ of the polarimetric variable ($X$) is calculated using equation (5).

As the quality of ${\rho }_{HV}$ data also becomes worse in hail, NBF and BB similar to the $TXT$s of polarimetric variables, ${\rho }_{HV}$ can safely classify only weak NPE (Figure 6b). Therefore, a constant threshold value for ${\rho }_{HV}$ may cause incorrect PRE-NPE classification. CLEANER defined a reflectivity-weighted coefficient (${\rho }_{Z_H}$) by converting the observed $Z_H$ to a value from 0 to 1 using a convergent function as follows:

$${\rho }_{Z_H}\left(i\right)=1-{\mathrm{e}}^{-t\left(i\right)/T}\mathrm{and}t\left(\mathrm{i}\right)=\left[Z_H\left(i\right)+30.0\right]\times {\rho }_{HV}\left(i\right)+PIA\left(i\right),$$

(7)

where $PIA$ for $Z_H$ is corrected, and the observed ${\rho }_{HV}$ is used as a weighting factor. Herein, the increment of 30 dBZ for $Z_H$ considers a negative dBZ and $T$ is 20.0 (${\rho }_{Z_H}$ can reach 0.993 when $t$ is 5 T). This function is similar to the $Z_H$-${\rho }_{HV}$ distribution of precipitation echoes collected from Korean radar data that has the noise correction issue at low SNR [18]. The low threshold value for ${\rho }_{Z_H}$ can remove only weak NPE with low $Z_H$ and low ${\rho }_{HV}$, while preserving atypical PREs (see Figure 6g).

$Z_{DR}$ is not a general classification parameter because the range of $Z_{DR}$ for NPE is very wide, ranging from –8 to 8 dB, which covers the range of PRE. Figure 6c shows that both PRE with big-drop and BI are high, $Z_{DR}$ and both NBF and SC are small (negative) $Z_{DR}$ values. The high $Z_{DR}$ of NPE with weak $Z_H$ is outside of the $Z_H$–$Z_{DR}$ distribution for raindrops, which is a proportional relation, as confirmed by scattering simulation and radar measurements. Thus, this study defined the ratio between one-tenth of $Z_H$ and $Z_{DR}$ ($Z_{Ratio}$) at a linear scale as a feature parameter for weak AP and BI, as follows:

$$Z_{Ratio}\left(i\right)=10{\log }_{10}\frac{{10}^{0.1\times Z_{DR}\left(i\right)}}{{10}^{0.1\left[Z_H\left(i\right)/10\right]}}=Z_{DR}\left(i\right)-Z_H\left(i\right)/10.$$

(8)

Figure 6h shows the $Z_{Ratio}$ in a field that is mostly higher than 3.0 in NPE. Herein, the system bias of $Z_{DR}$ can properly be adjusted by the $Z_H$–$Z_{DR}$ mean relation from the drop size distribution data within the weak rain in real time.

${\Phi }_{DP}$ can effectively segregate NPE, especially AP and the interference echo, from PRE when $PIA$ is small because ${\Phi }_{DP}$ has a valid increment with the range due to only PRE. For example, in Figure 5d and Figure 6i, the offset value of ${\Phi }_{DP}$ was calculated approximately 75° based on Section 3.2.2; the reflectivity-based $PIA$ was bigger than 1.0 dB only in the north direction. This indicates that the large ${\Phi }_{DP}$ increment from the offset value is invalid except in the north direction just at that time. Thus, the difference between the observed ${\Phi }_{DP}$ and its offset value ($\Delta {\Phi }_{DP}$) can be used as a new feature parameter compared with reflectivity-based $PIA$:

$$\Delta {\Phi }_{DP}\left(r\right)={\Phi }_{DP}\left(r\right)-{\Phi }_{DP}{}^{OFF}.$$

(9)

3.3.3. Classification Conditions for Non-Precipitation Echo

The quality of polarimetric measurements is low when the correlation between hydrometeors within the radar sample volume decreases. In other words, the spatial fluctuations of polarimetric variables for mixed targets such as hail, wet snow and ice crystals, are high with ${\rho }_{HV}$ reduction. This quality degradation is also related to NBF, beam broadening with range, and noise correction issues at low SNR [14,15,16,17,18]. Furthermore, NPE is observed frequently and intensely at low elevations; thus, changeable classification conditions with an increasing observation range and height are more effective for NPE removal and PRE conservation than unchangeable conditions. Similar to the concept of the GC identification module in Section 3.1, NPEs, except for GC echo, are classified from PRE if any of the feature parameters except for ${\rho }_{HV}$ and ${\rho }_{Z_H}$, independently exceeds each threshold value with its adjustment in

Table 3. Feature parameters and classification conditions for non-precipitation echoes.

Parameters	Constraints	Initial Thresholds	Threshold Adjustments
$TX{T}_{Z_{DR}}$	$PIA<1.5-0.3\varphi$	$>10.0-0.1Z_H$	$+5.0\left(r^2/r_{Max}{}^2\right)\left(h/H_{Top}\right)$
$TX{T}_{{\Phi }_{DP}}$	$PIA<1.5-0.3\varphi$ and $Z_H<25.0 \mathrm{dBZ}$	$>50.0$	$+\left(20.0+10.0PIA\right)\left(r^2/r_{Max}{}^2\right)\left(h/H_{Top}\right)$
$TX{T}_{{\rho }_{HV}}$	$PIA<2.0$ and ${\rho }_{HV}<0.7$ and $Z_H<30.0 \mathrm{dBZ}$	$>0.3-0.05PIA$	$+0.2\left(r^2/r_{Max}{}^2\right)\left(h/H_{Top}\right)$
${\rho }_{HV}$	$PIA<1.5$	$<0.6-0.2BK$	$-\left(0.6-0.03\varphi \right)\left(r^2/r_{Max}{}^2\right)\left(h/H_{Top}\right)$
${\rho }_{Z_H}$			$-0.6\left(r^2/r_{Max}{}^2\right)\left(h/H_{Top}\right)$
$Z_{DR}$	$PIA<0.5$ and ${\rho }_{HV}<0.92$	$<-2.5$	-
	no winter season	$>7.5+0.1\varphi$	-
$Z_{Ratio}$		$>2.5+0.5\varphi$	$+\left(5.0-0.25\varphi \right)\left(r^2/r_{Max}{}^2\right)\left(h/H_{Top}\right)$
$\Delta {\Phi }_{DP}$	$PIA<1.0-0.5BK$ and ${\rho }_{Z_H}<0.85$	$>40.0$	$+75.0PIA+2.0h$

. In contrast, ${\rho }_{HV}$ and ${\rho }_{Z_H}$ work when they fail to reach each threshold value. The threshold values are adjusted with the $PIA$, observation range ($r$) and height ($h$), observational elevation angle ($\varphi$) and maximum range ($r_{MAX}$) of the radar scan, and seasonal echo top height ($H_{Top}$ varying from 3 to 12 km) to consider beam broadening and vertical distribution of the hydrometeor. The constraints determine a boundary such that each feature parameter can work for every observation point with $Z_{MAX}{}^{NPE}$ (not described in Table 3, but applied, except for $Z_{DR}$–related classification conditions) to conserve PREs in NBF, attenuated areas, and the melting layer. The initial threshold values were also empirically determined to be sufficient to preserve atypical PREs such as hail, NBF, BB and ice crystals. In addition, when severe AP due to super refraction or beam ducting occurs, some threshold values can be adjusted more strictly with the NPE areal ratio near the radar site and the detection ratio of far-range GC to completely remove indistinguishable AP echoes.

4. Results

This section describes the results of NPE removal for various types of radar echoes through CLEANER by comparison with the FQC. The FQC algorithm was developed in 2015 using TXTs of polarimetric variables, ${\rho }_{HV}$, and $Z_H$ as the feature parameters as following [23] with the optimization process for each radar dataset.

4.1. Ground Clutters

Figure 8 shows two cases of $GCF$ for the static GC map and 1-h mean reflectivity with large-clustered clutter within 100 km range and far-range clutter due to super refraction, respectively. In this comparison, the FQC worked for all types of NPE and CLEANER only applied the GC identification step (the GC area is filled with filtered reflectivity with low $TXT$ of $Z_H{}^F$). The FQC failed to identify some GC areas overlapped with PRE (red lines in Figure 8) or weak GC near the radar site (black lines in Figure 8). The remaining GC in Figure 8c,g causes the overestimation in rainfall estimation. The dynamic and static identification of CLEANER completely identified large-clustered clutters, far range GC by super refraction, and even weak GC near the radar site (Figure 8d,h).

4.2. Anomalous Propagations

Figure 9 shows two different types of severe AP cases with ducting (top) and super refraction (middle and bottom). The FQC identified the GC even over the 200 km range. However, notably, the FQC left NPE by ship and AP echo because polarimetric variables had soft spatial textures or the ${\rho }_{HV}$ was not low enough to be classified as NPE (not shown here). CLEANER separated most of the AP from the PRE, which was mainly effective using invalid ${\Phi }_{DP}$ increments and applying additional classification conditions that are enhanced with the PRE/NPE areal ratio or the detection number of far-range GCs over 200 km.

4.3. Hail and Nonuniform Beam Filling

When hail or graupel particles exist in the radar sample volume, ${\rho }_{HV}$ can be decreased to approximately 0.8, and the $TXT$s of polarimetric variables can be increased in the hail boundary (Figure 10d). In Figure 10c, CLEANER correctly preserved the hail area and removed SC and AP while the FQC misidentified the hail and their boundaries and retained some AP echoes (Figure 10b).

NBF with quality degradation of the polarimetric fields is commonly observed when convective storms are lined up with the long radial direction and/or sampled with a relatively large beam width. This study separates the NBF area with reflectivity-based $PIA$ (because NBF is associated with high-reflectivity PRE) and applies relaxed classification conditions to the NBF area rather than to other areas. The high $PIA$ direction was consistent with the NBF and large ${\Phi }_{DP}$ area; CLEANER mostly conserved the PRE in the NBF area, in contrast to the FQC results (Figure 10f,g).

4.4. Bright Band, Snow and Ice Crystals

Figure 11 shows prominent BB cases of low and high elevation angles for different seasons and a snow case in a coastal area. CLEANER mostly conserved the BB area and even ice crystals using a variable threshold value with the beam height and seasonal echo top height, contrary to the FQC results. Ice crystals are usually observed with a high $Z_{DR}$ value, low ${\rho }_{HV}$ value and high spatial fluctuation in the polarimetric measurements to a level similar to NPE, especially the $Z_{DR}$ field. CLEANER excepted or relaxed the $Z_{DR}$-related classification conditions for the winter season (December, January and February) to classify ice crystals as PRE more correctly.

4.5. Applications to C- and X-band Polarimetric Radars

CLEANER was designed to overcome the quality degradation of polarimetric measurements due to meteorological and observational properties through physically-based feature parameters and adaptive classification conditions to various error factors, such as beam blockage, attenuation, beam broadening, hydrometeor and NBF in real time. Thus, this new QC algorithm can be applied to other radar systems of shorter wavelengths, i.e., C- or X-band, because they have similar meteorological and observational properties in polarimetric measurements for PRE and NPE as the S-band radar. In particular, $\Delta {\Phi }_{DP}$ more rapidly increases for shorter wavelength, but is still available in C- and X-band because $PIA$ used as adjustment or constraint factor also increases larger than S-band radar by applying a proper $Z_H$-$A_H$ relation for each radar wavelength. Figure 12 is the result for the C-band polarimetric radar of the air force and for the X-band polarimetric radar of KMA for research. In both cases, large ${\Phi }_{DP}$ increments were observed with polarimetric quality degradations due to BB and attenuation. CLEANER effectively classified GC and IF from PRE without additional optimization.

5. Conclusions

CLEANER is a new quality control algorithm designed to effectively separate all types of NPEs (except range-folding echo) from PRE for operational radar systems in Korea. The new feature parameters were based on the characteristics of ground clutter in the reflectivity field and meteorological consistency between polarimetric variables, and each parameter with different characteristics effectively classified a specific NPE. The classification conditions for each parameter independently worked and flexibly changed to consider meteorological artifacts such as NBF, attenuation and hydrometeor type and observational limitations such as terrain blockage, beam broadening, abnormal refraction and noise correction issues with low SNR.

The results of NPE removal by CLEANER showed better performance in various PRE and NPE cases than the results from the fuzzy logic-based algorithm in terms of both NPE removal and PRE conservation. The datasets collected by other radar networks with different radar wavelengths and scan strategies were also successfully processed by CLEANER without additional data training and optimizations, and the computation efficiency of CLEANER for real-time operation was improved compared with the fuzzy logic-based algorithm.