3.1. QC Results
The Gaussian distribution, also known as the normal distribution, is of significant importance in fields such as statistics and atmospheric sciences owing to its ability to approximate many natural phenomena and experimental data. By analyzing the Gaussian distribution plots, we can intuitively understand the central tendency, symmetry, and dispersion of the data.
Figure 4 presents the Gaussian distribution of FY2E TPW data innovation at three key time points (06:00 UTC on 8 July, 06:00 UTC on 9 July, and 06:00 UTC on 10 July 2013), representing the beginning, middle, and end of our study period, respectively. The figure shows the original data distribution alongside the results after applying the MCD and Isolation Forest QC methods.
Across all three time points, both QC methods demonstrate substantial improvements in data quality compared to the original distributions. The histograms for both methods show increased concentration around the center and reduced spread, indicating a more Gaussian-like distribution. This improvement is particularly evident in the reduction in extreme values and the smoothing of irregular peaks present in the original data.
However, the two methods exhibit distinct characteristics in their approach to data quality improvement. The MCD method (middle column) consistently produces smoother, more symmetrical distributions that closely approximate Gaussian curves. This is evident in the more uniform shape of the histograms and the closer fit of the red dashed Gaussian curve to the data. The MCD method’s effectiveness in optimizing the data distribution towards a Gaussian form is particularly noticeable in panels (e) and (h). This characteristic of the MCD method excels at optimizing the data distribution towards a Gaussian distribution, which may be more beneficial for variational assimilation systems based on Gaussian assumptions.
In contrast, the Isolation Forest method (right column) tends to preserve more of the original distribution’s features while still improving overall normality. This is visible in the slightly sharper peaks and marginally longer tails of the distributions, particularly in panels (c) and (f). This characteristic suggests that the Isolation Forest method may be more sensitive in identifying and processing anomalies, potentially preserving some physically significant extreme values. This unique advantage of preserving potentially important extreme information may be more applicable in certain extreme weather scenarios.
To quantitatively assess the effectiveness of the two QC methods, we calculated the skewness and kurtosis of the FY2E TPW data innovation after applying the MCD (EXPR2) and Isolation Forest (EXPR3) methods.
Table 3 presents these statistics for nine time points during the study period. The MCD method generally produced more consistent skewness values, with an average absolute skewness of 0.28 compared to 0.15 for the Isolation Forest method. This suggests that the MCD method was more effective in reducing the asymmetry of the data distribution. However, the Isolation Forest method maintained skewness values closer to zero in most cases, indicating a slightly better symmetry in the resulting distributions.
Regarding kurtosis, both methods showed varied results across different time points. The MCD method had an average kurtosis of −0.43, while the Isolation Forest method had an average of −0.20. The negative kurtosis values for both methods suggest that the resulting distributions are generally more platykurtic (flatter) than a normal distribution, with the MCD method producing slightly flatter distributions on average.
Note that at certain time points (e.g., 00:00 UTC on 9 July), the two methods produced quite different results, with the Isolation Forest method showing a higher positive kurtosis. This aligns with our previous observation that the Isolation Forest method may preserve more extreme values, resulting in heavier tails in some cases.
Overall, these metrics provide statistical evidence that both QC methods improved the normality of the data distribution, with each method showing distinct characteristics. The MCD method appears to be more consistent in reducing skewness, while the Isolation Forest method may be more effective in preserving the original distribution’s characteristics while still improving overall normality.
Figure 5 compares the boxplots of FY2E TPW data innovation before and after QC. In these boxplots, the central line represents the median, the lower and upper edges of the box indicate the first (Q1) and third (Q3) quartiles, respectively, and the whiskers extend to 1.5 times the interquartile range (IQR) beyond the box edges. Any points beyond the whiskers are considered potential outliers. Uncontrolled data (
Figure 5a) show evident distribution anomalies, with large upper and lower boundary ranges up to ±20 mm. Notably, at 12, 24, 36, 42, and 48 h lead times, numerous anomalies (filtered points) below the lower boundary appeared, which indicated a clear negative bias in innovation distribution at these times, thereby implying a model overestimation of PW values. Ideally, in a large sample with a standard normal distribution, the median should be centered between the upper and lower quartiles, with the boxplot symmetrical about the median line. However, in
Figure 5a, the median at almost every lead time is biased towards the upper quartile, thus revealing strong right-skewed distribution characteristics, which are consistent with the observations in
Figure 5.
After performing QC using the MCD method (
Figure 5b), the data distribution notably improved. The upper and lower limits of boxplots narrowed to ±10 mm. Except for 30 h and 42 h, the medians at each lead time were generally centered between the upper and lower quartiles, with boxplots showing near-symmetric shapes and exhibiting characteristics close to the standard normal distribution. This strongly demonstrates the excellence of the MCD method in eliminating noise and anomalies and effectively enhancing data concentration and symmetry.
The results after the Isolation Forest QC (
Figure 5c) showed similar improvements. Although the upper and lower limits of the boxplots were slightly larger than those of the MCD QC group, they were still markedly smaller than those of the uncontrolled group, thus successfully filtering out the anomalies. Positive innovation values after QC indicate that the observed precipitation exceeds the model forecast values, and vice versa. The MCD QC group shows a median average of about 2.5 mm, with a maximum of 4 mm, while negative value distributions are sparse, reaching a minimum of −7.5 mm. In comparison, the median of the isolated forest QC group was closer to zero, with similarly sparse negative value distributions.
To further quantify the improvement in data quality, we calculated the standard deviation of the innovation values at each lead time for the original data and after applying each QC method, as shown in
Table 4. The average standard deviation decreased from 4.38 mm in the original data to 2.86 mm after MCD QC and 2.55 mm after Isolation Forest QC. This reduction in standard deviation indicates a significant decrease in data spread and confirms the effectiveness of both QC methods in reducing data variability.
Notably, the Isolation Forest method showed a slightly lower average standard deviation compared to the MCD method, suggesting it may be more aggressive in reducing extreme values. This aligns with our previous observations of the Isolation Forest method’s characteristics in preserving certain data features while improving overall normality.
The standard deviation values also reveal that both QC methods are particularly effective at certain lead times. For instance, at the 42 h lead time, the standard deviation decreased from 4.40 mm in the original data to 3.35 mm with MCD and 1.90 mm with Isolation Forest, showing substantial improvements across all lead times.
These quantitative improvements in data distribution and variability, as evidenced by both the boxplots in
Figure 5 and the standard deviation values in
Table 4, underscore the effectiveness of both QC methods in enhancing data quality. Each method shows distinct characteristics that may be advantageous in different forecasting scenarios, with the MCD method generally producing more consistent distributions and the Isolation Forest method often achieving greater reductions in data variability.
Figure 6 and
Figure 7 illustrate the spatial distributions of the pass (
Figure 6A and
Figure 7A) and reject points (
Figure 6B and
Figure 7B) before and after applying the two QC methods, respectively. When analyzing the range of innovations, it is evident that the absolute values of innovations at rejection points generally exceed 15 mm. This indicates significant discrepancies between observations and model forecasts at these locations. In contrast, the innovations of pass points after QC are within the ±10 mm range, thereby suggesting that the data quality at these observation points is relatively high and meets the expected standards.
From the perspective of spatial distribution, most observation points in the western plateau region of Sichuan Province failed to pass the QC. This phenomenon may be attributed to the lack of observational data from the plateau region in the training model and inadequate utilization of observational data from these regions by the GSI system. This finding highlights the need to improve data QC methods in areas with complex terrain.
When comparing the performances of the two QC methods, we found that the Isolation Forest method eliminated more observation points located in the southeastern part of Sichuan Province than the MCD method. This may reflect the fact that the Isolation Forest method potentially adopts more stringent criteria when dealing with outliers, imposing higher quality requirements on the observational data in that region. Overall, the Isolation Forest method exhibited a higher sensitivity in identifying and filtering anomalous observation points, whereas the MCD method demonstrated advantages in maintaining overall data consistency and reliability.
3.2. Analysis of Simulated Circulation Fields
Before examining the precipitation forecasts, we first analyze the simulated circulation fields, as the large-scale circulation patterns play a crucial role in driving precipitation. The following analysis compares the circulation fields simulated by the four experimental groups, focusing on the 500 hPa geopotential height, vertically integrated water vapor flux, and 850 hPa wind vectors.
Circulation fields are key factors that determine moisture transport pathways, velocities, and total amounts, and they directly influence the spatiotemporal distribution and intensity of precipitation. Therefore, an in-depth analysis of circulation field configurations and characteristics is crucial for understanding and predicting precipitation activities. To further investigate the impact of assimilating data processed by different QC methods on rainfall forecasts, this study generated circulation field distributions for the four experimental groups (
Figure 8A–D).
The CTRL simulation results (
Figure 8A) show that the high-value area of the water vapor flux coincides precisely with the core position of the low-level southwest jet stream, with substantial numerical values capable of transporting sufficient moisture to the heavy rainfall center. As the simulation progresses, it is evident that as the subtropical high retreats eastward, this causes the jet stream to weaken and shift southeastward. This change in the large-scale circulation pattern is likely to influence the distribution and intensity of precipitation. Compared to the background circulation field based on ERA5 data, the high-value area of water vapor flux simulated by the CTRL experiment was biased towards the southeast, which can be attributed to the southward bias in the simulated position of the subtropical high.
In EXPR1 (
Figure 8B), compared to the CTRL, both the intensity and range of the moisture transport belt were weakened. In the 20°N–40°N and 105°E–115°E regions, the water vapor flux characterization between the 750 and 925 hPa layers showed a decrease in intensity, transitioning from high values of 4.0–4.7
to moderate values of 3.3–4.0
, and even to lower values of 2.7–3.4
thereby indicating reduced moisture transport intensity in this area. This change suggests that after assimilating the conventional data, the model simulation of moisture transport in this region underwent significant adjustments. The 850 hPa wind field showed changes in the low-level convergence areas, especially in the pattern at the edges of the original strong moisture transport belt. The southwestern airflow pattern across the entire region also exhibited subtle adjustments, resulting in a more complex structure. These changes indicate that the model’s description of atmospheric circulation was substantially adjusted after assimilating conventional data, and the weakening of moisture transport may have led to a reduced forecast precipitation intensity in certain areas.
The EXPR2 experiment (
Figure 8C) demonstrates significant changes brought about by assimilating the TPW data after QC processing using the MCD method. The intensity of the moisture transport belt was further enhanced, showing a denser and more widespread area of high water vapor flux values (4.7–5.4
) in the 20°N–40°N, 105°E–115°E region, which indicated high levels of water vapor flux in this area. Simultaneously, the range of the moisture transport belt expanded notably, extending further northeast. The 500 hPa geopotential height field showed further deepening of the upper-level trough in the northwest, thereby potentially leading to stronger cold air advection southward and increasing atmospheric instability. The 850 hPa wind field revealed significant low-level convergence at the leading edge and central areas of the strong moisture transport belt. Moreover, the southwest airflow within the region had become more vigorous and persistent. These features suggest that after assimilating the QC data processed using the MCD method, the WRF model is likely to forecast more extensive and intense precipitation processes.
The circulation field from the EXPR3 experiment (
Figure 8D) illustrates that assimilating the TPW data QC processed using the Isolation Forest method also brings about notable improvements. Compared to EXPR2, the intensity and range of the moisture transport belt were adjusted, showing a more concentrated high-intensity water vapor flux area in the 20°N–40°N and 105°E–115°E regions. This concentration may imply a more precise spatial distribution of precipitation. The 500 hPa geopotential height field shows that the depth of the upper-level trough in the northwest was moderate between CTRL and EXPR2, which may have brought about a more balanced atmospheric instability. The 850 hPa wind field shows that the low-level convergence area at the leading edge of the strong moisture transport belt was more distinct. The southwest airflow pattern was clearer and displayed a more organized moisture transport pathway. These features indicate that an assimilation scheme using the QC Isolation Forest method for QC may produce more balanced and refined precipitation forecasts.
3.3. Analysis of Precipitation Forecasts
3.3.1. Simulated Precipitation Distribution
Precipitation forecasting serves as a comprehensive indicator of the effectiveness of data assimilation.
Figure 9 presents the observed precipitation provided by the CMA from 06:00 UTC on 8 July 2013 to 06:00 UTC on 10 July 2013. The observed precipitation distribution (
Figure 9) shows that throughout the heavy rainfall event, the rain band was oriented northeast–southwest, primarily positioned above Mianyang–Chengdu–Ya’an, with persistent precipitation peaks particularly over the Chengdu area. At 06:00 UTC on 8 July 2013 (
Figure 9a), precipitation was mainly concentrated in the northeastern Sichuan Province, with the maximum rainfall reaching approximately 110 mm. As time progressed (from 12:00 UTC on 8 July to 06:00 UTC on 9 July), the precipitation area gradually expanded, with the center shifting towards central Sichuan Province, maintaining maximum rainfall amounts above 110 mm, which is classified as heavy rainfall. Subsequently (from 12:00 UTC on 9 July to 06:00 UTC on 10 July), the precipitation intensity gradually weakened to moderate-to-heavy rain levels of 40–68 mm, thereby highlighting the significant persistence of this precipitation event.
Figure 10 shows the 6-hourly accumulated precipitation distributions simulated by the four experimental groups from 06:00 UTC on 8 July 2013 to 06:00 UTC on 10 July 2013. The CTRL experiment (
Figure 10A) successfully reproduced the observed precipitation characteristics during the early stages of a heavy rainfall event. The simulation results showed high spatial consistency with the CMA observations and accurately captured the northeast–southwest orientation of the heavy rainband and its gradual shift from northeastern to central Sichuan. This indicates that the CTRL experiment was strongly formulated to accurately simulate the formation mechanism and evolution process of a heavy rainfall event during its initial stages.
However, in the later stages of the heavy rainfall event, particularly after 06:00 UTC on 9 July, the CTRL experiment substantially underestimated precipitation intensity. The simulated rainband was primarily concentrated in the central–eastern regions of Sichuan Province and the border areas of the Tibetan Plateau, which was slightly north of the observed rainband. Moreover, the simulated rainband appeared shorter and more dispersed, failing to adequately reproduce the observed characteristics of a continuous and extensive heavy rainband.
The EXPR1 group, which assimilated conventional observations (
Figure 10B), showed a notable weakening of the simulated precipitation intensity, especially during the peak of the precipitation event (00:00 UTC, 9 July) and its preceding period. The EXPR1 group only exhibited higher precipitation intensities in the stage leading up to the peak, which rapidly weakened afterward, thereby indicating a poor simulation performance during the sustained precipitation process. Compared to the CTRL experiment, the assimilation of conventional observational data in EXPR1 only improved the simulation of precipitation areas in the early stages of heavy rainfall, thus making some adjustments to the spatial distribution and local characteristics of precipitation during this phase. However, EXPR1 performed poorly in simulating the later stages after the precipitation peak, which were primarily characterized by an excessively rapid decrease in precipitation intensity.
The EXPR2 group, which underwent MCD QC processing (
Figure 10C), showed notable improvements over EXPR1 in simulating precipitation intensity. During the key phase from 06:00 UTC on July 8 to 06:00 UTC on 9 July, the EXPR2 maximum precipitation intensity reached heavy rainfall levels, with the rainband position and distribution closely matching the observations. This demonstrated that assimilating data processed using the MCD QC method can notably enhance the ability of the model to capture the spatial structure and intensity distribution of precipitation systems. This further validated the importance of introducing advanced QC and error-correction strategies prior to data assimilation.
The EXPR3 experiment, processed using the Isolation Forest method (
Figure 10D), showed an overall similarity to EXPR2 in the precipitation simulation, thereby demonstrating remarkable skill in reproducing the precipitation intensity and spatial distribution. However, after 06:00 UTC on 9 July, EXPR3′s simulated precipitation intensity weakened slightly, thus showing some degree of underestimation when compared with the observations. Additionally, the simulation of the spatial distribution details for the rainband in central–eastern Sichuan Province was inadequate, with a reduced continuity and extent of the rainband, thereby failing to capture the widespread and continuous heavy rainfall characteristics observed. This may be attributed to the imprecise capture of local data features by the IF method when performing QC on the FY2E TPW data. This leads to biases in the model’s simulation of moisture conditions and convective activities in these areas after assimilation.
3.3.2. Quantitative Precipitation Verification
To comprehensively quantify the precipitation simulation performance of the four experimental groups, this study employed the Fraction Skill Score (FSS) to evaluate the 6 h cumulative precipitation (
Figure 11). Evaluation thresholds were set according to the 6 h cumulative precipitation standards provided by the National Meteorological Center of China (0.1 mm, 4, 13, 25, 60, and 100 mm), corresponding to light rain, moderate rain, heavy rain, rainstorms, and severe rainstorms, respectively.
The four experimental groups exhibited distinct characteristics at various lead times during precipitation. The CTRL group generally performed well in simulating light to moderate rain but underperformed in simulating rainstorms and above, with FSSs consistently below 0.4. The EXPR1 group exhibited optimal performance in simulating all precipitation levels during the initial stage; however, its scores displayed a marked declining trend as the precipitation events progressed. Although EXPR1 outperformed the CTRL group in moderate rain and above, it still fell short of the EXPR2 and EXPR3 groups. After QC processing using the MCD and Isolation Forest methods, the EXPR2 and EXPR3 groups showed significant improvements in the precipitation forecast scores, particularly for moderate to heavy rain and severe rainstorm levels. During the critical precipitation stage, the FSSs exceeded 0.4, thereby demonstrating a relatively superior performance. This result indicates that appropriate QC of the FY2E TPW data before assimilation can effectively enhance the capability of the WRF model to simulate intense precipitation.
To evaluate the simulation performance of the entire precipitation event process comprehensively, we calculated the average FSSs across nine assimilation times to produce a mean FSS chart (
Figure 12). The results showed that the EXPR2 and EXPR3 groups, which underwent QC processing, considerably outperformed the CTRL and EXPR1 groups in simulating heavy rain and higher precipitation levels. Further comparison of these two QC groups revealed that EXPR3, which employed the Isolation Forest method, surpassed EXPR2, which used the MCD method, at moderate to heavy rain levels. However, EXPR2 outperformed EXPR3 during both rainstorms and severe rainstorms.
These results may reflect the complex interactions between data quality control, assimilation system characteristics, and extreme weather forecasting. Although the MCD method tends to optimize the data distribution towards a Gaussian distribution, in practical applications, it may indirectly enhance the model’s grasp of large-scale circulation features by smoothing extreme values, thus leading it to excel in extreme precipitation forecasts. In contrast, although the Isolation Forest method demonstrates advantages in identifying and retaining key extreme data points, the retained extreme values may introduce local instabilities during the assimilation process, particularly when the forecast model is highly sensitive to these extreme conditions. This could lead to a slightly lower forecast accuracy in certain extreme precipitation scenarios when compared with the MCD method.
To further quantify the performance of different experiments, we analyzed additional metrics including the Root Mean Square Error (RMSE), correlation coefficient (CC), and bias for each experiment at different assimilation times (
Table 5,
Table 6 and
Table 7). These metrics provided complementary insights to our FSS analysis. Additionally, we include results from experiments without quality control (No-QC) in these tables to highlight the importance of the quality control process in data assimilation. The No-QC results consistently showed higher RMSE, lower correlation coefficients, and larger biases compared to all other QC experiments, underscoring the critical role of proper quality control in improving forecast accuracy.
The analysis of these metrics across all experimental groups revealed several key findings:
Throughout the forecast period, all experimental groups demonstrated improvements over CTRL, which consistently showed the highest RMSE and bias values. However, the degree and nature of these improvements varied among EXPR1, EXPR2, and EXPR3.
EXPR1 showed a marked reduction in RMSE compared to CTRL, with values ranging from 9.39 to 24.62 mm. However, its performance was surpassed by both EXPR2 and EXPR3 at most time points. Interestingly, EXPR1 exhibited the highest initial CC (0.54), suggesting its potential strength in capturing immediate post-assimilation patterns. Yet, this advantage diminished over time, with CC values dropping as low as 0.06 in later stages. EXPR2, employing the MCD method, demonstrated notable RMSE reductions, particularly excelling in the early (8.43 mm at 06:00 UTC on 8 July) and late stages of the forecast. Its CC values, while variable, showed improvements over CTRL at several critical points. The MCD method’s performance in bias reduction was particularly impressive, maintaining generally low values throughout the forecast period. EXPR3, utilizing the Isolation Forest method, exhibited a more stable performance across all metrics. Its RMSE values, ranging from 7.57 to 15.90 mm, exhibited relatively less fluctuation compared to the other groups. While EXPR3’s CC values were not always the highest, they maintained a consistent level throughout the forecast, suggesting a steady reliability in pattern prediction. In terms of bias, EXPR3 achieved a balance between reduction and stability, with values consistently lower than CTRL and EXPR1.
The contrast between EXPR2 and EXPR3 is particularly intriguing. EXPR2 showed more pronounced improvements at specific time points, especially in RMSE reduction, aligning with its strong performance in extreme event prediction noted in our FSS analysis. Conversely, EXPR3’s strength lay in its consistency across all metrics and time points, resonating with its robust performance across various rainfall intensities observed earlier.
These findings not only corroborate our FSS analysis but also offer deeper insights into the characteristics of each QC method. The MCD method (EXPR2) appears to enhance the model’s capability in capturing critical features at specific stages, particularly beneficial for extreme event forecasting. In contrast, the Isolation Forest method (EXPR3) provides a more balanced improvement across the entire forecast range, potentially offering a more reliable overall performance.