*2.3. DA Configurations, Experimental Details and Observations*

#### 2.3.1. DA Configurations

The model used here is the operational global forecast system of China Meteorological Administration (CMA-GFS), whose original name was the Global/Regional Assimilation and Prediction System (GRAPES-GFS) [28]. The DA systems used in this study include the 4DVar system [23], and the recently developed 4DEnVar system and En4DVar system [24,26].

A dual-resolution framework with 1.0◦ for the inner loop and 0.5◦ for the outer loop, and 87 vertical layers are adopted by all DA systems. In the first assimilation window, the 4DEnVar system utilizes random perturbation samples with balanced constraints generated using the "randomcv" method [29]. This method can generate reasonable initial condition (IC) samples with balanced constraints by using the variational variable transform. Then, the 4DEnVar system updates the flow-dependent perturbation samples every 6 h in subsequent assimilation windows by assimilating perturbed observations. An extended-ensemble-sample-based localization method mentioned in Section 2.2.1 is applied in the 4DEnVar system. To alleviate the filter divergence problem, inflation, observation perturbation and SST perturbation approaches are applied [24].

The En4DVar system constructs the hybrid BEC by incorporating the ensemble covariance estimated by 60 ensemble members from the 4DEnVar system into the climatological BEC of the 4DVar system. The scalar weights of the climatological and ensemble covariances for the hybrid BEC in the En4DVar system are 0.25 and 0.8. Moreover, the ensemble covariance of the En4DVar system utilizes the same localization scheme as in the 4DEnVar system.

#### 2.3.2. Experimental Details

Observing system simulation experiment (OSSE) allows an objective assessment of the assimilation and forecast performances of a DA system when the "truth" state is known [25,30–32]. In this study, cycled assimilation experiments and corresponding initialized forecast experiments were performed based on the OSSE.

The design of OSSE is similar to Zhu et al. [24,26]. The background field in the first assimilation window and the "truth" state (or "truth") were generated by the lowresolution and high-resolution versions of the CMA-GFS model, which were initialized from the ERA-Interim 6 h forecast field and the ERA-5 reanalysis field, respectively. For a fair comparison, all assimilation experiments used the same background field in the first assimilation window.

Based on previous experiments assimilating conventional observations [24,26], further experiments adding AMSU-A radiance observations were carried out. The 1-week sensitivity experiments on the basis of the pressure at peak weight and the weighted average pressure were initially conducted to determine the vertical positioning method for the observation space localization. The most favorable vertical positioning method for the forecast performance was adopted. Then, three classes of experiments were designed, which covers a period of about 1 month starting from 0900 UTC 11 September 2016 and taking the first 2 days for spin-up. The first class includes the ensemble DA experiments using the 4DEnVar system and their initialized forecast experiments, and the second one contains the hybrid DA and forecast experiments with the En4DVar system that require the flow-dependent data produced by the first one. Additionally, the third class of experiments, i.e., the standard 4DVar DA and forecast experiments, were conducted for comparisons.

Each class includes two sets of DA experiments, respectively, incorporating only conventional observations [24,26] and both conventional and AMSU-A radiance observations (simply all types of observations, hereinafter), and two sets of corresponding initialized forecast experiments. Totally twelve sets of experiments were conducted to investigate the effects of adding AMSU-A radiance observations on the assimilation and forecast performances of the En4DVar system and its 4DVar and 4DEnVar component systems. The analyses of assimilating only conventional observations and assimilating all types of observations and their initialized forecasts were compared to assess the contributions of AMSU-A radiance observations. The experiments upon the standard 4DVar and 4DEnVar systems were conducted to provide the references for evaluating the performance of the En4DVar system when adding AMSU-A radiance observations.

#### 2.3.3. Observations

The "observations" were extracted from the "truth" state by using the transformations of observation operators and superimposing observation errors. The conventional observations used in this study were obtained from sounding and cloud-derived wind, and more details are presented in Zhu et al. [24]. Additionally, radiance from AMSU-A instruments of NOAA 15, 18, 19, NPP, and Metop A, B were also utilized. Sounding observations are sampled every 6 h, while both cloud-derived wind observations and AMSU-A radiance observations are sampled every 30 min. The radiance observations are assimilated using Version 12 of the RTTOV model [33] as the observation operator. To avoid the negative impacts of ground albedo and interpolation errors at upper layers, only channels 5–14 of the AMSU-A radiance observations were assimilated. Conventional observations cover most of the Northern Hemisphere, with a lower sampling density in the Southern Hemisphere. In contrast, radiance observations have a wider sampling range, which especially compensates for the low coverage of conventional observations in the Southern Hemisphere (Figure 1).

**Figure 1.** Spatial distribution of (**a**) conventional and (**b**) AMSU-A radiance observations valid during 0900–1500 UTC on 13 September 2016. The brown dots represent sounding observations, the blue dots represent cloud-derived wind observations, and the purple dots represent AMSU-A radiance observations.

#### *2.4. Evaluation Method*

In this study, the anomaly root mean square error (ARMSE) [31,32,34] and anomaly correlation coefficient (ACC) metrics were used to assess the random error and correlation of the analyses and forecasts against the "truth", respectively. The globe was divided into Northern Extratropics (20◦N∼90◦N), Southern Extratropics (20◦S∼90◦S) and Tropics (20◦S∼20◦N) for calculating the statistical results of these metrics. Moreover, a score card, which is marked with the significance of performance difference, was used to conveniently exhibit the performance difference between two forecasts initialized from different analyses in terms of ARMSE and ACC. Note that the analyses and forecasts from the 4DEnVar system are its ensemble mean analyses and deterministic 10-day forecasts initialized from these ensemble mean analyses. For more details about the evaluation methods refer to Zhu et al. [24,26].

#### **3. Results**

#### *3.1. Vertical Positioning Method*

In this subsection, the vertical positioning method was determined by a set of sensitivity experiments. The purpose of these experiments is to investigate the effects of two vertical coordinate definitions of AMSU-A radiance observations including the pressure at peak weight and the weighted average pressure on the forecast skill of the 4DEnVar system.

Figure 2a shows the scorecard of the 4DEnVar-initialized forecasts assimilating all types of observations with the pressure at peak weight as the vertical coordinates of AMSU-A radiance observations against those assimilating only conventional observations in terms of ACC and ARMSE. Encouragingly, the addition of AMSU-A radiance observations leads to significant improvements of the forecasts, especially in the Southern Extratropics and Tropics, except the degradation in the late period of the medium range over the Northern Extratropics (Figure 2a). Meanwhile, similar impacts of the AMSU-A radiance observations on the forecasts can be observed when the weighted average pressure is used as their vertical coordinates, but the degradation shown in Figure 2a is alleviated (Figure 2b). Therefore, the weighted average pressure was finally chosen to define the vertical coordinates of AMSU-A radiance observations in this study.

**Figure 2.** The scorecards of the 4DEnVar-initialized geopotential height (GZ), temperature (T), zonal wind (U) and meridional wind (V) forecasts assimilating all types of observations with (**a**) the pressure at peak weight and (**b**) the weighted average pressure as the vertical coordinates of AMSU-A radiance observations against those assimilating only conventional observations. The filling size of the triangle shows the difference significance of anomaly correlation coefficient (ACC) or anomaly root mean square error (ARMSE) between the evaluated and reference forecasts. The largest filling size represents very significant difference, and the other two decreasing filling sizes represent significant and insignificant differences. The green upward-pointing (purple downward-pointing) triangles are plotted if the evaluated forecast is better (worse) than the reference forecast. No triangles indicate equivalent.

#### *3.2. Effects of AMSU-A Radiance Observations on Analysis Quality*

After the vertical positioning method was determined, the effects of AMSU-A radiance observations on the analysis qualities of the DA systems were evaluated. Based on the En4DVar system and its two components, the analysis errors of assimilating all types of observations were compared with those of assimilating only conventional observations so as to investigate whether the AMSU-A radiance observations benefit the analysis quality in different DA systems. The results of the 4DVar and 4DEnVar component systems were used as the references to assess the effectiveness of the En4DVar system on assimilating AMSU-A radiance observations.

Figure 3 shows the contributions of AMSU-A radiance observations to the decreases in analysis error in the En4DVar system and its two components. It is found that all three DA systems reduced the ARMSE on all vertical layers except very few layers over Northern Extratropics and Tropic when the AMSU-A radiance observations joined the analyses. In particular, the decreases in ARMSE in all basic variables except specific humidity are most significant in the Southern Extratropics, especially in the stratosphere where conventional observations are sparsely distributed (Figure 3, column 2). However, the most significant improvement in the specific humidity analysis is located in the Tropics (Figure 3, row 4) where the water vapor content is high. As for the comparisons among three DA systems, they have different performances on different variables in different regions. The improvement in the 4DEnVar ensemble mean analysis is more (less) significant than in the 4DVar analysis on geopotential height (specific humidity), and comparable on zonal wind and temperature. It is more obvious on temperature (zonal wind and temperature) at the middle (upper) layers in the Tropics (Northern and Southern Extratropics), but less obvious on temperature at the lower layers, and on zonal wind at the upper (middle and lower) layers in the Tropics (Northern and Southern Extratropics). The improvement in the En4DVar analysis is generally between those in the analyses from its two component systems. There is larger improvement in zonal wind (temperature) at the middle layers in the Southern Extratropics and on the layers below 100 hPa in the Tropics (at the upper layers in the Tropics). Smaller improvement in temperature is in the middle and upper troposphere in the Northern Extratropics (Figure 3, rows 2 and 3).

The effects of adding AMSU-A observations on the error structures of the En4DVar and 4DVar analyses and the 4DEnVar ensemble mean analyses were shown in Figure 4. First, the analysis errors of all three DA systems are significantly reduced in most regions, indicating that the AMSU-A radiance observations have an overall positive effect on the analysis quality. Second, it is found that AMSU-A radiance observations most significantly reduces the analysis errors of the geopotential height, zonal wind and temperature in the Southern Extratropics, especially near 60◦S, where conventional observations are sparsely distributed. In addition, the analysis errors of the specific humidity are significantly reduced not only in the Southern Extratropics, but also in the Tropics (Figure 4, row 4). Finally, the improvement of analysis by the En4DVar is generally between those by the 4DVar and 4DEnVar. These results are consistent with the findings in Figure 3.

**Figure 3.** The anomaly root mean square error (ARMSE) differences between the analyses of assimilating all types of observations and those of assimilating only conventional observations by the 4DVar (black), 4DEnVar (red) and En4DVar (blue) systems in the Northern Extratropics (**left** column), Southern Extratropics (**middle** column) and Tropics (**right** column). The results of geopotential height (GZ; units: gpm), zonal wind (U; units: m/s), temperature (T; units: K) and specific humidity (Q; units: g/Kg) are ploted in rows 1–4, respectively. The green line denotes zero.

**Figure 4.** The zonally averaged anomaly root mean square error (ARMSE) differences between assimilating all types of observations and assimilating only conventional observations for the 4DVar (**left** column), 4DEnVar (**middle** column) and En4DVar (**left** column) analyses. The results of geopotential height (GZ; units: gpm), zonal wind (U; units: m/s), temperature (T; units: K) and specific humidity (Q; units: g/Kg) are ploted in rows 1–4, respectively.

#### *3.3. Effects of AMSU-A Radiance Observations on Forecast Skill*

Given that the analysis errors of the DA systems are significantly reduced by adding AMSU-A radiance observations, we next focus on whether the improved analysis could benefit the forecast skill as well.

From the comparisons between the geopotential height forecasts initialized from the analyses with and without including AMSU-A radiance observations in all three DA systems, it can be found that AMSU-A radiance observations can generally reduce the geopotential height forecast errors (Figure 5). The largest improvements are mainly located at the middle and upper layers in the Southern Extratropics, followed by the Northern Extratropics and Tropics, which is consistent with the analysis error distributions (Figure 3). In addition, the improvement of the 4DVar-initialized forecast is more obvious than those of the 4DEnVar- and En4DVar-initialized forecasts in the Northern Extratropics, but comparable in the Southern Extratropics and Tropics.

**Figure 5.** The time-variation of the anomaly root mean square error (ARMSE) differences between assimilating all types of observations and assimilating only conventional observations for the geopotential height forecasts (units: gpm) initialized by the 4DVar (**left** column), 4DEnVar (**middle** column), and En4DVar (**right** column) systems. The results in the Northern Extratropics, Southern Extratropics and Tropics are ploted in rows 1–3, respectively.

Figure 6 shows the effects of AMSU-A radiance observations in the En4DVar, 4DVar, and 4DEnVar systems on the zonal wind forecast errors. AMSU-A radiance observations in all these DA systems generally reduces the zonal wind forecast errors. The locations where the 4DVar- and 4DEnVar-initialized zonal wind forecasts are improved or degraded are generally consistent with the geopotential height. However, inconsistently, the En4DVarinitialized zonal wind forecast shows an improvement at the late period in the Northern Extratropics.

**Figure 6.** Same as Figure 5, except the zonal wind forecasts (units: m/s).

Adding AMSU-A radiance observations to all three DA systems also reduces most of the temperature forecast errors, with the largest improvement in the Southern Extratropics (Figure 7). Quite different from the geopotential height and zonal wind, the largest improvements in the temperature forecasts are located in the stratosphere, and middle and lower troposphere in the Southern Extratropics (Figure 7, row 2), consistent with the reduced analysis errors (Figure 3h). In addition, while the 4DVar-initialized forecast shows a persistent improvement in the Northern Extratropics, the 4DEnVar- and En4DVarinitialized forecasts performs neutrally. In contrast, the 4DEnVar and En4DVar systems show larger improvements than 4DVar in the Southern Extratropics and the Tropics. In particular, the En4DVar system shows the largest improvement.

**Figure 7.** Same as Figure 5, except the temperature forecasts (units: K).

Figure 8 shows the effects of AMSU-A radiance observations in all DA systems on the specific humidity forecasts. Similar to other variables, adding AMSU-A radiance observations steadily improves the specific humidity forecasts of all DA systems except for very few lead days in the Northern Extratropics. The improvement of the 4DVarinitialized forecast is also more significant than those of the 4DEnVar- and En4DVarinitialized forecasts in the Northern Extratropics (Figure 8, row 1). However, different from other variables, the largest improvement in the specific humidity forecasts is mainly distributed in the lower troposphere of the Southern Extratropics (Figure 8, row 2). In addition, there are significant improvements on the first few lead days in the Tropics (Figure 8, row 3), consistent with the regions where analysis errors are significantly reduced (Figure 3l).

**Figure 8.** Same as Figure 5, except the specific humidity forecasts (units: g/kg).

Overall, the differences of the 4DVar-initialized forecast performances between assimilating all types of observations and assimilating only conventional observations are statistically significant for almost all lead days in the Southern Extratropics and Tropics and the first few lead days in the Northern Extratropics (Figure 9a). It is encouraging to note that adding AMSU-A radiance observations to the 4DEnVar and En4DVar systems with the weighted average pressure as the vertical coordinates in the observation space localization also has significant positive effects on forecasts. While there are similar improvements in the Southern Extratropics and Tropics for the 4DEnVar- and En4DVar-initialized forecasts, the improvements are less statistically significant than those of the 4DVar-initialized forecast at the last few lead days. In addition, the impacts of AMSU-A observations in the 4DEnVar and En4DVar systems on the medium-range forecasts in the Northern Extratropics are neutral to slightly worse (Figure 9). It is reasonable considering that the 4DVar system uses model space localization, which can simulate close to the true atmospheric state [21,35]. In contrast, the observation space localization may hinder the transfer of some information from the radiance observations.

**Figure 9.** Same as Figure 2, except the forecasts assimilating all types of observations against assimilating only conventional observations initialized by the (**a**) 4DVar, (**b**) 4DEnVar and (**c**) En4DVar systems, respectively.

### **4. Discussion**

This study investigated the effects of incorporating AMSU-A radiance observations on the En4DVar system. Unlike most En4DVar systems that utilize the ensemble covariance produced by the locally solved EnKF class or the ensemble of globally solved 4DVars, this system introduces the ensemble covariance provided by the globally solved 4DEn-Var system using an economical observation space localization [26]. To take into account the information of AMSU-A radiance observations at other vertical layers, a weighted average hypsometry was proposed to define the vertical coordinates of radiance observations. The sensitivity experiments indicates that the new hypsometry approach has a wider range of positive effects on the 4DEnVar deterministic forecasts than the traditional peak-based approach.

The impacts of adding AMSU-A radiance observations on the assimilation and forecast performances of the En4DVar system were systematically assessed through 1-month OSSEbased assimilation experiments and its corresponding initialized forecast experiments. The results of the 4DVar and 4DEnVar components are also given as the references for more systematic evaluation of the En4DVar system in assimilating radiance observations. The analyses of all three DA systems benefit from AMSU-A observations, especially in the Southern Extratropics, where conventional observations are sparsely distributed. It is encouraging that the 4DEnVar system using observation space localization improved the analyses on the upper layers of the Northern and Southern Extratropics more significantly than the 4DVar system using model space localization. The improvement in the En4DVar analyses is generally between those of the standalone 4DVar and 4DEnVar components. In terms of ACC and ARMSE, three DA systems further improved the forecasts when adding AMSU-A radiance observations to the ICs. There is a steady improvement in the Southern Extratropics and Tropics, but the impact on the later lead days in the Northern Extratropics is neutral or even slightly negative. In the Northern Extratropics, the improvement of forecast by 4DVar is more significant than by 4DEnVar and En4DVar.

Future improvements in the assimilation of radiance observations based on the En4DVar system will focus on increasing the types of observations and adjusting the filtering radius of localization. In order to further improve the analysis quality, the En4DVar system needs to continue adding more radiance observations with complex multi-peak distribution weighting functions such as those from AMSU-B instruments. In addition, the broad satellite channel weighting function has a significant influence on the filtering radius of localization, and too larger or too small filtering radius will limit the assimilation performance. More flexible and adaptive localization techniques need to be developed for satellite DA with localization in observation space.

Moreover, although encouraging results were obtained using observation space localization method in assimilating AMSU-A observations with a single-peak distribution of weighting function, model space localization has proven to be more beneficial for assimilating radiance observations [21,35]. Therefore, future attempts will also be made to develop efficient model space localization method for the ensemble component of the En4DVar system, in order to obtain better results when assimilating radiance observations with complex multi-peak distribution weighting functions.

**Author Contributions:** Conceptualization, S.Z. and B.W.; writing—original draft preparation, S.Z.; writing—review and editing, S.Z., B.W., L.Z., J.L., Y.L., J.G., S.X., Y.W., W.H., L.L., Y.H., X.W., B.Z. and F.C. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded by the National Key Research and Development Program of China (2018YFC1506703) and the Innovation Group Project of Southern Marine Science and Engineering Guangdong Laboratory (Zhuhai) (No. 311022006).

**Data Availability Statement:** The observations were supported by the Global Telecommunications System (https://public.wmo.int/en/programmes/global-telecommunication-system accessed on 1 February 2021). The ERA-5 reanalysis and the ERA-Interim 6 h forecast are available at https: //apps.ecmwf.int/data-catalogues/era5/?class=ea&stream=oper&expver=1&type=an accessed on 1 February 2021 and https://apps.ecmwf.int/datasets/data/interim-full-daily/levtype=sfc/ accessed on 1 February 2021. All data used in this study are available from the authors upon request.

**Acknowledgments:** The assimilation and forecast experiments were performed on the high-performance computer PI-SUGON of the China Meteorological Administration.

**Conflicts of Interest:** The authors declare no conflict of interest.
