Retrieval of Atmospheric Water Vapor Content in the Environment from AHI/H8 Using Both Physical and Random Forest Methods—A Case Study for Typhoon Maria (201808)

Abstract

The advanced imagers onboard the new generation of geostationary satellites could provide multilayer atmospheric moisture information with unprecedented high spatial and temporal resolutions, while the physical retrieval algorithm (One-Dimensional Variational, 1DVAR) is performed for operational atmospheric water vapor products with reduced resolutions, which is due to the limited computational efficiency of the physical retrieval algorithm. In this study, a typical cost-efficient machine learning (Random Forecast, RF) algorithm is adopted and compared with the physical retrieval algorithm for retrieving the atmospheric moisture from the measurements of Advance Himawari Imager (AHI) onboard the Himawari-8 satellite during the typhoon Maria (201808). It is found that the accuracy of the RF-based algorithm has much high computational efficiency and provides moisture retrievals with accuracy 35–45% better than that of 1DVAR, which means the retrieval process can be conducted at full spatial resolution for potential operational application. Both the Global Forecast System (GFS) forecasts and the AHI measurements are necessary information for moisture retrievals; they provide added value for each other.

1. Introduction

Tropical cyclones (TCs), either hurricanes or typhoons, are typical high-impact weather events with strong winds and heavy precipitation, often causing huge damage and disasters to society. Notably, these damages have ever been intensified over the past 50 years under a warming climate background [1,2,3]. The intensity of a typhoon after striking land usually slows down since less water vapor supplies to the cyclone. The distribution of atmospheric moisture carried by a storm system or in the surrounding environment is a distinctly important contributor to influencing its intensity and moving path [4,5,6]. The nearby and remote precipitation brought by a typhoon or motivated circulation system needs the supply of moisture [7,8]. Research shows that different extents of water vapor (WV) transport supplements to two respective typhoons with similar intensities or tracks will lead to completely discrepant life cycles and disaster impacts [9,10], embodying in the intensity, location and rainfall distribution [11]. As a whole, the precise descriptions of moisture fields are extremely important to better understanding, numerically simulating, and predicting the TC’s development and movement.

The reanalysis datasets could provide relatively reliable humidity fields and are widely used for comprehending the mechanism of typhoon development and movement [12,13]. They are commonly generated by combining various observations including various satellite remote sensing data with numerical weather prediction models through an assimilation system [14,15]. The latest ERA5 (the fifth generation ECMWF atmospheric reanalysis) reanalysis dataset produced by the European Centre for Medium-Range Weather Forecasts (ECMWF) has high accuracy and the highest spatio-temporal resolutions among all the reanalysis datasets, which could provide multilayer WV from 1000 hPa to 1 hPa with the temporal resolution of 1 h and horizontal resolution of 0.25° [16]. However, compared to the current geostationary advanced imager observing systems, the ERA5 provides atmospheric thermodynamic information at a quite low spatio-temporal resolution.

The Advanced Himawari Imager (AHI), onboard the Himawari-8 (H8) satellite, launched in 2014, started a new era for the new-generation geostationary satellite observing system [17]. It provides full disk InfraRed (IR) observations every 10 min with 2 km spatial resolution at nadir view for IR spectral bands over the East Asian region. H8 is stationed geostationary at the nadir of 140.7°E, which has an apparent resolution advantage over the reanalysis datasets. Observations with high spatio-temporal resolution are crucial to capture fast-changing weather systems [18,19,20]. For example, Schmit et al. [21,22] show that movement or development of clouds, convection, severe storms, and tropical cyclones are continuous even in 1 min intervals. Lee et al. [23] demonstrated that atmospheric profile products with 10 min intervals, derived from the AHI, have the capability on depicting low-level moisture advection and increasing instability in the pre-landfall environment of a typhoon. Di et al. [24] pointed out that moisture spatial variations within 12 to 16 km resolution are non-negligible, especially over land and in the pre-convection environment, and this information is an obvious indicator of several storm occurrences/development. Besides, evaluations on reanalysis datasets and regional model forecast products using GEO WV measurements from advanced imagers onboard the geostationary weather satellites both show non-negligible moisture bias at higher and lower troposphere [25,26]. Especially, the reanalysis dataset is not a near real-time product that has a period of lag time, and it is not available for operational applications. In contrast, high spatio-temporal atmospheric thermodynamical information promptly retrieved from geostationary imagers’ measurements (e.g., AHI), such as atmospheric vertical temperature profiles, atmospheric vertical moisture (VM) profiles, total precipitable water (TPW), layered precipitable water (LPW), and the atmospheric stability indices, are irreplaceable information to understand the process and mechanism of rapidly developing convective systems (e.g., TCs), as well as to improve their predictions [27].

The current operational atmospheric profile retrieval algorithm applied for high-resolution GEO imagers such as the AHI, and Advanced Baseline Imager (ABI), is the iterative-based one-dimensional variational (1DVAR) method [19,28]. While limited by the computational efficiency, instead of pixel by pixel, the retrievals are performed on Field-of-Regard basis, and one Field-of-Regard contains a 5 × 5 pixel box to meet the latency requirement for both full disk and regional production [19,23], for example, the retrievals must be produced for full disk coverage within 10 min to keep the temporal resolution of 10 min. It means the spatial resolution of atmospheric profile products is largely reduced from 2 km to 10 km due to the high computational cost on processing the large volume of Level 1B data if the retrieval is performed at pixel level. A machine learning-based approach is an alternative scheme that can fully use the sensor’s original resolution information since this kind of approach is cost-efficiently applied online as long as it is well trained offline. In addition, the machine learning algorithm sometimes has superior performance on the retrieval of the atmospheric profile, if it could also reproduce the nonlinearity relationship between the atmospheric parameters and radiance observations [29,30,31]. The purpose of this study is to demonstrate the feasibility of using rapid algorithms such as RF for producing atmospheric moisture information from advanced GEO imager measurements with original high spatio-temporal resolution and low latency. A typhoon case is chosen for the comparison between 1DVAR and machine learning-based algorithm (RF) on retrieving the atmospheric moisture parameters at the AHI original spatio-temporal resolution.

In this study, the 1DVAR algorithm and the RF-based algorithm are both applied to retrieve atmospheric moisture products such as VM, TPW, and LPW from the AHI measurements during typhoon Maria (201808) under clear skies at original spatio-temporal resolution (2 km at nadir view). Since the thermodynamic information in the environment is associated with TC development such as intensification, the high spatio-temporal resolution atmospheric water vapor information from AHI can be used for better understanding the TC process. The specific algorithm and data used here are described in Section 2. Section 3 shows the retrieval results from the RF-based algorithm. Section 4 exhibits the comparisons of results between the physical retrieval algorithm and the RF-based algorithm. The summary and discussions are given in Section 5.

2. Data and Methods

2.1. Data

2.1.1. Forecasts

Forecasts from the Global Forecast System (GFS) developed at National Centers for Environmental Prediction are used as input parameters or background fields in the RF-based and the 1DVAR algorithm, respectively. The GFS provides dozens of atmospheric and land-soil variables, including temperatures, winds, precipitation, soil moisture, and atmospheric ozone concentration at the temporal interval of three hours and spatial resolution of 0.5°. The specific variables of the GFS data adopted in the two types of retrieval algorithms include pressure at mean sea level (PRMSL_meansealevel), sea surface temperature (TMP_surface), along with relative humidity and temperature (TMP) at each pressure level. Section 2.2 outlines the specific time of data.

2.1.2. AHI Level1B

The AHI onboard the H8, launched in 2014, has three visible, three near-infrared, and ten IR spectral bands with 0.5 km, 1 km, and 2 km spatial resolution, respectively [32]. And it provides full disk measurements every 10 min. This study is focused on the retrieval of atmospheric water vapor profiles using Level 1B measurements of the AHI onboard the Himawari 8 (H8), and only IR measurements from water vapor and window bands (bands 8–16) are used in the retrieval process. Regarding using 9.6 µm channel, Li et al. [33] shows in Figure 1 (in their paper) that 9.6 μm channel also includes the information of low-level water vapor. Therefore, we thought 9.6 μm channel can be included in the establishment of the model in this study. In addition, both retrieval algorithms are only conducted over clear skies, therefore, the AHI level-2 cloud mask product is also needed for cloud screening. The algorithm procedure of cloud mask products please refer to [34] and its accuracy has been asserted in Wang [35]. Cloud mask product is used as a prerequisite in the operational macroscopic and microphysical property of cloud parameters and atmospheric parameters retrieval algorithms [36,37].

2.1.3. ERA5

The ERA5, as the fifth-generation reanalysis product generated by the ECWMF, absorbs various kinds of observations through the data assimilation system, which provides information on variable atmospheric parameters with high accuracy at the temporal resolution of 1 h and horizontal resolution of 0.25° [16]. The accuracies of temperature and WV profiles have been asserted by various assessments with remote sensing measurements and radiosonde observations [25,38]. The WV profiles of the ERA5 are used as labels for the RF-based algorithm and the validation resources (independent) for both algorithms. To be specific, specific humidity (q), mean sea level pressure (MSL), sea surface temperature (SST), the TPW (300–1000 hPa), and LPW of the upper (700–900 hPa), middle (900–700 hPa) and lower (700–300 hPa) levels, respectively, are to be retrieved. The TPW and LPW are calculated by the following equation:

$$\mathrm{LPW}=\frac{1}{\mathrm{g}} {{\displaystyle \sum }}_{\mathrm{i}=1}^{\mathrm{n}}\overline{{\mathrm{q}}_{\mathrm{k}}}\Delta {\mathrm{P}}_{\mathrm{k}}$$

(1)

where $\overline{{\mathrm{q}}_{\mathrm{k}}}=\left({\mathrm{q}}_{\mathrm{k}}-{\mathrm{q}}_{\mathrm{k}-1}\right)/2 \mathrm{and} \Delta {\mathrm{P}}_{\mathrm{k}}={\mathrm{P}}_{\mathrm{k}-1}-{\mathrm{P}}_{\mathrm{k} }$, ${\mathrm{q}}_{\mathrm{k}}$ means specific humidity at k level; ${\mathrm{P}}_{\mathrm{k}}$ is atmospheric pressure at k level; N means all the atmospheric levels.

Table 1. Introduction of the AHI infrared bands.

Band Number	Central Wavelength (μm)	Resolution at Nadir (km)
8	6.2	2.0
9	6.9
10	7.3
11	8.6
12	9.6
13	10.4
14	11.2
15	12.3
16	13.3

.

2.1.4. Dropsonde

Dropsonde observations during typhoon Maria (201808) are also used to validate the retrievals independently, the dropsonde data is downloaded from https://doi.org/10.5281/zenodo.4671631 accessed on 9 July 2018. These dropsonde observations are obtained by releasing the downward-facing sounder from the aircraft, which was equipped with a range of observation instruments including sensors of GPS antenna, GPS receiver, temperature, pressure, etc. For your information, these soundings have also been used in other studies [39].

2.2. Methods

2.2.1. Maria Typhoon

‘Maria’ is a super typhoon with a maximum intensity of level 17, a central minimum pressure downs to 925 hPa [40], and a long duration of maintenance generated in the northwest Pacific Ocean on 4 July 2018, which caused severe economic damage [31]. There are several reasons for choosing this typhoon in our study: (1) first, this typhoon is the most famous one in 2018 and has been studied the most frequently with a lot of information to refer to; (2) this typhoon had a long duration, high intensity, stronger impact than others and a more obvious rapid strengthening phase. In our opinion, this typhoon has high research value; and (3) the typhoon is strong and long-lasting, so it carries more vapor and supplies more vapor from the environmental field than other typhoons.

2.2.2. RF-Based Algorithm

The RF method is a typical machine learning integrated model proposed by Breiman [41]. It has obvious advantages in handling classification and prediction-related problems effectively and efficiently, which includes but are not limited to: (1) the procedure of training can be processed in parallel, which has a great advantage in enhancing the algorithm speed, especially for the case of large samples; (2) the importance ranking of features (input parameters) can be given directly, which is beneficial for understanding the model performance and the physical means behind the ranking; and (3) using random sampling with put-back, the RF model has small variance and powerful generalization. Due to those advantages, the RF method has been recently applied in various studies and applications, for example, Turini et al. [42] and Kuehnlein et al. [43] established new approaches for rainfall estimation from high spatial and temporal resolution satellite measurements using RF. Lee et al. [30] used RF to obtain a highly accurate water vapor retrieval model, and Zhang et al. [44] found that RF can effectively improve the retrieval accuracy of TPW from thermal IR remote sensing observations.

The RF-based algorithm adopted in this study aims to retrieve the atmospheric moisture profile products such as VM, TPW, and LPW during the typhoon Maria (201808) from 4 to 11 July 2018. First, a set of data containing the inputs, the AHI measurements and the GFS forecasts, as well as the outputs, the ERA5, are formed. The AHI cloud mask product is used to clear sky pixels for retrievals. Only clear sky AHI pixels over ocean are used for retrievals, therefore, our retrievals are performed in the environmental region. The AHI and the GFS are temporally and spatially interpolated to the resolution of the ERA5 with the temporal resolution of 1 h and the horizontal resolution of 0.25° × 0.25° using the nearest distance method. In order to train and evaluate the RF-based retrieval model, a match-up dataset is developed through spatially and temporally collocating ERA5, GFS and AHI radiance measurements during typhoon Maria (2018). There are three steps for developing the training and test datasets: in the first step, for each ERA5 grid, the closest clear AHI pixel with 0.05 latitude/longitude degree (about 5 km) radius is selected to form a match, and the GFS data are also spatially and temporally interpolated to each ERA5 grid; in the second step, in order to reduce the data volume, 1 out of every 4 ERA5 grids is picked to finally form the size reduced ERA5-GFS-AHI matchup dataset (83,199 samples); in the third step, the size reduced matchup dataset is randomly divided into training (70% of the samples) and testing (30% of the samples). On the one hand, the training and testing datasets cover the spatial and temporal variations during typhoon period, the training is representative for this case, and since only one quarter of matchup dataset is used, the application to ABI measurements with 10 min temporal resolution and 2 km spatial resolution is considered somewhat independent. Equation (1) is used for the variable conversion, and

Table 2. Specific input and output variations of RF-based algorithm.

		Variables	Information Provided by These Variables
Input	GFS	PRMSL_meansealevel (hPa)	Minimum sea level pressure
		TMP_surface (K)	Sea surface temperature
		VM (1000 hPa/900 hPa/850 hPa/ 800 hPa/700 hPa/500 hPa/300 hPa) (g/kg)	Water vapor specific humidity at specific atmospheric pressure levels
		LPW (UP/MID/LOW) TPW (mm)	Layered precipitable water Total precipitable water
	AHI	IRX0620 (K)	Brightness temperature of the AHI band 08 which provides upper tropospheric moisture information
		IRX0700 (K)	Brightness temperature of the AHI band 09 which provides middle to upper tropospheric moisture information
		IRX0730 (K)	Brightness temperature of the AHI band 10 which provides low to middle tropospheric moisture information
		IRX0860 (K)	Brightness temperature of the AHI band 11
		IRX0960 (K)	Brightness temperature of AHI band 12
		IRX1040 (K)	Brightness temperature of the AHI band 13 which provides SST information
		IRX1120 (K)	Brightness temperature of the AHI band 14 which provides SST information
		IRX1230 (K)	Brightness temperature of the AHI band 15 which provides boundary layer moisture information
		IRX1330 (K)	Brightness temperature of the AHI band 16 which provides low level atmospheric temperature information
		IRX0620-IRX1120 (K)	Brightness temperature difference
		IRX0700-IRX1120 (K)
		IRX0730-IRX1120 (K)
		IRX1230-IRX1120 (K)
Output	ERA5	MSL (hPa)	Minimum sea level pressure
		SST (K)	Sea surface temperature
		VM (1000 hPa/900 hPa/850 hPa/ 800 hPa/700 hPa/500 hPa/300 hPa) (g/kg)	Atmospheric water vapor mixing ratio at specific pressure levels
		LPW(UP/MID/LOW) (mm) TPW (mm)	Layered precipitable water Total precipitable water

outlines the specific input/output variables. The entire dataset is randomly divided into two sets of training and test datasets (data size is 83,199) with 70% and 30% of the sample size (sample size: 100°E–180°E, 0°N–50°N), respectively. The training dataset is used to obtain several significate super-parameters of this algorithm including the number of trees, the maximum depth of a tree, and so on. The final-used super-parameters are listed in

Table 3. Super-parameters used in RF-based algorithm.

Super-Parameter	Definition	Number
1. n_estimators	the number of trees	300
2. max_features	the maximum number of predictors	“auto”
3. max_depth	the maximum depth of the tree	30

. The test dataset is to independently evaluate the algorithm’s performance. Note that all the results shown in Section 3 and Section 4 are based on the test dataset. Again, the RF-based algorithm is only applied to the AHI measurements over clear skies with the original spatial resolution of 2 km.

It is worth noting that the ERA5 is the best uniformly sampled atmospheric reanalysis, it has very good representativeness of atmospheric diurnal variation and can be well match with geostationary observations with high temporal resolution. The ERA5 dataset has very good quality, it is the first reanalysis to use a 10-member ensemble to assess atmospheric uncertainties through the 4D-Var data assimilation system, and it also uses many observations for data assimilation, thus can provide atmospheric conditions more accurately. The accuracy of the temperature and water vapor profiles has been confirmed by various assessments from remote sensing and radiosonde observations [25,37], and using ERA5 data as a reference to analyze vertical meteorological fields is highly advantageous [31]. The ERA5 data have a much improved spatial and temporal resolutions compared to its predecessor, and the vertical stratification has been increased from 60 to 137 layers. Therefore, although the spatial resolution of ERA5 data is relatively coarse, it is used as the true value reference for this study because of its high accuracy and uniform sampling. Moreover, the purpose of this study is to retrieve the water vapor information with high spatial and temporal resolutions, so it is necessary to use the independent data to validate the accuracy of the water vapor information obtained from AHI measurements using the RF and 1DVAR methodologies.

2.2.3. 1DVAR

The atmospheric moisture profile physical retrieval algorithm adopts the 1DVAR iterative technique based on optimal estimate theory [19,38,45,46]. In the AHI retrieval, the regression is used to generate the first guess with the AHI IR measurements and forecasts (GFS) as the predictor, and 1DVAR is then performed by adjusting the atmospheric profile state, X, to minimize a cost function:

$$J\left(X\right)={\left[Y^m-F\left(X\right)\right]}^T{E}^{-1}\left[Y^m-F\left(X\right)\right]+{\left[X-X_b\right]}^T\gamma B^{-1}\left[X-X_b\right]$$

(2)

where the X is profile vector (containing atmospheric temperature vertical profile, moisture vertical profile and surface skin temperature [47]) to be retrieved; E and B are observational and background error covariance matrix of the AHI measurements and first guess, respectively; Superscripts −1 and T denote inverse and transport of a matrix, respectively.$Y^m$ is the AHI IR band measurements; F is the radiative transfer model, which is the Pressure-Layer Fast Algorithm for Atmospheric Transmittance (PFAAST) model used in this study [48,49]. $\gamma$ is the regularization parameter or smoothing factor used for solution convergence [50]. The profile vector X can be solved through an iterative approach (the Quasi-Newtonian iteration used in our algorithm):

$$X_{n+1}=X_0+{\left(F^{\prime T}\left(X_n\right)·O^{-1}·F^{\prime }\left(X_n\right)+B^{-1}\right)}^{-1}·F^{\prime T}\left(X_n\right)·O^{-1}\cdot \left(F\left(X_0\right)-F\left(X_n\right)+F^{\prime T}\left(X_n\right)\left(X_n-X_0\right)\right)$$

(3)

where $F^{\prime T}$ is the Jacobian matrix of the radiative transfer model. A more detailed description of the algorithm can be found in the ABI legacy atmospheric profile (LAP) algorithm [19,28,46]. It should be noted that both E and B matrix are diagonal for simplicity and efficient calculations.

2.3. Evaluation Criteria

Table 4. Algorithm evaluation indicators.

Evaluation Indicators	Calculation Formulas	Range of Values	Optimum Value
RMSE	$\sqrt{\frac{1}{\mathrm{n}}{\displaystyle {\displaystyle \sum }_{\mathrm{i}=1}^{\mathrm{n}}}{\left({\mathrm{y}}_{\mathrm{i}}^{*}-{\mathrm{y}}_{\mathrm{i}}\right)}^2}$	$\left[0,+\infty \right)$	0
R	$\frac{\mathrm{Cov}\left({\mathrm{y}}_{\mathrm{i}},{\mathrm{y}}_{\mathrm{i}}^{*}\right)}{\sqrt{\mathrm{Var}\left[{\mathrm{y}}_{\mathrm{i}}\right]\mathrm{Var}\left[{\mathrm{y}}_{\mathrm{i}}^{*}\right]}}$	[–1, 1]
MAPE	$\frac{100\%}{\mathrm{n}}{{\displaystyle \sum }}_{\mathrm{i}=1}^{\mathrm{n}}\left|\frac{{\mathrm{y}}_{\mathrm{i}}^{*}-{\mathrm{y}}_{\mathrm{i}}}{{\mathrm{y}}_{\mathrm{i}}}\right|$	$\left[0,+\infty \right)$	0
MAE	$\frac{1}{\mathrm{n}}{{\displaystyle \sum }}_{\mathrm{i}=1}^{\mathrm{n}}\left|{\mathrm{y}}_{\mathrm{i}}^{*}-{\mathrm{y}}_{\mathrm{i}}\right|$	$\left[0,+\infty \right)$	0

Note: where ${\mathrm{y}}_{\mathrm{i}}^{*}$ denotes the retrievals and ${\mathrm{y}}_{\mathrm{i}}$ denotes the true values; $\mathrm{Cov}\left({\mathrm{y}}_{\mathrm{i}},{\mathrm{y}}_{\mathrm{i}}^{*}\right)$ is the covariance matrix of ${\mathrm{y}}_{\mathrm{i}}$ and ${\mathrm{y}}_{\mathrm{i}}^{*}$. $\mathrm{Var}\left[{\mathrm{y}}_{\mathrm{i}}\right]$ and $\mathrm{Var}\left[{\mathrm{y}}_{\mathrm{i}}^{*}\right]$ are the variances of ${\mathrm{y}}_{\mathrm{i}}$ and ${\mathrm{y}}_{\mathrm{i}}^{*}$, respectively.

lists the metrics used in this study to evaluate both retrieval algorithms. Root mean square error (RMSE) and Mean Absolute Error (MAE) are the two indicators used to measure the deviations between the retrievals and the true values. Mean absolute percentage error (MAPE) is a relative error indicator and it depends both on deviations and the true values. Each indicator nearer to 0 means the retrievals are better; the correlation coefficient (R) is to quantitatively provide the degree of linear correlation between the retrievals and true values.

3. Evaluations of RF-Based Algorithm

The atmospheric moisture profile retrievals from the RF-based algorithm are evaluated using the ERA5 as a reference and the evaluation indicators listed in Section 2.3 are used. This RF algorithm uses the inputs of combined the AHI IR band brightness temperature measurements and the GFS forecasts (GFS + AHI), the same inputs for the 1DVAR algorithm. The retrievals follow Schmit et al. [19], including atmospheric vertical moisture (VM) (moisture at each pressure level between 300 and 1000 hPa), total precipitable water (TPW, integrated from 300 to 1000 hPa), layered precipitation water (LPWs) (integrated from 300 to 700 hPa, 700 to 900 hPa, and 900 to 1000 hPa, respectively). Note that the evaluations are only conducted with the independent test dataset. The GFS moisture forecasts are also included in the evaluation, to see the added value of the AHI IR band measurements on the GFS forecasts.

The RMSE and MAPE statistical results of the VM retrievals at the pressure levels between 300 hPa and 1000 hPa from the GFS forecasts and RF-based algorithms are shown in Figure 1a,b, respectively. Figure 1c,d are similar but the LPW retrievals are assessed. It can be seen that the moisture RMSEs between 300 hPa and 1000 hPa are significantly reduced from the GFS to AHI retrievals, resulting in consistent improvement of LPWs (upper, middle, and low) from combined the AHI and the GFS over the GFS alone. Consistent improvement can also be found when the MAPE is used as an indicator, however, the error retrievals at 300 hPa are the largest and those at the middle levels of the troposphere are modest, which seems opposite to the RMSE results. This is due to the fact that moisture concentration at 300 hPa is usually quite small, and it increases in an exponential growth as close to the surface, it is reasonable that moisture retrievals at 300 hPa have a smaller absolute error but larger relative error. As a whole, with the AHI measurements combined with the GFS, the moisture retrievals from the RF-based algorithm have been largely improved over the GFS forecast, indicating the added value of the AHI to the forecast for applications. The specific error reductions or accuracy improvements (in percentage) of the RF-based algorithm from the AHI are listed in

Table 5. Accuracy improvements of the retrievals from RF-based algorithm using AHI+GFS as inputs over the GFS forecasts.

	RMSE			MAPE			MAE			R
	GFS	GFS+ AHI	Improvement (%)	GFS	GFS+ AHI	Improvement (%)	GFS	GFS+ AHI	Improvement (%)	GFS	GFS+ AHI	Improvement (%)
MSL	4.32	0.95	78.01	0.29	0.06	79.31	2.89	0.65	77.51	0.79	0.98	24.05
SST	1.21	0.45	62.81	0.27	0.10	62.96	0.81	0.29	64.20	0.98	0.99	1.02
VM _1000 hPa	1.58	0.57	63.92	7.87	2.63	66.58	1.14	0.39	65.79	0.87	0.98	12.64
VM _900 hPa	2.63	1.03	60.84	19.37	7.44	61.59	1.92	0.73	61.98	0.59	0.93	57.63
VM _850 hPa	2.82	1.02	63.83	25.25	9.01	64.32	2.11	0.72	65.88	0.53	0.93	75.47
VM _800 hPa	2.78	1.08	61.15	28.97	11.30	60.99	2.20	0.78	64.55	0.51	0.91	78.43
VM _700 hPa	2.50	1.08	56.80	40.62	18.24	55.10	1.98	0.79	60.10	0.45	0.89	97.78
VM _500 hPa	1.58	0.70	55.70	69.83	33.30	52.32	1.24	0.52	58.06	0.40	0.89	122.5
VM _300 hPa	0.33	0.14	57.58	111.46	46.40	58.37	0.26	0.10	61.54	0.26	0.94	261.54
TPW	10.10	3.76	62.77	19.47	6.66	65.79	7.99	2.64	66.96	0.64	0.94	46.88
LPW_LOW	1.76	0.66	62.50	9.68	3.45	64.36	1.25	0.45	64.00	0.82	0.97	18.29
LPW_MID	4.96	1.89	61.90	24.35	8.97	63.16	3.86	1.35	65.03	0.56	0.93	66.07
LPW_UP	5.14	2.07	59.73	41.11	16.54	59.77	4.07	1.49	63.39	0.48	0.91	89.58

, evaluated using RMSE, MAPE, MAE, and R indicators, respectively. It is interesting to see that the accuracy improvements (percentages) of the retrievals are comparatively consistent among MAPE, RMSE, and MAE, for all parameters.

Figure 1e provides the importance ranking for all the input variables. Overall, the GFS forecasts have a larger significance than the AHI measurements in the RF-based algorithm, indicating the importance of background. The relatively important of the AHI measurements are the brightness temperatures from band 12 (centered at 9.6 μm) as well as brightness temperatures differences between band 14 (centered at 11.20 μm) and band 15 (centered at 12.3 μm). Some information about moisture near the boundary is contained in these measurements, where the GFS moisture forecasts have relatively large errors (Lee et al., 2014). Interestingly, the accuracies of moisture parameters from the RF-based algorithm with the AHI alone data are relatively lower than those from the GFS+AHI (not shown). It indicates that the GFS forecasts and the AHI measurements from the satellite are both necessary information for moisture retrieval and they provide added value for each other.

Figure 2 illustrates grid-to-grid comparisons of the VM retrievals at 300, 700, and 1000 hPa, respectively, with the ERA5 based on the test dataset. As mentioned, since the spatial resolutions of the ERA5 and the retrievals are different, the grid-to-grid comparisons require to be down-sampled to temporal and spatial resolutions of the ERA5. The color represents the probability density of samples, and the assessment indicators are given in panels. It obviously shows that the GFS moisture forecasts have relatively a large inconsistency with the ERA5, especially for the moisture at the upper troposphere. With the AHI measurement added, the AVM retrievals from the RF-based algorithm agree with the ERA5 at grid level. Overall, the accuracy of retrievals is relatively lower for the moisture at the upper troposphere than the lower troposphere, which is accordant and dependent on the accuracy of the GFS forecast itself.

The spatial distribution of the multilayer WV retrievals and the GFS forecasts at 00 UTC on 5 July 2018 are shown in Figure 3. The IR brightness temperatures measurements for band 9 (6.9 μm), 10 (7.3 μm), and 14 (11.2 μm) are also exhibited in Figure 3a–c. Since both the RF-based and 1DVAR algorithms are only conducted for the clear AHI pixels, retrieved moistures are in the surrounding environment of the typhoon Maria. From Figure 3a–c, it can be seen that the retrieved moistures are in the northern and eastern of the typhoon. From Figure 3d–l, it indicates that the patterns of retrievals are coincident better with the ERA5 than with the GFS, especially for the northern of the typhoon. The GFS moisture forecasts tend to be dry, while the retrievals tend to be wet. The accuracy at low troposphere is better as a whole.

Figure 4 shows the differences between the ERA5 and the retrievals/GFS: (a) (b) for the upper LPW, (e) (f) for the middle LPW, (i) (j) for the lower LPW, and (c) (d) for the VM at 300 hPa, (g) (h) for the VM at 700 hPa, and (k) (l) for the VM at 1000 hPa, respectively. It can be seen that as a whole, the LPW and VM retrievals errors from the RF-based algorithm are smaller than those from the GFS forecasts. The retrieval errors from the RF-based algorithm are smaller for the lower layers and somewhat larger for the upper layers, which are consistent with those findings in Figure 2.

The dropsonde moisture observations are also used to assess the retrievals from RF-based algorithm after temporally and spatially matched. The match requires the observation time difference between dropsonde and the AHI less than 5 min and the spatial distance within 50 km. Limited by the observation time and clear sky condition, only two dropsonde observations over the ocean meet the requirements and are collocated. Figure 5 shows that except retrievals at the upper troposphere, most of the retrievals from the RF-based algorithm are even closer to the dropsonde observations than the ERA5. It also can be seen that, overall, the 1DVAR water vapor retrievals are close to RF-based results in both cases in terms of TPW and LPW (right panels), while at significant pressure levels, 1DVAR is slight better in the middle to upper troposphere while RF-based model is better in the lower to middle troposphere in these two particular cases (left panels).

4. Comparisons of Retrievals between RF-Based and 1DVAR Algorithms

The LPW retrievals from the RF-based and 1DVAR algorithms at different layers 00 UTC 5 July 2018 are compared in Figure 6. It exhibits the WV differences between the retrievals and the ERA5. Although the algorithms are both conducted for the same AHI measurement at pixel level under clear sky conditions, since the 1DVAR algorithm might fail in the convergence of iteration and will output invalid results for these nonconvergence pixel [46], therefore the data size of the retrievals from the 1DVAR algorithm is distinctly smaller. Besides, assessed by the ERA5, LPW retrievals from the RF-based algorithm are closer to the ERA5 than the 1DVAR. Especially, the overestimations on moisture are more noticeable in the 1DVAR algorithm, while the problems are visually alleviated in the RF-based algorithm at all layers in this case.

In addition, comparisons between the retrievals and the ERA5 at grid level with different cloudy scenarios are analyzed. With the help of the AHI cloud mask (2 km), the cloud coverage of each ERA5 grid (25 km) is determined. Comparing the accuracies of two sets of retrievals with different cloud coverages is possible. In

Table 6. Retrieval accuracies between 1DVAR and RF-based methods for different cloudy scenarios.

	RMSE (Unit: mm)			MAPE (Unit: %)			MAE (Unit: mm)			R
	1DVAR	RF	Improvement (%)	1DVAR	RF	Improvement (%)	1DVAR	RF	Improvement (%)	1DVAR	RF	Improvement (%)
Clear (0–10%)	1.17	0.70	40.17	6.24	3.44	44.87	0.92	0.49	46.74	0.71	0.86	17.44
Partly Cloudy (10–30%)	1.00	061	39.00	5.33	2.92	45.22	0.81	0.42	48.45	0.82	0.88	9.00
Cloudy (30–70%)	1.23	0.73	40.65	6.64	3.81	42.62	0.97	0.55	43.30	0.77	0.86	10.46
Overcast (70–100%)	1.11	0.71	36.04	6.18	3.53	42.88	0.90	0.50	44.44	0.72	0.85	15.29

, four scenes including clear, partly cloudy, cloudy, and overcast scenarios with 0–10%, 10–30%, 30–70%, and 70–100% cloud coverage, respectively, are considered in the comparisons. Using RMSE, MAPE, and MAE as indicators, retrievals from the RF-based algorithm have 35–45% percentage improvements than the 1DVAR for all four cloud scenes. The performance of the 1DVAR algorithm is relatively weaker in the cloudy scenarios while the accuracy of the RF-based algorithm has almost no scene dependency. Besides the retrieval accuracies, the computation efficiency between the RF-based and 1DVAR algorithms are compared in this case study. The total time cost of retrieving one full disk of Himawari-8 data with the 10 min and 2 km resolutions using the RF-based algorithm is about 7 min, which is much faster than 5 h and a half based on the traditional 1DVAR algorithm.

5. Summary and Discussion

The advanced IR imagers onboard the new generation of the geostationary satellite could provide multilayer moisture information with unprecedented high spatial and temporal resolutions, which are irreplaceable information to understand the process and mechanism of rapidly developing convective systems (e.g., TCs), those measurements are also the very useful improving simulation of prediction of TC. However, operational physical retrieval algorithm (1DVAR) is performed with reduced spatial resolutions due to limited computational efficiency for large data processing if the retrieval is performed at pixel level. In this study, a typical cost-efficient machine learning, the RF-based algorithm, is developed and compared with the 1DVAR algorithm for the measurements of the AHI onboard the H8 satellite during the typhoon Maria (201808). It is found that:

• The GFS forecasts and the AHI measurements are both necessary information for moisture retrievals and provide supplemental value for each other;
• The accuracy of atmospheric water vapor retrievals using the RF-based algorithm with representative training dataset is enhanced when compared with the 1DVAR algorithm (e.g., with 35–45% improvement in this case);
• The retrievals can be conducted at full resolution in operation with high computational efficiency if using a machine learning-based algorithm instead, potentially for real-time or near real-time quantitative applications of high spatio-temporal resolution satellite measurements.

It should be noted that building a representative training dataset is very important for the applications of RF model, due to limited computer resource on processing large volume of data (every 10 min with 2 km spatial resolution at nadir view) for training, the RF model in this study is trained with only limited independent data from the case, therefore it can be applied appropriately to the water vapor retrieval for this typhoon case, and the accuracy might be reduced when it is applied to another typhoon case. However, the main objective of this case study was to prove our hypothesis: if the machine learning model is well trained, then it can be used to extract atmospheric water vapor content efficiently and effectively from the high spatio-temporal resolution satellite observations for real-time or near real-time applications, which exhibits advantage of processing large volume of data with low latency when compared with the physical based 1DVAR methodology. For real-time or near real-time applications, representative training dataset can be achieved through either routinely updating training dataset, or building a representative training dataset covering seasonal, regional, and diurnal variations. Compared with the 1DVAR algorithm, when the same input data are used, the advantages of the machine learning approach for atmospheric profile retrieval include high computational efficiency, stable retrieval (e.g., less scene dependence), not relying on the radiative transfer model, less sensitive to the calibration bias of observations, etc. It is very useful for near real-time applications of high resolution (spatial, temporal, and spectral) satellite measurements such as geostationary imager and sounder measurements [51] with large data volume, those near real-time applications can be realized through training the model offline while applying the model online. In addition, multiple sources of data can be easily used together with machine learning techniques in the retrieval process, if they are temporally and spatially matched. Despite those advantages, using machine learning for sounding retrieval should be cautious, since it highly relies on the representativeness of training dataset as mentioned above, in addition, the RF method is lack of physical interpretation. Building a physical driven RF model is our future work, those including but are not limited to understanding the sensitivity of each channel in retrieval under various atmospheric conditions, the use of both temporal and spatial information in the RF-based retrieval model, the relative importance of other source of measurements (if available) in RF-based atmospheric water vapor retrieval, etc.

Since the retrieval accuracy and stability with RF method are highly dependent on the representativeness of the training dataset, the key to building a trustworthy model is the development of a training dataset that should cover all weather conditions and all seasons. Building such a training dataset should be the focus of future work on using machine learning for profile retrieval. In addition, the label data are also very important, the ERA5 is good moisture label data but the spatial and temporal resolutions are still not well coupled with geostationary measurements, using high resolution regional reanalysis for label data should be considered in future work. Due to the limitation of the case study, the numbers from the evaluation might be less representative, but that does not change the general conclusions and findings, for example, (1) and (3) mentioned above.