An Improved One-Dimensional Variational Method for a Ground-Based Microwave Radiometer

Abstract

Temperature and water vapor density profiles in the troposphere (from the surface to 10 km) can be retrieved from a ground-based microwave radiometer (MWR) at high temporal and moderate vertical resolution. The back-propagation neural network (BPNN) algorithm is commonly deployed in ground-based microwave radiometers. Some studies have shown that the accuracy of the BPNN retrieval algorithm is affected by training data with large deviations. In this paper, an improved 1D-VAR method is proposed, which can effectively correct the bias; the results show that the improved 1D-VAR method can provide more accurate inversion results. Compared to the BPNN and 1D-VAR methods, the root mean square errors of temperature for the improved 1D-VAR method are reduced by $60.8\%$ and $29.4\%$ during daytime and by $54.2\%$ and $49.7\%$ during nighttime, respectively.

1. Introduction

The atmospheric temperature and relative humidity profiles are important parameters of the atmospheric state, especially in the field of numerical weather prediction (NWP) [1,2]. Radiosonde observation is a useful method of detecting atmospheric temperature and humidity. This method involves launching sounding balloons equipped with sophisticated sensors, enabling highly accurate measurements of various atmospheric parameters. However, the inherent constraints of launch procedures and the substantial costs involved preclude continuous, real-time monitoring of atmospheric conditions [3]. To improve temporal coverage and resolution, microwave remote sensing techniques have been proposed [4,5]. In contrast to radiosonde observations (RAOBs), microwave remote sensing obtains the atmospheric parameters continuously, and data can be collected autonomously.

A microwave radiometer (MWR) is a passive remote sensing instrument with multiple frequency channels. It obtains a vertical distribution of atmospheric temperature and relative humidity using the V (50–60 GHz) and K (20–30 GHz) bands from the surface to an altitude of 10 km. In recent years, MWRs have achieved excellent performance when measuring atmospheric parameters, especially temperature and precipitable water vapor [6,7,8].

The retrieval of the temperature and relative humidity profile is an ill-posed problem. Various retrieval methods have been proposed, including physical algorithms, linear regression retrieval methods, neural network (NN) methods and one-dimensional variational (1D-VAR) methods [9,10]. Currently, most MWR instruments use the back-propagation neural network (BPNN) algorithm to retrieve atmospheric temperature and water vapor profiles [11]. However, in some regions, there are not enough historical atmospheric datasets or the quality of the datasets does not satisfy the neural network training requirements. Expanding the sample set by adding data from other sensors is one approach to solve these issues [12]

The one-dimensional variational method is a optimal estimation method that combines a precise forward atmosphere transport model with a priori information. Several studies have shown that 1D-VAR methods achieve good performance in the retrieval of atmospheric parameters [9,13,14,15,16]. Clive D. Rodgers introduced the basic principles of BPNN and 1D-VAR retrieval methods in detail and compared the two retrieval methods, elaborating on their advantages and limitations [17].

Recent research has investigated the integration of BPNN algorithms with MWR systems [18]. However, the efficacy of this approach continues to be bounded by constraints related to dataset dimensions and quality. In the absence of viable NN methods, an improved 1D-VAR method is introduced with an innovative iteration factor, which significantly improves the convergence efficiency of the algorithm. By incorporating this iteration factor, the method optimizes the iterative process, potentially leading to more accurate and computationally efficient retrievals of atmospheric temperature profiles from MWR observations. Cloud-induced scattering elevates retrieval errors under overcast conditions, with additional uncertainty arising from variations in cloud layer thickness. Therefore, our analysis focuses on clear-sky conditions [19].

This paper is organized as follows. The MWR and dataset used in this study are introduced in Section 2. the basic concept of the 1D-VAR retrieval method is presented in Section 3. Section 4 introduces the experimental setup. The results of the 1D-VAR retrieval method are described in Section 5. Section 6 provides a brief summary and conclusion.

2. Instruments and Datasets

2.1. Microwave Radiometer

For this study, a 22-channel MP-3000A MWR manufactured by Radiometrics Corporation (Boulder, CO, USA) was deployed in Harbin at 45.46° N, 126.41° E [20,21]. The MWR is a passive instrument; it measures the downwelling radiance for two spectral ranges. The 22 to 30 GHz (K-band) range is used to retrieve water vapor density and relative humidity profiles with 8 channels, with 22.234, 22.500, 23.034, 25.000, 26.234, 28.000 and 30.000 GHz center frequencies. The 51 to 60 GHz (V-band) range is used to retrieve temperature profiles with the remaining 14 channels with 51.248, 51.760, 52.280, 52.804, 53.336, 53.848, 54.400, 54.940, 55.500, 56.020, 56.660, 57.288, 57.964 and 58.800 GHz center frequencies. The MP-3000A MWR is equipped with surface temperature, humidity and pressure sensors, as well as an infrared sensor for measuring cloud base temperature. A BPNN retrieval model was built in the MWR, and the MWR was calibrated to ensure accuracy.

2.2. Datasets

The observed brightness temperatures (BTs) of the MWR used in this study were collected in April 2017 in Harbin at 45.46° N, 126.41° E. The retrieval products from the MWR are profiles of temperature (K) and water vapor density (g/m³). The atmospheric vertical profile from the surface to 10 km is divided into 58 layers with varying vertical resolutions at different altitude ranges. The resolutions of these profiles are measured as initial and final distances from the surface, e.g., from 0.05 km to 0.5 km, 0.1 km to 2 km, and 0.25 km to 10 km. Five-year (February 2012 to August 2016) RAOB data from Harbin station were used to train BPNN retrieval models (temperature and water vapor density) and one-year (August 2016 to September 2017) data were used for validation. Under typical balloon ascent conditions (20–30 min to reach 10 km altitude), the RAOBs can be assumed to represent vertical atmospheric profiles, as horizontal displacement remains limited within this altitude range (0–10 km). The simulated BTs were used as inputs for the BPNN retrieval model, obtained via calculations using the monochromatic radiative transfer model (MonoRTM) [22,23].

NCEP-2 is an improved version of the NCEP and has updated data assimilation techniques [24]. In this study, the five-year (January 2012–December 2017) NCEP-2 data with a spatial resolution of 2.5° × 2.5° were used as the background state of the 1D-VAR method. The red hollow circle in Figure 1 signifies the installed position of the MWR (microwave radiometer), while the blue dot designates the location surrounding the MWR utilized for reanalysis of the data and the red box represent the general area of Harbin City. The reanalysis data situated at the lower right corner position were used as the background state. The NCEP-2 reanalysis data encompass global atmospheric profile information from 1979 to the present, with four daily atmospheric profiles containing temperature and humidity data from the surface up to 17 pressure levels: 1000, 925, 850, 700, 600, 500, 400, 300, 250, 200, 150, 100, 70, 50, 30, 20 and 10 hPa. The 58 layers are interpolated and serve as the atmospheric background field ($x_b$). The radiosonde data were used as the truth values to validate the retrieval results.

3. Methodology

3.1. 1D-VAR Retrieval Method

In this study, a 1D-VAR method was used to retrieve temperature and humidity profiles. The method is based on Bayesian theory to obtain the maximum probability of atmospheric parameter values, given the BTs, using a prior knowledge:

$$P\left(x\right|y)={\displaystyle \frac{P\left(y\right|x\left)P\right(x)}{P\left(y\right)}}$$

(1)

where $P\left(x\right)$ and $P\left(y\right)$ refer to the PDF of the state vector and measurement, respectively. $P\left(y\right|x)$ represents the conditional PDF of y for a given x.

To estimate the optimal state vector (x), the prior information and the errors in state and measurement spaces are assumed to be independent and follow Gaussian distributions. The form of the atmospheric transfer model can be expressed as follows:

$$y=F\left(x\right)+ϵ$$

(2)

where $F(·)$ and $ϵ$ denote the atmospheric transfer model and system error, respectively.

The maximum a posteriori probability of the state vector (x) can be obtained using an iterative approach, which is tantamount to minimizing the cost function ($J\left(x\right)$):

$$J\left(x\right)={\left(x-x_b\right)}^T{B}^{-1}(x-x_b)+{\left[y-F\left(x\right)\right]}^T{R}^{-1}\left[y-F\left(x\right)\right]$$

(3)

where $x_b$ is the background state vector; B is the error covariance matrix of the background state vector; R is the observation error covariance matrix; and ^T and ⁻¹ represent the matrix transpose and inverse, respectively.

3.2. Improved 1D-VAR Retrieval Method

As described in Section 3.1, there are two key components in the 1D-VAR retrieval algorithm. One is the Jacobian matrix of the atmospheric radiative transfer model and the parameters of $J\left(x\right)$, which include $x_b$, B, R, $F\left(x\right)$ and K (Jacobian matrix). The other is the method that derives the optimal state vector by minimizing the cost function.

The Newton iteration method is widely employed to find the optimal solution for objective functions. In ideal circumstances, this method ensures that each iteration yields a value of the objection function smaller than the previous one [25,26].

$$x_{i+1}=x_i-{\left[{\nabla }_{x}g\left(x_i\right)\right]}^{-1}g\left(x_i\right)$$

(4)

where $x_i$ is the ith estimate of the atmospheric state vector and $g\left(x_i\right)$ represents the derivative of the cost function $J\left(x\right)$, where $g\left(x\right)$ can be written as

$$g\left(x\right)={\nabla }_{x}J\left(x\right)=B^{-1}(x-x_b)-{\left[{\nabla }_{x}F\left(x\right)\right]}^T{R}^{-1}\left[y-F\left(x\right)\right],$$

(5)

where ${\nabla }_{x}g\left(x\right)$ is the Hessian matrix of cost function $J\left(x\right)$:

$${\nabla }_{x}g\left(x\right)=B^{-1}+{\left[K\left(x\right)\right]}^T{R}^{-1}K\left(x\right)-{\left[{\nabla }_{x}K\left(x\right)\right]}^T{R}^{-1}[y-F\left(x\right)]$$

(6)

where K denotes the Jacobian matrix. K can be written as follows:

$$K\left(x\right)={\left(\begin{array}{ccc}{\displaystyle \frac{\partial y_1}{\partial x_1}}& ...& {\displaystyle \frac{\partial y_{22}}{\partial x_1}}\\ ⋮& \ddots & ⋮\\ {\displaystyle \frac{\partial y_1}{\partial x_{58}}}& ...& {\displaystyle \frac{\partial y_{22}}{\partial x_{58}}}\end{array}\right)}^T$$

(7)

where ${\displaystyle \frac{\partial y_i}{\partial x_j}}$ represents the sensitivity of the observed state (y) to perturbations in the state vector x, where j denotes the microwave radiometer observation channel (j = 1, 2, …, 22) and i denotes the atmospheric state corresponding to the altitude layer (i = 1, 2, …, 58).

The Newton–Gauss iterative formula is derived as follows:

$$x_{i+1}=x_i+{\left[B^{-1}+K_i^T{R}^{-1}{K}_i\right]}^{-1}\left[K_i^T{R}^{-1}(y-F\left(x_i\right))+B^{-1}(x_i-x_b)\right]$$

(8)

However, the Newton iteration method occasionally exhibits convergence issues when applied to optimize the objective function ($J\left(x\right)$). This convergence issue arises due to problems with the iteration step size. To address this, we introduce a step-size factor ($\alpha$) into the original Newton iteration method.

$$x_{i+1}=x_i-\alpha \ast {\left[{\nabla }_{x}g\left(x_i\right)\right]}^{-1}g\left(x_i\right)$$

(9)

Substituting Equations (5) and (6) into (9), the Newton iteration method is derived as follows:

$$x_{i+1}=(1-\alpha )x_i+\alpha \left[x_b+B{K}_i^T{(K_{i}B{K}_i^T+R)}^{-1}[y-F\left(x_i\right)+K_i(x_i-x_b)]\right]$$

(10)

where $\alpha =0.5$ in this paper.

3.3. Optimal Configuration of the 1D-VAR Method

The background state ($x_b$) is part of the a priori knowledge and represents the initial atmospheric state. As shown in Section 2.2, a background state vector is derived from NCEP-2 reanalysis data.

The background error covariance matrix (B) represents the correlation between the true atmospheric state and the background at varying altitudes. In this study, the background error covariance matrix (B) can be obtained via the NCEP-2 reanalysis data and RAOB data (00:00 and 12:00 UTC) and written as follows:

$$B=E\left[(x_{NCEP}-x_{RAOB}){(x_{NCEP}-x_{RAOB})}^T\right]$$

(11)

where $x_{NCEP}$ represents the NCEP reanalysis data and $x_{RAOB}$ represents the radiosonde data.

Figure 2 shows the background error covariance of temperature and water vapor density. Both the abscissa and ordinate axis represent altitude from the surface up to 10 km. The background error covariance matrix (B) comprises a $58\times 58$-dimensional matrix. The elements within the matrix represent the error covariances between the NCEP reanalysis data and the radiosonde data (true atmospheric state) at various altitudes. As shown in Figure 2, the temperature error covariance matrix presents a large covariance near the diagonal, especially within the 0 to 5 km altitude range, while the water vapor density background error covariance matrix exhibits large covariance at the lower altitude levels. Above 2 km, the low water vapor content results in smaller covariances for the water vapor density.

Under the assumption that there is no correlation between the 22 frequency channels, the observation error covariance matrix (R) can be written as $R=N+F$, where the diagonal matrix (N) represents the instrument noise and the diagonal matrix (F) represents the error of MonoRTM [27].

4. Experimental Setup

To eliminate an obstruction above the antenna, the MWR was positioned on the rooftop (as shown in Figure 3). An antenna shroud was placed below the antenna to shield it from background radiation reflecting from the ground surface.

However, the detection by the ground-based MWR sensor was susceptible to interference from solar radiation and the surface materials present at the observation location.

For the experimental validation, the impact of ground radiation background on the MWR required consideration. Comparative analysis was conducted between the inversion outcomes obtained during daytime and nighttime.

As shown in Figure 4, significant discrepancies were observed between ground sensor and radiosonde data at 00:00 and 12:00 (UTC time) during April 2017. At 00:00, the surface temperatures measured by the MWR were notably higher than those recorded by radiosondes, with an average error of 8.2 K and a maximum error of 16.2 K. Conversely, at 12:00, the MWR-measured surface temperatures were lower than the radiosonde data, with an average error of 4.1 K and a maximum error of 12.2 K. The bias in ground sensor measurements significantly impacts the retrieval results of the MWR. This issue can be effectively corrected using the improved 1D-VAR method.

5. Results

The experiments reported in this section utilized radiosonde data as reference truth values for validation and analysis. The output resulting from the 1D-VAR retrieval algorithm is evaluated quantitatively based on the root mean square error (RMSE), mean error (ME) and mean absolute error (MAE).

$$RMSE=\sqrt{{\displaystyle \frac{1}{n}}\sum _{i=1}^n{(R_i-O_i)}^2}$$

(12)

$$ME={\displaystyle \frac{1}{n}}\sum _{i=1}^n(R_i-O_i)$$

(13)

$$MAE={\displaystyle \frac{1}{n}}\sum _{i=1}^n\left|(R_i-O_i)\right|$$

(14)

where $R_i$ represents the RAOB data, $O_i$ represents the retrieval value and n is the sample size.

After excluding anomalous and cloudy instances, 52 clear-sky temperature and humidity soundings remained, which were separately assessed for daytime and nighttime to account for varying effects of background radiation on the ground sensor. The resulting statistics are presented in Figure 5 and Figure 6.

5.1. Statistics in Daytime

The statistical errors of the retrieval results for 52 cases during daytime are summarized in Figure 5. Figure 5a,b show the RMSerrors and the mean errors for the improved 1D-VAR(1D-VAR-$\alpha$,$\alpha =0.5$), 1D-VAR and BPNN methods. The improved 1D-VAR method achieves better performance than the BPNN and 1D-VAR methods; RMS temperature errors of the BPNN deviate considerably from the surface to a 10 km height, with a maximum RMSEgreater than 8.9 K. Figure 5a shows the RMS errors for the improved 1D-VAR, 1D-VAR and BPNN methods. The RMS errors of the improved 1D-VAR method are reduced by 0.5–1.4 K randomly within the range of 1–3 km compared to the 1D-VAR method. Figure 5b shows the mean errors for the three methods. The 1D-VAR method displays significantly higher errors (maximum ME = 2.18 K) in the 0–0.5 km layer, whereas the improved 1D-VAR reduces this maximum error to 1.27 K. Between 0.5 and 1 km, the improved 1D-VAR method shows a controlled error growth, remaining constrained within 1.5 K. The RMS errors and mean errors for water vapor density are shown in Figure 5c,d. Similar to the temperature profile retrievals, the BPNN retrieval method achieves the worst performance above 0.7 km. The maximum RMS errors of the improved 1D-VAR method and 1D-VAR method compared to the BPNN method are 0.2 g/m³ and 0.4 g/m³, respectively. The retrieval errors of the improved 1D-VAR method are significantly improved compared with the 1D-VAR method and BPNN method at most height levels.

5.2. Statistics in Nighttime

Figure 6a–d show the temperature and water vapor density error profiles retrieved via the improved 1D-VAR (1D-VAR-α,α = 0.5), 1D-VAR and BPNN approaches during nighttime. As shown in Figure 6a,b, the improved 1D-VAR method achieves the best performance below 6 km. In Figure 6b, the 1D-VAR method exhibits a non-monotonic mean error profile—decreasing initially before increasing with altitude (as shown by the red dashed trajectory within the range of 5–8 km). In Figure 6c, the RMS errors of the improved 1D-VAR are smaller than the those of the 1D-VAR method below 1 km. In the range of 1–6 km, the results of the improved 1D-VAR method and 1D-VAR are better than those of the BPNN method.

Table 1. Statistical significance in temperature and water vapor density profiles for clear-sky cases.

Time Period	Retrieval Method	Temperature (K)		Water Vapor Density (g/m3)
		RMSE	MAE	RMSE	MAE
Daytime	BPNN	4.29	2.38	0.76	0.47
	1D-VAR	2.38	1.84	0.77	0.46
	Improved 1D-VAR	1.68	1.28	0.68	0.40
Nighttime	BPNN	4.17	3.20	0.77	0.45
	1D-VAR	3.80	2.92	0.70	0.41
	Improved 1D-VAR	1.91	1.52	0.64	0.36

presents the comparative temperature and water vapor density retrieval outcomes from 0 to 10 km for the BPNN, 1D-VAR and improved 1D-VAR methods. Evidently, the improved 1D-VAR approach demonstrates superior performance over the BPNN and 1D-VAR methods for temperature and water vapor density profile retrieval.

6. Summary and Conclusions

The optimization and validation of numerical weather prediction models require accurate vertical atmospheric profiles with high temporal resolution. This requirement underscores the need for advanced measurement techniques capable of providing detailed atmospheric parameters across various altitudes. Ground-based MWRs have emerged as promising instruments to meet the aforementioned requirements, exhibiting remarkable capabilities in providing high-temporal-resolution vertical atmospheric profiles within the range of 0–10 km. However, the MWR’s built-in BPNN retrieval method has large retrieval errors both in temperature and humidity profiles in some cases. To meet the requirements of accuracy, we improved the 1D-VAR algorithm to retrieve the vertical profile of temperature and water vapor density by incorporating NCEP2 reanalysis profiles. The improved 1D-VAR retrieval method proposed in this paper improves the iteration efficiency by adding an iteration factor ($\alpha$) and obtains more accurate atmospheric temperature and water vapor density profiles. Experimental validation of the proposed technique was also demonstrated.

Through statistical comparisons between the three retrieval methods (BPNN, 1D-VAR and improved 1D-VAR), we conclude the following:

• The results of the BPNN retrieval method show large deviations when the ground measurement sensors are faulty, and the improved 1D-VAR method can effectively correct these errors.
• Compared to the BPNN and 1D-VAR methods, the RMSE of temperature of the improved 1D-VAR method are reduced by $60.8\%$ and $29.4\%$ during daytime and by $54.2\%$ and $49.7\%$ during nighttime, respectively.
• Compared to the BPNN and 1D-VAR methods, the accuracy of the improved 1D-VAR method regarding water vapor density is not significant. Although the RMS error from the surface to 10 km is reduced, the RMS errors from the surface to 1 km are higher than those of the BPNN method. In the next step, the improved 1D-VAR method and the BPNN method can be fused to achieve high retrieval results for water vapor density.

Moving forward, we will improve the parameter-tuning methods of the 1D-VAR inversion model to address the problem of adaptive tuning of stepping factors.