A Comparative Study of Bulk Parameterization Schemes for Estimating Cloudy-Sky Surface Downward Longwave Radiation

Abstract

Parameterization schemes (bulk formulae) are widely used to estimate all-sky surface downward longwave radiation (SDLR) due to the simple, readily available inputs and acceptable accuracy from local to regional scales. Seven widely used bulk formulae are evaluated using the ground measurements collected from 44 globally distributed flux measurement sites of five networks. The Bayesian model averaging (BMA) method is introduced to integrate multiple bulk formulae to obtain an estimate of cloudy-sky SDLR for the first time. The second multiple linear regression model of Carmona et al. (2014) performs the best, with BIAS, RMSE, and R² of zero, 20.13 W·m⁻² and 0.87, respectively. The BMA method can achieve balanced results that are close to the accuracy of the second multiple linear regression model of Carmona et al. (2014) and better than the average accuracy of seven bulk formulae, with BIAS, RMSE, and R² of −1.08 W·m⁻², 21.99 W·m⁻² and 0.87, respectively. In addition, the bulk formula of Crawford and Duchon (1999) is preferred if there is insufficient data to calibrate the bulk formulae because it does not need local calibration and has an acceptable accuracy, with BIAS, RMSE, and R² of 0.96 W·m⁻², 26.58 W·m⁻² and 0.82, respectively. The effects of climate type, land cover type, and surface elevation are also investigated to fully assess the applicability of each bulk formula and BMA. In general, there is no cloudy-sky bulk parametrization scheme that can be successfully applied everywhere.

1. Introduction

The radiation budget at the Earth’s surface is an important factor that determines land surface processes such as evapotranspiration, oceanic and atmospheric circulations [1,2]. Surface downward longwave radiation (SDLR, 4–100 μm) is one of four components required to calculate the Earth’s surface radiation budget [3,4,5]. Usually, a considerable part of the earth’s surface is covered by clouds [6]. In cloudy sky, SDLR is emitted from clouds (aerosol, water droplets, and ice crystals) and atmosphere columns (H₂O, CO₂, and O₃ molecules) between the cloud base and the surface. Numerous studies demonstrated that clouds have a nonneglectable contribution to SDLR, especially in the thermal infrared atmosphere window [7,8,9], because clouds can simultaneously absorb solar radiation and re-emit outgoing longwave radiation (OLR) to space and SDLR to the surface.

SDLR can be accurately measured using ground-based instruments (e.g., pyrgeometer). However, this method is relatively expensive and sensitive and has spatially sparse coverage at the global scale [10]. Therefore, many researchers have used remote sensing and meteorological data to estimate high spatial-temporal SDLR [11,12,13,14], and many SDLR retrieval algorithms have been proposed. These algorithms can be classified into three categories: physical, parameterized, and statistical methods [15,16]. Relatively simple parameterization schemes are highly praised in that they only require a limited input set of atmospheric variables and have acceptable accuracy [17,18,19]. For example, Wang and Liang [20] applied the bulk formulae of Brunt [21] and Brutsaert [22] to estimate all-sky SDLR from global available meteorological observations to calculate the decadal variation in SDLR. The general form of the parameterization scheme (also called the bulk formula) is expressed as

$$SDLR={\mathsf{\sigma }\mathsf{\epsilon }}_{eff}{T}_{eff}^4$$

(1)

where ${\mathsf{\epsilon }}_{eff}$ is the effective emissivity of the atmosphere under all-sky conditions, $T_e$ is atmospheric effective temperature, and σ is the Stefan-Boltzmann constant. Most of the bulk formulae are applicable to clear-sky cases [12,21,22,23,24], and few derive all-sky SDLR [17,25,26,27,28,29,30]. Under cloudy-sky conditions, the liquid water and ice absorb and emit longwave radiation more effectively than water in the vapor phase. Thus, clouds increase SDLR significantly, and cloud cover is usually introduced to calculate atmospheric effective emissivity [7,15,25].

Numerous studies have been conducted to analyze the advantages, shortcomings and applicability of these bulk formulae. For example, Kjaersgaard et al. [31] evaluated the performance of 20 clear-sky bulk formulae using long-term ground measurements at two sites in Demark. Their results suggested that four bulk formulae, proposed by Swinbank [32], Brutsaert [22], and Prata [23] and the Food and Agricultural Organization (1990) parameterization of the Brunt [21] model, have smaller discrepancies than the other bulk formulae. Kjaersgaard et al. [31] combined the above four models with the cloudy-sky bulk formulae of Korsgaard [33] and Crawford and Duchon [26] to evaluate the accuracy of SDLR estimates. The mean bias errors of the two combinations ranged from −8 to 8 W·m⁻², and the root mean square errors (RMSEs) ranged from 23 to 30 W·m⁻². Flerchinger et al. [34] investigated the accuracy of 13 clear-sky SDLR bulk schemes at 21 sites across North America and China. Their study indicated that the bulk schemes developed by Dilley et al. [35], Prata [23], and Angstrom [36] are good at predicting surface SDLR. Flerchinger et al. [34] combined the cloud-correction methods of Kimball and Idso [9], Unsworth and Monteith [37], and Crawford and Duchon [26] with the clear-sky formulation of Dilley and O’Brien [35] to obtain estimates of the all-sky SDLR. The results showed that the average root mean square deviation across all sites for three cloud corrections was approximately 24 to 25 W·m⁻² for 30-minute or hourly estimates. Zhu et al. [19] comprehensively assessed all-sky atmospheric emissivity using 13 clear-sky bulk schemes combined with 10 cloud corrections at five sites in high mountain regions of the Tibetan Plateau. The results showed that the best all-sky parameterization scheme for the five sites was Dilley and O’Brien’s A model [35] for clear-sky conditions combined with Sicart et al.’s A cloud correction [38], with a mean RMSE of 26.3 W·m⁻² and a mean MBE of zero. These clear-sky or all-sky bulk formulae have different accuracies at local scales. If they were applied directly to different areas, the inversion accuracy would be reduced and their applicability would be suppressed. Which bulk formula is best adapted to SDLR estimate under a specific circumstance remains unclear. In addition, selection of a suitable bulk formula lacks guidance.

A few studies explored new ways to estimate all-sky SDLR by taking advantage of existing bulk formulae with good performance. They found that the Bayesian model averaging (BMA) method is effective to estimate clear-sky SDLR [39,40]. The estimation potential of BMA for all-sky SDLR and its accuracy and applicability have not been determined. Thus, the purpose of this study is to investigate the accuracy and applicability of widely used cloudy-sky bulk formulae and the utility of BMA in estimating cloudy-sky SDLR. First, bulk formulae for cloudy-sky SDLR estimate are calibrated and integrated using BMA. Then, the bulk formulae and BMA are validated using ground measurements. Finally, the performance of the bulk formulae and BMA are assessed under the circumstances of different climate types, land covers and surface elevations.

2. Materials and Methods

2.1. Ground Measurements

The surface radiation network can provide long-term, accurate records of components of the surface radiation budget and meteorological observations, which can be used to detect important changes in the Earth’s radiation field at the Earth’s surface and support the validation and confirmation of satellite and computer model estimates [20]. After filtering the poor-quality data using the built-in quality control (QC) information, high quality ground-measured SDLR, surface downward shortwave radiation and meteorological variables (air temperature and relative humidity) collected at 44 globally distributed sites from 5 independent surface radiation networks, including 14 sites from the Baseline Surface Radiation Network (BSRN) [41], 12 sites from AmeriFlux [42], 8 sites from AsiaFlux [42], 6 sites from SURFRAD [43], and 4 sites from the Coordinated Energy and Water Cycle Observations Project (CEOP) [44] (Figure 1), are used in this study. These sites have a latitude span of 78°W to 70°E, with elevations ranging from 4 m to 5038 m, and they cover a variety of land types, including bare land, desert, cropland, grassland, and forest. The 44 sites represent different climate and ecosystem conditions.

Table 1. Selected sites in the surface radiation networks.

NO.	Short Name	Full Name	Latitude	Longitude	Elevation (m)	Land Cover	Climate Type	Temporal Resolution	Time Period
1	Bondville1	Bondville, Illinois	40.05	−88.37	213	Cropland	Dfa	3 min	2003–2005
2	Boulder1	Boulder, Colorado	40.13	−105.24	1689	Grassland	BSk	3 min	2003–2005
3	Fort Peck1	Fort Peck, Montana	48.31	−105.10	634	Grassland	BSk	3 min	2003–2005
4	Desert Rock1	Desert Rock, Nevada	36.63	−116.02	1007	Desert	BWh	3 min	2003–2005
5	Penn State1	PennState, Pennsylvania	40.72	−77.93	376	Cropland	Dfb	3 min	2003–2005
6	Sioux Falls1	Sioux Falls, South Dakota	43.73	−96.62	473	Cropland	Dfa	3 min	2003–2005
7	US-Blk2	Black Hills	44.16	−103.65	1718	Evergreen needle leaf forest	Dfb	30 min	2004–2006
8	US-Bo22	Bondville (companion site)	40.01	−88.29	219.3	Cropland	Dfa	30 min	2004–2006
9	US-Bkg2	Brookings	44.35	−96.84	510	Grasslands	Dfa	30 min	2004–2006
10	US-CaV2	Canaan Valley	39.06	−79.42	994	Grasslands	Cfb	30 min	2004–2006
11	US-NR12	Niwot Ridge Forest	40.03	−105.55	3050	Evergreen needle leaf forest	Dfc	30 min	2001–2003
12	US-FPe2	Fort Peck	48.31	−105.1	634	Grassland	BSk	30 min	2003–2005
13	US-Goo2	Goodwin Creek	34.25	−89.87	87	Grasslands	Cfa	30 min	2003–2005
14	US-MMS2	Morgan Monroe State Forest	39.32	−86.41	275	Deciduous broadleaf forest	Cfa	30 min	2001–2003
15	US-WBW2	Walker Branch Watershed	35.96	−84.29	283	Deciduous broadleaf forest	Cfa	30 min	2003–2005
16	US-Wrc 2	Wind River Crane Site	45.82	−121.95	371	Evergreen needleleaf forest	Csb	30 min	2003–2005
17	US-WCr2	Willow Creek	45.81	−90.08	520	Deciduous broadleaf forest	Dfb	30 min	2003–2005
18	US-MOz2	Missouri Ozark Site	38.74	−92.20	220	Deciduous broadleaf forest	Cfa	30 min	2003–2005
19	QHB3	Qinghai Flux Research Site	37.61	101.33	3250	Grasslands	BSk	15 min	2003–2004
20	MKL3	Mae Klong	14.58	98.84	231	Mix forest	Am	15 min	2003–2004
21	TKY3	Takayama	36.15	137.42	1420	Deciduous broadleaf forest	Dfb	15 min	2003–2005
22	TMK3	Tomakomai Flux	42.74	141.52	140	Deciduous needle leaf forest	Dfb	15 min	2001–2003
23	BKS3	Bukit Soeharto	−0.86	117.04	20	Evergreen broadleaf forest	Af	15 min	2001–2002
24	FJY3	Fujiyoshida	35.45	138.76	1030	Deciduous needleleaf forest	Cfa	15 min	2000
25	LSH3	Laoshan	45.28	127.58	340	Deciduous needleleaf forest	Cfc	15 min	2002
26	SKR3	Sakaerat	14.49	101.92	543	Evergreen broadleaf forest	Aw	15 min	2001–2003
27	Amdo4	AmdoTower	32.24	91.62	4695	Bare land	ET	60 min	2002–2004
28	BJ4	BJTower	31.37	91.90	4509	Bare land	ET	60 min	2002–2004
29	D1054	D105AWS	33.06	91.94	5038	Bare land	ET	60 min	2002–2004
30	Gaize4	Gaize	32.30	84.05	4416	Bare land	Dwb	60 min	2002–2004
31	BOU5	Boulder	40.05	−105.01	1577	Grasslands	BSk	1 min	2003–2005
32	CAR5	Carpentras	44.08	5.06	100	Cultivated	Csb	1 min	2003–2005
33	DAR5	Darwin	−12.43	130.89	30	Grasslands	Aw	1 min	2003–2005
34	LIN5	Lindenberg	52.21	14.12	125	Cultivated	Dfb	1 min	2003–2005
35	MAN5	Momote	−2.06	147.43	6	Grasslands	Af	1 min	2003–2005
36	NAU5	Nauru Island	−0.52	166.92	7	Rock	Af	1 min	2003–2005
37	NYA5	Ny-Ålesund	78.93	11.93	141	Tundra	ET	1 min	2003–2005
38	PAY5	Payerne	46.82	6.94	491	Cultivated	Dfb	1 min	2003–2005
39	REG5	Regina	50.21	−104.71	578	Cultivated	Dfb	1 min	2003–2005
40	E135	Southern Great Plains	36.61	−97.49	318	Grasslands	Cfa	5 min	2003–2005
41	TAT5	Tateno	36.05	140.13	25	Grasslands	Cfa	1 min	2003–2005
42	DAA5	De Aar	−30.7	23.99	1287	Desert	BWk	5 min	2002–2004
43	GVN5	Georg von Neumayer	−70.65	−8.25	42	Ice	EF	1 min	2003–2005
44	SBO5	Sede Boqer	30.86	34.78	500	Desert	BWk	1 min	2003–2005

Note: 1, 2, 3, 4, and 5 represent SURFRAD sites, AmeriFlux sites, AsiaFlux sites, CEOP sites, and BSRN sites, respectively.

gives detailed descriptions of these sites.

2.2. Methods

2.2.1. Cloudy-Sky Bulk Formulae

Cloudy sky is identified using the cloud fraction function c [26], which is defined as

$$\mathrm{c}=1-\frac{Sw\downarrow }{Sw{\downarrow }_0}$$

(2)

where $Sw\downarrow$ is the observed downward shortwave radiation, $Sw{\downarrow }_0$ is the theoretical value of downward shortwave radiation; when c > 0.05, we define the situation as cloudy sky.

The equation of the selected cloudy-sky bulk formulae are shown in

Table 2. Seven cloudy-sky bulk formulae for surface downward longwave radiation (SDLR) estimation.

Model	Bulk Formulae	Equation
1	Jacobs [29]	$\mathrm{SDLR}=SDL{R}_{clr}\left(1+a_{1}c\right)$
2	Lhomme et al. [27]	$\mathrm{SDLR}=SDL{R}_{clr}\left(a_2+b_{2}c\right)$
3	Maykut and Church [45]	$\mathrm{SDLR}=SDL{R}_{clr}\left(1+a_3{c}^{b_3}\right)$
4	Konzelmann et al. [28]	$\mathrm{SDLR}=SDL{R}_{clr}\left(1-a_4{c}^{b_4}\right)+d_4{c}^{e_4}\sigma T^4$
5	Crawford and Duchon [26]	$\mathrm{SDLR}=SDL{R}_{clr}\left(1-c\right)+c\sigma T^4$
6	Carmona1 [25]	$\mathrm{SDLR}=\left[\left(a_6+b_{6}T+d_{6}RH\right)\left(1-c\right)+c\right]\sigma T^4$
7	Carmona2 [25]	$\mathrm{SDLR}=\left(a_7+b_{7}T+d_{7}RH+e_{7}c\right)\sigma T^4$

Note: Carmona1 and Carmona2 denote the first and second multiple linear regression models of Carmona et al. (2014), respectively. The selected seven bulk formulae are abbreviated as Jacobs, Lhomme, MC, Konzelmann, Crawford, Carmona1 and Carmona2, respectively. $a$, $b$, d, and $e$ are the coefficients, $SDL{R}_{clr}$ is the clear-sky SDLR, $T$ is the screen-level temperature (unit: K), $\mathsf{\sigma }$ is the Stephen–Boltzmann constant (5.67·10⁻⁸ W·m⁻²·K⁻⁴), $RH$ is the relative humidity (%), and $c$ is the cloud fraction.

. Herein, we briefly described these bulk formulae. More details please refer to their respective papers. The bulk formula of Jacobs [29] is similar to that of Lhomme et al. [27], who proposed a cloudy-sky bulk formula by combining the bulk formula of Brutsaert [22] and a correction for cloudiness using the measurements obtained from two different sites on the Andean Altiplano, which is a vast plateau lying at an average altitude of 4000 m. The bulk formula of Maykut and Church [45] was developed using the five-year observations of the radiation climate near Point Barrow, Alaska. Konzelmann et al. [28] developed the bulk formula to estimate cloudy-sky SDLR using the meteorological measurements from various projects on West Greenland. Crawford and Duchon [26] developed an improved parameterization scheme for estimating effective atmospheric emissivity by explicitly incorporating the fractional cloudiness using the data from the Atmospheric Radiation Measurement (ARM) program. Carmona et al. [25] proposed two multiple linear regression models to estimate SDLR for all sky conditions over a sub-humid region in Argentina. Clearly, these bulk formulae are likely to be specific to the atmospheric circumstances under which they were developed, and should be handled with care when applying to other conditions.

According to the study of Guo et al. [40], the clear-sky bulk formula of Carmona et al. [25] has good performance at global scale. Thus, we selected their bulk formula to calculate the clear-sky SDLR in the cloudy-sky bulk formulae shown in Table 2. The clear sky bulk formula of Carmona et al. [25] is expressed as

$$SDL{R}_{clr}=\left(\alpha +\beta T_a+\gamma RH\right)\times \sigma T_a^4$$

(3)

where $\alpha$, $\beta$, and $\gamma$ are coefficients, and $T_a$ and $RH$ are the air temperature and relative humidity at screen level, respectively.

2.2.2. Bayesian Model Averaging

In this study, the Bayesian Model Averaging (BMA) method is introduced to integrate selected bulk formulae for predicting SDLR under all-sky condition for the first time. The BMA method uses the posterior probability as the weight to conduct a weighted average of the possible individual models [46,47]. The posterior probability reflects the models’ predictive performance and can be derived in the training period [46].

We employ $SDL{R}_p$ and $SDL{R}_o$ to denote the predicted and observed SDLRs at a given time, respectively $f_1,f_1,\cdots f_n$ is an ensemble of n bulk formulae (Table 2) used to predict $SDL{R}_p$. According to the total probability formula, the predictive probability density function (PDF) of $SDL{R}_p$ based on the multi-bulk formulae ensemble can be calculated as

$$\mathrm{p}\left(SDL{R}_p|f_1,f_1,\cdots f_n\right)={{\displaystyle \sum }}_{i=1}^n\mathrm{p}\left(SDL{R}_p|f_i\right)\mathrm{p}\left(f_i|SDL{R}_o\right)$$

(4)

where $\mathrm{p}\left(f_i|SDL{R}_o\right)$ is the forecast PDF of bulk formula $f_i$ alone, and $\mathrm{p}\left(f_i|SDL{R}_o\right)$ is the posterior probability of bulk formula $f_i$, which can reflect how well the bulk formula $f_i$ fits the observations. The PDF of BMA is a weighted average of the conditional PDFs of each individual bulk formula, weighted by their posterior model probabilities. The posterior probabilities of all single bulk formulae add up to one, i.e., ${{\displaystyle \sum }}_{i=1}^n\mathrm{p}\left(f_i|SDL{R}_o\right)=1$, and they can be viewed as weights $w_i$. Equation (4) can be rearranged as:

$$\mathrm{p}\left(SDL{R}_p|f_1,f_1,\cdots f_n\right)={{\displaystyle \sum }}_{i=1}^n{w}_i\mathrm{p}\left(SDL{R}_p|f_i\right)$$

(5)

Assuming the conditional PDF of $SDL{R}_p$ is normally distributed with mean value of E, and variance ${\sigma }^2$. Denote the $\mathrm{g}(·)$ to represent the associated Gaussian PDF of each bulk formula.

$$\mathrm{p}\left(SDL{R}_p|f_i\right)=\mathrm{g}\left(SDL{R}_p|\{E_i,{\sigma }_i^2\}\right)$$

(6)

$$\mathrm{p}\left(SDL{R}_p|f_1,f_1,\cdots f_n\right)={{\displaystyle \sum }}_{i=1}^n{w}_i\mathrm{g}\left(SDL{R}_p|\{E_i,{\sigma }_i^2\}\right)$$

(7)

The optimal estimation of SDLR by the BMA method is to derive the conditional expected value of $SDL{R}_p$, which can be written as follows:

$$\mathrm{E}\left(SDL{R}_p|f_1,f_1,\cdots f_n\right)={{\displaystyle \sum }}_{i=1}^n{w}_i{E}_i$$

(8)

As a result, the key problem is obtaining the posterior probabilities $w_i$ of each model, which makes the estimated $SDL{R}_p$ closest to the measurement $SDL{R}_o$. The $w_i$ can be estimated by maximum likelihood from the training data [48]. The EM algorithm is iterative and alternates between the Expectation (or E) step and the Maximization (or M) step to find the maximum likelihood value [49]. Firstly, we assume that all the bulk formulae have the same weight, then the Expectation step can be written as:

$$z_{i,t}=\frac{w_i\mathrm{g}\left(SDL{R}_{o,t}|\{E_{i,t},{\sigma }_i^2\}\right)}{{{\displaystyle \sum }}_{i=1}^n{w}_i\mathrm{g}\left(SDL{R}_{o,t}|\{E_{i,t},{\sigma }_i^2\}\right)}$$

(9)

where the $z_{i,t}$ denotes the probability that the bulk formula $i$ to be the best forecaster at training data $t$. The M step will estimate the $w_i$ using the $z_{i,t}$ calculated in E step:

$$w_i=\frac{1}{N}{\displaystyle \sum }_t{z}_{i,t}$$

(10)

$${\mathsf{\sigma }}_i^2=\frac{1}{N}{\displaystyle \sum }_t{{\displaystyle \sum }}_{i=1}^n{z}_{i,t}{\left(E_{i,t}-SDL{R}_{o,t}\right)}^2$$

(11)

where $N$ is the number of observations in the training dataset, n is the number of bulk formulae. The E and M steps are iterated to convergence, and then the $w_i$ for each bulk formula is obtained.

3. Results and Discussion

3.1. Calibration and Validation

Three indicators, the mean bias error (BIAS, Equation (12)), root-mean-square error (RMSE, Equation (13)) and determination coefficient (R², Equation (14)), are used to evaluate the performance of cloudy-sky bulk formulae.

$$\mathrm{BIAS}=\frac{1}{n}{\displaystyle \sum }_{i=1}^n{\left(SDL{R}_{e,i}-SDL{R}_{o,i}\right)}^2$$

(12)

$$\mathrm{RMSE}=\sqrt{\frac{1}{n-1}{\displaystyle \sum }_{i=1}^n{\left(SDL{R}_{e,i}-SDL{R}_{o,i}\right)}^2}$$

(13)

$${\mathrm{R}}^2=1-\raisebox{1ex}{$\frac{1}{n}{{\displaystyle \sum }}_{i=1}^n{\left(SDL{R}_{e,i}-SDL{R}_{o,i}\right)}^2$}\!\left/ \!\raisebox{-1ex}{${{\displaystyle \sum }}_{i=1}^n{\left(SDL{R}_{o,i}-\overline{SDL{R}_{o,i}}\right)}^2$}\right.$$

(14)

where $SDL{R}_{e,i}$ is the estimated SDLR, $SDL{R}_{o,i}$ is the observed SDLR, $\overline{SDL{R}_{o,i}}$ is the mean $SDL{R}_{o,i}$, and n is the number of samples.

The collected samples in Section 2.1 were used to calibrate and validate the selected cloudy-sky bulk formulae and validate the results integrated by BMA. We randomly divided the collected samples into two parts: two-thirds of samples were used to calibrate the bulk formulae to remove the potential error caused by the local coefficients; the remaining one-third of samples were used to validate the calibrated bulk formulae. To examine the performance of calibration, the bulk formulae with original coefficients were also validated using the same data.

Table 3. Variation of coefficient values for six cloudy-sky bulk formulae before and after calibration.

Bulk Formula	Coefficients	Original Coefficients	Adjusted Coefficients	Relative Difference (%)
Jacobs [29]	a1	0.26	0.17	−0.36
MC [45]	a2	0.22	0.12	−0.44
	b2	2.75	0.30	−0.89
Lhomme [27]	a3	1.03	1.06	0.03
	b3	0.34	0.07	−0.80
Konzelmann [28]	a4	1	0.29	−0.71
	b4	4	0.23	−0.94
	d4	0.952	0.34	−0.64
	e4	4	0.27	−0.93
Carmona1 [25]	a6	−0.88	0.33	−1.38
	b6	5.2·10−3	8.05·10−4	−0.85
	d6	2.02·10−3	3.90·10−3	0.93
Carmona2 [25]	a7	−0.34	0.55	−2.61
	b7	3.36·10−3	5.05·10−4	−0.85
	d7	1.94·10−3	2.57·10−3	0.32
	e7	0.213	5.6·10−2	−0.74

shows the variation of coefficients among six bulk formulae. The bulk formula of Crawford and Duchon [26] is not considered. This is due to fact that Crawford and Duchon [26] is a special case of Konzelmann [4], and the corresponding model coefficients are all one. Table 3 shows that the relative variation of the bulk formula coefficients is greater than 30% except for that of Lhomme [27], and more than half of the bulk formula coefficients are greater than 50%. This demonstrates the necessity of calibrating the bulk formulae.

The validation results are shown in Figure 2. Most of the samples are distributed around the 1:1 line except a small number of samples that have larger deviations. The statistical performance is shown in

Table 4. Validation results of seven calibrated cloudy-sky bulk formulae and the BMA method. The results for the original cloudy-sky bulk formulae and BMA method are also provided.

Bulk Formula	Original			Calibrated
	BIAS	RMSE	R2	BIAS	RMSE	R2
Jacobs [29]	7.63	31.25	0.85	−6.91	26.36	0.86
MC [45]	−16.78	31.68	0.85	−2.70	24.62	0.87
Lhomme [27]	29.76	47.15	0.83	−2.69	24.65	0.87
Konzelmann [28]	−25.55	34.68	0.86	−1.69	23.32	0.87
Crawford [45]	0.96	26.58	0.82	0.96	26.58	0.82
Carmona1 [25]	−8.80	31.72	0.79	4.44	24.21	0.83
Carmona2 [25]	−11.33	30.01	0.83	0.00	20.13	0.87
BMA	−5.47	27.28	0.85	−1.08	21.99	0.88

. The BIAS values of the calibrated bulk formulae range from −6.91 W·m⁻² to 4.44 W·m⁻², with an average of 2.77 W·m⁻², whereas those of the original bulk formulae are between −25.55 W·m⁻² and 29.76 W·m⁻², with an average of 14.40 W·m⁻². The RMSE of the calibrated bulk formulae range from 20.13 W·m⁻² to 26.58 W·m⁻², with an average of 24.27 W·m⁻², whereas those of the original bulk formulae are between 26.58 W·m⁻² and 47.15 W·m⁻², with a mean of 33.30 W·m⁻². The R² are slightly improved after calibration. The bulk formula of Carmona2 performs the best, with the lowest zero BIAS and RMSE of 20.13 W·m⁻² and second highest R² of 0.87.

The BMA yields balanced results, and its accuracy is only slightly worse than the bulk formula of Carmona2 and better than the other bulk formulae, with the highest R² of 0.88, second-lowest RMSE of 21.99 W·m⁻², and a relatively lower BIAS of −1.08 W·m⁻². The balance indicates that the three indicators of BMA are better than the average of those for seven selected bulk formulae. The performances of other bulk formulae are also acceptable, with BIAS ranges from −6.91 W·m⁻² to 4.44 W·m⁻², RMSE ranges from 23.32 W·m⁻² to 26.58 W·m⁻², and R² around 0.85. The coefficients of bulk formulae of Carmona are tunable during the calibration, whereas the ${\mathrm{SDLR}}_{clr}$ in other bulk formulae is provided as a priori and only the coefficients related to cloud fraction are tunable. This partly explains the good performance of Carmona2. Note the bulk formula of Crawford and Duchon [26] do not need calibration. The accuracy is acceptable, with a BIAS, RMSE and R² of 0.96 W·m⁻², 26.58 W·m⁻² and 0.82, respectively. Therefore, it is a good choice if sufficient data to calibrate the bulk formulae are lacking.

3.2. Effects of Climate Type

Bulk formulae mainly depend on the screen-level air temperature, relative humidity and cloud properties (e.g., cloud fraction) [15,18,26]. The primary criteria for climatic type classification are temperature and precipitation. As a result, we divided the sites as nine groups according to the Koppen climate classification system, including Af (tropical rainforest climate, 3 sites), Am (tropical monsoon climate, 1 site), Aw (tropical wet and dry or savanna climate, 2 sites), BS (semiarid, 5 sites), BW (desert climate, 3 sites), Cf (temperate or subtropical hot summer climates, 9 sites), Cs (Mediterranean climates, 2 sites), Df (warm summer continental climates, 13 sites), and ET (tundra climate, 5 sites). The performance of the cloudy-sky bulk formulae and BMA method were investigated considering different climate types.

As shown in Figure 3, there are large fluctuations in the BIAS and RMSE plots. At first glance, we cannot determine which bulk scheme is the best.

Table 5. Statistical results of seven cloudy-sky bulk formulae and the BMA method over different climate types. Af: tropical rainforest climate; Am: tropical monsoon climate; Aw: tropical wet and dry or savanna climate; BS: semiarid; BW: desert climate; Cf: temperate or subtropical hot summer climates; Cs: Mediterranean climates; Df: warm summer continental climates; and ET tundra climate.

Climate Type	BIAS (W/m2)
	Jacobs	MC	Lhomme	Konzelmann	Crawford	Carmona1	Carmona2	BMA
Af	10.87	16.35	16.29	7.11	2.59	1.00	−1.55	4.71
Am	5.15	1.97	1.83	−4.42	−8.55	−8.75	−10.02	−3.40
Aw	9.71	15.82	15.75	9.09	5.70	0.40	−3.16	5.76
BS	−5.61	−2.05	−2.06	6.16	14.08	13.88	5.97	3.51
BW	−7.11	−1.72	−1.73	6.79	12.69	6.59	−0.52	1.05
Cf	5.28	5.81	6.21	5.40	11.46	14.65	7.56	7.57
Cs	−16.36	−11.90	−11.88	−10.17	−7.48	−4.45	−8.18	−9.57
Df	−10.89	−7.27	−7.27	−5.37	−1.63	4.54	0.26	−3.54
ET	−27.66	−21.53	−21.59	−20.65	−21.37	−6.02	−6.52	−13.49
Climate Type	RMSE (W/m2)
	Jacobs	MC	Lhomme	Konzelmann	Crawford	Carmona1	Carmona2	BMA
Af	19.07	19.84	19.68	13.29	12.79	13.02	10.69	12.23
Am	21.03	16.26	16.18	16.00	20.26	24.59	22.07	17.48
Aw	22.84	23.39	23.28	18.28	18.42	18.78	15.22	17.16
BS	27.05	27.45	27.43	27.97	32.95	31.32	25.76	25.74
BW	24.94	24.56	24.62	24.96	30.20	29.75	24.12	23.15
Cf	34.79	30.31	30.89	29.29	37.72	37.68	28.91	30.96
Cs	24.77	21.05	21.09	19.10	20.26	19.33	18.04	18.52
Df	26.54	24.39	24.47	23.63	26.63	24.13	20.01	22.24
ET	31.79	25.84	25.89	25.10	27.49	15.84	15.40	19.56
Climate Type	R2
	Jacobs	MC	Lhomme	Konzelmann	Crawford	Carmona1	Carmona2	BMA
Af	0.19	0.16	0.17	0.15	0.18	0.19	0.18	0.20
Am	0.52	0.74	0.75	0.81	0.62	0.39	0.62	0.70
Aw	0.34	0.32	0.32	0.31	0.31	0.32	0.33	0.35
BS	0.76	0.73	0.73	0.73	0.72	0.71	0.74	0.75
BW	0.74	0.72	0.72	0.73	0.70	0.64	0.71	0.74
Cf	0.66	0.73	0.72	0.71	0.58	0.57	0.70	0.69
Cs	0.79	0.80	0.80	0.81	0.78	0.73	0.79	0.81
Df	0.78	0.79	0.79	0.77	0.71	0.73	0.79	0.78
ET	0.73	0.75	0.75	0.74	0.67	0.73	0.75	0.75

shows that all methods overestimate SDLR when the climate type is Cf and underestimate SDLR for the Cs and ET climate types. That bulk formulae appear to overestimate SDLR under hot and warm climates and underestimate SDLR when the atmosphere is dry and cold. The absolute value of the BIAS of Carmona1 is the lowest when the climate type is Af, Aw, Cs, and ET; the RMSE of Carmona2 is the lowest except when the climate type is Am, BS and BW. BMA achieves a balanced BIAS and RMSE between the best and worst model for each climate type and has a good performance in the R². The BIAS, RMSE and R² of Carmona1 range from −8.85 to 14.65 W·m⁻², 13.02 to 37.68 W·m⁻², and 0.19 to 0.73, respectively. The BIAS, RMSE, and R² of Carmona2 range from −10.02 to 7.56 W·m⁻², 10.69 to 28.91 W·m⁻², and 0.18 to 0.79, respectively. The BIAS, RMSE, and R² of BMA range from −13.49 to 7.57 W·m⁻², 12.23 to 30.96 W·m⁻², and 0.20 to 0.81, respectively. Based on the BIAS values in Table 5, we can identify the best bulk formula for each climatic type. Carmona1 performs the best over Af, Aw, Cs, and ET, Carmona2 performs better over BW and DF, Lhomme performs better over Am, MC performs better over BS, and Jacobs performs well over Cf. The performance of Carmona1 and Carmona2 are generally relatively better than the other methods. Due to the scarcity of the ground measured SDLR, the number of sites for each climate type are different. We did not find a clear relationship between the number of sites and the performance of bulk formulae over each climate type. We will explore this relationship once we accumulate abundant data.

3.3. Effects of Land Cover Type

The land cover type is another factor that regulates Earth-atmosphere interactions. We divided the sites into five types, i.e., desert (3 sites), bare land (4 sites), cropland (4 sites), grass (12 sites), and forest (14 sites), to investigate the accuracy of the bulk formula and the BMA method. The evaluation results are shown in Figure 4. Similar to Figure 3, there are larger spread in the BIAS and RMSE. Considering the BIAS and RMSE of each bulk formulae and the BMA over all land cover types, it is difficult to conclude which one performs the best or the worst as a whole. The statistical results in

Table 6. Statistical results of seven parameterization schemes and the BMA method over different land cover types.

Land Cover Type	BIAS (W/m2)
	Jacobs	MC	Lhomme	Konzelmann	Crawford	Carmona1	Carmona2	BMA
Desert	−7.11	−1.72	−1.73	6.79	12.69	6.59	−0.52	1.05
Bare land	−27.66	−21.53	−21.59	−20.65	−21.37	−6.02	−6.52	−13.49
Cropland	−11.46	−7.74	−7.73	−5.87	−2.21	3.46	−0.85	−4.25
Grassland	2.89	6.72	6.74	7.53	10.63	9.79	3.87	6.62
Forest	−14.16	−9.59	−9.58	−9.28	−7.83	−2.65	−4.71	−7.86
Land Cover Type	RMSE (W/m2)
	Jacobs	MC	Lhomme	Konzelmann	Crawford	Carmona1	Carmona2	BMA
Desert	24.94	24.56	24.62	24.96	30.20	29.75	24.12	23.15
Bare land	31.79	25.84	25.89	25.10	27.49	15.84	15.40	19.56
Cropland	26.10	23.74	23.81	22.88	25.77	23.49	19.60	21.63
Grassland	25.72	25.66	25.69	23.89	27.84	26.88	21.68	22.82
Forest	28.66	25.99	26.06	24.70	26.20	24.50	22.38	23.80
Land Cover Type	R2
	Jacobs	MC	Lhomme	Konzelmann	Crawford	Carmona1	Carmona2	BMA
Desert	0.74	0.72	0.72	0.73	0.70	0.64	0.71	0.74
Bare land	0.73	0.75	0.75	0.74	0.67	0.73	0.75	0.75
Cropland	0.78	0.79	0.79	0.78	0.72	0.73	0.79	0.79
Grassland	0.90	0.90	0.90	0.90	0.86	0.86	0.90	0.90
Forest	0.85	0.86	0.86	0.85	0.81	0.82	0.85	0.85

show that all methods underestimate the SDLR over bare land and forest, with BIAS ranging from −27.66 to −2.65 W·m⁻², and overestimate the SDLR over Grassland, with BIAS ranging from 2.89 to 10.63 W·m⁻². The R² of all methods are larger than 0.64. The RMSE of Carmona2 is better than that of the other methods under most conditions. According to the BIAS values, Jacobs performs the best over grassland, Carmona1 performs well over bare land and forest, and Carmona2 performs well over desert and cropland. BMA achieves a balanced BIAS and RMSE between the best and worst model for each land cover type.

3.4. Effects of Surface Elevation

The vertical zonality is an important component of the rule of territorial differentiation. The driving factors behind the vertical zonality are water vapor and temperature, and elevation is a direct quantitative index of vertical zonality. We divided the sites into four categories according to the elevation range (≤500 m (23 sites), 500–1000 m (8 sites), 1000–3000 m (7 sites), and ≥3000 m (6 sites)) and assessed the cloudy-sky bulk formulae and BMA method. Figure 5 shows a trend that the absolute values of the BIAS and the RMSE increase with surface elevation.

Table 7. Statistical results of the seven bulk formulae and the BMA method over different surface elevations.

Elevation	BIAS (W/m2)
	Jacobs	MC	Lhomme	Konzelmann	Crawford	Carmona1	Carmona2	BMA
H < 500	−7.24	−2.66	−2.66	−4.45	−4.07	0.56	−2.31	−2.92
500 < H < 1000	−5.38	−2.04	−1.93	1.66	7.42	11.97	5.36	2.04
1000 < H < 3000	−3.57	−0.05	−0.08	8.26	16.60	15.07	6.34	4.36
H > 3000	−45.72	−41.96	−41.90	−24.92	−15.01	−14.88	−20.03	−25.27
Elevation	RMSE (W/m2)
	Jacobs	MC	Lhomme	Konzelmann	Crawford	Carmona1	Carmona2	BMA
H < 500	25.21	22.89	22.92	20.52	21.93	19.01	16.91	19.05
500 < H < 1000	29.99	27.93	28.06	28.21	35.14	32.96	24.75	27.26
1000 < H < 3000	23.34	24.16	24.13	26.82	33.15	31.24	24.38	23.50
H > 3000	59.41	56.67	56.71	42.33	38.85	41.08	42.11	43.09
Elevation	R2
	Jacobs	MC	Lhomme	Konzelmann	Crawford	Carmona1	Carmona2	BMA
H < 500	0.91	0.92	0.92	0.91	0.89	0.89	0.92	0.91
500 < H < 1000	0.70	0.72	0.72	0.69	0.60	0.61	0.72	0.70
1000 < H < 3000	0.80	0.77	0.78	0.75	0.73	0.72	0.79	0.79
H > 3000	0.60	0.59	0.59	0.65	0.65	0.59	0.59	0.64

shows that the cloudy-sky bulk formulae perform best when the surface elevation is less than 500 m. The BIAS ranges from −7.25 to 0.56 W·m⁻², the RMSE ranges from 16.91 to 25.21 W·m⁻², and R² is approximately 0.9. The bulk formulae perform the worst when the surface elevation is higher than 3000 m. The bulk formulae underestimate the SDLR when the elevation is higher than 3000 m. The BIAS ranges from −45.72 to −14.88 W·m⁻², the RMSE ranges from 38.85 to 59.41 W·m⁻², and R² is approximately 0.6. The BMA achieves a balanced BIAS and RMSE between the best and worst model for each climate type. This is consistent with the climate and surface type evaluation results.

The air temperature, relative humidity and cloud fraction are used to predict cloudy-sky SDLR in the selected bulk formulae. The uncertainties of the input variables propagate into the predicted SDLR and cause unpredictability [50]. Even most of the clear-sky SDLR is emitted within the lowest layer of the atmosphere, air temperature and relative humidity at screen level cannot fully reflect the variability of the upper level temperature and water content. The accuracy of predicted SDLR is improved by incorporating the total precipitable water (TPW) [50,51]. Other factors, such as uncertainties in the cloud fraction, aerosol loads, heatwaves in hot summers, and temperature inversions in the lower atmospheric levels during cold winter night, also affect the predictability of the bulk formulae [27,34,50,52]. The positive BIAS over climate types Af and Cf might result from strong unstable air temperatures with high humidifies, and negative BIAS over climate type ET might relate to dry atmospheric conditions and strong inversions; In addition, the negative BIAS over land cover type bare land may be incurred by strong atmospheric inversions. Furthermore, the method of estimating cloud fraction with Equation (2) cannot account perfectly for cloudiness. For example, thin cloud types like cirrus have higher solar transmissivity, and resulting in an underestimating of cloud cover as well as SDLR. This partly explains the negative BIAS in SDLR when surface elevation is higher than 3000 m. Another possible explanation may be that other atmospheric constituents (e.g., ozone and CO₂) have relatively large contributions to the SDLR, but this contribution is not considered by the bulk formulae [4,15].

According to the investigation results on the impacts of climate type, land cover type and surface elevation, it did not provide a clear guidance on how to identify an optimal bulk formula under certain circumstances (climate type, land cover type, and surface elevation). For example, we may have three different bulk formulae to calculate SDLR for a specific site because this site belongs to certain climate type, land cover type and surface elevation at the same time. We can select the one with the best performance. Meanwhile, it is also advisable to integrate the bulk formulae via BMA, whose performance has been verified in Section 3. The BMA method conducts a weighted average of the individual bulk formula. The weight of each bulk formula is derived from the calibration results, and can reflect the predictive performance of each bulk formula. Therefore, the performance of the BMA is close to bulk formula that performs best, and better than the arithmetic average of those for seven selected bulk formulae.

4. Conclusions

SDLR is one of the four components required to calculate the Earth’s surface radiation budget. Bulk formulae are widely used to estimate all-sky SDLR due to the simple, readily available inputs and acceptable accuracy from local to regional scales. The accuracy and applicability of seven widely used bulk formulae were investigated using ground measurements collected from 44 globally distributed flux measurement sites in five networks. The BMA method was introduced to integrate multiple bulk formulae to obtain an estimate of cloudy-sky SDLR for the first time. The accuracy and applicability of the BMA method were also investigated using the same data. The following conclusions can be drawn from this study:

It is vital to calibrate cloudy-sky bulk formulae before use. If representative data to calibrate the bulk formulae are lacking, the bulk formula of Crawford and Duchon [26] is a good choice because it can achieve acceptable accuracy as a whole without local calibration.

Overall, the bulk formula of Carmona2 (i.e., the second multiple linear regression model of Carmona et al. [25]) performs the best, with a BIAS of zero, an RMSE of 20.13 W·m⁻², and an R² of 0.87.

The BMA method conducts a weighted average of the individual bulk formula. The weight of each bulk formula is derived from the calibration results, and can reflect the predictive performance of each bulk formula. Therefore, the BMA method can achieve balanced results that are close to those of Carmona2 and better than the arithmetic average of those for seven selected bulk formulae, with a BIAS, RMSE and R2 of −1.08 W·m⁻², 21.99 W·m⁻² and 0.87, respectively.

The investigation results on the impacts of climate type, land cover type and surface elevation did not provide a clear guidance on how to identify the bulk formula with an optimal performance at certain circumstances (climate type, land cover type and surface elevation). For example, we may have three different bulk formulae to calculate SDLR for a specific site because this site belongs to certain climate type, land cover type and surface elevation at the same time. We can select the one with the best performance or integrate the bulk formulae via BMA.

To summary, a bulk formula that can be successfully applied everywhere does not exist. The best and worst methods may differ under different climate types, land cover types and surface elevations. Thus, the best bulk formula should be site-specific.