Cloud Area Distributions of Shallow Cumuli: A New Method for Ground-Based Images

Abstract

We develop a new approach that resolves cloud area distributions of single-layer shallow cumuli from ground-based observations. Our simple and computationally inexpensive approach uses images obtained from a Total Sky Imager (TSI) and complementary information on cloud base height provided by lidar measurements to estimate cloud equivalent diameter (CED) over a wide range of cloud sizes (about 0.01–3.5 km) with high temporal resolution (30 s). We illustrate the feasibility of our approach by comparing the estimated CEDs with those derived from collocated and coincident high-resolution (0.03 km) Landsat cloud masks with different spatial and temporal patterns of cloud cover collected over the Atmospheric Radiation Measurement (ARM) Southern Great Plains (SGP) site. We demonstrate that (1) good (~7%) agreement between TSI and Landsat characteristic cloud size can be obtained for clouds that fall within the region of the sky observable by the TSI and (2) large clouds that extend beyond this region are responsible for noticeable (~16%) underestimation of the TSI characteristic cloud size. Our approach provides a previously unavailable dataset for process studies in the convective boundary layer and evaluation of shallow cumuli in cloud-resolving models.

1. Introduction

Shallow cumuli are observed frequently over many continental and marine regions of the world. They consist mainly of small (<1 km) clouds [1,2] and thus represent a great challenge for both observational and model studies. Cloud horizontal size and its diurnal changes substantially influence complex surface-atmosphere interactions, including energy fluxes at the land-atmosphere boundary and the Earth’s radiation budget [3,4,5]. The cloud size distribution is used to calculate mass flux and energy transport in cloud models and also for model evaluation against observations.

Although the term “cloud horizontal size” is commonly used to refer to cloud chord length (CCL) derived from ground-based radar-lidar [6] or aircraft [7] data, model studies typically restrict the word “size” to describe cloud equivalent diameter (CED) of a given cloud area [8]. As a consequence, the data-derived CCL and model-predicted CED often cannot be compared directly especially at short temporal and spatial scales as recently shown by Romps and Vogelmann [9].

Satellite observations can be used to measure cloud area [10,11], though the accuracy of the cloud area measurements depends strongly on the spatial resolution of the satellite sensor and the most frequent cloud size [12]. In particular, the spatial resolution must be high enough to resolve the majority of clouds within a given scene. Since shallow cumuli are typically small (<0.4 km) [6,11], only high-resolution satellite data are relevant for such demanding characterization. For example, Landsat provides the required data at 0.03 km horizontal resolution. However, Landsat data is sampled at only once every 16 days and this coarse temporal resolution is a key obstacle to capture the strong diurnal changes of shallow cumuli from space.

In contrast with satellite observations, ground-based sky cameras offer images with high temporal (<1 min) resolution [13,14] and these images could address the shortcomings of episodic satellite images highlighted above. For example, images of cloudy sky from ground-based Total Sky Imager (TSI) have been used operationally to obtain fractional sky cover within a 160° field-of-view (FOV) with high (30 s) temporal resolution [14]. Thus, the following question arises: Is it possible to estimate the cloud area (or CED) from TSI images? Here we answer this important question by introducing a new ground-based approach that leverages passive and active remote sensing and then demonstrate its initial performance for 5 days with continental single-layer shallow cumuli.

2. Approach

Our approach follows the same three major tasks used in analysis of high-resolution satellite images (e.g., [10,11]): (1) “cloud masking” task, which separates clear-sky and cloudy pixels, (2) “cloud labeling” task, which groups the selected cloudy pixels into individual clouds and (3) “quantitative analysis” task, which provides statistics of influential macrophysical parameters of clouds. Our approach uses the operational processing of the TSI images (Figure 1a–c) for the “cloud masking” task. It should be emphasized that cloud masking errors can propagate to cloud statistics. Two new tasks required prior to the “cloud labeling” task are outlined below: interpolation over image obstructions and projection to planar coordinates.

2.1. Interpolation over Image Obstructions

The TSI cloud mask images are obstructed by the shadowband, camera arm, camera housing and green lines (Figure 1c). These obstructions may divide a large cloud into two or more apparent smaller clouds. To address issues associated with these obstructions, we first expand the existing cloud mask to screen incorrectly classified pixels [15], then use 2D natural interpolation [16] to estimate the pixel mask within the obstructed regions (Figure 1d). The details and evaluation of this processes are described in full in Appendix A.

2.2. Projection of Cloud Mask from Hemispherical to Planar Coordinate System

Each image pixel of a hemispherical TSI image (Figure 1d) represents an approximately equal solid angle of the sky. The image resolution of the TSI was increased on 15 August 2011 from 352 × 288 pixels (original resolution) to 640 × 480 pixels (improved resolution). The area of the sky subtended by a pixel depends on cloud base height (CBH) and pixel zenith angle. We correct for a slight distortion of the TSI mirror [14] and apply the well-known spherical to planar coordinate transformation [17,18].

$$r=\mathrm{CBH}\tan \left({\theta }_p\right),$$

$$x_p=r\cos \left({\varphi }_p\right),$$

(1)

$$y_p=r\sin \left({\varphi }_p\right),$$

where CBH is derived from lidar measurements (Section 3), θ_p and φ_p are the zenith and azimuth angles for each pixel, respectively and x_p and y_p are resulting pixel coordinates. A single CBH is used for each image transformation, representing all clouds in the scene. This approximation is reasonable given the steady lidar observations of CBH. The transformed pixel coordinates (x_p, y_p) have irregular spacing. We use triangulation-based linear interpolation to sample the transformed pixel coordinates on a regular grid (Figure 1e). To verify the accuracy of the applied planar coordinate projection, we place a TSI instrument beneath a rectangular platform with known dimension and height (Figure 2) and retrieve platform dimensions using the collected TSI images. We obtain good agreement (~3%) between the known and retrieved dimensions under the control setting. Additional qualitative verification is obtained by overlaying the resulting cloud boundaries obtained from the post-processed TSI cloud mask with the coincident Landsat observation for the five selected dates (Figure 1f and Figure 3).

2.3. Computation of Cloud Areas and the Corresponding CED

Each matrix element in the planar projected cloud mask (Figure 1e) has the same area, allowing for simple calculation of cloud area. Cloudy elements are grouped with an 8-neighbor connectivity criterion, completing the “cloud labeling” task in satellite-based image analysis. The cloud area of the connected region is simply the sum of the element areas [19].

A sequence of TSI images is analyzed to produce statistics on CED, defined as 2√(A/π), similar to the satellite-based “quantitative analysis” task. In our analysis, all cloud areas are retained, including those that are truncated by the 130° FOV circle (Figure 1e). Results of our sensitivity study suggest that 130° FOV is a reasonable trade-off between the anticipated largest sample area (or largest CED) and unwanted influence of cloud side effects (see Appendix B). The cloud truncation at the perimeter limits the observation of large clouds and incorrectly identifies them as smaller clouds. To reduce the influence of this error, we require that clouds truncated at the 130° FOV extend to within a 122° FOV inner region, as determined by a sensitivity study to obtain the best agreement between TSI- and Landsat-derived CED (see Appendix C). This analysis is valid for images with low-to-moderate (<0.4) cloud fraction. The following section highlights the Landsat images and ground-based data required for evaluation of our approach.

3. Data

The integrated dataset includes the ground-based TSI images [20] the best estimate CBH from the ARM Active Remotely Sensed Clouds Locations (ARSCL) cloud product [21], wind speed and direction from the 915-MHz Radar Wind Profiler (RWP) measurements [22] and also high-resolution Landsat images. Hemispheric sky images and corresponding cloud masks are obtained from the TSI Model 880 [23] with enhanced control board and camera installed in 2011. All ground-based data are obtained from the Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) Southern Great Plains (SGP) site, located at 36°36′18″ N, −97°29′6″ E. We use the outlined ground-based and satellite data for selected 5 days with single-layer shallow cumuli to demonstrate the feasibility of our approach (Section 4). What follows is a short description of the collected data and the corresponding instruments.

We select desirable days with single-layer shallow cumuli using multi-year (2004–2017) datasets obtained from the coincident and collocated ground-based and satellite observations. A total of 471 potential days with single-layer shallow cumuli are identified from the Shallow Cumulus Value Added Product [24] using ground-based active and passive remote sensing. The Landsat satellite passes the SGP site infrequently: only once every 16 days at roughly 17:00 UTC (11 a.m. local standard time). We found only 5 days with Landsat images of single-layer shallow cumuli over the SGP site (Figure 4) within the 14-year period (2004–2017) considered here: 2006/05/15 (16:59 UTC), 2007/07/21 (17:01 UTC), 2007/09/23 (17:00 UTC), 2009/05/23 (16:55 UTC) and 2017/06/14 (17:07 UTC). All data described below represent the selected 5 days.A representative CBH and wind vector is determined for each TSI sky image, with 30-s temporal resolution. The ARSCL Value Added Product (VAP) resolves the cloud boundaries with 0.045-km vertical and 10-s temporal resolution [25]. We use a 30-min average of the ARSCL-derived CBH centered at the TSI-capture time as an estimated CBH for the imaged cloud field. The RWP reports the wind vectors with 60-m vertical and 1-h temporal resolution. We use the 1-h average RWP-derived wind vectors closest in time to each TSI image capture time. To reduce noise, we vector average the RWP winds from the three height range gates below, including and above the estimated CBH, ignoring any missing entries, to obtain 1-h averages of wind speed and direction at CBH. It should be mentioned that the vertical span of 180-m is typically less than the standard deviation of the estimated CBH.

The Landsat cloud masks with 0.03-km horizontal resolution identify the majority of shallow cumuli within a 50-km region surrounding the SGP site (Figure 4, top panel). Landsat capture times are within 5 min of 17:00 UTC at SGP and have root-mean-square geodetic accuracy of better than 50 m [26]. We retrieve cloud mask data [27] from the Landsat 5 (L5) and Landsat 8 (L8) missions (datasets courtesy of the US Geological Survey) using the Google Earth Engine online tool (https://code.earthengine.google.com) [28]. Then we combine the cloud masks retrieved for areas located within the 50-km region into a mosaic. The same approach is used for color images. The selected 5 days define different observational conditions with distinct spatial distribution of clouds and cloud fraction ranging by almost an order of magnitude from 0.04 (2007/09/23) to 0.33 (2017/06/14) (see

Table 1. Landsat cloud fraction (CF), cloud base height (CBH), wind speed and direction (dir.) at CBH, the estimated width and length of subareas (Figure 4b) for 130° FOV and 1-h averaging period. Numbers of clouds observed by TSI (N TSI) and Landsat (N Lsat).

Landsat Scene	CF (50-km)	CF Subarea	CBH (km)	Wind Speed (m/s)	Wind dir. (deg)	Width (km)	Length (km)	N TSI	N Lsat
2006/05/15	0.27	0.26	1.52	13.4	349	6.5	48.2	2445	297
2007/07/21	0.1	0.1	1.25	6.5	159	5.4	23.4	1483	87
2007/09/23	0.04	0.15 (1)	1.25	7.2	168	5.4	25.9	1700	60
2009/05/23	0.17	0.16	1.24	5	85	5.3	18	2017	80
2017/06/14	0.33	0.3	1.3	7.8	223	5.6	28.1	4093	85

⁽¹⁾ Note that values of CF calculated for the 50-km (second column) and smaller subarea (third column) are comparable for all days considered here except 23 September 2007 where the subarea CF exceeds the 50-km CF more than three times likely due to the strong diurnal variability of CF before noon.

). Also, we use the RWP-based averages of wind speed and direction at CBH together with the frozen turbulence assumption to estimate size and orientation of the subarea of each Landsat image surrounding the SGP site observed by the TSI (Figure 4, bottom panel). Their widths represent 130° FOV equivalent areas for a given CBH, while their lengths define 1-h equivalent distances for a given wind speed at CBH (Table 1). The main goal of the width and length estimation is to match “case study” areas obtained from Landsat (instantaneous snapshot) and TSI (1-h averaging) observations. Such area matching is required for comparing the basis statistics of the TSI- and Landsat-derived CEDs (Section 4). We determine the Landsat-derived CEDs similarly to those from TSI data (Section 2), except that Landsat-derived cloud area is never truncated due to FOV limitations (we censor one erroneous cloud on 14 June 2017, Figure 4b). Landsat cloudy pixels located within and outside of the red rectangle (Figure 4, bottom panel) are considered part of a given cloud (navy shaded clouds in Figure 4).

4. Results

Prior to comparing CED statistics obtained from the Landsat and TSI observations, we outline challenges and solutions to a balanced comparison. First, the satellite and ground-based observations likely have different sensitivities to cloud optical and geometrical properties, resulting in different cloud areas. Second, while an apples-to-apples comparison on a single coincident capture (see Figure 3) can be made, a comparison of cloud field statistics from the TSI and Landsat image requires employing the frozen turbulence assumption. In other words, we assume that the cloud field is evolving slowly enough that the single instantaneous Landsat capture is comparable to the TSI-observations acquired over 1-h period. This assumption may break down when the cloud field is non-stationary over the 1-h averaging period. Thus, the Landsat and TSI cloud masks may represent different cloud populations. Third, the TSI CEDs represent both truncated and non-truncated clouds, while the Landsat CEDs define non-truncated clouds only. As a result, TSI CEDs likely underestimate Landsat CEDs and this underestimation can be size-dependent. Fourth, the TSI can view cloud sides at large zenith angles, sometimes resulting in merged clouds. Finally, the TSI image interpolation over the shadowband region may introduce additional errors.

To quantify the combined impact of the sampling strategy (second challenge) and cloud truncation (third challenge) on the level of agreement between the TSI and Landsat CEDs, we introduce simulated TSI-like observations of the Landsat image. The TSI-like observations consist of 120 circular instances of the Landsat image, captured with TSI-equivalent 130° FOV for a 1-h averaging period along the wind direction, simulating a 30-s temporal resolution within the Landsat subarea (See Figure A3). The TSI-like observations observe each cloud multiple times, truncate clouds at the 130° FOV and retain only truncated clouds that extend to less than 122° FOV, as does the TSI.

The cloud fraction (CF) obtained from the TSI and TSI-like simulator reveal differences in spatial and temporal sampling (Figure 5a). The 15-min TSI average (red bars) indicates temporal variability over the 1-h observation period, while the TSI-like 15-min average indicates spatial variability within the Landsat subarea (green bars). The Landsat subarea CF (blue circles) and TSI-like 1-h average (green diamonds) show nearly perfect agreement, as expected. The TSI CF indicates non-stationary cloud fields on 2006/05/15, 2007/09/23 and 2009/05/23. The TSI 1-h and Landsat subarea averages (symbols) agree to within 10 percentage points on the two stationary days and 2006/05/15 due to a steady increase in CF over the 1-h averaging period. On the two remaining days: (1) 2007/09/23 contains an inaccurate TSI cloud mask caused by sun glare (see Figure 5b) as well as an inhomogeneous and non-stationary cloud field and (2) 2009/05/23 has a non-stationary cloud field.

Statistical comparisons of the cloud area distributions measured by the TSI and Landsat are made following the approaches introduced previously by others. In particular, the normalized cloud size distribution n(λ) is defined as

$$N=\frac{1}{S}{{\displaystyle \int }}_0^{\infty }n\left(\mathsf{\lambda }\right)\mathrm{d}\mathsf{\lambda },$$

(2)

where λ is the CED, S is the total area of sky observed and N is the number of clouds observed per unit area. The normalized cloud size distribution (Figure 6a) shows that the smallest clouds are most numerous, consistent with many other studies for example, [29]. The TSI resolves a wide range of cloud sizes from 0.01 km to about 3.5 km (see Appendix D for lower limit and Appendix C for upper limit), while Landsat has no upper bound on cloud size. The cloud size distribution n(λ) commonly extends over multiple decades in scale (0.01–10 km for shallow cumulus) [1], which does not easily suggest a central or mean cloud size.

On the other hand, the cloud cover density is defined as [7]

$$\mathsf{\sigma }\left(\mathsf{\lambda }\right)=\frac{1}{\mathrm{S}}a\left(\mathsf{\lambda }\right)n\left(\mathsf{\lambda }\right),$$

(3)

where a(λ) is the cloud area π(λ/2)². Note that σ(λ) commonly has peaked distributional shape that can be described using a summary statistic of cloud size. The cloud cover density represents the contribution of each CED to cloud cover, such that the integral of σ(λ) over λ results in the cloud fraction. We draw two cloud size statistics from the cloud cover density: the characteristic size λ_c is the expected value of the cloud cover density [7]

$${\mathsf{\lambda }}_{\mathrm{c}}={{\displaystyle \int }}_0^{\infty }\mathsf{\lambda }\mathsf{\sigma }\left(\mathsf{\lambda }\right)\mathrm{d}\mathsf{\lambda },$$

(4)

and the median of the cloud cover density, L₅₀, describes the cloud size for which half of the cloud fraction is derived from clouds smaller than L₅₀ [29].

The median L₅₀ and other percentiles of the cloud cover density σ(λ) (Equation (3)) illustrate differences in CED obtained from these three approaches (Figure 5b). The three approaches TSI, TSI-like and Landsat show mutual agreement when there is a high-quality TSI mask and Landsat L₅₀ is less than 2 km (2007/07/21 and 2009/05/23). The TSI and TSI-like interquartile ranges show excellent agreement on all days with correct TSI mask (Figure 5b, red and green bars). The percent difference of TSI compared to TSI-like L₅₀ ranges from −2% (2009/05/23) to +32% (2017/06/14), with a mean absolute percent difference of 9%. Similarly, the percent difference of TSI to TSI-like λ_c ranges from −2% to 8% with a mean absolute difference of 3%. This error is due to sensor differences, inclusion of cloud sides and interpolation and is considered secondary to the effect of cloud truncation and spatio-temporal differences (Appendix B discusses the sensitivity of CED on FOV). The cloud cover densities (Figure 6b,c, red and green bars) further demonstrate this close agreement. More generally, comparison of Landsat λ_c from clouds that appear within the TSI field of view (but are not sampled as in the TSI-like) with TSI returns a mean absolute percent difference of 7%. All central cloud sizes are presented in

Table 2. Comparison of median of the cloud cover density L₅₀ and characteristic size λ_c for the cloud cover density, σ(λ), in Figure 6b,c.

Date	Statistic	Lsat (whole, km) (1)	Lsat (cut-off, km) (2)	TSI-Like (km)	TSI (km)
2006/05/15	L50	2.03	1.64	1.60	1.63
	λc	2.48	1.75	1.73	1.69
2007/07/21	L50	1.11	0.79	0.93	0.93
	λc	1.04	0.92	0.92	0.93
2007/09/23	L50	1.09	0.96	1.13	1.78
	λc	1.35	1.32	1.27	1.79
2009/05/23	L50	1.39	0.96	1.17	1.15
	λc	1.33	1.16	1.20	1.30
2017/06/14	L50	2.60	1.77	1.74	2.30
	λc	2.79	2.30	2.05	2.06

⁽¹⁾ Whole refers to the primary analysis presented, where the clouds extending beyond the 130° FOV in the Landsat sub-area are kept whole (shaded clouds in Figure 4b). ⁽²⁾ Cut-off refers to the same clouds as in the primary analysis, except clouds extending beyond the 130° FOV boundary are truncated (red line in Figure 4b), the distribution of the cut-off clouds is not shown in Figure 6.

.

The full Landsat cloud sizes, which are never truncated, extend beyond the 130° FOV (blue clouds in Figure 4). Consequently, both TSI and TSI-like cloud cover densities fail to resolve clouds larger than about 3.5 km (Figure 6b,c; see 2006/05/15 and 2017/06/14). Not only are these clouds outside of the TSI field of view but the sinewy connections in the Landsat image suggest that closely-spaced clouds may be falsely merged by Landsat. One such example in the northeast region of Figure 4a, 2017/06/14, was omitted from the analysis for this reason. The radius of the circular sky observed in a single TSI image (Table 1) is comparable to 3.5 km, making this a sensible upper bound. We generalize this result to expect an upper bound of resolvable cloud size at the radius of the projected image, which is roughly two times the CBH for a 130° FOV. The overall percent error in L₅₀, taken as the difference over the Landsat value, is (19%, 16%, 17% and 12%) for the four days with high-quality TSI mask, in chronological order, for a mean error of 16% (Table 2). This error is biased low due primarily to the truncation of clouds at the 130° FOV and is evaluated only from cloud fields with small-to-moderate (<0.4) cloud fraction.

5. Conclusions

We develop a novel approach that resolves cloud area distributions of single-layer shallow cumuli from ground-based observations and estimates cloud equivalent diameter (CED) over a wide range of cloud sizes (about 0.01–3.5 km) with high temporal resolution (30 s). Our approach is computationally inexpensive and relies on sky images obtained from Total Sky Imager (TSI) and complementary information on cloud base height provided by lidar measurements. We illustrate performance of our approach using coincident 0.03-km Landsat cloud masks obtained for five periods with single-layer shallow cumuli over the Atmospheric Radiation Measurement (ARM) Southern Great Plains (SGP) site. We demonstrate that TSI and Landsat CEDs are in good (~7%) agreement for clouds that fall within the TSI field of view (FOV) limit (~3.5 km) and noticeable (~16%) underestimation of the TSI CEDs is due to large clouds that extend beyond the TSI FOV limit. We demonstrate that a similar sampling strategy of the TSI and Landsat data results in strong (~3%) agreement, indicating that the limited FOV is a primary influence on the comparison, while other factors associated with visibility of cloud sides, 2D interpolation over image obstructions and different sensitivities of the ground-based and satellite sensors to cloud properties are secondary. An extension of this approach could involve tracking clouds as they pass through the TSI FOV, enabling observation of larger clouds along the wind direction. Since many sites world-wide deploy sky imagers, including DOE ARM fixed and mobile facilities, our approach can be applicable to data collected at different locations and times. Our approach provides previously unavailable datasets that can contribute to improved understanding of the diurnal cycle of cumulus convection at various temporal and spatial scales and can offer a unique opportunity to carefully evaluate and improve large-eddy simulation (LES) results of shallow cumuli cloud size and cloud fraction [30,31,32].