The purpose of this effort is to provide current and future users of the PATMOS-x CDR knowledge of characteristics of the cloud fraction CDR behavior. The techniques described above allow a direct comparison of the PATMOS-x results to those from MYD35 and CALIPSO. This section will first show a comparison of PATMOS-x to MYD35 and analyze the regional differences in key quantities. Then, we use CALIPSO results to provide a quantitative analysis of both PATMOS-x and MYD35. The PATMOS-x cloud fraction uncertainty estimates will also be verified. Lastly, an analysis is performed to show how the spectral content of other sensors impacts the PATMOS-x cloud fraction.
3.1. Comparison of MODIS PATMOS-x to NASA MODIS MYD35
As stated above, the NASA AQUA/MODIS sensor has all the channels provided by the NOAA AVHRR. Applying the PATMOS-x cloud detection to MODIS observations allows a direct comparison to the NASA MODIS cloud detection products (MYD35). In this section, we compare the performance by season, averaged over the period 2003–2014. This discussion will point out the relative differences between MYD35 and PATMOS-x, and the following section will provide more quantitative measures of the MYD35 and PATMOS-x compared to CALIPSO/CALIOP. Again, all PATMOS-x results are generated using the corresponding AVHRR channels on MODIS, run through the PATMOS-x algorithm. All data has been mapped to an equal angle grid with a resolution of 2.5°.
Figure 2 shows the comparison of MYD35 and PATMOS-x for winter season, including day and night data. There are six panels in
Figure 2 and they are designed to give a concise summary of the comparison. Panel (a) shows the mean global cloud amount for PATMOS-x. As described in [
3], the PATMOS-x cloud fraction is computed over the 3 × 3 pixel array. Each pixel in the 3 × 3 array is cloudy if the naïve Bayesian cloud probability for that pixel exceeds 0.5; Panel (b) shows the difference (PATMOS-x–MYD35) cloud amount map; Panel (c) shows the mean cloud detection uncertainty as provided by the PATMOS-x naïve Bayesian algorithm; Panel (d) shows the anomaly correlation of the MYD35 and PATMOS-x time-series. Values of 1.0 indicate grid-cells where the year-to-year variations between MYD35 and PATMOS-x are similar. Inspection of the mean differences in Panel (b) and the anomaly correlations in Panel (d) allow one to separate differences that are systematic biases and those that are not; Panel (e) shows the linear trend applied to the PATMOS-x seasonal time-series. It is not corrected from ENSO or other known atmospheric oscillations; Panel (f) shows a scatterplot of the linear trend values from each grid-cell for MYD35 and PATMOS-x. In summary, these panels are meant to provide a convenient and efficient summary of the performance of PATMOS-x relative to MYD35 on a mean, inter-annual, and decadal basis.
As
Figure 2 shows, there is generally more cloud over ocean in the MYD35 than PATMOS-x. Comparison of Panel (b) with (a) shows this occurring in areas of low cloud amount (<0.5). Comparison with Panel (d) shows that these differences occur in regions with high values of anomaly correlations, and this would indicate that these differences are systematic biases (occur each year). There are also significant cloud fraction differences in winter in Siberia and Canada. Unlike the oceanic differences, these differences occur in regions with lower values of anomaly correlation and high values of uncertainty. Therefore, these differences between PATMOS-x and MYD35 point to true differences in the cloud detection performance. For example, MYD35 C6 does appear to have an issue of falsely detecting cloud over snow-covered land during the day [
26], and the comparisons shown here are consistent with that. Also, the differences might be contributed from different snow/ice ancillary maps used by MYD35 and PATMOS-x, but this question lies beyond the scope of this paper. While the likely cause of this issue is known, the analysis against CALIPSO in the next section will also provide insight into differences that are not already diagnosed. Lastly, in the bottom two panels are an analysis of the linear trends constructed for the time-series in each grid-cell. Panel (e) shows the linear trend from PATMOS-x. The linear trend from MYD35 was visually very similar. To make this point, Panel (f) shows a scatterplot of trends from MYD35 and PATMOS-x; Panel (f) shows that trends agree in magnitude and sign for most of the globe. This agreement is significant, since one of the most important uses of the PATMOS-x CDR is for multi-decadal climate analysis. For these studies, the stability of multi-decadal variation (for which the linear trend is a surrogate) is more important than the absolute values of the cloud fraction. As discussed later, PATMOS-x uses a prior cloud probability in its formulation, and this raises concern that the PATMOS-x cloud fraction trends may be influenced by this prior value. MYD35 has no such prior cloud probability constraint. The agreement in the trends between PATMOS-x and MYD35 is a reassuring sign that the PATMOS-x trends are indeed valid, and not overly controlled by the choice of prior cloud probability.
Figure 3 shows the same analysis presented in
Figure 2 applied to summer seasons. Comparisons for spring and fall were also made, but they provided no unique information and are therefore not shown. In summer, the Northern Land Masses are nearly free of snow, which greatly improves the cloud detection skill. This is confirmed by the decreased values of uncertainty in Panel (c) over the Northern Land Masses. The oceanic differences observed in winter remain in the summer. Over land, the cloud fraction differences are the largest over desert regions, and these differences are larger in summer than winter. The anomaly correlation values for the grid-cells with these differences are high, which again points to the differences being systematic and not influencing the year-to-year or decadal variations. Another difference with the winter results is the change in Antarctic results. In summer, the Antarctic region is mainly free of solar illumination, and the skill in cloud detection drops [
27]. For most of Antarctica, PATMOS-x shows much more cloud than MYD35. This region, though, is characterized by high uncertainties, and therefore disagreements are expected. The trends shown in Panels (e) and (f) show the same high level of agreement seen in summer for most regions. While not shown, the grid-cells with trend differences occur in regions with low anomaly correlations and in the predicted high uncertainties.
3.2. Comparison of MODIS PATMOS-x to NASA CALIPSO CALIOP
The CALIPSO/CALIOP sensor in the EOS A-Train provides a direct measure of the presence of cloud and has been used extensively to validate other cloud detection techniques. CALIOP is a lidar and provides nearly direct detection of cloud and other atmospheric scatterers with little dependence on solar illumination or surface characteristics. In this section, the 1 km cloud layer product from CALIOP is used to provide a direct estimate of cloud fraction to better assess the PATMOS-x cloud fraction.
Figure 1 and the associated text describe the physical meanings of these two cloud fractions. While there will certainly be differences due to the spatial scales of the PATMOS-x and CALIPSO cloud fractions, there is no better direct and instantaneous comparison from a space-borne sensor at this time.
It is important to note that both the PATMOS-x and MYD35 results have been tuned to optimize their performance. As described in [
2], the PATMOS-x naïve Bayesian approach was derived from the same type of CALIPSO/CALIOP observations used here but for the years 2007 and 2009. MYD35 has also used CALIPSO/CALIOP co-locations to help augment the manual derivation of cloud detection thresholds. In both of these algorithms, the threshold values (MYD35) or curves (PATMOS-x) are determined over large areas and long time periods. Therefore, we are confident that the results for this one year are representative for other years. As stated earlier, CALIPSO/CALIOP represents the best global validation source and using it as a reference in cloud fraction comparisons is still relevant, even though both PATMOS-x and MYD35 are tuned to CALIPSO/CALIOP.
The results in this section include the values of cloud fraction and the values probability correct (PC). Using the standard cross-comparison matrix shown in
Table 2, PC = (a + d)/(a + b + c + d).
As
Figure 1 shows, a comparison of the PATMOS-x cloud fraction from MYD02SSH and AVHRR/GAC is susceptible to spatial sampling issues. PC is computed by converting the cloud fractions to a binary clear or cloudy mask and comparing the agreement of these values using the formulism in
Table 2. When computing PC, two spatial filters were applied. The first one attempts to remove all spatial sampling differences by excluding points where the PATMOS-x and CALIPSO (P/C) values were not completely clear (cloud fraction = 0%) or completely cloudy (cloud fraction = 100%). These results are labelled as the 0/100 filter in the following tables. This filter demands homogeneity over 10-km for MYD02SSH and roughly 7-km for AVHRR/GAC. The other filter makes no attempt to remove spatial issues. In the second filter, the binary mask is clear if the cloud fraction is less than 50% and cloudy if the binary mask is greater than or equal to 50%. These results are labelled as to 50/50 filter. In terms of data loss, the 50/50 Filter excludes nothing, but the 0/100 Filter excludes roughly a third of the data. We expect truth to lie between the results of these two filters. While the 0/100 results are optimistic, they should allow for a differentiation of the true failings of the spectral cloud tests from the ambiguity caused by spatial sampling differences. When showing MYD35 and CALIPSO (M/C) results, the same two filters were applied. Comparisons of PATMOS-x relative to MYD35 (P/M) are also shown for both filter settings. Because these data are from the same pixels, there are no spatial sampling issues in the P/M results.
The following comparisons were computed for every day of the year 2013. The results are separated by day and night. While the CALIPSO cloud layer algorithm is not dependent on solar illumination, the same cannot be said for the PATMOS-x and MYD35 cloud detection algorithms. Solar illumination is a major factor in the selection of spectral tests in both the MYD35 and PATMOS-x approaches. The CALIOP instrument does have a day/night difference in performance due to the noise from solar contamination, but this is not accounted for here. The results are stratified by surface type, and the seven surface types used in the PATMOS-x naïve Bayesian Training are employed here [
6]. The Antarctic surface type includes Greenland. The distinction between ocean and other water surfaces is taken from the land-sea dataset used in the MODIS C6 [
28] processing. The water surface type includes all inland waters and some coastal waters.
Table 3 presents the results for the daytime analysis of 2013 and shows several characteristics worth noting. First, the P/C and M/C PC values agree within 2% for all surface types and are all above 95% with some values approaching 100% when the 0/100 filter is applied. The A/C and P/C cloud fractions generally agree within 1%. The 50/50 results are uniformly lower than the 0/100 results by as much as 13%. This is expected because the 50/50 results include all of the partly cloudy situations. Again, the P/C and M/C results are very consistent. The A/C values are generally 2% lower than the P/C results, except for the Arctic where the A/C value is 8% lower. The cause of this difference may be due to the inferior radiometric performance of AVHRR in cold regions compared to MODIS or the difference in the angular sampling in the AVHRR and MODIS CALIPSO co-locations. It is problematic that the A/C results beat the P/C results by 3% in the Antarctic. The cloud fractions in
Table 3 are computed using all results without any filter. The PATMOS-x and MYD35 results agree within 3% for most surfaces, with the largest exception being snow-covered land, where the MYD35 cloud fraction is 6% higher. The CALIPSO cloud fractions are generally 5% higher than PATMOS-x or MYD35 with the exception of snow-covered land, where MYD35 exceeds CALIPSO by 2%.
The nighttime 2013 comparisons to CALIPSO are shown in
Table 4. In general, cloud detection at night is more uncertain than during the day, and this is reflected in the PC values in
Table 3 compared to
Table 4. The global PC values dropped by 2% to 6%. The relative agreement between P/C and M/C values remain (with some exceptions) for both spatial filters The A/C shows the same pattern of Arctic degradation and Antarctic improvement relative to P/C, as seen during the day. At night, the snow PC values for A/C are lower than P/C, and this might be due to AVHRR performance at cold temperatures. The M/C values in Antarctic are much higher (7%–10%) compared to P/C. This might be explained by the additional spectral tests in the MYD35 algorithm that are designed for high-latitude nighttime cloud detection. The nighttime cloud fractions show more disagreement for the frozen surfaces than during the day. For example, the PATMOS-x and MYD35 Arctic values are 10% lower than CALIPSO, and MYD35 nighttime Antarctic cloud fraction is 19% lower than CALIPSO and 14% lower than PATMOS-x. PATMOS-x nighttime land fractions appear to be 10% lower than both CALIPSO and MYD35. However, the PC values for P/C are similar to M/C for this surface type. An overriding conclusion from
Table 3 is that the additional spectral information used in MYD35 does not dramatically alter the mean performance relative to the AVHRR spectral information. Undoubtably, the spectral information improves the performance, but this analysis indicates this improvement occurs in a relatively small amount of the data.
The results in
Table 3 and
Table 4 differ from those shown in GEWEX Cloud Climatology Assessment Report. In that report, the global cloud amounts given from CALIPSO was 73% and the PATMOS-x (AVHRR) and MODIS (MYD35) cloud amounts were 68% and 69%, respectively. There are several potential reasons for this. First, the data in GEWEX are not corrected for latitude, and therefore sample the higher latitudes more than lower latitudes due to the sampling characteristics of sub-synchronous satellites flying in the EOS-A-train. Also, the GEWEX CALIPSO data was generated from the 5-km resolution CALIPSO/CALIOP Cloud Layer products. The 5-km product is more sensitive, and would detect more cloud than the 1-km product used here. These numbers are not meant to serve as absolute reference values. They are simply a metric for the comparison of the three data sources in the context of this analysis.
The PATMOS-x to MYD35 (P/M) comparisons are also included in
Table 3 and
Table 4. Since PATMOS-x was generated on MYD02SSH, the comparisons are of exactly the same pixels and no spatial sampling difference exists. For the daytime land surface types, the 50/50 P/M values are 7% higher than the P/C or P/M values. One could imagine that daytime land with surface heating-driven convection would present the most small-scale cloudiness (and associated spatial sampling issues). Small scale cloud is also ubitiquous over the open oceans in both the day and night, and these surfaces also show larger PC values for P/M than for P/C or M/C. These differences could potentially be used to estimate the degradation in the PC values relative to CALIPSO due to spatial sampling differences in the 50/50 PC results in
Table 3 and
Table 4. While this indicates the 50/50 PC values maybe underestimated, this correction was not applied.
3.4. Sensitivity of PATMOS-x Cloud Fraction to Prior Cloud Amount Assumptions
One additional source of uncertainty in the PATMOS-x cloud fraction CDR is the prior cloud probability value in the naïve Bayesian formulation. The prior cloud probability values are simply the assumed mean cloud fraction for each of the surface types. The actual values used are given in [
2] and are close to the mean of the day and night CALIPSO values in
Table 3. In the case of no information, the naïve Bayesian cloud detection will return the prior cloud probability values. In the case where none of the cloud detection tests are definitely clear or cloudy, the prior cloud probability can influence the final posterior cloud probability. In this section, we run the PATMOS-x cloud detection scheme in the AVHRR/3b configuration on MYD02SSH, but we increase the prior cloud probabilities by 10% and these results are labeled PATMOSx_10.
Figure 5 shows the impact on the PATMOS-x cloud fraction by changing the prior cloud probability by 10%. The format and contents are identical to that used in
Figure 2 and
Figure 3, except for Panel (e). Panel (b) shows the difference in cloud fraction due to the 10% change in prior cloud probability. As was the case in
Figure 2 and
Figure 3, cloud amount differences are seen in oceanic regions with low cloud fraction (<0.5). The increase in the prior cloud probability by 10% can cause increases of 10% in the cloud fraction in these regions. Other noticeable changes are the increases in cloud amount in the Polar Regions. These regions have higher uncertainties, and would therefore show more sensitivity to the prior cloud probability assumption. In all, a 10% change in the prior cloud probability increases the global cloud amount by 6%. The high values of the anomaly correction in Panel (d) indicate that the changes in cloud fraction are systematic. The bottom two panels show the impact on the linear trends. Panel (e) shows a difference map and Panel (d) shows a scatterplot. The trend difference shows the largest differences occur in the areas of high uncertainty, which is consistent with the pattern of the cloud fraction differences. There are trend differences with a pattern echoing the trend patterns seen in
Figure 2 and
Figure 3. However, the magnitude of these differences are much less than the absolute values seen in
Figure 2 and
Figure 3.
While this analysis certainly did reveal a non-negligible sensitivity of the PATMOS-x results to the assumed prior cloud probability, the assumed error in the prior probability of 10% is likely too large. If the results of
Table 3 and
Table 4 are used, the difference in the global cloud fraction between MYD35, PATMOS-x, and CALIPSO is less than 6%, with the difference being less for many surface types. Therefore, the expected impact due to realistic errors in the mean prior probabilities is less than shown in
Figure 5. However, the difference in a particular small region’s mean prior cloud probability to that of its surface type may be larger, and the change in the prior cloud probability across surface type boundaries may introduce artifacts in the spatial distribution of cloud fraction. No obvious artifacts are seen in
Figure 5, but this remains an issue to be investigated further. Most importantly, the linear trend shown in Panel (e) shows no significant sensitivity to the prior cloud probability perturbation.
3.5. Sensitivity Based on Spectral Content
As stated above, the PATMOS-x cloud detection algorithm is designed to adjust to the channels available. PATMOS-x can currently process 10 different spectral bands and use them in 16 different tests. The tests involving the AVHRR channels are described in [
6], and the additional tests are described in [
29]. In this section, we again use the MYD02SSH data as a test-bed to explore the impact of these channels on the PATMOS-x cloud detection performance. The DNB refers to the day-night band, which is visible-near infrared nighttime channel on VIIRS. Note that the DNB tests are excluded since MODIS does not have an analogous channel.
Figure 6 shows comparisons of the global cloud fraction for all seasons from 2003 to 2014. The upper panel in
Figure 6 shows the mean PATMOS-x cloud fraction using the MODIS channels in
Table 4. This result is not from MYD35 data but is generated by PATMSO-x using MYD02SSH data. In describing these figures, it is worth noting the global pattern of cloud fraction uncertainty shown in
Figure 2 and
Figure 3. The upper right panel in
Figure 6 shows the difference in MODIS and VIIRS cloud fractions. The significant differences occur in the high latitudes, which coincide with the regions of higher cloud detection uncertainty. VIIRS cloud fractions are higher in these regions. A cause of this behavior is the skill provided by the 6.7 µm water vapor channel in detecting clear polar region [
30]. Without this channel, the very cold surfaces at high latitudes are sometimes classified as cloud. The other notable difference is the Tibetan Plateau and the Andes.
The lower left panel in
Figure 6 shows the difference between the AVHRR in the channel 3b configuration compared to MODIS. The striking feature of this image is the general increase in cloud over the ocean reported by AVHRR/3b relative to MODIS. Comparison to the top left panel shows that this occurs primarily in oceanic regions with cloud fractions <0.5. These regions are also dominated by low cloud. The 6.7 and 1.38 µm tests predominately act to detect high cloud. In the naïve Bayesian logic, each test contributes to the final answer. The presence of tests that are not sensitive to low cloud may reduce the final sensitivity of the cloud mask to low clouds. One can therefore hypothesize that the absence of these high-cloud sensitive tests in the AVHRR/3b mask may increase the sensitivity to low cloud. The switch AVHRR/3b configuration also decreases the cloud detection off of the coast of Antarctica. The other features in the AVHRR/3b comparison are similar to the VIIRS comparison. The lower right panel in
Figure 6 shows the difference in global cloud fraction from the AVHRR/3a and AVHRR/3b detection results. This panel shows that the switch of the AVHRR channel configuration has little impact on the cloud detection over most of the globe. AVHRR/3b shows slightly more cloud in some regions (
i.e., The Sahel) where the surface characteristics of the 1.6 and 3.75 µm channels are most uncertain and variable.
Table 5 shows the sensitivity of the CALIPSO PC metrics as a function of the different spectral contents for each sensor. The results in
Table 5 do confirm the belief that more spectral information improves the global performance of the PATMOS-x cloud fraction. However, the variation in the PC values is less than 3% for non-snow/ice covered surface types. Only in the Antarctic does PATMOS-x run with MODIS channels have the highest PC value. For ocean, MODIS is in fact the worst performing, but the variation in PC is small. This does raise the suspicion that the naïve Bayesian formulation used in PATMOS-x may not be optimally utilizing the additional spectral signatures offered by MODIS. This decrease in performance in the ocean surface type is consistent with the decrease in oceanic cloud amount in
Figure 6c. Except for the desert surface type, the NOAA-19/AVHRR/GAC PC values are less than those seen for the MYD02SSH/AVHRR. The differences are biggest for Arctic (8%). While radiometric differences and the angular differences in co-locations exist, it is not clear if these can explain all of these differences.