Next Article in Journal
Detection of Oil near Shorelines during the Deepwater Horizon Oil Spill Using Synthetic Aperture Radar (SAR)
Previous Article in Journal
An Automated Approach to Map Winter Cropped Area of Smallholder Farms across Large Scales Using MODIS Imagery
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Impact of AVHRR Channel 3b Noise on Climate Data Records: Filtering Method Applied to the CM SAF CLARA-A2 Data Record

1
Swedish Meteorological and Hydrological Institute, Folkborgsvägen 17, SE-601 76 Norrköping, Sweden
2
Department of Meteorology, University of Reading, Whiteknights, P.O. Box 217, RG6 6AH Reading, Berkshire, UK
3
Deutscher Wetterdienst, Frankfurter Str. 135, 63067 Offenbach, Germany
*
Author to whom correspondence should be addressed.
Remote Sens. 2017, 9(6), 568; https://doi.org/10.3390/rs9060568
Submission received: 12 April 2017 / Revised: 19 May 2017 / Accepted: 1 June 2017 / Published: 6 June 2017
(This article belongs to the Section Atmospheric Remote Sensing)

Abstract

:
A method for reducing the impact of noise in the 3.7 micron spectral channel in climate data records derived from coarse resolution (4 km) global measurements from the Advanced Very High Resolution Radiometer (AVHRR) data is presented. A dynamic size-varying median filter is applied to measurements guided by measured noise levels and scene temperatures for individual AVHRR sensors on historic National Oceanic and Atmospheric Administration (NOAA) polar orbiting satellites in the period 1982–2001. The method was used in the preparation of the CM SAF cLoud, Albedo and surface RAdiation dataset from AVHRR data—Second Edition (CLARA-A2), a cloud climate data record produced by the EUMETSAT Satellite Application Facility for Climate Monitoring (CM SAF), as well as in the preparation of the corresponding AVHRR-based datasets produced by the European Space Agency (ESA) Climate Change Initiative (CCI) project ESA-CLOUD-CCI. The impact of the noise filter was equivalent to removing an artificial decreasing trend in global cloud cover of 1–2% per decade in the studied period, mainly explained by the very high noise levels experienced in data from the first satellites in the series (NOAA-7 and NOAA-9).

Graphical Abstract

1. Introduction

Satellite-based climate data records have become increasingly important for climate monitoring and climate change studies because of their maturity and the gradually increasing length of their covered observation period. The latter circumstance leads especially to better confidence in the determination of climate trends as well and also to a strengthening of the overall statistical significance as a climate data record. However, the increasing length of data records inevitably leads to variations in the quality of data due to factors such as changes in the behavior of individual sensors and/or changes in sensor design where original spectral channels (often called “heritage channels”) only exist as a sub-set of all channels. This could then lead to new problems since the revised sensor performance (often clearly improved compared to predecessors in terms of stability and signal to noise ratio) can be misinterpreted as an artificial trend or discontinuity in the long-term measurement series. In conclusion, the longer measurement series we have for one sensor or sensor family, the more we have to work with mitigation methods to avoid introducing artificial trends in climate data records.
The work with homogenization of climate data records is an immense task, and it has many aspects that need consideration depending on the sensor or sensor family. In this paper, we want to highlight one particular feature that is specific to climate data records based on the Advanced Very High Resolution Radiometer (AVHRR, [1]). The feature to be discussed is the impact of radiometric noise in AVHRR channel 3b at 3.7 micron on AVHRR-derived climate data records. The noise problem, producing herring-bone patterns of quite varying intensity in images, was identified early as a potential problem for climate monitoring applications (e.g., [1], page 101: “The noise in Channel 3 makes it difficult to use data from this channel in climatological studies”). Some noise filtering procedures were developed [2,3] and had some success, but no method evolved into any integral and vital part of the standard AVHRR pre-processing software packages, which are now widely used to prepare data for climate monitoring purposes. Thus, it is still up to each data producer to deal with this problem and take the necessary precautions.
One reason for why the problem has been largely ignored (or possibly dealt with by more simple lowpass filtering methods) has been its intermittent appearance among the early satellites in the National Oceanic and Atmospheric Administration (NOAA) satellite series. For some satellites (e.g., NOAA-7, NOAA-9 and NOAA-12) and for some periods of the satellite lifetime, the problem has been significant, but, for others (e.g., NOAA-11 and NOAA-14), the problem has been less pronounced. After introduction of the third version of the AVHRR sensor (AVHRR/3, first appearing on satellite NOAA-15 launched in 1998), the core interference problem was finally solved technically, which further limited the interest in the problem. This evolution of channel 3b sensor performance is nicely summarized in [4] (Table 1). However, the increasing interest in creating climate data records based on AVHRR data, which is now the longest available multispectral image data record available (with data since 1978), means that this issue arises again and must be dealt with.
This paper presents a method for filtering channel 3b noise based on a dynamic filtering approach utilizing the recorded time variability of the noise and the dependence on the scene temperatures. The method has primarily been used in the preparation phase of one particular climate data record: the CLARA-A2 data record [5]. The acronym stands for CM SAF (Climate Monitoring Satellite Application Facility, www.cmsaf.eu) cLoud, Albedo and surface RAdiation dataset from AVHRR data—Second Edition. CLARA-A2 covers a 34-year period (1982–2015) and we will demonstrate the impact with and without a channel 3b noise filter on the resulting cloud products.
Section 2 will introduce the problem of channel 3b noise in greater depth with regard to the cloud screening process of AVHRR imagery with examples given for one of the early polar orbiting NOAA satellites. The method for reducing the impact of the noise is presented in Section 3 and full-scale results based on the entire 34-year data record are presented in Section 4. The impact and validity of the filtering procedure for the CLARA-A2 data record is discussed in Section 5, with final conclusions given in Section 6.

2. The Importance of Channel 3b Noise for Multispectral Cloud Screening of AVHRR Data

To illustrate what the noise problem could really mean for cloud screening methods applied to AVHRR imagery, we present two examples taken from one NOAA-7 orbit from 1 January 1983 with the first scanline recorded at 00:07 UTC. We have selected two test areas, with positions displayed in Figure 1, where noise problems were found to be particularly serious. Figure 2 and Figure 3 are composed from AVHRR Global Area Coverage (GAC) data with a horizontal resolution of approximately 4 km. These figures are examples taken from a NOAA-7 period when the channel 3b noise for that satellite reached remarkably high levels. The applied cloud processing method is the same as being used for the generation of CLARA-A2 [5,6].
The first example in Figure 2 (corresponding to Test area 2 in Figure 1) illustrates typical problematic conditions for night-time and twilight AVHRR imagery over a region near Antarctica where the visible part of Antarctica has very cold surface temperatures. The typical herringbone pattern can be seen everywhere in Figure 2A and most pronounced for thick midlevel clouds and other cold surfaces (e.g., Antarctic surfaces in the lower portion of Figure 2A). The effect on the resulting cloud mask image is illustrated in Figure 2B. The most striking false features in the cloud mask product is the noisy striped pattern seen over presumably cloud free areas over ocean (upper part of the image) and over the Antarctic surface (at the bottom of the image). For most other parts, the cloud screening seems to work satisfactorily, indicating that cloud tests other than the ones related to channel 3b are safely able to detect most clouds. Thus, we find most problems over cloud-free parts of the image where other cloud tests are not indicating clouds.
Figure 3 shows another portion of the same AVHRR GAC orbit over the Arabian peninsula and adjacent regions (Test area 1 in Figure 1), now observed under exclusively night-time conditions. The herringbone pattern is not as easily seen in Figure 3A as in the previous Figure 2A, but the effects on the cloud type product in Figure 3B are substantial. The noisy pattern of false clouds almost dominates the scene. The two examples shown here are indeed extreme cases selected from conditions with very high levels of channel 3b noise. Nevertheless, they clearly illustrate the vulnerability of the cloud screening process to these unwanted fluctuations and perturbations of the infrared radiances and brightness temperatures.
The two examples in Figure 2 and Figure 3 indicate that the impact of channel 3b noise in the cloud screening process varies depending on the situation. For example, it is not always the areas with most evident visible noise patterns in the radiance images (i.e., Figure 2A and Figure 3A) that create the largest changes of the cloud mask product. To understand this, we need to recall some of the most essential principles for the cloud screening process. As elaborated in detail in [7,8], the 3.7 micron channel offers a very important complementary capability for cloud screening compared to the more traditional visible and infrared channels. For visible channels, clouds are generally brighter than the surface and the same is true for infrared channels provided that the signal is inverted so that the coldest targest are shown as bright targets and vice versa. However, this method will fail for most clouds located over snow surfaces or for low (warm) clouds over bright land surfaces (e.g., deserts). But since water clouds reflect strongly in the 3.7 micron channel as opposed to bright surfaces on Earth, this channel is particularly useful as a complement to the visible and infrared cloud tests. Furthermore, these clouds’ ability to reflect sunlight also means that they will not act as perfect blackbodies at night. Thus, by comparing their brightness temperatures to the measurements at longer infrared wavelengths (where they act more like blackbodies), clouds can be detected also at night even if cloud temperatures are close to the surface temperatures. At the same time, cloud transmissivities for thin cirrus clouds are larger at 3.7 microns than at longer wavelengths. This means that if studying e.g., the brightness temperature difference between the channel at 3.7 microns and the channel at 12 microns at night, we get a positive difference for thin cirrus clouds and a negative difference for thick water clouds. The cloud-free surface generally does not show these differences at night, although some compensations or corrections have to be applied over desertic regions where surface emissivities may vary (in CLARA-A2, this is done using MODIS-derived surface emissivity climatologies). Because of this, AVHRR channel 3b is quite important in the cloud screening process. It also follows that, since this channel is also central for the determination of other cloud properties (like cloud phase and cloud droplet characteristics), these noise problems will also increase the uncertainty in the retrieval of cloud products other than the cloud mask.
From this background, it follows that if periodically and spatially varying noise is added to the channel 3b measurement, the night-time and twilight cloud separability in infrared AVHRR channel data could be seriously affected. If the noise has a large amplitude (as illustrated in Figure 2 and Figure 3), it creates artificial brightness temperature variations in channel 3b and consequently also in all brightness temperature difference features involving this channel. In the worst case, a wave-like pattern of alternating false low cloud types and thin cirrus clouds may appear in the cloud typing process despite truly and completely cloud-free conditions. This is exactly what is illustrated in Figure 3B. Without the extra noise in the measurement, we would not have any remarkable brightness temperature differences between measurements in the 3.7 micron and the 12 micron channels, and we would have achieved completely cloud free conditions in the cloud mask product. However, due to the strong noise with a high amplitude, the result is instead turned into a dominantly but falsely cloudy state. Bad conditions can also occur in twilight conditions (here defined as the solar zenith interval 80–95 degrees—often associated with the time of the daily minimum of the surface temperature), but, for higher sun elevations, the reflected component measured in the 3.7 micron channel eventually dominates over the emitted one and the problems diminish.

3. Development of a Dynamic Channel 3b Noise Filter

The complex behavior of the channel 3b noise means that no standard lowpass filters with fixed size can be used to remove it, simply because the noise pattern varies in time and space in AVHRR imagery. One particularly difficult feature is that, for higher noise levels, the pattern of the noise changes, often yielding noise with a larger spatial scale (i.e., longer spatial wavelengths). Even though more complex methods have been attempted on high resolution AVHRR imagery [2,3], their applicability to a large data record of historic and coarse resolution AVHRR GAC orbits has not yet been demonstrated.
The noise filtering method developed in the preparation phase of the CLARA-A2 data record should be seen as a data rescue action rather than as a data recovery action. Thus, we want to retrieve as much as possible of the true Earth observation signal without risking destroying true signals (also pointed out specifically in [2]). More clearly, the filter must only be used when needed and not in noise-free or close to noise-free situations. If applying it in the latter situation, the true signal could be altered in a way that fine resolution details are lost, i.e., true brightness temperature differences at the finest scales might be removed. Thus, the use of the filter has to be tightly linked to the actual level of the channel 3b noise for each AVHRR sensor. The natural choice for linking the filter to noise levels is to make use of measured space count variability of the sensor onboard the satellite. For each scan of the sensor, there is a measurement into deep space that is used as the lower calibration reference. The space count would be a stable value in undisturbed conditions, but this measurement is also affected by the external interference disturbance onboard the space platform causing the channel 3b noise problem. Thus, a measurement of the variance of the space count would be a good measure of the channel 3b noise level. Figure 4 gives the noise equivalent delta temperature (NeΔT) derived from the space view measurements for all AVHRR sensors on satellites NOAA-7, NOAA-9, NOAA-11, NOAA-12 and NOAA-14 (i.e., the satellites carrying the AVHRR/2 sensor, remembering that the CLARA-A2 data record only uses data from the AVHRR/2 and AVHRR/3 sensors). The noise is first estimated from the raw counts observed when looking at space and is derived from data aggregated over a complete orbit. Here, we note that, due to the electronic clamping performed on-board the AVHRR, there is no variation in the mean value of the space view counts. The derived counts noise is then converted into noise in radiance space using the AVHRR calibration equation with the average instrument gain determined over the given orbit. The radiance noise is then converted into the NeΔT using the Planck function at a fixed scene brightness temperature, in this case 300 K. We can, in particular, see the problematic behavior of channel 3b for satellites NOAA-7, NOAA-9 and NOAA-12 while the other satellites show less of a problem, with high noise levels occurring only sporadically. Notice also that for NOAA-7 the high noise levels were reduced back to more normal levels in autumn 1983 by specific satellite operations, but this lasted only for a couple of months until noise levels started to increase again.
In addition to a dependency on actual noise levels, we also need to deal with the fact that noise levels will be more serious for cold image targets than for warm targets. This is a consequence of the nonlinear transfer of original counts and radiances into brightness temperatures using the inverse Planck function. Thus, we need to apply a filter that is dynamic in its size and influence area, and which is a function of channel 3b noise levels and the actual scene temperature.
Several types of filters were tested, including the Wiener filter proposed in [2]. However, the final choice was to use a Median filter, which replaces the central pixel value in the filter kernel of size N with the median value in the kernel. The main advantage of the Median filter is the treatment in areas where ‘no data’—pixels exist, which frequently occurs in historic AVHRR data. The median filter may change a pixel value from ‘no data’ to the median of the surrounding pixels radiance values and vice versa, but, as long as we have valid radiances in other AVHRR channels for this pixel, the results are not changed significantly. Most important is that if we also have ‘no data’ in all other channels, the impact will be zero, i.e., this will result in ‘no data’ in the cloud product regardless of having performed filtering or not. The median filter also preserves cloud edges better than mean filters. The filter was also found to be about three times faster than the Wiener filter, which is an important aspect when processing very large datasets.
The methodology can be summarized as follows:
(1)
A Median filter is applied with dynamic kernel size as a function of channel 3b noise levels.
(2)
The size of the used circular kernel is varied from radius of two GAC pixels for low noise levels (up to 0.1) to a radius of seven GAC pixels for the highest noise levels (above 1.25). A radius of 2 means that in total 13 pixels including the considered central pixel are included. For noise levels between 0.1 and 1.25, the radius is linearly interpolated between the two and rounded down to nearest integer.
(3)
A slightly modified definition of the noise level nl was used compared to what is shown in Figure 4. Instead of using the standard deviation of the space count we use the standard deviation of the internal black body count. Both quantities are highly correlated but the black body count gives a slightly smoother dependency curve which better resemble the measurement variability of Earth view measurements. After applying a constant scale factor, nl is adjusted to gives values which approximately match the range of values in Figure 4.
(4)
A post-processing restoral approach is applied in addition to reduce the effect of erroneously removing pixel-scale true clouds over warm surfaces.
The third step here is a way of introducing a scene temperature dependency on the filtering method. If not applying any restoral method, small-scale features (e.g., cumulus clouds or small holes in cloud decks) that are correctly depicted in the warmer parts of AVHRR scenes (i.e., not produced by noise) might risk to be lost after filtering. The restoral method checks the temperature correction T for each pixel and judges if this is a reasonable correction or not by checking the pixel temperatures. If the correction is not found to be reasonable (i.e., exceeding a maximum T M A X range), the value is restored back to its original channel 3b brightness temperature.
An optimal determination of the parameter T M A X is crucial for the restoral method to be efficient. It should be linked to the actual noise levels in radiance space rather than to brightness temperatures to avoid too strong nonlinear effects due to the dependency on the Planck Function. We have chosen to formulate T M A X using a reference temperature at 270 K and with a linear dependency on the noise level n l . We first define that at a temperature 270 K plus a temperature deviation T M A X 1 the maximum allowed temperature difference T M A X 1 should follow the relation given by
Δ T M A X 1 ( 270 + Δ T M A X 1 ) = 15 n l
Via the Planck Function B λ ( T ) , we can then calculate the corresponding radiance difference Δ R M A X (resulting for a temperature Δ T M A X 1 warmer than the studied temperature) by
Δ R M A X = B λ = 3.7 μ m ( 270 + 2 Δ T M A X 1 ) B λ = 3.7 μ m ( 270 + Δ T M A X 1 )
We will now use this radiance difference and keep it fixed to recalculate the corresponding Δ T M A X ( T ) to be used for all other temperatures than 270 K in the relation
Δ T M A X ( T ) = B λ = 3.7 μ m 1 [ B λ = 3.7 μ m ( T ) + Δ R M A X ] T
In this way, we get a more realistic calculation of the impact of the noise than if we had formulated this entirely in brightness temperature space. The reason is that the noise is a constant addition to the true radiance measured by the sensor regardless of its actual value (thus, without temperature dependence).
Table 1 describes the resulting Δ T M A X ( T ) values for some selected temperatures for the restoral method. Results are given here for one low noise level category and one high noise level category. We notice the strong dependency on the noise level so that the allowed temperature difference after filtering is higher for cases with very high noise levels. Notice, however, that, as an extra precaution, we always perform filtering if both the original and filtered channel 3b brightness temperatures are colder than a certain value (here, 263 K), acknowledging the fact that, for cold temperatures, the impact of channel 3b noise is likely to be very high and further augmented by poor radiometric resolution effects. Thus, our restoral method will then be less justifiable or relevant and should therefore be discarded.
A problem with this method is that, if just relying on channel 3b brightness temperatures for the determination of T and Δ T M A X ( T ) , the noise itself might have altered the scene temperature so much that it is not truly representative any more. Thus, in order to not be too sensitive to the noise effect in very cold situations, we have applied a combined use of 3.7 micron and 11 micron brightness temperatures as follows:
(1)
At night (defined as situations with less than 1% reflectivity measured in AVHRR channel 1), use 11 micron brightness temperatures as the reference when calculating Δ T M A X ( T ) .
(2)
During the daytime, use the maximum of the original and the filtered 3.7 micron brightness temperatures as the reference when calculating Δ T M A X ( T ) .

4. Results

4.1. Impact on Individual GAC Scenes

Two examples of how filtered products compare to unfiltered products are given in Figure 2C and Figure 3C.
Visual inspection indicates a very satisfactory performance of the filtering in Figure 2C. Obviously, cloud-free regions over the ice-free ocean are now restored completely, getting rid of the noisy pattern of false clouds. The visible part of Antarctica in the lower part of the GAC scene is now described more realistically, revealing large cloud-free areas over the Antarctic continent.
The other example in Figure 3C shows a less efficient case where the filtering is only capable of removing some of the false clouds. It shows that, in some situations, very serious noise patterns create conditions that are not possible to restore completely and only some reductions can be achieved. In particular, in regions where false clouds dominate, a median filter (or any other of the tested filters) cannot recover the original signal very well.

4.2. Regional Differences in Monthly Climatologies

To study the full impact of the filtering process, one has to look at the impact on real climatologies based on the aggregation of a large number of GAC orbits. This cannot be done easily by simple prototyping efforts because of the need for a very extensive processing of data. However, we will take advantage of the fact that two complete but slightly different versions of the CLARA-A2 data record were produced. This is explained by the fact that the first reprocessing effort had to be repeated after discovering some minor technical problems affecting a small part of the GAC dataset and also because of a minor arithmetic error in the calculation of monthly climatologies. This reprocessing offered a chance to introduce the new noise filtering procedure and to study the full impact of it over the entire 34-year period. Thus, by comparing the two versions of the data record and by considering the other changes made to the processing, it is possible to study the sole impact of the channel 3b noise filtering.
In summary, the following changes were introduced in the second and final reprocessed version of the CLARA-A2 data record:
(1)
Updated method of removing overlap between consecutive GAC orbits;
(2)
Removal of an incorrect rounding error in the calculation of monthly means;
(3)
Blacklisting of a number of NOAA satellite orbits in the period 2000–2004 due to scan motor problems;
(4)
Introduction of the channel 3b noise filter.
The first item regarding the orbit overlap treatment had negligible impact on final climatologies since it only affected a small fraction of GAC orbits, and every affected orbit only impacted a few scan lines (out of more than 12,000 lines) per GAC orbit. The second item had a much larger effect on results since it meant that values were rounded to the nearest lower integer value in the first CLARA-A2 processing effort. In the case of cloud fraction, it meant that results were biased low with a value between 0–1% in absolute values. This feature becomes quite noticable for very cloudy regions where overcast gridboxes have been systematically given a cloud fraction value of 99% instead of 100%. For any other region, the cloud fraction is distributed over a larger value range, and the difference will be smaller but still negative (i.e., underestimated). However, in practice, the correction would be close to 1% in global averages since the overcast situations dominate over fractionally cloudy or cloud free cases. However, since this correction would be valid and relatively stable over the entire data record, it can be adequately accounted for in this particular study. The third item is actually mostly irrelevant for the channel 3b noise study since this affected only a few months of data from NOAA-14 at the end of its lifetime. Thus, if restricting the study to the period 1982–1999, we would see the full effect of channel 3b noise filtering provided that we also take into account the ~1% increase in cloud fraction due to the correction of the rounding error.
Figure 5 shows the impact of the channel 3b noise filtering and the rounding error correction for the mean CLARA-A2 cloud fraction of the months of January and July 1983 from the NOAA-7 satellite. In both these months, the channel 3b noise levels were very high (see Figure 4). Thus, the figures illustrate more or less the maximum effects that were encountered during channel 3b noise filtering.
We conclude that the effect of the filtering is much larger than the effect of the rounding error. In fact, filtering is able to change monthly mean cloud cover by more than 25% in some regions compared to the maximum change of 1% from the rounding error. For example, Australia stands out in particular in the results for July 1983 (Figure 5B) where reductions larger than 25% can be seen almost over the entire continent. Other regions with large reductions are southern Africa and the southern part of South America in July 1983 and parts of Antarctica in January 1983. The filter obviously remove clouds over large areas (red parts in Figure 5), but it also leads to large increases in cloudiness in some regions. The increases are most pronounced over sea ice regions during polar winter seasons. This is explained by the specific cloud mask thresholding sequence differences over land and sea in very cold situations at night. Previously (with no filter applied), conditions with high noise levels would allow temperatures to vary wildly in channel 3b in very cold situations. Since this was known to seriously impact brightness temperature tests, those tests were simply turned off when temperatures fell below a threshold (230 K). However, the filtering process now causes a large number of those pixels to register as warmer than 230 K, and be analyzed using the brightness temperature tests. This leads to a labelling of more cloudy pixels than before. Furthermore, many of these tests also have an additional condition that the variance in channel 3b should not be too high. Large variance over sea ice indicates either noise in channel 3.7 or ice cracks. The filtering procedure generally reduced this variance over sea ice regions, which also leads to the detection of more clouds. These circumstances lead to an apparent asymmetry in the resulting changes over the two Polar Regions. However, if just separating the effects over sea ice and land portions, respectively, the effects appear to be rather similar over both Polar Regions.

4.3. Contribution to 34-Year Trend in Cloudiness

Figure 6 shows the globally averaged cloud cover for the two CLARA-A2 versions and a difference plot. In the period 2000–2003, a significant fraction of GAC orbits were removed which caused large changes in results. After 2003, there is an almost constant change of just below 1% cloud cover except for some spikes caused by the removal of some additional erroneous GAC orbits. This is the impact of correcting for the rounding error. In the period 1982–1999, we should only see the combined effect of the corrected rounding error and the introduction of the median filter to channel 3b brightness temperatures. It is clear that, since the rounding effect should be approximately constant over the years (varying marginally with the global mean cloud cover), the filtering procedure generally leads to reduced cloudiness (i.e., the difference is generally smaller than 1%). This reduction is largest in the beginning of the period (1982–1988 with the NOAA-7 and NOAA-9 satellites) with an extreme negative peak in July 1983 (corresponding to the situation illustrated in Figure 5B). The variability in the impact of the filtering effect is also largest in this early period.
As indicated by Figure 5, the regional impact might be quite different from the impact given by the global mean figures in Figure 6. This is further exemplified in Figure 7, showing the impact (i.e., difference after filtering) for the tropical region and the South Pole region. It is clear that, for lower latitudes, the effect of filtering is mainly the removal of (presumably) false clouds, while, at higher latitudes, the effect alternates between mainly removing clouds in the polar summer and adding clouds in the polar winter.

5. Discussion

Results in Section 3 imply that AVHRR channel 3b noise is capable of degrading cloud climate data records considerably, especially during periods of high noise levels. If assuming that corrections are reasonable, results show that ignoring noise effects might contribute to an overall decreasing trend of about 1–2% per decade in cloud cover over the period (with largest overestimations in the beginning of the period). In addition to this, noise will increase the variability of climate relevant parameters in an unrealistic way. For the CLARA-A2 data record specifically, this appeared to be most serious for the Polar Regions, where noise effects caused seasonally varying and opposed effects.
The artificial trend might be specific for CLARA-A2 due to the heavy use of cloud tests based on the brightness temperature differences between channel 3b and the other infrared channels. However, other data records making heavy use of AVHRR channel 3b data (e.g., PATMOS-x [9] and International Satellite Cloud Climatology Project (ISCCP) [10]) are also likely to be influenced negatively by AVHRR channel 3b noise.
No filtering method will probably ever be capable of completely recovering the original AVHRR noise free measurement, especially not in situations with very high noise levels that completely dominate the measurement. However, the use of dynamically varying filters has some success even with these very problematic conditions, as demonstrated in this study. Thus, applying filters will definitely have some data rescuing value. As such, the method is probably only applicable to AVHRR data because of the very specific origin and character of the noise caused by interference with other systems onboard the NOAA satellite platform.
Despite the seemingly modest impact of the noise on global trends, we cannot ignore the fact that accuracy requirements of measurements to be used for climate change studies are very strict and demanding [11]. In that perspective, the noise problem has to be tackled appropriately together with other calibration and navigation issues of AVHRR GAC data.
The results presented in this paper should be seen as a first more extensive and systematic effort of handling the channel 3b noise problem in the production of an AVHRR GAC based climate data record. The method was not only used for CLARA-A2 generation, but it has also been used in a project for generating an AVHRR-heritage cloud data record [12] of the ESA Climate Change Initiative (CCI) programme [13]. The latter project aims at studying the specific essential climate variable (ECV) denoted “cloud properties” and the project is formally named ESA-CLOUD-CCI [14]. The filtering functionality will also be added to the general AVHRR calibration and pre-processing software package PyGAC (under GNU General Public License, available at https://pygac.readthedocs.io/en/develop/) [15].
Future improvements of the methodology are possible. Especially, the filtering procedure should preferably be carried out in radiance space rather than in brightness temperature space in order to avoid the strongly nonlinear effects (in particular affecting cold scene temperatures) resulting when applying the inverse Planck function. However, this requires some improved flexibility of the currently used state-of-the-art calibration and pre-processing tools. A first demonstration of such a radiance-based filtering approach applied to the entire AVHRR GAC dataset has recently been made by National Aeronautics and Space Administration (NASA) [16]. This method does not use pre-existing knowledge of temporally varying noise characteristics but is instead based on a segmentation of individual GAC orbits into smaller segments, where a Fast Fourier transform analysis and filtering can be made. This method is very interesting, and results will be studied and compared to the current method in the near future. The most interesting aspect of these future studies of the two methods will be to check the balance between the removal of truly artificial noise features and the unwanted removal of small scale true features in the filtered AVHRR scenes.
Finally, improved descriptions of noise and uncertainty characteristics, as well as a considerably upgraded infrared AVHRR calibration methodology, are foreseen as important outcomes of the European Union Horizon 2020 project Fidelity and uncertainty in climate data records from Earth Observations (FIDUCEO, [17]). This will contribute to better means of handling radiance noise problems in fundamental climate data records (FCDR) such as the AVHRR GAC data record.

6. Conclusions

A method for reducing the impact of noise in the 3.7 micron spectral channel in climate data records derived from coarse resolution (4 km) global measurements from AVHRR data has been presented.
The impact of the noise was demonstrated for two selected cases from the NOAA-7 satellite in a period when noise levels were extremely high. Based on some fundamental characteristics of the noise, a dynamic size-varying median filter was suggested to be operated and guided by measured noise levels and being dependent on scene temperatures for individual AVHRR sensors. The impact of applying the noise filter was demonstrated for two selected monthly cloud climatologies as well as for the entire data record from 1982 to 2015. Globally, the filter generally reduced cloud cover leading to the removal of a decreasing global trend of about 1–2% per decade in the period 1982–2001. However, the impact of the filter showed strong regional and seasonal dependencies. For low latitudes, cloud cover was generally reduced, while, for the Polar Regions, corrections were alternating positively or negatively depending on season (positively in polar winter, negatively in polar summers). Thus, not only the global trend was affected but also the climate variability in different regions.
The method has been used in the preparation of the CLARA-A2 data record as well for preparing the corresponding AVHRR-based datasets produced in the ESA project ESA-CLOUD-CCI. Future improvements of the methodology can be achieved if applying filters in radiance space instead of in brightness temperature space to avoid the nonlinear dependence from the Planck function. Such an approach has recently been applied [17] and corresponding results will be compared with the results of the current method in the future. In addition, improvements based on a better characterization of the noise are anticipated as an outcome of the EU Horizon 2020 project FIDUCEO.

Supplementary Materials

Supplementary File 1

Acknowledgments

This work was funded by EUMETSAT in cooperation with the national meteorological institutes of Germany, Sweden, Finland, the Netherlands, Belgium, Switzerland and United Kingdom. The authors are very grateful to reviewers for their advice on improvements of the manuscript.

Author Contributions

Nina Håkansson and Karl-Göran Karlsson conceived and designed the experiments and carried out the prototyping of the method; Jon Mittaz prepared and provided the AVHRR channel 3b noise information, Timo Hanschmann and Abhay Devasthale prepared and analysed the resulting climate data records; and Karl-Göran Karlsson wrote the paper.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Cracknell, A.P. The Advanced Very High Resolution Radiometer; Taylor & Francis Ltd.: Oxfordshire, UK, 1997. [Google Scholar]
  2. Simpson, J.J.; Yhann, S.R. Reduction of noise in AVHRR channel 3 data with minimum distortion. IEEE Trans. Geosci. Remote Sens. 2002, 32, 315–328. [Google Scholar] [CrossRef]
  3. Warren, D. AVHRR channel-3 noise and methods for its removal. Int. J. Remote Sens. 1989, 10, 645–651. [Google Scholar] [CrossRef]
  4. Trishchenko, A.P.; Fedosejevs, G.; Li, Z.; Cihlar, J. Trends and uncertainties in thermal calibration of AVHRR radiometers onboard NOAA-9 to NOAA-16. J. Geophys. Res. 2002, 107, 4778. [Google Scholar] [CrossRef]
  5. Karlsson, K.-G.; Anttila, K.; Trentmann, J.; Stengel, M.; Meirink, J.F.; Devasthale, A.; Hanschmann, T.; Kothe, S.; Jääskeläinen, E.; Sedlar, J.; et al. CLARA-A2: The second edition of the CM SAF cloud and radiation data record from 34 years of global AVHRR data. Atmos. Chem. Phys. 2017, 17, 5809–5828. [Google Scholar] [CrossRef]
  6. Dybbroe, A.; Thoss, A.; Karlsson, K.-G. NWC SAF AVHRR cloud detection and analysis using dynamic thresholds and radiative transfer modelling—Part I: Algorithm description. J. Appl. Meteorl. 2005, 44, 39–54. [Google Scholar] [CrossRef]
  7. Karlsson, K.-G.; Johansson, E.; Devasthale, A. Advancing the uncertainty characterisation of cloud masking in passive satellite imagery: Probabilistic formulations for NOAA AVHRR data. Remote Sens. Environ. 2015, 158, 126–139. [Google Scholar] [CrossRef]
  8. Musial, J.P.; Hüsler, F.; Sütterlin, M.; Neuhaus, C.; Wunderle, S. Probabilistic approach to cloud and snow detection on Advanced Very High Resolution Radiometer (AVHRR) imagery. Atmos. Meas. Tech. 2014, 7, 799–822. [Google Scholar] [CrossRef]
  9. Heidinger, A.K.; Foster, M.J.; Walther, A.; Zhao, Z. The pathfinder atmospheres extended (PATMOS-X) AVHRR climate data set. Bull. Am. Meteorol. Soc. 2014, 95, 909–922. [Google Scholar] [CrossRef]
  10. Rossow, W.B.; Schiffer, R.A. Advances in understanding clouds from ISCCP. Bull. Am. Meteorol. Soc. 1999, 80, 2261–2287. [Google Scholar] [CrossRef]
  11. Ohring, G.; Wielicki, B.; Spencer, R.; Emery, B.; Datla, R. Satellite instrument calibration for measuring global climate change. Bull. Am. Meteorol. Soc. 2005, 86, 1303–1313. [Google Scholar] [CrossRef]
  12. Stengel, M.; Stapelberg, S.; Sus, O.; Schlundt, C.; Poulsen, C.; Thomas, G.; Christensen, M.; Carbajal Henken, C.; Preusker, R.; Fischer, J.; et al. Cloud property datasets retrieved from AVHRR, MODIS, AATSR and MERIS in the framework of the Cloud cci project. Earth Syst. Sci. Data 2017, in press. [Google Scholar]
  13. Hollmann, R.; Merchant, C.; Saunders, R.; Downy, C.; Buchwitz, M.; Cazenave, A.; Chuvieco, E.; Defourny, P.; de Leeuw, G.; Forsberg, R.; et al. The ESA climate change initiative: Satellite data records for essential climate variables. Bull. Am. Meteorol. Soc. 2013, 94, 1541–1552. [Google Scholar] [CrossRef]
  14. Stengel, M.; Mieruch, S.; Jerg, M.; Karlsson, K.-G.; Scheirer, R.; Maddux, B.; Meirink, J.; Poulsen, C.; Siddans, R.; Walther, A.; et al. The clouds climate change initiative: Assessment of state-of-the-art cloud property retrieval schemes applied to AVHRR heritage measurements. Remote Sens. Environ. 2015, 162, 363–379. [Google Scholar] [CrossRef]
  15. Devasthale, A.; Raspaud, M.; Dybbroe, A.; Hörnquist, S.; Karlsson, K.-G. PyGAC: An open-source, community-driven Python interface to preprocess more than 30-year AVHRR Global Area Coverage (GAC) data. 2017; in press. [Google Scholar]
  16. A Consistent Long-Term Cloud and Clear-Sky Radiation Property Dataset from the Advanced Very High Resolution Radiometer (AVHRR)—Climate Algorithm Theoretical Basis Document, NOAA Climate Data Record Program CDRP-ATBD-0826 Rev. 1. (2016). Available online: http://www.ncdc.noaa.gov/cdr/operationalcdrs.html (accessed on 12 April 2017).
  17. Woolliams, E.; Mittaz, J.; Merchant, C.; Dilo, A. Harmonization and Recalibration: A FIDUCEO Perspective. Available online: http://www.fiduceo.eu/content/harmonization-and-recalibration-fiduceo-perspective (accessed on 12 April 2017).
Figure 1. Test areas chosen for visualization of problematic channel 3b noise conditions in one selected NOAA-7 AVHRR GAC orbit from 1 January 1983 (see Figure 2 and Figure 3).
Figure 1. Test areas chosen for visualization of problematic channel 3b noise conditions in one selected NOAA-7 AVHRR GAC orbit from 1 January 1983 (see Figure 2 and Figure 3).
Remotesensing 09 00568 g001
Figure 2. Channel 3b noise effects in a portion of an AVHRR GAC orbit near Antarctica (Test Area 2 in Figure 1) from 1 January 1983 (near 00 UTC). (A) colour composite image based on brightness temperatures of the three AVHRR infrared channels 3b, 4 and 5 at 3.7, 11 and 12 microns, respectively; (B) cloud mask product for the same image (orange is cloudy, blue is cloudfree water and green is land (including snow covered land); (C) cloud mask product after using the channel 3b noise filter; (D) difference plot where red indicates either removal or adding of clouds.
Figure 2. Channel 3b noise effects in a portion of an AVHRR GAC orbit near Antarctica (Test Area 2 in Figure 1) from 1 January 1983 (near 00 UTC). (A) colour composite image based on brightness temperatures of the three AVHRR infrared channels 3b, 4 and 5 at 3.7, 11 and 12 microns, respectively; (B) cloud mask product for the same image (orange is cloudy, blue is cloudfree water and green is land (including snow covered land); (C) cloud mask product after using the channel 3b noise filter; (D) difference plot where red indicates either removal or adding of clouds.
Remotesensing 09 00568 g002
Figure 3. Channel 3b noise effects in a portion of an AVHRR GAC orbit over Egypt and the Arabian peninsula (Test Area 1 in Figure 1) from 1 January 1983 (near 00 UTC). (A) colour composite image based on brightness temperatures of the three AVHRR infrared channels 3b, 4 and 5 at 3.7, 11 and 12 microns, respectively; (B) cloud mask product for the same image (using the same colour table as in Figure 2B,C); (C) cloud mask product after using the channel 3b noise filter; (D) difference plot where red indicates either removal or adding of clouds.
Figure 3. Channel 3b noise effects in a portion of an AVHRR GAC orbit over Egypt and the Arabian peninsula (Test Area 1 in Figure 1) from 1 January 1983 (near 00 UTC). (A) colour composite image based on brightness temperatures of the three AVHRR infrared channels 3b, 4 and 5 at 3.7, 11 and 12 microns, respectively; (B) cloud mask product for the same image (using the same colour table as in Figure 2B,C); (C) cloud mask product after using the channel 3b noise filter; (D) difference plot where red indicates either removal or adding of clouds.
Remotesensing 09 00568 g003
Figure 4. Channel 3b noise, expressed as noise equivalent temperature variation (NeΔT) at the 300 K level, during the lifetime of satellites NOAA-7, NOAA-9, NOAA-11, NOAA-12 and NOAA-14.
Figure 4. Channel 3b noise, expressed as noise equivalent temperature variation (NeΔT) at the 300 K level, during the lifetime of satellites NOAA-7, NOAA-9, NOAA-11, NOAA-12 and NOAA-14.
Remotesensing 09 00568 g004
Figure 5. Cloud fraction difference for January 1983 (A) and July 1983 (B) for the CLARA-A2 climatology after applying channel 3b noise filter and a rounding error correction (approximately −0.01 everywhere). Observe that the figure shows unfiltered minus filtered CLARA-A2 results, i.e., red colours indicating reduction after filtering and blue colours an increase after filtering. The globally averaged cloud fraction difference in January 1983 was +0.4% and in July 1983 + 4.2%.
Figure 5. Cloud fraction difference for January 1983 (A) and July 1983 (B) for the CLARA-A2 climatology after applying channel 3b noise filter and a rounding error correction (approximately −0.01 everywhere). Observe that the figure shows unfiltered minus filtered CLARA-A2 results, i.e., red colours indicating reduction after filtering and blue colours an increase after filtering. The globally averaged cloud fraction difference in January 1983 was +0.4% and in July 1983 + 4.2%.
Remotesensing 09 00568 g005
Figure 6. (A) global mean cloud fraction (% cloud cover) for the entire time period 1982–2015 for the final version of CLARA-A2 and the original CLARA-A2 version; (B) mean cloud fraction difference between filtered and unfiltered results. The impact of channel 3b filtering is visible in the period 1982–1999 (see text).
Figure 6. (A) global mean cloud fraction (% cloud cover) for the entire time period 1982–2015 for the final version of CLARA-A2 and the original CLARA-A2 version; (B) mean cloud fraction difference between filtered and unfiltered results. The impact of channel 3b filtering is visible in the period 1982–1999 (see text).
Remotesensing 09 00568 g006
Figure 7. Mean cloud fraction difference (% cloud cover) in the tropical region (Tropics) and the South Pole region for the entire time period 1982–2015 between the final version of CLARA-A2 and the original CLARA-A2 version. The impact of channel 3b filtering is visible in the period 1982–1999 (see text).
Figure 7. Mean cloud fraction difference (% cloud cover) in the tropical region (Tropics) and the South Pole region for the entire time period 1982–2015 between the final version of CLARA-A2 and the original CLARA-A2 version. The impact of channel 3b filtering is visible in the period 1982–1999 (see text).
Remotesensing 09 00568 g007
Table 1. Example of maximum allowed temperature differences ( Δ T ) after Median filtering for two different noise level categories and as a function of some selected scene temperatures (column 1). Notice that the applied method uses a finer temperature resolution of the tabulated values than presented here.
Table 1. Example of maximum allowed temperature differences ( Δ T ) after Median filtering for two different noise level categories and as a function of some selected scene temperatures (column 1). Notice that the applied method uses a finer temperature resolution of the tabulated values than presented here.
Temperature (K) Δ T M A X (K) Noise Level 0.1 Δ T M A X (K) Noise Level 1.25
22015.8 *74.0 *
23010.3 *64.3 *
2406.4 *54.8 *
2504.0 *45.9 *
2602.5 *37.5 *
2701.630.0
2801.023.5
2900.718.1
3000.513.8
3100.310.5
3200.38.0
* If both the filtered and original 3.7 micron brightness temperatures are colder than 263 K, no restoral is applied at all.

Share and Cite

MDPI and ACS Style

Karlsson, K.-G.; Håkansson, N.; Mittaz, J.P.D.; Hanschmann, T.; Devasthale, A. Impact of AVHRR Channel 3b Noise on Climate Data Records: Filtering Method Applied to the CM SAF CLARA-A2 Data Record. Remote Sens. 2017, 9, 568. https://doi.org/10.3390/rs9060568

AMA Style

Karlsson K-G, Håkansson N, Mittaz JPD, Hanschmann T, Devasthale A. Impact of AVHRR Channel 3b Noise on Climate Data Records: Filtering Method Applied to the CM SAF CLARA-A2 Data Record. Remote Sensing. 2017; 9(6):568. https://doi.org/10.3390/rs9060568

Chicago/Turabian Style

Karlsson, Karl-Göran, Nina Håkansson, Jonathan P. D. Mittaz, Timo Hanschmann, and Abhay Devasthale. 2017. "Impact of AVHRR Channel 3b Noise on Climate Data Records: Filtering Method Applied to the CM SAF CLARA-A2 Data Record" Remote Sensing 9, no. 6: 568. https://doi.org/10.3390/rs9060568

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop