Saturation Correction for Nighttime Lights Data Based on the Relative NDVI

Abstract

DMSP/OLS images are widely used as data sources in various domains of study. However, these images have some deficiencies, one of which is digital number (DN) saturation in urban areas, which leads to significant underestimation of light intensity. We propose a new method to correct the saturation. With China as the study area, the threshold value of the saturation DN is screened out first. A series of regression analyses are then carried out for the 2006 radiance calibrated nighttime lights (RCNL) image and relative NDVI (RNDVI) to determine a formula for saturation correction. The 2006 stable nighttime lights (SNL) image (F162006) is finally corrected and evaluated. It is concluded that pixels are saturated when the DN is larger than 50, and that the saturation is more serious when the DN is larger. RNDVI, which was derived by subtracting the interpolated NDVI from the real NDVI, is significantly better than the real NDVI for reflecting the degree of human activity. Quadratic functions describe the relationship between DN and RNDVI well. The 2006 SNL image presented more variation within urban cores and stronger correlations with the 2006 RCNL image and Gross Domestic Product after correction. However, RNDVI may also suffer “saturation” when it is lower than −0.4, at which point it is no longer effective at correcting DN saturation. In general, RNDVI is effective, although far from perfect, for saturation correction of the 2006 SNL image, and could be applied to the SNL images for other years.

1. Introduction

Nighttime lights data obtained by the Operational Linescan System (OLS) on board Defense Meteorological Satellite Program (DMSP) satellites was discovered to have a high correlation with urban characteristics such as Gross Domestic Product (GDP), power assumption, and population levels in the late 20th century [1]. Since then, many more studies in a wide range of disciplines have been conducted to determine useful indicators of human activities [2,3,4,5,6,7,8,9,10,11,12]. The National Aeronautics and Space Administration (NASA) and the National Oceanic and Atmospheric Administration (NOAA) launched the Suomi National Polar Partnership (Suomi-NPP) satellite carrying the first visible infrared imaging radiometer suite (VIIRS) instrument in 2011 and the VIIRS has several improvements over the OLS [13]. Because the VIIRS record began in 2011, any studies prior to that date must rely on DMSP/OLS for nighttime lights imagery. Thus, the importance of the OLS dataset has not decreased.

Scientists at NOAA’s National Geophysical Data Center (NGDC) are in charge of processing the nighttime lights data. They remove natural noises, including moonlight disturbances, aurora light displays, and forest fires, leaving mostly man-made light. They then produce a satellite-year dataset: data from all orbits of a given satellite in a given year are averaged over all valid nights. These datasets, in combination known as the World Stable Lights dataset [14], include data from 1992 to 2013 and are distributed to the public. It has been proven that these datasets can basically depict the degree of human activities at night [15]. However, with six-bit quantization and limited dynamic range (DN from 0 to 63), the recorded data are saturated in the bright cores of urban centers. The distortion caused by saturation might not be very significant in terms of the whole world, because urban centers take up a relatively small area; however, when estimating and comparing GDP, power assumption, or CO₂ emissions based on the nighttime light of local regions densely covered by cities, especially for metropolitan districts, the saturation might be too severe to be corrected.

Several studies have focused on the correction of saturation in the World Stable Lights dataset; thus, some correction methods have been developed. We have classified them into four categories, as follows:

(a) The first and best approach is to eliminate saturation by adjusting the sensor itself to utilize the dynamic satellite gain settings to produce unsaturated images [16,17]. The OLS visible band detector is typically operated in a high gain setting to detect moonlit clouds. A limited set of observations have been obtained at low lunar illumination, when the gain of the detector was set significantly lower than its typical operational setting (sometimes by a factor of 100). At this low gain setting, only bright cores of urban centers were detected [16]. By combining these sparse data acquired at low gain settings with the operational data acquired at high gain settings, scientists at the NGDC produced a global radiance calibrated nighttime lights (RCNL) product without saturation. (More detailed information about the procedure to generate the product is described in [17].) However, this approach is cost-intensive and is available only for a limited number of years (1996, 1999, 2000, 2002, 2004, 2006, 2010) [18].

(b) In the second correction method, regression models are applied, assuming that the change characteristic of the DN of the unsaturated area is the same as that of the saturated area in a given region. This means that if we can establish the relationship between the cumulative number of pixels and the corresponding DN in an unsaturated area, we could predict the cumulative DNs in a saturated area. Hara et al. used a linear correction model [19], and Letu et al. later developed it into a cubic model [20]. However, there are two defects to this kind of approach. One is that it cannot be applied at the pixel scale; namely, it is not a correction of the images but rather a correction just of the statistical results. The other is that the change characteristics differ significantly in different cities, which hinders its application at larger scales [21].

(c) In the third method, saturated pixels are corrected via regressions using pixels from each stable light image and the non-saturated OLS data. For instance, Letu et al. calculated a regression formula using the 1999 stable nighttime lights (SNL) image and the 1996 RCNL image, and then corrected the saturated pixels of the stable lights image using that formula [22]. However, the assumption in this approach is that the light intensity gradient in a saturated area does not change between the base year and the year to be corrected, which might be reasonable for developed countries over a short time span but is far from convincing for emerging countries during any time span, or for any country over a long time span.

(d) The last method is to utilize other kinds of datasets than the nighttime lights data themselves. The datasets to be used should reflect the relative contents of the nighttime lights data. One of the datasets successful for this approach is the Normalized Differences Vegetation Index (NDVI), which is based on the rationale that urban features, including nighttime lights, should be inversely correlated with vegetation abundance. Lu et al., Cao et al., and Pandey et al. combined nighttime lights data with NDVI data to enhance the mapping of urban areas [23,24,25]. Zhang et al. created a more concise index, the Vegetation Adjusted NTL Urban Index (VANUI), to overcome the saturation problem [18]. However, the intentions of these studies were to improve urban mapping rather than to correct the nighttime lights data. This means that the new indexes are not nighttime lights anymore, and thus may no longer be used to estimate things such as GDP and CO₂ emissions. Nevertheless, the idea of light correction using the NDVI is quite inspiring and promising.

In this study, we aimed to detect the relationship between the DNs of nighttime lights data and the NDVI. To this end, we propose an applicable method to reduce the saturation problem for the images of stable lights. First, we discuss the threshold of DNs above which the saturation phenomenon begins. Second, we conduct a regression analysis of the relationship between DNs and the NDVI at the pixel scale, using the 2006 RCNL image and the SPOT-VGT NDVI product to determine what kind of formula could best depict the relationship. Third, we make corrections for the saturated parts of the 2006 SNL image using the formula. Last, our correction results are tested and evaluated. Our study area is located in China.

2. Does Saturation Exist only if DN Equals 63?

Many previous studies refer to the saturation defect of the DMSP/OLS data, especially in urban studies, but few of them mention that the saturation phenomenon occurs when the DN reaches a certain level. This seems to imply that when the DN equals 63, the pixel is saturated, because the upper limit for the detection capacity of the sensor is DN = 63. However, this oversimplified viewpoint might not be accurate due to the boundary effect, and more importantly, due to sub-pixel saturation. The data, collected originally with a pixel size of 0.56 km, are spatially averaged onboard a satellite in $5\times 5$ pixel blocks to smooth the data to provide spatial resolution of 2.8 km. Under such circumstances, saturated and unsaturated pixels are certain to be averaged together so the DN of the resultant data appears less than 63 [17]. Ziskin et al. tried setting DN = 55 as a threshold to prevent saturated pixels from contaminating the calibrated result [17]. Letu et al. tested whether a linear relationship existed between the DNs of the 1996 RCNL image and the 1999 SNL image and found that pixels with DN > 20 might be saturated [22]. The linear relationship is reasonable according to how the calibrated image was produced. However, the result may not be as solid as the method because the two images are not for the same year. A gap of three years inevitably distorts the linear relationship; therefore, careful inspection is needed to properly determine the critical point for saturation and thus to distinguish which part should be corrected and which should not. This way, only necessary corrections are made later.

The 2006 RCNL image is one without saturation that we could obtain. The original images needed to produce this calibrated image were obtained by satellite F16; thus, we selected the SNL image coded F162006 as the corresponding data to determine where the critical point was. To this end, we performed two tests, one of which is similar to that carried out in [22]. The other test was to examine the distribution rules of the DNs of the two images. First, the relationship between the mean of the DNs of the 2006 RCNL image and the corresponding DN of the 2006 SNL image F162006 at the pixel scale (Figure 1) was tested. This showed that the relationship remains linear roughly when the DN < 50, and becomes increasingly distorted the higher the DN is over 50. To quantify this change, we performed a set of regressions. The regression formula is as follows:

$$\mathrm{Mean}{\left(\mathrm{DN}\left(\mathrm{calibrated}\right)\right)}_{\mathrm{DN}\left(\mathrm{stable}\right)=\mathrm{i}}=\mathrm{slope}·\mathrm{i}+\mathrm{intercept},0<\mathrm{i}\le \mathrm{UL},$$

(1)

where slope is the coefficient of the DN of the SNL image. UL is the upper limit of the DN of the SNL image, which was set from 15 to 63 in each round of regression. We then plotted the coefficient of determination (R²) and the coefficient slope for each regression (Figure 2). In the ideal situation, both indicators should be close to 1. However, the R² decreases sharply when the UL is greater than 50, and the coefficient slope gets further away from 1 when the UL exceeds 50. This indicates that the linear relationship between the DNs of the two images disappears when the DN is above 50.

Next, the distribution rules of the DN values of the 2006 RCNL image and the 2006 SNL image were examined (Figure 3 and Figure 4). These show that when the DN > 50, especially above 55, the distribution rule of the DNs of pixels in the 2006 SNL image becomes badly distorted due to the saturation problem, compared with that of the 2006 RCNL image.

It can be concluded from both tests that pixels may start to become saturated when the DN is larger than 50, or at most 55, and that the saturation is more serious when the DN is larger. To be prudent, we classified all pixels into three categories: unsaturated pixels (DN ≤ 50), slightly saturated pixels (50 < DN ≤ 55), and severely saturated pixels (DN > 55). The unsaturated part needs no correction, while the severely saturated part will be corrected later. Strictly speaking, the slightly saturated part should also be corrected; however, any kind of correction may introduce new errors, and we cannot guarantee, for the middle range, that the amelioration by correction is larger than the error caused. Consequently, we decided not to correct the slightly saturated part in this study.

3. Exploring the Relationship between the NDVI and Nighttime Lights

Previous study results support the idea that the NDVI or the similar Net Primary Productivity (NPP) is negatively correlated with urban characteristics. One indicator of this is the intensity of nighttime lights [18,26,27]. It seems that the NDVI is a good resource to help correct the saturation problem of nighttime lights data. Although there appears to be no empirical study about the quantitative relationship between the NDVI and nighttime lights until now, we suspect that the real NDVI, without further processing, is not suitable for saturation correction at the national or global scale because it varies significantly according to the local climate [28,29,30]. For instance, the real NDVI for a city located in an arid area may be quite different from that for a similar city in a humid region. Therefore, RNDVI (relative NDVI) for urban areas, which is defined below to eliminate disturbance due to local natural factors, is suggested as an alternative measure.

$$\mathrm{RNDVI}=\mathrm{real}\mathrm{NDVI}-\mathrm{interpolated}\mathrm{NDVI},$$

(2)

where interpolated NDVI means the NDVI interpolated by NDVI of non-urban areas, which generally reflects local climate conditions.

Our major data source for RNDVI was the SPOT-VGT images (http://free.vgt.vito.be). These are 10-day composites (36 images per year) with a 1-km spatial resolution. After processing the primary VGT data into NDVI, we retrieved the maximum value of the 36 images for year 2006 for each pixel [31]. We thus obtained a yearly composite (real NDVI). To create the RNDVI image, we also need the 2006 SNL image to assist in determining which pixel belongs to urban regions and should be updated by RNDVI. In this study, we took DN = 20 as the threshold, which means that the interpolation should be performed for pixels with DN > 20 using the NDVI of pixels with DN ≤ 20. If the threshold is lower, the area to be interpolated would be larger; thus the precision of the interpolation would be greatly reduced. If the threshold is higher, the pixels to be analyzed later by regression would be greatly decreased. We did some calculations and found that DN = 20 was a good compromise for China and it was thus adopted. Different regions may have different optimal thresholds and thus relevant calculations should be performed to help determine it. With respect to the interpolation method, we adopted the natural neighbor interpolation based on its simplicity of calculation and rationality of interpolation results. In addition, the extremely low NDVI of water bodies should be set to null before interpolation, otherwise the interpolation results would be severely influenced. The interpolation was processed using ArcGIS 10.2, and the procedure is shown in the following flowchart (Figure 5).

Figure 6 and Figure 7 are the real NDVI and RNDVI for China, which suggests that RNDVI is an indicator without natural noises, and better reflects the degree of human activities than the real NDVI. Further quantitative comparison is performed later in this section.

A series of regressions was performed using Stata 13.0 to identify the fittest order of function needed to describe the relationship between DN − 20 of the 2006 RCNL image and RNDVI. Ideally, RNDVI should be 0 if the DN = 20, according to the procession rule above. Consequently, the regression equation should be a homogeneous equation, namely, the constant term should be forced to be 0. The results of the linear regression, quadratic regression, and quartic regression are shown below in

Table 1. Results of different order regressions between DN − 20 and RNDVI for the 2006 RCNL image.

	R2	Coefficient
RNDVI	0.42	−534.56
RNDVI2	0.48	1793.04
RNDVI4	0.29	7554.81

. Although all three regressions are significant and effective, the quadratic function depicts the relationship best and thus was selected to correct the saturated area of the 2006 SNL image.

To help prove that RNDVI is a better indicator than the real NDVI for saturation correction, more regressions were performed between the real NDVI and corresponding DNs for those pixels with DN > 20. The R² of linear, quadratic, and quartic regressions were below 0.2 (

Table 2. Results of different order regressions between DN − 20 and the real NDVI for the 2006 RCNL image.

	R2	Coefficient
NDVI	0.14	77.89
NDVI2	0.08	90.90
NDVI4	0.04	120.68

), and were significantly lower than those of RNDVI and DN − 20 (Table 1). This demonstrates statistically that RNDVI is probably a much better indicator for correcting the saturation of nighttime lights data.

However, there are some other things we should also pay attention to. As shown in Figure 8, when RNDVI is below −0.4, the distribution of scatters seems quite discrete and with strong uncertainty, although its trend still fits the quadratic curve generally. This indicates that RNDVI, and therefore NDVI, might also suffer a “saturation problem” when it is low enough (corresponding roughly to the pixels where the DN of the nighttime lights is saturated the most severely). Therefore, the NDVI might no longer be effective for correcting light saturation when RNDVI is below −0.4. However, this problem was ignored in our study to reduce the complexity.

4. Saturation Correction Results and Evaluation

In light of the conclusion of Section 3, the quadratic function was adopted in the regression between the DN and RNDVI, and the formula is as follows:

$$\mathrm{DN}-20=1793.04\times {\mathrm{RNDVI}}^2.$$

(3)

The result is significant at the 0.01 level with an R² of 0.48. Using the Raster Calculator Tool in ArcGIS 10.2, the corrected DNs of the saturated pixels were obtained using the RNDVI image and Equation (3). If the corrected DN was lower than 55, it remained unchanged. The corrected pixels were integrated into the final corrected image with the uncorrected pixels. Figure 9 shows the final corrected nighttime lights image along with the 2006 SNL and RCNL images in China.

A total of 31,467 DNs were corrected, with the maximum DN value increased from the original 63 to 877.104. We selected the Beijing–Tianjin and Xi’an–Xian’yang regions as two examples with which to demonstrate our correction results (Figure 9). These are typical urban areas located in the north and west of China, respectively. As shown in Figure 9, the corrected image resembles the RCNL image more in terms of presenting variations and spatial details, which means that the degree of saturation of DNs in the original images of urban centers has been significantly reduced.

Three assessments were performed to evaluate the correction results quantitatively. First, we visually compared the corrected DN, the DN of the 2006 SNL and RCNL images from transects in the typical regions. Two groups of straight lines were added on Figure 9, one of which passed through the Beijing–Tianjin central area (40.26°N, 116.04°E) to (38.78°N, 117.57°E); the other passed through the Xi’an–Xian’yang central area (34.19°N, 109.18°E) to (34.40°N, 108.51°E). We then extracted the three groups of DN values from the corrected, 2006 SNL, and RCNL images, respectively, by the lines. Every group of DN values was exported, assigned a serial number (No.), and plotted on a curve (Figure 10 and Figure 11). The figures show that the DN values in urban centers have been significantly improved in the corrected image. The fine spatial details are successfully described, in contrast to their condition in the original images, and they are very similar to those in the calibrated images.

Next, we conducted two regressions with DN > 0. One is between the 2006 SNL and RCNL images and the other is between the corrected and 2006 RCNL images.

Table 3. Results of the regression between the DNs of the 2006 RCNL image and those of the other images.

	2006 SNL Image	Corrected Image
R2	0.53	0.65
RMSE	30.53	26.39

shows the R² and RMSE of these two regressions, illustrating that the DNs after correction were much more relevant linearly to those of the 2006 RCNL image (R² 0. 65 vs. 0. 53 and RMSE 26.39 vs. 30.53).

In addition to the three images, we also tried to test the results using other data. In earlier studies, it was found that a high correlation exists between GDP and the accumulative DN of the unsaturated nighttime lights in a region [16]. Two regressions were also performed for provinces in China between GDP and the sums of DNs of both the corrected data and the original data. The correlation coefficient was 0.8461 for the original data, and 0.8626 for the corrected data. This also indicates that our correction is meaningful.

All of the above tests suggest that the method proposed in this study to utilize RNDVI to correct saturation for the nighttime lights image is basically effective and successful, although it only offers limited improvements. Most of the corrected DNs, however, are still smaller than they should be, especially in the core urban area (Figure 10 and Figure 11). This might have occurred for three major reasons. First, defects in the NDVI data could be to blame, as was mentioned in Section 3. RNDVI, and therefore NDVI, may suffer “saturation,” especially when lower than −0.4 (where DNs suffer the most severe saturation). Thus, it loses its authenticity and validity in these areas. The second major reason might be that not all samples follow the uniform relationship, as presumed, and not all places conform to the pattern of brighter lights along with lower NDVI. In contrast, urbanization in some areas might result in more vegetation cover [18]. The third reason could be that the slightly saturated samples (50 < DN ≤ 55 for F162006) have not been corrected at all.

5. Discussion and Conclusions

There is a growing need for more detailed and accurate characterization of urban areas at global or national scales. Nighttime lights data obtained by DMSP/OLS are some of the data most widely used to depict urban characteristics. However, the light saturation problem in urban cores has hindered its application. Many scholars have endeavored to alleviate it by developing a lot of different methods. For DMSP/OLS images, this study concludes, pixels might start to suffer saturation when the DN > 50, or at most 55, and the saturation is more serious when the DN is larger. To be prudent, we classified all pixels into three categories: unsaturated pixels (DN ≤ 50), slightly saturated pixels (50 < DN ≤ 55) and severely saturated pixels (DN > 55). Considering that any correction may introduce new errors, we could not guarantee that the amelioration by correction would cover the error it might introduce to the middle range, so we decided not to correct the slightly saturated pixels in this study.

RNDVI, derived by subtracting the interpolated NDVI from the real NDVI, was utilized to correct the saturation of nighttime lights images. RNDVI is significantly better at reflecting the degree of human activity compared to the real NDVI because it eliminates background noise. DN = 20 was set as the threshold to distinguish urban and non-urban areas, with the intention to have enough samples in the two categories for significance of regressions and low interpolation biases. Generally, quadratic functions described well the relationship between the intensity of nighttime lights and RNDVI. All of the evaluations performed proved that the proposed saturation correction method was effective at abating the saturation problem. Nevertheless, we should also note that the correction is not yet sufficient in core urban areas, which might have three causes: the “saturation” of RNDVI, statistical errors, and ignorance about slightly saturated pixels.

In terms of the application of our work, it was mainly focused on demonstration of a new saturation correction method for nighttime lights data from DMSP/OLS and could be extended to other regions and other years. Both DMSP/OLS nighttime lights data and SPOT-VGT NDVI data are globally available. Therefore, it is only necessary to undergo the procedure described in Figure 5 when applying the method to other regions. Applying the method directly at a global scale might obtain a statistically significant result but is not recommended. The reason is that not all places conform to the pattern of brighter lights along with lower NDVI. Therefore, application at national or regional scale is better. Temporally, the method could be applied to other adjacent years with SNL images but without RCNL images, under the assumption that the relationship between DN − 20 and RNDVI does not change with years. However, caution should be observed because the World Stable Lights dataset lacks comparability. SNL images from other years should be adjusted to the 2006 SNL image through inter-calibration prior to saturation correction. The invariant region method is one of the most widely used methods for inter-calibration and thus could be utilized. As a benefit from the preliminary inter-calibration, the final saturation corrected images of different years would surely be comparable. Because we will utilize different NDVI data for different years, it is not difficult to understand that the final saturation-corrected images will also present different light intensity gradients in the same saturated areas, which is an advantage over previous methods based on RCNL images only.

Numerous studies have used NDVI or relevant indicators to monitor land cover dynamics at the global or national scales. RNDVI, as proposed and used in this study, can more directly characterize the impact of human activities on vegetation when compared with the real NDVI. It is suggested that vegetation density is negatively related to temperature in certain areas [32]. However, these relationships vary a lot in different regions with various natural conditions. Relative vegetation indexes may provide the potential for achieving a unified framework for field studies. Furthermore, in the domain of remote sensing, saturation problems are not rare because every man-made sensor has some detection deficiency. No man-made sensor has perfect spatial, temporal, and radiant resolution. For example, particles with aerodynamic diameters of < 2.5 μm, also called PM_2.5, tend to suffer a similar problem at high densities [33]. Sometimes the problem may be severe enough to impede the application of the data, in which case it seems wise to utilize other data relevant to the saturated data to help correct them. This kind of practice is meaningful, although it is unlikely to be perfect.