DMSP-OLS Radiance Calibrated Nighttime Lights Time Series with Intercalibration

Abstract

The Defense Meteorological Satellite Program-Operational Linescan System (DMSP-OLS) stable lights products are made using operational OLS data collected at high gain settings, resulting in sensor saturation on brightly lit areas, such as city centers. This has been a paramount shortcoming of the DMSP-OLS stable lights time series. This study outlines a methodology that greatly expands the dynamic range of the OLS data using observations made at different fixed-gain settings, and by incorporating the areas not affected by saturation from the stable lights product. The radiances for the fixed-gain data are computed based on each OLS sensor’s pre-flight calibration. The result is a product known as the OLS radiance calibrated nighttime lights. A total of eight global datasets have been produced, representing years from 1996 to 2010. To further facilitate the usefulness of these data for time-series analyses, corrections have been made to counter the sensitivity differences of the sensors, and coefficients are provided to adjust the datasets to allow inter-comparison.

1. Introduction

Nighttime lights are unique among global remote sensing data products for their high correlation to human activities. They have been used in studies regarding population [1], power consumption [2], GDP [3,4], mapping city boundaries [5,6], gas flaring volume [7], CO₂ emissions [8], material stocks [9], etc.

Elvidge et al proposed a systematic workflow on making nighttime lights composites from Defense Meteorological Satellite Program (DMSP) Operational Linescan System (OLS) observations [10]. Since 1994 National Geophysical Data Center (NGDC) has been producing annual global cloud-free nighttime lights datasets, named OLS Stable Lights products. Automatic algorithms have been developed for screening the quality of nighttime visible band observations to remove areas with undesirable properties, such as contamination by sunlight or moonlight, presence of clouds, lights from ephemeral sources, and background noise. Currently OLS Stable Lights products from 1992 to 2012 are distributed on the website of NGDC [11].

DMSP flies on a sun-synchronized polar orbit, with local overpass times at descending node and ascending node roughly at 0830 and 2030, respectively. DMSP-OLS is an oscillating scan device with two bands (visible (VIS) and thermal-infrared (TIR)) designed to map clouds in both day and night. With a swath of 3000 km, it is able to provide daily global coverage. The visible band uses a photomultiplier tube (PMT) to collect photons. The gain setting of the OLS can be changed in a wide range, making it able to adapt to scenes both in daytime and nighttime. The gain is adjusted automatically according to the predicted brightness in the scene. The result has 6-bit quantization, with digital numbers (DN) ranging from 0–63 and limited dynamic range. For nighttime overpasses, the gain setting is usually set to the highest level to enable observation of moonlit clouds. Therefore, pixels in bright areas such as city centers often reaches a DN of 63, leaving no further details to be recognized. Such pixels are termed to be “saturated”.

To overcome this problem, Elvidge et al proposed a “radiance calibration” approach [12]. This method requires the collection of DMSP-OLS VIS band at three different fixed-gain levels: low, medium and high. In fixed-gain mode the radiances can be calculated based on the preflight sensor calibration. The strategy is to operate the OLS in 24 h increments under each gain setting, across the set of nights to either side of the new moon. The lowest gain setting is set to avoid saturation on the brightest lights present on the Earth’s surface. The upper range of the medium gain setting overlaps the lower end of radiances observable at the low gain setting. Similarly, the upper range of the high gain setting overlaps the lower end of radiances observable at the medium gain setting. The objective is to collect unsaturated data on all detectable lights, greatly expanding the dynamic range of the final cloud-free composite. Elvidge et al defined the optimized fixed-gain setting as 15 dB, 35 dB, and 55 dB (decibels, which is the log of amplitude) [13]. These setting were adopted for most of the acquisitions of the fixed-gain data since 1999 as shown in Table 1. This technique resembles the HDR (High Dynamic Range imaging) in digital photography. This study created a composite over the U.S. using a set of fixed-gain observation made mainly in 1997.

This method was further developed in Ziskin et al [13] adding in a ramped weighting scheme to smooth out the discontinuities in the gain setting switch zones. In addition, to extend the product to even lower brightness levels and to augment the limited number of fixed-gain data collections, the stable lights product for the corresponding year is blended in. Ziskin et al created a 2006 global composite using fixed-gain observation plus stable lights.

This study outlines the production of a time series of Radiance Calibrated global nighttime lights products, spanning all the years where suitable fixed-gain data collections were made. In addition, we performed an intercalibration of the products.

2. Methodology

The generation of radiance calibrated nighttime lights products can be divided into three steps. First, fixed-gain data need to be collected and processed into global fixed-gain cloud-free composite data for three gain settings (low, medium and high). Second, cloud-free composite data from the three gain settings are merged. Last, the stable lights product is blended in with the merged global fixed-gain composite. These three steps will be elaborated in the following sections.

2.1. Preparation of Fixed-Gain Imagery Collection

2.1.1. Instrument Gain

The OLS has an analog amplifier and post-amplifier with constant gains as well as VDGA (Variable Digital Gain Amplifier) gain. The gain of the PMT also contributes to the overall system gain for night scenes. Primary control of the fixed-gain setting is achieved through ground command of the VDGA, which has values from 0 to 63 dB. The term “gain setting” used throughout this study refers to the VDGA gain.

The OLS is calibrated prior to the launch, so called preflight calibration, by characterizing the DN to radiance for all VDGA gain steps. The relationship of the radiance and DN can be described with the following equation.

$$\frac{DN}{R}=\frac{D{N}_{Max}}{R_{Sat}(G)}$$

(1)

where DN_Max is 63 in 6-bit quantization, R_Sat(G) (W/cm²/sr) is the saturation radiance of the instrument at gain setting G (dB), which is measured in preflight calibration. For example, the R_Sat of OLS on satellite F16 can be represented as below and in Figure 1.

$$R_{Sat}\left(G\right)=\frac{R_0}{{10}^{(\frac{G}{20})}}$$

(2)

where R₀ is the saturation radiance at zero VDGA gain, and is measured 5.3E-6 (W/cm²/sr) for satellite F16.

Most of the fixed-gain data are collected under gain settings fixed at 15/35/55 dB. At these gain settings, the represented radiances at the same DN is 100/10/1 times to that at 55 dB.

The primary objective of the on-orbit gain control with VDGA is to generate imagery of clouds with consistent contrast. Under operational situations, the VDGA gain is adjusted according to prediction of solar and lunar illumination every 0.4 ms by an onboard along scan gain control (ASGC) algorithm. A bidirectional reflectance distribution function (BRDF) algorithm further adjusts the gain in the scan segment where the illumination angle equals the observation angle. However, the gain setting used is not recorded in the data stream. It can only be partially calculated backward from the relative solar and lunar positions, and constants stored onboard.

Figure 1. The relationship between gain setting and minimum/saturation radiance for satellite F16. Note the DN values in the selected VDGA gains correspond to radiances that differ by factors of 10 [14].

Figure 1. The relationship between gain setting and minimum/saturation radiance for satellite F16. Note the DN values in the selected VDGA gains correspond to radiances that differ by factors of 10 [14].

During the low-moon segments of the lunar cycle when the data is less useful for USAF operations, the VDGA gain setting can be fixed to a specific value by request. Usually when the DMSP satellite is approaching the end of its designed lifetime, a new satellite will be placed in the orbit with the old satellite operating as a backup. The authors kept requesting fixed-gain operations with the old satellite to avoid interruption on the primary mission of DMSP whenever possible. As a result, a total of seven annual observation sets were collected with gain settings shown in Table 1.

Table 1. List of Radiance Calibrated products and source datasets used.

Representative Satellite/Year	Fixed-Gain	Satellite/Period of Observation
F12_1997	24	F12/19960316–19960324
		F12/19970105–19970212
	40	F12/19960316–19960324
	50 (b)	F12/19970105–19970212
	STB*	F12/1997
F12_1999	15	F12/19990119–19991211
	35	F12/19990119–19991211
	55 (b)	F12/19990119–19991211
	STB	F14/1999
F12_2000	15	F12/20000103–20001229
		F15/20000303–20000508
	35	F12/20000103–20001229
		F15/20000303–20000508
	55 (b)	F12/20000103–20001229
		F15/20000303–20000508
	STB	F15/2000
F14_2003	15	F14/20021230–20031127
		F15/20021230–20030205
	35	F14/20021230–20031127
		F15/20021230–20030205
	55 (b)	F14/20021230–20031127
		F15/20021230–20030205
	STB	F15/2003
F14_2004	15	F14/20040118–20041216
	35	F14/20040118–20041216
	55 (b)	F14/20040118–20041216
	STB	F15/2004
F16_2006	15	F16/20051128–20061224
	35	F16/20051128–20061224
	55 (b)	F16/20051128–20061224
	STB	F16/2006
F16_2010	15	F16/20100111–20101209
		F16/20110129–20110731
	35	F16/20100111–20101209
		F16/20110129–20110731
	50 (b)	F16/20100111–20101209
		F16/20110129–20110731
	STB	F18/2010

* Blended Stable Lights Product; (b): Base Gain, other fixed-gain data are calibrated to match this gain setting by applying associating multipliers.

Table 1. List of Radiance Calibrated products and source datasets used.

Representative Satellite/Year	Fixed-Gain	Satellite/Period of Observation
F12_1997	24	F12/19960316–19960324
		F12/19970105–19970212
	40	F12/19960316–19960324
	50 (b)	F12/19970105–19970212
	STB*	F12/1997
F12_1999	15	F12/19990119–19991211
	35	F12/19990119–19991211
	55 (b)	F12/19990119–19991211
	STB	F14/1999
F12_2000	15	F12/20000103–20001229
		F15/20000303–20000508
	35	F12/20000103–20001229
		F15/20000303–20000508
	55 (b)	F12/20000103–20001229
		F15/20000303–20000508
	STB	F15/2000
F14_2003	15	F14/20021230–20031127
		F15/20021230–20030205
	35	F14/20021230–20031127
		F15/20021230–20030205
	55 (b)	F14/20021230–20031127
		F15/20021230–20030205
	STB	F15/2003
F14_2004	15	F14/20040118–20041216
	35	F14/20040118–20041216
	55 (b)	F14/20040118–20041216
	STB	F15/2004
F16_2006	15	F16/20051128–20061224
	35	F16/20051128–20061224
	55 (b)	F16/20051128–20061224
	STB	F16/2006
F16_2010	15	F16/20100111–20101209
		F16/20110129–20110731
	35	F16/20100111–20101209
		F16/20110129–20110731
	50 (b)	F16/20100111–20101209
		F16/20110129–20110731
	STB	F18/2010

* Blended Stable Lights Product; (b): Base Gain, other fixed-gain data are calibrated to match this gain setting by applying associating multipliers.

2.1.2. Data Processing

Global composites of retrieved orbit data at each fixed-gain setting are made with methodology that is essentially the same as what is carried out for the Stable Light products described in Baugh et al [15].

Each input OLS orbit is comprised of VIS and TIR data of the same area. Flag images are then calculated for the orbit to identify the properties of each pixel. These properties are DAYTIME, NIGHTTIME MARGINAL, ZERO LUNAR ILLUMINANCE, CLOUDS PRESENT, and NO DATA (Table 2). Flag images have dimensions the same as OLS orbit data with values of 16-bit integers and are processed bit-wise. Therefore each pixel can belong to more than one category by turning specific bits on. The flag data along with VIS and TIR data are further constrained between latitudes of 75°N and 65°S. They are then reprojected into 30 arc-second grids, which is about 1 km near the equator.

The composite only takes parts of images which have best quality. In other words, only pixels with DAYTIME and NIGHTTIME MARGINAL flags off, ZERO LUNAR ILLUMINANCE on, CLOUDS PRESENT off, and NO DATA off will be included in the composite. The compositing process takes both visible data and their companion flag images to produces a suite of output files. The cloud-free coverage (CF_CVG) image counts the number of pixels that made their way into the composite, and summation of DN (SUM_DN) image shows the summed DN of those pixels. Dividing SUM_DN with CF_CVG makes average DN image (AVG_DN). During the compositing process, images are further constrained to take only pixels near the center of swath to attain the highest quality pixels.

Table 2. Types of OLS flags and their meaning.

Flag Name	Indication	Prefer State
DAYTIME	Area of projected pixel is in daytime.	OFF
NIGHTTIME MARGINAL	Area of projected pixel is in transition zone between daytime and nighttime.	OFF
ZERO LUNAR ILLUMINANCE	Area of projected pixel is not affected by lunar illumination.	ON
CLOUDS PRESENT	Area of projected pixel is covered by clouds.	OFF
NO DATA	The pixel does not contain valid data in OLS imagery.	OFF

Table 2. Types of OLS flags and their meaning.

Flag Name	Indication	Prefer State
DAYTIME	Area of projected pixel is in daytime.	OFF
NIGHTTIME MARGINAL	Area of projected pixel is in transition zone between daytime and nighttime.	OFF
ZERO LUNAR ILLUMINANCE	Area of projected pixel is not affected by lunar illumination.	ON
CLOUDS PRESENT	Area of projected pixel is covered by clouds.	OFF
NO DATA	The pixel does not contain valid data in OLS imagery.	OFF

2.2. Merge Fixed-Gain Data

2.2.1. Pre-Processing

The overall processing flow for merging fixed-gain images is shown in Figure 2. The inputs are: CF_CVG and SUM_DN data for low (L), medium (M), and high (H) fixed-gain settings.

For each fixed-gain setting, a pair of values depicting lower and higher thresholds in DN to be included is provided (DN_RANGE). Also provided are multipliers from preflight calibration results that convert DN from each gain setting to its equivalent DN (DN_EQ) at the selected base-gain (usually 55 dB). The multipliers can be calculated by the following equation, which is derived from Equation (2).

$$\text{M}\left(G_1,G_2\right)={10}^{0.05(G_2-G_1)}$$

(3)

where M(G₁,G₂) is the multiplier that calibrates DN from gain setting G₁ to its DN_EQ at gain setting G₂. For example, the multiplier for fixed-gain at 15 dB will be 100 because it detects radiance 100 times lower than at 55 dB (see Figure 1).

Figure 2. Overall flow of merging fixed-gain data focusing on pre-processing.

Figure 2. Overall flow of merging fixed-gain data focusing on pre-processing.

In the beginning of the processing, the CF_CVG and AVG_DN data are filtered to exclude pixels in AVG_DN data that are outside of DN_RANGE for their fixed-gain settings. CF_CVG data are then summed up for pixels with non-zero SUM_DN, which will become one of the final outputs from the merging of fixed-gain data.

The filtered AVG_DN can then be applied with the multipliers respectively to produce filtered average equivalent DN (AVG_DN_EQ) imagery. Likewise the DN_RANGE can also be converted into base-gain equivalent DN thresholds (DN_EQ_RANGE). The filtered AVG_DN_EQ and DN_EQ_RANGE are then fed into the actual merging process.

2.2.2. Merge Process

As described in Ziskin et al., weighting is required in the merging process to compensate for the spatial discontinuity caused by filtration with DN_RANGE [13]. The weight is calculated following the principle that a pixel in fixed-gain imagery with DN closer to higher bound of DN_RANGE will be given smaller weighting to lessen its effect as it is approaching the point of discontinuity and saturation, while raising the weighting of the other fixed-gain setting to cover the deficiency. In other words, the weighting scheme ensures a smooth transition from one fixed-gain setting to another.

Figure 3 shows an example of weighting calculation for medium fixed-gain imagery. Weighting factors for other fixed-gain settings are calculated in a similar manner. Given an AVG_DN_EQ of a certain fixed-gain setting, the first step is to check if the AVG_DN_EQ also falls in the DN_EQ_RANGE of the other two fixed-gain settings. If the answer is positive, a weighting factor against that fixed-gain setting needs to be calculated.

For a certain pixel within the fixed-gain images, if the AVG_DN_EQ of medium fixed-gain imagery $AVG\_DN\_E{Q}_M$lies within the DN_EQ_RANGE of high fixed-gain setting, then the weighting factor for medium fixed-gain in respect to low fixed-gain setting $W_M\left(L\right)$ will be calculated as Equation (4):

$$\begin{array}{c}{W}_M\left(L\right)=\frac{\left(DN\_EQ\_RANG{E}_{M,upper}-AVG\_DN\_E{Q}_M\right)}{\left(DN\_EQ\_RANG{E}_{M,upper}-DN\_EQ\_RANG{E}_{M,lower}\right)}\end{array}$$

(4)

Likewise, if $AVG\_DN\_E{Q}_M$lies within the DN_EQ_RANGE of low fixed-gain setting, then the weighting factor for medium fixed-gain in respect to high fixed-gain setting $W_M\left(H\right)$ will be calculated as Equation (5).

$$W_M\left(H\right)=\frac{\left(AVG\_DN\_E{Q}_M-DN\_EQ\_RANG{E}_{M,lower}\right)}{\left(DN\_EQ\_RANG{E}_{H,upper}-DN\_EQ\_RANG{E}_{M,lower}\right)}$$

(5)

In Equations (4) and (5), DN_EQ_RANGE is described as $DN\_EQ\_RANG{E}_{i,j}$, where symbol i denotes fixed-gain settings (L, M, H) and j denotes for upper or lower DN_EQ_RANGE setting. The weighting factors in respect to other fixed-gain settings are then averaged and set to zero if it became negative as described in Equations (6) and (7). This yields the actual weighting factor $W_M$ for medium fixed-gain setting.

$$W_M=\{\begin{array}{cc}{W}_M\left(L\right)& if only W_M\left(L\right) is calculated\\ W_M\left(H\right) & if only W_M\left(H\right) is calculated\\ \left[W_M\left(L\right)+W_M\left(H\right)\right]/2& \end{array}$$

(6)

$$W_M=\{\begin{array}{cc}0& if W_M<0\\ W_M & \end{array}$$

(7)

The merged fixed-gain sum of adjusted DN (SUM_DN_EQ) is then calculated as below

$$SUM\_DN\_EQ={{\displaystyle \sum }}^{\text{}}{W}_i\times N_i\times AVG\_DN\_E{Q}_i i=L,M,H$$

(8)

where $SUM\_DN\_EQ$ is the merged fixed-gain SUM_DN_EQ, $W_i$, $N_i$, and $AVG\_DN\_E{Q}_i$ are weighting factor, cloud-free coverage counts, and AVG_DN_EQ for each fixed-gain settings respectively.

Figure 3. Flow of weighting factor generation and fixed-gain imagery merging.

Figure 3. Flow of weighting factor generation and fixed-gain imagery merging.

To better elucidate the idea of calculating weighting factors, Figure 4 shows the merging process for a transect over Los Angeles, the United States. Note in Figure 4c that as the weighting factor drops when AVG_DN_EQ approaches saturation, the weighting factor of the neighboring fixed-gain setting raises to cover the deficiency.

Figure 4. How weighting factors are calculated and applied in merging fixed-gain data. (a) Fixed-gain data of Los Angeles, U.S. with selected transect over city center (upper row: AVG_DN, lower row: CF_CVG). (b) (Upper) AVG_DN_EQ and AVG_DN of fixed-gain data over selected transect (Lower) Cloud-free coverage counts of fixed-gain data over selected transect. (c) Weighting factor of fixed-gain data over selected transect. (d) Sum of weighted fixed-gain DN_EQ over selected transect. (e) Merged fixed-gain imagery over Los Angeles area.

Figure 4. How weighting factors are calculated and applied in merging fixed-gain data. (a) Fixed-gain data of Los Angeles, U.S. with selected transect over city center (upper row: AVG_DN, lower row: CF_CVG). (b) (Upper) AVG_DN_EQ and AVG_DN of fixed-gain data over selected transect (Lower) Cloud-free coverage counts of fixed-gain data over selected transect. (c) Weighting factor of fixed-gain data over selected transect. (d) Sum of weighted fixed-gain DN_EQ over selected transect. (e) Merged fixed-gain imagery over Los Angeles area.

2.3. Blend Stable Lights Product

Due to limited availability of DMSP-OLS data with fixed-gain settings, the number of orbits collected at each fixed-gain setting is usually about ten times less than those collected in normal operations [13]. Therefore, Stable Lights products are blended into the merged fixed-gain imagery as another high-gain-setting dataset to improve coverage.

Stable Lights products are nighttime lights datasets with fires and other ephemeral lights removed based on their high brightness and short duration. Background noise is also removed by setting thresholds based on visible band values found in areas known to be free of detectable lights. The global composite is restricted within 75°N and 65°S, and gridded into a global latitude-longitude grid (Platte Carrée projection) with a resolution of 30 arc seconds. The grid size is approximately a square kilometer at the equator. In 2010, NGDC released the version 4 time series of Stable Lights, which currently spans the years 1992–2012 [11,15].

However, due to the on-the-fly gain adjustment of operational DMSP-OLS observations and the lack of a record of OLS gain settings, no feasible multiplier is available to convert the Stable Lights product DNs to match the base-gain of the merged fixed-gain imagery. To obtain a proxy between the Stable Lights product and merged fixed-gain imagery, linear regression was performed with AVG_DN_EQ and AVG_DN of a subset of pixels collected from merged fixed-gain imagery and the corresponding Stable Lights product respectively. The collection was made to include pixels with values within DN_RANGE specified for the Stable Lights product. Since the merged fixed-gain imagery is calibrated to the highest gain as the base-gain, which is comparable to the average gain settings in operational observations, DNs below saturation of the Stable Lights product and DN_EQs of merged fixed-gain imagery can be directly compared to each other. Therefore, the general linear regression is carried out in this analysis. Additional rules are applied when selecting pixels to be included, which are (1) the AVG_DN_EQ in merged fixed-gain imagery have to be larger than 0; and (2) AVG_DN_EQ has to be lower than four times of the AVG_DN from the corresponding pixel in Stable Lights product. These additional rules are set to avoid absurd samples and potential outliers caused by noise being included in the regression.

The result of the linear regression is applied to the Stable Lights product and input into the blending process. Since the Stable Lights product is restricted with its own DN_RANGE that results in discontinuity as well, a weighting scheme similar to merging fixed-gain data is also applied to ensure a smooth blending. Finally, in the blending of the Stable Lights Product with the merged fixed-gain data, there is a merged fixed-gain pixel set that was excluded from the blending process. These are pixels that had very low AVG_DN values in the Stable Lights product, but considerably higher values in the fixed-gain AVG_DN_EQ. After investigation it was found these were bright pixels from fires that elevated the merged fixed-gain values, but had been filtered from the Stable Lights product. Such event is most influential in rural areas. Thus, this merged fixed-gain exclusion set is defined by three criteria: (1) the pixels have to be on land; (2) the merged fixed-gain AVG_DN_EQ is 10 times larger than the Stable Lights AVG_DN; and (3) the Stable Lights AVG_DN is less than the threshold set for rural lights.

The overall flow of blending the Stable Lights product is shown in Figure 5 with the blending process enlarged in Figure 6. Table 1 shows the satellite/year of the Stable Lights product used to produce each Radiance Calibrated product.

The comparison of the Radiance Calibrated product with the corresponding Stable Light product is shown in Figure 7. The Tokyo Metropolitan area is taken as an example for being one of the world’s brightest areas. From the transect over the center of Tokyo Metropolitan, it is clear how much detail was lost in the Stable Light product which is now restored in the Radiance Calibrated product.

Figure 5. Flow of Stable Lights product blending with merged fixed-gain imagery.

Figure 5. Flow of Stable Lights product blending with merged fixed-gain imagery.

Figure 6. Detailed flow of merged fixed-gain imagery blending with Stable Lights product.

Figure 6. Detailed flow of merged fixed-gain imagery blending with Stable Lights product.

Figure 7. Comparison of Stable Light product (F16_2006) and Radiance Calibrated product (F16_20051128–20061224) over Tokyo Metropolitan. (a) Stable Light product of Tokyo Metropolitan; (b) Radiance Calibrated product of Tokyo Metropolitan; (c) Transect of Stable Light product over the center of Tokyo Metropolitan; (d) Transect of Radiance Calibrated product over the center of Tokyo Metropolitan; (e) Overlay of transects from Stable Light product (green) and Radiance Calibrated product (red).

Figure 7. Comparison of Stable Light product (F16_2006) and Radiance Calibrated product (F16_20051128–20061224) over Tokyo Metropolitan. (a) Stable Light product of Tokyo Metropolitan; (b) Radiance Calibrated product of Tokyo Metropolitan; (c) Transect of Stable Light product over the center of Tokyo Metropolitan; (d) Transect of Radiance Calibrated product over the center of Tokyo Metropolitan; (e) Overlay of transects from Stable Light product (green) and Radiance Calibrated product (red).

2.4. Intercalibration

With the release of the multi-year Radiance Calibrated product time-series, it is anticipated that there will be high interest in comparing the Radiance Calibrated products from different years. However, due to the lack of on-board calibration, DN values cannot be converted to exact radiance. To enable intercomparison of these annual products, an empirical intercalibration was performed. This effort comes in two steps. First, the inter-satellite instrument difference is addressed. Second, the inter-annual difference is addressed to counter the bias generated by sensor degradation and other sources of bias.

2.4.1. Inter-Satellite Calibration

The inter-satellite calibration adjusts the DN in Radiance Calibrated product to correct the difference caused by sensitivity discrepancies between instruments on different satellites. The Radiance Calibrated products are made based on the highest fixed-gain setting available, called base gain in this study. In other words, the DN values in the Radiance Calibrated products (DN_EQ) can be deemed as taken from the reading of sensors set at the base gain with very large dynamic range. By applying multipliers shown in Table 3 to corresponding Radiance Calibrated products following their satellite and base gain, the adjusted DN values are now theoretically equivalent to the OLS onboard the F16 at gain setting of 55 dB. The multipliers in Table 3 are calculated from the radiance of DN 1 at the base gain of each satellite’s OLS. The data is taken from the preflight calibration report for each satellite [14].

Table 3. Inter-satellite Calibration Multiplier.

Satellite	Base Gain (dB)	Multiplier	Radiance @ DN1 (W/cm2/sr)
F12	55	0.96	1.44E-10
F14	55	0.82	1.23E-10
F15	55	0.90	1.35E-10
F16	55	1.00	1.50E-10

Table 3. Inter-satellite Calibration Multiplier.

Satellite	Base Gain (dB)	Multiplier	Radiance @ DN1 (W/cm2/sr)
F12	55	0.96	1.44E-10
F14	55	0.82	1.23E-10
F15	55	0.90	1.35E-10
F16	55	1.00	1.50E-10

Most of the Radiance Calibrated Products are made from data collected using a single satellite. For the products using multiple satellites (see Table 1), the inter-satellite calibration is carried out before the merging process.

2.4.2. Inter-Annual Calibration

Biases other than instrument differences are addressed by performing inter-annual calibration to the Radiance Calibrated product time series. A similar effort was demonstrated in Elvidge et al. for comparison between Stable Lights products from 1994 to 2008 [7]. The authors chose the island of Sicily, Italy as reference, and assumed that the sum of lights there had not changed in the time period of interest. Since the dynamic range of the Stable Lights product only ranges from DN 0 to 63, Sicily Island was sufficient to provide samples within this dynamic range. A second order polynomial regression was used to address the curvature in correlation when the DN values approached saturation in the Stable Lights products.

Being freed from the saturation problem, Radiance Calibrated products need a reference area which not only is stable enough but also provides samples with brightness ranging from very low light to the world’s highest levels. Los Angeles was taken as the reference for the Radiance Calibrated products for two reasons. First, Los Angeles has long been a mature metropolis and the light change is negligible. Second, being a metropolis, it can provide samples with high DNs from the city center, as well as low DNs from the suburban area.

Furthermore, in the previous study, the Stable Lights product of F121999 was selected as the reference year for it gives the most saturated pixels from the reference area among all annual composites. It was chosen because there is no way to correct the DN of an already saturated pixel for it already lost its ability to represent the actual brightness. However, there is no such restriction when it comes to selecting the reference year among annual Radiance Calibrated products. Therefore, the annual composite for 2006 is selected as it gives a fair cloud-free coverage over the world.

The result of linear regression for intercalibrating the Radiance Calibrated products is shown in Table 4 and Figure 8.

Table 4. Inter-annual Calibration Result.

Satellite_Year	Slope (a)	Intercept (b)	R2	Number of Pixels *
F12_19960316–19970212	0.915	4.336	0.971	20,540
F12_19990119–19991211	0.780	1.423	0.980	20,846
F12-F15_20000103–20001229	0.710	3.658	0.980	20,866
F14-F15_20021230–20031127	0.797	3.736	0.980	20,733
F14_20040118–20041216	0.761	1.062	0.984	20,848
F16_20051128–20061224	1.000	0.000	1.000	21,044
F16_20100111–20101209	1.195	2.196	0.981	20,848
F16_20100111–20110731	1.246	−1.987	0.981	20,848

* Number of pixels included in the regression analysis.

Table 4. Inter-annual Calibration Result.

Satellite_Year	Slope (a)	Intercept (b)	R2	Number of Pixels *
F12_19960316–19970212	0.915	4.336	0.971	20,540
F12_19990119–19991211	0.780	1.423	0.980	20,846
F12-F15_20000103–20001229	0.710	3.658	0.980	20,866
F14-F15_20021230–20031127	0.797	3.736	0.980	20,733
F14_20040118–20041216	0.761	1.062	0.984	20,848
F16_20051128–20061224	1.000	0.000	1.000	21,044
F16_20100111–20101209	1.195	2.196	0.981	20,848
F16_20100111–20110731	1.246	−1.987	0.981	20,848

* Number of pixels included in the regression analysis.

Figure 8. Scatter plots showing the different annual Radiance Calibrated products compared to the reference year.

Figure 8. Scatter plots showing the different annual Radiance Calibrated products compared to the reference year.

2.4.3. Verifying the Effect of Inter-Annual Calibration

To verify the effect of inter-annual calibration, Elvidge et al took advantage of overlapping satellite-years of Stable Lights products to show that after inter-annual calibration the discrepancy between the satellite-years had been suppressed [7]. However, there is no such luxury of overlapping satellite-years for the Radiance Calibrated products. Therefore, a different approach has been developed.

Given that the Radiance Calibrated products are free from saturation even in the brightest part of cities, it is assumed that large cities in developed countries should behave stably over time. For this study, 12 cities are chosen: Tokyo, Bangkok, Moscow, Berlin, Paris, Madrid, Beijing, New York, Mexico City, New Delhi, Taipei, and London. An 11 × 11 pixel box, with the center located at the brightest pixel in each city, was used as the sampling area. The coordinates of the selected centers and the results are shown in Figure 9.

Figure 9. List of city samples, center coordinates, and sum of lights vs. years.The red line indicates inter-annual calibrated and blue line indicates not inter-annual calibrated lights

Figure 9. List of city samples, center coordinates, and sum of lights vs. years.The red line indicates inter-annual calibrated and blue line indicates not inter-annual calibrated lights

After inter-annual calibration, the temporal change of sum of lights is generally suppressed. It is clear that before inter-annual calibration, years before the 2006 reference year are overestimated, while 2010 is underestimated. To summarize, before intercalibration, the overall sum-of-lights appears to decrease in these large cities. While, after intercalibration, the sum-of-lights trend becomes stable or even shows an increase. This is the expected result for these large well-developed cities.

3. Applications

3.1. Restriction on Converting DN to Radiance

Many users would like to relate the DN values in the Radiance Calibrated products to actual radiance. The fixed-gain image avoids the automatic gain variation by VDGA which is not recorded in the data stream. So theoretically, by applying the preflight sensor calibration, the actual radiance could be derived from the DN. Nevertheless, even under such controlled operation, there is no means to address the degradation of the equipment without an on board calibration device. Moreover, the uncertainty brought by blending Stable Light products, which were taken under variable gain settings, make it even more unlikely to derive the exact radiance from the Radiance Calibrated product. As a result, it is suggested that the Radiance Calibrated product is only suitable for analyses which do not require actual radiance. Nevertheless, the conversion factor from DN to radiance for the Radiance Calibrated products is listed in Table 3 for reference. Note all products are inter-satellite calibrated to the base gain setting of 55 dB of satellite F16.

3.2. Comparing Radiance Calibrated Product with Stable Lights Product

The time-series of Radiance Calibrated products provides an exceptional opportunity to discover trends in nighttime lights that was not perceivable by the time-series of Stable Lights product as well as a single year of Radiance Calibrated product. This leads to insights on how the lights in cities and rural areas change differently over time. Nevertheless, DN values in Radiance Calibrated products are not directly comparable with those in Stable Lights product mainly for two reasons.

First, their data are collected by different gain settings by various satellites. Stable Lights products are made with observations collected with a varying VDGA gain, while fixed-gain data used in the Radiance Calibrated products are collected with a fixed VDGA gain. Although for observations in low-moon nights used in the Stable Lights products, the VDGA gain remains close to the highest level in the base gain defined in Radiance Calibrated products, there is another factor to be considered.

Saturation in the Stable Lights product causes non-linear representation of actual radiance. Because the OLS does onboard averaging to produce its global coverage data, saturation does not happen immediately when radiance reaches the maximum level. On the contrary, as the actual radiance grows, the observed DN value fails to follow the radiance growth linearly, causing a gradual transition into a plateau of full saturation. Such effect can be seen in Figure 7e, and this behavior can be observed starting at about DN = 35. On the contrary, the Radiance Calibrated products are immune to the saturation effect. Therefore the actual radiance represented by DN values below the saturation level of the Stable Lights products (i.e., DN = 63) can differ in these two products.

That being said, qualitative comparison of the two products still reveals interesting findings. Figure 10 shows the comparison of national sum of lights of the Radiance Calibrated products before and after inter-annual calibration and the inter-annual calibrated Stable Lights product [7].

Figure 10a shows the result for Trinidad and Tobago, a country composed by two islands with its GDP largely contributed by oil export. In the 1990s the sum of lights for both Radiance Calibrated and Stable Lights are close. This indicates that most light sources are dim and do not reach the saturation level. Since year 2000, the country entered a small-scale oil boon [16]. Between years 2000 to 2004, the sum of lights from the Radiance Calibrated products starts to rise above the Stable Lights, showing growth in areas that are saturated in the Stable Lights product, while it appears that the lights from rural areas stayed the same. After year 2001, the high GDP growth rate in petroleum sector pushed its GDP as well as the sum of lights [17,18]. Note that since the sum of lights from both products grew, and that the Radiance Calibrated product departs even further from Stable Lights product. This indicates that both cities and rural areas have benefitted from the GDP boost, and areas with saturated Stable Lights in the cities also increased.

Figure 10. Comparison of the national sum of lights for the Radiance Calibrated products before/after inter-annual calibration and the inter-annual calibrated Stable Lights product, for (a) Trinidad and Tobago, (b) Rwanda and (c) The United States.

Figure 10. Comparison of the national sum of lights for the Radiance Calibrated products before/after inter-annual calibration and the inter-annual calibrated Stable Lights product, for (a) Trinidad and Tobago, (b) Rwanda and (c) The United States.

For countries with little to no areas of saturated nighttime lights, like Rwanda shown in Figure 10b, the sum of lights from the Radiance Calibrated products follows closely to that of the Stable Lights products. However, due to limited years’ availability of the Radiance Calibrated products, detailed change could not be detected as well as the Stable Lights products can with complete annual data. In extreme contrast to Rwanda, the United States, as shown in Figure 10c, has a sum of lights from the Radiance Calibrated products which is always higher than the Stable Lights products. Furthermore, although the sum of lights from the Radiance Calibrated products shows a slight drop, which might be due to the introduction of proper capped lighting, the Stable Lights sum of lights remains at nearly the same level over time. This result reveals the variation in the sum of lights which were unperceivable using the Stable Lights product, and can now be observed by the Radiance Calibrated product.

Given the ability of excavating hidden variations, it is now possible to refine the studies that utilized the sum of nighttime lights as a proxy to other human activity indicators, e.g., GDP, power usage, population, etc. For example, in Figure 10a it is shown that the Radiance Calibrated products better represent the change of GDP than Stable Lights products. Another application using Nighttime Lights is to map the change of city overtime. Due to the saturation of Stable Lights product in city centers, such study was limited to map city boundaries. For the first time it is possible to map the changes of internal city structures using the Radiance Calibrated product as shown in Figure 7.

4. Conclusion and Discussion

This study describes an improved methodology to generate global cloud-free Radiance Calibrated nighttime lights products from DMSP-OLS data, overcoming the saturation problem present in the DMSP Stable Lights product. The basic method was pioneered by Elvidge et al in the late 1990s [12]. The new method introduces several improvements over the previous published methods [12,13], including a refined ramp-blending scheme and intercalibration methodology. This study also expanded the dataset to multiple years. To facilitate time-series analysis using the Radiance Calibrated products, inter-satellite as well as inter-annual calibration is carried out to give comparability between the annual composites. By comparing sum of lights from centers of large and stable cities before and after inter-annual calibration, the results show that the inter-annual calibration suppresses the fluctuation of sum of lights over time.

Because the DMSP-OLS does not carry an on-board calibration device for the visible band, it is not possible to track how actual radiance is being converted to DN even if the auto-adjust gain setting is fixed. Instead we rely on the preflight calibration made for each OLS instrument prior to launch. Therefore, even though the Radiance Calibrated products are made from data collected under much more controlled setting than the Stable Lights products, the radiance values should still be considered relative and not absolute.

Radiance Calibrated products are not directly comparable to legacy Stable Lights products for two reasons, the way VDGA gain was set and the effect of saturation. Nevertheless, Radiance Calibrated products produced in this study can reveal changes of nighttime lights over time free from saturation. This is a great advantage over the Stable Lights products. By comparing both products over time, the story of changes between cities and rural areas can be told. Furthermore, studies carried out using the Stable Lights products are now given a chance to refine their results by recovering the lost nighttime lights in saturated areas.

Last but not least, the DMSP Radiance Calibrated nighttime lights time series will not be continued in the future because no fixed-gain imagery has been collected after 2010. The processing of DMSP-OLS data into Stable Lights products has been discontinued in lieu of using the high quality low light imaging data collections from the NASA-NOAA Visible Infrared Imaging Radiometer Suite Day Night Band (VIIRS-DNB) [19]. However, the time-series of Radiance Calibrated products provides a precious record of non-saturated nighttime lights which can be compared with products produced using VIIRS-DNB which is also free from saturation.