1. Introduction
The widespread and increasing use of satellite observations of anthropogenic night light for a variety of applications, ranging from socioeconomic to ecologic, is a testament to the unique information content of these data. Whereas the DMSP-OLS series of meteorological satellites was the only source of satellite night light data for many years, there have been more than 10 satellites with nocturnal imaging capability launched since 2000 [
1]. However, none of the non-commercial night light sensors provide both sub-hectometer spatial resolution and multispectral visible bands. Sub-hectometer spatial resolution is necessary to resolve the urban light field at the decameter characteristic scale of the urban mosaic, while multispectral (or preferably hyperspectral) imaging is necessary to distinguish the diversity of light sources used to illuminate the built environment. The importance of the color spectrum of lighting sources has been recognized for its potential impact on ecosystems [
2,
3,
4,
5,
6], astronomical light pollution [
7,
8], melatonin suppression [
9] and as an epidemiological correlate to multiple types of cancer [
10,
11].
While future night light sensors will likely incorporate multispectral low light imaging capability, there has been only one option for sub-hectometer multispectral night light imaging with extensive geographic coverage over the past two decades: astronaut photos taken from the International Space Station (ISS). At the time of writing, NASA’s Gateway to Astronaut Photography of Earth (
https://eol.jsc.nasa.gov accessed on 9 July 2022) contains over 30,000 photos of ~2500 cataloged urban lightscapes taken from the ISS. There is considerable redundancy, as many of these photos are multi-shot sequences from slightly differing view geometries because the camera’s field of view changes slightly due to spacecraft motion. In addition, many are panoramic photos of large regions containing multiple cities with insufficient resolution for many scientific applications.
Despite these limitations, a subset of the collection of astronaut photos of urban lightscapes are of sufficient spatial resolution, sharpness and exposure to be potentially useful for spectral characterization of urban lightscapes—particularly, as no other geographically extensive source of comparable imagery is available for the past two decades. Variations in location, date, view geometry, resolution and exposure impose limitations on comparative analyses, but with cross calibration it is possible to obtain a spectral characterization of the diversity of light sources contributing to urban lightscapes over the past decade. Spectral characterization of multiple urban lightscapes can provide a basis for quantifying intra and interurban variability in night light brightness and extent. However, the disparity between the centimeter scale of most light sources and the decameter spatial resolution of even the most detailed of astronaut photos implies the potential for subpixel spectral mixing to occur in the sensor’s Instantaneous Field of View (IFOV) of individual pixels.
The objective of this study was to characterize the spectral properties of a diverse collection of urban lightscapes—specifically, to quantify the luminance and color distribution of the spectral feature space of multispectral night light. In order to accommodate the spectral mixing of multiple light sources within individual image pixels, a spectral mixture analysis was conducted on a mosaic of intercalibrated color images to identify spectral endmembers bounding the spectral feature space. The spectral endmembers form the basis of a linear spectral mixture model that can be inverted to yield endmember fraction estimates for each RGB image pixel. The trivariate distributions of endmember fractions quantify the spectral diversity of individual urban lightscapes, as well as the aggregate of the full mosaic.
2. Data
All color images were obtained as Nikon raw files (.nef) from NASA’s Gateway to Astronaut Photography of Earth (
https://eol.jsc.nasa.gov accessed on 9 July 2022). The .nef raw files were converted to digital negative (.dng) format using Online Converter (
https://www.onlineconverter.com/nef accessed on 9 July 2022), imported to Adobe Camera Raw v4.0, standardized to common brightness temperatures (2200 K & 5500 K; 0 tint) and saved as 32 bit RGB uncompressed TIFF (.tif) images. Linear tone curves were retained with default settings for brightness (+50) and contrast (+25). Default settings were also retained for sharpening (+25) and noise reduction (luminance: 0; color: +25). All of the subsequent analysis was performed using ENVI and IDL software.
A total of 122 high quality photographs were considered for the analysis. All were shot with Nikon D3S, D4 or D5 cameras. The majority were shot with either a Nikkor 400 mm f/2.8D IF-ED lens (67 photos) or a Nikkor 180 mm f/2.8 AF-D lens (26 photos). A comparison of the effects of view geometry and lens focal length (hence, image dimensions and ground sample distance) is shown in
Figure 1. The primary photo selection criteria were high spatial resolution and minimal blur. Blur can result from either camera movement during exposure or atmospheric water vapor scattering, as illustrated in
Figure 2. The tradeoff between the image sharpness and background noise level of the photos is determined by both exposure time and ISO setting, as well as noise reduction and sharpening settings used in raw conversion. Almost all the photos were acquired at maximum aperture (f/2.8 for both lenses). As shown in
Figure 3, most photos were acquired with exposures in the 1/4 to 1/60 s range with ISO settings near 10,000. From this set of 122 candidate photos, 18 were chosen on the basis of image quality, spatial resolution and geographic diversity. City names (as provided by NASA), dates and image IDs are given in
Table 1.
The effects of atmospheric absorption and scattering were analyzed using the MODTRAN radiative transfer model [
12] to simulate atmospheric transmittance for different model atmospheres. Modeled transmittance profiles were convolved with spectral response functions for the Nikon D3S [
13] to yield weighted transmittance estimates for the R, G and B image channels. Spectral responses are very similar for the D3S, D4 and D5 sensors [
14], with the exception of somewhat broader band red channel sensitivity for the D5. Only London and Las Vegas were acquired with the D5. As shown in
Figure 4, molecular absorptions are more pronounced for the Tropical than for the Mid-Latitude Winter model atmosphere, but the overall amplitude and curvature of the transmittance profiles are nearly identical. Amplitude is far more sensitive to the visibility (Aerosol Optical Depth; AOD) than model atmosphere, as indicated in
Figure 4. In principle, it should be possible to obtain location-specific visibility estimates from the Aerosol Robotic Network (AERONET) database (
https://aeronet.gsfc.nasa.gov accessed on 9 July 2022). Unfortunately, time and location-coincident estimates were not available for any of the 18 sites chosen for this analysis, so a common visibility of 23 km (MODTRAN default) was used for all corrections. Channel-specific correction terms are obtained from the complement of the response-integrated transmittance and added to the unit-normalized DN value of each image pixel.
3. Methods
Originally developed for the spectroscopic analysis of lunar and Martian substrates, the linear spectral mixture model assumes that spectrally distinct materials within the sensor Instantaneous Field of View (IFOV) contribute to an aggregate mixed reflectance (or radiance) spectrum in proportion to their relative areal fraction of the IFOV [
15,
16,
17,
18]. Given N spectrally distinct endmember spectra and a D (>N) channel multispectral image, a linear spectral mixture model can be given as a system of D channel-specific linear mixing equations, each containing N terms to represent the areal fraction of each spectral endmember in the observed spectrum. In matrix notation:
CF =
O, where
C is the NxD matrix of endmember reflectances,
F is the vector of endmember fractions to be estimated and
O is the observed spectrally mixed pixel vector. If D > N, the system is overdetermined, allowing for the possibility of a unique, or at least optimal, solution. Inversion of the linear model yields an estimate of the fraction of each endmember possibly present in the observed mixed spectrum.
The spectral endmembers on which the linear mixture model is based may be chosen a priori from field or laboratory spectra, or from a spectral mixture analysis of all the observed spectra to be modeled [
19,
20]. While there exist a multitude of ways to select spectral endmembers, all require either assumptions about how many and which endmembers may be present, or a characterization of the observed spectra to identify endmembers relative to all the spectra available. In the latter case, some form of dimensionality reduction is generally applied to the D dimensional feature space of observed spectra. If the topology of the lower dimensional projection of the spectral feature space suggests linear mixing, a convex hull can be circumscribed bounding all or most of the observed spectra in the projection. In the projection, the endmembers reside at the apexes of the space, with binary mixtures occurring along the line segments between each adjacent pair of endmembers and N endmember mixtures occurring within the bounding hull [
21,
22,
23].
With multispectral land surface reflectance, large spectrally diverse collections of spectra can provide an approximation of the full global feature space [
24]. The identification of spectral endmembers bounding the composite feature space provides a basis for a general mixture model for land cover reflectance. Sensor-specific standardized endmembers identified from such compilations indicate that feature spaces for Landsat [
25,
26], MODIS [
27] and Sentinel 2 [
28] all have similar mixing space topology bounded by common spectral endmembers representing rock and soil substrates, green vegetation and water (for ice-free landscapes). This allows for the identification of sensor-specific standardized spectral endmembers upon which standardized spectral mixture models may be based. Inversion of the linear spectral mixture model using standardized spectral endmembers is effectively a change of basis from reflectance to land cover fraction. This change of basis from higher dimensional reflectance to lower dimensional land cover fraction renders continuous variations in the landscape that are interpretable in the context of the most distinct physical properties impacting its form and function.
Because anthropogenic light sources (e.g., bulbs, tubes and LEDs) have scales on the order of centimeters, and the IFOV of the camera pixels have scales on the order of decameters (at ISS altitudes), we can treat the radiance incident on the camera sensor as a spectral mixture of multiple light sources coming from a combination of direct and reflected light within the pixel IFOV. While some pixels may be dominated by a single light source, the most general case can include multiple light sources with both direct illumination and upward reflected light. Thus, the strategy is to characterize the spectral distribution of night light color imaged from a variety of urban lightscapes to derive standardized spectral endmembers for visible night light. Following the approach described by [
24], a spectrally diverse mosaic of 18 urban lightscapes is constructed from MODTRAN-calibrated photos. For comparison, mosaics are constructed for two different white balance calibrations. A 2200 K calibration corresponds to the color temperature of high-pressure Sodium lamps widely used for urban street lighting, while a 5500 K calibration corresponds to the color temperature of sunlight. Both mosaics are shown for comparison in
Figure 5. The daylight-relative 5500 K calibration illustrates the preponderance of warm spectrum lighting used in large areas of all 18 lightscapes, while the warm spectrum calibration of 2200 K shows greater contrast within and among lightscapes. All subsequent analyses are performed on the 5500 K calibrated mosaic for consistency with the more broad-spectrum daylight source.
The spectral feature space of the lightscape mosaic allows for identification of the number and identity of spectral endmembers, the linearity of spectral mixing, and the trivariate distribution of the entire set of illuminated pixels. A principal component transform is applied to identify the apexes corresponding to spectral endmembers and to characterize the linearity of spectral mixing.
Figure 6 clearly shows four apexes bounded by binary linear mixing lines. Since the principal component transform maximizes variance, the first PC is analogous to the overall luminance, while the second and third PCs reveal a planar triangular mixing space bounded by the three brightest spectral endmembers representing white, yellow and red sources (lower right). Note that very few pixels are fully red, with the majority clustering in the interior of the red apex and having a spectrum closer to orange than red. The 3D spectral feature space can be considered a tetrahedral pyramid, with the brightest white, yellow and red sources forming a triangular base with a gray axis extending to the dark apex.
The white, yellow, red and dark endmembers span the feature space and therefore form the basis of the WYRK linear mixture model. However, the number of RGB color channels (3) is less than the number of endmembers (4), rendering the problem underdetermined. This can be resolved by adding a unit sum constraint equation (
F1 + F2 + F3 + F4 = 1) to the system. In this analysis the linear mixture model is inverted using the familiar least squares solution:
(CTC)−1CT [
29] to invert the four-endmember linear mixture model for each RGB vector in the lightscape mosaic. The resulting distribution of fraction estimates is generally well-bounded [0, 1] for each endmember, with a small number (912/5,000,000~0.018%) of blue light sources having slightly negative values for the yellow endmember fraction. The distribution of dark fraction estimates has 95% of pixel spectra with dark fractions > 0.78. As expected for such a strongly linear mixing space, the RMS misfit to the four-endmember linear model is less than 10
−7 DN for all pixel spectra.
4. Results
As the dark endmember fraction simply modulates brightness as the complement to the white, yellow and red endmember fractions, the essential information content of the WYRK model can be conveyed with an RGB composite of the WYR fractions. When [R, G, B] = [R, Y, W], the resulting composite maintains a comparable color composition to the 2200 K calibrated RGB mosaic. Compare
Figure 7 (top) to
Figure 5 (top). The WYR composite shows a somewhat more uniform brightness distribution as the transformation to endmember fraction offsets the greater brightness of whiter light sources in most of the lightscapes. This is most apparent in the greater spectral diversity of the London and Las Vegas lightscapes in the WYR composite compared to the 5500 K RGB composite. Another benefit of the WYR model is that pixels saturated in a specific endmember appear either red, green, blue, cyan magenta or yellow to indicate which endmembers are saturated.
One consequence of the equalizing effect of the endmember fraction basis is to skew the distribution of fractions heavily on the white fraction axis corresponding to overall brightness. This is apparent in the slightly darker cast of the WYR composite in
Figure 7 compared to the 5500 K RGB composite in
Figure 5. The extent of the dimmer, more diffuse light sources can be emphasized by displaying Log
10 of WYR fractions, as shown in
Figure 7. The primary spectral effect of this brightening is to emphasize the large areas of diffuse warm spectrum light.
Bivariate fraction distributions from orthogonal projections of the WYRK feature space are shown for eight contrasting urban lightscapes in
Figure 8. While each individual lightscape feature space has topology consistent with the composite space, some differences are apparent, primarily in the relative abundance of saturated pixels on the binary mixing line spanning the white and yellow endmembers. This is most apparent for London, Berlin and Las Vegas. There is also some orange saturation near the red endmember on the binary mixing line spanning red and yellow. This is most apparent for Mecca and Los Angeles. In addition, there are a few examples of cool spectrum light sources with negative fractions of the yellow endmember. This is most apparent in London, Berlin and Las Vegas. The considerable variation in the skewness of the fraction spaces, most apparent in the white–yellow projection, is a result of differences in actual brightness distributions, atmospheric opacity and image exposure.
5. Discussion
Implicit in this analysis is the assumption that the varying exposures of the individual photos will partially compensate for differences in atmospheric path length and aerosol optical depth among photos, making a comparison of the relative RGB differences meaningful. The intercalibration and transmittance correction to the relative RGB values used here is far simpler than the absolute calibration procedure proposed by [
30]. However, absolute radiances are not required for the spectral mixture analysis described here. The comparison of modeled MODTRAN transmittances clearly indicates that the primary effect of both path length and AOD is to shift the bias of the transmittance without significantly affecting the slope, suggesting that the wavelength-dependent Rayleigh scattering is the primary influence on relative RGB values. If this is indeed the case, then intercalibration to a common color temperature and compensation for relative transmission losses among R, G and B channels should allow for intercomparison of the RGB spectral feature spaces. The consistency of the overall topology of the most spectrally distinct feature spaces shown in
Figure 8 suggests that this is the case. This consistency suggests that the relative transmission compensation proposed here could be complementary to the more rigorous calibration proposed by [
30] for studies not requiring absolute calibration.
This intercomparison of 18 urban lightscapes highlights both the consistencies and variations in their spectral feature spaces. As emphasized by the comparison in
Figure 1, a retroactive analysis of archival photographs cannot compensate for unavoidable variations in view geometry and time of acquisition. Some in situ studies of urban lightscapes have investigated the effects of view geometry [
31,
32,
33] and time of observation [
34] and found both to be significant. Future satellite missions could potentially minimize these variations, but with astronaut photos they can only be acknowledged and factored into interpretation.
The unavoidable aliasing of narrowband lamp spectra by broadband RGB sensors will render many different light sources indistinguishable in photos. The comparison of G/R and B/G ratios given by [
14] for 50 common street lamp spectra forms a continuum ranging from warm spectrum low pressure Sodium and LED to nearly flat spectrum ceramic metal halide, fluorescent and LED. Only green fluorescent and LED deviate from this continuum. Even in a luminously diverse environment such as the Las Vegas Strip, a spectral mixture analysis of 360 channel SpecTIR hyperspectral night light imagery by [
35] found the spectral feature space of the brightest lights to be effectively four-dimensional, spanned by high pressure Sodium, incandescent, and two distinct metal halide spectra. In addition, [
13,
36] also identified red and blue neon spectra from the same hyperspectral data. In contrast, the photograph of Las Vegas used in this analysis shows two distinct continua extending from the dark endmember to a warm spectrum orange source (presumably low-pressure Sodium) and a saturated nearly white source that may conflate a variety of bright sources that were overexposed in the ISS photograph.
Despite the saturation resulting from varying exposures of different photos, the feature spaces of the eight individual lightscape examples in
Figure 8 are self-consistent relative to the feature space of the full mosaic of 18 lightscapes. Despite varying degrees of exposure evident in the white–yellow projection of the space, all have similar continua extending to an orange source near the red endmember in the yellow–red projection. This very likely reflects the widespread use of high- and low-pressure Sodium lamps for street lighting. This is apparent in the Log
10 transformed mosaic, where these warm spectrum sources are seen to be widespread throughout all 18 cities—including those with significant areas of cooler spectrum lighting (e.g., Calgary, Kuwait, Las Vegas and London). While cooler spectrum LED streetlights are replacing warm spectrum Sodium lamps in many urban areas, there is apparently some controversy related to both the color cast of warm spectrum Sodium lamps (e.g., [
37]), as well as the brightness of LED lamps [
38] in some cities. Despite the aforementioned aliasing of lamp spectra by broadband RGB sensors, the photos used in this analysis clearly distinguish warm and cool spectrum mixing trends within the individual lightscape feature spaces.
A full resolution comparison of the WYR and Log
10WYR composites for Paris illustrates several general features apparent in the 18 lightscapes analyzed here, and many of the other 122 with sufficient resolution and sharpness to resolve decameter-scale features. As can be seen in
Figure 9, the brightest lights tend to be point sources, and sometimes large thoroughfares such as the Avenue de Champs-Élysées and the Boulevard Périphérique. These point sources are often high luminance white, or saturated on the yellow–white mixing line. However, the Paris image shows a variety of colors for isolated point sources. In contrast, the lower luminance pixels that form the bulk of the distribution are warm spectrum street lighting on the dark–orange continuum. As most street lights are designed to project light downward, the lower luminance warm spectrum distribution is expected to be primarily upward scattering from street light reflected from ground surfaces and building sides, while the brighter point sources may include upward directed lighting on building facades that would be more apparent in oblique orientation photos. Comparisons with calibrated VIIRS Day Night Band imagery indicates that this reflected background luminance is 1 to 2 orders of magnitude dimmer than the brightest point sources that are able to dominate even a hectometer-scale IFOV, such as VIIRS DNB (~700 m) [
39].
In summary, the NASA archive of astronaut photos of urban lightscapes currently contains more than 100 decameter resolution RGB images of sufficient quality to allow for a spectral mixture analysis of the areal distribution of different color light sources from almost 100 cities around the world. While intercomparisons of absolute radiance are complicated by variations in view geometry, atmospheric path length, exposure and aerosol optical depth, a relatively simple standardization of color temperature and correction for the relative transmissivity of red, green and blue image channels can allow for a comparative spectral mixture analysis of the relative luminance distributions and spectral feature spaces of these urban lightscapes and, possibly, for multitemporal change analyses on longer time scales.