The Influence of Signal to Noise Ratio of Legacy Airborne and Satellite Sensors for Simulating Next-Generation Coastal and Inland Water Products

Abstract

Presently, operational ocean color satellite sensors are designed with a legacy perspective for sampling the open ocean primarily in the visible domain, while high spatial resolution sensors such as Sentinel-2, Sentinel-3, and Landsat8 are increasingly used for observations of coastal and inland water quality. Next-generation satellites such as the NASA Plankton, Aerosol, Cloud and ocean Ecosystem (PACE) and Surface Biology and Geology (SBG) sensors are anticipated to increase spatial and/or spectral resolution. An important consideration is determining the minimum signal-to-noise ratio (SNR) needed to retrieve typical biogeochemical products, such as biomass, in aquatic systems, and whether legacy sensors can be used for algorithm development. Here, we evaluate SNR and remote-sensing reflectance (R_rs) uncertainty for representative bright and dim targets in coastal California, USA. The majority of existing sensors fail to meet proposed criteria. Despite these limitations, uncertainties in retrieved biomass as chlorophyll or normalized difference vegetation index (NDVI) remain well below a proposed threshold of 17.5%, suggesting that existing sensors can be used in coastal systems. Existing commercially available in-water and airborne instrument suites can exceed all proposed thresholds for SNR and R_rs uncertainty, providing a path forward for collection of calibration and validation data for future satellite missions.

1. Introduction

Ocean color remote sensing provides a cost-effective, synoptic method for deriving information about the ecologically relevant constituents of the coastal ocean and inland waters [1]. These observations are used to infer ocean health, species composition, water quality, carbon cycling, fluxes of key variables, and biodiversity [2]. The coastal zone, or transition between land and ocean biomes, is the largest ecotone on Earth [3], and therefore an important target for remote sensing. There has been a steady call for better remote sensing methods and increased resolution at the coastal transition zone: The National Research Council [4] noted “simply sustaining the current capabilities of ocean color remote sensing will fall short of supporting the array of [required] ocean color applications;” and the 2017 NASA Decadal Survey [5] challenges the community of practice to support humanity’s ability to thrive by using Earth observations to support our economy, national security, public safety, and quality of life.

Ocean color provides a depth-integrated, but nonetheless near-surface, measurement of the biotic and abiotic constituents that interact with light in aquatic environments. The challenge is to interpret the signature [6,7,8,9]. The magnitude of water-leaving radiance (L_w(λ)_, W m⁻² nm⁻¹ sr⁻¹, abbreviated to L_w for convenience) is highly variable, ranging from dim values in clear, deep water to bright values at water’s edge and in turbid or productive waters. The spatial (~1 km) and temporal (~daily) resolution from legacy satellite platforms such as SeaWiFS, MODIS, and VIIRS is of marginal use in coastal waters [10]. Low signal-to-noise ratio (SNR) measurements of L_w in the blue and red spectral domains result in negative values using standard reprocessing, leading to poor discrimination of pigments from colored dissolved organic matter (CDOM) and poor estimates in the ultraviolet (UV). Aerosol and trace gas plumes from continental sources complicate the task of atmospheric correction, as does cloud cover. Atmospheric correction schemes are also problematic for productive coastal waters. Issues include the use of nonzero near-infrared (NIR) reflectance and poor SNR values, complicating atmospheric correction based on short-wave infrared (SWIR) wavelengths [11,12,13].

Next-generation satellites such as the Plankton Aerosol, Cloud and ocean Ecosystem (PACE) Ocean Color Instrument (OCI) and proposed missions such as the Hyperspectral Infrared Imager (HyspIRI; [14]), now replaced by the NASA Surface Biology and Geology (SBG) sensor, are expected to extend measurements into shorter wavelengths. The UV is potentially useful for discriminating red tides [15], identifying point sources for pollution [16], and improving atmospheric correction, particularly in turbid coastal waters [17,18,19]. However, most existing calibration/validation (cal/val) sensors, as well as spectral radiometers, exhibit poor performance in the UV. All of these issues make aquatic remote sensing challenging.

There is also increasing demand for high (~10 s of meters) spatial resolution and increased (~5–10 nm) spectral resolution. For example, Muller-Karger et al. [2] identified a requirement for a new generation of satellite sensors with (1) spatial resolution on the order of 30–100 m, (2) spectral resolution on the order of 5–10 nm, (3) radiometric quality with SNR > 800, and (4) temporal resolution of hours to days. This so-called H4 imaging (high spatial, temporal, and spectral resolution, as well as high radiometric quality) does not currently exist, but in preparation for future missions existing satellites and airborne sensors may serve as reasonable proxies. For example, the multispectral imager (MSI) aboard Sentinel-2 has 10–60 m native spatial resolution, while the ocean land imager aboard Landsat8 has 30 m native resolution. The “classic” airborne visible infrared imaging spectrometer (AVIRIS-C; referred to hereafter as AVIRIS) provides both high spatial (~5–60 m) and spectral (~400–2500 nm) resolution, and has been the platform of choice for simulating HyspIRI products [20]. A fundamental question that has not been carefully addressed is whether some combination of existing airborne and satellite sensors (Figure 1) are appropriate for development and intercomparison of algorithms capable of utilizing H4 sensing in coastal and inland waters (c.f. [2,21]).

Mouw et al. [21] summarized some of the issues with remote sensing of coastal and inland waters, noting that there is often a tradeoff between high SNR and dynamic range for ocean sensors, and that ocean color sensors often saturate over bright targets, such as turbid coastal and inland waters, due to limited dynamic range. SNR recommendations were therefore for an SNR of >100–200 for SWIR, >600 for NIR, and >1000 for UV-VIS bands, so long as they do not saturate over clouds or bright targets. Muller-Karger et al. [2] recommended SNR > 800 for the UV to visible range (UV-VIS). Wang and Gordon [22] specifically evaluated NIR and SWIR bands for atmospheric correction, and identified minimum requirements of ~200–300 for NIR and ~100 for SWIR, with recommendations of ~600 and ~200 respectively. Qi et al. [23] argued that for the open ocean, SNR of at least 400 for visible bands and >600 for NIR bands are necessary, which is consistent with [22]. As also noted by [23], SNR values listed for at-sensor radiances are difficult to interpret given the wide range of typical at-sensor radiance (L_typ) values, particularly for coastal and inland waters, and it would be more useful to present values in terms of L_w, remote sensing reflectance (R_rs, sr⁻¹), or water-leaving reflectance (ρ_w). Indeed, the PACE Science Definition Team report [24] provides requirements for uncertainty estimates of water leaving reflectance as both percent and absolute values, in addition to SNR for L_typ.

The goals of this manuscript were threefold. First, directly compare commonly used legacy airborne and spaceborne sensors with spatial and/or spectral resolution comparable to H4 sensing for representative coastal targets. Second, use the radiometric uncertainty values to estimate the uncertainty in deriving chlorophyll and kelp biomass as proxies for widely used algorithms that might be applied to H4 sensors. Finally, determine whether existing commercially available airborne and in-water instrumentation can provide high quality estimates of L_w (and therefore R_rs and ρ_w) that exceed proposed SNR and uncertainty estimates for cal/val activities at spatial and temporal scales relevant to coastal and inland waters.

2. Materials and Methods

2.1. Field Sites and Targets

The sites chosen for this analysis (

Table 1. Summary of data collection and processing parameters.

Sensor	Date	Resolution (m)	Atmospheric Correction	Source
Lake Tahoe
AVIRIS	10-Apr-14	14.6	HyspIRI Preparatory Campaign (ATREM)	JPL
OLI	27-Sep-17	30	iCOR	USGS
MSI	14-Sep-17	10, 20, 60	iCOR	ESA
OLCI	14-Sep-17	300	iCOR	ESA
RE	27-Sep-17	5	FLAASH	Planet Labs
C-AERO	13-Sep-17	4.4	Not Necessary	GSFC
San Francisco Bay
AVIRIS	2-Oct-17	16.2	HyspIRI Preparatory Campaign (ATREM)	JPL
OLI	2-Sep-17	30	iCOR	USGS
MSI	17-Sep-17	10, 20, 60	iCOR	ESA
OLCI	13-Sep-17	300	iCOR	ESA
RE	9-Sep-17	5	FLAASH	Planet Labs
C-AERO	8-Sep-17	4.4	Not Necessary	GSFC
Coal Oil Point
AVIRIS	27-Jun-17	16.9	FLAASH	JPL
OLI	16-Jun-17	30	iCOR	USGS
MSI	26-Jun-17	10, 20, 60	iCOR	ESA
RE	3-Jul-17	5	FLAASH	Planet Labs
Elkhorn Slough
PRISM	24-Jul-12	0.5	ATREM	JPL

) were based on satisfying the following three criteria: (a) The sites provide a wide range in L_typ (at-sensor radiance used for derived products), and are, thus, representative of common coastal and inland water targets; (b) the locations were the targets of data collection (in-water, airborne, and satellite) from multiple platforms, and are well characterized as part of historical and ongoing remote sensing studies; and (c) the sites are spatially large enough to use a geostatistical approach to calculate SNR (Figure 2; [25]).

Lake Tahoe is situated between California and Nevada (USA) in the Sierra Nevada mountains. It is the third largest alpine lake globally and is representative of a low-biomass (mesotrophic) inland water body. It has historically been used for estimation of chlorophyll [26] and for vicarious calibration and uncertainty estimates for simulated SeaWiFS bands [27]. The high altitude (1895 m elevation) and typically clear atmospheric conditions simplify atmospheric correction [28,29]. As part of the Coastal High Acquisition Rate Radiometers for Innovative Environmental Research (C-HARRIER) campaign, in-water and airborne data were collected for Lake Tahoe on 13 September 2017 (see below for instrument details). The HyspIRI Airborne Preparatory campaign routinely imaged Lake Tahoe from 2013–2015, with less frequent collections from 2016–2018.

San Francisco Bay, California, USA was chosen as a complementary site to Lake Tahoe because it was also routinely imaged during the HyspIRI Airborne Preparatory campaign, and also because it was included in the C-HARRIER campaign on 8 September 2017. High-resolution imagery and in-water data were collected from Grizzly Bay, while lower resolution imagery from the same time frame were obtained (for the Ocean Land Colour Imager (OLCI) aboard Sentinel-3) using San Pablo Bay, which is physically adjacent to Grizzly Bay. San Francisco Bay represents an optically complex eutrophic end member, and has been imaged extensively for aquatic vegetation, landcover, and water quality (e.g., [30,31,32,33]).

Coal Oil Point is situated in the middle of the Santa Barbara Channel (SBC), California, USA, within the Campus Point State Marine Conservation Area. Coal Oil Point is part of the SBC Long Term Ecological Research (LTER) program, which has extensively studied ocean and land processes’ influence on giant kelp ecosystems via in situ and remote sensing approaches (e.g., [34,35,36,37]). This particular portion of the SBC was chosen for the presence of a large, relatively homogenous, and persistent giant kelp bed.

In addition to the target sites used for in-water, airborne, and satellite remote sensing, an additional site was included for in-water data only that provides an ultra-oligotrophic environment. Crater Lake, Oregon, USA is a deep caldera (594 m maximum depth) at a reference elevation of 1882 m [38]. It is considered to be one of the “clearest” natural waters in existence, rivaling the Sargasso Sea [39]. We used measurements from this location (27 July 2015) as a clear water end member for comparison to [23], who provided an estimate of the best R_rs uncertainty estimates from field measurements in clear oceanic waters.

2.2. In-water, Airborne, and Satellite Sensors

For this study, matched low-altitude flights (Lowest Safe Altitude (LSA), ~100 m above the surface) and near-surface profiling were used to collect above- and in-water optical measurements, respectively, using state-of-the-art microradiometers with 10 decades of dynamic range and 15 Hz sampling. For the objectives considered here, the principal differences between the two embodiments are as follows: (a) The above-water Compact-Airborne Environmental Radiometers for Oceanography (C-AERO) instrument suite has a narrow field of view (2.5° full angle) and wavelengths spanning 320–1640 nm; and (b) the in-water Compact-Optical Profiling System (C-OPS) instrument suite was equipped with the Compact-Propulsion Option for Profiling Systems (C-PrOPS) accessory, wherein two small digital thrusters are combined with hydrobaric buoyancy control to provide a vertical sampling resolution of 1 mm or less, thereby allowing all data products to be derived from 313–875 nm [40]. In-water ancillary data were collected for each field deployment, including surface measurement of total chlorophyll-a (TChla) using high performance liquid chromatography (HPLC; [41]), and CDOM by absorption spectroscopy [42]. The radiometric and ancillary data were processed using the Processing of Radiometric Observations of Seawater using Information Technologies (PROSIT) software package to provide estimates of L_w and R_rs from in-water and above-water measurements. While a full atmospheric correction was not applied given the low flight altitude, skylight reflectance was removed as per standard NASA above-water reflectance protocols [40].

2.3. AVIRIS Imagery

Images for Lake Tahoe, Grizzly Bay, and Coal Oil Point were obtained from the NASA Jet Propulsion Laboratory (JPL) AVIRIS site, and specifically from the HyspIRI Airborne Preparatory campaign website for Lake Tahoe and Grizzly Bay. Data were downloaded as Level 2 (L2) reflectances with a standard atmospheric correction applied [43]. Data for Lake Tahoe and Grizzly Bay were from 10 April 2014 and 2 October 2015, respectively (Table 1). Imagery was kept at native pixel and wavelength resolution of 14.6 and 16.2 m respectively (both at ~10 nm spectral resolution from 400–2500 nm). For the Lake Tahoe image, the data were subsequently spatially binned using ENVI (Harris Geospatial) to 60 m resolution. An AVIRIS image for Coal Oil Point in the Santa Barbara Channel was obtained from the NASA Jet Propulsion Laboratory for 27 June 2017. Data were downloaded as Level 1 (L1) orthocorrected at-sensor radiance, and subsequently atmospherically corrected using the Fast Line-of-sight Atmospheric Analysis of Hypercubes (FLAASH; Harris Geospatial) software.

2.4. Landsat8 Operational Land Imager

The Operational Land Imager (OLI) imagery was accessed from the USGS EarthExplorer portal. Images were chosen as close in time to the in-water measurements after taking into account cloud cover and atmospheric conditions. The three selected images were from 16 June 2017, 2 September 2017, and 27 September 2017 for Coal Oil Point, Grizzly Bay, and Lake Tahoe, respectively. The L1C data were atmospherically corrected using the iCOR plugin [44] for the SNAP (European Space Agency; ESA) software package.

2.5. Sentinel-2, Sentinel-3 Imagery

MSI (Sentinel-2) and OLCI (Sentinel-3) imagery were accessed from the European Space Agency as L1 files and processed using iCOR in SNAP. S2A-MSI data were from 26 June 2017, 14 September 2017, and 17 September 2017 for Coal Oil Point, Lake Tahoe, and San Francisco Bay. To meet constraints of uniformity and sufficient number of pixels, San Pablo Bay (adjacent to Grizzly Bay, which is located in Suisun Bay) was used for the analysis.

2.6. RapidEye Imagery

Data from the RapidEye-2 sensor were obtained from Planet Labs, Inc. as Level 3A orthorectified TIFF images. The data were atmospherically corrected using FLAASH. Images were obtained for 9 September 2017, 27 September 2017, and 3 July 2017 for Grizzly Bay, Lake Tahoe, and Coal Oil Point, respectively.

2.7. PRISM Imagery

Neither AVIRIS Next Generation (AVIRIS-NG) nor the JPL Portable Remote Imaging Spectrometer (PRISM) were available for our selected sites, but one image for Elkhorn Slough, California, USA using PRISM was analyzed to evaluate modern airborne sensors capable of supporting H4 imaging. Data were obtained from JPL as Level 2 atmospherically corrected data as described by [45]. The deep-water channel in Figure Video 2 [45] was used for analysis.

2.8. Calculation of Signal-to-Noise Ratios and Uncertainty

The signal-to-noise ratio (SNR) of a sensor can be estimated as the ratio of the mean signal for an invariant target to the standard deviation of the signal. The SNR of an orbital or airborne sensor is often measured in the laboratory, based on a standard target with (typically) spectrally uniform albedo of 5% for instruments designed to image water [46]. Estimates of SNR convolve multiple sources of uncertainty (noise), which can include instrument artifacts and uncertainties revealed as part of instrument characterization (discussed in greater detail by [23]). An alternative method, referred to as “geostatistical SNR”, applies a semivariogram to a spatially uniform target from field data. This provides an SNR that is relevant to the investigator [25] and can include the effects of atmospheric correction, if applicable (C-AERO is flown at LSA during clear-sky conditions, and in the principal plane of the sun, to minimize atmospheric effects as well as glint), thus providing SNR for L_w rather than at-sensor radiance. For this method to be applicable, it is assumed that the data exhibit stationarity and isotropy, and that the selected region of an image does not change spatial resolution [25].

Here we apply a geostatistical SNR approach, generally following the methodology outlined in [25]. For each sensor a representative region of the image was identified that meets the above criteria. For C-AERO, which collects data along a linear transect, the data were comprised of a narrow swath or transect across the target region. For the imaging sensors, a two-dimensional matrix was used (see Figure 2 for representative examples). SNR was determined for each wavelength by calculating the semivariance, γ(h) over the distance between pixel pairs (h) using the MATLAB (Mathworks Inc.) packages variogram and variogramfit. A theoretical semivariogram was calculated with the data to estimate the nugget, sill, and range (c.f. [25,47]). The SNR was then calculated as the mean signal ($\overline{z},$ water-leaving radiance in this case) divided by the square root of the nugget variance (C₀):

$$\mathrm{SNR}=\frac{\overline{\mathrm{z}}}{\sqrt{{\mathrm{C}}_0}}.$$

(1)

The non-zero intercept, or nugget (n, or C₀), determines the degree of unresolved variability, or noise, and is the non-zero limit of ${\mathsf{\gamma }}_{\mathrm{h}}$ when h approaches zero, where h is the lag and ${\mathsf{\gamma }}_{\mathrm{h}}$ is the semivariance as a function of the lag. The range (a) determines how quickly in space the variability reaches a global maximum, while the sill (C₁) determines the total resolved variance, and the range beyond which pixel proximity does not correlate with the spatial structure of the data. Various theoretical variogram models can be used depending on the structure of the data. Three of these theoretical models are formulated as:

$${\gamma }_h=c_0+c_1[1-\exp \left(-\frac{h^2}{{\mathrm{a}}^2}\right)$$

(2)

$${\gamma }_h=c_0+c_1\left[1-\exp \left(-\frac{h}{\mathrm{a}}\right)\right]$$

(3)

$${\gamma }_h=\{\begin{array}{ll}{c}_0+c_1\left(\frac{h}{\mathrm{a}}\right),& h\le a\\ c_0+c_1,& h>a\end{array}$$

(4)

where (2) is a Gaussian, (3) is an exponential, and (4) is a bounded linear model. For this analysis the top-performing model was determined based on the r² value of the fit, with the majority of the semivariograms exhibiting r² values greater than 0.8; some wavelengths for AVIRIS data were discarded when no model could be fit with reasonable results. As discussed by [47], real data may be more complex than these theoretical models, with multiple nodes or inflections at distances less than saturation. We note that the geostatistical approach would classify spatial variability in R_rs for a given scene as noise when it may in fact be real variability (structure) in the environment. For this analysis, this bias was primarily in the OLCI-derived SNR, given the large (300 m) pixel size at native resolution.

2.9. Uncertainties in R_rs and Derived Geophysical Products

Following determination of SNR, we also calculated R_rs uncertainty by converting the water-leaving radiance to corresponding R_rs values and converting the percent noise of the mean signal to the corresponding R_rs value. Following [48], the same weightings in a 3-band configuration were used for calculation of chlorophyll (chla) from OLI and MSI. For kelp, the NDVI was calculated. The relevant algorithms are:

$${\log }_{10}\left(\mathrm{chla}\right)={\mathrm{a}}_0+{{\displaystyle \sum }}_{\mathrm{i}}^4{\mathrm{a}}_{\mathrm{i}}\left({\log }_{10}({\mathrm{R}}_{\mathrm{bg}}\right)$$

(5)

where $R_{bg}$ is the ratio of maximum blue to green reflectance, and a0–a4 are weighting functions for instrument-specific bands (c.f. [48,49,50,51]), and for NDVI,

$$\mathrm{NDVI}=({\mathrm{R}}_{\mathrm{rs}}^{\mathrm{NIR}}-{\mathrm{R}}_{\mathrm{rs}}^{\mathrm{red}})/({\mathrm{R}}_{\mathrm{rs}}^{\mathrm{NIR}}+{\mathrm{R}}_{\mathrm{rs}}^{\mathrm{red}})$$

(6)

where $R_{rs}^{red}$, $R_{rs}^{NIR}$ are the remote-sensing reflectances for each band and sensor combination (c.f. [35]).

These data were then used to propagate uncertainty using the standard OCx band-ratio algorithm for chlorophyll retrievals following [23], and for NDVI for the kelp targets using the following equation:

$$\mathsf{\delta }\mathrm{NDVI}=\frac{2{\mathrm{R}}_{\mathrm{rs}}^{\mathrm{red}}{\mathrm{R}}_{\mathrm{rs}}^{\mathrm{NIR}}}{{({\mathrm{R}}_{\mathrm{rs}}^{\mathrm{NIR}}+{\mathrm{R}}_{\mathrm{rs}}^{\mathrm{red}})}^2}\sqrt{{\left(\frac{1}{{\mathrm{SNR}}^{\mathrm{NIR}}}\right)}^2+{\left(\frac{1}{{\mathrm{SNR}}^{\mathrm{red}}}\right)}^2}$$

(7)

where $\delta chla$ and $\delta NDVI$ are the calculated uncertainties with units of mg m⁻³ for chla and dimensionless units (−1 to 1) for NDVI.

3. Results

3.1. Minimal Uncertainty from Field Observations

Comparison of SNR, or uncertainty, for different sensors benefits from an initial comparison to the lowest uncertainty field measurements as an objective standard. Previous analyses [23,52] used a skylight-blocked approach (SBA) to measure above-water reflectance in very clear (oligotrophic) marine waters. This approach estimated that the “state-of-the-art” uncertainties from these measurements were ~1% for typical blue water R_rs (443 nm) of 0.01 sr⁻¹, and suggested this value (1% uncertainty) as an achievable minimum uncertainty for field measurements. Here, we provide data obtained from C-OPS with C-PrOPS demonstrating how this uncertainty estimate varies as a function of water clarity and type (Figure 3). We confirm that uncertainty of ~0.0001 sr⁻¹ is achievable across a wide wavelength range (313–875 nm) and water masses, providing a realistic baseline for comparison of airborne and satellite-derived uncertainty estimates. The average uncertainty from C-OPS with C-PrOPS across the three sites was 0.42% with a maximum value of 1.35% at 380 nm, demonstrating an ability to routinely provide high-quality in-water observations from ultra-oligotrophic to optically complex estuarine waters.

3.2. Airborne and Satellite-Derived Uncertainty

Comparison of SNR and R_rs uncertainty for the airborne and satellite platforms shows consistent performance across sites (Figure 4). It is assumed that an SNR of two is the absolute minimum requirement for scientifically relevant data. It is often assumed that the at-sensor signal for aquatic targets is ~5% of L_typ. Consequently, for a proposed at-sensor SNR of 400–1000, the corresponding atmospherically corrected SNR should be 20–50. Given that there are other potential sources of variance, doubling that minimum SNR to 40–100 provides a reasonable new threshold. For the NIR and SWIR domains, the signal would typically be much less than 5%, but previously recommended SNR targets are also lower.

At native resolution (i.e., without spatial binning) RapidEye-2 exhibited the lowest SNR performance and correspondingly highest R_rs uncertainty, followed by AVIRIS with a strong bias towards degraded performance in the blue for R_rs uncertainty, and a peaked increase in SNR at ~500 nm with a rapid drop-off towards the blue and red wavelengths. AVIRIS also suffered from data loss below ~400 nm and above ~750 nm after atmospheric correction because of negative radiances. OLI, MSI, and OLCI are comparable, with higher SNR values in the visible and fairly low R_rs uncertainty (~0.0002–0.0006 sr⁻¹). C-AERO consistently out-performed the other sensors (Figure 5) with very low R_rs uncertainty, approaching that of in-water measurements (Figure 3) with SNR values >100 across all wavelengths, exceeding 1000 for the UV and blue wavelengths in mesotrophic waters.

Atmospherically corrected PRISM data were only available for Elkhorn Slough. SNR and uncertainties were improved in comparison to AVIRIS (Figure 6), with much better performance in the blue part of the spectrum (<500 nm) as expected, because PRISM is blue-optimized for aquatic retrievals [53,54]. The fairly low SNR (comparable to AVIRIS) is largely due to the calculations being based on native resolution; AVIRIS pixels were ~20 m resolution while PRISM were 0.5 m (Table 1). Spatial binning of the PRISM data would be expected to substantially increase the observed SNR. Comparison of radiometric uncertainty demonstrates that PRISM exhibits ~2–10× less R_rs uncertainty compared to AVIRIS when processed similarly (Figure 6). The apparent discrepancy between SNR and Rrs uncertainty for AVIRIS and PRISM can be accounted for by examining the data for each scene. L_w values for AVIRIS were ~2× higher in the visible than for PRISM. Therefore, there was more signal in the AVIRIS image and the resulting uncertainty was also larger. The L_w for AVIRIS was on average 2.86× higher than for PRISM, while the R_rs uncertainty was 2.61× higher. The mean uncertainties were 2.49% for PRISM and 17.5% for AVIRIS, resulting in an average percent noise that was ~7× higher for AVIRIS compared to PRISM despite the comparable SNR. Despite the improvement in R_rs uncertainty with PRISM there was still a loss of data (negative radiances) below ~380 nm after atmospheric correction.

For kelp at Coal Oil Point, only four sensors were evaluated based on spatial resolution requirements and data availability: AVIRIS, RapidEye-2, OLI, and MSI. Kelp provides a bright target compared to typical aquatic scenes, so the data are presented as percent uncertainty for R_rs to simplify the comparison across all wavelengths (Figure 7). The SNR and radiometric uncertainty for the kelp were comparable to that of the optically bright San Francisco Bay and Elkhorn Slough results (not shown). For the four sensors, the same general pattern was observed with poorest performance from RapidEye-2 and comparable performance for OLI, MSI, and AVIRIS (Figure 7). OLI was the only sensor able to meet the PACE specified percent uncertainty ranges of 20% for 350–400 nm, 5% for 400–600 nm, and 10% for 660–710 nm. AVIRIS was below the threshold for 400–600 and 650–710 nm, but suffered from data loss (negative radiances) after atmospheric correction for wavelengths less than ~500 nm, and would not meet the 10% threshold if the PACE guidelines of 10% uncertainty were extended from 710 to 900 nm.

3.3. Derived Chlorophyll and NDVI

Data for all sensors were converted to R_rs and then used to calculate chla for C-AERO, AVIRIS, OLI, MSI, and OLCI, and NDVI (for kelp) using AVIRIS, OLI, MSI, and RapidEye-2. In-water validation data were not available for sensors other than C-AERO, so $\mathsf{\delta }\mathrm{chla},$ and $\mathsf{\delta }\mathrm{NDVI},$ are presented as percent uncertainty due to the estimated error and not in direct comparison with measured in-water values. Reported uncertainty is, therefore, only an indication of the sensor performance and does not account for potential uncertainties or biases in the underlying algorithm, except for the C-AERO data.

For the two C-AERO flights, measured TChla was 0.444 mg m⁻³ (Lake Tahoe) and 2.251 mg m⁻³ (Grizzly Bay), which would classify these waterbodies as mesotrophic and eutrophic, respectively [55]. The corresponding C-AERO data using the OC3M algorithm were 0.419 mg m⁻³ and 4.818 mg m⁻³ chla respectively, or 5.73% and 114.06% relative percent difference (RPD). The larger RPD for Grizzly Bay is presumed to be related to the optical complexity and spatial heterogeneity of the target [31] rather than the sensor. When OC3M was applied to the closest temporally and spatially coincident data from C-AERO and C-OPS for Grizzly Bay (same data collection but for a region of the Bay not used for the SNR calculation, because the flight line was shorter), the calculated values were 8.076 and 9.191 mg m⁻³ chla, respectively, or an RPD of 12.13%. For comparison, and noting that these values are not true matchups, calculated chla for Lake Tahoe for AVIRIS, OLI, MSI, and OLCI was 0.118, 0.568, 0.620, and 0.240 mg m⁻³, respectively, while for Grizzly Bay it was 3.576, 2.751, and 3.516 mg m⁻³ for AVIRIS, OLI, and MSI, and 17.708 mg m⁻³ for San Pablo Bay from OLCI.

A comparison of estimated biomass as chla and NDVI from the sensors for Lake Tahoe, San Francisco Bay, and Coal Oil Point is provided in Figure 8. As summarized by [23], typical global chlorophyll uncertainty is ~5%, while the acceptable total uncertainty for chla retrievals in the open ocean have been set at 35%, i.e., approximately 25% each for the inversion of remote and in situ optical measurements [56]. For coastal California, a recent assessment provided average errors of ~11%–14% when using in situ matchups with coincident satellite overpasses [57].

Assuming that $\mathsf{\delta }\mathrm{chla}$ and $\mathsf{\delta }\mathrm{NDVI}$ can be partitioned into the combined uncertainty of instrument and atmospheric correction error (evaluated here), algorithm performance, matchup requirements, and uncertainty in field measurements [57], a reasonable threshold is that no more than half the combined uncertainly should be attributable to the sensor and atmospheric correction. This would provide a first-order threshold of between 2.5%–17.5% uncertainty, with the caveat that the algorithm performance may be much worse than typical open ocean uncertainty in complex coastal waters (e.g., RPD > 100% for Grizzly Bay when relaxed spatial/temporal matchup criteria were employed). With the exception of AVIRIS for $\mathsf{\delta }\mathrm{chla}$ at native spatial resolution, $\mathsf{\delta }\mathrm{chla}$ and $\delta NDVI$ uncertainties were well within this proposed threshold of 17.5% uncertainty.

4. Discussion

Numerous investigators have estimated at-sensor SNR for existing satellite sensors, including similar geostatistical analysis for OLI and MSI [58,59]. The results obtained here for those two sensors were actually better than expected, with an average SNR in the visible bands (for bright and dim targets) of ~28 for OLI and ~48 for MSI compared to at-senor SNR of ~124 and ~110 [59] for the same sensors, which would be reduced to ~6 if 5% of the signal were attributed to L_w. This improved SNR is not attributable to the semivariogram method, because we obtained comparable at-sensor SNR values (data not shown) when processing the scenes without atmospheric correction. We note that for OLI, MSI, and OLCI, the iCOR atmospheric package was used. This processing scheme tiles the full scene into 15 × 15 km regions to calculate aerosol optical thickness (AOT), with the assumption that the atmosphere is still homogenous at this scale, while high spectral variation is maintained [44,60]. This effectively smooths and increases the SNR of the NIR/SWIR bands used for atmospheric correction, a method which was also recommended by [23] to improve SNR for NIR/SWIR bands. Despite this correction, OLCI generally performed less capably than expected (evaluated using SNR, R_rs uncertainty, and derived chla), given the larger pixel size (300 m) and same atmospheric correction. This is likely due to violation of the homogeneity requirement. San Pablo Bay was chosen (rather than Grizzly Bay) given the small physical size of the bays being evaluated and the large pixel size, but San Pablo Bay is still spatially heterogeneous (e.g., Figure 2). As would be expected, uncertainty in derived chla decreased in Lake Tahoe compared to San Pablo Bay for OLCI, consistent with spatial heterogeneity increasing with increasing pixel size. The geostatistical approach would treat underlying environmental variability at this coarse spatial resolution as “noise”, suppressing the calculated SNR for OLCI.

For AVIRIS, [25] reported SNR was ~20–160 for sediment-laden water in the visible bands (also using San Francisco Bay) with peak values in the green. Our results peaked at 580 nm with a range in the visible of 6.3–60.5, consistent with bright water exhibiting ~37.5% albedo, and generally consistent with [25]. We note that AVIRIS, PRISM, and RapidEye used a pixel-by-pixel atmospheric correction, limiting potential improvement from NIR and SWIR smoothing.

While SNR is a frequently presented metric of sensor performance, less commonly reported are radiometric uncertainties expressed as R_rs (sr⁻¹). Here we estimated both SNR and R_rs uncertainty using the same methods and for similar targets, allowing direct comparison of the sensors. The comparison of PRISM data from Elkhorn Slough with AVIRIS data for Grizzly Bay (Figure 3 and Figure 6) shows that at native resolution, geostatistical SNR were similar for the two sensors. However, the R_rs uncertainty is 5–10 times lower for PRISM compared to AVIRIS, with much better performance in the blue part of the spectrum. Given the desire to document uncertainty in all derived products, it is arguably more useful to report instrument performance as R_rs uncertainty rather than as an SNR.

Regardless of the selected metric, direct comparison of the sensors with comparable targets identifies the challenge in meeting proposed SNR and/or uncertainty estimates for existing and next-generation sensors in real-world scenarios. The three most common uses of airborne remote sensing for ocean color investigations are vicarious calibration, algorithm validation, and basic research. From the perspective of SNR, establishing a set of thresholds or limits based solely on the calibration and characterization uncertainty of the sensor with no other influences (e.g., environmental) provides a starting point for comparative analyses, as follows: (1) For a theoretical research limit, there must be more calibrated signal than noise, which is SNR = 2; (2) for a theoretical validation limit, the noise must be within 15% of the calibrated signal, which is SNR = 8; (3) for a theoretical calibration limit, the noise must be within 5% of the calibrated signal, which is SNR = 50. For existing and proposed satellites, similar SNR thresholds have been proposed, ranging from SNR = 30–100 (after accounting for the albedo of typical aquatic targets), as described above. The PACE Science Definition Team [24] provides comparable R_rs uncertainty values of 0.0057 or 20% for ρ_w (350–400 nm), 0.0020 or 5% for ρ_w (400–600 nm), and 0.0007 or 10% for ρ_w (660–710 nm). For dim targets, such as Lake Tahoe, all of the existing image-based sensors struggle to meet either criteria, particularly in the UV and blue wavelengths. C-AERO easily meets these requirements, but it is also designed as a point-sensor for calibration and validation activities. Bright targets, such as San Francisco Bay, exhibit better performance for image-based sensors, with PRISM, OLI, MSI, and OLCI all meeting the proposed R_rs uncertainty thresholds for the majority or all of the bands, and C-AERO again easily satisfying the requirements.

In regards to calibration, validation, and research, none of these existing image-based remote sensors have a performance that could support an operational calibration limit. Operational validation is possible for several sensors over turbid waters or for a reduced spectral range over blue or turbid waters (PRISM, MSI, OLCI, OLI, and AVIRIS). Research activities for an operational limit are not limited in turbid waters except for AVIRIS at shorter wavelengths. For operational research over blue waters, the principal limitations are for OLI, OLCI, and AVIRIS, which are restricted primarily to wavelengths spanning the UV, blue, and green domains. For very bright targets, such as kelp, AVIRIS and OLI can achieve reasonable SNR and uncertainty levels (Figure 7), while MSI shows degraded but reasonable performance.

RapidEye provides the worst performance for kelp, but a significant tradeoff is the 5 m spatial resolution it provides. For this analysis we specifically analyzed all sensors at native pixel resolution. Spatial binning would, of course, improve SNR and uncertainty, and has been recommended in the past to improve SNR and to remove artifacts such as striping (e.g., [48,49,58,59,61]), but this also defeats the point of having high spatial resolution imagery (e.g., [48]). Using giant kelp (Macrocystis pyrifera) and bull kelp (Nereocystis luetkeana) as relevant examples, historical aerial kelp surveys conducted by the California Department of Fish and Wildlife employed 2 m pixel resolution. Cavanaugh et al. [35] were able to reproduce the giant kelp canopy area in southern California using 10 m SPOT data, while [34] recently extended the record by merging 30 m historical Landsat and OLI data. However, as noted in previous analyses, monitoring of smaller or less dense kelp beds requires higher spatial resolution imagery [62].

Stekoll et al. [63] successfully used airborne imagery to map bull kelp in the Northern California Current, but studies of bull kelp’s temporal and spatial trends remain largely absent from the published literature despite the success in monitoring giant kelp. This lack of research is likely due to observation and environmental challenges, e.g., the small patch sizes formed by bull kelp for a brief (few months) period during autumn require high spatial resolution and sufficient SNR not available through existing sensors. Thus, while global 60–100 m spatial resolution may be sufficient to capture aquatic water column properties, important ecological targets would likely be missed at that resolution, and global spatial resolution of 10–30 m is both technically possible and preferred [64].

While SNR and R_rs uncertainty are one set of quantitative metrics, it is also important to consider how sensor design influences retrieval of higher-level products such as chla and NDVI. For the instrument systems tested, uncertainties were obtained for water column chla and kelp NDVI that would be compliant with our proposed maximum of 17.5% uncertainty, with the exception of AVIRIS at native resolution (Figure 8). This is in striking contrast to the often substandard quantitative radiometric performance for the same sensors. We note that we did not assess algorithm or sensor biases associated with the data retrievals given the lack of direct matchups for all sensors, and that these biases may be substantial for regional assessments [65]. Our results are consistent with the numerous publications that have applied these sensors to coastal and inland waters successfully, demonstrating that commonly used band-ratio algorithms are reasonably robust even when using less-than-optimal data. Comparable radiometric and biogeochemical uncertainties across sensors also suggest that existing sensors have similar enough spatial, spectral, temporal, and radiometric characteristics to form virtual constellations or sensor webs [48,66,67], improving the temporal return rate of existing high spatial resolution sensors such as MSI and OLI.

We did not assess variability in atmospheric correction schemes because our focus was on using standard (research or operational) products. We note, however, that there were obvious atmospheric correction issues with several of the data sets, while one instrument suite (C-AERO) completely avoids a need for atmospheric correction (because the flight altitude is so low). Regarding the former, there were substantial data dropouts (zero or negative L_w) for AVIRIS and PRISM, especially in the blue and UV wavelengths but also in the red and NIR. This is particularly problematic for next-generation sensors such as PACE, because it would be challenging to provide calibration or validation data from existing platforms other than in-water sensors and C-AERO, and it would be impossible to apply a large majority of the hyperspectral airborne data already collected, e.g., HyspIRI [14], to algorithm development activities if the full spectrum is required without first substantially improving the atmospheric correction for aquatic targets. Conversely, improvements to atmospheric correction would immediately improve existing data (e.g., [68]) and likely contributed to better than anticipated SNR for OLI and MSI.

Looking forward, this analysis suggests that present operational sensors provide adequate sensitivity to apply typical band-ratio algorithms to assess high spatial resolution capabilities. Existing platforms could also be used in a virtual constellation or sensor web to simulate high(er) temporal resolution. Of the two remaining components in H4 remote sensing, existing sensors, for the most part, do not meet the requirements for high spectral resolution and high fidelity. In particular, AVIRIS uncertainties are unacceptably high for moderately dim targets, while standard atmospheric processing of AVIRIS results in significant spectral data loss, even over bright targets such as kelp. Next-generation airborne sensors such as PRISM and AVIRIS-NG can address these shortcomings, but have yet to be widely and routinely deployed for aquatic remote sensing.

While in-water measurements remain the standard for calibration and validation activities (e.g., [23,40]), we also demonstrate that C-AERO, which is based on commercial off-the-shelf technology, can provide equivalent high-fidelity data, which meet or exceed proposed criteria for H4 remote sensing, with the caveat that in the current configuration C-AERO is multispectral rather than hyperspectral. C-AERO provides an ability to collect calibration, validation, and research-quality data rapidly over much larger spatial areas than is possible using traditional in-water measurements, so long as point-based rather than image-based sensing is acceptable, and is particularly relevant for next-generation sensors that extend into the UV and SWIR wavelengths.

5. Conclusions

We provide a direct comparison of AVIRIS, OLI, MSI, OLCI, and RapidEye (with limited evaluation of PRISM) and demonstrate that presently operational airborne and satellite sensors capable of two-dimensional imaging are comparable in SNR and R_rs uncertainty for typical coastal targets. C-AERO, which produces one-dimensional transects, provides the highest SNR and lowest R_rs uncertainties. Despite the poor radiometric performance of the imaging sensors compared to recommended thresholds, the derived biogeochemical products, chla and NDVI, are retrieved with compliant accuracy. None of the existing imaging sensors can consistently produce data compatible with H4 remote sensing requirements, but aspects of H4 remote sensing can be addressed with existing data. Next-generation airborne sensors such as PRISM and AVIRIS-NG should immediately be prioritized over AVIRIS-Classic, which exhibits poor performance for aquatic targets. Finally, existing commercial off-the-shelf technology (C-AERO) exceeds all radiometric requirements for calibration and validation of multi-wavelength sensors extending into the UV, NIR, and SWIR, providing an ability to rapidly and inexpensively collect data at spatial and temporal scales relevant to coastal and inland waters.