Analyzing Satellite Ocean Color Match-Up Protocols Using the Satellite Validation Navy Tool (SAVANT) at MOBY and Two AERONET-OC Sites

Abstract

The satellite validation navy tool (SAVANT) was developed by the Naval Research Laboratory to help facilitate the assessment of the stability and accuracy of ocean color satellites, using numerous ground truth (in situ) platforms around the globe and support methods for match-up protocols. The effects of varying spatial constraints with permissive and strict protocols on match-up uncertainty are evaluated, in an attempt to establish an optimal satellite ocean color calibration and validation (cal/val) match-up protocol. This allows users to evaluate the accuracy of ocean color sensors compared to specific ground truth sites that provide continuous data. Various match-up constraints may be adjusted, allowing for varied evaluations of their effects on match-up data. The results include the following: (a) the difference between aerosol robotic network ocean color (AERONET-OC) and marine optical Buoy (MOBY) evaluations; (b) the differences across the visible spectrum for various water types; (c) spatial differences and the size of satellite area chosen for comparison; and (d) temporal differences in optically complex water. The match-up uncertainty analysis was performed using Suomi National Polar-orbiting Partnership (SNPP) Visible Infrared Imaging Radiometer Suite (VIIRS) SNPP data at the AERONET-OC sites and the MOBY site. It was found that the more permissive constraint sets allow for a higher number of match-ups and a more comprehensive representation of the conditions, while the restrictive constraints provide better statistical match-ups between in situ and satellite sensors.

1. Introduction

Space-borne ocean color sensors are launched with initial calibrations, but the following various conditions contribute to the degradation of these calibrations over time: exposure to solar radiation, jostling during launch, aging sensors, and power-supplies. The pre-launch calibration must be adjusted during orbit [1]. NASA’s on-orbit vicarious calibration technique is applied to correct for these conditions (as well as any other within the system), and reduce uncertainty, resulting in data that are better suited for making ocean color observations. Traditionally, calibration and validation is a lengthy process; however, the Naval Research Laboratory (NRL) favors rapid, frequent calibrations. This is spurred by the recent launch of the following several sensors in a relatively short period of time: NOAA-20 Visible Imaging Radiometer Suite (VIIRS), and the Sentinel 3-A and 3-B Ocean Land Colour Imager (OLCI) sensors [2]. In order to enable these sensors to become operational quickly, we have developed the satellite validation navy tool (SAVANT) system to allow faster acquisition of an optimal number of ground-truth match-ups for calibration and validation purposes. SAVANT is currently at version 1.4, which was used to produce the graphs and statistics used in this study, and is available for use at https://www7330.nrlssc.navy.mil/7331/savant (accessed on 6 July 2021) in Supplementary Materials.

For this study, two primary sources of data, at three locations that monitor in situ ocean color, the Marine Optical Buoy (MOBY) and two Aerosol Robotic Network Ocean Color (AERONET-OC) sites, are used. The Aerosol Robotic Network currently has thirty-six active or historical ocean color SeaPRISM instruments. These locations are spread across various coastal and inland areas, and allow for multiple measurements per day, per active location. The SeaPRISMs take measurements centered around local noon, daily. These measurements are screened for cloud contamination and processed using the protocols developed by Zibordi [3], before SAVANT ingests them, and then they are matched up against the available satellite data. The in situ sites used for this study are Venise, located in the Adriatic Sea off of Venice, Italy; Wave-Current-Surge Information System (WaveCIS), located in the Northern Gulf of Mexico, offshore of Louisiana (for the SeaPRISM sites we are using the NASA spellings for Venice and WavCIS in this study); and MOBY, located in the Pacific Ocean, west of Lanai, HI (Figure 1). These regions were chosen because they cover a range of water types and the instruments are regularly calibrated with extensive data collections over many years. The polar-orbiting SNPP VIIRS satellite used in this study passes over these locations within a few hours of local noon each day. These data are stored in the SAVANT database, along with the following meta-data describing the collection: satellite and solar angles, observed wind speed at the SeaPRISM, and data quality control flags determined during satellite processing (through the Naval Research Laboratory’s automated processing software, calibrated as described by [4]). The MOBY data are also measured three times per day and have their own quality assessment, which is available based on the local conditions at the buoy.

SAVANT’s quality controls (Figure 2) include setting constraints for (a) the geographical box size of the average satellite data (1 km, 9 km, and 25 km), (b) the maximum temporal difference between the satellite overpass and the in situ measurement, (c) limiting the maximum observed windspeed (defaults to 8 ms⁻¹ maximum) and aerosol optical thickness (defaults to 0.2), (d) the minimum and maximum valid normalized water-leaving radiance values, (e) the maximum coefficient of variance within the satellite box (measured standard deviation divided by measured mean, per match-up), (f) the number of valid satellite pixels (50% of the 5 × 5 box, for example, requiring 13 pixels to be valid), and (g) applying the following various quality control flags: high glint, maximum aerosol iteration [5], high polarization, stray light, absorbing aerosol, moderate sun glint, navigation failure, turbid water, low water-leaving radiance, epsilon out of range, and the high satellite and solar zenith flags.

Previously, we set constraint thresholds and adjusted the individual constraints [6], in order to observe the impact of each individual constraint on the spectral ocean color and in situ match-ups. We defined protocols by determining what constraints were the closest to the optimum (based on statistical performance) match-up for use at the WaveCIS site. This allowed the reduction in the uncertainty in derived satellite products. With well-defined, well-tested sets of constraint protocols, the next step is to apply them to various locations for further research and verification.

2. Materials and Methods

The satellite data used for this study are from the VIIRS instrument on board the S-NPP satellite [7]. These satellite data are processed through NRL’s Automated Processing System (APS), which uses the Sea-viewing Wide Field-of-view Sensor (SeaWiFS) Data Analysis System (SeaDAS) l2gen code [8] as a base for processing satellite data through to level-2. During the processing of the data, vicarious calibration using gains determined by NRL are applied [2], the data are bidirectional reflectance distribution function corrected (BRDF) [9], and atmospherically corrected [10]. The data are processed through to level-2, extracted in a 5 × 5 box, and inserted into the SAVANT database. These pixels are centered around the ground-truth stations. SAVANT’s web-facing front end allows the end user to select between the center pixel only (1 × 1), a 3 × 3 box, or the full 5 × 5 box. APS runs in near-real-time producing satellite data at the three areas of interest discussed, and the data are extracted into SAVANT once the processing is completed.

The WaveCIS site south of Louisiana observes case 2 waters [11], with contributions from the Atchafalaya River resulting in varied, biologically active waters with significant sediment concentrations. The WaveCIS site is jointly maintained by the Naval Research Laboratory and Louisiana State University and has an extensive history of observations. Of the three sites studied, the WaveCIS site has the most dynamic water type.

The Venise site was chosen as a mixed-water, predominantly case-1 region [12], because of the extensive history of SeaPRISM measurements at that site, with routine calibration of the primary instrument and a well-calibrated instrument put into place while the primary one is being re-calibrated. Further, the water conditions off the coast of Italy are interesting and varied; at times, there is diverse biology in the water causing changes in space-observed ocean color at multiple wavelengths. These changes can add additional uncertainty to match-ups, and further understanding and research into those daily, in-water changes allows for better calibration and validation of space-borne sensors.

The MOBY observatory in the waters of Hawaii is the community standard ocean-color calibration site [13]. The stable, case-1 blue water site stands in stark contrast to the area covered by the WaveCIS sensor. The geographic position of MOBY also affords less wind and cloud cover due to mountains on three nearby Hawaiian islands [13]. Additionally, the MOBY observations are an entirely different type of measurement than those measured by the SeaPRISM. MOBY utilizes an underwater system taking water-leaving radiance at three discrete depths versus the above-water sun-tracking photometer radiometer of the SeaPRISM sites. The near-real-time, near-continuous record provided by the MOBY stations are rigorously calibrated [14].

Using established, informed protocols [15,16,17], the geographical coverage is broadened to assess different water types in order to assess these screening protocols. The statistics, mean ratio, R-squared, and slope of results from the screening protocols at the WaveCIS site and Venise SeaPRISM, as well as the frequency and distribution of absolute percent error across the data set, are examined. Once the screening protocols employed at these AERONET-OC sites are studied, the protocols are applied at MOBY, the standard calibration site [18]. The hyper-spectral water-leaving radiance at MOBY is normalized and compared against the satellite-derived normalized water-leaving radiance.

The satellite data are constrained by default to exclude any pixels that have the following quality flags: cloud contamination, high top of atmosphere radiance, land, or atmospheric correction failure. Once the processing algorithms determine one of these conditions exists a bad data value is set for that pixel, and no normalized water-leaving radiance or other derived products are available. Any pixels that are thus flagged are excluded and unavailable in SAVANT match-up results. All pixels that pass these standards are ingested into SAVANT for match-up purposes where the user is able to apply additional constraints (Figure 2). Each pixel and its various meta-data are stored for extraction and filtration in the match-up process. The meta-data includes quality flags, as well as APS software version and sensor processing version (accounting for changes in vicarious validation independent of APS software changes). Location, time, and date information, and other data required for match-ups to proceed are also present.

The AERONET-OC data are processed and made available by NASA, and the SAVANT software pulls down all available level 1.5 and 2.0 data. Previously acquired data is refreshed regularly to maintain a direct correlation between SAVANT holdings and any corrections NASA may make. The initial assessment utilizes WaveCIS data [6]. The Venise data are provided and maintained by Giuseppe Zibordi. Quality-controlled NOAA MOBY data are provided by NOAA Star.

Two match-up protocols with differing constraints, ’permissive’ and ‘strict’, are examined. The permissive match-up protocol does not involve further quality flag or angle constraints, but it does require that all visible wavelengths pass the following tests: at least 50% (Figure 2f) of visible light normalized water-leaving radiance pixels (in the case of a 5 × 5 box; this means 13 or more pixels) must have a positive radiance value and are limited to a maximum value of 3 (Figure 2d), and the chlorophyll value must be valid. The pixel that contains the in situ station must also be valid, and the time difference between satellite overpass and in situ measurement must be less than three hours. The SeaPRISM windspeed must also be less than eight meters per second and the SeaPRISM aerosol optical thickness cannot exceed a value of 0.2 (Figure 2c).

The strict constraints limit the solar zenith angle and sensor zenith using the high solar zenith (HISOLZEN) and high sensor zenith (HISATZEN) flags. This is consistent both with NASA and NOAA’s protocols for match-ups [15,16]. These constraints also incorporate additional quality flags for high glint (HIGLINT), stray light (STRAYLIGHT), navigation failure (NAVFAIL), and low water-leaving radiance (LOWLW). These protocols have been previously observed to reduce the number of match-ups, while improving the overall statistics [6]. The summary of protocols can be seen in

Table 1. Exclusion criteria for the permissive and strict protocols.

Protocol	Permissive	Strict
Box Size	5 × 5 and 1 × 1	5 × 5 and 1 × 1
Time Difference	+/−3 h, +/−1 h, +/−0.5 h	+/−3 h, +/−1 h, +/−0.5 h
Flags	CLDICE, LAND, ATMFAIL, HILT (processing default)	Permissive and HIGLINT, STRAYLIGHT, NAVFAIL, LOWLW
nLw	0–3	0–3
Windspeed	≤8 ms−1	≤8 ms−1
AOT	≤0.2	≤0.2
in situ Pixel Valid	Yes	Yes
Solar Zenith	Not restricted	≤75°
Sensor Zenith	Not restricted	≤60°
Additional Requirements	Chlorophyll, all nLw wavelengths valid	Chlorophyll, all nLw wavelengths valid.

.

The products used for this analysis are the normalized water-leaving radiance from satellite and the MOBY water-leaving radiance (Lw1) using the top (in-water at a depth of 1.5 m [13]) and middle (in-water at a depth of 5 m [13]) arms of the buoy. Those data are then normalized [19]. The SeaPRISM water-leaving radiance is Lwn f/Q, which is BRDF-corrected normalized water-leaving radiance [3], noted as the instruments also provide a non-BRDF corrected product. As these are different methods of in situ observation, differences in the match-ups are expected [20]. The statistics examined are the regression slopes or the rate of change between in situ and satellite data; the coefficient of variation (R²) or goodness of fit; and the mean ratio (the total of the in situ and satellite match-up ratios divided by the number of match-ups for a given match-up criteria). Additionally, the bias and mean absolute error (MAE), when comparing the permissive and strict datasets, are analyzed [21].

During the course of this research the spatial resolution of the sensor was also considered. The standard practice is a 5 × 5 box with the center pixel containing the in situ station. The statistics of this 5 × 5 box are then compared against the in situ data, while also being used for quality control. A high standard deviation from the mean can indicate outliers in the data. This 5 × 5 box also helps to account for changes in the water between in situ and satellite measurements. The standard time window of +/−3 h allows for large change in the water, ranging from biological processes through the course of the day, movement of sediment and other inorganic matter, and variations in light in the water due to time of day. Regional, coastal calibration for SNPP VIIRS has been examined using SeaPRISM sites previously and methodically constraining the data has been shown to be of importance in optically complex waters [22]. The impact of time on a variable water column is considered, with a focus on Venise due to the potential influence of nearby Italian rivers. In order to better quantify the changes in the water, the difference between the spatial resolution of a 5 × 5 box and the in situ pixel in it is examined in addition to examining discrete time difference windows (+/−3 h, +/−1 h, and +/−0.5 h).

3. Results

3.1. WaveCIS AeroNET Site

The initial analysis was focused on the WaveCIS sensor in the Northern Gulf of Mexico (

Table 2. Strict and permissive results at WaveCIS. June 2014–May 2019 data. Values closest to one in bold, for slope, R², and mean ratio. Lowest values in bold for bias and mean absolute error.

Protocol/Wavelength	Match-Ups	Slope	R2	Mean Ratio	Bias	MAE
5 × 5 Permissive @ 410 nm	235	0.8918	0.4667	1.0587	−0.006	0.1078
1 × 1 Permissive @ 410 nm	236	0.9012	0.4562	1.0939	0.0017	0.1093
5 × 5 Strict @ 410 nm	82	0.9672	0.5251	1.0921	0.0106	0.1108
1 × 1 Strict @ 410 nm	84	0.9957	0.5348	1.1201	0.0214	0.1074
5 × 5 Permissive @ 443 nm	235	0.8803	0.7528	0.9174	−0.0579	0.1093
1 × 1 Permissive @ 443 nm	236	0.8842	0.7423	0.9377	−0.0525	0.1115
5 × 5 Strict @ 443 nm	82	0.9105	0.7226	0.9517	−0.0396	0.1113
1 × 1 Strict @ 443 nm	84	0.9173	0.7184	0.9696	−0.0316	0.1096
5 × 5 Permissive @ 486 nm	235	1.0207	0.9138	1.007	0.0077	0.099
1 × 1 Permissive @ 486 nm	236	1.0217	0.9038	1.0169	0.0109	0.1054
5 × 5 Strict @ 486 nm	82	1.041	0.8895	1.0436	0.0259	0.1089
1 × 1 Strict @ 486 nm	84	1.0467	0.8734	1.069	0.0377	0.1142
5 × 5 Permissive @ 551 nm	235	0.9822	0.9426	0.9617	−0.0253	0.0915
1 × 1 Permissive @ 551 nm	236	0.9826	0.9314	0.9693	−0.0223	0.0956
5 × 5 Strict @ 551 nm	82	0.9974	0.9304	1.0028	−0.0018	0.0947
1 × 1 Strict @ 551 nm	84	0.9991	0.9166	1.0232	0.007	0.1046
5 × 5 Permissive @ 671 nm	235	0.9007	0.8919	0.8586	−0.0258	0.0369
1 × 1 Permissive @ 671 nm	236	0.8906	0.8611	0.8679	−0.0259	0.0389
5 × 5 Strict @ 671 nm	82	0.9057	0.8558	0.9142	−0.0183	0.0353
1 × 1 Strict @ 671 nm	84	0.9173	0.8768	0.9418	−0.0147	0.0331

). During this analysis, the strict protocol (Table 1) was determined to have limited the statistical advantage over the more robust match-ups of the permissive protocol. Allowing for a more representative sample in the green case-2 [23] waters results in the match-ups agreeing more closely, when considering the goodness of fit (R²), but other statistics are also impacted by restricting the data when considering the slope and mean ratio. However, the 5 × 5 permissive box has values closer to one across the analyzed statistics for the 486 band. The lower bias values correlate with ‘closer-to-one’ mean ratio in several bands. The permissive protocol has a lower mean absolute error in the 443 nm, 486 nm, and 551 nm. Both the bias and mean absolute error are the lowest in the strict 1 × 1 for the 671 nm.

When the 5 × 5 data are graphed together (Figure 3), the strict protocol shows a tighter grouping in the data. Several outlying match-ups are removed by the strict match-up constraints. In previous tests of the individual match-up criteria, the stray-light flag was responsible for the greatest reduction in the available match-ups [6], and in this case we have lost approximately 65% of the match-ups when applying the strict criteria.

When applying the permissive and strict constraints to only the center pixel, a similar shift is observed (Figure 4). The range of the data is much closer to the 1:1 line in the strict constraint box.

3.2. Venise AERONET Site

The permissive results at the Venise site result in nearly as many match-ups over the same period as the WaveCIS results (Figure 5), and the statistics are comparable and within the expected range. The slopes of each trend-line are close to a 1:1 line.

Comparing the 5 × 5 satellite box to the center pixel only, the match-ups produce similar statistical results (

Table 3. Venise data, June 2014–May 2019. Permissive data protocol at 5 × 5 and 1 × 1 box sizes. Values closest to one in bold, for slope, R², and mean ratio. Lowest values in bold for bias and mean absolute error.

Protocol/Wavelength	Match-Ups	Slope	R2	Mean Ratio	Bias	MAE
5 × 5 @ 410 nm	281	1.077	0.4416	1.3317	0.1136	0.2131
1 × 1 @ 410 nm	280	1.0578	0.4156	1.3567	0.109	0.2181
5 × 5 @ 443 nm	281	0.9989	0.7714	1.0759	0.027	0.1586
1 × 1 @ 443 nm	280	0.9965	0.758	1.0847	0.0268	0.167
5 × 5 @ 486 nm	281	1.1562	0.8777	1.1926	0.1899	0.2279
1 × 1 @ 486 nm	280	1.1525	0.8708	1.1955	0.1897	0.2331
5 × 5 @ 551 nm	281	1.0426	0.9067	1.0689	0.06	0.1258
1 × 1 @ 551 nm	280	1.038	0.9112	1.0733	0.0603	0.1279
5 × 5 @ 671 nm	281	1.0414	0.7744	1.5243	0.023	0.0431
1 × 1 @ 671 nm	280	1.0161	0.7149	1.7937	0.0247	0.0482

). The greatest change in both the goodness of fit and the mean ratio occur in the 671 nm, both showing more favorable numbers for the 5 × 5 box; the goodness of fit improves by 8.3% and the mean ratio improves by 15%. The bias is reduced by 6.88%, and the mean absolute error is reduced by 10.6%. These changes are significant for bands with less signal in the water-leaving radiance, as those bands are more susceptible to noise and out-of-band response. The mean ratio is improved for the other wavelengths by using a 5 × 5 box, but the scale of improvement is lower.

In order to test the change seen in the 5 × 5 box over the center pixel, as an aspect of changing water conditions (especially at 443 nm) (Figure 6), we examine the difference between the center pixel, with our standard +/−3 h time window compared to the center pixel with a tighter +/−1 h time period and a +/−0.5 h time period (

Table 4. Venise permissive protocol, June 2014−May 2019. Center (in situ) pixel only, at +/−3, 1, and 0.5 h. Values closest to one in bold, for slope, R², and mean ratio. Lowest values in bold for bias and mean absolute error.

Protocol/Wavelength	Match-Ups	Slope	R2	Mean Ratio	Bias	MAE
+/−3 h, 410 nm	281	1.0578	0.4156	1.3567	0.109	0.2181
+/−1 h, 410 nm	229	1.0783	0.431	1.3813	0.1222	0.2189
+/−0.5 h, 410 nm	183	1.0524	0.3705	1.3548	0.1141	0.2283
+/−3 h, 443 nm	281	0.9965	0.758	1.0847	0.0268	0.167
+/−1 h, 443 nm	229	1.0008	0.7633	1.0966	0.0357	0.1664
+/−0.5 h, 443 nm	183	0.9847	0.7173	1.087	0.0268	0.1765
+/−3 h, 486 nm	281	1.1525	0.8708	1.1955	0.1897	0.2331
+/−1 h, 486 nm	229	1.1594	0.871	1.2074	0.1983	0.2349
+/−0.5 h, 486 nm	183	1.1507	0.84	1.1995	0.1895	0.2405
+/−3 h, 551 nm	281	1.038	0.9112	1.0733	0.0603	0.1279
+/−1 h, 551 nm	229	1.0369	0.9102	1.0771	0.0613	0.1227
+/−0.5 h, 551 nm	183	1.0265	0.8856	1.0721	0.0527	0.1308
+/−3 h, 671 nm	281	1.0161	0.7149	1.7937	0.0247	0.0482
+/−1 h, 671 nm	229	1.0011	0.6668	1.7658	0.0246	0.0478
+/−0.5 h, 671 nm	183	0.8564	0.4715	1.7587	0.0185	0.0522

). This helps to account for the changes to the water mass over the course of a day—the changes occurring between the in situ measurement and satellite overpass. When excluding data greater than an hour in time difference, using this protocol, a loss of 62 (+/−1 h) and 109 (+/−0.5 h) match-ups are observed, decreasing our match-up numbers to 229 and 183 match-ups, respectively. When examining an in situ pixel match-up, the statistics favor limiting the difference in observation times to +/−1 h or less, although that change is not statistically significant across all wavelengths when using the permissive protocol.

When examining the strict protocol (Figure 7), the statistical change is similar to the permissive protocol. Both the 5 × 5 statistical box and the center pixel have strengths (

Table 5. Venise data, strict protocol. June 2014–May 2019. Values closest to one in bold, for slope, R², and mean ratio. Lowest values in bold for bias and mean absolute error.

Protocol/Wavelength	Match-Ups	Slope	R2	Mean Ratio	Bias	MAE
5 × 5 @ 410 nm	94	1.1191	0.5439	1.3365	0.1421	0.1999
1 × 1 @ 410 nm	96	1.093	0.458	1.3471	0.1374	0.2058
5 × 5 @ 443 nm	94	0.995	0.8265	1.0702	0.0242	0.144
1 × 1 @ 443 nm	96	0.9966	0.7887	1.0748	0.03	0.1563
5 ×5 @ 486 nm	94	1.18	0.9152	1.2253	0.2294	0.2459
1 × 1 @ 486 nm	96	1.2001	0.9112	1.2389	0.2417	0.2568
5 × 5 @ 551 nm	94	1.0468	0.9026	1.0714	0.0683	0.1313
1 × 1 @ 551 nm	96	1.0537	0.891	1.0847	0.0785	0.1361
5 × 5 @ 671 nm	94	1.055	0.8011	1.4529	0.0253	0.0437
1 × 1 @ 671 nm	96	1.0823	0.7888	2.0668	0.358	0.05

), yet, overall, the 5 × 5 box performs better than the center pixel. This becomes especially evident in the higher wavelengths (486 nm, 551 nm, and 671 nm), where the 5 × 5 box is improved, for the slope, goodness of fit, and mean ratio, with less bias and mean absolute error in all three bands. The difference in the goodness of fit is most improved using the 5 × 5 box, for the following several wavelengths: 18.75% (410 nm), 4.8% (443 nm), and 1.6% (671 nm). The mean absolute error is also lower for the 5 × 5 box at both 410 nm and 443 nm, and the bias is lower for the 5 × 5 box at 443 nm as well. Only 410 nm has a lower bias for the 1 × 1 box. The bias is lowest for the 1 × 1 strict match-up set, only for the 410 nm band, and the mean absolute error is the lowest in all the wavelengths for the 5 × 5 strict match-up set. Given this trend and the previous temporal restrictions, the temporal differences were examined again, with the strict protocol and working closer in time.

When the time is decreased from +/−3 h to +/−1 h, the results decrease from 96 match-ups to 75., and decreasing to +/−0.5 h results in 61 match-ups. The results are a reflection of the uncertainty reduction using the remaining constraints in the strict protocol. The blue 410 nm channel has the lowest bias and mean absolute error for the +/−3 h window (

Table 6. Venise data, strict protocol. June 2014−May 2019. Center (in situ) pixel only, at +/−3, 1, and 0.5 h. Values closest to one in bold, for slope, R², and mean ratio. Lowest values in bold for bias and mean absolute error.

Protocol/Wavelength	Match-Ups	Slope	R2	Mean Ratio	Bias	MAE
+/−3 h, 410 nm	96	1.093	0.458	1.3471	0.1421	0.1999
+/−1 h, 410 nm	75	1.1184	0.5336	1.3545	0.148	0.2065
+/−0.5 h, 410 nm	61	1.0979	0.4764	1.3732	0.1484	0.2192
+/−3 h, 443 nm	96	0.9966	0.7887	1.0748	0.0242	0.144
+/−1 h, 443 nm	75	0.9911	0.8322	1.0745	0.0229	0.1433
+/−0.5 h, 443 nm	61	0.9795	0.8082	1.0767	0.0173	0.154
+/−3 h, 486 nm	96	1.2001	0.9112	1.2389	0.2294	0.2459
+/−1 h, 486 nm	75	1.1794	0.9216	1.2253	0.2292	0.246
+/−0.5 h, 486 nm	61	1.175	0.9263	1.2208	0.2216	0.2391
+/−3 h, 551 nm	96	1.0537	0.891	1.0847	0.0683	0.1313
+/−1 h, 551 nm	75	1.0434	0.9261	1.0708	0.065	0.1177
+/−0.5 h, 551 nm	61	1.0394	0.928	1.0737	0.0618	0.1112
+/−3 h, 671 nm	96	1.0823	0.7888	2.0668	0.0253	0.0437
+/−1 h, 671 nm	75	1.0319	0.7785	1.4574	0.0231	0.0409
+/−0.5 h, 671 nm	61	1.0762	0.7764	1.5449	0.0272	0.0406

), but the statistics are the lowest for the +/−1 h and +/−0.5 h windows across the remaining wavelengths. Overall, when grouped together (+/−0.5 and 1 h), the more temporally restrictive match-ups perform better across our statistical analysis than the +/−3 h default. Once again, the changes are not necessarily statistically significant enough to warrant excluding potentially valid match-ups.

The strict protocol follows the results at WaveCIS, with a significant reduction (~67%) in the match-ups (Figure 8). However, the statistics are impacted across most of our analysis for each wavelength (

Table 7. Venise 5 × 5 permissive versus 5 × 5 strict. June 2014–May 2019. Values closest to one in bold, for slope, R², and mean ratio. Lowest values in bold for bias and mean absolute error.

Protocol/Wavelength	Match-Ups	Slope	R2	Mean Ratio	Bias	MAE
Permissive, 410 nm	281	1.077	0.4416	1.3317	0.1136	0.2131
Strict, 410 nm	94	1.1191	0.5439	1.3365	0.1421	0.1999
Permissive, 443 nm	281	0.9989	0.7714	1.0759	0.027	0.1586
Strict, 443 nm	94	0.995	0.8265	1.0702	0.0242	0.144
Permissive, 486 nm	281	1.1562	0.8777	1.1926	0.1899	0.2279
Strict, 486 nm	94	1.18	0.9152	1.2253	0.2294	0.2459
Permissive, 551 nm	281	1.0426	0.9067	1.0689	0.06	0.1258
Strict, 551 nm	94	1.0468	0.9026	1.0714	0.0683	0.1313
Permissive, 671 nm	281	1.0414	0.7744	1.5243	0.023	0.0431
Strict, 671 nm	94	1.055	0.8011	1.4529	0.0253	0.0437

). The permissive protocol performs the closest to one across the slope analysis in all the wavelengths, with some differences more significant than others, as the permissive protocol improves by the most at 410 nm, with an improvement of 3.8%, and the least at 443 nm, with a difference of only 0.4%. The goodness of fit improves for the strict protocol in every wavelength, except 551 nm, where the 5 × 5 box improves by only 0.5%, but the strict protocol goodness of fit for 410 nm is an 18.8% improvement. The bias is the lowest for the permissive protocol across all wavelengths, except the 443 nm, and the difference at that wavelength is only 1.2%.

3.3. MOBY

For the MOBY–VIIRS match-ups, the permissive protocol results in fewer match-ups over the same time period as the previous areas of study (Figure 9), but this is expected. MOBY experienced several outages, due to severe weather conditions and ships damaging the buoy. With a smaller data set, the statistics remain quite good. The mean ratio for the blue and green channels is quite close to one, and the slope is also very close to one across all the blue wavelengths. MOBY is in very clear blue waters, so the green and red channels have less signal, and that is reflected in the statistics (

Table 8. MOBY data, June 2014–May 2019. Permissive data protocol at 5 × 5 and 1 × 1 box sizes. Values closest to one in bold, for slope, R², and mean ratio. Lowest values in bold for bias and mean absolute error.

Protocol/Wavelength	Match-Ups	Slope	R2	Mean Ratio	Bias	MAE
5 × 5, 410 nm	232	1.0207	0.3314	1.0762	−0.0176	0.272
1 × 1, 410 nm	239	1.0173	0.325	1.0778	−0.0212	0.2795
5 × 5, 443 nm	232	0.9994	0.2468	1.043	0.0343	0.2194
1 × 1, 443 nm	239	0.9981	0.2479	1.0444	0.0337	0.2221
5 × 5, 486 nm	232	1.0238	0.0788	1.0607	−0.0052	0.1328
1 × 1, 486 nm	239	1.0223	0.0869	1.0623	−0.0053	0.1348
5 × 5, 551 nm	232	0.8743	0.0002	0.9738	−0.0227	0.0536
1 × 1, 551 nm	239	0.8655	0.001	0.9972	−0.0221	0.0584
5 × 5, 671 nm	232	0.3015	0.0004	0.5797	−0.0158	0.0159
1 × 1, 671 nm	239	0.2008	0	0.6233	−0.0167	0.0169

).

It is worth noting that for both the satellite and in situ data, the range of each discrete wavelength of normalized water-leaving radiance falls within a fairly well-defined upper and lower bound. This is visually apparent in the plots, and represents the strength of the light signal reflected from the waters around the sensor. The 671 nm has the smallest bounds, increasing slightly in the 551 nm, and growing larger from 486 nm through to 410 nm. This is an expected reflection of the case-1 waters, and differentiates them from the case-2 waters at WaveCIS.

Comparing the 5 × 5 statistical box to the center pixel reveals no trend in our statistics. There are a few changes, but these changes are wavelength-dependent. This is to be expected, due to the location of the MOBY sensor; the spatial stability is a result of the homogeneous waters, allowing for MOBY’s use in calibrating satellites, regardless of their spatial resolution. The bias is lower for the 410 nm, 486 nm, and 671 nm for the 5 × 5 box, while the 5 × 5 box also has a lower mean absolute error across all the wavelengths. Due to this stability in the water column, we forewent including the temporal analysis at MOBY.

The strict protocol (

Table 9. MOBY data, June 2014−May 2019. Strict data protocol at 5 × 5 and 1 × 1 box sizes. Values closest to one in bold, for slope, R², and mean ratio. Lowest values in bold for bias and mean absolute error.

Protocol/Wavelength	Match-Ups	Slope	R2	Mean Ratio	Bias	MAE
5 × 5, 410 nm	68	0.9983	0.3727	1.0538	−0.0599	0.2782
1 × 1, 410 nm	69	0.9953	0.3856	1.0409	−0.0752	0.2604
5 × 5, 443 nm	68	0.99	0.271	1.0306	0.016	0.2175
1 × 1, 443 nm	69	0.9902	0.2891	1.022	0.0096	0.1987
5 × 5, 486 nm	68	1.0067	0.0896	1.0439	−0.0267	0.129
1 × 1, 486 nm	69	0.9975	0.0816	1.0307	−0.0375	0.1294
5 × 5, 551 nm	68	0.9042	0.015	0.9947	−0.0138	0.0483
1 × 1, 551 nm	69	0.8739	0.0153	1.01	−0.0177	0.0529
5 × 5, 671 nm	68	0.2411	0.0021	0.6907	−0.0139	0.0148
1 × 1, 671 nm	69	0.105	0.0002	0.8026	−0.0181	0.0193

) was examined. In both the permissive and strict protocols, the 5 × 5 box (Figure 9c) impacts 671 nm and 551 nm channels at MOBY more than the 486 nm, 443 nm, and 410 nm, suggesting that the limited signal in these wavelengths results in more uncertainty when focusing on a narrower spatial region. The bias is the lowest for the 5 × 5 strict box across all the wavelengths, except for 443 nm. The mean absolute error is lower for the blue channels of 410 nm and 443 nm using the 1 × 1 box, but the 5 × 5 box has a lower mean absolute error for the remaining bands. While, overall, the 1 × 1 (not pictured) does better, the differences are, in some cases, less than a few tens of a percent or smaller.

When comparing the 5 × 5 permissive protocol to the 5 × 5 strict protocol (

Table 10. MOBY data. Permissive versus strict, 5 × 5 statistical box. June 2014–May 2019. Values closest to one in bold, for slope, R², and mean ratio. Lowest values in bold for bias and mean absolute error.

Protocol/Wavelength	Match-Ups	Slope	R2	Mean-Ratio	Bias	MAE
Permissive, 410 nm	232	1.0207	0.3314	1.0762	−0.0176	0.272
Strict, 410 nm	68	0.9983	0.3727	1.0538	−0.0599	0.2782
Permissive, 443 nm	232	0.9994	0.2468	1.043	0.0343	0.2194
Strict, 443 nm	68	0.99	0.271	1.0306	0.016	0.2175
Permissive, 486 nm	232	1.0238	0.0788	1.0607	−0.0052	0.1328
Strict, 486 nm	68	1.0067	0.0896	1.0439	−0.0267	0.129
Permissive, 551 nm	232	0.8743	0.0002	0.9738	−0.0227	0.0536
Strict, 551 nm	68	0.9042	0.015	0.9947	−0.0138	0.0483
Permissive, 671 nm	232	0.3015	0.0004	0.5797	−0.0158	0.0159
Strict, 671 nm	68	0.0021	0.0021	0.6907	−0.0139	0.0148

), the goodness of fit improves when using the strict protocol. This is the most significant in the 551 nm and 671 nm. The value of the change is small, but statistically significant, for these low values. There are changes in the other wavelengths, but none as clearly significant as these; the initial goodness of fit of 0.0002 (551 nm) and 0.0004 (671 nm) are not meaningfully different than a value of zero. This improvement is especially important in these bands with low signal, as the waters here are dominated by the other bands.

4. Discussion

Our SAVANT software contains the data from MOBY and multiple AERONET-OC sensors in a variety of water types. Additionally, the data are processed for various satellite sensors, including OLCI Sentinel 3A and Sentinel 3B at 300 m, and NOAA-20 VIIRS (at the same 750 m as SNPP VIIRS). Future research around these sensors will broaden the scope of our understanding. In addition to being a much higher resolution than VIIRS, the OLCI sensors also pass-over earlier in the day than either of the VIIRS sensors. Biological processes, sun and sensor angles, and resolution combined present interesting opportunities for research at these locations. Gathering these continuing datasets for calibration and validation will also have a supporting role in continued climate science [24], as it will enable the automatic examination and graphing of data over several years; a multi-year time series of calibrated data could be used to show the trends caused by climate change.

SAVANT does have the following limitations: It serves as a data aggregation and display system. Data can be downloaded in comma-separated files, and also displayed as visual graphs with the user-selected exclusion criteria, but the system does not have the capability to recommend vicarious gains or other changes to satellite processing. The system was also designed around the fixed locations of SeaPRISM and MOBY, and does not currently support moving in situ measurements, such as those taken by a scientist on a ship. The back-end database was designed to be forward thinking, and mobile readings could be added in the future; the system was designed with as many of the metadata fields, in order to be open-ended and allow flexibility. Initially, it was only ingesting and storing a single SeaPRISM and a single satellite at that SeaPRISM site, but in such a way that it was accommodating additional satellites and in situ platforms, and locations only required changes to the ingest scripts to get the new data sources. The goal of SAVANT is not merely comparing satellites to SeaPRISM and MOBY, but any comparison of ocean color data that can be ingested into the database.

Across three regions, our results show that the strict constraints result in an improvement in the statistical values, while the permissive constraints allow for a more robust statistical representation of the conditions in the water. When examining the case-2 waters at WaveCIS, and the potentially complex, predominantly case-1 waters at Venise, the 5 × 5 box gave a better representation of the water surface and compensated for temporal changes within the allowed +/−3 h overpass window. The time difference between the satellite measurement and that of the in situ sensor produced slight statistical improvement when allowing only measurements taken within a 0.5–1 h difference. While significantly reducing the pool of potential match-ups, the value of this statistical improvement should be considered, especially in waters with the potential for rapid changes during the course of a day. The most significant increase in validation statistics, when applying the strict constraints, was observed at MOBY. The 671 nm and 551 nm bands were the most improved, as the signal in those channels is much lower in the steady case-1 waters, and they are therefore more impacted by issues caused by stray light, high sensor or solar angles, and low water-leaving radiance values.

There are remaining uncertainties, and these should be addressed in future research. For the purposes of this study, only the SNPP VIIRS sensor was tested, and the analysis of other space-borne ocean color sensors require further refinement. Differences in the spectral wavelengths, geometric resolution, and in situ site overpass times may impact the quality of the match-ups for other sensors. While chlorophyll data are produced by the ocean color processing, currently neither MOBY nor the SeaPRISM sites offer near-real-time in-water biology measurements. Utilizing in situ data can lead to improvements in the satellite algorithms for derived properties [25], and the understanding of satellite error when examining IOPs [26]. Further, while MOBY data are hyper-spectral and the closest wavelength to the satellite sensor is selected for match-up purposes, the SeaPRISM data are multi-spectral. The wavelengths are close to, but not exactly centered on, the same wavelengths as multi-spectral satellite data. While this might be acceptable for quick-look comparisons, for calibration and validation purposes the selected SeaPRISM data must be band-shifted [27,28,29], but SAVANT only stores the multi-spectral SeaPRISM data at their original wavelengths.

One focus area that would strengthen the statistical analysis and correlation in future research is to address the seasonal variability in the optically complex water. This study examined several years (June 2014 through to May 2019) of satellite and in situ comparisons. However, when examining the time-series plots in the optically complex water around WaveCIS (Figure 10), we see that there are peaks and troughs in both the satellite and in situ measurement that correlate with seasonal changes, which other studies have examined in optically complex waters [30]. This is one area in which expanding the use of tools such as SAVANT, to examine seasonable variability across different regions, could present a more robust picture of the processes impacting satellite validation statistics; the multi-year time series across varied water types could be examined, looking at changes and trends in the seasons. Waters are not just changing during the course of the day, but experience changes, both seasonally and long term, as a result of climate change. Utilizing tools such as SAVANT, with expanding data sets covering varied time series, represents one approach to tracking climate change [31].

5. Conclusions

With the present study, we have demonstrated the utility of SAVANT for the validation of satellite data, and avenues for the future expansion of the automated production of validation graphs and statistical information about match-ups has been demonstrated. Across all three regions, the permissive protocol allows for a more comprehensive statistical picture. However, the strict protocol provides better statistical results across all the regions for the traditional statistics of slope, goodness of fit, and mean ratio, while the bias and mean absolute error present more of a mixed picture. These findings must be weighed against the benefit of allowing the more rapidly changing waters, with diverse biological impacts on remote sensing of ocean color, to be compared, using a more relaxed, inclusive approach that allows for broad, representative match-ups between in situ and satellite measurements, especially with evidence of less systematic bias and absolute error. The difference between the number of available match-ups is consistent with other studies, and is to be expected [32].

Allowing for a 5 × 5 box analysis is favorable in the optically complex water, for some wavelengths, and makes less difference in the blue water. This is because of the changing nature of case-2 waters. The optically complex water can be further impacted by restricting the match-up time window down to 0.5–1 h between the satellite and in situ measurement, which further accounts for the rapid changes in such waters. However, again, reducing the number of match-ups increases the evidence of bias and mean absolute error.

In the pristine blue waters off Hawaii, the strict protocol results in a significant statistical change across the 551 nm and 671 nm wavelengths. This site was chosen based on the stability of the water color for ocean color calibration. The additional quality flag and zenith angle constraints clearly provide more benefit in the case-1 waters, where the signal is more susceptible to the influence of angles and stray light.

Future research into the variability caused by biology, potential differences in various satellite sensors (differing spectral wavelengths, geometric resolution, and overpass time), as well as against additional in situ sensors and sensor types, is recommended. Using SAVANT to examine the expected seasonal change [33] is another area of future research. Additionally, an expansion of the statistics used to examine match-ups will allow for a more complete and robust comparison of the impacts of different protocols [21]. Additional statistics and study in the MOBY area are especially recommended due to the performance of goodness of fit values and the general statistics of the 671 nm wavelength. The addition of support for mobile in situ stations to allow for validation using shipboard or drone-produced [34] measurements is another area of improvement needed as SAVANT moves forward. The addition of other in situ data sources would also allow for an expansion of which products are compared, which is important, as products such as backscattering are important “for the study of ocean biology and oceanic carbon estimations” [35]. The framework of SAVANT allows for the examination of multiple statistics across varied regions, with a range of potential future research opportunities.