In-Flight Retrieval of SCIAMACHY Instrument Spectral Response Function

Abstract

The instrument Spectral Response Function (ISRF) has a strong impact on spectral calibration and the atmospheric trace gases retrievals. An accurate knowledge or a fine characterization of the ISRF shape and its FWHM (Full width at half maximum) as well as its temporal behavior is therefore crucial. Designing a strategy for the characterization of the ISRF both on ground and in-flight is critical for future missions, such as the spectral imagers in the Copernicus program. We developed an algorithm to retrieve the instrument ISRF in-flight. Our method uses solar measurements taken in-flight by the instrument to fit a parameterized ISRF from on ground based calibration, and then retrieves the shape and FWHM of the actual in-flight ISRF. With such a strategy, one would be able to derive and monitor the ISRF during the commissioning and operation of spectrometer imager missions. We applied our method to retrieve the SCIAMACHY instrument ISRF in its different channels. We compared the retrieved ones with the on ground estimated ones. Besides some peculiarities found in SCIAMACHY channel 8, the ISRF results in other channels were relatively consistent and stable over time in most cases.

1. Introduction

One of the key components that is needed to perform spectral calibration and to proces Level 1 and Level 2 data is the instrument spectral response function (ISRF) which is sometimes referred to as a slit function. Additionally, the lower the accuracy of the ISRF is, the higher the error on the retrieval is, i.e., larger systematic errors of the retrieved quantities. As a matter of fact, one of the origins of the instrument introduced errors is insufficient knowledge of the ISRF during the mission. It is one of the most important parameters for level 2 retrieval. It is additionally crucial to probe any variations over time or during different observing cases (e.g., [1]). Some studies have in fact investigated the ISRF impact on data retrievals and calibration for different instruments in-flight, like GOME-2 (e.g., [2,3]). Furthermore, another study [4] reported that the retrieved in-flight ISRF in the UV improved the level 1 data in the period of 2003–2011.

Requirements for achieving high accuracies of retrieved ISRF(s) in-flight and for probing its stability over time are becoming common for the next generation instruments. We developed a method using mathematical functions and algorithms for the retrieval of the instrument ISRF in-flight. The motivation was to calculate it with the best accuracy possible on the fly. We applied our method for the retrieval of SCIAMACHY (SCanning Imaging Absorption spectroMeter for Atmospheric CHartographY) instrument’s slit functions in-flight. We developed a model that used SCIAMACHY solar measurements to obtain its ISRF on a daily basis over its lifetime.

SCIAMACHY is a scanning nadir and limb spectrometer covering the wavelength range from 212 nm to 2386 nm in eight channels (

Table 1. The SCIAMACHY instrument channels. The spectral resolution in column 3 is also the FWHM (Full width at half maximum) of the corresponding ISRF Instrument Spectral Response Function).

Channel Number	Spectral Range (nm)	Spectral Resolution (nm)
1	214–334	0.24
2	300–412	0.26
3	383–628	0.44
4	595–812	0.48
5	773–1063	0.54
6	971–1773	1.48
7	1934–2044	0.22
8	2259–2386	0.26

). It is a joint project of Germany, the Netherlands and Belgium and was launched in February 2002 on the ENVISAT platform. Designed for a lifetime of five years, it performed measurements until April 2012, when the contact to the satellite was lost. The project is now in phase F. SCIAMACHY was designed to measure column densities and vertical profiles of trace gas species in the mesosphere, in the stratosphere and in the troposphere [5]. It can detect O${}_3$, H${}_2$CO, SO${}_2$, BrO, OClO, NO${}_2$, H${}_2$O, CO, CO${}_2$, CH${}_4$, N${}_2$O, O${}_2$, and can provide information about aerosols and clouds. In addition to the spectrally resolved measurements of the radiance reflected from the Earth’s atmosphere, the polarization of the incoming light is measured with seven broadband sensors.

2. Materials and Methods

2.1. Background

We developed and ran the SCIAMACHY operational processor to retrieve both Level 1 and Level 2 data [6]. We extracted solar measurements (Figure 1) from the processed Level 1b data and used it as an input for our ISRF retrieval algorithm.

Ground based calibration measurement campaign OPTEC5 [7] provided an estimation of the ISRF shape in each of the SCIAMACHY science channels [8]. Furthermore, from the commissioning estimations, various mathematical functions were recommended (

Table 2. Recommended analytical functions for the ISRF function in seven channels of the SCIAMACHY instrument. Channel 7 was not included in the table as it was not analyzed in this study. The second column’s functions were retrieved from on ground measurements during the OPTEC5 campaign. The third column’s were obtained during the commissioning phase.

Channel Number	OPTEC5 Campaign	Commissioning Phase
1	Simple hyperbolic	Simple hyperbolic
2	Simple hyperbolic	Simple hyperbolic
3	Simple hyperbolic	Simple hyperbolic
4	Gaussian	Simple hyperbolic
5	Gaussian	Simple hyperbolic
6	Gaussian	Gaussian
8	Compound hyperbolic	Voigt

). We used both of these estimations as starting inputs for our simulations. Like our presented analysis, several studies had already tested various parameter functions to constrain the ISRF shape of different instruments like OMI [9] and OCO-2 [10].

2.2. Fitting the ISRF

We modeled the solar measurements by convolving highly resolved reference spectra [11,12] of the sun with a simulated ISRF function. The convolution is expressed like the following:

$$S_{fit}=S_{\odot }\otimes ISRF$$

(1)

where $S_{\odot }$ is the solar spectrum. After the convolution, the resulting spectra are pixel sampled to match SCIAMACHY wavelength axis in order to fit the modeled $S_{fit}$ against a SCIAMACHY solar measurement $S\left(\lambda \right)$, while the ISRF is the free component to be estimated. We retained the ISRF function that provided the best fit of the solar measurements (Equation (1)) in each channel.

Some regularization is usually necessary to solve such convolution type integrals, in our case the regularization is applied by assuming a specific parameter function (Table 2) for the ISRF in each case. For the fitting, we used the ${\chi }^2$ minimization method to retrieve the in-flight ISRF from each individual measurement (orbit) separately. This method is also sometimes used for the retrieval of the ISRF [13]. We calculated the ${\chi }^2$ of a fitted solar measurement from Equation (1), like:

$${\chi }^2=\sum _{i=1}^N\frac{{(S\left({\lambda }_i\right)-S_{fit}({\lambda }_i;x_M))}^2}{{\sigma }_i^2}$$

(2)

where N is the number of pixels per measurement in each tested orbit. The individual measurement per pixel is defined by $S\left({\lambda }_i\right)$ and its corresponding error by ${\sigma }_i$. The ISRF free-parameters (in Equations (3)–(7)) are defined by $x_M$, where M is the number of parameters per function. For each set of parameters $x_M$, we normalized the calculated ISRF function, i.e., equality to the unit of its integral, before simulating the measurement $S_{fit}$. Since the number of free parameters is small, we used the brute force search method, as it was more appropriate in our case and more reliable since it excludes local minima that might affect our results. We imposed a few constraints on each free parameter x by letting them vary between pre-defined values only, in order to be compatible with the physical and technical specifications of the existing estimated ISRFs of SCIAMCHY. Overall, the functions (Equations (3), (4), (6), (7)) width were initially constrained within the range $0.01\times$ FWHM and 5× FWHM (Table 1) depending on the channel, while their other free parameters were constrained based on assumptions from our knowledge of the instrument, as described below. We then searched the M-dimension parameter space grid with the highest reasonable resolution. These conditions allowed us to find the global minimum in this space. In each point of the grid, we computed the ${\chi }^2$ (Equation (2)). The best fit parameters occur at the grid point of the lowest ${\chi }^2$, and thus yielding the best fit ISRF function. We additionally deduced the corresponding confidence level of each fitted measurement.

The ISRF is parameterized by a mathematical function that we modeled for each tested channel. Thus in each channel, we selected the mathematical function that provided the best fit of all fitted solar lines (at various wavelengths) simultaneously. We chose the parameter functions based on the recommended ones from ground based and commissioning estimations (Table 2). Accordingly, we tested in channels 1 to 6 a Gaussian (Equation (3)) and a Hyperbolic function (Equation (4)). The Gaussian is defined in two wavelength regions like the following:

$$ISRF\left(\lambda \right)=\left\{\begin{array}{cc}a\times e^{-\frac{{(\lambda -{\lambda }_0)}^2}{2{C_1}^2}},\hfill & \mathrm{if}\lambda <{\lambda }_0.\hfill \\ a\times e^{-\frac{{(\lambda -{\lambda }_0)}^2}{2{C_2}^2}},\hfill & \mathrm{if}\lambda \ge {\lambda }_0.\hfill \end{array}\right.$$

(3)

where ${\lambda }_0$ is the central wavelength of the ISRF. $C_1$ was left as a free parameter. On the other hand, and despite being a free parameter, $C_2$ depended on the value of $C_1$. In fact, $C_2$ was constrained within 0.98 $\times C_1$ and 1.02 $\times C_1$ range for each $C_1$ value. The relation between these two parameters allowed the tracking of any asymmetry of the simulated ISRF function, by up to 2%. The hyperbolic function is defined like

$$ISRF\left(\lambda \right)=\frac{a}{{(\lambda -{\lambda }_0)}^4+c^2}$$

(4)

where the FWHM is determined by, but not equal to c.

Following the existing estimations and recommendations in channel 8, we tested both a Voigt (Equation (5)) and a compound hyperbolic function (Equation (7))

$$ISRF\left(\lambda \right)=G\left(\lambda \right)\otimes L\left(\lambda \right)$$

(5)

where G(λ) is a Gaussian function (Equation (3)) and L(λ) is a Lorentzian function as defined below (Equation (6)):

$$L\left(\lambda \right)=\frac{\gamma }{\pi ({\lambda }^2+{\gamma }^2)}$$

(6)

where the FWHM of the Lorentzian function is $2\times \gamma$. We defined the compound hyperbolic function like:

$$ISRF\left(\lambda \right)=\frac{a_1}{{(\lambda -{\lambda }_0)}^2+c^2}+\frac{a_2}{{(\lambda -{\lambda }_0)}^4+c^2}$$

(7)

In the simulation of each solar spectrum, we normalized the ISRF. We consequently fixed the value of a = 1 in Equations (3)–(5). We similarly fixed a${}_1$ = 1 in Equation (7), while a${}_2$ was left as a free parameter, but yet constrained within 0–10. To take into account any shifting impact on the ISRF, we left ${\lambda }_0$ [nm] as a free parameter, and constrained it within −5% and 5% of the pixel size in each channel. This allowed us to retrieve the ISRF in-flight whether it was slightly shifted or not.

2.3. SCIAMACHY Solar Measurements

We used all SCIAMACHY available daily solar measurements from Level 1b data taken over the entire lifetime of the mission, but excluded decontamination and anomaly orbits.

In channel 7, the measurements are contaminated by straylight from a light leak. The stray light prevents any retrieval of atmospheric parameters from this channel. Therefore we excluded this channel from our study.

We wanted to deduce the ISRF in each channel and additionally investigate its FWHM variation across the channel. In order to test that, we tried to retrieve the ISRF at different wavelengths within the same channel. We therefore needed to investigate as many solar lines or spectral structures as possible at various wavelengths in each channel. The selection of the lines was however challenging due to a few constraints. We needed strong lines from the theoretical spectra that should also be spectrally resolved by SCIAMACHY. Additionally, the measured line should cover enough pixels within the line’s spectral range (a few nanometers) to avoid an over-parameterization of the model and also be resolved to fit a relatively complex curve. These are key points in order to perform a reliable and statistically meaningful fitting.

We identified a few solar lines per channel at different wavelengths across the channel from the reference spectra. However, for the fitting we only used the ones that were well resolved by SCIAMACHY (

Table 3. Identified solar lines for the fitting in different SCIAMACHY channels. The lines that were not resolved were not fitted. Column 3 indicates whether a line had acceptable fits for our analysis. ${}^{\left(1\right)}$Channel 8 line was an exception as this was the only one we could identify, which we used for testing.

	Line ID	$\mathit{\lambda }$ Range (nm)	Resolved	Acceptable Fit
Channel 1	Line 1	277–282.5	yes	yes
	Line 2	283.5–287	yes	no
	Line 3	301.5–302.7	yes	no
Channel 2	Line 1	330–331.5	yes	yes
	Line 2	391–394.5	yes	yes
	Line 3	396–398	yes	yes
Channel 3	Line 1	483–488	yes	yes
	Line 2	514–520	yes	yes
	Line 3	524–530	yes	yes
	Line 4	531–535	yes	yes
Channel 4	Line 1	654–659	yes	yes
	Line 2	760–764	partially	-
	Line 3	804–806.5	no	-
Channel 5	Line 1	853–856.5	yes	yes
	Line 2	837–842	no	-
	Line 3	865–868.5	yes	yes
	Line 4	880–884	yes	yes
	Line 5	940–943.5	partially	-
Channel 6	Line 1	1590–1600	partially	-
	Line 2	1635.5–1650	no	-
	Line 3	1668–1675	no	-
	Line 4	1193–1215	yes	yes
	Line 5	1565–1581	yes	yes
	Line 6	1705–1717	partially	-
Channel 8	Line 1	2280.5–2282.2	partially	(yes)${}^{\left(1\right)}$

). Furthermore, channels 6 and 8 contained a large number of dead and bad pixels [14] which reduced significantly the number of usable pixels for the fitting. This has been detected early in the mission and it increased over time. Channel 8 in particular lacked the necessary number of pixels covering our identified solar lines across the entire channel. This was an additional limiting factor in this study.

In Table 3, we present the identified solar lines and indicate whether they were resolved by SCIAMACHY. Ultimately for each channel we used and presented the results that were obtained with spectrally resolved available solar lines. Channel 8 had one line only which we used despite it being only partially resolved. This allowed us to test our algorithm in this channel and probe the relative variation of its ISRF over time.

3. Results

For comparison, Figure 2 shows SCIAMACHY pixel masks per channel over the entire lifetime of the mission. We only show channels 1, 2, 6, and 8 for comparison between the UV/VIS and the SWIR bands. In channels 1 and 2, most pixels are good, as highlighted in green, over most of the mission lifetime. The vertical stripes that occur every several months correspond to the decontamination and the anomaly orbits, which we excluded in our work. On the other hand, in the infrared channels 6 and 8, a large number of pixels were flagged as bad/dead over an important part of the mission lifetime. A pixel is flagged as bad based on 11 criteria derived from different in-flight measurements. Typically a bad flag means the pixels had no signal (disconnected), were too noisy, or had excessively large leakage current. The reason for bad pixels is a substrate/light-detecting layer mismatch and induced damages because of proton impact [14].

We fitted each daily measurement (orbit) of the resolved lines in Table 3. In each channel, we retained the parameter function (Equations (3)–(5) and (7) depending on the channel) that overall (over the mission lifetime) provided the best fits of all available lines concurrently. The deduced fits of the measured solar lines that had <50% confidence levels over most of the mission lifetime were not used for our analysis (

Table 4. The ISRF fitted function per channel and the averaged FWHM over the mission lifetime per measurement based on this study’s results. Channel 8 results are shown in parenthesis because of their peculiarity and also as the measured line was not fully resolved.

	Fitted Analytical Function	Solar Line	Time Averaged FWHM (nm)
Channel 1	Hyperbolic	277–282.5	0.25 ± 0.004
Channel 2	Hyperbolic	330–331.5	0.24 ± 0.004
		391–394.5	0.25 ± 0.005
		396–398	0.26 ± 0.004
Channel 3	Gaussian	483–488	0.448 ± 0.01
		514–520	0.436 ± 0.01
		524–530	0.43 ± 0.01
		531–535	0.44 ± 0.01
Channel 4	Hyperbolic	654–659	0.426 ± 0.01
Channel 5	Hyperbolic	853–856.5	0.574 ± 0.01
		865–868.5	0.554 ± 0.01
		880–884	0.537 ± 0.01
Channel 6	Hyperbolic	1193–1215	1.468 ± 0.03
		1565–1581	1.34 ± 0.03
Channel 8	(Compound hyperbolic)	2280.5–2282.2	(>0.3)

), e.g., line 2 in channel 1 (Table 3) was not analyzed in this study. In order to investigate the stability of the ISRF over time, we calculated the FWHM, or width at the half of their maximum values, of the retrieved ISRFs between 2002 and 2012. We could therefore probe its variation on a daily basis.

Figure 3, Figure 4, Figure 5, Figure 6, Figure 7, Figure 8 and Figure 9 show examples of the corresponding SCIAMACHY measurements in orbit 8884 including their modeled counterparts, as well as their relative fitting residuals. The theoretical highly resolved corresponding solar lines are shown to verify the consistency between the measured and the theoretical spectra. The corresponding deduced ISRFs are shown in Figure 10.

Column 2 of Table 4 shows the retained mathematical functions per channel from our results. Table 4 (column 4) presents the time averaged FWHM of the ISRF(s) over the mission lifetime, in all channels and for each fitted spectral line. The temporal variation of the FWHM(s) are shown in Figure 11, Figure 12, Figure 13, Figure 14, Figure 15, Figure 16, Figure 17 and Figure 18.

4. Discussion

4.1. The ISRF in the UV/VIS Channels

The corresponding Figure 11, Figure 12, Figure 13, Figure 14 and Figure 15 show stable FWHMs in time, within the error bars, except in channel 1. In this channel, the FWHM increases not linearly beyond the error bars with the time. Its value increases from ≃0.23 nm to reach ≃0.256 nm in 2008, where it stays until mid 2009, and then drops down to ≃0.252 nm and stays there until the end of the mission in 2012. Furthermore, its average value over the life time of the mission is 0.258 ± 0.005 nm. The other UV/VIS channels (channels 2–5) do however not show such a variation or trend. They are pretty stable over time within the error bars. Furthermore, the observed sharp decrease of the optical throughput in channels 1 & 2 [7] does not provide an obvious explanation as to why the FWHM varies over time in channel 1, since we did not see a similar trend in channel 2. This perhaps would still need a further investigation.

UV/VIS channels have an averaged FWHMs that are consistent within a few percents with the on ground estimated ones (Table 1), except channel 4 where it is a little over 10% higher. In our simulation, we found that channel 4’s ISRF is shifted all the time by at least $\frac{1}{20}$ of a pixel, which is about 0.01 nm.

The signal to noise ratios in all the UV/VIS channels were very similar as well as the ISRF resolution, or the average number of pixels per FWHM. Therefore, these two properties do not appear to be a reason for the temporal variation in Channel 1 and the particularity of Channel 4. However, we only had one line per channel in both and could therefore not make a final statement about their behavior or deviation. Further investigations might be needed to consolidate these results. This could include fitting SCIAMACHY non-solar spectra measurements at these wavelengths.

In channels 2, 3 and 5, we had a few measured lines per channel, and thus a few retrieved ISRF(s). The retrieved FWHMs in channel 2 increased on average from one region around ∼330 nm to another ∼390 nm by up to 5%. However, this does not indicate a trend within the channel. In any case, more lines in this channel, or at least between 330 and 390 nm, would have been necessary to better analyze any possible variation within the channel. Channel 5 results show a slight trend where the average FWHM increases by ≃2% as a function of the wavelength beyond the error bars. Since we only covered the region from ≃850 to 885 nm, which is only ∼10% of the channel spectral range, we cannot confirm a possible trend within this channel. Furthermore, in channel 3 the averaged FWHM of the measured lines were consistent with each other within the error bars.

4.2. The ISRF in the NIR Channels

As shown in Figure 2, channel 6 and 8 pixels were mostly flagged as bad/dead with a very little amount of usable or available pixels or wavelengths, highlighted in green in Figure 2. Additionally, the signal to noise ratios in these channels were significantly lower than in the UV/VIS channels. We still identified a few solar lines that we tried in this analysis (Table 3). Although SCIAMACHY did not resolve the line in channel 8 (Figure 9), we fitted the measurement to basically test our ISRF retrieval method in this channel and to probe the relative temporal variation of its FWHM.

Channel 6 FWHM are on average relatively stable in the long term (Figure 16). There were no trends detected like in channel 1. However, line 1 of channel 6 showed a seasonal-like variation beyond the error bars. We could distinguish eight distinct temporal periods between 2003 and 2012. In each, the FWHM varied by ≃0.6 nm, nevertheless the average (Table 4) was overall constant and very consistent with the ground based estimated value (Table 1). A comparison to the periodical variation of the temperature did not show any correlation [6]. However, in both cases the periods duration are close to a year. Line 2 at longer wavelength in the same channel and close to the edge of channel 6+ is a lot more constant within the error bars (Figure 16), yet its value diverges from the on ground one. The difference of the two lines results in a same channel made their interpretation more intriguing, and thus may require further investigation.

Channel 8 (Figure 17) shows a stable, within the error bars, FWHM up until December 2008, nevertheless it is over 20% higher than the on ground value. Afterwards, between late 2008 and mid 2009, the FWHM becomes very noisy before jumping into a larger value that is almost twice the on ground known FWHM, where it stays until the end of the mission. As a matter of fact, the sudden variations of the FWHM started before the decontamination on 19-December 2008/3-January-2009. Besides, channel 7 & 8 were peculiar cases as the detectors were operated at the lowest temperatures. The throughput was in the beginning hampered by the growth of an ice layer in the optical light path over time. In fact, immediately after the detectors were cooled to operating temperatures, the ice layer began accumulating—and not uniformly over the light detecting area [16]—and the average throughput decreased rapidly (Figure 18), and thus the detector temperature increased. This can be seen in the behavior of the corresponding temperature curve (Figure 19). The ISRF does also depend on the structure (microscopic and macroscopic) of the ice layer or its smoothness. The ice does not only absorb light but also scatters photons, which ultimately fall on different parts of the detector. This generally leads to a widening of the ISRF and an increase of the intensity in the outer wings. This generated a scattering of the light and likely the widening of the ISRF of channel 8. Due to the particularity of channel 8 including the fact that the fitted line was only partially resolved does not yield an absolute result regarding the ISRF in this channel.

5. Conclusions

We developed a method using mathematical algorithms to recover the ISRF function (sometimes referred to as a slit function) of space spectrometers and imagers in-flight. Our method was used to retrieve the ISRF(s) of SCIAMACHY instrument in seven channels 1–6 and 8, by using its daily solar measurements independently of each other and over its entire lifetime. By fitting these measurements with an algorithm using the brute force search method, we retrieved daily in-flight ISRFs in each tested channel. We lastly compared our results to the on ground defined ones and probed the ISRF’s width behavior over time.

The FWHM of the retrieved ISRF were consistent within a few percents (up to about 10%) with their pre-defined values from ground based estimation in channels 1–6. They were stable over time in the UV/VIS channels 2–5. Channel 1 showed however a long term temporal trend revealing a non linear increase of the FWHM value by ≃10% in the first seven years of the mission. In the SWIR channel 6, the deduced ISRF’s FWHM were on average stable over the entire lifetime of the mission. Nevertheless, despite being stable on average, the ISRF from one of the spectral regions in channel 6 showed a seasonal-like variation of its FWHM by ≃40%, which is hard to interpret as this was not observed in the second retrieved ISRF’s within the same channel. The SWIR Channel 8’s ISRF FWHM values were on the other hand very different than the ground based estimation, and it additionally showed a sudden change of its value around year 2009. This was due to the known peculiarity of this SWIR channel mainly because of non-uniform ice accumulation on its detector.

In this study we did not see any particular variation or trend of the FWHM as a function of wavelengths across the channels that had a few solar lines. Channel 5 showed a slight increase by up to 2% of the FWHM as a function of the wavelength. However, we could only analyze a very small spectral region of this channel. Thus, one cannot settle with these results nor fully confirm them, as there were not enough measured and also resolved solar lines per channel for the purpose of this analysis.

Ultimately this approach and algorithms retrieved SCIAMACHY’s in-flight ISRF using its solar measurements. It also showed that the signal-to-noise ratio was not a limiting factor in the retrieval. However, the spectral resolution has been a key factor to perform and endorse the retrieval results. Designing a strategy for the characterization of the ISRF both on ground and in-flight is in any case important for spectral imager missions and their data processing; our algorithm could eventually be tested for that purpose.