Experimental OMPS Radiance Assimilation through One-Dimensional Variational Analysis for Total Column Ozone in the Atmosphere

Abstract

This experiment is the first ultraviolet radiance assimilation for atmospheric ozone in the troposphere and stratosphere. The experiment has provided better understanding of which observations need to be assimilated, what bias correction scheme may be optimal, and how to obtain surface reflectance. A key element is the extension of the Community Radiative Transfer Model (CRTM) to handle fully polarized radiances, which presents challenges in terms of computational resource requirements. In this study, a scalar (unpolarized) treatment of radiances was used. The surface reflectance plays an important role in assimilating the nadir mapper (NM) radiance of the Ozone Mapping and Profiler Suite (OMPS). Most OMPS NM measurements are affected by the surface reflection of solar radiation. We propose a linear spectral reflectance model that can be determined inline by fitting two OMPS NM channel radiances at 347.6 and 371.8 nm because the two channels have near zero sensitivity on atmospheric ozone. Assimilating a transformed reflectance measurement variable, the N value can overcome the difficulty in handling the large dynamic range of radiance and normalized radiance across the spectrum of the OMPS NM. It was found that the error in bias correction, surface reflectance, and neglecting polarization in radiative transfer calculations can be largely mitigated by using the two estimated surface reflectance. This study serves as a preliminary demonstration of direct ultraviolet radiance assimilation for total column ozone in the atmosphere.

1. Introduction

Atmospheric ozone plays a crucial role in the series of intricate feedback mechanisms that dynamically link the troposphere and the stratosphere. It blocks most of the harmful ultraviolet solar radiation from reaching the Earth’s surface and impacts air quality near the surface. Daily measurements of the ozone distribution are an important component of a more realistic treatment of the stratosphere in operational weather forecasts. This is especially true in the vicinity of jet streams at the mid and high latitudes, which play a major role in the formation and steering of tropospheric weather systems, including large-scale thunderstorm complexes and hurricanes. The improved forecasting of jet streams should lead to the improved long-term forecasting of tropospheric weather [1]. Numerical weather prediction (NWP) centers worldwide are extending their data assimilation and forecasting codes to provide a more realistic handling of the stratosphere. Infrared hyperspectral radiances for measuring atmospheric ozone have been directly assimilated for weather forecasts. Both infrared and microwave sounding radiances are directly assimilated in support of weather forecasting. Historically, the infrared (IR) and microwave (MW) sounding products, which are retrieved from infrared and microwave radiances, have contributed to weather forecasting. Indeed, the algorithm including the cost function and data quality control as well as bias correction in retrieval algorithms are similar to those of direct radiance assimilation systems. However, direct radiance assimilation has two unique advantages: the best prior information and better consistencies among various observations and with the forecast system.

Currently, the assimilation of ozone information from satellite-borne ultraviolet (UV) sensors is only conducted in retrieval space because of the challenge of radiative transfer modelling and the lack of accurate knowledge about UV data quality control and bias correction for direct radiance assimilation. However, if difficulties can be overcome, direct UV radiance assimilation offers several advantages compared to retrievals. Reliable prior information on ozone from the NWP 6-h forecast would greatly reduce the uncertainty that exists in retrievals. The determined ozone amount obtained directly from radiances would be more consistent with the NWP model because the NWP 6-h forecast is an important part of the cost function used for minimization. At the same time, there are a number of differences between UV radiance and microwave/infrared radiances. UV radiances are reflected solar radiation from the atmosphere and surface, while microwave and infrared (except for IR shortwave channels) radiances are emitted from the atmosphere/surface themselves. Therefore, UV radiances highly depend on both the zenith and azimuth angles of the sun and the sensor viewing direction. In addition, Solar irradiance can vary with time by 2% at 300 nm [2].

In direct radiance assimilation systems, IR and MW brightness temperatures are actually assimilated. The ratio between the maximum and minimum brightness temperatures is generally less than a factor of 2. However, the ratio between the maximum and minimum radiances for UV sensors can be larger than 100. The large difference in measurements may cause instability in deriving geophysical parameters such as ozone amount in the atmosphere. This article aims at sharing our experience and very limited experimental results in directly using Joint Polar Satellite System (JPSS) Ozone Mapping and Profiler Suite (OMPS) radiances. This article is organized as follows: We introduce the Joint Polar Satellite System in Section 2; the OMPS sensor data record (calibrated radiance) is described in Section 3; Section 4 reviews the community radiative transfer model (CRTM), the core tool for radiance assimilation; Section 5 describes a very simple radiance assimilation scheme; the experimental results of the OMPS radiance assimilation for atmospheric ozone are given in Section 6; the last section leads to the summary and discussions.

2. Joint Polar Satellite System

The Joint Polar Satellite System (JPSS) is a new generation polar-orbiting operational environmental satellite system in the United States (https://www.jpss.noaa.gov/mission_and_instruments.html, accessed on 25 August 2021). JPSS is a collaborative program between the National Oceanic and Atmospheric Administration (NOAA) and its acquisition agent, the National Aeronautics and Space Administration (NASA). The Suomi-National Polar-Orbiting Partnership (SNPP) satellite, launched on 28 October 2011, is the prototype of the JPSS program, originally designed as a preparatory program for the aborted National Polar-orbiting Operational Environmental Satellite System (NPOESS). The first JPSS satellite (JPSS-1) was launched on 18 November 2017. After successful commissioning, JPSS-1 was renamed NOAA-20. NOAA-20 leads SNPP by 50 min in the same orbit. The JPSS-2 launch date is currently scheduled in 2022 (https://www.wmo-sat.info/oscar/satellites/view/jpss_2, accessed on 25 August 2021). JPSS-3 and JPSS-4 are scheduled in 2027 and 2032, respectively. JPSS will provide continuity of critical, global observations of Earth’s atmosphere, oceans, and land through 2038.

SNPP and all JPSS satellites observe the Earth’s surface twice a day, once during the day and once at night, flying 830 km high in a polar orbit around 1:30 p.m., crossing the equator at local time. Both SNPP and NOAA-20 have continued supporting accurate forecasts and provide advanced warning for severe weather, such as hurricanes, deadly tornado outbreaks, heat waves, droughts and floods, snowfall, and wildfires. The satellites carry the Advanced Technology Microwave Sounder (ATMS), the Cross-track Infrared Sounder (CrIS), the Visible and Infrared Imaging Radiometer Suite (VIIRS), the Ozone Mapping and Profiler Suite (OMPS), and the Clouds and the Earth’s Radiant Energy System (CERES) on the same platform.

The ATMS is a cross-track scanner with 22 channels of temperature and humidity sounding measurements over a frequency range from 23 to 183 GHz. The ATMS provides sounding observations for atmospheric temperature, moisture, and clouds as well as surface temperature, snow, and ice [3]. All channels are sampled every 1.1° at 96 scan positions, with a cross track swath width of 2500 km, which is significantly wider [4] than AMSU-A or MHS. ATMS channels 1 and 2 have a large beam width (low spatial resolution) of 5.5 degrees; channels 3 to 16 have a medium beam width of 2.2 degrees; and channels 17 to 22 have a small beam width of 1.1 degrees. The CrIS is a Fourier transform spectrometer (Michelson interferometer) with 2223 un-apodized (2211 Hamming apodized) spectral channels over three wavelength ranges: long-wave infrared (LWIR), middle-wave infrared (MWIR), and short-wave infrared (SWIR) [5]. CrIS scans a 2200 km swath width (±50°) with 30 Earth scene views (i.e., field of regards). Each scene consists of nine fields of view, arrayed as a 3 × 3 array of 14 km diameter spots at a nadir spatial resolution. The CrIS provides hyperspectral measurements for atmospheric sounding of three-dimensional temperature, pressure, moisture profiles, and trace gases: carbon dioxide, monoxide, ozone, and methane. VIIRS succeeds the NOAA Advanced Very HighResolution Radiometer (AVHRR) and the Moderate Resolution Imaging Spectroradiometer (MODIS) on the NASA Aqua and Terra satellites. The VIIRS supports operational environmental monitoring and numerical weather forecasting, with 22 imaging and radiometric bands covering wavelengths from 0.41 to 12.5 microns [6], providing the sensor data records more than twenty environmental data records, including clouds, sea surface temperature, ocean color, polar wind, vegetation fraction, aerosol [7], fire, snow and ice, vegetation, and other applications. The Ozone Mapping Profiler Suite (OMPS) is composed of three spectrometers: a nadir mapper for total column ozone, a nadir profiler, and a limb profiler for ozone profiling. Since this article is about OMPS radiance assimilation, we will introduce the OMPS sensor data record in detail in the next section. The CERES is a three-channel radiometer [8], measuring radiance at a shortwave channel (0.3–5.0 µm), a total channel (0.3–>50 µm), and an infrared window channel (8–12 µm). CERES products contain estimates of instantaneous filtered radiance, unfiltered radiance, flux at the top of the atmosphere (TOA), and scene type (see https://www.nasa.gov/feature/langley/ceres-at-20-a-continuous-record-of-earths-radiation-budget, accessed on 25 August 2021).

SNPP and all JPSS satellites carry multiple sensors, simultaneously observing the Earth from the ultraviolet spectrum to visible, infrared, and microwaves. For historical reasons and heritage, the OMPS sensor data record (SDR) data uses a radiance unit of ${\mathrm{Wcm}}^{-3}{\mathrm{sr}}^{-1}$. The VIIRS SDR uses a radiance unit of ${\mathrm{mWm}}^{-2}{\mathrm{sr}}^{-1}{\mathrm{nm}}^{-1}$. The unit used for the VIIRS SDR and OMPS SDR is equivalent or the same. Both CrIS and ATMS SDRs use a radiance unit of ${\mathrm{mWm}}^{-2}{\mathrm{sr}}^{-1}{\mathrm{cm}}^{-1}$, which is the same as the unit used by numerical weather prediction (NWP) centers. Using a CRTM utility tool, we converted CrIS and ATMS radiance to the same unit as OMPS and VIIRS [9]. Figure 1 shows radiances of the OMPS nadir profiler (black line, NP) and the OMPS Nadir Mapper for total column ozone (blue line, NM), the VIIRS bands (red triangles, VIIRS) of a moderate spatial resolution, and the CrIS (black line, infrared) over the Libya desert I site on 2 March 2021. To display the figure of entire radiance clearly, we plotted the OMPS NP and NM radiances as well as the VIIRS radiance in reflective bands on the top panel of Figure 1. It is noted that VIIRS channel 6 over desert may be saturated because of the small dynamic range since the primary use of VIIRS channel 6 is over oceans. The VIIRS infrared radiance (red triangles) and CrIS radiance (black) are displayed on the middle panel. The green spectral line is the CrIS inner spectral gap filling using the CrIS gap filling algorithm [10]. The extrapolated CrIS spectrum to shortwaves beyond 2550 ${\mathrm{cm}}^{-1}$ is not displayed here because of strong solar radiation contamination. Even though no new information is added, the CrIS spectral gap filling is still quite useful for us to compare the CrIS observations with the VIIRS broadband measurements. The collocated VIIRS measurements averaged over the CrIS field of view and the simulated VIIRS observations through the spectral convolution of the CrIS radiances with the VIIRS sensor response function data have been successfully used to estimate the CrIS geolocation accuracy [11]. The ATMS radiances for 22 channels are shown in the bottom panel.

One may notice that the SNPP radiance changes from more than 200 ${\mathrm{mWm}}^{-2}{\mathrm{sr}}^{-1}{\mathrm{nm}}^{-1}$ for VIIRS channel 5 centered at 670 nm (i.e., 14,925 ${\mathrm{cm}}^{-1}$) and downward ${10}^{-10}{\mathrm{mWm}}^{-2}{\mathrm{sr}}^{-1}{\mathrm{nm}}^{-1}$ for ATMS channel 1 at 23.8 GHz. Even for ATMS itself, the dynamic range of radiance is more than 1000 times. This is also true for the CrIS radiance itself. If we plot the SNPP measurements in brightness temperatures as shown in Figure 2, the ratio between the maximum and minimum is less than 25 times, which is much smaller than the ratio in radiance. The ratio for the ATMS brightness temperatures is less than a factor of 2. The ratio for the CrIS brightness temperatures is generally (except for cases of wildfires) smaller than a factor of 2 as well. This may partly explain why users prefer to assimilate the brightness temperatures for microwave and infrared measurements instead of radiances even though sensor noise in radiance is not very dependent on scene radiance. The noise in the brightness temperature can strongly depend on the scene brightness temperature. In addition, users are more familiar with brightness temperatures than radiance for quality control and cloud detection.

3. Ozone Mapping and Profiler Suite

The OMPS heritage from the NASA Solar Backscatter Ultraviolet (SBUV) and Total Ozone Mapping Spectrometer (TOMS) series of instruments began operationally with the Nimbus-7 TOMS and the NOAA SBUV/2 series of instruments [12]. The advanced suite is composed of three hyperspectral instruments: a nadir mapper (OMPS-NM) for total column ozone, a nadir profiler (OMPS-NP), and a limb profiler (OMPS-LP) for ozone profiling [13]. The OMPS instrument suite has two telescopes and two solar diffusers. Of the solar diffusers, one solar diffuser works bi-weekly. The other solar diffuser serves as reference, working semiannually. One telescope is used by the limb profiler. The OMPS-NM and OMPS-NP grating spectrometers share the second telescope [14,15]. A dichroic filter reflects the 250–310 nm light to the NP spectrometer and transmits the 300–380 nm light to the NM spectrometer. The NP and NM spectrometers overlap from 300–310 nm, where the dichroic filter transitions from reflection at 300 nm to transmission at 310 nm. Two-dimensional (340 pixels along the spectral dimension and 740 pixels in the cross-track spatial dimension) charge-coupled devices (CCD) are located at the NM spectrometer’s focal plane. The nadir profile spectrometer deploys a two-dimensional CCD of 340 (spectral) by 390 (spatial). The entire CCD is not used, but part of it is sensitive to photons within defined spectral ranges.

The nadir mapper has a 2800 km cross-track swath divided into 35 fields of view of nearly equal angular extent. The NM SDR data contain 196 channel radiances. The nadir profiler SDR data content of SNPP and NOAA-20 is different. In SNPP OMPS-NP SDR, there are 147 channels, and the spatial resolution is 250 by 250 km². NOAA-20 NM SDR includes 151 channels, and the spatial resolution is enhanced to 50 by 50 km². The geolocation uncertainty is less than 5 km.

Together, NM and NP spectrometers can measure reflected UV radiances between 250 nm and 380 nm. Figure 3 shows the optical depths of absorbing gases: O₃, SO₂, NO₂, O₄, Bro, and ClO. Below 340 nm, O₃ absorption dominates. Both NM and NP measurements have high sensitivity to the ozone profile in the atmosphere. The global mean value of SO₂ is about 1 ppm or 0.1 Dobson Units (DU). The effect for such a low SO₂ on OMPS measurements is negligible. However, the eruption and outgassing of volcanoes can dramatically increase the local SO₂ amount in the atmosphere. The local SO₂ amount can be larger than 10 DU. The volcanic ash and SO₂ gas can affect stratospheric temperatures and atmospheric chemistry as well as cause aviation hazards [16]. The olive-colored line in Figure 3 represents the optical depth for the SO₂ of 10 DU. One can use OMPS NM measurement to retrieve the high SO₂ amount [17].

In this study, we only assimilate OMPS-NM SDR and geolocation data. Latitude, longitude, sensor and solar zenith and azimuth angles, date and time, and quality flags from the NM geolocation granule file are used. The Earth-view radiances, solar irradiances, and channel wavelengths as well as quality flags from the NM SDR granule file are also used. The wavelengths in both the NM and NP measurements are changed slightly depending on the date/time and orbit. The solar irradiance is measured weekly with a reflective diffuser at the entrance aperture. It may be worth mentioning that the solar irradiance in the SDR data file is corrected for a 1 AU Sun and Earth distance. In the CRTM calculation, the Sun–Earth distance is considered in the outputted solar irradiance.

4. Community Radiative Transfer Model (CRTM)

CRTM is a sensor-based radiative transfer model. It supports more than 100 sensors including sensors on most meteorological satellites and some from other remote sensing satellites. The CRTM is composed of four important modules for gaseous transmittance, surface emission and reflection, cloud and aerosol absorption and scattering, and a solver for radiative transfer. The CRTM is designed to meet users’ needs. Many options are available for users to choose from. The CRTM provides surface emissivity to the users; allows users to select a subset of channels for a given sensor; turns off scattering calculations; computes radiance at aircraft altitudes; computes aerosol optical depth only; and completes parallel calculations of the CRTM. The public interface contains the initialization modules for choosing sensor/sensors and full channels or a subset of channels. The CRTM forward model module is used to simulate satellite measured radiances, which can be used to verify measurement accuracy, uncertainty, and the long-term monitoring of radiances. The K-matrix module can be used to compute Jacobian values (i.e., radiance derivative with respect to geophysical parameters), which are used for the inversion processing in retrieval and radiance assimilations. Using the tangent-linear and adjoint modules are equivalent to using the K-matrix module, which is also applied to radiance assimilation at some numerical prediction centers. The CRTM is a FORTRAN library for users to link to their own code instead of a graphic user interface. At the CRTM initialization step, the user selects the sensor/sensors and surface emissivity/reflectance look-up tables. Developers may incorporate their own expertise into the CRTM for any desired application. The gaseous transmittance describes atmospheric gaseous absorption so that one can utilize remote sensing information in data assimilation/retrieval systems for atmospheric temperature, moisture, and trace gases such as CO₂, O₃, N₂O, CO, and CH₄ [18]. The aerosol module is fundamental to acquiring aerosol type and concentration for the study of air quality. The cloud module contains optical properties of six cloud types, providing radiative forcing information for weather forecasting and climate studies [19]. The CRTM surface model includes surface static and atlas-based emissivity/reflectivity for various surface types. There are two radiative transfer solvers that have been implemented in the CRTM. The advanced double-adding (ADA) solver [20] was chosen as a baseline. The advanced matrix operator method (AMOM) is a current baseline, and ADA is optional. The successive order of interaction (SOI) for radiative transfer [21] developed at the University of Wisconsin has also been implemented in the CRTM.

The latest released CRTM version is 2.4. The CRTM version 3.0 beta version has also been released for testing. The CRTM 3.0 beta inherits the new features from the version 2.4. Users can now use cloud and aerosol optical tables in either binary or netCDF format. In previous CRTM releases, an on-line aerosol modeling in Goddard Chemistry Aerosol Radiation and Transport (GOCART) was developed and implemented within GMAO’s GEOS-5 earth system model [22] and was later coupled with NCEP’s NEMS version of GFS to establish the first interactive atmospheric aerosol forecasting system at NCEP, NGAC [23]. The recently added aerosol model is the Community Multi-scale Air Quality (CMAQ) model, which is more focused on regional areas. In previous releases, the CRTM was developed for un-polarized scattering atmosphere. The polarization calculation in the CRTM only comes from polarized surface emissivity and reflectivity. However, un-polarized natural light can be polarized by scattering from molecules, aerosols and clouds, and by the reflection and scattering from surfaces [24]. The polarization of light in the atmosphere–surface system contains important signals that have led to major scientific breakthroughs that could not have been achieved by only studying scalar radiance [25]. For example, the polarization measured by microwave sensors provides unique information to calculate the sea surface wind speed [26], sea surface wind vector [27], and sea ice [28]. The French satellite-based POLarization and Directionality of the Earth Reflectance (POLDER) instrument has been used to obtain the particle shape of ice clouds [29]. In the CRTM version 3.0 beta, major work was conducted to extend the unpolarized treatment to a fully polarized treatment with an update to the radiative transfer solver and to add polarized atmospheric optical property calculations and surface emissivity/reflectance calculations. For the reasons of computational efficiency and easy software maintenance, we used a polarized radiative transfer form that holds the reciprocity principle [30]. With the reciprocity principle, we extended the CRTM Advanced Doubling-adding (ADA) algorithm to vectorized radiative transfer just by changing the dimension size from n_Angles to n_Angles x n_Stokes and saved about 50% in computational time in the radiative transfer solver. The extension of the CRTM AMOM is slightly more complicated [30]. For a scalar RT model, a single element of the phase matrix for intensity is sufficient, while all six independent elements in the phase matrix are required for a fully polarized RT model.

In addition to the software changes, there is significant effort to generate aerosol and cloud optical look-up tables offline. The look-up tables contain the extinction scattering coefficients, six expansion coefficient sets for the Greek matrix used in fully polarized radiative transfer, and asymmetry factors. Backscattering coefficients are newly added for active sensor applications. The OpenMP function was also implemented so that one can perform parallel CRTM calculations either at a profile level or at a channel level.

Solar irradiance is important to UV and visible radiance simulation, although normalized radiance and the N value (see Section 6) are independent from the solar irradiance. One can see from Equations (1) and (2) in [30] that the radiation source is linearly proportional to solar irradiance in the radiative transfer equation within the ultraviolet and visible spectrums, where the thermal emission is negligible. The radiance solution of the radiative transfer equation is linearly proportional to solar irradiance as well. To understand the physics behind this, one can imagine the atmosphere and surface systems as a black box and as having a boundary at the top of the atmosphere (e.g., Karman line at 100 km). The satellite measured UV and visible radiation are the reflected solar irradiance from the boundary, which is a linear product of solar irradiance and the reflectance at the boundary. The reflectance will not be changed because the solar radiation does not alternate the optical properties of the atmosphere and the surface during the very short period of time.

Solar irradiance is also subject to non-local thermodynamic equilibrium (non-LTE) effects [31,32]. It was found that below 210 nm, the non-LTE effects contribute significantly to the solar spectral irradiance (SSI) and SSI variability. However, above 210 nm, the non-LTE SSI variability from the non-LTE and LTE results are very similar, and the SSI difference between the LTE and non-LTE are very small (less than 1%) and should be understood as unimportant to our current study, which applies to OMPS Nadir Mapper wavelengths from 300–380 nm without considering the non-LTE effects.

With the changes described above, CRTM version 3.0 therefore has a new capability for UV radiance simulation and assimilation. The transmittance coefficients are obtained from the CRTM internal transmittance software [18] based on line-by-line calculations [33]. The internal surface reflectance is still under development. In this study, we use the reflectance option from the user inputs, and the surface is assumed to be Lambertian and unpolarized in the UV radiance simulations and assimilations. For OMPS NM, the surface reflectance can be estimated by fitting the measurements at two central wavelengths or at two channels. We believe that this study may be the first application of CRTM version 3.0 (to be publicly released) using OMPS UV radiance measurements.

5. A Simple Radiance Assimilation Algorithm

Satellite radiance data have been used in many applications: radiance assimilation in supporting of weather forecasts; retrievals of environmental data records; and long-term radiance records for climate studies. Microwave and infrared observations are unique to studying atmospheric temperature changes in both the troposphere and stratosphere. Direct satellite radiance assimilation has greatly improved weather forecasting. Passive microwave observations alone contribute to around 30−40% of the overall improvement of short-range forecast performance and provide around 15−30% of the total number assimilated observations [34]. UV radiances contain the most unique and the most important information about ozone amount in the atmosphere. UV radiances, which are measured from the sensor on the satellite platform, contain sufficient sensitivity to stratospheric ozone, which plays an essential role in absorbing harmful UV radiation. Therefore, direct UV radiance assimilation is highly desirable. However, four-dimensional radiance assimilation at NWP centers demands very large computational resources and significant manpower. In the context of this study, these resources are beyond our capability. For demonstration purposes, we adopted a one-dimensional variational (1DVAR) scheme to assimilate OMPS NM radiance. Similar to an operational radiance assimilation system, a 6-h forecast is used as the background. The radiance assimilation in our 1DVAR was first conducted at the time and location of the satellite observations, and then the ozone increments from the 1DVAR were propagated to NWP model grids and at the time of analysis through an adjoint technique. The observation data are four-dimensional. The treatment of the observation time is called the first guess at appropriate time (FGAT). The 3DVAR FGAT has been used in radiance assimilations at NWP centers [35]. FGAT compares the observations with the background at the observation time.

The 1DVAR technique had been very successful and has demonstrated clear and consistent benefits for forecast skill and has been implemented operationally at the European Centre for Medium-Range Weather Forecasts (ECMWF) in June 1992 [36]. The 1DVAR technique is very common in generating environmental data records. The 1DVAR algorithm is the core of the NOAA Microwave Integrated Retrieval System (MiRS) system [37] that has been implemented operationally at the U.S. National Oceanic and Atmospheric Administration since 2007 [38,39]. The MiRS has geenrated many environmental data records (i.e., satellite retrieval products) for NOAA-18, NOAA-19, Metop-B and C AMSUA-MHS, SNPP ATMS, NOAA-20 ATMS, DMSP F17 and F18 SSMIS, GCOM-W1 AMSR-2, and GPM GMI. These satellite products are atmospheric profiles of temperature and water vapor, cloud liquid water, ice water content, rainfall rate, snow cover and snow water equivalent, snow fall rate, surface temperature and microwave emissivity, and sea ice concentration (https://www.star.nesdis.noaa.gov/mirs/highresolutionv.php, accessed on 25 August 2021).

Radiance assimilation and 1DVAR retrieval all aim to minimize a cost function that weighs the relative contribution of background (a priori) information and satellite observations. Assuming that the errors in the observations and in the prior information are neither biased nor correlated and have Gaussian distributions, the best estimate will minimize the cost function [40]:

$$J\left(x\right)=\frac{1}{2}{\left(x-x_b\right)}^T{B}^{-1}\left(x-x_b\right)+\frac{1}{2}{\left(Y^M-F\left(x\right)\right)}^T{E}^{-1}\left(Y^M-F\left(x\right)\right)$$

(1)

where $x$ is a retrieved state vector of $n$ elements. The state vector may include temperature, water vapor, and cloud water content at each atmospheric layer. The first item on the right represents the penalty for departing from the background $x_b$ (a priori). The departure is weighted by a $n\times n$ error covariance matrix $B^{-1}$. The diagonal elements of $B^{-1}$ are the standard deviations of the state variables while the off-diagonal elements are the error covariance among the state variables, for example between the temperature and water vapor profiles. The second term represents the penalty for $m$ simulated radiances $F\left(x\right)$ (or brightness temperatures) departing from $m$ observed radiances (or brightness temperatures) $Y^M$, weighted by the sum of the instrument and forward $m\times m$ modeling error covariance matrix $E^{-1}$. $F$ is a forward operator.

The minimum of the cost function may be found from an iterative process. It computes descent directions that are gradients of the cost function (Equation (1)) to the state vector $x$:

$${\nabla }_{x}J\left(x\right)=B^{-1}\left(x-x_b\right)-K^T\left(x\right)E^{-1}\left(Y^M-F\left(x\right)\right)$$

(2)

where $K^T\left(x\right)$ is the adjoint model, which is equivalent to the transpose of Jacobian radiance with respect to the state vector. Mathematically speaking, the local minimum may be obtained by setting (gradients) Equation (2) to zero. However, we can only have an explicit solution in the 1DVAR case. For the 3DVAR case, the solution may be approached within the inner loop of the minimization. For the 1DVAR case, using a first order Taylor series expansion:

$$F\left(x\right)=F\left(x_b\right)+K\left(x\right)\left(x-x_b\right)$$

(3)

It is obvious that Equation (3) is only valid when the Jacobian operator $K\left(x\right)$ is linear or when the increment $\left(x-x_b\right)$ is small enough. Inserting Equation (3) into Equation (2) and letting Equation (2) equal zero (where the cost function $J\left(x\right)$ will have a local minimum), we obtain

$$\left(x-x_b\right)={\left[B^{-1}+K^T\left(x\right)E^{-1}K\left(x\right)\right]}^{-1}{K}^T\left(x\right)E^{-1}\left(Y^M-H\left(x_b\right)\right)$$

(4)

Taking an iterative approach, we may rewrite Equation (4) as

$$\begin{array}{c}\left(x_{k+1}-x_b\right)={\left[B^{-1}+K^T\left(x_k\right)E^{-1}K\left(x_k\right)\right]}^{-1}{K}^T\left(x_k\right)E^{-1}\left(Y^M-F\left(x_b\right)\right)\\ ={\left[B^{-1}+K^T\left(x_k\right)E^{-1}K\left(x_k\right)\right]}^{-1}{K}^T\left(x_k\right)E^{-1}\left(Y^M-F\left(x_k\right)+F\left(x_k\right)-F\left(x_b\right)\right)\\ ={\left[B^{-1}+K^T\left(x_k\right)E^{-1}K\left(x_k\right)\right]}^{-1}{K}^T\left(x_k\right)E^{-1}\left(Y^M-F\left(x_k\right)+K\left(x_k\right)\left(x-x_b\right)\right)\end{array}$$

(5)

or

$$∆x_{k+1}={\left[B^{-1}+K^T\left(x_k\right)E^{-1}K\left(x_k\right)\right]}^{-1}{K}^T\left(x_k\right)E^{-1}\left(Y^M-F\left(x_k\right)+K\left(x_k\right)∆x_k\right)$$

(6)

Equation (6) is in “n-form”, which is the preferred form for cases in which the number of state variables is less than the number of observations [40].

If the number of state variables is larger than the number of the observations, it is better to use the “m-form”:

$$∆x_{k+1}=B{K}^T\left(x_k\right){\left[K\left(x_k\right)B{K}^T\left(x_k\right)+E\right]}^{-1}\left(Y^M-F\left(x_k\right)+K\left(x_k\right)∆x_k\right)$$

(7)

where $∆x_{k+1}=x_{k+1}-x_b$ and $∆x_k=x_k-x_b$.

The increment at each step can be large, so Equation (3) may not be accurate for a non-linear problem since the increment is the departure directly from the background instead of the small change with respect to the previous iteration value at each step. It is known that the inversion is an ill-posed problem (i.e., there are multiple possible solutions that are consistent with the satellite observations). In MiRS, the values of the diagonal elements in the observation error covariance matrix are increased if the differences between the observations and the simulations are large. The adjustment seems reasonable and mitigates the ill-posed problem. In radiance assimilation, the minimization of the cost function is composed of the inner loop and outer loop iterations. The background and observational error covariance matrices are fixed in both the inner and outer loops. For every outer loop step, there are many dozens of inner loop iterations. Within inner loop iterations, the CRTM is only called one time, and the background remains the same. The minimization is conducted for the fixed Jacobian and fixed background. For the next outer loop step, the CRTM is called again, and the background is updated as well. The process ensures small increments at each outer loop step since the background is updated toward the final state. It may seem that the observation information content is utilized more than once. However, this is just an internal process. For the entire radiance assimilation process, the information used is the 6-h forecast, satellite radiances, and other observations.

In this study, we follow similar steps as the operational 3DVAR FGAT radiance assimilation. The time interpolation utilizes the observations with the background at the observation time. Our simple radiance assimilation scheme is similar to the operational 3DVAR FGAT scheme but with a one-dimensional variational procedure and error covariance matrix in the cost function. In 1DVAR, the cost function minimization problem can be explicitly solved, while the solution in 3DVAR minimization can only be approached in the inner loop iteration.

Table 1. Summary of 1DVAR retrieval, the simple radiance assimilation scheme used in this study, and operational radiance assimilation.

		MiRS 1DVAR Retrieval	Simple Radiance Assimilation 1DVAR FGAT	Radiance Assimilation 3DVAR FGAT
Satellite radiance		$\surd$	$\surd$	$\surd$
Other observations				$\surd$
First guess		May be different from background	Same as background	Same as background
Background	1D	$\surd$
	3D		With spatial & temporal interpolation	With spatial & temporal interpolation
Error covariance	Background	$\surd$	$\surd$	$\surd$
	Observation	$\surd$	$\surd$	$\surd$
Iteration	Outer loop	RT model called, observation error matrix is changed if O-B is large	Background changed and RT model called	Background changed and RT model called
	Inner loop			$\surd$
Data thinning				$\surd$
Cost function	1D	$\surd$	$\surd$
	3D			$\surd$
Output	Analysis grid at analysis time		Increments propagated to grids at analysis time	Increments propagated to grids at analysis time
	Orbit	$\surd$

summarizes the similarities and the differences between the 1DVAR retrieval, the simple radiance assimilation scheme used in this study, and the operational 3DVAR radiance assimilation process. OpenMP [41] is also implemented for parallel processing in the radiance assimilation procedure used here.

6. Experimental OMPS Radiance Assimilation

Infrared and microwave radiances (actually brightness temperatures) have been operationally assimilated at NWP centers in support of weather forecasting. The infrared and microwave radiance assimilation has greatly improved the analysis of temperature, water vapor, and ozone in both the troposphere and stratosphere. Visible and ultraviolet radiances have not been assimilated because of a number of challenges, for example, the availability of a fast and accurate radiative transfer model and an appropriate bias correction scheme. The CRTM has been successfully supporting operational infrared and microwave radiance assimilation at NWP centers. Recently, the CRTM has been extended with the capability for ultraviolet radiance simulations and assimilations [30]. The extended CRTM can be run for fully polarized radiative transfer calculations or for scalar (i.e., intensity) calculations. However, in radiance assimilation, the fully polarized CRTM is 20 times slower than the scalar CRTM because Rayleigh scattering is present in all 91 layers. A method is needed to significantly reduce the computation time for pure Rayleigh scattering through a look-up table technique or a machine learning approach in the future. An artificial intelligence (AI) based radiative transfer model can be 100 to 1000 times faster than a traditional RT model [42]. We found that the large difference between the fully polarized and scalar CRTM calculations can be mitigated using corresponding surface reflectance. In this study, we have used the scalar CRTM for OMPS radiance simulation and assimilation. To further minimize the computational demand, we selected nine OMPS NM channels, similar to the channels used in ozone retrieval. The NDEV8 ozone product was retrieved from the Version 8 total ozone algorithm [12]. The algorithm uses a look-up table technique by matching radiances. A radiative transfer lookup table was generated using standard ozone profiles offline.

For radiance assimilation, the selection of observations, bias correction strategy, error covariance matrix, and the minimization technique are critical. The background error covariance matrices were derived from one day of global analysis data. The observation error covariance matrix was based on experiments at several analysis times. We used the 1DVAR technique in the minimization. We followed the steps of operational 3DVAR radiance assimilation. The error covariance matrices remained unchanged, and the background was updated during the outer loop iteration. The increments from the minimization were then propagated back to the model grids at the desired analysis time.

OMPS ultraviolet radiance assimilation is very different from assimilating infrared and microwave brightness temperatures. There are two key differences between microwave/infrared and ultraviolet/visible radiance assimilations. The first key difference is the radiation source. The radiation source within microwave and infrared spectrums is the thermal emission of the atmosphere and Earth’s surface. The radiation over visible and UV spectral ranges is the solar radiation of sunlight. The short-wave infrared radiation can be affected by solar radiation, but they are not assimilated at numerical prediction centers. The second key difference is emissions vs. scatterings. Microwave and infrared radiation are dominated by emissions/absorptions, while UV and visible radiation are dominated by scatterings. For this reason, this is an initial experiment to assimilate OMPS radiance for atmospheric ozone in troposphere and stratosphere. Normalized radiance, a ratio between OMPS radiance and OMPS measured solar irradiance, is better for assimilation than radiance itself. The UV solar spectrum can change by a few percent [2], while the solar irradiance used in the CRTM is static. As one can see from Figure 4, the normalized radiance can vary by a factor of more than one hundred, which would be extremely challenging for the minimization procedure.

UV surface reflectance plays an important role in the CRTM simulations because most OMPS NM channels can see the surface under clear-sky conditions. Furthermore, there are large uncertainties in surface reflectance. In the ozone retrieval, one channel at 331 nm is used to estimate surface reflectance, and this reflectance is assigned to all other channels in the retrieval. Using CRTM, we calculate the total transmittance from the surface to the OMPS NM instrument and can calculate the accumulated radiance sensitivity to the ozone in the atmosphere (see Figure 5).

As shown in Figure 5, radiance at 331 nm is somewhat affected by atmospheric ozone. This suggests that it may be better to use ozone-independent channels to estimate the surface reflectance. Our proposed two channels (see symbol o in Figure 5) have high transmittance (high sensitivity to surface) and nearly zero sensitivity to ozone in the atmosphere.

In fact, one can see from Figure 4 that by using one constant reflectance (black line), the error in the CRTM simulation error increases for wavelengths longer than 331 nm since surface reflected solar radiation increases with wavelength as well. This large error can be reduced (see cyan line in Figure 4) if we use the surface reflectance interpolated or extrapolated from the estimated reflectance at the two wavelengths marked with “o” in Figure 5. This indicates that surface reflectance is not constant for the wavelengths between 250 nm and 380 nm. The linear spectral dependence of the surface reflectance can be a good approximation for this study. Based on the linear assumption, we analyze the difference of the normalized radiance between global OMPS NM measurements and the CRTM simulations for a time window of 6 h.

The scalar radiative transfer can be accurate for OMPS NM radiance simulations with an accurate effective surface reflectance. For this test, we chose a case over a Libya desert calibration site where the surface UV reflectance is low, only about 3% [43]. We first derived the surface reflectance at the wavelengths of 347.6 nm and 371.8 nm using polarized CRTM. Then, we interpolated or extrapolated the two reflectance points for other OMPS NM channels. A total of nine channels were selected to be used in the simulation. The last three of the nine channels are insensitive to ozone in the atmosphere. The other six channels are sensitive to the ozone amount in the atmosphere. We first calculated the absolute difference between the OMPS NM measurement and the CRTM simulation. The error given in this section is the absolute difference divided by the measurement in percentage. As shown in

Table 2. Comparison of OMPS normalized radiance (OMPS NM) by solar irradiance with the CRTM Stokes, scalar, and modified scalar calculations. The land surface reflectance is interpolated or extrapolated from the surface reflectance at 347.6 nm and 371.8 nm. The latter two reflectance are calculated by fitting OMPS NM measurements at the two wavelengths. The scalar model uses the same surface reflectance as Stokes. The modified scalar model uses the surface reflectance by fitting the measurements with the scalar model calculations at the two wavelengths.

Channel Number	Wavelength (nm)	OMPS NM	Stokes Model Error (%)	Scalar Model Error (%)	Modified Scalar Model Error (%)
61	324.53	0.06646	−0.030	4.228	−0.015
75	330.38	0.07006	−0.029	4.168	−0.014
77	331.20	0.06837	−0.015	4.183	−0.015
82	333.28	0.07068	−0.014	4.188	−0.014
94	338.30	0.06935	−0.029	4.167	−0.014
103	342.06	0.68780	−0.015	4.115	−0.015
117	347.90	0.06615	−0.015	4.006	−0.015
127	352.07	0.06422	−0.016	3.955	−0.016
175	372.09	0.05607	−0.018	3.674	−0.018

, the simulated radiances using the polarized CRTM agree with OMPS NM measurements very well (<0.1%). However, using the scalar CRTM model, the error is higher than 3.6% (5th column in Table 2). This result implies that the difference between the polarized and scalar CRTM simulations can be significant. If we use the same scalar CRTM model to derive the surface reflectance at the two wavelengths mentioned above and interpolate or extrapolate the two reflectance for other channels, the scalar CRTM simulation can also achieve a high accuracy (see the last column of Table 2). The estimated surface reflectance using the polarized CRTM simulation can be off the estimate by a few percent by using the scalar CRTM calculation. We also chose a case over Dome C site that is a radiometric calibration site in Antarctica. The surface reflectance in the UV spectra is very high [44], larger than 0.9. Again, the scalar radiative transfer can be accurate to OMPS NM radiance simulations with the effective surface reflectance.

As shown in Figure 4, the spectral variation of the normalized radiance can be 100 times larger between the low and high normalized radiances. In the ozone retrieval, the logarithm of the normalized radiance (so called $N_{value}$) was used instead of the normalized radiance itself. The $N_{value}$ is defined as [45]

$$N_{value}=-100\times lo{g}_{10}\frac{Radiance}{solar\_flux}$$

(8)

We replotted Figure 4 using the $N_{value}$. One can see from Figure 6 that the variation of $N_{value}$ is much smaller, less than a factor of 4. The CRTM simulation and the OMPS NM measurements generally agree well. However, we see a noticeable difference for large wavelengths between observations (red line) and CRTM simulations using a constant surface reflectance (black line).

Again, the CRTM simulation using two surface reflectance values is in better agreement with the observations. Large differences in the $N_{value}$ are also found for wavelengths between 300 nm and 305 nm, where the dichroic filter transitions from reflection at 300 nm to transmission at 305 nm.

In this OMPS NM radiance assimilation, we use the $N_{value}$ in the cost function equation, Equation (1). The Jacobian equation will be the derivative of the $N_{value}$ with respect to geophysical parameters. That is

$$\frac{\partial N_{value}}{\partial x}= \frac{-100}{\ln \left(10\right)\times Radiance}\times \frac{\partial Radiance}{\partial x}$$

(9)

To run the assimilation experiments, we used both analysis and forecast data from the ECMWF modelling system. ECMWF analysis data are available at 00Z, 06Z, 12Z, and 18Z. ECMWF forecast data are available at 03Z and at 09Z from an initialization time at 00Z and at 15Z and at 21Z from an initialization time at 12Z. We conducted the first experiment at 06Z on 2 March 2021. The background was the mean of the 3 h and 9 h forecasts from 00Z. The satellite data within the time window between 03Z and 09Z were assimilated. We compared the total column ozone in Dobson Units between our assimilation and the ECMWF analysis at 06Z. OMPS NM channels 61, 75, 77, 82, 94, 103, 117, 127, and 175 at the wavelengths 324.52, 330.36, 331.20, 333.28, 338.29, 342.05, 347.89, 352.06, and 372.09 in nm were assimilated. The nine channels were similar to the channels that are used in ozone retrievals. As shown in Figure 4, channels 117 and 175 are suitable for estimating the surface reflectance, and Channels 61, 75, 77, 82, 94 and 103 have good sensitivities to ozone. Figure 7 compares the difference of the normalized radiance for those channels between OMPS observations and the CRTM simulations for solar zenith angles less than 80 degrees. The CRTM radiative transfer model is a plane-parallel model that does not take the Earth’s curvature into account. Therefore, for solar zenith angles larger than 80 degrees, the CRTM simulation is not accurate. The collocated ECMWF forecast and the OMPS NM global data between 03Z and 09Z on 2 March 2021 were used. Since the collocated data for aerosols and clouds are not available, we did not apply the data control except for in the condition where the relative difference between the OMPS observations and the CRTM simulations were within 10%. Applying the condition excludes 4% of the data. In Figure 7, the standard deviations vary from 1.0% to 2.6%. The averaged bias and standard deviation are 1.4% and 1.8%, respectively.

In the following, we directly assimilate OMPS NM UV radiance. Due to the very limited computational resource, this radiance assimilation is based on a one-dimensional variation technique that is different from the three-dimensional variation algorithm used at NWP centers. This radiance assimilation technique is different from the MiRS retrieval algorithm. The similarity and difference among this algorithm, the MiRS retrieval algorithm, and three-dimensional radiance assimilation are given in Table 1. Figure 8 is the ECMWF global total ozone amount (DU) distribution from the 06Z analysis. The ozone amount in northern hemisphere is higher than that over southern hemisphere. The 6-h forecast is relatively close to analysis. The global mean difference and standard deviation between analysis and the 6-h forecast is about 0.22 DU and 3.60 DU, respectively (not shown). The difference for some small regions can be up to 50 DU. Our OMPS NM radiance assimilation slightly improves the ozone product. For the areas where the OMPS NM covers and the solar zenith angles are less than 80 degrees, our OMPS NM radiance assimilation slightly improves the ozone product. For example, the bias and standard deviation between the ECMWF analysis and our OMPS NM radiance assimilation at 06Z 2 March 2021 are 0.286 and 5.262 DU, respectively (not shown), while the bias and standard deviation between the ECMWF analysis and the ECMWF 6-h forecast for those radiance assimilated data points are 0.348 and 5.269 DU, respectively. The standard deviation improvement from our radiance assimilation is quite small. Nevertheless, the result is encouraging for a first demonstration of ultraviolet radiance assimilation. To look at the differences on a global map, one can look into the differences between two absolute differences: the absolute difference between the ECMWF ozone analysis and the ECMWF 6-h forecast and the absolute difference between the ECMWF ozone analysis and our OMPS NP radiance assimilation:

$$∆O_3=\left|O_3\left(analysis\right)-O_3\left(6h forecast\right)\right|-\left|O_3\left(analysis\right)-O_3\left(OMPS radiance\right)\right|$$

(10)

Equation (10) is often used in data analysis. A positive value of $∆O_3$ here means the absolute value of the second term on the right of Equation (10) is smaller (i.e., an improvement), which implies that $O_3\left(OMPS radiance\right)$ is better than $O_3\left(6h forecast\right)$, while negative values indicate that the absolute error of the assimilated analysis was larger than that of the 6-h forecast.

Figure 9 is the difference of the two absolute differences defined by Equation (10). We can see more positive values than negative values on the map. The assimilated OMPS radiance data are within $\pm 3$ h of the analysis time at 06Z 2 March 2021. Since we did not have aerosol and detailed cloud information for this study, a climatological aerosol model profile was used. The ECMWF forecast cloud liquid water and ice cloud water content are used and modeled as non-precipitating clouds with a constant effective radius. Quality control fully depends on the convergence of the minimization. Since the 6-h background is good and close to the analysis, our convergence criteria were simple: simply check whether the cost function has decreased during the iteration. The non-convergent cases are likely due to high amounts of aerosols and clouds. We also analyzed the OMPS NM radiance assimilation at 12Z on the same day. The outcome was very similar to the result at 06Z, with a small improvement seen using the OMPS NM radiance assimilation. The bias and standard deviation between the ECMWF analysis and our OMPS NM radiance assimilation at 12Z 2 March 2021 are 0.725 and 4.601 DU, respectively, while the bias and standard deviation between the ECMWF analysis and ECMWF 6-h forecast for those radiance assimilated data points are 0.777 and 4.605 DU, respectively. The geolocation of the OMPS orbital data around 06Z are different from that around 12Z.

7. Conclusions

This study is the first experiment to assimilate ultraviolet radiances for ozone amount analysis in the troposphere and stratosphere. The initial result is encouraging, although the improvement in the atmospheric ozone is small. The assimilation experiments were accomplished through several developments:

• Through this study, we found that assimilating the $N_{value}$ can overcome the difficulty of handling the large dynamic range of the normalized radiance (see Figure 4 and Figure 6). The definition of the $N_{value}$ is given in Equation (8), which is the logarithm (base 10) of the normalized radiance. The Jacobian of the $N_{value}$ can be calculated from Equation (9).
• Specifying an accurate surface reflectance is the most challenging aspect of the problem. The reflectance depends on many variables such as the surface type, surface roughness, sensor and solar zenith, and azimuth angles, with the coarse spatial resolution of the measurements (50 by 50 km²) contributing to sub-FOV inhomogeneity. In addition, the bias correction is more challenging than the bias correction in assimilating microwave and infrared measurements. In this study, the bias correction is mitigated in the treatment of surface reflectance. Since the assumption of a single constant surface reflectance is not accurate for the radiance simulation over longer wavelengths (see Figure 4), we propose using two estimated surface reflectance by fitting OMPS normalized channel radiances at 347.6 and 371.8 nm. The normalized channel radiances at 347.6 and 371.8 nm do not have significant sensitivity on atmospheric ozone (see Equation (4)). By assuming that the instantaneous surface reflectance changes linearly with wavelength, the spectral surface reflectance at 331 nm can be calculated from these two estimated surface reflectance values. This approach mitigates the uncertainty due to unknown surface reflectance and the bias between the measurements and simulations. This approach can also mitigate errors by using scalar radiative transfer calculations.
• To date, the CRTM model has not had the capability of simulating either fully polarized or scalar ultraviolet radiances. In this work we have extended the capability to simulate and assimilate ultraviolet radiances and have treated the radiances as scalar measurements for computational reasons. Fully polarized radiance calculations require more than 20 times the CPU time than the scalar radiance assimilation. In a fully polarized calculation for the four components of the Stokes vector, all of matrices are expanded by a factor of 4 by 4; therefore, the matrix inversion requires more than 16 times the computational resources compared with the scalar representation.

Other aspects to be considered:

• As part of the enhancements to CRTM allowing for both the scalar and polarized treatment of ultraviolet radiances, CRTM 3.0 will allow users to load cloud and aerosol optical look-up tables in either the current binary format or in the netCDF format. In addition to the current operational GOCART global aerosol modeling, Community Multi-scale Air Quality (CMAQ), which is more focused on regional applications, is also included in the CRTM. The openMP function is implemented in the CRTM as well. Users can simulate radiances using parallel computations at the atmosphere profile level or at the sensor channel level. However, as noted above, the computational requirements for fully polarized calculations are considerably larger than those for scalar calculations.

The result of this experimental OMPS NM radiance assimilation shows that the error in bias correction, surface reflectance, and neglecting polarization in radiative transfer calculations are largely mitigated by using the two estimated surface reflectance values. In the future, we may include the OMPS NM channel at 331 nm to estimate three surface reflectance values. A non-linear spectral reflectance model based on the three reflectance values may be applied to all other channels in the future.

The experiments reported here are a first step toward determining the feasibility of full ultraviolet radiance assimilation for a tropospheric ozone. While the experimental design was a simplified version of a full 3DVAR FGAT operational system, it allowed for the identification of the key challenges associated with assimilating these data and for assessing the potential within a 1DVAR framework. Since the results presented here are quite preliminary, we acknowledge that further study and efforts by the scientific community are needed, particularly assimilation experiments at operational NWP centers. It is hoped that the results shown here will be helpful for future research efforts.