Evaluating the Value of CrIS Shortwave-Infrared Channels in Atmospheric-Sounding Retrievals
Round 1
Reviewer 1 Report
Comments for author File: 
Comments.pdf
Author Response
Response to Reviewer #1
We would like to thank the reviewer for their excellent comments and for helping to make this a better paper. We made the suggested changes and discuss our specific responses in detail below.
line 150ff: This section gives a good introduction to the peculiarities of the O-E retrieval, but it is lacking a brief explanation on how the a priori is defined. Is it a complex covariance matrix encoding typical profile shapes or a simple Tikhonov-like smoothness constraint? This might be relevant for the retrieval behaviour using channels of different spatial localisation (if e.g. the covariance matrix does not allow for strong variations that might be invisible to LW measurements, but detectable to SW measurements).
We employ a regression operator as a first guess for T(p) and H2O(p); however, our a-priori error estimates are larger than the regression performance and are based on a static climatology for T(p), q(p), and all the trace gases, surface emissivity, and clouds. Since the regression operator has a dynamic and highly variable response to our spatially-derived cloud clearing we use a form of Tikhonov-like smoothing in the O-E retrievals with an embedded information content analysis of the geophysical noise (including the cloud clearing error) that derives a highly scene-dependent smoothing parameter. In Section 2.1 (line 157) we cite references for the algorithm – including the algorithm theoretical basis document – from which the reader can glean the details of the algorithm. We added a reference to the NUCAPS operational algorithm theoretical basis document (ATBD) and additional discussion at lines 157-159 and 190-193 to briefly summarize the a-priori for all retrievals.
The reference added for the ATBD is Barnet, C.D.; Divakarla, M.; Gambacorta, A.; Iturbide-Sanchez, F.; Tan, C.; Wang, T.; Warner, J.; Zhang, K.; Zhu, T. NOAA Unique Combined Atmospheric Processing System (NUCAPS) Algorithm Theoretical Basis Document; National Oceanic and Atmospheric Administration: Washington, D. C., USA, 2021; pp. 110.
line 350ff: The use of pseudo-brightness temperature differences seems a lot of work for a simple linear scaling of radiances that is removed later when the differences between measured and calculated radiances are weighted by the covariance matrix. Is the major intent of this section to educated communities, in which the use of actual brightness temperature and the involved non-linear ill-defined conversion is still prevalent?
The main message we are trying to convey is that the conversion of radiances to brightness temperature is not linear and whatever is done in the conversion of radiances must also be done in the conversion of the noise. The major intent of this section is two-fold. First, we would like to educate the community on how to use infrared radiances and how the NASA/NOAA IR retrieval community solved this issue by use of pseudo-BT’s. You are correct, that if done correctly, it does cancel out; however, the global NWP data assimilation community has done this incorrectly for ~20 years (i.e., converting radiances to BT without amplification of the noise) and we are trying to communicate that. Second, we wish to show the community that SW-only instruments are plausible and we are attempting to demonstrate this with real operational data.
line 509ff: I am curious with regard to the intent of the lengthy eigenvector discussion. A PCA is a very useful tool and it can be used here to show "how much information" is contained within the different regions. The major shortcoming is that this discussion cannot relate the information to the target quantities of interest. The related discussion on averaging kernels deliver exactly that, but restricted on the employed channels.
- Can you derive information about information on temperature DOF etc. from the linear regression? The diagnostics of derivation of A parameters and forecast accuracy should give some insight into this?
This is an interesting point and could be the basis of a follow-on study. The regression coefficients are, in essence, empirical averaging kernels as they represent the average over the training ensemble for each of the configurations studied. We felt an analysis of the performance statistics (bias and standard deviation of the retrievals as well as averaging kernels from the O-E steps) would be a better assessment of the information content of the SW versus LW configurations. Per your comment below we have added to our discussion of the AK’s details about the DOF for temperature and water vapor. These are now quantified for each of the configurations in the Figure 9 caption (lines 923-933).
- The outcome of the linear regression is used as initial guess in the O-E retrieval. A reasonable value is necessary for convergence, but the effort spent here should not be necessary. Would an initial guess derived from a single band (e.g. LW) not be sufficient to estimate the accuracy of the O-E retrieval using different bands? Or is the intent to fully demonstrate the feasibility whole retrieval system? Please clarify.
Our overarching intent is to compare complete retrieval systems without any influence from the denied bands. Many instrument vendors are considering SW-only configurations because of the advantages for smaller footprint sizes, lower cost, and lower launch risk. Also, the CrIS instrument on S-NPP has already had loss of bands – as discussed in the paper – such that we did not assume we had any access to the channels in the denied bands. Given the comments from reviewer #2 we have added some discussion of the intended audience and goals of this paper (lines 27–30) and lines 744-753 (along w/ comments requested by reviewer #2)
line 654: The optimization process may be beyond the scope of the paper, but the finally chosen channels should not. Please provide a table or a data-addendum listing the channels employed in the different O-E setups.
We added an Appendix with a table listing the channel selections for all retrieval steps (i..e, T(p), q(p), surface parameters, and trace gases) and for each configuration in this study. We reference the Appendix at line 729.
line 788ff: The discussion using the averaging kernel matrix diagnostic is sadly rather short, even though it gives the most information on how the measurements improve the different quantities. It would be interesting to see the (averaged) trace of the averaging kernel matrices for temperature and water vapour for the different retrievals to see how many DOF in a table to have a quantified comparison. The figures are very illustrative also about the distribution of information, but it is difficult to directly compare the retrievals due to the small differences.
We edited the caption of Figure 9 (lines 923-933) with quantities for the DOFs of temperature and water vapor to substantiate our discussion. Specifically, we say “The sum of all values along the vertical axis of these AK vectors (i.e., the trace of AK matrices) is known as the degrees-of-freedom-for-signal (DOFS) and summarizes retrieval skill as follows: (left) LW+MW+SW system has DOFS(Tp) = 3.6 ± 0.4 and DOFS(H2Ovap) = 2.4 ± 0.7, (middle) LW+MW system has DOFS(Tp) = 3.3 ± 0.4 and DOFS(H2Ovap) = 2.4 ± 0.7, and (right) MW+SW has DOFS(Tp) = 3.7 ± 0.3 and DOFS(H2Ovap) = 2.3 ± 0.6.”
In particular, I do not understand how removing information (i.e. the LW channels) should improve the temperature product. The LW+MW+SW retrieval should - by definition - contain the most information about temperature and water vapour.
If removing the LW channels improves the temperature information, this must be an artefact because some background gases are not retrieved anymore. I.e. some of the information previously required to constrain other targets may now improve temperature. This makes the results very difficult to interpret, in particular as there may be not enough information left to retrieve a trace gas profile, but it might still have an impact.
One would expect that the LW+MW+SW system should always outperform all other configurations; however, a multi-instrument (i.e., CrIS+ATMS) and multi-band retrieval with real data can have many competing issue – beyond retrieval artifacts. For example, there could be real calibration issues where the instruments and/or bands are not perfectly calibrated and co-located. This might be relevant when comparing the CrIS+ATMS versus CrIS-only results. The fact that the LW-band does have strong interference from ozone, nitric acid, and other gases could be a reason why removal of the LW band improve results since errors in ozone can confound the convergence of LW w.r.t. the other bands and ATMS. It is worth noting that the LW+MW+SW system has been optimized over two decades, first with Aqua/AIRS/AMSU and later with Metop/IASI/AMSU/MHS and finally with S-NPP/CrIS/ATMS. The other configurations have only had the quick-look optimization that we have performed in this paper. Thus, the small improvement of the LW+MW over the LW+MW+SW system is something we will explore in a future paper.
With the exception of ozone, the trace gas retrievals cannot impact the results presented here, since in NUCAPS, the trace gases are retrieved after temperature (T) and water vapor (q). That is, we designed the system to have minimum sensitivity to trace gas errors, given that many of the trace-gas retrievals are considered experimental. We have realistic climatological a-priori’s for all trace gases in our systems, so it is not likely that removal of non-ozone trace gases will have any impact on the T/q results.
In the case of ozone, the loss of information content in the LW-band is real and does play a small, indirect role in the performance assessment (since ozone is treated as a geophysical source of noise, and now the geophysical noise is larger). Again, we want to assess the full-retrieval capability for each band – including the loss of information about the trace gases that cannot be retrieved.
We added a few sentences to clarify this discussion in lines 744–753.
In line 667 you mention that three trace gases were not retrieved in setups without LW channel. The retrieval should be able to work with those trace gases enabled (obviously no sensible information should be retrieved). But with the same setup the information content of temperature should be comparable.
- Please explain why temperature improves for a reduced number of channels.
- Did you examine using the three setups with all trace gases derived?
- Figure 3 indicates that the SW channels have much better localization below 500 hPa. Shouldn’t this be reflected in an increased information content at this altitude in the LW+MW+SW retrieval compared to the LW+MW?
For item #1 see discussion in previous section.
For item #2 we cannot derive some of the trace gases when certain bands are denied. When certain bands are removed we no longer have sensitivity to gases that have unique signatures in those bands. For example, when removing the LW band we are left with negligible sensitivity to ozone in the other bands. When removing the SW-band we completely lose all sensitivity to carbon monoxide. We could select other channels in the remaining bands; however, the sensitivity will be so low (or possibly zero in the case of some gases) that the retrieval will simply return the a-priori – hence, it is more realistic for an operation system to simply turn it off. On the other hand, a negligible sensitivity to a gas also implies that it will not be a strong interference gas and will not have a large impact of temperature or moisture. Therefore, we did not feel that a constant a-priori of the trace gases between configurations was warranted or desirable.
For item #3 we expect that the SW channels can have better performance because of the quality of the channels. But this is a complicated trade-off between vertical resolution, noise, and interference signals that we are attempting to explore in this paper. Fig. 3 is only showing half the subject – the Jacobian or what we are calling the “signal”. Yes, the SW channels have better vertical localization, but they also have significantly higher noise – and both signal and noise both matter for the SW. The SW also has better “spectral purity” – that is less sensitivity to interference signals – but again, not enough to overcome the degradation in noise. Future work will explore the impact of lowering the CrIS-noise in the SW-band; however, that is a much more difficult experiment and it out of scope for this paper.
Reviewer 2 Report
This manuscript contains an overview of NOAA’s operational retrievals, NUCAPS, with references to the authors’ work with CLIMCAPS. Their investigations cover various topics including their retrieval techniques, channel selections, information content, cloud clearing procedures and signal to noise comparisons. However, this manuscript does not contain any relevant experiments or comparisons pertaining to data assimilation.
The classic definition of data assimilation requires a forecast model. There is no mention of an atmospheric model being used in deriving NUCAPS or CLIMCAPS retrievals. The only mention of an atmospheric model being used is the ECMWF global model for verification. There are also many variants of data assimilation including optimum interpolation, variants of the Kalman Filter, and 1 – 4DVAR. Each has its own strengths and weaknesses, thus their own implementations. Blanket statements about the behavior of specific instruments (CrIS) in data assimilation are not correct in all cases. An example of this is the sentence that starts in line 71. Depending on the type of data assimilation, the sentence could be correct. Other types of data assimilation invoke physical mass-wind balance equations. Some are 1-dimensional and do not have enough information. Yet others have no such constraints. There are several of these types of comments throughout the manuscript.
Your target audience is not specified in this manuscript. I am assuming your data assimilation references are targeted at numerical weather prediction centers, as stated in your concluding sentence. In this case, your temperature precision plots in figure 7 (c and d), the standard deviations are about an order of magnitude higher than what are commonly used by numerical weather prediction centers. They typically use 1 - 6 hour forecasts. I fail to see the relevance of this work for data assimilation by numerical weather prediction centers.
The paragraph starting at line 761 gives reasons why you are not using non-LTE channels, yet your conclusion states “the SW-band is a viable replacement for the LW-band in retrieval systems and probably in data assimilation”. You haven’t proven this statement.
This manuscript would be fine as an overview of NUCAPS and CLIMCAPS without the data assimilation speculations.
Author Response
Response to Reviewer #2
We would like to thank the reviewer for their excellent comments and for helping to make this a better paper. We made the suggested changes and discuss our specific responses in detail below.
This manuscript contains an overview of NOAA’s operational retrievals, NUCAPS, with references to the authors’ work with CLIMCAPS. Their investigations cover various topics including their retrieval techniques, channel selections, information content, cloud clearing procedures and signal to noise comparisons. However, this manuscript does not contain any relevant experiments or comparisons pertaining to data assimilation.
Our goal in this work was two-fold. First, to perform retrieval experiments that could provide guidance to other applications, such as data assimilation and second to explore the specific value of the CrIS SW-band. We have added some discussion of the intended audience and goals of this paper (lines 27–30) and lines 744-753 (along w/ comments requested by reviewer #1).
We will discuss our relevance to DA in response to your other concerns below.
The classic definition of data assimilation requires a forecast model. There is no mention of an atmospheric model being used in deriving NUCAPS or CLIMCAPS retrievals. The only mention of an atmospheric model being used is the ECMWF global model for verification. There are also many variants of data assimilation including optimum interpolation, variants of the Kalman Filter, and 1 – 4DVAR. Each has its own strengths and weaknesses, thus their own implementations. Blanket statements about the behavior of specific instruments (CrIS) in data assimilation are not correct in all cases. An example of this is the sentence that starts in line 71. Depending on the type of data assimilation, the sentence could be correct. Other types of data assimilation invoke physical mass-wind balance equations. Some are 1-dimensional and do not have enough information. Yet others have no such constraints. There are several of these types of comments throughout the manuscript.
Yes, we agree that we made a major mistake in not qualifying which data assimilation systems we were referring too. You are correct that we are talking about NWP data assimilation applications and, specifically, the global models. We are aware that both NCEP/GFS and ECMWF require the modifications we are discussing and that they are currently using the CrIS instrument improperly (i.e., conversion of radiances to BT without amplification of the noise) and, therefore, would not have the correct impact if they assimilate the CrIS SW band. We are not aware if other centers or other DA applications are also misusing the CrIS (or other infrared) instruments. Again, since instrument vendors are pursuing SW-only designs, we felt this paper is relevant to the data assimilation community. There were numerous small changes to many lines throughout the manuscript where we changed “DA” to “NWP global DA”
Your target audience is not specified in this manuscript. I am assuming your data assimilation references are targeted at numerical weather prediction centers, as stated in your concluding sentence. In this case, your temperature precision plots in figure 7 (c and d), the standard deviations are about an order of magnitude higher than what are commonly used by numerical weather prediction centers. They typically use 1 - 6 hour forecasts. I fail to see the relevance of this work for data assimilation by numerical weather prediction centers.
We added a statement at lines 27 to explicitly state our goal to provide guidance to the global data assimilation community through our experience with operational retrieval systems. We also re-enforce our intent at line 529, line 744
Fig. 7 represents the state-of-the-art in operational atmospheric retrieval systems for systems that have global yields (i.e., function in cloudy environments for all Earth locations) and have no dynamic dependence on model temperature or moisture or trace gas fields. We have made no claim that these retrievals should be assimilated in place of the radiances. We are only using these retrievals - in a relative sense - to show the impact of loss of bands as guidance to the NWP global data assimilation community.
When we compute statistics of GFS vs. ECMWF we find the differences are about half of the retrieval statistics shown in Fig.7 (in lower troposphere it is 0.5 K for T(p) and 10% for q(p)). The differences between GFS and ECMWF are roughly equal to our retrievals near the tropopause. These retrievals have very little knowledge of the specific scene we are retrieving. Trace gases have relatively simple a-priori’s and our regressions are based on a static relationship to ECMWF – i.e., we have no knowledge of any model field for the specific scenes we are retrieving. In addition, the particular focus day chosen for our performance assessments were not included in any training. For example, we do have a state-of-the-art static climatology that shows ~4K, ~30%, 15% RMS differences for T, q, and O3 for the focus day we chose and while the regression a-priori is much better than that we impose large error estimates (i.e., regularization of our O-E retrievals) to ensure our retrieval are stable over the entire globe. Models, as you point out, have specific knowledge of the scene from the previous cycles and, therefore, their a-priori is substantially better than a-priori knowledge used in our retrievals.
We also frequently compare to the CLIMCAPS LW+MW+SW+ATMS system that uses a model a-priori (MERRA-2) instead of a regression operator for our T(p), q(p), and O3(p) a-priori. It has similar statistical performance as GFS with respect to ECMWF and with respect to dedicated radiosondes and other in-situ measurements – as one might expect given the improvement in the a-priori knowledge.
As stated, we clarified the goal of our work as having relevance specifically to the NCEP and ECMWF global DA models; however, some of our methodology (e.g., use of pseudo-BT’s) may be guidance to other applications, including DA. We also hope to encourage NWP centers to use the CrIS-SW band and to entertain the idea of SW instruments in the future.
The paragraph starting at line 761 gives reasons why you are not using non-LTE channels, yet your conclusion states “the SW-band is a viable replacement for the LW-band in retrieval systems and probably in data assimilation”. You haven’t proven this statement.
We are not using non-LTE in our regression step for the MW+SW configuration; however, we are using non-LTE channels in the O-E for all configurations, including the MW+SW – therefore, it is appropriate to say that the SW-band is a viable replacement. This is a robust conclusion because we are doing a full denial of the LW-band – that is, it is not used in any context within our systems – and MW+SW+ATMS behaves as well as the LW+MW+ATMS – which would be the system most likely to be operational.
We did run a system in which we used the non-LTE channels in our regression step and it behaved slightly worse for the CrIS-only system so we chose to show the system without non-LTE channels for the regression operator for both CrIS+ATMS and CrIS-only so those two systems could be compared directly. We hypothesized in the text that the reason the regression performs poorly for CrIS-only MW+SW is because the LW channels (through the eigenvectors) need to be present in the training of the regression coefficients. We have reason to believe that the real reason the MW+SW system degrades has to do with the CrIS instrument noise, which is one to two orders of magnitude higher than the LW for cold scenes. We have some results to show that this is, in fact, the reason for the degradation but the results are too preliminary for this paper. We do say in our final paragraph that our results suggest we need improvements in the non-LTE correction (relevant to both retrievals and DA applications) and that future instrument designs should have lower noise if they do not employ a LW-band
At line 945 we also added the caveat that our results show the CrIS SW is a viable replacement for the LW when the ATMS instrument is included and made some changes in the final paragraph to clarify that more work is needed on non-LTE corrections.
This manuscript would be fine as an overview of NUCAPS and CLIMCAPS without the data assimilation speculations.
We hope that the changes described above will be sufficient for us to say that our retrieval results and methodologies we employ (i.e., the pseudo-BT discussion) are relevant to the NWP DA community. In the past few years both NCEP and ECMWF have been testing the CrIS-SW band in their DA systems as a result of our research and discussions. They will report their results in separate publications and we were hoping they could cite our paper as the impetus for their experiments.
Reviewer 3 Report
This manuscript is well written and provides great background on CrIS shortwave infrared channel retrievals. The authors talked a lot about the radiance DA. The focus is on the CrIS product retrievals.
P1L16: “SW” to “shortwave (SW)”
P2L91: “(i) non-LTE effects that are historically difficult to model;” references needed. Similar to (ii) and (iii).
P5L210: “From light to dark, cloud fractions are 0%, 42% and 88%.” Why 0%, 42%, and 88%?
P6L266: “in the scientific literature.” References needed.
P6L270-271: “All of our conclusions, however, would be unchanged if another forward operator was used.” What does it mean of different forward operator? What are the differences between different forward operators?
P7L323: “using using” to “using”
Author Response
Response to Reviewer #3
We would like to thank the reviewer for their excellent comments and for helping to make this a better paper. We made the suggested changes and discuss our specific responses in detail below.
This manuscript is well written and provides great background on CrIS shortwave infrared channel retrievals. The authors talked a lot about the radiance DA. The focus is on the CrIS product retrievals.
P1L16: “SW” to “shortwave (SW)”
Done, edited line 16 from “SW” to “shortwave (SW)”
P2L91: “(i) non-LTE effects that are historically difficult to model;” references needed. Similar to (ii) and (iii).
We have citations within the sections that discuss topics (i), (ii), and (iii). We felt it would read better if we simply said “these will be discussed in more detail in Sections 2.2.1 and 2.2.2” instead of adding references at this introductory stage.
P5L210: “From light to dark, cloud fractions are 0%, 42% and 88%.” Why 0%, 42%, and 88%?
This figure is real data and we purposely searched for a scene in which there was high cloud contrast between the fields-of-view. The 0%, 42%, and 88% are our retrieved estimates of the cloud amount for this scene. We edited the Figure 1 caption (lines 250–253) to make this clear.
P6L266: “in the scientific literature.” References needed.
We were trying to be sufficiently vague since this is off-topic but you are keeping us honest and we will include a reference. The best coverage of this topic that I have seen is Chapter 2 of Andrews, D.G., J.R. Holton and C.B. Leovy 1987. Middle Atmospheric Dynamics. Academic Press p.-2-489. ISBN 0-12-058575-6, 489 pgs.
P6L270-271: “All of our conclusions, however, would be unchanged if another forward operator was used.” What does it mean of different forward operator? What are the differences between different forward operators?
The differences are too detailed and too numerous for this paper. The main point; however, is that the DA application uses CRTM (Community Radiative Transfer Model) and we are using SARTA and both models are extremely accurate. The differences are really due to implementation within the application. We added the following sentence to reflect our meaning (lines 320–223): “For example, the Community Radiative Transfer Model (CRTM) that is used by NWP has the same accuracy and characteristics as SARTA and the main differences are in their application-specific implementation”
P7L323: “using using” to “using”
Done.
Round 2
Reviewer 2 Report
There is still no proof that this retrieval scheme is consistent with any numerical weather prediction data assimilation system. There are several comments suggesting the opposite. Retrievals are one-dimensional, NWP data assimilation systems that have an atmosphere are at least 3-dimensional. Data assimilation systems also have quality requirements of spatial (3D) (and sometimes temporal, 4D), physical and statistical consistencies, which are only 1-dimensional in a retrieval.
The real test of LW vs SW will require cycling a global data assimilation system where the analysis errors (improvements) are allowed to propagate through the atmosphere with time. Retrievals are NOT capable of mimicking this important test.
The estimations made for non-LTE effects and bi-directional reflectance may be adequate for NUCAPS and CLIMCAPS but I have not seen where they are adequate for NWP data assimilation. Yin [3],Chen et al. [4], and DeSouza-Machado et al. [5] have made considerable (order of magnitude) improvements but still fall short of being replacements for the long wave channels. 1K degree biases [3] in the non-LTE channels during the day are still too large. There is also no mention of how to account for auroras and other solar wind issues in the Mesosphere-Ionosphere. The various “contaminants” at night can also make their use limited. Similar concerns are warranted for [4] as wave amplitude, period and direction are not adequately known to solve surface emissivity and other surface parameters.
Line 77: “interaction with model dynamics” is only true of 4-dimensional data assimilation systems. 3-dimensional systems have both physical and statistical constraints but do not have interactions with the forecast model or its adjoint.
Line 96: (ii) “measures reflected solar irradiance” is not discussed in 2.2.1 or 2.2.2.
Line 98 and 393: You are correct, most data assimilation groups do not account for the non-linear effects of the Planck function. It is also not that relevant as the NEDT/NEDN is not the dominant (largest) error.
Line 130: You have not proven that you can use NUCAPS to provide guidance for DA applications.
Paragraph line 871: You have shown that the various combinations are adequate for NUCAPS retrievals but have not shown anything related to data assimilation.
Author Response
Response to reviewer #2
We would like to thank reviewer #2 for a rapid and constructive review. We have carefully examined the suggestions and thank reviewer #2 for helping us improve the manuscript. Our response to the individual items is provided below.
There is still no proof that this retrieval scheme is consistent with any numerical weather prediction data assimilation system. There are several comments suggesting the opposite. Retrievals are one-dimensional, NWP data assimilation systems that have an atmosphere are at least 3-dimensional. Data assimilation systems also have quality requirements of spatial (3D) (and sometimes temporal, 4D), physical and statistical //consistencies, which are only 1-dimensional in a retrieval.
Response: We agree that there are substantial differences between retrievals and 1DVAR, 3DVAR, 4DVAR, and ensemble data assimilation approaches. We are mostly focusing on the similarities in the analysis where both retrievals and data assimilation minimize a cost function and they share many components of how error sources, including forward model error, instrument error, and background (or a-priori) errors are realized. We reviewed each reference we made to data assimilation within the manuscript and softened, and in some cases removed, discussion about data assimilation.
The real test of LW vs SW will require cycling a global data assimilation system where the analysis errors (improvements) are allowed to propagate through the atmosphere with time. Retrievals are NOT capable of mimicking this important test.
Response: That is correct and is why there is parallel work to test these concepts in NWP OSE’s. The skill of the SW-band will be assessed by experts at NCEP, GMAO, and ECMWF and reported in separate papers. Again, we softened our statements about the value of using retrievals for assessment of data assimilation.
The estimations made for non-LTE effects and bi-directional reflectance may be adequate for NUCAPS and CLIMCAPS but I have not seen where they are adequate for NWP data assimilation. Yin [3],Chen et al. [4], and DeSouza-Machado et al. [5] have made considerable (order of magnitude) improvements but still fall short of being replacements for the long wave channels. 1K degree biases [3] in the non-LTE channels during the day are still too large. There is also no mention of how to account for auroras and other solar wind issues in the Mesosphere-Ionosphere. The various “contaminants” at night can also make their use limited. Similar concerns are warranted for [4] as wave amplitude, period and direction are not adequately known to solve surface emissivity and other surface parameters.
Response: With over 20 years of operational use of the DeSouza-Machado method in operational Aqua/AIRS, Metop/IASI+AMSU+MHS, and S-NPP and JPSS/CrIS+ATMS we have seen no evidence of measurement impacts of the mesospheric effects, such as those mentioned. We agree that the items you mention are all plausible and are of potential concern; however, they are outside the scope of the present work. Most likely any scene with uncorrected non-LTE, aurora, etc. would be rejected since our minimization residuals would be high and if it disagreed with the microwave it could explain the degraded performance we see with the CrIS-only system. It should be recognized that this paper alone will not cause NWP to drop the LW band in favor of the SW CrIS band; however, we do hope that this paper will encourage some to investigating the potential of using SW CrIS channels for the information content they contribute. We agree that more research is needed on non-LTE corrections. We would also add there is no reason why the LW band should be immune to the effects you mentioned and there are many similar issues that can be raised about approximations that many applications assume are correct for the LW CrIS band.
The issue you raise about the emissivity/reflectivity in [4] is actually related to item (ii) of line 96 comment and will be addressed there.
We modified line 916-917 from:
“Finally, more work should be done on operational corrections for non-LTE when instruments do not have LW or microwave information that can be utilized for non-LTE correction training.
To:
“The NUCAPS retrieval results demonstrate that the SW-bands can be used for CrIS; however, it is possible that non-LTE corrections are not currently sufficiently accurate for data assimilation applications. Additional research may be necessary for operational non-LTE corrections for NWP applications, especially for new satellites that do not have LW or microwave information that can be utilized for non-LTE correction training. “
We also added a sentence at the end of 2.2.2: “It is worth noting that if there are large non-LTE correction errors the retrieval will be rejected due to failed convergence of the minimization. Other applications may need to test the residuals of channels sensitive to non-LTE.
Line 77: “interaction with model dynamics” is only true of 4-dimensional data assimilation systems. 3-dimensional systems have both physical and statistical constraints but do not have interactions with the forecast model or its adjoint.
Response: We agree and removed “due to the interaction with model dynamics” from line 76/77.
Line 96: (ii) “measures reflected solar irradiance” is not discussed in 2.2.1 or 2.2.2.
Response: We modified the following paragraph
“… because the SW band (i) is subject to non-LTE effects that are historically difficult to model; (ii) measures reflected solar irradiance, in particular glint over ocean surface, which introduces additional, complex considerations in forward operator calculations; and (iii) is subject to non-linear effects of the Planck function. All three issues will be discussed in more detail in Sections 2.2.1 and 2.2.2 and have been resolved in modern retrieval systems. “
to read as follows
“… because the SW band is (i) subject to non-LTE effects that are historically difficult to model; (ii) sensitive to reflected solar irradiance; and (iii) subject to non-linear effects of the Planck function. Issues (i) and (iii) will be discussed in more detail in Sections 2.2.1 and 2.2.2, respectively. For issue (ii) NUCAPS solves for spectral emissivity and spectral effective reflectivity (i.e., we allow for changes to the solar irradiance) and, as such, it is part of the NUCAPS state vector and error budget. In applications were the surface emissivity, reflectivity, and solar irradiance are assumed known special care must be taken - especially over specular reflecting ocean and lake regions. Some daytime scenes may need to be ignored (i.e., glint conditions)“
Line 98 and 393: You are correct, most data assimilation groups do not account for the non-linear effects of the Planck function. It is also not that relevant as the NEDT/NEDN is not the dominant (largest) error.
In hyperspectral infrared retrievals we have robust results that demonstrate our RMS performance metric monotonically degrades with increasing instrument noise for AIRS, CrIS, and IASI. For some spectral regions, the instrument noise has some margin but in other spectral regions we feel that instrument noise could be significantly improved. For example, the AIRS instrument team invested heavily in reducing the noise in the 650-690 and cm-1 and 2380-2400 cm-1 spectral regions, but that specification was relaxed for IASI and CrIS. Our results in this paper are consistent with the fact that the CrIS SW band has higher noise than the AIRS SW band and higher noise than the CrIS LW band. If data assimilation is more accurate than retrievals, then data assimilation should be even more sensitive to instrument noise. We also do not want to send a message to instrument vendors that the instrument noise is over-specified for modern instruments such that they can relax it further.
It might be possible that data assimilation can get away with ad-hoc noise estimates for the LW band, but one of the main points of this paper is that any objective assessment of the hyperspectral infrared SW bands with constant NEDT – which is what NWP centers appear to be inclined to do – would be incorrect by 2 orders of magnitude. In the MW band it is in error by an order of magnitude.
We are not sure what correction to our paper is being requested, but we stand behind our results and feel that this warrants publication because this aspect has been overlooked by NWP centers and other retrieval systems for decades.
Line 130: You have not proven that you can use NUCAPS to provide guidance for DA applications.
Response: Historically, the NUCAPS retrieval system has been used to provide global NWP centers with preliminary information (i.e., guidance) on new hyperspectral infrared instruments. The initial instrument characteristics and channel lists were provided to all NWP centers for AIRS, IASI, and CrIS and some of those channel sets are still in use. As we said in the first review, the AIRS, IASI, and CrIS instruments all employ a SW band that has, for the most part, been ignored by NWP. The NUCAPS (and CLIMCAPS) retrievals have used these channels for over two decades. The research presented in this paper is exploratory and the information content analyses is relevant to global NWP centers worldwide – especially when considering which CrIS bands to assimilated when bands fail. We are utilizing the CrIS instrument and NUCAPS system because it is a relatively new instrument and this instrument has lost bands on S-NPP. There is also familiarity with CrIS at NWP centers and, therefore, it is a good example for the weather community.
We are using a single-satellite retrieval and can isolate the specific instrument impacts through our denial experiments. This has proven valuable in the early mission phases. The reality is that doing these kinds of experiments in operational NWP centers is expensive and difficult.
Both retrievals and data assimilation employ subsets of channels and both attempt to characterize the instruments they are using. The lessons learned that we included within this paper are derived from experience using SW radiances within 20+ years of operational retrievals. We believe this is relevant as guidance to NWP DA applications especially given that (a) instrument vendors are moving towards SW solutions and (b) the operational NWP modeling centers have very little experience with the utilization of the SW.
But we are not going to belabor this point. We have modified lines 128-129 from: “We can use NUCAPS to provide guidance for DA applications using hyperspectral infrared sensors”
To: “We will use NUCAPS retrievals to assess and intercompare the information content of the LW, MW, and SW bands of CrIS.”
Paragraph line 871: You have shown that the various combinations are adequate for NUCAPS retrievals but have not shown anything related to data assimilation.
Response: We removed “and DA systems” from line 874 and added the following sentence at the end of the paragraph: “These retrieval results and the discussions within Section 2 and 3 suggest that a low noise SW band could be a plausible replacement for the LW-band in future instruments; however, more work is required to fully demonstrate that.”
Author Response File: Author Response.pdf
