On the Accuracy and Consistency of Quintuple Collocation Analysis of In Situ, Scatterometer, and NWP Winds

Round 1

Reviewer 1 Report

The manuscript presents an extension of the triple / quadruple collocation approach which has been used to validate data from different sources (e.g. satellite missions, point data, model output). It provides a generalisation to an arbitrary number of observing systems, and demonstrates the approach with an example of five systems. The accuracy of the approach is evaluated using a sensitivity study based on a large number of system configurations as input.

Overall, the text is well-structured and formulated clearly.

Some comments:

- the paragraph starting on line 161, ending on line 180: it is not fully clear from the description if the error covariance e_ij is taken into account here or not. Please clarify in the text.

- same paragraph: I’m guessing that in the case described here, one of the systems has been used as a calibration reference? Please make this more clear in the text.

- same paragraph: you are taking logarithms to “linearize” your equations. This is, in general, not a valid approach since it depends on the properties of the measurements. Can you please elaborate why this can be applied here?

- line 250: ends mid-sentence, it appears that something is missing here.

- Fig. 1: where do you think the discrepancy between ASCAT-A and –B comes from? I’d assume they would be more similar over the shown range of spatial scales.

- Fig 5 / paragraph from line 388 to 406 (correlations between ECMWF/ASCAT-A/ASCAT-B): ASCAT-A and –B are processed independently (in particular with independent instrument calibration tables during the L1 processing) so it’s not clear which part of the processing chain that could introduce this kind of (negative) error correlation. Could this be an effect of the sampling (orbit configuration), and the number of winds assimilated by ECMWF from each instrument?

Author Response

See attached file

Author Response File: Author Response.docx

Reviewer 2 Report

Summary:

This manuscript present an extension of the triple collocation methodology to quintuple collocation in addition to a more general set of solutions for higher-order collocations. The presented approach develops the collocation model, develops an approach to develop estimates of the accuracy of the approach using synthetically generated noise and finally applies the approach to a quintuple collocation dataset. The authors provide discussion of the approach and the relationship between analytical and least-squares solution to the problem, the latter of which are developed through the application of logarithms to develop an ordinary linear system relating error covariances.

 

Recommendation:

Overall, I thought the paper was well motivated in terms of working to identify a more general framework for multiple collocation analysis and to develop approaches for assessing the accuracy of estimates. Unfortunately, I found the general development of the model to be severely lacking in clarity and thus found it difficult to understand the many finer points of this extended model and associated discussion. That said, I do think the formulation is relevant and potentially important. I do not doubt that the authors are intimately familiar with the collocation framework nor do I necessarily question any specific results of the manuscript. Thus, I do believe this work is worthy of acceptance but with a major revision to include more extensive development of the model and use of notation.

 

Comments:

1.     The most significant issue I have is that the development of the model does not feel self-contained to any extent. The authors begin with a relatively straightforward development of model equations relating system observations to truth through linear scaling and offsets. However, they quickly begin to use what I would consider non-standard notation for many of the terms. For example, they use <> brackets for denoting averaging over samples (why not simply use an overbar), and further then redefine the averages <x_i> by another variable name M_i (presumably, [M]ean).  They then develop an equation for covariances; there, a capital C_ij is used but that sort of makes me think of general covariance matrices. Equation (4) is provided where the authors possibly redefine Cij  but it is not clear, nor is it clear in what manner this equation was derived and its relationship to the ordering of systems by spatial resolution. At various points the authors note the potential for dropping or including different error covariances and how they may or may not be included. I strongly suggest developing a more formal “complete” model for equation (1) (or explict statement of the system of equations) that includes the all the different error components, and then develop simplifications thereupon. The authors then move on to the development of matrix equations (see lines 158  - 181) with little rigorous development of notation. Development of the appropriate vectors and matrices and their elements, similar to the type of development in the appendix, would substantially benefit the manuscript. In fact, the development for a consistent example (say for either 3 or 4 observing systems) would be very helpful. Note that the authors originally use the notation “k” for samples but then at some point switch to using k to index the different systems (e.g. equation 7). More consistency should be used.

2.     It is still not precisely clear to me why the spatial representativeness error components cannot be determined through the analysis. It would seem that adding more collocations (and hence more equations) would allow one to determine more parameters of a general model. What is the added benefit of having more than 3 collocations, if not for the potential to develop a model that can be used to identify/constrain more parameters?

3.     How is the spatial representativeness distance “s” determined? Is this simply the distance between each non-ECMWF sample point and the nearest ECMWF grid point? Is it appropriate to treat the ECMWF grid point as an “instantaneous” point in space to support descriptive statistics of the spatial variability for distances less than the effective spatial resolution of the ECMWF background?

4.     There are other alternatives and closely related models to multiple collocation such as the three-cornered hat method (there are n-dimensional extensions as well). Given the focus here on the development of a general n-system collocation, it would be interesting to discuss the relationships and potential advantages to these alternative methods.

5.     One of the results appears to indicate that the least-squares solution is up to a factor of 2 different than the model average solution. Does this mean one should not use the least squares solution? If not, then what is the utility of the development of the least square approach outlined in Equation 6.

6.     Equation (13): How is this derived? Throughout the manuscript, the authors discuss the number of potential models, solvable, and unsolvable. But, this terminology is difficult to follow. On one hand, it seems they argue that the system is generally overdetermined, so it is not clear exactly what is *not* solvable. Perhaps a more thorough development of the model and all the relationships and parameters that need to be solved for would better clarify.

Author Response

See attached file

Author Response File: Author Response.pdf

Reviewer 3 Report

This manuscript is a follow-on to a paper published by the same authors in 2021. Their main objective is to compare collocated ocean wind vector data from three or more sources (scatterometers, buoys, numerical weather prediction models) in a way that allows them to derive meaningful sets of regression coefficients and to quantify and compare the individual accuracies and effective spatial resolutions of the input data sets. In the paper from 2021, data from four different sources were compared. In this latest work, a fifth data set is added, and a more generalized and more sophisticated mathematical approach is formulated in order to discuss fundamental properties of this kind of multi-dimensional regression analysis in more detail and to come to more robust conclusions regarding the qualities of the wind data sets under consideration.

I don't have personal experience with this kind of multi-dimensional data analysis, so it is difficult for me to tell how the proposed approach compares to the existing literature in this field. However, my impression from the manuscript is that the authors know very well what they are doing, and they have invested a lot of effort to develop and explain their mathematical approach, to study carefully what it can do under what conditions and if the input data satisfy these conditions, and to quantify the properties of the individual wind data sets based on useful quality measures. The manuscript is well written and includes adequate figures and tables to illustrate the findings. It shows that the authors have many years of experience in this field. I don't see any obvious issues that would require a revision before the manuscript can be accepted for publication.

 

Accordingly, this is one of the rare cases in which I can recommend the first submitted version of a manuscript for publication "as is" without any need for modifications.

Author Response

See attached file

Author Response File: Author Response.docx

Round 2

Reviewer 2 Report

I found the authors' response to address my primary concerns. It is clear that a standard notation has not generally been used prior and it is also true that brackets may be used to denote averaging (though I usually see this to indicate ensemble averaging, as the authors note).  Their addition of Appendix A and additional clarifications have improved the ability to follow their proposed approach.

Small edits/comments:

Line 168: Should a1=1  not 0?  

Line 208: Check number of referenced equations (8) and (9); do these need to be updated?

 

Author Response

See attached file

Author Response File: Author Response.pdf