Impact of BeiDou Observations on the Accuracy of Multi-GNSS PPP in a Function of Observing Session Duration within Europe—Analysis Based on Open-Source Software GAMP

Round 1

Reviewer 1 Report

In their Multi-GNSS study, the authors investigated how the inclusion of the BeiDou navigation system in the assessment affected PPP results across Europe, and they even claim to have seen an improvement in results with the addition of the BeiDou system. The title of the article fully expresses what is intended to be done in the study, but some structural improvements are required in the body part. There are also problems in the presentation of the results and their compsrison to the existing studies in the field. In the study, it is emphasized that BeiDou makes a positive contribution in some parts, while in some parts it is stated that the results deteriorate. The reader is confused because the authors are not able to express this clearly. If the above-mentioned issues are reviewed and the manuscript is rearranged, it is considered that the issues discussed in this paper will contribute to the reader in the relevant field. In this regard, some points are detailed below and some points are marked up in the body text and given as an attachment file to this review:

 

#1 The abstract has not been written properly. It should be rearranged by giving the importance and hypothesis of the paper in the first part, by explaining what has been done in the following parts, and by giving the most striking result in the last sentence.

#2 The introduction is generally well written, but there are minor errors in the use of English and these should be revised. The manuscript contains similar errors all through the text, and these should be revised once more carefully. Some of these errors are marked in the attached document and the authors need to review them and make the necessary improvements.

#3 Most parts of the text use something called STD Coefficient. What does it mean? If this means standard deviation, it should only be used as standard deviation or standard deviation values. Markings related to this are also made manually in the attached document.

#4 The color scheme used in Figure 2 is a bit confusing. One color is used for the minus values ​​of the same numeric values, and another color is used for the plus values. For example, blue is used for -0.01 m and orange is used for + 0.01. The same color should be used for results with the same numeric value but different sign. The important thing here is; using different color when the magnitude changes. This should not be affected by the plus or minus sign. Accordingly, the color scale in the figure should be re-arranged.

#5 In Figure 3, there is an opposite situation to that in Figure 2. Figure 2 uses light tones (yellowish) for the precision results, whereas Figure 3 uses darker tones (gradual darker shades of blue) for the precision results. This is a really confusing and momentarily misleading situation for a reader going through Figs 2 and 3. It makes it difficult to read and interpret. Therefore, the same coloring should be used for colors with varying precision in Figures 2 and 3.

#6 It seems that the 8-hour solution is not included in the charts. Probably because there is no significant difference between the results of 6 and 24 hours, and 8-hour results just neglected! But it would be useful to express this once in the text.

#7 Considering the loss of precision in the figures, different scales are used vertically and horizontally. But this really complicates the interpretation. Therefore, in general, an additional study should be done in order to give the horizontal axis and vertical axis results on the same scale as possible. The horizontal and vertical axis may differ in scale, but the horizontal axis results of the same sub figures (for example, north and east) are expected to be on the same scale. All figures should be reviewed and rearranged throughout the manuscript. Detailed comments about this are also presented in the attached document.

 

#8 In Chapter 3.3, between lines 267 and 270, the statement "More significant differences and related improvements for the North component can be noted in all processing variants. The reasons for this phenomenon are not clear. This issue needs to be further addressed." is included. Did the authors refer to the literature on this subject? If they did not, they should definitely review the literature on this matter and share a few sentences with us.

#9 The regression analysis results given in Table 8 can not be clearly understood! I guess only the "slope" values ​​are given in the table. However, what do the changes in those values mean and how the reader should interpret them should be expressed in the text.

 

#10 Another issue that confuses the reviewer; Why are the curves in top sub figures in Figure 4 represented with a linear function but not with a logarithmic one!? Can you explain this a little bit?

 

#11 In Figure 8, no information in regard to the length of the observation session could be found. The observation session length of  (0.5h, 1h, 2h, 3h, 4h, 6h and 24h) must be included in the graphs.

#12 Is there a special purpose for the analysis results to be given only for AUBG? In this regard, it is useful to satisfy the curiosity of the reader.

#13 Page 16, lines 396 to 401. It is emphasized here and occasionally in some other parts of the text that adding BeiDou to the analysis distorts the results obtained from the longer observation sessions and this is due to bad orbital and clock errors of the Galileo, Glonass and BeiDou systems. Wouldn't it be  same also for short observation sessions? Please comment on this.

#14 Lines 429-439. The authors only interpret their own findings in the manuscript. However, they should also compare their results with those of the studies in the literature (for example, given in the introduction) and interpret them here indicating the advantages and disadvantages of the study.

#15 The conclusion contains structural failures. Here, the main motivation of the study, which is clearly stated in the title, should first be emphasized once again, and then the summary of findings and recommendations should be listed. Other comments regarding the Conclusion are given in detail in the attached document.

Comments for author File: Comments.pdf

Author Response

Reviewer #1 (replies to comments).

We would like to thank you for the detailed analysis of the manuscript and valuable suggestions. We believe that all introduced changes add value to the paper and significantly improve it.

In their Multi-GNSS study, the authors investigated how the inclusion of the BeiDou navigation system in the assessment affected PPP results across Europe, and they even claim to have seen an improvement in results with the addition of the BeiDou system. The title of the article fully expresses what is intended to be done in the study, but some structural improvements are required in the body part. There are also problems in the presentation of the results and their compsrison to the existing studies in the field. In the study, it is emphasized that BeiDou makes a positive contribution in some parts, while in some parts it is stated that the results deteriorate. The reader is confused because the authors are not able to express this clearly. If the above-mentioned issues are reviewed and the manuscript is rearranged, it is considered that the issues discussed in this paper will contribute to the reader in the relevant field. In this regard, some points are detailed below and some points are marked up in the body text and given as an attachment file to this review:

Thank you for highlighting this issue. We agree that the analysis and presentation of the results could be improved. According to your detailed suggestions, we clarified the manuscript's structure. We also improved the clarity of confusing sentences. Finally, we also try to improve language. We believe that taking into account these remarks will significantly contribute to the value of our work.

 

#1 The abstract has not been written properly. It should be rearranged by giving the importance and hypothesis of the paper in the first part, by explaining what has been done in the following parts, and by giving the most striking result in the last sentence.

Thank you for this remark. Suggested improvements were introduced to the text.

 

#2 The introduction is generally well written, but there are minor errors in the use of English and these should be revised. The manuscript contains similar errors all through the text, and these should be revised once more carefully. Some of these errors are marked in the attached document and the authors need to review them and make the necessary improvements.

Thank you for pointing this out. According to the suggestion, the necessary improvements were introduced to the text.

 

#3 Most parts of the text use something called STD Coefficient. What does it mean? If this means standard deviation, it should only be used as standard deviation or standard deviation values. Markings related to this are also made manually in the attached document.

Thank you for highlighting this. Indeed, STD means standard deviation. Due to our mistake, in some cases, it was marked as STD instead of SD. This has been corrected in the text.

 

 

#4 The color scheme used in Figure 2 is a bit confusing. One color is used for the minus values ​​of the same numeric values, and another color is used for the plus values. For example, blue is used for -0.01 m and orange is used for + 0.01. The same color should be used for results with the same numeric value but different sign. The important thing here is; using different color when the magnitude changes. This should not be affected by the plus or minus sign. Accordingly, the color scale in the figure should be re-arranged.

Corrected.

 

#5 In Figure 3, there is an opposite situation to that in Figure 2. Figure 2 uses light tones (yellowish) for the precision results, whereas Figure 3 uses darker tones (gradual darker shades of blue) for the precision results. This is a really confusing and momentarily misleading situation for a reader going through Figs 2 and 3. It makes it difficult to read and interpret. Therefore, the same coloring should be used for colors with varying precision in Figures 2 and 3.

Corrected.

 

#6 It seems that the 8-hour solution is not included in the charts. Probably because there is no significant difference between the results of 6 and 24 hours, and 8-hour results just neglected! But it would be useful to express this once in the text.

Indeed the 8-hour solution is not included into the analysis as well as in the charts. As has rightly been noted, the reason was no significant difference between the results of 6 and 24 hours. An appropriate comment has been added to the text.

 

#7 Considering the loss of precision in the figures, different scales are used vertically and horizontally. But this really complicates the interpretation. Therefore, in general, an additional study should be done in order to give the horizontal axis and vertical axis results on the same scale as possible. The horizontal and vertical axis may differ in scale, but the horizontal axis results of the same sub figures (for example, north and east) are expected to be on the same scale. All figures should be reviewed and rearranged throughout the manuscript. Detailed comments about this are also presented in the attached document.

Thank you for noticing this. We agree that using different scales in the figures complicates the interpretation. Suggested improvements were introduced to the text.

 

 

 #8 In Chapter 3.3, between lines 267 and 270, the statement "More significant differences and related improvements for the North component can be noted in all processing variants. The reasons for this phenomenon are not clear. This issue needs to be further addressed." is included. Did the authors refer to the literature on this subject? If they did not, they should definitely review the literature on this matter and share a few sentences with us.

Thank you for pointing this out. Suggested improvements were introduced to the text.

 

#9 The regression analysis results given in Table 8 can not be clearly understood! I guess only the "slope" values ​​are given in the table. However, what do the changes in those values mean and how the reader should interpret them should be expressed in the text.

Thank you for this remark. Indeed the "slope" values ​​are given in the table. We agree that comments on these values are too narrow. According to suggestion, the description, what the changes in presented values mean, and how the reader should interpret them were added to the text.

 

 #10 Another issue that confuses the reviewer; Why are the curves in top sub figures in Figure 4 represented with a linear function but not with a logarithmic one!? Can you explain this a little bit?

Thank you for highlighting this. We decided to use a linear function as such a way of representation was most often chosen in similar studies. Appropriate explanations also were added to the text.

 

 #11 In Figure 8, no information in regard to the length of the observation session could be found. The observation session length of  (0.5h, 1h, 2h, 3h, 4h, 6h and 24h) must be included in the graphs.

Corrected.

 

#12 Is there a special purpose for the analysis results to be given only for AUBG? In this regard, it is useful to satisfy the curiosity of the reader.

Thank you for this remark. Our intention was to perform a detailed analysis for one randomly selected example station. We simply chose the first one on the station list. An appropriate comment has been added to the text.

 

#13 Page 16, lines 396 to 401. It is emphasized here and occasionally in some other parts of the text that adding BeiDou to the analysis distorts the results obtained from the longer observation sessions and this is due to bad orbital and clock errors of the Galileo, Glonass and BeiDou systems. Wouldn't it be same also for short observation sessions? Please comment on this.

Thank you for pointing this out. It is our mistake. This was corrected in the text.

 

 

#14 Lines 429-439. The authors only interpret their own findings in the manuscript. However, they should also compare their results with those of the studies in the literature (for example, given in the introduction) and interpret them here indicating the advantages and disadvantages of the study.

Thank you for this remark. An appropriate explanation has been added to the text.

 

#15 The conclusion contains structural failures. Here, the main motivation of the study, which is clearly stated in the title, should first be emphasized once again, and then the summary of findings and recommendations should be listed. Other comments regarding the Conclusion are given in detail in the attached document.

Thank you for pointing this out. We agree with this comment. This part of the text was corrected.

 

 

Reviewer 2 Report

This paper investigated the accuracy of PPP in the EUREF Permanent Network (EPN) with an open-source software (GAMP). Especially, the impact of BDS observations on the multi-GNSS PPP. It is a good work.

But a main problem is that I can not BDS-3 was used in the manuscript. As we known, BDS-3 has now provided a global service. I think that BDS-3 need to be analyzed in the manuscript. Because BDS-2 can be seen too few satellites in Europe.

Author Response

Reviewer #2 (replies to comments).

This paper investigated the accuracy of PPP in the EUREF Permanent Network (EPN) with an open-source software (GAMP). Especially, the impact of BDS observations on the multi-GNSS PPP. It is a good work.

But a main problem is that I can not BDS-3 was used in the manuscript. As we known, BDS-3 has now provided a global service. I think that BDS-3 need to be analyzed in the manuscript. Because BDS-2 can be seen too few satellites in Europe.

We would like to thank you for your valuable suggestions that helped us to improve our manuscript. We agree that the visibility of BDS-2 satellites is heavily limited. However, our goal was to analyze the impact of BeiDou observations on the accuracy of Multi-GNSS PPP within Europe based on open-source software. For analysis, we used GAMP, which is well-known in the surveying community, is based on RTKLIB software, is user-friendly, and allows for sophisticated GNSS data analysis.

However, using such type of software involves some risk. In our case, the GAMP software is not ready for processing BeiDou-3 observations. Although the user can select a navigation system in a binary setup mode, official settings do not allow the selection of preferred signals to be processed. Consequently, in our analysis, only BeiDou-2 observations contributed to solutions.

All these issues are described in the manuscript. We also pointed out that using the full potential of BeiDou, which in the GAMP case should be possible after a software upgrade, significantly improves the presented results.

So far, we have made some preliminary analysis based on PRIDE PPP-AR software, where we can choose satellites from which observations will be used in the processing. Based on these limited studies, we expected SD reductions from 20% to 60% when the full BeiDou constellation will be used (in comparison to BeiDou-2 only) depending on the position component, session length, and GNSS system used. Also, some other authors report improvements of solution when BeiDeo-2 only data are replaced by BeiDou_2+3 observations (e.g. Hu et al. 2022). Some short remarks concerning these results were added to the text.

 

 

Hu, J.; Li, P.; Zhang, X.; Bisnath, S.; Pan, L. Precise point positioning with BDS-2 and BDS-3 constellations: ambiguity resolution and positioning comparison, Advances in Space Research. 2022, 70, 7, 1830-1846. https://doi.org/10.1016/j.asr.2022.06.056.

Reviewer 3 Report

In this manuscript, the authors examined the performance of open-source software (like GAMP) in multi-GNSS PPP processing, with a focus on the impact of adding the Beidou constellation in areas with sparse Beidou coverage like Central Europe. Based on data from the EPN network, thorough statistical analyses were carried out covering multi-GNSS scenarios.

Major comments:

One of the main objectives of the manuscript is to examine open-source software performance (like GAMP) in multi-GNSS PPP processing. While the end results all appeared to have small error values, it is not clear on its performance without comparing to some baseline values. Maybe the authors could include some numerical values from others' literature or industry standards.

Please provide more details on the GNSS receivers used to collect the data. For instance, receiver make(s), model(s), antenna type(s), etc. Are the EPN stations all using the same type(s) of receivers? These are important information for others to replicate similar studies and shouldn't be neglected.

As some of the receivers are located at relatively high latitude regions (about 60 degrees), it is necessary to check on the Solar activities during the selected 10 days. Because strong geomagnetic activities can cause carrier phase disturbances in high latitude regions, which may skew the statistical results.

The chosen receivers are relatively clustered in a small region. Is this the requirement of GAMP's PPP technique? Maybe elaborate a bit on GAMP's methodology. Consider including a flow chart on the GAMP processing used in the manuscript.

How does the Central Europe results compare to regions with dense Beidou coverage? The authors may expand on this point in a future work section.

 

Author Response

Reviewer #3 (replies to comments).

 

We would like to thank you for your valuable suggestions that helped us to improve our manuscript.

In this manuscript, the authors examined the performance of open-source software (like GAMP) in multi-GNSS PPP processing, with a focus on the impact of adding the Beidou constellation in areas with sparse Beidou coverage like Central Europe. Based on data from the EPN network, thorough statistical analyses were carried out covering multi-GNSS scenarios.

Major comments:

One of the main objectives of the manuscript is to examine open-source software performance (like GAMP) in multi-GNSS PPP processing. While the end results all appeared to have small error values, it is not clear on its performance without comparing to some baseline values. Maybe the authors could include some numerical values from others' literature or industry standards.

Thank you for pointing this out. Suggested information was introduced to the text.

 

Please provide more details on the GNSS receivers used to collect the data. For instance, receiver make(s), model(s), antenna type(s), etc. Are the EPN stations all using the same type(s) of receivers? These are important information for others to replicate similar studies and shouldn't be neglected.

Thank you for this remark. Suggested information was introduced to the text.

 

As some of the receivers are located at relatively high latitude regions (about 60 degrees), it is necessary to check on the Solar activities during the selected 10 days. Because strong geomagnetic activities can cause carrier phase disturbances in high latitude regions, which may skew the statistical results.

Thank you for highlighting this issue. According to the suggestion, the Solar activities during the selected ten days were checked and introduced to the text.

 

The chosen receivers are relatively clustered in a small region. Is this the requirement of GAMP's PPP technique? Maybe elaborate a bit on GAMP's methodology. Consider including a flow chart on the GAMP processing used in the manuscript.

Thank you for pointing this out. Indeed, the chosen receivers are relatively clustered in a small region. It is not the requirement of GAMP's PPP technique but a premise for our analysis: investigate the impact of BeiDou observations on the accuracy of Multi-GNSS PPP over the European area, where the number of visible BeiDou SVs is significantly smaller than in Asian-Pacific regions.

According to the suggestion, some additional information on GAMP's methodology was added to the text.

How does the Central Europe results compare to regions with dense Beidou coverage? The authors may expand on this point in a future work section.

Thank you for pointing this out. We agree that comparing results achieved within Central Europe to results obtained in regions with dense Beidou coverage may yield interesting conclusions. We also agree that it is worth analyzing in future work.