Evaluation and Analysis of the Accuracy of Open-Source Software and Online Services for PPP Processing in Static Mode

Round 1

Reviewer 1 Report (New Reviewer)

Comments to the paper “Evaluation and analysis of the accuracy of open-source software and online services for PPP processing in static mode”, Remote Sensing manuscript ID 2297402.

 

There are already a number of papers addressing online services for PPP processing as well as papers who have used open-source software as processing tools. However, the topic of the current paper is interesting as it addresses quite a large number of both online services and open-source software. I think the paper has the potential too meet the requirements for a reviewed paper. Below are a few comments:

 

First and foremost, the authors should address the structure of the text and the use of the English language. Some parts of the paper (especially in sections 5, 6 and 7) are a bit repetitive, and the authors should give parts of the text a more stringent and compact form. Also, some sentences are unclear, and some sentences are grammatically incomplete (e.g. lines 14-15, 50-51, 57-60, 75-77, 90-91, 311-314, ….)

When abbreviations are used for the first time, they should be spelled out. e.g. CSRS-PPP is used several times before it is spelled out in line 210.

 

Concerning the ground truth coordinates, why does the authors use coordinates obtained from Scripps OPAC? SOPAC is one of twelve processing centers who contribute to the official IGS products. Daily solutions from individual processing centers deviate at the several-mm to cm level from the official coordinates supplied by IGS.  Also, the authors should quantify the quality of the ground truth coordinates.

 

Concerning section 2, the authors should make an attempt to further harmonize the description and information of capabilities of the different software and processing tools, e.g. web-page for downloading, static/kinematic processing, observables with respect to systems, frequencies and signals, batch-processing/GUI, accordance with conventions and standard formats, ambiguity resolution, estimator, documentation and user friendliness, required datafiles, output,.. I realize that this is difficult but encourage the authors to further systematize the information. For some software and processing tools, parts of the information will not be available and can then be let out. Some of the abovementioned information is available via tables 1 and table 2, but the actual text should also be standardized/harmonized. Important information to include is whether software/tools are being actively maintained with respect to conventions and formats. Information regarding availability/robustness of the online-services should be included (i.e. will the user always get results upon submitting a data-file – e.g. the Trimble RTX-service processes only data from a limited number of GNSS-antennas, the MagicGNSS-service have been off-line/not available for longer periods of time).

The date of the current software version should be included.

 

Concerning section 3

In line 251-252: “….. observables between receiver ? and satellite ? can be described as [15]”. Should probably be: “….. observables between receiver j and satellite ? can be described as [15]”, as subscript i is used to denote frequency.

 

Please also state that effects due to e.g. relativity, phase wind-up and tidal effects are omitted in the described model. The handling of the tropospheric delay should be further elaborated.

 

Line 271: what is “other residual errors”? Please include also notation for multipath and different hardware delays, when using the ionospheric-free linear combination.

 

The content in table 3 is not complete. What is the source for the different files, e.g. file with ocean loading parameters and DCB-file? Do all ERP-, clk- and sp3-files stem from the “IGS final solution”. Abbreviations should be explained/spelled out.

 

Concerning the use of RMSE as quality indicators, the authors could also include “mean offset” and “standard deviation” (precision number computed based on differences from mean offset”). Standard deviations are included in e.g. table 4.

 

Concerning the large discrepancies for RTKLIB; these discrepancies are much larger than expected and should be further investigated. Could the reason for the large discrepancies be due to operator error? incompatible data formats, ..? Numerous authors have used the RTKLIB software with results at the cm-level.

 

Concerning large discrepancies for some stations (line 337-339), the authors should verify their explanation by comparing with daily coordinates from IGS.

 

The fonts in figures 2 and figure 5 are too small.

 

In line 370-372: “The E, N and U components of the 45 IGS stations of each software were averaged to establish a value that represents the performance of the software.” Do you with “E, N and U components” mean “standard deviation of E-, N- and U-components”? Instead of taking the average of individual standard deviations, why do you then not compute overall standard deviations base on sum of all squared deviations and overall number of deviations?

Concerning information from each software, the most correct is probably to compute overall standard deviations based on averaging individual variances (and not average of individual standard deviations).

 

In order to increase the resolution in Figure 4, the erroneous output from RTKLIB should be omitted. Also, it would be interesting to show ratio-numbers between standard deviations based on deviation from ground truth and standard deviations reported by the software.

 

In table 6; please describe the meaning of and formulae to compute “Total Uncertainty”.

 

The results of the analysis should be compared to similar work carried out by other authors, also concerning the distribution of errors in Nort, East and Heigth (the distribution in North and East could also depend on latitude of receivers).

Author Response

Response to Reviewer #1 Comments (1^st round)

Comments and Suggestions for Authors

Reviewer #1- General Comment: Comments to the paper “Evaluation and analysis of the accuracy of open-source software and online services for PPP processing in static mode”, Remote Sensing manuscript ID 2297402. There are already a number of papers addressing online services for PPP processing as well as papers who have used open-source software as processing tools. However, the topic of the current paper is interesting as it addresses quite a large number of both online services and open-source software. I think the paper has the potential too meet the requirements for a reviewed paper. Below are a few comments:

Authors’ Response to General Comment: The Authors would like to thank Reviewer #1 for this positive general comment about our manuscript. Reviewer #1 completely followed the main objective of the paper. Thanks.

Reviewer #1- Comment 1: First and foremost, the authors should address the structure of the text and the use of the English language. Some parts of the paper (especially in Sections 5, 6 and 7) are a bit repetitive, and the authors should give parts of the text a more stringent and compact form. Also, some sentences are unclear, and some sentences are grammatically incomplete (e.g. lines 14-15, 50-51, 57-60, 75-77, 90-91, 311-314, ….)

Authors’ Response to Comment 1: The Authors have addressed in detail the revisions suggested by Reviewer #1, regarding the structure of the text of the manuscript. Within this context, the Authors significantly have improved English language, especially in those sections recommended by Reviewer #1. Additionally, the incomplete sentences indicated by Reviewer #1 are now clear and complete. Thank you very much for these significant recommendations.

Reviewer #1- Comment 2: When abbreviations are used for the first time, they should be spelled out. e.g. CSRS-PPP is used several times before it is spelled out in line 210.

Authors’ Response to Comment 2: The Authors thank Reviewer #1 for her/his valuable comment. To address this recommendation, the Authors described the abbreviations when they are first mentioned. These changes are highlighted in green color in the body of the revised manuscript.

Reviewer #1- Comment 3: Concerning the ground truth coordinates, why does the authors use coordinates obtained from Scripps OPAC? SOPAC is one of twelve processing centers who contribute to the official IGS products. Daily solutions from individual processing centers deviate at the several-mm to cm level from the official coordinates supplied by IGS. Also, the authors should quantify the quality of the ground truth coordinates.

Authors’ Response to Comment 3: The Authors thank Reviewer #1 for this outstanding comment. The Authors did select the coordinates published by SOPAC because they are used as a reference in other studies, as is the case of the work published by Guo (2015) in the GPS Solutions Journal. Also, the Authors compared the coordinates published by SOPAC with the daily solution of the IGS, finding millimeter differences, in none of the cases, the difference did exceed 3 mm. On the other hand, the precision with which the reference coordinates were obtained in SOPAC never exceeded 3 mm. Next, the reference corresponding to Guo (2015) is provided:

Guo, Q. (2015). Precision comparison and analysis of four online free PPP services in static positioning and tropospheric delay estimation. GPS solutions, 19(4), 537-544. https://doi.org/10.1007/s10291-014-0413-5

Reviewer #1- Comment 4: Concerning Section 2, the authors should make an attempt to further harmonize the description and information of capabilities of the different software and processing tools, e.g. web-page for downloading, static/kinematic processing, observables with respect to systems, frequencies and signals, batch-processing/GUI, accordance with conventions and standard formats, ambiguity resolution, estimator, documentation and user friendliness, required datafiles, output,. I realize that this is difficult but encourage the authors to further systematize the information. For some software and processing tools, parts of the information will not be available and can then be let out. Some of the abovementioned information is available via tables 1 and table 2, but the actual text should also be standardized/harmonized. Important information to include is whether software/tools are being actively maintained with respect to conventions and formats. Information regarding availability/robustness of the online-services should be included (i.e. will the user always get results upon submitting a data-file – e.g. the Trimble RTX-service processes only data from a limited number of GNSS-antennas, the MagicGNSS-service have been off-line/not available for longer periods of time).

Authors’ Response to Comment 4: The Authors thank Reviewer #1 for this interesting recommendation. Relevant information on the evaluated software and online services was added. The Authors decided not to add all the information requested by Reviewer #1 because Section 2 would be too large, so readers are encouraged to visit the manuals or websites of each of the software and online services for more detailed technical information about them. In addition, information about the TRIMBLE online service was incorporated in a paragraph to specify compatibility of antennas in processing. Finally, the MagicGNSS online service has been available, at least, during the last 2 years. The user manual of every software details the processing strategies, the compatibility with RINEX formats and some precise products, the compatibility of GNSS constellations and frequencies, compatibility with ambiguity resolution, input files, output files, operating system where it runs, models for mitigation of the troposphere, etc. Therefore, the Authors strongly recommend readers to consult the manuals for more information, since the objective of our study is to evaluate the software and online services as published by their respective Authors.

Reviewer #1- Comment 5: The date of the current software version should be included.

Authors’ Response to Comment 5: The Authors thank Reviewer #1 for this valuable recommendation. To address it, the year corresponding to the version of each software and some online services evaluated was added in the revised version of the paper.

 

Concerning Section 3

Reviewer #1- Comment 6: In line 251-252: “….. observables between receiver ? and satellite ? can be described as [15]”. Should probably be: “….. observables between receiver j and satellite ? can be described as [15]”, as subscript I is used to denote frequency.

Authors’ Response to Comment 6: The Authors thank Reviewer #1 for this interesting comment. Due to the suggestion of another Reviewer, the Authors decided to delete Section 3 as the equations presented in this section are well-known in the literature. On the other hand, citations were added where readers can find more detailed information about the PPP mathematical model. Thanks a lot for the suggestion.

Reviewer #1- Comment 7: Please also state that effects due to e.g. relativity, phase wind-up and tidal effects are omitted in the described model. The handling of the tropospheric delay should be further elaborated.

Authors’ Response to Comment 7: Thanks for this recommendation. Due to the suggestion of another Reviewer, the Authors decided to delete Section 3 as the equations presented in this section are well-known in the literature. On the other hand, citations were added where readers can find more detailed information about the PPP mathematical model.

Reviewer #1- Comment 8: Line 271: what is “other residual errors”? Please include also notation for multipath and different hardware delays, when using the ionospheric-free linear combination.

Authors’ Response to Comment 8: This is an excellent observation of Reviewer #1, thanks. Due to the suggestion of another Reviewer, the Authors decided to delete Section 3 as the equations presented in this section are well-known in the literature. On the other hand, citations were added where readers can find more detailed information about the PPP mathematical model.

Reviewer #1- Comment 9: The content in table 3 is not complete. What is the source for the different files, e.g. file with ocean loading parameters and DCB-file? Do all ERP-, clk- and sp3-files stem from the “IGS final solution”. Abbreviations should be explained/spelled out.

Authors’ Response to Comment 9: The Authors appreciate the comment from Reviewer #1. To address this revision, the Authors incorporated more understandable information to Table 3. The main precise products such as satellite orbits, correction to clocks, earth rotation parameters correspond to the final solution of the IGS. The ten open-source software evaluated have in common the input of the precise products: orbits and clock, PCO/PCV. It was sought to standardize the precise input products for the 10-software evaluated with the objective of not having discrepancies due to the precision of the precise products generated by the different IGS analysis centers. The final precise products of the main IGS were used in all the software evaluated.

Reviewer #1- Comment 10: Concerning the use of RMSE as quality indicators, the authors could also include “mean offset” and “standard deviation” (precision number computed based on differences from mean offset”). Standard deviations are included in e.g. table 4.

Authors’ Response to Comment 10: Thank you for this recommendation, to address it, the Authors have included the values of the “mean offset” in Table 4 and Table 5.

Reviewer #1- Comment 11: Concerning the large discrepancies for RTKLIB; these discrepancies are much larger than expected and should be further investigated. Could the reason for the large discrepancies be due to operator error? incompatible data formats, ..? Numerous authors have used the RTKLIB software with results at the cm-level.

Authors’ Response to Comment 11: The Authors thank Reviewer #1 for this important comment. The Authors reprocessed again the RINEX files in 2.11 format belonging to the 45 IGS stations, this time using the demo5 b34b version of RTKLIB. The new results improved significantly, and ENU coordinates in the order of centimeters were found. The manuscript was updated with the new coordinates from the open-source software RTKLIB.

Reviewer #1- Comment 12: Concerning large discrepancies for some stations (line 337-339), the authors should verify their explanation by comparing with daily coordinates from IGS.

Authors’ Response to Comment 12: This is an excellent recommendation, thank you. To address it, the Authors reviewed the daily solutions published by the IGS of the BOGT, HRAO, SALU and SCTB stations. Maximum differences of 5 mm were found between the 7 days. Therefore, the discrepancies of these stations are mainly due to the quality of the observations, since the final solutions of the IGS show significant discrepancies from one day to the next. Also, the performance of software and online services have an impact on these discrepancies, since the software that has the ambiguity resolution option did not show large discrepancies. Also, Figures 2 and 5 were regenerated, and only 20 of 45 IGS stations were illustrated. The 20 IGS stations with the least discrepancy were selected, however, for the statistical analysis and calculation of the RMSE the majority of the IGS stations were used. The new Figures 2 and 5 are more legible, the Y component was standardized to the same scale, except for the RTKLIB software due to its large values of differences with respect to the reference coordinates.

Reviewer #1- Comment 13: The fonts in Figures 2 and Figure 5 are too small.

Authors’ Response to Comment 13: Thanks for the recommendation about the above-mentioned Figures of the manuscript. New figures with higher quality were generated and the fonts were designed with a larger size. The new Figures 2 and 5 can be seen in the updated version of the manuscript.

Reviewer #1- Comment 14: In line 370-372: “The E, N and U components of the 45 IGS stations of each software were averaged to establish a value that represents the performance of the software.” Do you with “E, N and U components” mean “standard deviation of E-, N- and U-components”? Instead of taking the average of individual standard deviations, why do you then not compute overall standard deviations base on sum of all squared deviations and overall number of deviations?

Authors’ Response to Comment 14: This is a very interesting recommendation, thanks for mentioning it. The E, N and U components are those obtained by each software in the 45 IGS stations, and the standard deviation represents the precision with which each component was obtained by the different software. Unfortunately, only RTKLIB, PPPLib and GIPSY X showed the precisions with which the ENU coordinates were obtained. The objective of Figure 4 is to illustrate the relationship between the differences between the reference coordinates and those obtained with the software, and the precision with which the coordinates were obtained. Therefore, the solutions of the IGS stations were obtained with high precision, but the accuracy was low. This is perfectly illustrated in Figure 4 of the revised version of the manuscript.

Reviewer #1- Comment 15: Concerning information from each software, the most correct is probably to compute overall standard deviations based on averaging individual variances (and not average of individual standard deviations).

Authors’ Response to Comment 15: Thank you for this observation. The Authors understand the concern of Reviewer #1about the use of the overall standard deviation instead of the use of individual standard deviation. The Authors will clarify this issue as follows. In general, the overall standard deviation is value of the global variation of the measurements. This is a quite interesting parameter if one is interested in the general uncertainty of the values. Within this context, if measurements are collected properly, the overall standard deviation may capture every source of variation. However, in the study presented in this paper, the Authors are exploring the local performance of the measurements considering different software. This is mainly the reason why the individual standard deviation is used. Thus, to keep the main objective of the research, the Authors have decided to maintain the statistical analysis of the date considering the individual standard deviation. The Authors hope Reviewer #1 understand this decision. Thanks.

Reviewer #1- Comment 16: In order to increase the resolution in Figure 4, the erroneous output from RTKLIB should be omitted. Also, it would be interesting to show ratio-numbers between standard deviations based on deviation from ground truth and standard deviations reported by the software.

Authors’ Response to Comment 16: The Authors thank Reviewer #1 for this important recommendation. As mentioned in the response to comment 11, the results of the reprocessing of RTKLIB improved significantly, therefore, a new Figure 4 was generated. Table 4 shows the standard deviation corresponding to each ENU component of the differences between the real coordinates (reference) of the IGS stations and those estimated with the software. The standard deviation of the actual coordinate deviations is higher than the standard deviations reported by the software. Also, at the end of Section 4.1 of the revised manuscript, information is added describing the differences between the standard deviations reported by the software and the standard deviations of the ENU component deviations.

Reviewer #1- Comment 17: In table 6; please describe the meaning of and formulae to compute “Total Uncertainty”.

Authors’ Response to Comment 17: Thank you for this recommendation. Information was added in the revised version of the manuscript describing the total error and the mathematical equation that calculates it.

Reviewer #1- Comment 18: The results of the analysis should be compared to similar work carried out by other authors, also concerning the distribution of errors in Nort, East and Heigth (the distribution in North and East could also depend on latitude of receivers).

Authors’ Response to Comment 18: Thank you for this comment. To address it, the Authors included a paragraph in the discussion section where the results achieved in our study are discussed with those published in other works. Also, it was discussed about the effect of the geographical location of the stations and their precision, which, for our results, did not find a possible correlation between location and precision that deserves to be mentioned.

Author Response File: Author Response.docx

Reviewer 2 Report (New Reviewer)

Comments to the paper "Evaluation and analysis of the accuracy of open-source software and online services for PPP processing in static mode"

 

The paper focuses on the evaluation of the accuracy of open-source software and online services for PPP processing in static mode. Although the paper's topic seems interesting, the manuscript contains several weaknesses. First of all, the review of the literature was done in a brief way. Many papers dealing with the analyzed issue have been omitted. In my opinion, result presentations should also be improved.

 

Detailed comments:

 Lines 14-15

“only a few studies have evaluated the performance of different open-source software and have focused extensively on online free PPP services". That is not true: the literature review should be more detailed.

 Lines 103-104-212

“There is a knowledge gap about detailed research on open-source software and online services for PPP processing”. This is not entirely true. The Authors have omitted many of the accessible studies. Below are only a few examples.

 

 ·       Malik J.S. Performance analysis of static precise point positioning using open-source GAMP. Artificial Satellites. 55(2), 41–60 (2020). https://doi.org/10.2478/arsa-2020-0004.

·       Yigit C.O., Gikas V., AlcayS., Ceylan A. Performance evaluation of short to long term GPS, GLONASS and GPS/GLONASS postprocessed PPP. Surv. Rev. 46, 155–166 (2014). https://doi.org/10.1179/1752270613Y.0000000068.

·       Alkan R.M., İlçi V., Ozulu İ.M., Saka M.H. A comparative study for accuracy assessment of PPP technique using GPS and GLONASS in urban areas. Measurement. 69 (2015) 1-8. ISSN 0263-2241. https://doi.org/10.1016/j.measurement.2015.03.012.

·       Malik J.S. Performance analysis of static precise point positioning using open-source GAMP. Artificial Satellites. 55(2), 41–60 (2020). https://doi.org/10.2478/arsa-2020-0004.

·       Mohammed, I. H., Ataiwe, T. N., & Al Sharaa, H. (2021). Accuracy Assessment of a Variety of GPS Data Processing, Online Services and Software. Geomatics and Environmental Engineering, 15(4), 5–19. https://doi.org/10.7494/geom.2021.15.4.5

·       Dawidowicz, K.; Bakuła, M. Impact of BeiDou Observations on the Accuracy of Multi-GNSS PPP in a Function of Observing Session Duration within Europe—Analysis Based on Open-Source Software GAMP. Remote Sens. 2023, 15, 158. https://doi.org/10.3390/rs15010158

  In my opinion, this part of the text should be improved. The Authors should describe the review of literature in more detail and highlight against this background the innovation of their own studies.

 

 Line 118 “Summary of open-source PPP software and online free PPP services”.

In my opinion, this chapter is too long. The authors can only leave both tables and in the description, I propose to focus on the fundamental differences between the services and software. The information that is included in the tables does not need to be repeated in the text.

 Line 247 (Table 2)

In the second line of Table 2, in many columns, the authors include citations of references. In my opinion, the citations can remain, but the name of the software developer should be added beforehand.

 Line 248 “GNSS PPP mathematical model”

I don’t know if this chapter is necessary. As the basic phase and code measurement equations for the PPP solution of a dual-frequency GNSS receiver can be considered the same for all presented services and software, I am not sure if they all use the presented ionospheric free combinations to mitigate the first-order ionospheric delay. Whether the authors have checked this?

All equations presented in this chapter are well-known in the literature, and there is no sense in repeating them. It will be sufficient to mention that “GNSS PPP mathematical model can be found in…”

 Line 289

“it was decided to standardize to only GPS (L1, L2)”

In my opinion, it is a serious drawback, as multi-GNSS processing has become a standard approach. Also, it will be interesting to analyze shorter (sub-daily) measurement sessions or the kinematic approach. For non-scientific users, such measurements are of great interest. Surveyors are mostly not interested in daily sessions as they are time-consuming and non-economic.

 Line 321 (Figure 1)

Authors present on figure GPS sites as well as Tectonic plates. Please explain the purpose of presenting Tectonic plates, as in the text authors do not mention it at all.

 Line 335 (Figure 2)

The figure is unreadable. Additionally, the Authors applied different ranges on the x-axis, making it challenging to analyze the results. Please improve this. Would only a few most prominent/interesting stations be enough to present results?

 Line 366 (Figure 3)

RMSE above bars are poorly readable. Are they really needed if they can be read off the x-axis?

Additionally, RTKLib results clearly differ from the others. In my opinion, the Authors should try to find a possible reason for this.

 Line 378 (Figure 4)

As mentioned above: RTKLib results clearly differ from the others. The Authors should try to find a possible reason for this.

 Line 397 (Figure 5)

As Fig. 2

 Line 417 (Figure 6)

In my opinion, the Authors should explain in the text why services and software are analyzed/presented separately.

 Lines 448-450

“The precisions presented in previous sections of the paper have demonstrated the potential benefit of using PPP in static mode in comparison with relative positioning”

I can’t find where in the text Author compare PPP results with “relative positioning”…

Please elaborate.

 Lines 454-458

“The Authors of this paper believe that the fundamental factor in the accuracy of the PPP technique is the processing software”

Accuracy may depend on algorithms rather than software… This sentence should be rewritten for clarity.

 “In this context, few studies have been done on the performance of software that processes static PPP, consequently, the scientific community does not have a benchmark study to identify the best-performing software for their experiments. This fact justifies the study presented in this paper”

As previously stated: this is not true… there are quite a lot of studies dealing with the analyzed subject.

 Lines 459-460

“In addition, every PPP open-source software contains different processing strategies, algorithms, and filtering techniques.”

These differences should also be studied in the paper, as mentioned previously (Line 118).

 Lines 500-504

Unclear part of the text: “the difference between PRIDE-PPPAR and GIPSY X…”  “This analysis reveals the efficiency of online PPP services to obtain high-precision positioning similar to that obtained by a scientific software such as GIPSYX”. But PRIDE-PPPAR and GIPSY X are both scientific software, not online services. Please rewrite this part of the text.

 Lines 515-516 and 538-540

This is almost the same conclusion. Please merge it into one sentence.

 

Conclusions:

The manuscript could be acceptable to publish but after major corrections in the text.

 

Author Response

Response to Reviewer #2 Comments (1^st round)

Comments and Suggestions for Authors

Reviewer #2- General Comment: Comments to the paper "Evaluation and analysis of the accuracy of open-source software and online services for PPP processing in static mode". The paper focuses on the evaluation of the accuracy of open-source software and online services for PPP processing in static mode. Although the paper's topic seems interesting, the manuscript contains several weaknesses. First of all, the review of the literature was done in a brief way. Many papers dealing with the analyzed issue have been omitted. In my opinion, result presentations should also be improved.

Authors’ Response to General Comment: The Authors thank Reviewer #2 for this important general comment. In general, to address this general comment, the Authors included more references to the revised version of the manuscript. Also, the Authors addressed one by one every recommendation coming from Reviewer #2. Responses to Reviewer #2's comments are shown below.

Reviewer #2- Comment 1: Lines 14-15 “only a few studies have evaluated the performance of different open-source software and have focused extensively on online free PPP services". That is not true: the literature review should be more detailed.

Authors’ Response to Comment 1: Thanks for the comment. The Authors understand the concern of Reviewer #2 about the fact that very few papers are included in the discussion about the literature review of the topic of this research. The Author will clarify this concern in the following lines. In lines 14-15, the Authors refer to the fact that very few studies have been carried out about the evaluation of many open-source software under the same conditions, in general, in the literature are technical publications about the evaluation of only one or two software or services online. In the present paper, the Authors integrated an evaluation of several of them, this is mainly why the Authors declared that very few studies are reported in the literature about it. Anyhow, to address this recommendation, the Authors incorporated more information (lines 14-15) to better explain the state of the art of the research problem explored in this paper.

Reviewer #2- Comment 2: Lines 103-104-212 “There is a knowledge gap about detailed research on open-source software and online services for PPP processing”. This is not entirely true. The Authors have omitted many of the accessible studies. Below are only a few examples.

• Malik J.S. Performance analysis of static precise point positioning using open-source GAMP. Artificial Satellites. 55(2), 41–60 (2020). https://doi.org/10.2478/arsa-2020-0004.
• Yigit C.O., Gikas V., AlcayS., Ceylan A. Performance evaluation of short to long term GPS, GLONASS and GPS/GLONASS postprocessed PPP. Surv. Rev. 46, 155–166 (2014). https://doi.org/10.1179/1752270613Y.0000000068.
• Alkan R.M., İlçi V., Ozulu İ.M., Saka M.H. A comparative study for accuracy assessment of PPP technique using GPS and GLONASS in urban areas. Measurement. 69 (2015) 1-8. ISSN 0263-2241. https://doi.org/10.1016/j.measurement.2015.03.012.
• Malik J.S. Performance analysis of static precise point positioning using open-source GAMP. Artificial Satellites. 55(2), 41–60 (2020). https://doi.org/10.2478/arsa-2020-0004.
• Mohammed, I. H., Ataiwe, T. N., & Al Sharaa, H. (2021). Accuracy Assessment of a Variety of GPS Data Processing, Online Services and Software. Geomatics and Environmental Engineering, 15(4), 5–19. https://doi.org/10.7494/geom.2021.15.4.5
• Dawidowicz, K.; Bakuła, M. Impact of BeiDou Observations on the Accuracy of Multi-GNSS PPP in a Function of Observing Session Duration within Europe—Analysis Based on Open-Source Software GAMP. Remote Sens. 2023, 15, 158. https://doi.org/10.3390/rs15010158

In my opinion, this part of the text should be improved. The Authors should describe the review of literature in more detail and highlight against this background the innovation of their own studies.

Authors’ Response to Comment 2: This is a very outstanding recommendation. The above-mentioned papers are within the scope of the main objective of the present research, thank you for recommending them. Thus, within this frame of reference, The Authors incorporated and described such references in the revised version of the manuscript.

Reviewer #2- Comment 3: Line 118 “Summary of open-source PPP software and online free PPP services”. In my opinion, this chapter is too long. The authors can only leave both tables and in the description, I propose to focus on the fundamental differences between the services and software. The information that is included in the tables does not need to be repeated in the text.

Authors’ Response to Comment 3: The Authors thank Reviewer #2 for this comment. However, since the Authors considered the present structure of the paper based on other published articles, where information is summarized about each open-source software or online service in detail, and then, such summarized information is incorporated in Tables, the Authors believe that it is important to maintain the software description design in its current form in the revised manuscript. In this way, readers will have an overview of each software and online service, which can be used to understand other important features of these software and online services. To justify this decision, it is important to mention that the papers reporting this structure of description of software and online services are mentioned below:

• Guo, Q. (2015). Precision comparison and analysis of four online free PPP services in static positioning and tropospheric delay estimation. GPS solutions, 19(4), 537-544. https://doi.org/10.1007/s10291-014-0413-5
• El-Mowafy, A. (2011). Analysis of web-based GNSS post-processing services for static and kinematic positioning using short data spans. Survey review, 43(323), 535-549. https://doi.org/10.1179/003962611X13117748892074
• El Shouny, A., & Miky, Y. (2019). Accuracy assessment of relative and precise point positioning online GPS processing services. Journal of Applied Geodesy, 13(3), 215-227. https://doi.org/10.1515/jag-2018-0046
• Farah, A. (2016). Accuracy evaluation for online Precise Point Positioning services. Journal of Geomatics, 10(1), 12-18.

Reviewer #2- Comment 4: Line 247 (Table 2). In the second line of Table 2, in many columns, the authors include citations of references. In my opinion, the citations can remain, but the name of the software developer should be added beforehand.

Authors’ Response to Comment 4: This is a good observation. Thank you. To address this concern, the Authors included the name of the software developer.

Reviewer #2- Comment 5: Line 248 “GNSS PPP mathematical model”. I don’t know if this chapter is necessary. As the basic phase and code measurement equations for the PPP solution of a dual-frequency GNSS receiver can be considered the same for all presented services and software, I am not sure if they all use the presented ionospheric free combinations to mitigate the first-order ionospheric delay. Whether the authors have checked this? All equations presented in this chapter are well-known in the literature, and there is no sense in repeating them. It will be sufficient to mention that “GNSS PPP mathematical model can be found in…”

Authors’ Response to Comment 5: Thank you for this outstanding recommendation. To overcome this issue, the Authors decided to delete the Chapter titled “GNSS PPP mathematical model” because the equations presented are well-known within the GNSS community, therefore, the Authors only cited some works where the mathematical model for PPP is explained in detail.

Reviewer #2- Comment 6: Line 289 “it was decided to standardize to only GPS (L1, L2)”

In my opinion, it is a serious drawback, as multi-GNSS processing has become a standard approach. Also, it will be interesting to analyze shorter (sub-daily) measurement sessions or the kinematic approach. For non-scientific users, such measurements are of great interest. Surveyors are mostly not interested in daily sessions as they are time-consuming and non-economic.

Authors’ Response to Comment 6: The Authors would like to thank Reviewer #2 for this comment. In fact, the Authors agree with Reviewer #2 since multi-GNSS processing has become a standard high-precision approach today. However, not all the evaluated software and online services are available for multi-GNSS and multi-frequency processing. Therefore, the GNSS constellation and common frequencies were searched for all software and online services. On the other hand, short measurement sessions were not considered in this study because long measurement periods were established to obtain the best performance of each software and online service. Since in short period measurements sometimes the best accuracies are not reached due to the long convergence time when the entire ambiguities are not resolved, as is the case for most of the evaluated software and services. The authors plan to evaluate the kinematic PPP mode with a focus on Structural Health Monitoring in bridges in a future publication.

Reviewer #2- Comment 7: Line 321 (Figure 1), Authors present on figure GPS sites as well as Tectonic plates. Please explain the purpose of presenting Tectonic plates, as in the text authors do not mention it at all.

Authors’ Response to Comment 7: Thank you for this observation. Reviewer #2 is correct. To overcome this concern, the Authors generated Figure 1 again without tectonic plates boundaries as they are not studied in this paper. The updated Figure 1 can be seen in the revised version of the paper.

Reviewer #2- Comment 8: Line 335 (Figure 2). The figure is unreadable. Additionally, the Authors applied different ranges on the x-axis, making it challenging to analyze the results. Please improve this. Would only a few most prominent/interesting stations be enough to present results?

Authors’ Response to Comment 8: This is a good comment, thank you. Figures 2 and 5 were generated again considering 20 IGS stations with the least errors. Also, the "Y" component was adjusted to a single range for a better analysis. In Figure 2 only RTKLIB has a different range in the Y component due to the large errors it reached. In general, the new Figures 2 and 5 are of higher quality. Please refer to the revised version of the manuscript where they are located.

Reviewer #2- Comment 9: Line 366 (Figure 3). RMSE above bars are poorly readable. Are they really needed if they can be read off the x-axis? Additionally, RTKLib results clearly differ from the others. In my opinion, the Authors should try to find a possible reason for this.

Authors’ Response to Comment 9: Thank you for this observation. The Authors increased the size of the font representing the RMSEs in each bar for better visualization. Also, the observations of the 45 IGS stations were reprocessed with a new version of the RTKLIB software and results with a lower degree of error were obtained. The new Figure 3 illustrates the RTKLIB software enhancement.

Reviewer #2- Comment 10: Line 378 (Figure 4). As mentioned above: RTKLib results clearly differ from the others. The Authors should try to find a possible reason for this.

Authors’ Response to Comment 10: The Authors would like to thank Reviewer #2 for this important comment. As mentioned in the response to comment 9, the reprocessed results in the new version of RTKLIB were better and they are illustrated in the new version of Figure 4.

Reviewer #2- Comment 11: Line 397 (Figure 5). As Fig. 2

Authors’ Response to Comment 11: Thank you for this comment. Figure 5 was regenerated, as explained in the response to comment 8.

Reviewer #2- Comment 12: Line 417 (Figure 6). In my opinion, the Authors should explain in the text why services and software are analyzed/presented separately.

Authors’ Response to Comment 12: Thanks for the observation. The Authors chose to present/discuss software and online services separately because it is a way of making it clear to the reader the differences between open-source software and online services in the implementation of PPP. Also, the Authors consider that is correct to present the software separately from the online services, since when certain constants are described in the software, they can help to the better understanding of the text. For example, the operating system where the software runs, most of the times, does not apply to online services. On the other hand, the Authors in the discussion section jointly analyze the total RMSE of online software and services. Finally, it is common to find this type of structure in other publications, for example, when they analyze online services that process PPP and relative positioning, it is common to describe them separately, within this frame of reference, first, they describe the PPP services and then the relative services.

Reviewer #2- Comment 13: Lines 448-450. “The precisions presented in previous sections of the paper have demonstrated the potential benefit of using PPP in static mode in comparison with relative positioning”. I can’t find where in the text Author compare PPP results with “relative positioning”… Please elaborate.

Authors’ Response to Comment 13: Thanks for this observation. The coordinates that were considered as reference were published by SOPAC and were obtained through relative processing with the GAMIT/GLOBK software, therefore, the estimated coordinates with the PPP software and online services in static mode were compared with those obtained by the method of relative processing. This clarification was added in the methodology section to clarify this concern. Please refer to the revised version of the manuscript.

Reviewer #2- Comment 14: Lines 454-458. “The Authors of this paper believe that the fundamental factor in the accuracy of the PPP technique is the processing software.” Accuracy may depend on algorithms rather than software… This sentence should be rewritten for clarity. “In this context, few studies have been done on the performance of software that processes static PPP, consequently, the scientific community does not have a benchmark study to identify the best-performing software for their experiments. This fact justifies the study presented in this paper.” As previously stated: this is not true… there are quite a lot of studies dealing with the analyzed subject.

Authors’ Response to Comment 14: The Authors would like to thank Reviewer #2 for this important comment. In the Discussion section, the wording was modified to specify that the processing algorithm of each software is one of the most important factors in the accuracy of the PPP technique in static mode. On the other hand, more relevant information was added about the few studies that exist where they evaluate a large number of software or online services under the same processing conditions. Published papers where several scholars evaluate software performance, commonly analyze only one software and no more than 10. In our study we evaluated 10 open-source software under the same processing conditions. This could be the main contribution of our manuscript.

Reviewer #2- Comment 15: Lines 459-460. “In addition, every PPP open-source software contains different processing strategies, algorithms, and filtering techniques.” These differences should also be studied in the paper, as mentioned previously (Line 118).

Authors’ Response to Comment 15: The Authors understand the concern of Reviewer #2 about the above-mentioned issue. The Authors will try to overcome this problem as follows. It is difficult to incorporate in detail each internal feature of software and online services, firstly, because they are not known (in the case of online services) and secondly, the document would be saturated with information. Therefore, the Authors decided to select the most important ones, they are summarized in Tables 1 and 2, where one of the most important algorithms is ambiguity resolution because not all software and online services have this option enabled.

Reviewer #2- Comment 16: Lines 500-504. Unclear part of the text: “the difference between PRIDE-PPPAR and GIPSY X…”  “This analysis reveals the efficiency of online PPP services to obtain high-precision positioning similar to that obtained by a scientific software such as GIPSYX”. But PRIDE-PPPAR and GIPSY X are both scientific software, not online services. Please rewrite this part of the text.

Authors’ Response to Comment 16: Thank you for this comment/observation. The text has been reinstated for a better understanding. The changes are in green color in the revised version of the manuscript.

Reviewer #2- Comment 17: Lines 515-516 and 538-540. This is almost the same conclusion. Please merge it into one sentence.

Authors’ Response to Comment 17: The Authors would like to thank Reviewer #2 for such a valuable observation. Conclusions were combined as recommended by Reviewer #2.

Reviewer #2- Comment 18: Conclusions. The manuscript could be acceptable to publish but after major corrections in the text.

Authors’ Response to Comment 18: Thank you very much for considering our work for publication after major revisions. We did a considerable effort to respond all your recommendations. We appreciate a lot all your comments.

Author Response File: Author Response.docx

Reviewer 3 Report (New Reviewer)

Dear Authors,

thank you for conducting this study and preparing this article. I believe it will be useful for the positioning community to get an overview of the different software/service solutions.

Your analysis is straightforward but novel and I therefore recommend it for publication.

I do advise, however, to modify the document slightly before publication:

1. I believe Section "3. GNSS PPP mathematical model" should be dropped completely. Readers who are interested in the theory should check one of the cited references. This section is not needed to understand the rest of your paper. And even more importantly: The tools you analysed do not all use the specified observation equations and linear combinations. Many employ the Melbourne-Wübbena linear combination and possibly even other approaches. After reading Section 3, a reader might be inclined to believe that this is how all the solutions work, which would be misleading.

2. Figure 2, as one of the main results of your work, should be made more readable and I would advise to use the same scaling for the y axis for all solutions but RTKlib and to explicitly write in the Figure caption that RTKlib has a different scaling. Many readers will not read the whole document but rather quickly skim the Figures. Then, a consistent scaling will help immensly in comparing the different software solutions.

3. The errors you get for the RTKLIB processing are too large to be attributed to inferior algorithms/models alone. Please provide more details as to what might be the cause for these discrepancies. Are you sure that you configured the software correctly?

Author Response

Response to Reviewer #3 Comments (1^st round)

Comments and Suggestions for Authors

Reviewer #3- General Comment: Dear Authors, thank you for conducting this study and preparing this article. I believe it will be useful for the positioning community to get an overview of the different software/service solutions. Your analysis is straightforward but novel and I therefore recommend it for publication. I do advise, however, to modify the document slightly before publication:

Authors’ Response to General Comment: The Authors would like to thank Reviewer #3 for this positive comment and for recommending our work for publication after minor revisions. Again, thanks a lot. In this sense, please find as follows our responses to each of your comments.

Reviewer #3- Comment 1: I believe Section "3. GNSS PPP mathematical model" should be dropped completely. Readers who are interested in the theory should check one of the cited references. This section is not needed to understand the rest of your paper. And even more importantly: The tools you analysed do not all use the specified observation equations and linear combinations. Many employ the Melbourne-Wübbena linear combination and possibly even other approaches. After reading Section 3, a reader might be inclined to believe that this is how all the solutions work, which would be misleading.

Authors’ Response to Comment 1: Thank you very much for this observation. To address it, Section 3 corresponding to the PPP-GNSS mathematical model was removed.

Reviewer #3- Comment 2: Figure 2, as one of the main results of your work, should be made more readable and I would advise to use the same scaling for the y axis for all solutions but RTKlib and to explicitly write in the Figure caption that RTKlib has a different scaling. Many readers will not read the whole document but rather quickly skim the Figures. Then, a consistent scaling will help immensly in comparing the different software solutions.

Authors’ Response to Comment 2: The Authors would like to thank thank Reviewer #3 for this important observation. Figures 2 and 5 were generated again following the recommendations of Reviewer #3.

Reviewer #3- Comment 3: The errors you get for the RTKLIB processing are too large to be attributed to inferior algorithms/models alone. Please provide more details as to what might be the cause for these discrepancies. Are you sure that you configured the software correctly?

Authors’ Response to Comment 3: Thank you very much for this observation. We understand the concern about the software configuration. Please find the justification as follows. The 45 IGS stations were reprocessed for the 7 days with a new version of the RTKLIB software and a different configuration from the first processing. The errors of the results decreased significantly, in the new Figure 2 and 3 these changes can be seen. The manuscript was updated with the new results achieved by the RTKLIB software.

Author Response File: Author Response.docx

Reviewer 4 Report (New Reviewer)

A good and interesting paper

Only a few small notes:

- page 2, rows 75-79: "very popular" appear in contrast with "small number", please explain better;

- lack of reference about APPS (page 5, rows 218-224);

- lack of reference about Trimble Center Point RTX (page 6, rows 232-239);

- figures 4, 6 and 7 looks to big and could be rescaled. 

Author Response

Response to Reviewer #4 Comments (1^st round)

Comments and Suggestions for Authors

Reviewer #4- General Comment: A good and interesting paper. Only a few small notes:

Authors’ Response to General Comment: The Author are happy to read the above general comment. Reviewer #4 followed all our work. Thank you very much for the positive observation. Please find next a comprehensive response to all your recommendations. Again thanks.

Reviewer #4- Comment 1: page 2, rows 75-79: "very popular" appear in contrast with "small number", please explain better;

Authors’ Response to Comment 1: Thank you for this observation. We understand this confusion. The sentence was improved for a better understanding. Please find the changes in the revised version of the manuscript.

Reviewer #4- Comment 2: lack of reference about APPS (page 5, rows 218-224);

Authors’ Response to Comment 2: The Authors would like to thank Reviewer #4 for this outstanding note. To address this revision, the Authors incorporated references to the APPS online PPP service.

Reviewer #4- Comment 3: lack of reference about Trimble Center Point RTX (page 6, rows 232-239);

Authors’ Response to Comment 3: Thanks for this valuable observation. The Authors included the corresponding references to the description of the Trimble Center Point RTX online service.

Reviewer #4- Comment 4:  figures 4, 6 and 7 looks to big and could be rescaled. 

Authors’ Response to Comment 4: Thank you for this observation about those Figures. In this sense, Figures 4, 6 and 7 were generated again for a better understanding.

Author Response File: Author Response.docx

Round 2

Reviewer 2 Report (New Reviewer)

Thank you for revised paper. I do accept the explanations and changes made in the paper. In my opinion this manuscript is acceptable to publish in the present form.

This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.

Round 1

Reviewer 1 Report

As I said in my first review:

To complete the analysis, numerical results and corresponding graphs for the  standard deviation in the calculation of the coordinates of the different software  (for those that offer this information) are needed. This standard deviations in comparison with the differences between reference coordinates and those estimated by the software (accuracy) should complete the analysis. Those standard deviations must be very close to the accuracy to ensure that a particular software gives good results.

 

 

Author Response

Response to Reviewer #1 Comments (2^nd round)

Comments and Suggestions for Authors

As I said in my first review:

To complete the analysis, numerical results, and corresponding graphs for the standard deviation in the calculation of the coordinates of the different software (for those that offer this information) are needed. This standard deviations in comparison with the differences between reference coordinates and those estimated by the software (accuracy) should complete the analysis. Those standard deviations must be very close to the accuracy to ensure that a particular software gives good results.

Response: We appreciate Reviewer #1 comment. To address this comment, standard deviations for the X, Y, and Z components were extracted, respectively. For the updated version of the manuscript, the authors transformed the ECEF coordinates to the E, N, and U topocentric local system considering the reference coordinate of each station as the origin of the local system. As a result, the standard deviations for the E, N, and U coordinates were calculated from the standard deviations given by the software (for the X, Y, and Z coordinates) considering the error propagation law. The differences in the standard deviations of the X, Y, and Z, and E, N, and U coordinates were negligible, even less than 1 mm. Unfortunately, only 3/10 of open-source software and 3/5 of online PPP services reported standard deviations. To demonstrate the above fact, in the updated version of the manuscript, we provide a brief analysis about it. New Figures can be found in the document, please refer to Figures 4-7.

Author Response File: Author Response.docx

Reviewer 2 Report

1. The positioning accuracy in the X, Y and Z components is meaningless. Since the positioning accuracy in the horizontal component is higher than that of vertical component, it should be converted into E, N and U components.

2. Observation data from one day is not enough, more days are suggested.

Author Response

Response to Reviewer #2 Comments (2^nd round)

Comments and Suggestions for Authors

1. The positioning accuracy in the X, Y and Z components is meaningless. Since the positioning accuracy in the horizontal component is higher than that of vertical component, it should be converted into E, N and U components.
2. Observation data from one day is not enough, more days are suggested

Response: We appreciate the recommendations of Reviewer #2. Regarding the first observation, the X, Y, and Z coordinates were transformed to E, N, and U components, respectively. The reference coordinate of each station was implemented to establish the origin of the E, N, and U topocentric local system. The precision in the horizontal component was better than the vertical. On the other hand, with respect to the second recommendation of Reviewer #2, six more days of GPS observations were included and processed from January 1 to 7, 2020. The E, N, and U coordinates of those seven days were averaged, and the results were considered to analyse the precision and the construction of new figures (please refer to Figures 4-7 in the updated version of the manuscript). The average of the E, N, and U coordinates for the seven days contributed in reducing the differences with respect to those of a single day campaign.

Author Response File: Author Response.docx

Reviewer 3 Report

Thank you for the revision. I am happy to see a lot improvements. However, as stated in my first review, it is still needed to investigate the reason of the difference of softwares. In addition, please make sure that the basic settings are the same, for example, the observations (which channel), the stochastic model, the error model, the posterior residual editing and etc. In this way, one can safely compare the performance. Good luck to your manuscript.

Author Response

Response to Reviewer #3 Comments (2^nd round)

Comments and Suggestions for Authors

Thank you for the revision. I am happy to see a lot improvements. However, as stated in my first review, it is still needed to investigate the reason of the difference of softwares. In addition, please make sure that the basic settings are the same, for example, the observations (which channel), the stochastic model, the error model, the posterior residual editing and etc. In this way, one can safely compare the performance. Good luck to your manuscript.

Response: We would like to thank Reviewer #3 for this valuable observation. It is difficult to attribute a particular reason for the differences obtained with all the used software. In principle because the configuration of open source is different from those of online PPP service. Even though, some of the basic settings could be the same, some of them are not. For example, some of the software cosidered in our paper does not consider the ambiguity resolution option. Hence, it is well-demonstrated in the literature that the ambiguity resolution helps to improve positioning accuracy and minimize convergence time. In other words, the different software used in the experiment depends on the processing capabilities of each of them. Based on the above facts, it could be expected to have differences (milimeter level) in the performance of the different software implemented in our research. Thus, we added more information in the text of the manuscript to address this concern. In addition, new values for the standard deviations for the coordinates were calculated and analyzed only for 3/10 of open-source software and 3/5 of online PPP services which reported these values. The modifications can be located in the updated version of the manuscript. Again, thanks a lot for this suggestion!

Author Response File: Author Response.docx

Round 2

Reviewer 2 Report

All my concerns have been addressed. I recommend that this article be accepted in its current form.

Reviewer 3 Report

Thank you very much for the revision. Unfortunately, I don't believe the manuscript has been sufficiently improved to warrant publication. As I stated in my previous review, I am not confident about its conclusion as the author doesn't check the source code. As far as i am concerned, there are bugs in the open source softwares, which must be corrected before comparison. In addition, i want to see detail comparison of few softwares, instead of many softwares with general introduction.