OBU for Accurate Navigation through Sensor Fusion in the Framework of the EMERGE Project

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The authors present a report on their activities related to the EMERGE project.  The paper provides an overview of the proposed OBU system and its components and of the sensor fusion principle.  However, in my opinion, it lacks the scientific rigor expected for a publication in a scientific journal.  In particular, the experimental results as they are presented do not sufficiently support the conclusions.

Some detailed comments:

Figure 1: please add a color legend (which layer is which color)?  Also, 7 different colors are used, while only 4 layers are described.  To be clarified.

Figure 2: the link with Figure 1 is not obvious.  I’d suggest showing the broker in Fig 1 too.

Section 4.1.1 should probably be renumbered into 4.2.

Line 459.  The data set is extremely limited (4 minutes only). If there is any possibility to present additional data, it would make the results more convincing. 

Figure 8 lacks context.  The experiment is said to cover 240 seconds, so why is this graph only covering 85 seconds?   What do the three operating regimes listed on line 479 correspond to?

Table 1: what does the mean (\mu) refer to?  How was it computed?  What was the reference trajectory?

Table 1: “SV Used” is confusing: why are there only 11.2 SVs used in GNSS-only and 26.2 in GNSS+PointPerfect?   It seems to me that using PointPerfect corrections cannot increase the number of satellites used…  To be clarified.

Table 1, “SV tracked”: same comment: it is not logical that using PointPefect corrections would increase the number of tracked satellites.

Line 504 and following paragraph is confusing.  From lines 488-490, it seems that the experiment always used multi-constellation data.  The only difference between the two sets being the usage or not of the PP correction.   However, the paragraph starting on line 504 speaks about doubling the number of used satellites and improving the DOP.  This must be clarified: what was exactly the configuration used in the two columns of Table 1?

Figure 10: this figure would seem to indicate that SF is worse than GNSS, which contradicts the conclusion.  To be clarified.

Figure 10: the acronym “SF” used in the legend should be added in the list of acronyms.

Figure 11: “position and velocity differences during the vehicle trajectory”:  Clarify what “differences” refers to (difference between what and what)?  This figure represents errors: what is the reference trajectory against which the error is computed?  On this figure too, SF seems to perform way worse than GNSS-only…

Figure 13: how to interpret this figure?  How can the reader know if the blue trajectory is more accurate than the red trajectory?

Figure 13: add a description of the red and blue curves in the legend.

Line 568-575: conclusions should not introduce new material or ideas not discussed in the text.  I’d suggest moving this paragraph into a “discussion” section.

Line 578: “Specifically, the Xsens MTi-630 inertial sensor presents the noise density and stability levels required in this application”.  In my opinion, this is not explicitly shown in the paper.  It is more an a-priori assumption.  It should then be removed from the conclusion.

Line 579 “presents excellent results when integrated with…”.  As mentioned above, the experimental results seem to prove the contrary, as GNSS-only seems to be more accurate. 

Line 589-591: “In this way, the satellite navigation system could maintain the performance value…” as far as I understand, this is only an assumption, not demonstrated by the authors.  It should not be presented as a conclusion of the work.

 

Author Response

General comment
The authors present a report on their activities related to the EMERGE project.  The paper provides an overview of the proposed OBU system and its components and of the sensor fusion principle.  However, in my opinion, it lacks the scientific rigor expected for a publication in a scientific journal.  In particular, the experimental results as they are presented do not sufficiently support the conclusions.

Response
Thank you for your valuable comment. According to this general comment and to specific comments about numerical results, we have performed a new measurement campaign and processed collected data with a different approach, to identify a reference trajectory (very close to the ground truth) and evidence accuracy improvements achieved through IMU sensor fusion when GNSS is degraded.

Comment 1
Figure 1: please add a color legend (which layer is which color)?  Also, 7 different colors are used, while only 4 layers are described.  To be clarified.

Response 1
Thank you very much for your feedback on Figure 1. We have modified it to represent each layer (5 layers overall) and external block correctly. We also added a color Legend in Figure 1. Furthermore, we added some comments to specify the OBU COMM and CRYPTO ENGINE blocks as external subsystems of the architecture: in the final part of section 2, “The OBU COMM and CRYPTO ENGINE blocks are external subsystems ensuring remote communication and IP-based services access.”.

Comment 2
Figure 2: the link with Figure 1 is not obvious.  I’d suggest showing the broker in Fig 1 too.

Response 2
Thank you for your feedback. The broker depicted in Figure 2 describes the interconnection process between the different components of the system, defining through arrows the directions of flows to distinguish between unidirectional and bidirectional information flows. In addition, the broker manages communications with external systems. However, we have preferred to make a separate diagram that represents in a more clear way the information exchange between the SW elements.

Comment 3
Section 4.1.1 should probably be renumbered into 4.2.

Response 3
Thank you for your feedback. An editing error occurred, we have fixed it in the revised version.

Comment 4
Line 459.  The data set is extremely limited (4 minutes only). If there is any possibility to present additional data, it would make the results more convincing. 

Response 4
Thank you for your comment. In the new campaign of measurements, we have increased the duration of datasets to 10 minutes. We have performed various onroad tests to tune the setup and perform the final rounds of measurements, on which the presented numerical results are based.

Comment 5
Figure 8 lacks context.  The experiment is said to cover 240 seconds, so why is this graph only covering 85 seconds?   What do the three operating regimes listed on line 479 correspond to?

Response 5
Thank you for your feedback. The idea behind Figure 8 was to present results regarding the power consumption of the OBU and highlight the system's energy autonomy. The results correspond to a selected time interval (85 seconds) where the three identified power regimes are verified: low, normal and high. Thanks to your consideration, we believe the results presented in this section lacked context. Therefore, we have made the following changes:
- remove lines 502-504;
- remove Figure 8 “OBU Power Consumption”.
The following lines summarize the aspects of energy autonomy in the OBU: in section 5.1, “The backup battery connected to the Protected 18650 Li-Ion Separable Battery Holder has a capacity of 3000 mAh (at 3.7 V). Assuming OBU consumption in the high operating mode (average power of 5 W and average current of 1 A), the system autonomy, regarding energy, is about 2 hours.”

Comment 6
Table 1: what does the mean (\mu) refer to?  How was it computed?  What was the reference trajectory?
Comment 7
Table 1: “SV Used” is confusing: why are there only 11.2 SVs used in GNSS-only and 26.2 in GNSS+PointPerfect?   It seems to me that using PointPerfect corrections cannot increase the number of satellites used…  To be clarified.
Comment 8
Table 1, “SV tracked”: same comment: it is not logical that using PointPefect corrections would increase the number of tracked satellites.
Comment 9
Line 504 and following paragraph is confusing.  From lines 488-490, it seems that the experiment always used multi-constellation data.  The only difference between the two sets being the usage or not of the PP correction.   However, the paragraph starting on line 504 speaks about doubling the number of used satellites and improving the DOP.  This must be clarified: what was exactly the configuration used in the two columns of Table 1?

Response to comments 6-9
Thank you for your feedback. In Table 1, (\mu) denotes statistical mean; its meaning/computation method along the table is different for each considered item: (\mu) is calculated in different ways when it refers to a number of satellites, or a position, or an error measure, and so on. For what concerns “SV Used” and “SV tracked”, Table 1 contains the GNSS parameters measured in experimental tests with two different configurations: a first configuration using the information from two constellations, i.e. GPS+GALILEO, and a second configuration using all available GNSS constellations and the corrections offered by the PointPerfect augmentation service, i.e. GNSS+PointPerfect. These experiments were carried out in the same place where on-the-road vehicle tests have been performed, but with the objective of evaluating the performance of the GNSS system individually. For these preliminary accuracy/validation tests, we used fixed known positions (pre-calculated from long RTK processes), not a reference trajectory. The duration of the tests was 85 minutes (around 5000 s) at known positions and tests were carried out simultaneously using the two configurations described above on two identical GNSS devices (Ublox ZED F9P). Moreover, you are right: the descriptions/names of the configurations were incorrect. We modified the description in Table 1 and added an explanation of both configurations: GPS+GALILEO and GNSS+PointPerfect. The increase of the SVs is explained in this case because two GNSS constellations are used with the first configuration, and four with the second configuration.

Comment 10
Figure 10: this figure would seem to indicate that SF is worse than GNSS, which contradicts the conclusion.  To be clarified.
Comment 11
Figure 10: the acronym “SF” used in the legend should be added in the list of acronyms.
Comment 12
Figure 11: “position and velocity differences during the vehicle trajectory”:  Clarify what “differences” refers to (difference between what and what)?  This figure represents errors: what is the reference trajectory against which the error is computed?  On this figure too, SF seems to perform way worse than GNSS-only…
Comment 13
Figure 13: how to interpret this figure?  How can the reader know if the blue trajectory is more accurate than the red trajectory?
Comment 14
Figure 13: add a description of the red and blue curves in the legend.
Comment 15
Line 568-575: conclusions should not introduce new material or ideas not discussed in the text.  I’d suggest moving this paragraph into a “discussion” section.
Comment 16
Line 578: “Specifically, the Xsens MTi-630 inertial sensor presents the noise density and stability levels required in this application”.  In my opinion, this is not explicitly shown in the paper.  It is more an a-priori assumption.  It should then be removed from the conclusion.
Comment 17
Line 579 “presents excellent results when integrated with…”.  As mentioned above, the experimental results seem to prove the contrary, as GNSS-only seems to be more accurate. 
Comment 18
Line 589-591: “In this way, the satellite navigation system could maintain the performance value…” as far as I understand, this is only an assumption, not demonstrated by the authors.  It should not be presented as a conclusion of the work.

Response to comments 10-18
Thank you for your valuable comments. As explained in the response to the general comment, we have performed a new measurement campaign and processed collected data differently, using a hybrid approach that includes simulation of GNSS degradation. In this way, we are able to both identify a reference trajectory, very close to the ground truth (with an accuracy of a few centimeters), and to evidence accuracy improvements achieved through IMU sensor fusion when GNSS is degraded. Tests have been performed in good visibility conditions, so that GNSS+augmentation can provide a reference trajectory with very good accuracy (as certified by the PointPerfect augmentation service). At the same time, we have simulated a degraded GNSS by adding noise/errors to it, and computed the accuracy improvement that can be achieved in this case through IMU sensor fusion. In this way, by comparison with a reference trajectory, we can evidence that, when GNSS is not optimal, SF with IMU is better than GNSS+augmentation. Figures, legends, descriptions, and conclusions have been added/updated accordingly.

Reviewer 2 Report

Comments and Suggestions for Authors

The manuscript describes a complete GNSS+IMU navigation system for autonomous vehicles, starting from the hardware components, communication infrastructure and including computing algorithms. The manuscript is a very thorough description of the implementation of the system using EKF filtering. In addition to the algorithms, the HW components and the software architecture is also clearly described. 

 

But the combination of GNSS and IMU with EKF is hardly novel, when there is already a lot of articles about it. Point perfect is rather new, and it is therefore more interesting, and it seems to significantly improve the positioning accuracy with small cost. The V2X method is introduced, and it would indeed provide new options with collaborative positioning, but it is not used in the manuscript.

 

The listings of pseudocode cross-reference the mathematical models, which are not introduced yet. It would be better to show the mathematical models first, and only after them the pseudocode.  

 

GNSS IMU fusion is the most important in situations where the view to the sky is at least partially blocked and in existence of multi-path propagations. In this work, the system was only tested in an open field, where the full sky is always available. In this case, the IMU does not have much effect, and the result is not very exiting. In this case, the IMU fusion does not help very much, as shown in Figure 11 and 13. Numerical analysis of the GNSS+IMU would have been necessary.

 

The power consumption is shown in Figure 8, but it remains a mystery why the consumption varies so much, when the system worked in low, normal and high power modes. Also the Figure 8 lacks the unit in the y-axis. 

 

Minor comments

 - In Algorithm 1, line 15, the term ZUPT is used before introduction

 - The pseudocode in ALgorith 2, may contain an error. The find function in line 14 potentially returns a vector of values. In that case the comparison in level 14 is insufficient.

Author Response

Comment 1
The manuscript describes a complete GNSS+IMU navigation system for autonomous vehicles, starting from the hardware components, communication infrastructure and including computing algorithms. The manuscript is a very thorough description of the implementation of the system using EKF filtering. In addition to the algorithms, the HW components and the software architecture is also clearly described. 
Comment 2
But the combination of GNSS and IMU with EKF is hardly novel, when there is already a lot of articles about it. Point perfect is rather new, and it is therefore more interesting, and it seems to significantly improve the positioning accuracy with small cost. The V2X method is introduced, and it would indeed provide new options with collaborative positioning, but it is not used in the manuscript.

Response to comments 1-2
Thank you for your feedback. We agree: the proposed fusion is not a novel idea, and neither V2X nor collaborative algorithms are used in the presented data. Indeed, the main objective of this work is to illustrate an architecture/framework to integrate state-of-the-art positioning (exploiting a bleeding edge augmentation service) with the architecture of the EMERGE project, that defines an ecosystem for smart mobility, powered by V2X and the modern networking based on 5G, MEC, redundant and heterogeneous networking. We have updated the text to better point out this aspect, e.g. in the concluding section.

Comment 3
The listings of pseudocode cross-reference the mathematical models, which are not introduced yet. It would be better to show the mathematical models first, and only after them the pseudocode.  

Response 3
Thank you for your feedback. We have checked that the concepts were introduced in the most appropriate positions. We have chosen to use Algorithm 1 as a conceptual basis for the mathematical models that are subsequently shown. This approach aims to facilitate gradual understanding, allowing readers to establish the sequence of steps being explained.

Comment 4
GNSS IMU fusion is the most important in situations where the view to the sky is at least partially blocked and in existence of multi-path propagations. In this work, the system was only tested in an open field, where the full sky is always available. In this case, the IMU does not have much effect, and the result is not very exiting. In this case, the IMU fusion does not help very much, as shown in Figure 11 and 13. Numerical analysis of the GNSS+IMU would have been necessary.

Response 4
Thank you for your valuable comment. We have performed a new measurement campaign and evolved the processing approach to fully exploit the good quality of GNSS in clear sky conditions and, at the same time, evidence the improvement achievable through sensor fusion when the GNSS quality degrades. The “clean” GNSS, together with augmentation, is used to compute a trajectory very close to the ground truth (with a precision of a few centimeters). The GNSS quality degradation has then been simulated adding noise to the same data before post-processing them; the improvement provided by IMU sensor fusion is computed and evidenced just under the conditions of this simulated degradation. Regarding the numerical analysis of the GNSS+IMU, we have included in the article a comparison, obtained through the Root Mean Square Error method, to assess the enhancements on the trajectory provided by the algorithm compared to the trajectory obtained from degraded GNSS.

Comment 5
The power consumption is shown in Figure 8, but it remains a mystery why the consumption varies so much, when the system worked in low, normal and high power modes. Also the Figure 8 lacks the unit in the y-axis. 

Response 5
Thank you for your feedback. The idea behind Figure 8 was to present results regarding the power consumption of the OBU and highlight the system's energy autonomy. The results correspond to a selected time interval (85 seconds) where the three identified power regimes are verified: low, normal and high. Thanks to your consideration, we believe the results presented in this section lacked context. Therefore, we have made the following changes:
- remove lines 502-504;
- remove Figure 8 “OBU Power Consumption”.
The following lines summarize the aspects of energy autonomy in the OBU: in section 5.1, “The backup battery connected to the Protected 18650 Li-Ion Separable Battery Holder has a capacity of 3000 mAh (at 3.7 V). Assuming OBU consumption in the high operating mode (average power of 5 W and average current of 1 A), the system autonomy, regarding energy, is about 2 hours.”

 

Minor comments

Comment 6
 - In Algorithm 1, line 15, the term ZUPT is used before introduction

Response 6
Thank you for your feedback. The term ZUPT is now defined, before the pseudocode, as part of a short introduction to the subsequent subsections.

Comment 7
 - The pseudocode in ALgorith 2, may contain an error. The find function in line 14 potentially returns a vector of values. In that case the comparison in level 14 is insufficient.

Response 7
Thank you for your feedback on Algorithm 2. Indeed, line 14 could potentially return a vector of values if many GNSS epochs fell inside the interval t_IMU +- epsilon_GNSS. However, since IMU sensors typically have a much higher update rate compared to GNSS measurements, this scenario should not occur. The IMU data is updated more frequently (every 0.01 seconds) than GNSS (every 1 second) in our experiments, ensuring that the comparison in line 14 deals with scalar values, thus maintaining the integrity of the algorithm's logic.

Reviewer 3 Report

Comments and Suggestions for Authors

This article presents a new approach to the implementation of an on-board unit (OBU) dedicated to the navigation process. navigation process. The OBU is equipped with a Xsens MTi-630 AHRS inertial sensor, a multi-frequency constellation/multi-frequency GNSS receiver with ublox ZED-F9P module and communication interfaces for accessing the PointPerfect augmentation service. These issues are currently being developed and refined and this is due to the high interest and demand from industry. The article is interesting, but still requires the authors to address the technical parameters of the proposed solution (they must give the technical parameters of the developed system, such as the parameters of the network including the algorithms implemented in it and the conditions under which the tests were carried out). Please also provide the technical parameters of the GNSS receiver used.
There are several shortcomings in the paper such as:
1 The literature presented is incomplete and should be improved with more recent studies.
2 Please explain in detail the principle of operation of the developed system model shown in Fig. 6
3 Please explain all quantities used in the formulae.
4 The paper lacks information as to what lies beneath the GNSS signals whether they are GPS, Galileo or perhaps all at the same time please clarify this and what number of satellites were seen by the system.
5 The results and conclusions are sparsely discussed please correct this.

Author Response

General comment
This article presents a new approach to the implementation of an on-board unit (OBU) dedicated to the navigation process. navigation process. The OBU is equipped with a Xsens MTi-630 AHRS inertial sensor, a multi-frequency constellation/multi-frequency GNSS receiver with ublox ZED-F9P module and communication interfaces for accessing the PointPerfect augmentation service. These issues are currently being developed and refined and this is due to the high interest and demand from industry. The article is interesting, but still requires the authors to address the technical parameters of the proposed solution (they must give the technical parameters of the developed system, such as the parameters of the network including the algorithms implemented in it and the conditions under which the tests were carried out). Please also provide the technical parameters of the GNSS receiver used.
There are several shortcomings in the paper such as:

Response
Thank you for your valuable comment. According to this general comment and to specific comments in the remainder, we have performed a new measurement campaign and processed collected data with a more targeted approach, to also evidence accuracy improvements achieved through IMU sensor fusion when GNSS is degraded. We have also put an effort to provide more information and details about all elements the system is composed of.

Comment 1
1 The literature presented is incomplete and should be improved with more recent studies.

Response 1
Thank you for your feedback. We have put an effort to extend, update, and refine the literature survey in the Introduction.

Comment 2
2 Please explain in detail the principle of operation of the developed system model shown in Fig. 6

Response 2
Thank you for your comment. The experimental setup description (i.e. section 4 of the paper) has been extended to explain and clarify the system depicted in Fig. 6, according to your request.

Comment 3
3 Please explain all quantities used in the formulae.

Response 3
Thank you for your feedback. We have put an effort to improve readability and clarify the meaning of formulae.

Comment 4
4 The paper lacks information as to what lies beneath the GNSS signals whether they are GPS, Galileo or perhaps all at the same time please clarify this and what number of satellites were seen by the system.

Response 4
Thank you for your comment. We have put an effort to describe in a clearer way the sources of positioning data. Table 1 has been modified and information describing the GNSS configurations used has been added. Table 1 contains the GNSS parameters measured in experimental tests with two different configurations: a first configuration using the information from two constellations, i.e. GPS+GALILEO, and a second configuration using all available GNSS constellations and the corrections offered by the PointPerfect augmentation service, i.e. GNSS+PointPerfect. The number of satellites detected by the receiver configured with GPS+GALILEO signals was, on average, 28. We tracked 24 satellites (on average) and, at the end of the experiment, 11 satellites were used in the FIX process (on average). In the configuration that used all GNSS constellations and the PointPerfect service (i.e. GNSS+PointPerfect), an average of 37 satellites were detected; we were able to track the majority of them, and the average number of satellites used in the FIX process was 26.

Comment 5
5 The results and conclusions are sparsely discussed please correct this.

Response 5
Thank you for your comment. We have made a new campaign of measurements and processed data in a more targeted way, to provide more significant conclusions and to better evidence the improvement provided by IMU sensor fusion when the GNSS degrades. The discussion of results and conclusions have been reworked according to your comment and to new results.

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

Although the new version is a clear improvement, some additional modifications are still needed before publication.

The abstract should be adapted to the new version.  For example, lines 12-13 are not applicable anymore as the correction rate is not discussed in the paper.

Section 5.1.1 remains confusing: lines 558-562 indicate that the results are based on simulations, while lines 566-567 give the street were the test was conducted.  To be clarified (simulation or real-life?).

In 5.1.1, it is unclear if augmentation has been used or not.  It seems not from the accuracy plots, but this should be mentioned explicitly for correct interpretation of the results.  If I understand correctly, only the GPS+Galileo case and the GPS+Galileo+INS case have been compared.  No test is presented using the full GNSS system + augmentation + INS.  I think it is useful to write a few words to explain the reason for that. 

The conclusion still refers to aspects not addressed in the paper, such as on line 608: "the Xsens MTi-630 607 inertial sensor presents the noise density and stability levels required in this application".  This is never demonstrated in the paper.  Only the datasheet value is provided.

 

Author Response

General Comment

Although the new version is a clear improvement, some additional modifications are still needed before publication.

General response

We would like to thank again the editor and the anonymous reviewers for their insightful comments and suggestions, which helped us in substantially improving the quality of the manuscript. In the following, a point-by-point response to each of the reviewer’s comments is reported.

 

Comment 1

The abstract should be adapted to the new version.  For example, lines 12-13 are not applicable anymore as the correction rate is not discussed in the paper.

Response 1

Thank you for your comment. After the revision, we forgot to update this part of the abstract to the new set of experimental data and numerical results. We have corrected the text according to your comment. Furthermore, the text of all the Abstract has been revised, for a more smooth reading.

 

Comment 2

Section 5.1.1 remains confusing: lines 558-562 indicate that the results are based on simulations, while lines 566-567 give the street were the test was conducted.  To be clarified (simulation or real-life?).

Response 2

Thank you for your comment. We agree: upon reviewing the text, we found that the term "simulation" was used somewhat ambiguously. Indeed, tests were performed in real-life open-sky environments, open to public traffic, near our laboratory, where visibility is clear. We chose to have flexibility in the testing environment and to define the vehicle trajectories freely, simulating appropriately the degradation of the GNSS signal, rather than confining the tests and their reproducibility to a specific environment. In the revised version, we have corrected the misleading sentence and refined the text for a better understanding.

 

Comment 3

In 5.1.1, it is unclear if augmentation has been used or not.  It seems not from the accuracy plots, but this should be mentioned explicitly for correct interpretation of the results.  If I understand correctly, only the GPS+Galileo case and the GPS+Galileo+INS case have been compared.  No test is presented using the full GNSS system + augmentation + INS.  I think it is useful to write a few words to explain the reason for that. 

Response 3

Thank you for your comment. In the revised text, we have specified that augmentation was not used on the GNSS data processed by the sensor fusion algorithm and shown on the plots for accuracy performance comparison. The ground truth was obtained using GNSS + augmentation in open-sky conditions. But, as we have specified in the revised text of the paper, in the specific context of the application (namely vehicular navigation in urban environments) it would not have been beneficial/relevant to also include results from GNSS system + augmentation + INS. Therefore, we chose to focus on the primary data (GPS and Galileo) and demonstrate how INS provides additional information on the vehicle's position and orientation. This improves position estimation and brings it closer to the ground truth, particularly in certain conditions such as degraded GNSS signal.

 

Comment 4

The conclusion still refers to aspects not addressed in the paper, such as on line 608: "the Xsens MTi-630 607 inertial sensor presents the noise density and stability levels required in this application".  This is never demonstrated in the paper.  Only the datasheet value is provided.

Response 4

Thank you for your comment. Line 608 has “survived” in the revised draft as a spurious text that was no more pertinent. Now we have removed it. Furthermore, the text of all the concluding section has been revised, for a more smooth reading.