Formal Analysis and Detection for ROS2 Communication Security Vulnerability

Round 1

Reviewer 1 Report

The manuscript presents a novel approach to detecting and analyzing security vulnerabilities in ROS2 by using formal methods. The use of a state transition system to model vulnerabilities, along with the integration of the CIA model and LTL for defining security properties, is a unique contribution to the field. This formal methodology is innovative and can potentially advance the state of research in robotic communication security.

The manuscript provides a well-rounded examination of the security issues in ROS2. It touches upon existing research, identifies gaps, and proposes a new method to address these concerns. The literature review is adequate, and the authors clearly understand the broader context of their work.

While the authors have made significant contributions, they should also discuss the limitations of their approach in more depth. Potential limitations could include the scalability of their tool or the specificity of LTL for certain types of vulnerabilities. Suggestions for improvement could involve exploring additional formal methods or extending the tool to cover a broader range of potential security issues.

The authors mention future work on runtime verification, which is encouraging, but they should also consider providing a roadmap for integrating this with their current work. It would be beneficial to discuss how these future directions could mitigate current limitations.

The manuscript is well-written, but minor grammatical corrections are necessary to ensure clarity and professionalism.

The research design and methodology are sound, and the results are presented clearly, though they could be enhanced with additional data visualization.

It is recommended that the authors discuss potential scalability concerns and the specificity of LTL in greater detail.

The implications for future research, particularly concerning runtime verification, are intriguing and should be expanded upon.

Minor editing of English language required

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

Thank you for an interesting manuscript on ROS2 security, which is indeed important. There is, however, one thing that puzzles me about discussing the security of (just) ROS itself. It’s always a good idea to lock the OUTSIDE doors in one’s house, not the INSIDE doors. Hence the idea of keeping *computers* running ROS secure, and not focus on ROS’ security itself. In this light, nodes and topics are not necessary “insecure” or “vulnerable”. I think many ROS-powered systems do (or may be able to) work in this paradigm, having security considered on the application level (inspired from here: http://wiki.ros.org/Security). The introduction here: https://ieeexplore.ieee.org/document/8794451 (“the ROS designers made a conscious decision to exclude security mechanisms”) also supports this idea. I would also consider this work at least for the related literature section.

Besides, I would also consider the options based on ROSBridge to allow the interaction of outside nodes with a ROS-based system and their implications on security.

There’s also this document which may be relevant to the topic: ROS 2 Robotic Systems Threat Model – http://design.ros2.org/articles/ros2_threat_model.html which could be considered for the related literature section.

Otherwise, the manuscript has a logical and coherent structure. It is also written in a concise manner, but please improve legibility - there are a couple of sentences without a predicate throughout the manuscript.

Arguably, as said above, the significance of this research can be disputed. I see the authors did not consider the so-called grey literature in the related work section, but practitioners cannot be excluded, given the topic specificity, as – in such cases - sometimes more relevant insights can be provided by practicians rather than by academia. And there is a lot of work on ROS security, outside the academia, which could be referenced to.

And another (and last) minor remark: the formulation of the title may be worth improving (I mean “based on formal model”).

There are a couple of sentences without a predicate throughout the manuscript. Also please re-check the punctuation. 

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 3 Report

Robotic systems are extensively utilized across various industries, leading to a growing concern regarding the security of communication between robots and their components. The security of ROS2, as a foundational framework for developing robotic systems, directly impacts the overall security of these systems. Therefore, it is crucial to investigate and address the security aspects of ROS2. The study presented in the article takes a formal approach to evaluate the communication security of ROS2. Initially, a state transition system is employed to identify potential vulnerabilities in ROS2, incorporating its communication mechanism and the fundamentals of penetration testing. Subsequently, the CIA model is applied to define security properties using LTL, building upon the vulnerability model. A tool for detecting vulnerabilities in ROS2 applications is then developed based on this model and security properties. Experimental testing on various ROS2-based applications reveals vulnerabilities in ROS2 that necessitate additional protective measures.

Some notes for improving the manuscript:

I recommend authors use the free app Grammarly to eliminate grammatical and stylistic errors! Some are mentioned below in the review.

Abbreviations are used in the abstract (and not only) that are not explained – it is true that later in the text of the paper there is a description, but it is accepted that at their first appearance, they are explained to the reader, for example, ROS, CIA, LTL, DDS, etc.

Same as ROS1-> Similar to ROS1

Same as unauthorized subscription -> Similar to unauthorized subscriptions

In Section II, We list some -> In Section II, we list some

Rescue robots [14,15], house-hold robots [16,17], telemedicine robots [18], drones [19], etc. – incomplete sentence

Although ROS2 uses DDS Security and SROS2 [31] To enhance its security and usability, but it is not absolutely secure. – unclear sentence…

Therefore, as long as the intruder node join the communication domain => Therefore, as long as the intruder node joins the communication domain

In my opinion, the word fail is not written as one word!

And the other states is the same as S6. =? And the other states are the same as S6.

Then these information will be corresponded to the vulnerability models. => Then this information will correspond to the vulnerability models.

• S5 = {idle, check1, auth, comm, wait_s, check2, service, call, success, fail}.

There are ten states in S3. -> here it should be S5, not S3.

linear temporal logic (LTL) – authors could cite some literature sources as well as give some details about this logic

It is not clear whether the models in Figures 1...7 are author's or from some literary sources – if it is the latter, then the sources should be cited!

Figure 6. stealing basic data of action. => Figure 6. Stealing basic data of action.

Figure 7. unauthorized action call. => Figure 7. Unauthorized action call.

Figure 14. experimental environment => Figure 14. Experimental environment

And the tool also performs runtime verification in the process of vulnerability detection. The implementation flow of this module is shown in Figure 13. => The tool also performs runtime verification in the process of vulnerability detection. The implementation flow of this module is shown in Figure 10. – perhaps it should be Figure 10 here.

In point 5.3.2 The implementation flow of this module is shown in Figure ??. – missing figure reference – it should be Figure 12.

In point 5.3.3 The implementation flow of this module is shown in Figure ??. – missing figure reference – it should be Figure 13.

Out of 36 sources, only 7 are in the period of the last 5 years - 2019 to 2024. I think it is good to cite more recent sources, considering the topicality of the issue.

I think the authors should clarify their contributions, especially in the part of the article up to point 6.2. Results and Analysis, i.e. in the previous points!

Mentioned above

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Thank you for addressing the concerns raised in the initial review and for the revisions made to your manuscript titled "ROS2 Communication Security Vulnerability Detection Based on Formal Method." The responses to the comments and the corresponding changes in the manuscript are appreciated and enhance the overall quality of the paper. Below are further comments and recommendations based on your revisions and the current version of the manuscript.

- Original Comment: The manuscript should discuss the limitations of the approach in more depth, particularly concerning the scalability of the tool and the specificity of LTL for certain types of vulnerabilities.

**Author's Response: You have outlined future considerations for validating additional security specifications such as authentication and identity authorization, which is encouraging.

##Follow-up: Your response addresses future extensions well. However, it would be beneficial to include a brief discussion in the manuscript regarding the current scalability of the tool and how it performs under different operational scales or environments. This would provide readers with a clearer understanding of the tool's applicability in real-world scenarios.


-Original Comment: Consider providing a roadmap for integrating runtime verification with current work.

**Author's Response: You have described future integration possibilities in the final paragraph of the future work section.

##Follow-up: The addition is noted and appreciated. To strengthen this section further, consider including a conceptual diagram or flowchart illustrating the proposed integration pathway. This visual aid would help clarify the integration process for readers.

- Original Comment: Enhance the results section with additional data visualization.

**Author's Response: You mentioned difficulties in representing data graphically due to large differences in data values and believe current tables and figures are sufficient.

## Follow-up: Understanding the challenge with data variability, it might still be beneficial to explore alternative data visualization techniques, such as logarithmic scales or divided bar charts, which could help in presenting complex data more effectively.

- Original Comment: Minor grammatical corrections are necessary.

** Author's Response: Grammar has been checked and corrections have been implemented.

## Follow-up: The effort to improve the manuscript's language quality is acknowledged. Please ensure that the final proofreading phase captures any remaining issues, potentially by a professional editor, to guarantee the manuscript meets publication standards.


The manuscript has significantly improved following the revisions. A few additional enhancements, particularly regarding the visualization of complex data and further clarification on scalability, could make the paper even stronger. I recommend accepting the manuscript after these minor issues are addressed.

Minor editing of English language required

Author Response

Thanks again for your review. Here are our revisions and responses.

Follow-up 1: Your response addresses future extensions well. However, it would be beneficial to include a brief discussion in the manuscript regarding the current scalability of the tool and how it performs under different operational scales or environments. This would provide readers with a clearer understanding of the tool's applicability in real-world scenarios.

Response 1:  We accept it. We have added a discussion of the environment in which the tool can be used and its extensibility in Section 6.3.2 of the manuscript, and the experimental result in that section is about the performance of the work under systems of different sizes. We believe this could illustrate the utility and limitations of the tool.

Follow-up 2: The addition is noted and appreciated. To strengthen this section further, consider including a conceptual diagram or flowchart illustrating the proposed integration pathway. This visual aid would help clarify the integration process for readers.

Response 2: Thanks for your suggestion. In fact, within our manuscript, we have integrated a runtime verification module into the tool. This module is capable of verifying whether the system under attack can meet the CIA security properties based on the ROS 2 system state information recorded during the occurrence of a vulnerability. This can be seen in section 5.3.1. In future work, our primary aim is to isolate this module so that it can operate independently within the ROS2 system, serving a function analogous to that of a monitor. In addition, we have drawn a brief flowchart in the final chapter.

Follow-up 3: Understanding the challenge with data variability, it might still be beneficial to explore alternative data visualization techniques, such as logarithmic scales or divided bar charts, which could help in presenting complex data more effectively.

Response 3: We accept it. We have transformed the data from the original table into two line charts (Figure 16) based on the range of values, which indeed provides a better representation of the variations in the tool's runtime.

Follow-up 4: We accept it. The effort to improve the manuscript's language quality is acknowledged. Please ensure that the final proofreading phase captures any remaining issues, potentially by a professional editor, to guarantee the manuscript meets publication standards.

Response 4: Thank you for your reminder. We will do our best to ensure the accuracy and standardization of language quality.

Reviewer 2 Report

Thank you, I have no further remarks. Just one minor recommendation: change "paper" with "article" throughout the manuscript.

Author Response

Thanks again for the review. We have taken your advice and have replaced all “paper” with “article” in the text.