CANPerFL: Improve In-Vehicle Intrusion Detection Performance by Sharing Knowledge

Round 1

Reviewer 1 Report

1. Abstract- Max 200-250 words- Start with Background, then add Problem Definition, then scope. Then highlight what is the Aim/Objective of the paper, and what novelty is there in the proposed technique, and what is proposed in terms of Hardware and in the last lines, highlight clearly in what %age and in what parameters the proposed methodology is better as compared to existing techniques and what is the overall analysis of proposed technique. And even stress on the outcome of some research on the hardware proposed. 

2. In literature Section, min 20-25 Papers should be highlighted. And start with Last Name of the author, and then use et al. where it is required and then add reference number and then add year name. And make sure that papers from only SCI/SSCI/SCIE Journals should be added. And every paper should be elaborated in 3-5 Lines in separate Paragraph and every paper should be written in Past tense, regarding what was proposed, what is the novelty and what experimental results/conclusions are there. At end of Related works, highlight in 9-15 lines what overall technical gaps are observed that led to the design of proposed methodology. 

3. At the final section, Give the Heading:

Experimentation and Results Under This, add the first sub heading: Simulation Parameters and all the parameters of simulation.Then add Results and every sub heading of the parameter should be there and every parameter should be given with Theory, Mathematical Formula, Table with Data and Graphs. And min 3-5 existing techniques to be compared. And then add Analysis Section. 

4. Make the conclusion more detailed and stress on the proposed technique with novelty and experimental results and then focus on the analysis and then add Future work and future work should be technical defined with regard to the additional work to be performed on the existing works in the future.

 

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 2 Report

Comments: This paper designs an CAN bus IDS using federated learning. Three real-car datasets are collected and utilized to experiment with the proposed IDS. It focuses on enhancing the vehicle manufacturer's IDS by training a global model rather than just thinking at the individual vehicle level. The topic of this paper which aims to solve the privacy and performance concerns about the application of intrusion detection models on different car manufacturers is forward-looking and with significant value.

 

 

In my opinion, the current form of the manuscript includes different issues, as well as aspects that should be clarified. In particular:

1. The specific details of the training and testing processes need to be explained, such as which CAN data features were used.
2. The persuasive power of demonstrating data heterogeneity only from the aspects of ID and data generation amount is not enough. For example, there may be situations where the IDs are different, but the payloads are the same due to KIA upgrades. Of course, this has little impact on the contribution of this article.
3.  Some more specific training parameters are not provided. Since the model parameters such as training epochs, data used for training, and time are different, is it reasonable to compare the local, Fed, and Fed-FN models directly?

4. Why does the local model of BMW not perform well? Is it possible that it is due to parameter adjustments during training, and have the optimal parameters been found?
5. There is a lack of direct training and detection time performance indicators. For manufacturers, the effectiveness of an IDS still has to be judged in combination with the attack detection rate and time cost. Ultimately the manufacturer will still deploy the IDS on vehicles.
6. Why the F1 score decreases as the data increases from 10K to 20K in KIA's local model?
7. There are some inconsistencies in the wording that need to be optimized.

Comments for author File: Comments.pdf

Author Response

Please see the attachment.

Author Response File: Author Response.docx