TXAI-ADV: Trustworthy XAI for Defending AI Models against Adversarial Attacks in Realistic CIoT
Round 1
Reviewer 1 Report
Comments and Suggestions for AuthorsThe manuscript presents a clear and logically rigorous adversarial attack detection approach named TXAI-ADV, which leverages deep learning and machine learning classifiers to swiftly detect and prevent attacks in a CIoT environment. However, there are still some issues that need to be addressed:
1. Why were the one benign and three adversarial attacks proposed in the article chosen?
2. Punctuation marks are needed after the formulas.
3. It is recommended to convert the figures in the manuscript into vector graphics, as the current representation of the figures is not clear. Additionally, the captions of the figures need further elaboration.
4. I would like to know if the method proposed in this paper is effective in the field of security, such as pedestrian retrieval in intelligent surveillance. Please discuss the prospects of the proposed method in the security field based on the following articles:
[1] Pedestrian 3D Shape Understanding for Person Re-Identification via Multi-View Learning
[2] Enhancement, integration, expansion: Activating representation of detailed features for occluded person re-identification
[3] 3D person re-identification based on global semantic guidance and local feature aggregation
[4] Occluded person re-identification with deep learning: a survey and perspectives
[5] Pedestrian Re-ID based on feature consistency and contrast enhancement
5. Please summarize the effectiveness and future prospects of the proposed method in the conclusion section.
Comments on the Quality of English LanguageThere are some grammatical errors in the text and the language needs further revision.
Author Response
Comment: Why were the one benign and three adversarial attacks proposed in the article chosen?
Response: Thanks for the comment.
Action: The proposed attacks are novel and trending in research; that’s why we proposed these attacks.
Comment: Punctuation marks are needed after the formulas.
Response: Thanks for the comment.
Action: We ensure you we added the punctuation marks after each formula.
Comment: It is recommended to convert the figures in the manuscript into vector graphics, as the current representation of the figures is not clear. Additionally, the captions of the figures need further elaboration.
Response: Thanks for the comment.
Action: We updated the captions of the figures.
Comment: I would like to know if the method proposed in this paper is effective in the field of security, such as pedestrian retrieval in intelligent surveillance. Please discuss the prospects of the proposed method in the security field based on the following articles:
- Pedestrian 3D Shape Understanding for Person Re-Identification via Multi-View Learning
- Enhancement, integration, expansion: Activating representation of detailed features for occluded person re-identification
- 3D person re-identification based on global semantic guidance and local feature aggregation
- Occluded person re-identification with deep learning: a survey and perspectives
- Pedestrian Re-ID based on feature consistency and contrast enhancement
Response: Thanks for the comment.
Action: Based on the provided articles, we added a discussion of the proposed method's prospects in the security field.
Comment: Please summarize the effectiveness and future prospects of the proposed method in the conclusion section.
Response: Thanks for the comment.
Action: We updated the “Conclusion.” section.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
Comments and Suggestions for Authors
Electronics MDPI
Review the paper: TXAI-ADV: Trustworthy XAI for Defending AI Models Against Adversarial Attacks in Realistic CIoT Environment
Paper No:2948553
Dear Authors,
Congratulations on your choice of research topic. You're right, a lot of scientists focus on IIoT, and we forget about CIoT, so I think your topic is very important. As a reviewer, I express my admiration and curiosity about your paper. My task (duty) is to point out the comments, here they are:
Note 1: I would not use the word Environment in the title CIoT is enough.
Note 2: Keywords: add the abbreviation AI to “Artificial Intelligence”, because you use the abbreviation at work. Add the name and abbreviation ML to your keywords, because you often use this word at work.
Note 3: Section: Literature review, please change the title of this section to: Background to (of) research (analysis). I think this section has too narrow a review of literature to call it a “Literature review”. In addition, you did not use SLR , and this methodology is now required in many publications in a section called “Literature Review” so you better rename the section.
Note 4: In your work you use different, unnecessary nomenclature for CIoT, here are examples: CIoT environment, CIoT network, CToT networs, CToT system, CToT systems, so clean up, I suggest you use only CIoT without these unnecessary additions. This remark applies to the entire work, including the abstract.
Note 5: Do not use the word “acticle” better paper or study.
Note 6: Arrange the equations , after where: (add two dots) then describe all the notations used in the given equation, and this is not always the case with your equations, the reader guesses what Z, W, etc. means. This note applies to all equations.
Note 7: Section: Discussion and Section Conclusions.
I’m wondering if you could combine these sections, because Discussion is so short and it’s more like a retrospective than a discussion, unless other reviewers indicate to expand these sections, then you’ll have to do it, your research is important to me, your model is great and the whole methodology is great too, but that’s my opinion.
Best wishes
Reviewer
3 April 2024
Author Response
Comment: I would not use the word Environment in the title CIoT is enough.
Response: Thank you for the suggestion.
Action: We updated the manuscript title by removing the word Environment.
Comment: Keywords: add the abbreviation AI to “Artificial Intelligence” because you use the abbreviation at work. Add the name and abbreviation ML to your keywords, because you often use this word at work.
Response: Many thanks for the comment. We have addressed this comment. We updated the keywords by adding the abbreviation AI to “Artificial Intelligence” and ML.
Action:
Comment: Section: Literature review, please change the title of this section to: Background to (of) research (analysis). I think this section has too narrow a review of literature to call it a “Literature review”. In addition, you did not use SLR , and this methodology is now required in many publications in a section called “Literature Review” so you better rename the section.
Response: Thanks for the comment. We have added more publications in the “Literature Review” section.
Action:
Comment: In your work you use different, unnecessary nomenclature for CIoT, here are examples: CIoT environment, CIoT network, CToT networs, CToT system, CToT systems, so clean up, I suggest you use only CIoT without these unnecessary additions. This remark applies to the entire work, including the abstract.
Response: Many thanks for noticing.
Action: We updated the manuscript using only CIoT.
Comment: Do not use the word “acticle” better paper or study.
Response: Thank you for the comment.
Action: We updated the manuscript by removing the word “article”.
Comment: Arrange the equations , after where: (add two dots) then describe all the notations used in the given equation, and this is not always the case with your equations, the reader guesses what Z, W, etc. means. This note applies to all equations.
Response: Many thanks for the comment. We updated all the equations.
Action:
Comment: Section: Discussion and Section Conclusions.
I’m wondering if you could combine these sections, because Discussion is so short and it’s more like a retrospective than a discussion, unless other reviewers indicate to expand these sections, then you’ll have to do it, your research is important to me, your model is great and the whole methodology is great too, but that’s my opinion.
Response: Many thanks for the suggestion.
Action: We expanded the “Discussion” section by adding a detailed discussion of the study’s findings.
Author Response File: Author Response.pdf
Reviewer 3 Report
Comments and Suggestions for AuthorsThe author proposed a novel approach using deep learning and machine learning classifiers, including the SHAP technique, to detect and prevent adversarial attacks in CIoT environments with 96% accuracy, enhancing interpretability and accuracy in attack detection.
However, there are some questions that need to be clarified.
1. Algorithm1 shows the detection method. What is the complexity of the algorithm? How can it be effectively be used in real-time?
2. How does the proposed adversarial retraining technique compare to existing methods for enhancing IDS effectiveness against adversarial attacks in CIoT environments? Elaborate in the discussion section.
3. Can you elaborate on the specific features of the SHAP technique that make it effective in analyzing the impact of class features on the model's performance in detecting adversarial attacks? It is better to list up (in points) the features for easy reading.
4. In what ways does the study address the limitations and vulnerabilities of compressed audio DNNs in the context of adversarial attacks, and how does the stochastic compression method contribute to model resilience?
5. What practical implications do the findings of this study have for the broader field of IoT device security, particularly in terms of mitigating privacy violations and potential personal injury risks associated with adversarial attacks on smart devices?
The above questions can be clarified within different sections or the discussion section.
Author Response
Comment: Algorithm1 shows the detection method. What is the complexity of the algorithm? How can it be effectively be used in real-time?
Response: Thanks for the comments. The algorithm complexity and real-time applications are discussed in Section 3 (Material and Methods) and “Discussion” section.
Comment: How does the proposed adversarial retraining technique compare to existing methods for enhancing IDS effectiveness against adversarial attacks in CIoT environments? Elaborate in the discussion section.
Response: Thanks for the comments.
Action: We updated the “Discussion Section”.
Comment: Can you elaborate on the specific features of the SHAP technique that make it effective in analyzing the impact of class features on the model's performance in detecting adversarial attacks? It is better to list up (in points) the features for easy reading.
Response: Thanks for the comments. We added more details about the specific features of SHAP.
Comment: In what ways does the study address the limitations and vulnerabilities of compressed audio DNNs in the context of adversarial attacks, and how does the stochastic compression method contribute to model resilience?
Response: Many thanks for the comment. The detailed limitations and vulnerabilities of compressed audio DNNs in the context of adversarial attacks are discussed in the “Discussion” section.
Comment: 5. What practical implications do the findings of this study have for the broader field of IoT device security, particularly in terms of mitigating privacy violations and potential personal injury risks associated with adversarial attacks on smart devices?
Response: Thanks for the comments. The details of the practical implications are provided in the “Discussion” section.
Author Response File: Author Response.pdf
Reviewer 4 Report
Comments and Suggestions for AuthorsAfter reading the entire article, I noticed a number of aspects related to:
Abstract. It presents several aspects, but not very clearly outlined. I suggest the authors to create a more concise abstract specifying very clearly the object of the research, the methodology used, the results and conclusions of their study.
Introduction. Remove the word Motivation and 1.1. Research contribution and leave only the paragraphs. What is the gap that your article covers compared to other studies? Remove 1.2. Organization.
Literature review. I suggest the authors to structure this section by explaining more clearly the factors or causes that influence the research undertaken by them.
Materials and methods. There are some bolded notions that should be explained in context and not highlighted.
Experimental results and analysis. The authors present the obtained results and interpret them with the help of tables and graphs.
Conclusions. The authors present some of their own contributions to the research, but they do not present the limits of their research or future research directions. I suggest the authors to realize these aspects.
Author Response
Comment: Abstract. It presents several aspects, but not very clearly outlined. I suggest the authors to create a more concise abstract specifying very clearly the object of the research, the methodology used, the results and conclusions of their study.
Response: Many thanks for the suggestion. We updated the abstract.
Comment: Introduction. Remove the word Motivation and 1.1. Research contribution and leave only the paragraphs. What is the gap that your article covers compared to other studies? Remove 1.2. Organization.
Response: Thanks for the comment.
Action: We updated the manuscript, removing motivation and subsections 1.1 and 1.2.
Comment: Literature review. I suggest the authors to structure this section by explaining more clearly the factors or causes that influence the research undertaken by them.
Response: Thank you for the comment. we updated the “Literature Review” section.
Comment: Materials and methods. There are some bolded notions that should be explained in context and not highlighted.
Response: Thanks for the suggestion.
Action: We updated the Material and Methods section.
Comment: Experimental results and analysis. The authors present the obtained results and interpret them with the help of tables and graphs.
Response: Thank you for the comment.
Comment: Conclusions. The authors present some of their own contributions to the research, but they do not present the limits of their research or future research directions. I suggest the authors to realize these aspects.
Response: Thanks for the comment. We updated the “conclusion” section.
Author Response File: Author Response.pdf
Reviewer 5 Report
Comments and Suggestions for AuthorsThe paper presents a novel approach named TXAI-ADV that classifies to detect and rapidly attacks in a CIoT environment. This analysis aids in selecting optimal features for the adversarial attack dataset. The proposed approach claimed a 96% accuracy rate. This research is significant as it contributes to the prevention of adversarial attacks in CIoT.
Here are some suggestions to enhance the scientific soundness:
Refining the methodology section by providing insights into the choice of specific models over other methods and perhaps discussing any challenges faced during their implementation.
Additionally, list the limitations of the study and outline more specific future research directions based on the limitations or challenges encountered during your study.
Finally, there is a need to add a comparative analysis table that highlights the key research contributions in comparison to contemporary studies, chosen parameters, implemented models, and the results to validate claims.
Comments on the Quality of English LanguageThe manuscript is well-written, with a clear and coherent structure. Minor improvements could be made by proofreading to resolve some typo errors present in the manuscript.
Author Response
Comment: Refining the methodology section by providing insights into the choice of specific models over other methods and perhaps discussing any challenges faced during their implementation.
Response: Thanks for the comment. The method choice and its challenges are discussed in the “Discussion” section.
Comment: Additionally, list the limitations of the study and outline more specific future research directions based on the limitations or challenges encountered during your study.
Response: Thank you for your kind comment. The study limitations and future directions are provided in the “Conclusion” section.
Comment: Finally, there is a need to add a comparative analysis table that highlights the key research contributions in comparison to contemporary studies, chosen parameters, implemented models, and the results to validate claims.
Response: Thanks for the comment.
Action: We generated the novel dataset; there is no comparative analysis table with contemporary studies.
Author Response File: Author Response.pdf
Reviewer 6 Report
Comments and Suggestions for AuthorsDear Authors
the submitted manuscript deals with a very interesting topic, i.e. the usage of machine Learning techniques to detect adversarial attacks on sensitive data. It is well presented, clear and with a balanced organization of the sections. Usage of English language is fine. In this reviewer’s opinion, the manuscript can be published after positive reply to some minor comments that aroused through the reading.
Please find below a detailed list of these minor concerns.
My best regards
Line 170: since the presented technique has been used on an available dataset, it is maybe useful to report information regarding the availability of these data: where they can be downloaded, as example. Some brief information regarding the composition of the dataset are also important to the readers.
Line 176: some description of the used features also can be useful.
Line 186 and following: the four described attacks have been used with the same frequency? Or there are some used more often than others? This aspect is much important for the following results.
Line 250: did the train/test splitting have been performed just once, or more than one time to assure reproducibility? If yes, did the authors use some seed to fix and reproduce the splitting procedure?
Line 319: “According to Dong and Deng”: are the authors referring to some publication? Or to a book? In any case, a reference has to be added at this point.
Line 347: “Experimental Results and Analysis” section: some general comments regarding this section are now reported:
1) The tables with the results of the Machine Learning Model are very interesting. A final Table or maybe better a graph reporting the comparison between the outputs of the models would be very useful for a comprehensive understanding of the phenomena described.
2) Confusion matrices (Figure 3 and Figure 5) are too small, and the numbers and the axis labels are not visible.
3) Figure 7 is slightly bigger, but stil not readable (numbers are too small)
4) Figure 4 is also too small to be correctly read. Since the Figures are summarizing the results, they have to be much clear. Please take in mind that MDPI format permits to the figures to be larger than the text limits.
Author Response
Comment: Line 170: since the presented technique has been used on an available dataset, it is maybe useful to report information regarding the availability of these data: where they can be downloaded, as example. Some brief information regarding the composition of the dataset are also important to the readers.
Response: Thanks for the comment.
Action: The dataset description is in the section “Experimental Dataset Creation” and is available on request.
Comment: Line 176: some description of the used features also can be useful.
Response: Thanks for the comment.
Action: The features used in this study are provided in Figure 2.
Comment: Line 186 and following: the four described attacks have been used with the same frequency? Or there are some used more often than others? This aspect is much important for the following results.
Response: Thanks for the comment.
Action: We used the four described attacks with the same frequency to ensure the reproducibility and consistency of the ML models.
Comment: Line 250: did the train/test splitting have been performed just once, or more than one time to assure reproducibility? If yes, did the authors use some seed to fix and reproduce the splitting procedure?
Response: Thanks for the comment.
Action: We performed the train/test splitting once and trained the model based on this splitting.
Comment: Line 319: “According to Dong and Deng”: are the authors referring to some publication? Or to a book? In any case, a reference has to be added at this point.
Response: Thanks for the comment.
Action: We added the reference at point “According to Dong and Deng.”
Comment: Line 347: “Experimental Results and Analysis” section: some general comments regarding this section are now reported: The tables with the results of the Machine Learning Model are very interesting. A final Table or maybe better a graph reporting the comparison between the outputs of the models would be very useful for a comprehensive understanding of the phenomena described.
Response: Thanks for the comment.
Action: We added the comparison graph for the proposed models.
Comment: Confusion matrices (Figure 3 and Figure 5) are too small, and the numbers and the axis labels are not visible.
Response: Thanks for the comment.
Action: We updated the Confusion matrices (Figure 3 and Figure 5).
Comment: Figure 7 is slightly bigger, but still not readable (numbers are too small)
Response: Thanks for the comment.
Action: We updated the Figure 7.
Comment: Figure 4 is also too small to be correctly read. Since the Figures are summarizing the results, they have to be much clear. Please take in mind that MDPI format permits to the figures to be larger than the text limits.
Response: Thanks for the comment.
Action: We updated the size of Figure 4.
Round 2
Reviewer 1 Report
Comments and Suggestions for AuthorsThis manuscript solves my doubts very well. I recommend accepting this manuscript and publishing it.
Reviewer 3 Report
Comments and Suggestions for AuthorsThe authors responded well to the comments.
