Round 1

Reviewer 1 Report

 In my opinion, the article has been done carefully and well, and it offers a good contribution. So, I recommend accepting this article, but after making the following modifications:

 1-    The abstract contains many abbreviations without definitions, such as NFC, RFID, and ATQA.

2-    In Equations (2), some symbols are not defined, please correct this.

3-    In the induction section, the authors should add more studies related to this article. 

4-    To make more sense, the concluding section should be expanded. In the conclusion, the study's contributions should be emphasized. What distinguishes this research from earlier research? In what ways does this article contribute to the field's understanding? At the conclusion of this work, the limitation should be mentioned.

 ==============================

Author Response

Author's Reply to the Review Report (Reviewer 1)

We would like to thank the reviewer for careful reading, helpful comments, and constructive suggestions, which have significantly improved the presentation of our manuscript titled ‘Deep Learning Aided RF Fingerprinting for NFC Relay Attack Detection’ (Manuscript ID: electronics-2140936).

We have carefully considered all comments from the reviewer and revised our manuscript accordingly. The manuscript has been double-checked, and the typos and grammar errors have been corrected. In the following section, we summarize our responses to each comment. We believe that our responses have well addressed all concerns of the reviewer. 

Response to Reviewer #1:

1. The abstract contains many abbreviations without definitions, such as NFC, RFID, and ATQA.

Response: Thank you for pointing out this problem in the manuscript. We have now introduced all the abbreviations and given the full version of their first usages in the abstract and the main body as well. We continue to use the same abbreviation throughout the whole article.

 

 

2. In Equation (2), some symbols are not defined. Please correct this.

Response: Thank you for pointing out this problem in the manuscript. The symbols in performance metrics equations are replaced with more comprehensible notations, and definitions for symbols and notations used are now given. The modifications can be found in the 1st paragraph of subsection 5.1.

 

 

3. In the introduction section, the authors should add more studies related to this article.

Response: Thank you for the above suggestion. We have added more works on Distance Bounding protocols as relay attack countermeasures and works on RF fingerprinting, which are related to our article in the Introduction section.

The added reviews of related works on distance bounding protocols are located in the 3rd paragraph in the Introduction section. The added reports on RF fingerprinting studies are located 10th paragraph in the Introduction section.

 

 

4. To make more sense, the concluding section should be expanded. In the conclusion, the study's contributions should be emphasized. What distinguishes this research from earlier research? In what ways does this article contribute to the field's understanding? At the conclusion of this work, the limitation should be mentioned.

Response: Thanks for the above suggestion. We have rephrased and expanded the Conclusion section and emphasized and listed our current article's contributions in the 1st paragraph of the Conclusion section. A discussion and comparison of the proposed method with previous works on relay countermeasures is given in the 2nd paragraph in the Conclusion section. In addition, we added a Limitations and Future Works section to elaborate limitations of current work, research directions that worth further investigation, and our future work in this section.

 

Reviewer 2 Report

This paper develops a method that can effectively detect and identify NFC relay attacks by waveforms of transmitted signals based on RF fingerprinting and deep learning.

The description of related work in the Introduction section should be enriched. There is a related line of work that studies  techniques of information embedding (see. e.g.,

A. Zaidi, and L. Vandendorpe, "Coding schemes for relay-assisted information embedding" IEEE Transactions on Information Forensics and Security, vol. 4, no. 1, pp. 70-85, Jan. 2009.

 

Author Response

Author's Reply to the Review Report (Reviewer 2)

We would like to thank the reviewer for careful reading, helpful comments, and constructive suggestions, which have significantly improved the presentation of our manuscript titled ‘Deep Learning Aided RF Fingerprinting for NFC Relay Attack Detection’ (Manuscript ID: electronics-2140936).

We have carefully considered all comments from the reviewer and revised our manuscript accordingly. The manuscript has been double-checked, and the typos and grammar errors have been corrected. In the following section, we summarize our responses to each comment. We believe that our responses have well addressed all concerns of the reviewer.

Response to Reviewer #2:

1. The description of related work in the Introduction section should be enriched. There is a related line of work that studies techniques of information embedding (see. e.g., A. Zaidi, and L. Vandendorpe, "Coding schemes for relay-assisted information embedding" IEEE Transactions on Information Forensics and Security, vol. 4, no. 1, pp. 70-85, Jan. 2009.)

Response: We gratefully appreciate your valuable suggestion. We have supplemented the literature reviews on relay attack solutions, and the corresponding modifications are located in the 5th paragraph in the Introduction section.

Information Embedding (IE) is a technique to embed an extraneous data signal into the host signal imperceptibly in order to prove the authenticity and reliability of data content. One of the noteworthy applications of IE is infrastructure-aided IE in wireless networks, where multi parties work cooperatively to reinforce watermark signals or reject unauthorized signals to secure the distribution of data. The nominated work studies relay-assisted cooperative IE and provide a coding scheme for implementing cooperative IE.

However, in an NFC relay attack, an attacker simply relays the signal as a whole and does not involve decoding, analyzing, or modifying the content. An NFC reader can still get fooled even with encryption methods in the protocol. While IE watermark approaches can verify whether the source (or publisher) of the data content (e.g., audio, image, or video signal) is legit, it can not verify if the source of the transmitted signals (e.g., a transceiver) is the authentic legit device. That is to say, IE methods are less likely to serve as a very effective countermeasure for NFC relay attacks.

Therefore, current countermeasures of RF/NFC relay attacks still mainly focus on Distance-Bounding protocols and ambient-based methods to verify the prover-verifier proximity in the physical space. Other means even include using a faraday cage to isolate the authentic tag (or smartcard). While in our article, we prove the feasibility of an RF fingerprinting-based solution.

However, we still mention the nominated lines of work in the article, located in the 5th paragraph in the Introduction section. In addition, we continue to pay attention to this line of research and hope to include the topic in our future works.

 

Reviewer 3 Report

The paper primarily presents one of the major pitfalls in using NFC for applications like PKES, smart card transactions, etc. As the information transmitted by RFID tags and readers is in air, it is dubious of securing the information. Among one such attack is Relay attack, the signals from the reader and tag are communicated even when they are far away, this is possible by relaying the signals from the reader to tag and vice-versa. In relay attacks encryption methods don't actually matter as the signal is relayed as whole.

Suggestions for further improvement:

1. Dataset split:
Authors have used 70% training split, 10% validation and 20% test split. To increase the model’s accuracy and validation, authors could have performed k-fold cross-validation; this might result in a tough choice between ANN and CNN.

2. Data Collection:
Majority of the data is wired relay attacks, but in real time scenarios the attacks are performed in wireless fashion. Data could have included wireless relay attacks through mediums like Bluetooth, cellular networks , bluetooth, Zigbee etc.

3. Measuring Techniques:
During measuring, there is not much change in distances between adversary tag and NFC Reader. RF fingerprinting and Distance Based approach should be used to give more precise classification.

4. Feasibility:
Deep Learning algorithms are heavy computational. Running them on production scale micro controllers would compromise the ease of use and quickness of NFC tags. A lightweight model would be a future research prospect for this solution.

 

Author Response

Author's Reply to the Review Report (Reviewer 3)

We would like to thank the reviewer for careful reading, helpful comments, and constructive suggestions, which have significantly improved the presentation of our manuscript titled ‘Deep Learning Aided RF Fingerprinting for NFC Relay Attack Detection’ (Manuscript ID: electronics-2140936).

We have carefully considered all comments from the reviewer and revised our manuscript accordingly. The manuscript has been double-checked, and the typos and grammar errors have been corrected. In the following section, we summarize our responses to each comment. We believe that our responses have well addressed all concerns of the reviewer.

Response to Reviewer #3:

1. Dataset split: Authors have used 70% training split, 10% validation and 20% test split. To increase the model’s accuracy and validation, authors could have performed k-fold cross-validation; this might result in a tough choice between ANN and CNN.

Response: We gratefully appreciate your careful reading and constructive suggestions. K-fold cross-validation can fully use the dataset and provide a more convincing evaluation of the considered models. Meanwhile, The result yielded by a train-validation-test scheme may depend on a specific data-splitting method.

Throughout our experiment phase, we conducted several (instead of just one) training with dataset shuffled and the same set of tuned hyperparams, observing that the performances did not vary or fluctuate much. Nevertheless, the reviewer’s concern is of importance for our further study, and we will show the results of a more comprehensive evaluation in our future work.

Currently, the CNN yields a slightly higher model performance than an ANN. A reasonable explanation could be the effect of the kernels——while the RF fingerprints are more likely to gather on the bit pulses in the waveform, CNN’s kernels can lock on those local features and yield better performance.

 

2. Data Collection: Majority of the data is wired relay attacks, but in real time scenarios the attacks are performed in wireless fashion. Data could have included wireless relay attacks through mediums like Bluetooth, cellular networks , bluetooth, Zigbee, etc.

Response: We gratefully appreciate your valuable suggestion. In this article, we introduced the wireless implementation of an NFC relay attack using WiFi. Since related reports demonstrated its easy conduct (via smartphones, etc.) and wide-range features, WiFi-based relays have recently gained more popularity among thieves. Hence we regarded WiFi as a representative implementation of a wireless relay attack in this article. Bluetooth, cellular networks, and Zigbee are also feasible ways to conduct relay attacks. The realizations of these wireless relays are similar, and we would like to include implementations other than Wifi in future works.

 

3. Measuring Techniques: During measuring, there is not much change in distances between adversary tag and NFC Reader. RF fingerprinting and Distance Based approach should be used to give more precise classification.

Response: We gratefully appreciate your valuable suggestion. Our corresponding response is as follows.

Regarding the dataset acquisition phase, changing distances between the tag and reader can introduce variations to the waveforms and improve the diversity of the dataset. In our NFC setting, the communication distance is relatively near (about a few centimeters) compared to other wireless signals such as WiFi or UHF. This is due to the power suppression that usually operates on the on-the-shelf product-level NFC readers, which lowers its range for NFC transaction safety considerations. As a result, we observed that the signals become too weak for the relay attack to succeed when adversary tags are placed too far away from the reader, so we did not include relayed signals collected from varied distances in our dataset for the moment being.

Nevertheless, we have already introduced variations to our dataset, but instead of varying the distance, we included various positions (e.g., the basic ‘upward’ position, tilted, in different angles as well) that can actually cause changes to the signal waveforms, in order to, at least, avoid collecting the samples in a static, fixed condition.

Regarding the methodology, it is possible to combine RF fingerprinting and DB protocols, making them work jointly and improving a defensive solution's overall accuracy and reliability. However, our idea is that, in most cases, distance bounding protocols require modification to the transmission protocol that might hinder the implementation of such applications in real life. In this article, our endeavor is to provide a method that avoids modifications to the transmission protocol.

 

4. Feasibility: Deep Learning algorithms are heavy computational. Running them on production-scale microcontrollers would compromise the ease of use and quickness of NFC tags. A lightweight model would be a future research prospect for this solution.

Response: We gratefully appreciate your suggestion. This is a very constructive suggestion since the ultimate goal of the current research is to port the method of this paper to a lightweight implementation.

Computation cost should be considered when the deep-learning-based method is to deploy on microcontroller devices that are usually resource-constrained. Possible ways could be utilizing quantization in our model or lightweight training frameworks such as TF Lite to adapt to a micro-controller-oriented model. However, the work is not discussed in this paper due to space limitations, and we would like to bring up the work in our future papers.

Round 2

Reviewer 2 Report

The authors have revised their manuscript in a satisfactory manner; I recommend Acceptance.