Survey of Security Issues in Memristor-Based Machine Learning Accelerators for RF Analysis

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The survey by Willian L. et al. differentiates itself from other security surveys by focusing on a specific crossing field, memristor-based machine learning accelerators for RF analysis. So, quite a little background knowledge on memristors, machine learning accelerators, and RF signal analysis is needed and introduced to the readers at the beginning. After the introduction, the manuscript is structured as a standard security survey, analyzing various threat models and potential countermeasures. As a survey on security, a vast, if not complete, scope of security threats was discussed. Compared with the threat model discussion, the countermeasures section could be more organized. Understandably, some cited works focus on general chip design or machine learning techniques indirectly related to memristors or RF applications because this topic is an immature field. Even so, readers may find most concepts inspirational or still applicable to the new memristor hardware platform. This survey could successfully bring the attention of readers from different backgrounds to this interdisciplinary topic. The last section listed some open problems without further comments or discussion, making it less insightful than previous sections. Overall, all concepts from various fields are well explained. The language is natural, fluent, and enjoyable to read.

A few minor specific comments are listed below:

1.      Although most are scratches and superficially simple, the figures only sometimes convey the ideas effectively. For example, In Fig. 2, two raw (temporal) signals are drawn, and no reactions or consequences exist. Instead, adding an extraction step instead of just raw signals can convey that fingerprints are not raw signals but need to be extracted. Adding a checkmark and a cross mark can convey that fingerprints can identify friendly and adversarial signals.

2.      Some references could be completed, like ref 19, 24, and more with missing links and dates. By the way, it would be encouraging if the authors could list any progress using a memristor-based or ML approach “that can quickly and accurately detect whether or not signals are present in their current channel and what channels are free around them.”

3.      There is a misused term in the example shown in Figure 3. ROC is approximately equal to TWN, and the country in b) should be referred to as PRC based on the flag. To make the caption more rigorous, “could occur in a different country” could be “could occur in a different country or region.”

4.      In section 5.3, while many remote attacks are conducted with limited prior knowledge, remote attacks can be any type (BB, WB, or GB), depending on how much the attackers may know beforehand. Thus, the authors may want to adjust the statement “where physical access 351 to the device is not available, and are therefore treated as BB access” accordingly.

5.      Figures 5 and 6 may be combined into one, as they focus on the same topic, “chiplet security,” and using two similar figures for this last piece looks overkill.

 

 

Author Response

Thanks so much for taking the time to review our paper! Our response is outlined briefly below:

General comments:

- Compared with the threat model discussion, the countermeasures section could be more organized

Response: Many of the countermeasures we outlined cut across multiple disciplines/ domains, and we found it difficult to provide any meaningful organization beyond the sections provided.

-  The last section listed some open problems without further comments or discussion, making it less insightful than previous sections.

Response: We provided more references to earlier paper sections and/or external sources in the Open Problems section

Specific comments:

Point 1 Response: We modified Figure 2 in order to more clearly indicate an extraction/ identification step of the RF signal

Point 2 Response: We combed through out reference list and tried to add dates and/or links as appropriate. In some cases, data/ links were simply not displaying under the latex template, so we modified the .bib file in order for the information to show.

Point 3 Response: We made corrections to the caption in Figure 3 as advised.

Point 4 Response: We corrected the use of "BB access" in section 5.3 as advised

Point 5 Response: We created a new figure to better exemplify the ideas we were attempting to convey with Figure 5.

Reviewer 2 Report

Comments and Suggestions for Authors

The manuscript entitled “Survey of Security Issues in Memristor-based Machine Learning Accelerators for RF Analysis” provides a timely and relevant survey of security issues in memristor-based machine learning accelerators for RF analysis, including three threat models: supply chain attacks, physical attack and remote attack. It is well-structured and presents a solid foundation of current knowledge. However, to enhance the clarity and academic rigor of the manuscript, the authors should consider addressing the following aspects:

1. Broaden the scope to include newer studies to provide a comprehensive view of the field. For example, some recent works on memristor PUFs are only updated until 2022, which is already 2 years ago, and many are before 2020. There have been many new studies in the recent years that should be included.

2. Clearly highlight what sets this survey apart from existing literature in a dedicated section. More tightly linking the background of RF analysis with the algorithms performed by memristors could help differentiate this work from existing literature.

3. Include more detailed explanations of three different security threats and defenses and differences between them, supplemented by concise examples. Simplified diagrams and examples would help improve layperson's understanding of these issues. And also, describe how memristors can address specific problems in RF analysis, respectively.

4. Consider creating new diagrams that compile key information from representative works in the field, which would enrich the entire article. Currently, a total of 6 figures are included, with Figures 5 and 6 related to chiplets, which are only a minor section (Section 6) of the paper, creating a mismatch.

5. Conduct thorough proofreading to correct grammatical errors and ensure consistency in formatting as per journal guidelines. For instance, References 22 and 77 lack the year, and some references have the year in bold while others do not, which should be standardized according to the journal's requirements. It is also unclear whether many references (e.g., 70-72) are from journals, conferences, or books.

Author Response

Point 1 Response: We conducted another search of the research field and added a new memristor PUF paper as a reference. 

Point 2 Response: Another point was added to the bulleted list in Section 1 to better explain what sets this survey apart from existing literature.

Point 3 Response: We believe the provided examples and diagrams are sufficient to illustrate potential threats. Section 4.4 provides several examples of memristors being applied to specific problems in RF analysis.

Point 4 Response: We replaced Figure 5 to remove the emphasis from chiplets, and instead focus on the neccessary heterogeneity of any memristor-abled systems.

Point 5: We combed through out reference list and tried to add dates and/or links as appropriate. In some cases, data/ links were simply not displaying under the latex template, so we modified the .bib file in order for the information to show.

Round 2

Reviewer 2 Report

Comments and Suggestions for Authors

1. Figure 3 uses real-world countries as examples, which is unnecessary and may appear unprofessional or biased. It is recommended to replace these with neutral identifiers such as "Country A, B, C" to avoid potential bias and maintain a professional tone.

2. Several abbreviations, such as FFT, RFML, and IFF, are used only once in the document. It is generally unnecessary to introduce abbreviations if they are not frequently used throughout the text. It would be clearer and more readable to simply use the full terms in these instances.

Comments on the Quality of English Language

2. Several abbreviations, such as FFT, RFML, and IFF, are used only once in the document. It is generally unnecessary to introduce abbreviations if they are not frequently used throughout the text. It would be clearer and more readable to simply use the full terms in these instances.

Author Response

Thank you for taking another look at our paper!   Per your feedback we made the following changes:   Country flags:    - Removed specific countries and flags from Figure 3   Abbreviations:   - Changed the following abbreviations to their full form/ removed abbreviation: - RFML - IFF - RSM - BCC - ECA - EPA - TERM - RNN - CNN - IC - Ag - LFI - BBICS - CF/CFs - COTS - SEM - NoC   Kept the following abbreviations as-is: - RF - ML - CMOS - PUF - NN - DNN - WB - GB - BB - IP   Some abbreviations were left in despite only being used once/ a few times, as the objects themselves are more commonly referred to using the abbreviation than the full term: - CPU - GPU - FPGA - ASIC - IoT - CAD - AES - DES - CIA - PDN - FFT - UCIe - ODSA - CXL