A Survey on Memory Subsystems for Deep Neural Network Accelerators

Round 1

Reviewer 1 Report

1. This survey should include a description of the conventional memory architectures for DNN accelerators. The following works should be discussed:
   10. Li et al., "SmartShuttle: Optimizing off-chip memory accesses for deep learning accelerators," 2018 Design, Automation & Test in Europe Conference & Exhibition (DATE), 2018, pp. 343-348, doi: 10.23919/DATE.2018.8342033.
   11. Putra et al., "ROMANet: Fine-Grained Reuse-Driven Off-Chip Memory Access Management and Data Organization for Deep Neural Network Accelerators," in IEEE Transactions on Very Large Scale Integration (VLSI) Systems, vol. 29, no. 4, pp. 702-715, April 2021, doi: 10.1109/TVLSI.2021.3060509.
   12. Marchisio et al., "DESCNet: Developing Efficient Scratchpad Memories for Capsule Network Hardware," in IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 40, no. 9, pp. 1768-1781, Sept. 2021, doi: 10.1109/TCAD.2020.3030610.
2. The figures are blurry. They should be exported in vectorial form (.pdf, .eps, .svg, …). It is bad conduct to make screenshots of figures from other papers. All the figures should be redrawn by the authors.
3. 3 is badly designed and does not give a clear indication of how the execution of a NN in a Neurocube is performed.
4. 10 presents the RAPIDNN architecture in a very abstract way. It can be enhanced to include more details in each box.
5. 18 is drawn in a bad quality. There are 2 red dots, and the right part seems cut.
6. 30 contains (a) and (b). They should be described in the caption.

Author Response

Dear Reviewer,

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

1. The survey presented in this paper represents an interesting topic and it represents a read of great value in the field. However, the paper feels like a strict technical report and it needs to be drastically reconstructed for acceptance in a scientific journal of this level. Please find my comments below that need to be addressed.  

   1. The survey should contain a significantly higher number of cited articles
   2. Images of higher quality should be used in the article
   3. What is the main novelty of this paper? What are the main differences between this paper and other surveys? Please define the novelty and address it in the introduction and in the abstract. 
   4. According to which criteria the cited papers were selected. Please describe the paper selection methodology according to PRISMA statement. Please add PRISMA methodology section and add graphical representation. 
   5. In the discussion section, please add a detailed comparison of presented systems with quantifiable data. Add graphical comparison where possible.
   6. Discuss the novelty of the paper in the conclusion

Author Response

Dear Reviewer,

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 3 Report

The paper is very interesting and very well written. However, some minor adjustments should be made to improve the manuscript's quality. 

1. In abstract at the line 11 add space between intelligence and (AI)
2. In abstract, at the line 24 remove comma after 2-
3. Set maximum of five keywords in alphabetical order 
4. The main contributions are fine. However, could you please indicate hypothesis in the same form 
5. Center the Figures 2,3,6,7,8,9,10,11...33
6. Before Conclusion add section Discussion and comment all the results. Rename the Conclusion section to Conclusions since there is more than one conclusion. 
7. In Conclusions section extract only the most important information from the discussion. However, the extracted information must provide answeres to the hypothesis given in the introduction section 

Author Response

Dear Reviewer,

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 4 Report

My comments on this work are as follows:
- first and foremost, this review lacks a ranking of methods, relative to certain factorials indicating accelerated computation. The reviewed methods should also be considered by their cost, failure rate, simplicity. It is worth to extend the information about their specific applications.
- Another important thread is to point out examples of hardware solutions to be used, so that this work has practical value,
- in the paper, the authors use the word proposed, which would suggest that they are proposing certain architectures, if this is the case, you still need to clearly state what is new in relation to existing architectures,
- Finally, I think this paper will be a valuable journal entry, when it makes clear which architecture is for what, and where to get it for practical soluctions. It may also be worth discussing the problem of using real-time neural networks in Internet of Things systems to increase the value of this work. 

Author Response

Dear Reviewer

Please see the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

The effort made by the authors in addressing the reviewers' comments from the previous round is appreciated. The quality of the manuscript is significantly increased.

Minor Comments

1. Figs. 23 and 25 exceed the page margin.
2. Table 1 exceeds the page margin.
3. In Section 1: “Though other existing surveys report DNN accelerators in overall but, This article will provide an analysis of the various memory architectures used in DNN accelerators depending upon the many possibile design scenarios.” It seems that is used a wrong punctuation (“but, This”) between the sentences. Moreover, there is a typo: “possibile” => “possible”.

Reviewer 2 Report

The paper can be published.

Reviewer 4 Report

Most of my comments have been addressed I have no further remarks on this work.