Motor Imaging EEG Signal Recognition of ResNet18 Network Based on Deformable Convolution
Round 1
Reviewer 1 Report (New Reviewer)
The manuscript entitled “Motor Imaging EEG Signal Recognition of ResNet18 Network Based on Deformable convolution” proposes a motor imaging EEG signal classification method based on the fusion of the improved ResNet18 convolutional neural network (CNN) with the deformable convolutional Network (DCN). The original signals are transformed into short-time Fourier transform (STFT) time-frequency maps. The proposed method is experimentally validated, achieving high classification performance.
The manuscript is well-written and easy to follow. The presented study is interesting and has potential practical application. The utilized techniques are described in detail. The results are well presented (with room for some visual improvements) and appropriately discussed.
However, here are some comments I would like the authors to address before the manuscript is considered for publication:
1. Please provide a short overview of the manuscript’s main contributions before the last paragraph of the Introduction section.
2. The literature review is somewhat modest, with only 10 references addressed in the Introduction section (the rest are used to explain utilized technical concepts). However, the application of deep CNNs with various two-dimensional signal representations has become a hot research topic recently. Therefore, I would like to suggest the authors supplement the introductory part with some of the recent studies on this topic to briefly illustrate the state-of-the-art performances of the CNNs and time-frequency representations (including STFT and alternative time-frequency distributions) in many different applications today and provide an interested reader with examples. Please consider briefly mentioning the following papers for illustration purposes: 10.1007/s10044-020-00921-5, 10.1109/ACCESS.2021.3139850, 10.1109/TNNLS.2020.3008938.
3. Please try improving the resolution of Figure 6, as some parts are hardly visible.
4. Please increase the text size in Figure 7.
5. In Figures 12 and 13, what is the difference between subfigures a) and b), as they have the same captions?
6. In the Conclusion section, please provide some limitations of the proposed method.
7. In the Conclusion section, please also provide some directions for future research.
Author Response
Please see the attachment.
Author Response File: 
Author Response.pdf
Reviewer 2 Report (New Reviewer)
Good work. I recommend the authors do a final revision to the manuscript. I noticed you had checked it using a professional English proofreader service but there are some issues that only we, formal researchers, cannot leave pass. For example, units should be separated from the quantity; do not use the form "In [20], the authors ...", instead use their last names (obviously with et al. if so) as you would desire to be mentioned, and so forth.
Plus, the discussion could be considered shallow. In fact, you are just mentioning what figures and tables have, but I would be happy to read more about your thoughts and possible explanations.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Reviewer 3 Report (New Reviewer)
This paper show motor imaging EEG signal recognition of ResNet18 Network 3 based on deformable convolution. I have some comments:
1.The experimental data selected in this paper is too single source. It is suggested that the author experiment on multiple groups of EEG data, and give the source location (for example, website) and the corresponding conclusions, to avoid the problem of limited applicable environment.
2. Author should compare their results with existing methods published in recent 5 years.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Reviewer 4 Report (New Reviewer)
In this study, the authors propose a method to prevent possible data loss that may occur due to noises that are naturally present in EEG signals. This method based on fusing the improved ResNet18 network with the Deformable Convolutional Network (DCN). It is argued that the proposed method is more successful than the classical methods. I think that the work is generally fluent and understandable. Improvements should be made in the following points.
1- In Line 35: "EEG signals have the characteristics of weak amplitude, complex background noise, randomness, significant individual differences, and contain a lot of time and space information, making it challenging to achieve reliable recognition." Please support this claim with literature.
2- Line 98: There is a hyphen in a word. It should be corrected. "... operations, each position p0 on the output fea-ture..."
3- In the 3.2. Improved ResNet18 network model building section, a table tha reveal the changes made on ResNet18 to make DCN should be added for comparison.
4- Among the subjects A01-A09 presented in Table 6, it is seen that the most successful structures in A08 and A09 are different structures. What is your opinion about the reason for this?
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Round 2
Reviewer 1 Report (New Reviewer)
The authors have adequately addressed most comments. The manuscript has been improved after revisions.
Two minor comments regarding the authors' response and revised manuscript still need to be addressed.
1. When referring to the added reference [9] in the manuscript's text, please use the author's surname and not the initial of the first name ("N et al" - "N" is not the surname). Please also make sure this is correctly done for all references cited in the manuscript's text.
2. In response to my comment 5, the authors correctly explained the difference between the two subfigures. Moreover, this difference was clear to me from the context prior to the revisions. However, the authors need to change the captions of subfigures in the revised manuscript to reflect this difference ((a)-left, (b)-right).
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.
Round 1
Reviewer 1 Report
Main issues: (1) The proposed method is not described well, and the methods on which it relies are poorly presented. The description of DCN repeats the description in Dai et al., 2007 with some typos as detailed below, and its training is not well explained as detailed in the specific comments. Most importantly, the main contribution of the paper, i.e., the integration of DCN with ResNet18 within a residual layer is not explained at all. (2) The method for evaluating the performance is not explained: how were the EEG datasets divided into training and testing? We the networks pre-trained on other datasets?
1. Introduction:
1.1. The first paragraph includes some misleading claims. Considering the first sentence: the BCI does not convert human intentions to EEG! This conversion is the result of the dipoles generated by neural activity in the brain - the BCI interpret the EEG activity to extract the intention. The second sentence claims that BCI create a channel for information exchange – but most BCIs communicate only one-way.
1.2. Second paragraph:
1.2.1.what does “poor stability” mean in terms of EEG? Do you mean non-stationary? Stability usually refers to systems rather than signals – use a proper term. The message of this sentence was made very well in the first sentence of the Abstract and can be just repeated here.
1.2.2.Second sentence (starting with Therefore) is trivial: of course improving EEG recognition would improve the accuracy of EEG-based BCIs.
1.2.3.The 3rd and 4th sentences are not related to each other – either move the 3rd sentence to another place, or start a new paragraph with the 4th sentence. In either case the importance of feature extraction to EEG-based BCI should be clarified before that sentence.
1.2.4.The rest of the paragraph should also be divided into two paragraphs describing previous classification work and previous work on channel selection in separate paragraphs.
1.3. Third paragraph: The first sentence is not clear – are you referring to your Network or in general to a problem with Networks that include deformable modules?
2. Section 2:
2.1. Symbols: in line 94 and 98 should be p_0 and in line 97 should be p_n. In line 104, the first symbol should be p and the second should be q!
2.2. Figure 1 and 3 are the same as Figure 1 and 2 in Dai et al, 2017, but it is not mentioned that the Figures are reproduced with permission. Getting a permission and stating that those Figures are taken from Dai et al, 2007 is a must.
2.3. Text and Equations in lines 91-107 are almost directly taken from Dai et al, 2007 – but with some typos as noted in 4.1 – maybe they should be moved to an appendix.
2.4. Figure 2: this is similar to Figure 5 in Dai et al., 2007 – only shown on a different picture.See 4.2. In any case, the caption should be extended to explain: why the pixels on the right (for the DCN) are not on a standard grid. My understanding is that the pixels are placed on a standard grid – and the difference is only in the offsets. Furthermore, there is no clear difference in the resulting picture.
2.5. The paragraph between Figure 2 and 3 does not explain DCN well.
3. Section 3:
3.1. First paragraph in 3.1 should be re-edited. In particular, the first two sentences; the elements of ResNet should be in plural; “gradient disappearance” should be “gradient vanishing”.
3.2. Section 3.1: explain what the residual part id doing.
3.3. Section 3.2: here and elsewhere use the term “receptive field” rather than sensing or feeling field.
3.4. Section 3.2: the first paragraph repeats what was already explained (about the merit of DCN) and should be omitted.
3.5. Section 3.2 second paragraph: This should have been the main paragraph that explains the integration of DCN with ResNet but it does not.
3.5.1. The main issue to be described in details is how the DCN is integrated with in the residual layer.
3.5.2.The two residual layers that combined DCN (the second and third residual layers – should be shown separately – one after the other – and not as “X2” of a single layer.
3.5.3.There is no need to specify the size of the other “standard” convolution and max pooling layers, since they already appear in Figure 5.
4. Section 4:
4.1. Section 4.1 regarding BCIC IV dataset 2b, 2a and 3: the description of the dataset is poor. It is based on well-known datasets, including two figures that are reconstructed from the BCI competition paper, but there are some discrepancies:
4.1.1.There are only 2 training sessions not 3. The text refers inconsistently to both data sets and data groups, while it seems it means dataset all the time.
4.1.2.The caption of figure 8 should state that it is with feedback.
4.1.3.It is not clear that MI is performed throughout the time the feedback is given.
4.1.4.It is not mentioned that the data set includes EOG data and was collected to test EOG removal algorithms.
4.1.5.Figure 9: again reproduced from competition paper
4.1.6.Dataset 2a: the recorded electrodes are not mentioned.
4.1.7.IN SUMMARY: it is critical to provide only the information that is relevant to the current paper and provide references to the proper literature on the competition. FINALLY: DO NOT show figures unless reproduction was permitted.
4.2. Section 4.2: Labels of Figure 11 and 12 should be in English
4.3. Section 4.3: It is not clear how the performance was evaluated. In particular, were the networks pre-trained on other datasets? How were the EEG datasets divided into training and testing data? Was k-fold cross validation used?
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Reviewer 2 Report
The authors had presented a novel method of integrating DCN and ResNet18 for EEG signal analysis. There are a few issues to be addressed before the paper can be considered for acceptance:
1. Suggest having network structure tested on all dataset used in the study to conclude the best number of n (size of convolutional kernel).
2. Provide proper citation in Table 3 and Table 5 for compared existing work.
3. Justify the use of different models for comparison for different datasets.
4. The authors should also compare the DCN-ResNet18 with other variations of ResNet (at least 4-5) to strengthen the quality of the proposed model.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Round 2
Reviewer 1 Report
The idea of using DCN for BCI is promising - but the methods are poorly explained, and the manuscript includes many Figures from other sources. The response did not address critical issues and even raised new ones.
1. The first sentence is still a poor definition of BCI.
2. Figure 1 and 4 are the same as Figure 1 and 2 in Dai et al, 2017, but it is not mentioned that the Figures are reproduced with permission. Getting a permission and stating that those Figures are taken from Dai et al, 2007 is a must. As far as I understand Dai et al., 2017 paper is by a different group – not by any of the authors of this paper.
3. Figure 1 and 2: caption should be extended to explain the Figures. In Figure 1: I do not see the 9 points of the Kernel in the middle layer of the DCN, and the connection from the bottom layer to the middle one is not clear.
4. The explanation of the residual structure is poor, and the Figure that was added to explain it (Figure 3) is taken from another paper without any permission.
5. In what sense is the Network similar to ResNet18 – it does have residual connections (shortcuts) – but includes less convolution layers (9 rather than 16).
6. There is no need to specify the size of the convolution and max pooling layers in the text, since they already appear in Figure 5.
7. Section 4.1 includes text and figures from other sources.
1. How many test examples were used? What is the confidence interval of the reported accuracy?
