Estimating the Frequencies of Maximal Theta-Gamma Coupling in EEG during the N-Back Task: Sensitivity to Methodology and Temporal Instability

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

This paper explores the pre-processing effects on theta-gamma coupling reproducibility. Several pipeline variants are created that are used on benchmark LFP animal data as well as human EEG recordings. Filtering methods, epoch selection strategy, PAC measure, etc were varied to study their effect. I think this is a very important paper; many researchers use processing steps as black box units without carefully considering the potential effect of the algorithm selection or the algorithm parameters. For this reason, I welcome this paper and encourage the authors to continue their investigation. 

The paper is well written, the structure is clear and understandable, language usage and style is good, I have not found mistakes or typos.

Couple of comments and/or suggestions that perhaps should be addressed in the final version of the paper:

- It seems that the eegfiltnew filtering method performs more poorly than the original filter or the filtfilt Butterworth. Could the authors explain why this could be? eegfiltnew is a linear phase filter with no phase distortion

- Did the authors considered testing the CleanLine line noise removal tool as well (https://github.com/sccn/cleanline) that removes 50 or 60Hz noise without notch filtering and distorting the spectrum?

- What is the reason that normalisation produces poorer reproducibility results?

- The Discussion could be improved by analysing the reasons why these differences among algorithms arise and providing recommendations which method should or should not be used in TGC calculations.

 

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 2 Report

Comments and Suggestions for Authors

The authors explored several options at different steps of the calculation, applying the resulting algorithms to EEG data of 16 healthy subjects performing the n-back working memory task, as well as a benchmark recording with previously reported strong PAC. By comparing the results for the two halves of each session, The authors estimated reproducibility at the time scale of a few minutes. For the benchmark data, the results were largely similar between the algorithms and stable in time. However, for the EEG, the results depended substantially on the algorithm, while also showing poor reproducibility, challenging the validity of using them for personalizing brain stimulation. Further research is needed on the PAC estimation algorithms, cognitive tasks, and other aspects to reliably determine and effectively use TGC parameters in neuromodulation.

This is an interesting study, but there are still some issues that need to be addressed before publication.

1 To verify the generalization of the model, authors should use more datasets.

2 Authors should compare more recently published methods as baseline methods.

 

3 Authors should discuss more EEG related work in the introduction, such as [1-3].

[1] Multi-View Spatial-Temporal Graph Convolutional Networks with Domain Generalization for Sleep Stage Classification. IEEE TNSRE (2021)

 

[2] Multi-modal physiological signals based squeeze-and-excitation network with domain adversarial learning for sleep staging. IEEE Sensors Journal. 2022

 

[3] Hybrid spiking neural network for sleep electroencephalogram signals. Sci. China Inf. Sci. 65(4) (2022)

Comments on the Quality of English Language

Minor editing of English language required

Author Response

We are thankful to the reviewer for the evaluation of the manuscript and valuable comments.

"1 To verify the generalization of the model, authors should use more datasets."
We agree that the generalization of the conclusions increases when they are validated on additional independent data. This is an important direction for future research. We have underlined this in the following sentence, conluding a new paragraph about algorithm recommendations (which we added following a suggestion by Reviewer 1): "That said, in the present sample, the highest ICC estimates were obtained for the ‘eegfiltnew’ algorithm (0.2 and 0.5 for phase and amplitude), making it a promising starting point for further methodological exploration on independent datasets."

"2 Authors should compare more recently published methods as baseline methods."
We thank the reviewer for highlighting the value of method comparison. Taking into account recent developments is also very important. This is why we examined the PAC calculation approaches (incorporating their steps into the compared algorithms) from a number of papers on TGC published in the last five years, including refs. 18 (Malenínská et al., 2021), 21 (Chung et al., 2019), 28 (Caiola et al., 2019), 29 (Combrisson et al., 2020), 35 (Musaeus et al., 2020), 36 (Gong et al., 2021), 39 (Rustamov et al., 2022), 43 (Jones et al., 2020), 45 (Hammer et al., 2021), 47 (Attaheri et al., 2022), 52 (Papaioannou et al., 2022), 54 (Martinez-Cancino et al., 2020), and 64 (Gordon et al., 2022). To highlight the contemporary approaches to EEG analysis even more, we have now mentioned in the discussion section the three recent papers listed in the next suggestion.

"3 Authors should discuss more EEG related work in the introduction, such as [1-3].
[1] Multi-View Spatial-Temporal Graph Convolutional Networks with Domain Generalization for Sleep Stage Classification. IEEE TNSRE (2021)
[2] Multi-modal physiological signals based squeeze-and-excitation network with domain adversarial learning for sleep staging. IEEE Sensors Journal. 2022
[3] Hybrid spiking neural network for sleep electroencephalogram signals. Sci. China Inf. Sci. 65(4) (2022)"

As these papers deal with innovative approaches to EEG analysis, we have mentioned them in the discussion section, in the context of the prospects of developing methods for online TGC estimation:
"To this end, it is important to investigate algorithms for online estimation of the currently dominant PAC frequencies. One of the tools that may help in this regard is deep learning, due to its ability to adaptively extract information from EEG signals [66–68], which may aid in dealing with short data segments."

Reviewer 3 Report

Comments and Suggestions for Authors

W artykule wskazano możliwość wykorzystania specyficznych/zindywidualizowanych częstotliwości maksymalnego sprzężenia theta-gamma (TGC) do regulacji stymulacji mózgu. Poruszona tematyka jest aktualna, a nawet wydaje się konieczna w kontekście dostępności licznych algorytmów szacowania PAC.

Metodologia oraz analizy i podsumowanie przeprowadzonych badań zostały przygotowane bardzo dobrze, logicznie i przejrzyście. Jedynie wzmianki o litaturach starszych niż 10 lat budzą wątpliwości. W dziedzinie neuronauki publikacje mające już pięć lat uznawane są za nierzetelne, jednak literatura sprzed 10 lat budzi poważne wątpliwości w sytuacji bardzo intensywnego rozwoju badań w zakresie poruszanej problematyki.

Author Response

We thank the reviewer for the assessment of the manuscript and appreciation of the importance of its topic. 

"Only mentions of literature older than 10 years are questionable. In the field of neuroscience, publications that are already five years old are considered unreliable, but literature from 10 years ago raises serious doubts in the situation of very intensive development of research in the field of the discussed issues."

We agree that taking into account recent developments is very important. This is why we examined the PAC calculation approaches (incorporating their steps into the compared algorithms) from a number of papers on TGC published in the last five years, including refs. 18 (Malenínská et al., 2021), 21 (Chung et al., 2019), 28 (Caiola et al., 2019), 29 (Combrisson et al., 2020), 35 (Musaeus et al., 2020), 36 (Gong et al., 2021), 39 (Rustamov et al., 2022), 43 (Jones et al., 2020), 45 (Hammer et al., 2021), 47 (Attaheri et al., 2022), 52 (Papaioannou et al., 2022), 54 (Martinez-Cancino et al., 2020), and 64 (Gordon et al., 2022). As regards mentions of older literature, they are explained by the fact that, although the new papers often contain different combinations of PAC calculation steps, the steps themselves remain largely the same. For example, a recent paper [64] (Gordon et al., 2022) employing the cutting-edge technique of closed-loop stimulation relies for TGC calculation on the Mean Vector Length (MVL) modulation index introduced by Canolty et al. back in 2006 [7]. To highlight the contemporary approaches to EEG analysis even more, we have now mentioned in the discussion section the three recent papers suggested by Reviewer 2.

Reviewer 4 Report

Comments and Suggestions for Authors

The paper is well written and presented. The conducted experiments demonstrate a reproducibility problem which is naturally explained by a critically small number of participants. The problem is mitigated with a larger number of participants, that can be addressed in the future research. 

Author Response

We thank the reviewer for evaluating the manuscript and providing thoughtful comments.

"The conducted experiments demonstrate a reproducibility problem which is naturally explained by a critically small number of participants. The problem is mitigated with a larger number of participants, that can be addressed in the future research."

Indeed, the validity of the conclusions can be enhanced by studying larger samples. At the same time, this would lead to increased reproducibility only if we were dealing with group-average theta-gamma coupling parameters. With our focus on individual measurements (necessary for personalizing brain stimulation), there is no a priori tendency for them to have smaller variation between measurements with increased number of subjects, only a tendency towards more accurate estimation of this variation (as described in the last paragraph of the discussion section).

Round 2

Reviewer 2 Report

Comments and Suggestions for Authors

The authors have solved all my problems.

Comments on the Quality of English Language

Minor editing of English language required