Advanced Analysis of Electroretinograms Based on Wavelet Scalogram Processing
Round 1
Reviewer 1 Report
Dear authors,
Thank you for submitting your article on the usage of wavelet based feature to classify electroretinograms in a pediatric group and an adult one.
Although this is an interesting study, your small sample size (for both groups) is insufficient to determine the clinical validity of your experiments. The discriminatory power of such testing is only of statistical and clinical importance if data is significant. Also the methods for feature extraction, selection and classification algorithms should be well explained in order to transform your work to a clinical sound one. To reliably assess the contribution of wavelet transform in the classification process, different methods should be employed. Thus,you will need a larger patient sample to satisfactorily address this issue, as well as improve your study design and used methods in order to determine whether clinically important differnces could have been missed or discovered.
I advice also to extensively review the english and the structure of the manuscript.
If you believe that you can revise or expand this study to robustly demonstrate the power of the usage of wavelet in this topic, I encourage the authors to resubmit this manuscript.
Author Response
Thank you for submitting your article on the usage of wavelet based feature to classify electroretinograms in a pediatric group and an adult one.
Although this is an interesting study, your small sample size (for both groups) is insufficient to determine the clinical validity of your experiments. The discriminatory power of such testing is only of statistical and clinical importance if data is significant. Also the methods for feature extraction, selection and classification algorithms should be well explained in order to transform your work to a clinical sound one. To reliably assess the contribution of wavelet transform in the classification process, different methods should be employed. Thus, you will need a larger patient sample to satisfactorily address this issue, as well as improve your study design and used methods in order to determine whether clinically important differnces could have been missed or discovered.
I advice also to extensively review the english and the structure of the manuscript.
If you believe that you can revise or expand this study to robustly demonstrate the power of the usage of wavelet in this topic, I encourage the authors to resubmit this manuscript.
Answer: Thank you for your comment. The current paper present initial study of the research for the original database (see the ref. 19). We fully understand the need of exceed the database.
The next step will be to validate results presented in the paper on new bigger database that we are working on. This database includes 1974 electroretinogram signals.
Also we have improved the description of the feature extraction and classification algorithms description in section 3.
The novelty of the current study relates to the processing of pediatric signals, which is as we know has limited description in the scientific literature. Additionally, we propose an approach to wavelet feature extraction that improves the accuracy of both adult and pediatric diagnosis classification.
The limitations of the work: The work provides initial study results on a rather limited database. In the next step, the database will be assumed to be exceeding.
Reviewer 2 Report
The authors implemented an extended analysis of the ERG signals and conduct a classification task over adult and pediatric group in this study. The results demonstrate the potential use of ERG in diagnostic task. The introduction is developed well. However, the rest of the paper still needs some improvements:
1. It’s difficult for the readers to understand the pipeline by purely reading the text in the introduction and methods. Please add a schematic figure to demonstrate the pre-processing of the data and the downstreaming feature ranking and classification. Specifically, please show one example of the masks generated from the preprocessing of the scalogram.
2. In Figure 3, please make the font on x and y-axis large for better visualization.
3. The result section before 5.1 seems very messy in term of writing. This need to be improved.
4. In the result section, please specify how many features were selected after the feature selection by DT.
5. Please clarity the “feature importance” used in ranking the feature. Is that an impurity-based feature importance or a permutation feature importance? And please define the one used in the paper.
6. Please add a horizontal bar plot demonstrating the feature ranking.
7. Please discuss the potential disadvantages using feature importance on the training set instead of an unseen held-out test set.
8. In Table 3, please merge the Mean and STD into same line using +- sign for better visualization. I would also suggest that to separate the adult and pediatric result into two tables. It’s difficult for readers to relate the text description in 5.1 and 5.2 otherwise.
9. Please split the discussion and conclusion section, and extensively and constructively elaborate both sections, including limitations and future works. Specifically, since ERG is time-series signal, I recommend to add a few references/discussion related to graph representation (visibility graph etc.) of EEG/other signals in the application of a classification task (such as sleep scoring).
Author Response
The authors implemented an extended analysis of the ERG signals and conduct a classification task over adult and pediatric group in this study. The results demonstrate the potential use of ERG in diagnostic task. The introduction is developed well. However, the rest of the paper still needs some improvements:
Answer: Thank you for your comment
- It’s difficult for the readers to understand the pipeline by purely reading the text in the introduction and methods. Please add a schematic figure to demonstrate the pre-processing of the data and the downstreaming feature ranking and classification. Specifically, please show one example of the masks generated from the preprocessing of the scalogram.
Answer: Thank you for your comment, Illustrations are added in the new Fig.3.
- In Figure 3, please make the font on x and y-axis large for better visualization.
Answer: Thank you for your comment, Figure font is corrected, now it is fig.5.
- The result section before 5.1 seems very messy in term of writing. This need to be improved.
Answer: Thank you for your comment, The result section is improved by reorganizing and moving part corresponding to methods in section 3.
- In the result section, please specify how many features were selected after the feature selection by DT.
Answer: Thank you for your comment, The number of selected features is added for each case.
- Please clarity the “feature importance” used in ranking the feature. Is that an impurity-based feature importance or a permutation feature importance? And please define the one used in the paper.
Answer: Thank you for your comment, In the section 5 we have clarified that the impurity-based features importance is used in the paper.
- Please add a horizontal bar plot demonstrating the feature ranking.
Answer: Thank you for your comment, here we considering all features with non-zero impurity. The more in depth feature analysis is required to have the extended database which is going to be the next of our research.
The novelty of the current study relates to the processing of pediatric signals, which is as we know has limited description in the scientific literature. Additionally, we propose an approach to wavelet feature extraction that improves the accuracy of both adult and pediatric diagnosis classification.
The limitations of the work: The work provides initial study results on a rather limited database. In the next step, the database will be assumed to be exceeding.
- Please discuss the potential disadvantages using feature importance on the training set instead of an unseen held-out test set.
Answer: The feature importance was determined during the training stage. To provide unbiased estimation of the impurity of features the k-fold cross-validation was applied.
- In Table 3, please merge the Mean and STD into same line using +- sign for better visualization. I would also suggest that to separate the adult and pediatric result into two tables. It’s difficult for readers to relate the text description in 5.1 and 5.2 otherwise.
Answer: Thank you for your comment, the table columns change.
- Please split the discussion and conclusion section, and extensively and constructively elaborate both sections, including limitations and future works. Specifically, since ERG is time-series signal, I recommend to add a few references/discussion related to graph representation (visibility graph etc.) of EEG/other signals in the application of a classification task (such as sleep scoring).
Answer: Thank you for your comment. The section is renamed to Conclusion which summarize the results obtained in the paper. The current work is limited for only the wavelet based feature extraction applied to the machine learning. Probably the visibility graph and other graph representation methods will be analyzed in the feature. If you can recommend us some particular references you think we must include, please send them to our know. Also the novelty and limitations description is added at the end of the introduction section.
Reviewer 3 Report
1: English should be strongly improved..
2: The abstract should be summarized, and there should be two lines of the description of the framework of the method.
3: It was not a clear explanation of the method. Authors should explain the method step by step with the image of each step.
4: The novelty of this method is limited as it is quite common to leverage such models. What is the novelty of the paper use of previous technique, please explain the reason for the steps? The main weakness of the manuscript is that it is not clear which real improvement is possible with the method proposed by authors. Authors do not give one real reason (experiment, computational time, summary table, etc.) why their method should be preferred.
5: The method section should be re-organized. The motivation of your method should be strengthened as well.
6:Please report the implementation details of the methods for good reproducibility.
7:The results are not compared with the recent published papers and so it helps authors to update the references as well.
8:In the introduction, the authors did not provide a strong motivation of the paper and the obtained results. In addition, they should discuss the main contributions of their work in details after the motivation part. Then they should summarize the main structure of their paper in brief at the end of the introduction.
9:The authors did not discuss the advantages of the proposed method over the other existing results.
10:In my opinion, the conclusion part of this paper should be modified. It is better to add some summaries different from the abstract to reflect the innovation and contribution of the manuscript. The conclusion can also add some follow-up research content to enhance the continuity of the research in this aspect.
11: The authors did not perform any contrast or illumination adjustment technique as a pre-processing as the images are collected from different cameras with various parameters settings.
12: The authors should state in the paper what kind of the software package has used to obtain the required results.
13:Complexity: Please give some theoretical analysis in the time complexity of the current paper.
14:In addition, authors should provide the comments of the cited papers after introducing each relevant work. What readers require is, by convinced literature review, to understand the clear thinking/consideration why the proposed approach can reach more convinced results. This is the very contribution from authors. In addition, authors also should provide more sufficient critical literature review to indicate the drawbacks of existed approaches, then, well define the main stream of research direction, how did those previous studies perform? Employ which methodologies? Which problem still requires to be solved? Why is the proposed approach suitable to be used to solve the critical problem? We need more convinced literature reviews to indicate clearly the state-of-the-art development.
15:When we are talking about performance it is important to mention the computing platform on which it is measured. Besides, it is important to state if all methods were measured on the same platform.
Author Response
1: English should be strongly improved.
Answer: Thank you for your comment. The grammar is improved.
2: The abstract should be summarized, and there should be two lines of the description of the framework of the method.
Answer: Thank you for your comment. The abstract part is rewritten.
3: It was not a clear explanation of the method. Authors should explain the method step by step with the image of each step.
Answer: Thank you for your comment. The illustrations are added in the fig.3.
4: The novelty of this method is limited as it is quite common to leverage such models. What is the novelty of the paper use of previous technique, please explain the reason for the steps? The main weakness of the manuscript is that it is not clear which real improvement is possible with the method proposed by authors. Authors do not give one real reason (experiment, computational time, summary table, etc.) why their method should be preferred.
Answer: The novelty of the current study relates to the processing of pediatric signals, which is as we know has limited description in the scientific literature. Additionally, we propose an approach to wavelet feature extraction that improves the accuracy of both adult and pediatric diagnosis classification.
The limitations of the work: The work provides initial study results on a rather limited database. In the next step, the database will be assumed to be exceeding.
The paper includes the comparison of the current existed approach using 4 features and our one which extend that results leading to the classification accuracy improvement.
5: The method section should be re-organized. The motivation of your method should be strengthened as well.
Answer: To the section 3. Materials and methods we have added the step-by-step visualization for the wavelet scalogram preprocessing, which includes initial scalogram, binarized version, binary version without local artifacts, as well as connected components. Also we have added the pipeline of data preprocessing: we use our database to get signals and well as diagnosis-related information. The signals are processed by classical approach and wavelet-transform. After that we solve the classification task by decision trees methods by testing possibility of different features to predict target – diagnosis.
6:Please report the implementation details of the methods for good reproducibility.
Answer: To the sections 3 we have added citation to the Python libraries used to obtained results, namely PyWT, OpenCV and scikit-learn
7:The results are not compared with the recent published papers and so it helps authors to update the references as well.
Answer: Thank you for your comment. As we mentioned above, we make comparison with exited method. The more wide comparison of the suggested results with other recently published techniques of ERG signal preprocessing will be done in the feature work as we are noticed in the conclusion.
8:In the introduction, the authors did not provide a strong motivation of the paper and the obtained results. In addition, they should discuss the main contributions of their work in details after the motivation part. Then they should summarize the main structure of their paper in brief at the end of the introduction.
Answer: The new introduction includes the following changes. The strong motivation to understand in more detail the ERG signals start in a first glance with building a signal’s database which includes adult and children. In this way, waveforms can be analyzed in terms of physiological process and compared according to waveform morphology.
9:The authors did not discuss the advantages of the proposed method over the other existing results.
Answer: Thank you for your comment. The answer has been given in previous issues
10:In my opinion, the conclusion part of this paper should be modified. It is better to add some summaries different from the abstract to reflect the innovation and contribution of the manuscript. The conclusion can also add some follow-up research content to enhance the continuity of the research in this aspect.
Answer: Thank you for your comment. The conclusion was rewritten.
11: The authors did not perform any contrast or illumination adjustment technique as a pre-processing as the images are collected from different cameras with various parameters settings.
Answer: Thank you for your comment. The images are obtained by digital signal processing methods and by “different cameras”. The original signals are obtained using specialized measurement system.
12: The authors should state in the paper what kind of the software package has used to obtain the required results.
Answer: Thank you for your comment. To the sections 3 we have added citation to the Python libraries used to obtained results, namely PyWT, OpenCV and scikit-learn
13:Complexity: Please give some theoretical analysis in the time complexity of the current paper.
Answer: Thank you for your comment. The analyzed task does not assume high computation efficiency rather than the accuracy of the finial classification that was the aim of our research.
14:In addition, authors should provide the comments of the cited papers after introducing each relevant work. What readers require is, by convinced literature review, to understand the clear thinking/consideration why the proposed approach can reach more convinced results. This is the very contribution from authors. In addition, authors also should provide more sufficient critical literature review to indicate the drawbacks of existed approaches, then, well define the main stream of research direction, how did those previous studies perform? Employ which methodologies? Which problem still requires to be solved? Why is the proposed approach suitable to be used to solve the critical problem? We need more convinced literature reviews to indicate clearly the state-of-the-art development.
Answer: Thank you for your comment.
The authors of article [1] described all the possible methods for investigating ERG signals that are found in the scientific literature including comparative analysis of the methods, their advantages, and disadvantages. There is no need for a more detailed description, since the Introduction section gives the motivation for this study, and the Related Work contains articles related to the ERG signal processing method used in the study.
[1] Behbahani, S., Ahmadieh, H., & Rajan, S. (2021). Feature Extraction Methods for Electroretinogram Signal Analysis: A Review. IEEE Access.
15:When we are talking about performance it is important to mention the computing platform on which it is measured. Besides, it is important to state if all methods were measured on the same platform.
Answer: Thank you for your comment. The analyzed task does not assume high computation efficiency rather than the accuracy of the finial classification that was the aim of our research.
Round 2
Reviewer 2 Report
The authors have addressed all my comments.
Author Response
Thank you
Reviewer 3 Report
Please read my all-previous comments carefully and explain them in your manuscript as well. You just give me the response but didn't update in the manuscript.
Author Response
Thank you for review, concerning your comments the following modifications has been done with the new version of the paper .
4: The novelty of this method is limited as it is quite common to leverage such models. What is the novelty of the paper use of previous technique, please explain the reason for the steps? The main weakness of the manuscript is that it is not clear which real improvement is possible with the method proposed by authors. Authors do not give one real reason (experiment, computational time, summary table, etc.) why their method should be preferred.
Answer:
The following text was added to the End of the Introduction Section
The novelty of the current study relates to the processing of pediatric signals, which is as we know has limited description in the scientific literature. Additionally, we propose an approach to wavelet feature extraction that improves the accuracy of both adult and pediatric diagnosis classification.
The limitations of the work: The work provides initial study results on a rather limited database. In the next step, the database will be assumed to be exceeding.
The paper includes the comparison of the current existed approach using 4 features and our one which extend that results leading to the classification accuracy improvement.
7:The results are not compared with the recent published papers and so it helps authors to update the references as well.
Answer: Thank you for your comment. As we mentioned above, we make comparison with exited method.
The following text was added to the Conclusion Section
The more wide comparison of the suggested results with other recently published techniques of ERG signal preprocessing will be done in the feature work as we are noticed in the conclusion.
8:In the introduction, the authors did not provide a strong motivation of the paper and the obtained results. In addition, they should discuss the main contributions of their work in details after the motivation part. Then they should summarize the main structure of their paper in brief at the end of the introduction.
Answer:
The following text was added to the Introduction Section
The strong motivation to understand in more detail the ERG signals start in a first glance with building a signal’s database which includes adult and children. In this way, waveforms can be analyzed in terms of physiological process and compared according to waveform morphology.
Also in the end of the Introduction we have added the brief structure of the paper
- The authors did not discuss the advantages of the proposed method over the other existing results.
Answer: Thank you for your comment.
The next text is added to the New Discussion section
The proposed method is based on the use of a continuous wavelet transform. Compared with Fast Fourier Transform, Linear Prediction, Windowed Fourier Transform, and other common methods, continuous wavelet transform provides variable window size, high resolution at low and high frequencies, the ability to obtain detailed information during rapid frequency changes and is also suitable for extracting features from non-stationary signals.
Varadharajan [1] compared the results of the analysis in the time domain and the continuous wavelet transform in determining the ERG parameters. The continuous wavelet transform method was more accurate than time domain analysis methods and confirmed that it can differentiate between normal and abnormal ERG, especially in the early diagnosis of glaucoma.
A research group led by Penkala [2] focused on extracting information about the time-frequency characteristics of ERG a- and b-waves using wavelet analysis. According to their results, low-frequency components prevailed (both in a-waves and b-waves), and their temporal distribution depended on the brightness of the light pulses.
Barraco [3] used wavelet analysis to extract the characteristics of the ERG a-wave. The work was focused on diagnosing two pathologies: achromatopsia and congenital stationary night blindness. The results showed that the number of dominant frequencies and the time of their occurrence in both studied diseases can reflect the state of retinal photoreceptors. A comparison of individual pathological cases with a healthy control group showed that both components of the incidence of the disease are shifted toward lower frequencies. Other studies [4-8] also point to the use of wavelet analysis. It should be noted that wavelet analysis is used directly in the above studies. In other words, there is no extraction of parameters from wavelet scalograms.
- Varadharajan, S. A novel method for separating the components of the clinical electroretinogram / S. Varadharajan, K. Fitzgerald, V. Lakshminarayanan // Journal of Modern Optics. – 2007. – Vol. 54, no. 9. – P. 1263-1280.
- Penkala, K. Improvement of the PERG parameters measurement accuracy in the continuous wavelet transform coefficients domain / K. Penkala, M. Jaskuła, W. Lubiński // Annales Academiae Medicae Stetinensis. – 2007. – Vol. 53. – P. 58-60.
- Barraco, R. Wavelet analysis of human photoreceptoral response / R. Barraco, D. P. Adorno, M. Brai // 2010 3rd International Symposium on Applied Sciences in Biomedical and Communication Technologies (ISABEL 2010). – IEEE. 2010. – c. 1-4.
- Multifocal electroretinography, glaucoma diagnosis by means of the wavelet transform / J. M. M. Jiménez [и др.] // 2008 Canadian Conference on Electrical and Computer Engineering. – IEEE. 2008. – c. 000867-000870.
- Glaucoma detection by wavelet-based analysis of the global flash multifocal electroretinogram / J. M. Miguel-Jiménez [et al.] // Medical engineering & physics. – 2010. – Vol. 32, no. 6. – P. 617-622.
- Multifocal ERG wavelet packet decomposition applied to glaucoma diagnosis / J. M. Miguel-Jiménez [et al.] // Biomedical engineering online. – 2011. – Vol. 10, no. 1. – P. 1-13.
- Barraco, R. An approach based on wavelet analysis for feature extraction in the a-wave of the electroretinogram [текст] / R. Barraco, 8. D. P. Adorno, M. Brai // Computer methods and programs in biomedicine. – 2011. – Vol. 104, no. 3. – P. 316-324.
Barraco, R. ERG signal analysis using wavelet transform / R. Barraco, D. Persano Adorno, M. Brai // Theory in Biosciences. – 2011. – Vol. 130, no. 3. – P. 155-163.
11: The authors did not perform any contrast or illumination adjustment technique as a pre-processing as the images are collected from different cameras with various parameters settings.
Answer: Thank you for your comment. The images are obtained by digital signal processing methods and by “different cameras”. The original signals are obtained using specialized measurement sensor system.
The following text was added to the Section Materials and methods , subsection Database
All signals were recorded at the same place and using the very same data acquisition protocols.
13:Complexity: Please give some theoretical analysis in the time complexity of the current paper.
Answer: Thank you for your comment. The analyzed task does not assume high computation efficiency rather than the accuracy of the finial classification that was the aim of our research.
This comment was added to the Section Materials and methods , subsection Machine learning pipeline
14:In addition, authors should provide the comments of the cited papers after introducing each relevant work. What readers require is, by convinced literature review, to understand the clear thinking/consideration why the proposed approach can reach more convinced results. This is the very contribution from authors. In addition, authors also should provide more sufficient critical literature review to indicate the drawbacks of existed approaches, then, well define the main stream of research direction, how did those previous studies perform? Employ which methodologies? Which problem still requires to be solved? Why is the proposed approach suitable to be used to solve the critical problem? We need more convinced literature reviews to indicate clearly the state-of-the-art development.
Answer: Thank you for your comment.
The authors of article [1] described all the possible methods for investigating ERG signals that are found in the scientific literature including comparative analysis of the methods, their advantages, and disadvantages. There is no need for a more detailed description, since the Introduction section gives the motivation for this study, and the Related Work contains articles related to the ERG signal processing method used in the study.
[1] Behbahani, S., Ahmadieh, H., & Rajan, S. (2021). Feature Extraction Methods for Electroretinogram Signal Analysis: A Review. IEEE Access.
The citation to the mentioned article, as well as its content was added to the section Related Work
15:When we are talking about performance it is important to mention the computing platform on which it is measured. Besides, it is important to state if all methods were measured on the same platform.
Answer: Thank you for your comment. The analyzed task does not assume high computation efficiency rather than the accuracy of the finial classification that was the aim of our research.
This comment was added to the Section Materials and methods , subsection Machine learning pipeline
