PRRGNVis: Multi-Level Visual Analysis of Comparison for Predicted Results of Recurrent Geometric Network

Round 1

Reviewer 1 Report

SUMMARY. This article proposes a method of multi-level visual analysis to improve the protein structure comparison between predicted and actual structures. The main point is that this method uses Artificial Neural Networks (ANNs), particularly Recurrent Geometric Networks (RGNs). As case studies, the paper presents two applications of the proposed method.

 

GENERAL COMMENTS. The topic looks interesting, and the paper is not so complex to follow [See minor comments]. My main concern is related to the paper structure and how the case studies are analyzed and presented.

  * The structure of the paper lacks the methodology or plan of how this research was carried out. For example, although the paper structure is simple (related work, method, demonstration, and conclusion), no a roadmap of the paper allows the readers to understand WHY the authors follow this structure. Even it is not clear the rationale process used to define the method. A new section 3.1 that explains the process of building the method would probably be welcome.

 

  * The reader commonly expects that the case studies try to highlight an aspect of the method. In the paper, there is no deeper discussion about the case study selection. In addition, a final subsection would be recommended that summarizes and compares the results obtained from the case studies.

 

 

MINOR COMMENTS.

- An English and writing edition is required. It is easy to read typos and mistakes in the documents, e.g., no space between words parenthesis, wrong capital letter, style for titles in (sub)sections, etc.

- I would recommend naming the proposed method because it is easier to identify when the paper is speaking the proposal.

- Figure 1 is too small.

- Related to GENERAL COMMENTS, it might be a good idea to introduce a "Methodology" Section.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

This paper describes a method to improve the accuracy of protein structure predictions using the Recurrent Geometric Network.  The authors propose three levels of evaluation.  First, they propose using three methods (GDT_TS, dRMSD, and TM-Score) to evaluate the predictions. They propose using distance deviation and torsion angle to compare the differences between the predicted structure and actual structure. The last step of the proposed process is to use the Ramachandran plot to assess the stability of the predicted structure based on locations of residues in allowed and disallowed regions.

 

Overall, the approach proposed by the authors is interesting and has potential merit. However, the quality of the prepared manuscript is so poor that it is impossible to fully evaluate the proposed methods.  Therefore, the manuscript will require significant revision before it will be possible to determine if the proposed approach is valid.

 

Major Critiques:

 

1.     There are several places, such as the second paragraph of the Introduction, where the authors provide a laundry list of authors and approaches that they used but there is no context for why this is relevant to the current work. The paragraph reads like a table in text form without any connection to the rest of the paper.

 

2.     The text in the majority of the figures is too small to read so there is no way to know if the claims the authors are making is valid. In addition, there seems to be little connection between the text and the figures in some cases.  For example, what is the relevance of Figure 1.  From the text it seems like an overview of the method but it is hard to know this for sure from the figure legend.

 

3.     There are major missing sections of the paper.  There is no Materials and Methods section, so it is not possible to evaluate the approaches used.  There is no Results or Discussion section but instead there is a “Related Work” section.  It is impossible to know where the summary of related work ends and where the experimental work starts.  The manuscript needs to be reformatted into a more traditional format so it is clearer what was done, how it was done and what it means.

 

4.     Most of the approaches were hard to evaluate due to the formatting of the paper but it was possible to evaluate the “Structural Stability” section.  I am not convinced that the approach that is being used really determines stability.  There are residues in experimentally determined structures that sit in unallowable regions, and these are stabilized by other interactions within the protein.  Using this as a criterion for stability does not take compensatory interactions into account. A more careful discussion of why this assumption is valid is necessary to convince me that this is really a measure of stability.

 

5.     The author claim to have demonstrated the validity of their approach (Section 4) without providing any figures, tables or data to support this validation.  It is not possible to determine if this validation really occurred without any data regarding the tests that were done using the CASP dataset.

 

6.     The authors make the claim in the conclusions that this approach can be used with other predictive network results, but they do not provide any evidence to support this claim.  Results from at least one other tool is necessary to show the accuracy of this statement.

 

Minor Critiques:

1.     There are a number of places where abbreviations/acronyms are used but not defined.  For example, on line 29, RGN is introduced but not defined.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 3 Report

This manuscript describes a comparative analysis of the protein structure prediction to the actual structure. This would help to identify and measure the similarities between any two proteins for further studies such as an evolutionary relationship and drug design. There are a few points that should be clearly addressed and given more discussion. Some typos and misused upper-case letters throughout the manuscript should be checked and corrected.
1. The title of the manuscript does not clearly state the main field of study (protein structure prediction/comparison). RGN should be written in full words.  Maybe something like 'Multi-level visual analysis of protein structure prediction by the recurrent geometric network' or others. Please clarify this point and propose a new title.

2.  The author should clearly state which measures are a commonly used evaluation between the prediction results and the actual structure. How good is the developed measure? The discussion of the pros/cons of the original measure and the new one should be stated.

3. Please make a clearer picture for Figure 1. The bigger size of the words and letters in Figure would be desired.

4. Please provide the source code or program if possible. Then, the others can then use your measure in the same way.

5. Some grammar, word choices, and word orders should be checked.

6. Some misused uppercase or lowercase letters, for example:

- line 1: 'Machine Learning' should be 'machine learning'.

- line 4: 'Multi-level' should be 'multi-level'.

- Figure 1 caption: 'the Mixed Line and Bar chart' should be 'the mixed line and bar chart'. 'at the structural difference level' should be 'At the structural difference level.

- line 48: 'Secondly, At' should be 'Secondly, at'.

- line 50: 'Finally, For' should be 'Finally, for'.

- line 55: 'Firstly' should be 'firstly'.

- line 96: 'Deep learning' should be 'deep learning'.

- line 140, 150, 155: 'Where' should be 'where'. This 'where' is an explanation after an equation. Therefore it should be in the same sentence as the equation. Actually, after completing an equation, we should put ',' after the equation and then a new line starting with 'where'. Please correct this point.

- the word 'Ramachandran plot' should be written the same throughout the manuscript. (Plot or plot?)

- please check if the starting letters of words should be uppercase or lowercase throughout of the manuscript.

 

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

The author team has improved the paper to address the comment from the first review. 

I am still recommending to create an isolated section "Methodology," which clearly specifies the goal of the paper (expected results and comparisons) and the steps used to carry out the process.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

This is a significant improvement over the original submission but still required additional editing.  The edits provide a more clearer roadmap to follow in the organization of the paper and what is previous work and what is new in this work.  That said, this still doesn't adhere to the "Instructions for Authors" published by the journal and it is not clear why that is that case..  There are also significant grammatical and formatting errors throughout the paper.  A subset of these include.

 

RGN should be defined in the main body of the manuscript and not just in the abstract.

 

There should be comma on line 32

 

Figure 1 still hard to read and fonts are way too small.  Some of the structures are very faint and not easy to read.  Finally, this figure is not referenced in the main body of the manuscript at all and should either be deleted or be referred to in the manuscript.

 

Line 108 – “For” should not be capitalized.

 

Line 134-135 – not correct grammer.

 

There are a couple words misspelled in Figure 2 and it should be placed in the text after it is referred to and not before.  This is true of several other figures as well.

 

The abbreviation "FM" is used on line 131 before it is defined on line 143.

 

Line 285 – should be Table 1

 

Table 1 is not very clear.  For example, is the 5.32 Å for the dRMSD on the TBM line referring to the average value <3.2 for the 10% of similarity standards?  It is really hard to interpret the table and the validity of the author’s conclusions as written.

 

Line 347 - There is a reference made to Figure 17b when the data being referred to is in Figure 17a. 

 

There needs to be a clearer rational for why T0315 and T0324 don’t follow rules.  Could be there on lines 348-349 but not defined as that is what is being discussed.

 

This is a small sample of the grammatical errors and unclear writing throughout the manuscript and there are too many to list them all.  A significant proofreading is necessary to bring the language up to publication standards.

 

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.