A Peptides Prediction Methodology with Fragments and CNN for Tertiary Structure Based on GRSA2

Round 1

Reviewer 1 Report

The authors present a broad review of computational tools used in the prediction of single molecular structures of small proteins (under 49 amino acids in length). They compare their approach to other leading packages used in protein structure prediction. I feel that while the overall analysis does not support their contention that GRSA2-FCNN represents a new state-of-the-art in small protein structure prediction it does represent an improvement in some respects compared to currently available algorithms. If the authors wish to stand by their statement significant additional analysis will be necessary. (As well as a discussion on how accurate their declared improvement can be based on a dataset of 41 peptides).

However, as the paper does show an alternative mechanism to reach broadly similar results to that of I-TASSER/AlphaFold/etc. it does warrant publication.

Changes requested.

1. Please have the manuscript proof-read/corrected by a native speaker.

2. The authors refer to a single native structure - which is unlikely given the limited size of these peptides, which I would consider to be highly likely to be very mobile in solution. The authors should strongly motivate why the pdb structure used as the central comparison (and indicate those that are generated under solution parameters very distinct from those to be expected in vivo (ie.  high precipitant, extreme pH etc).

3. The authors should consider using NMR-generated structures, which will provide a broader description of the mobility of peptides and - while making comparisons more complex - may improve the comparison.

4. Figure 4 needs to be reorientated to more closely view the differences between the red & green structures

5. Figure 5 is of questionable value, as it represent the 5 best predictions. How do AlphaFold2/I-TASSER/etc compare? This figure is potentially redundant as the same information is broadly shown in Figure 9?

6. In Figure 6 and the associated discussion. The authors combine the -FCNN and -SSP predictions into one group. This approach needs to be justified. Is there any metric that can be used to select which of -FCNN and -SSP are proposed as representing the GRSA2 approach? If not it seems they have had "two attempts", whereas the competing software only had one? Please elaborate on why such an approach is valid.

7. In Figure 7 and associated discussion -FCNN and -SSP are now treated separately? See comment above. This subdivision (16-30aa) is the only one in which GRSA outperformed the competition (by 11 to 10).

8. In Figure 8 the performance difference between GRSA2-FCNN and other approaches is rather stark. I would suggest the authors perform a secondary analysis on the scores used for ranking. While the best performing alogrithms can be identified by high TS/GDT-TS a more useful comparison to place GRSA2 approaches in context would be the difference between the GRSA2 outcome scores and the "winning" algorithm. While cases where many/most algorithms perform well are interesting, the cases where there are significant differences in performance deserve highlighting.

8. Figure 9 should include all algorithms compared. It is also not clear from this figure why GRSA2-FCNN is the new state-of-the art. A plot of performance over the entire dataset is required to make such a claim.

9. The figures (axis legends) are rather pixelated and need cleaning.

Author Response

Thanks a lot for your valuable revision of our paper. In this new version, we made all the corrections that you kindly send us.
Attached to this submission you will find the answers to all your observations.

Best regards!
The authors.

 

Author Response File: Author Response.pdf

Reviewer 2 Report

Authors presented a new method to predict tertiary structure of the small proteins and peptides benchmarked on peptides with the size up to 49 amino-acid residues. As the method provided mixed accuracy of the predictions, authors should extend the testing set of peptides/proteins, improve analysis and compare to at least one more method to clearly illustrate advantages of their approach.



Comments:

• Authors should clearly explain how the NMR ensembles were handled during the comparison of the predicted models.
• Authors should explain why there is a limit of the size of the proteins/peptides which structures can be predicted and if they plan to extend it.
• Plots of average RMSD, RG and secondary structure of predicted models to experimental structures should be added to the manuscript.

• As the method is on average not better than existing tools, authors should describe other unique properties, such as computational time needed to obtain models and add the comparison to at least one more modern tool designed to predict structure of peptides, such as APPTEST.
• To better assess the performance of the method more alpha, beta and alpha+beta systems should be added for the comparison, some examples being: 1E0L, 1ED7, 1RES, 2HEP, 1QHK, 1E0G.
• Is the method providing estimations of the model (fragment) quality?
• Is it possible to use secondary structure (or any other data) as input to enhance quality of the results?

Author Response

Thanks a lot for your valuable revision of our paper. In this new version, we made all the corrections that you kindly send us.
Attached to this submission you will find the answers to all your observations.

Best regards!
The authors.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Many thanks to the authors for providing the updated draft. I am convinced by their responses and the changes made to the text.

However, I'm greatly surprised to see that although the authors claim to have improved the quality of the figures, figure legends for Fig 6-11 remain pixelated and grammatical highlights from powerpoint (?) are visible in Figure 2. Please address these prior to publication.

Secondly, I would again urge the authors to have a native speaker look through the manuscript as the changes made to the abstract (and elsewhere) need grammatical correction.

Author Response

Dear reviewer:

Thank you again for your observations. The remarks you suggested were important to improve the quality of our paper and were very valuable to us.

In this second version of our paper, we have focused on improving the quality of the images so that they do not look pixelated, we hope they look better in this version.

We include now in the cover letter the answers to your observations.

Specifically, we consulted a professional who works in the communication area and is a British speaker who revised the paper and helped to improve its quality.

Thanks again for your comments,

Sincerely

The authors

Author Response File: Author Response.pdf

Reviewer 2 Report

Authors significantly improved the manuscript, however, some details still remain fuzzy, e.g.:

- 2HEP has 42 aa residues, not 85 in the single structure from the pdb; how do authors obtain an 85-residue model?

- 1E0G is a mix of alpha and beta, so why in table 1 its secondary structure is marked as "none"?

- why did the authors select such a non-standard definition of RMSD? In typical definition RMSD depends only on the square of the differences in atom positions after optimal superposition and does not include Rg; what is the advantage of using a more complicated formula of RMSD?

- unit of RMSD should be showed at figures

- authors still does not analyze carefully the erratic results of their method for the tested systems - while some peptides and proteins are predicted with very good accuracy, some others are predicted with much lower quality (e.g. in 30-40 aa residue range), which does not seem to be connected strictly to their size and secondary structure

- authors should show how well their method predicted secondary structure elements comparing to other methods (e.g. showing the correlation between helices, sheets, turns and coils in predicted and experimental models)

- authors should correct language of the manuscript before submission

 

Author Response

Dear reviewer:

Thank you again for your observations. The remarks you suggested were important to improve the quality of our paper and were very valuable to us.

In this second version of our paper, we have focused on improving the comparison of the proposed method, adding a new analysis with statistical tests of our results, and evaluating why it is better in some groups and worst in others.

We include in the cover letter the answers to your observations. Specifically, we consulted a professional who works in the communication area and is a British speaker who revised the paper and helped to improve its quality.

Thanks again for your comments,

Sincerely

The authors

Author Response File: Author Response.pdf

Round 3

Reviewer 2 Report

Authors improved the manuscript, however, one issue still remains unanswered regarding the metrics used to evaluate the methods, which strongly impact evaluation of the results. I agree that RMSD is a commonly used metric, but that is not the point. The point is that RMSD is defined in the references provided by authors (AlphaFold2 and TM-Score) differently that in the manuscript under revision (equation 3) - most important difference is that standard RMSD definition does not include radius of gyration:

"The most commonly used metric in this category is the root-mean-square deviation, RMSD, in which the root-mean-square distance between corresponding residues is calculated after an optimal rotation of one structure to another ( 11 )."

Authors should clearly state if they used the eq 3 to calculate RMSD or the standard RMSD definition (please take a look at e.g. wikipedia equation 1: https://en.wikipedia.org/wiki/Root-mean-square_deviation_of_atomic_positions), because the results between these two metrics would be completely different. If authors used equation 3 to calculate the metrics, I strongly suggest changing its name to not confuse readers.

 

Authors should expand the description of Figure 13 to explain what it shows exactly and provide what property is shown at the y-axis.

 

Author Response

Dear reviewer:

Thank you for your valuable observations they were too important for us to improve the quality of our paper.

In this new version of our manuscript, we modified it with the observations you made.

Thanks again,

Sincerely

The authors

Author Response File: Author Response.pdf