Development of a Predictive Tool for the Parametric Analysis of a Turbofan Engine

Round 1

Reviewer 1 Report

This is an interesting study, in which a predictive tool was developed using machine learning techniques to provide parametric analysis to turbofan engines. The topic is of high importance and interest to the general scientific community, and in particular to the aeronautic division.

In particular, the authors conducted a wide range of numerical simulations on a well-established and reliable software (GasTurb14), obtaining a database of operating conditions and the engine behavior. Taking advantage of this database, they directed part of the data for model training in machine learning procedure, resulting in a robust predictive tool that was further validated with a subset of the database - the validation data.   The paper was prepared, written, organized and presented with good characteristics. The research outcome content is useful. The predictive tool shows good agreement with the database, even when new set of parameters are inputed. The methodology employed can be implemented for other types of analysis in several scientific areas.   My only comments refers to the figures 9-16, if there could be an efficient visualization format to present the data, as most of the plots show low data density, and could be improved to promote a more interesting evaluation of the datasets. Regarding figure 9 and 14, they are table, and should be labeled as such.   For these reasons above mentioned, i suggest the publication of the manuscript after minor explanations and possible modifications.

Author Response

Respected Sir. Thank you very much for taking the time to review this manuscript. As per your suggestions, The quality of figures (9-16) has been improved. As the desired results were taken from MATLAB, so figures were saved in jpg files which are pasted in the updated manuscript.

Regards

Reviewer 2 Report

After carefully reviewing the entire paper, I have identified several points that need to be addressed:

1.      Clarify Boundary Conditions: In the paper, make sure to provide a more detailed explanation of the boundary conditions used in your analysis. Include equations, diagrams, or additional text that elucidates how these conditions were applied and their significance to the study.

2.      Rewrite the Abstract: A well-crafted abstract is crucial for conveying the essence of your work. Ensure that the abstract is clear, concise, and provides a comprehensive summary of your research, including the key findings and their significance.

3.      Grammar and Language: Review and revise the paper for english grammar and language issues.

4.      New Research and Citations: Update the paper's references section with more recent and relevant research. Ensure that all sources are properly

5.      Explanation of Figures and Tables: For each figure and table in the paper, provide clear and concise explanations. Make sure the captions are informative and not just descriptive.

Dear Sir,

Overall, the manuscript is well-structured and presents a valuable contribution to the field. However, there are some language and grammatical issues that need attention. I recommend a thorough proofreading and editing to improve clarity and readability. Additionally, the abstract could be revised for greater conciseness and to more effectively highlight the research's significance. Proper citation and referencing, along with the incorporation of recent research, should be ensured. Providing more comprehensive explanations for figures and tables would enhance the paper's accessibility to a wider audience. Addressing these points will significantly strengthen the paper's overall quality and impact.

Author Response

Respected Sir, Thank you very much for taking the time to review this manuscript. Please find the detailed responses below and the corresponding revisions/corrections highlighted as Yellow in the re-submitted files.

1. "Clarify Boundary Conditions"
   Respected Sir, 

   Equations 2-4 are from Mattingly’s book, Aircraft Engine Design [1]. Steady, one-dimensional flow is assumed with the fluid behaviour considered as a perfect gas. Specific thrust depends on the velocity ratio and overall static temperature ratio, has been added in the revised manuscript.

2.  Rewrite the Abstract
   
   Respected Sir, as per your kind suggestion, the abstract has been updated as highlighted in green.
3. Grammar and Language
   Respected Sir, in the revised manuscript the grammar has been checked and corrected by using Grammarly-free software.
4. New Research and Citations
   
   Thank you for your suggestions. 7 new citations have been added in the revised manuscript
5.  Explanation of Figures and Tables
   Thank you for your suggestions. Details have been added in the revised manuscript. 

Reviewer 3 Report

The article presents the development of a predictive tool for parametric analysis of a turbofan engine. The tool is essentially based on neural networks trained on a dataset generated by a commercial software for gas turbine simulation.

In the reviewer's opinion, the study does not make significant contributions either to the field of gas turbine simulation or to the field of deep learning algorithms. However, the proposed construction is technically sound, was well conducted and may be of interest to the specialized community involved with the topic. The manuscript is satisfactorily well written, clear and well presented. Bibliographical references are apparently representative and up-to-date. Figures and tables are of satisfactory quality.

The following minor review points should be addressed by the authors.

1- In lines 130-132, the following statement is found: “Professedly, this research is unprecedented as the use of supervised machine learning and deep learning for the prediction of parametric analysis of a turbofan engine has not been researched before.” Here a question naturally arises: has no turbofan engine other than the F100-PW229 been analyzed in this way before? If so, this issue should be emphasized; otherwise, it should be better contextualized in the revised manuscript.

2- It is important to comment the basic principles and assumptions involved in Equations 2 and 4.

3- In line 206, reference is made to “five models”. Later, in section 3.3, it becomes clear that “four models” were actually built, one for each output variable. It is necessary to correct or clarify.

4- In Figs. 5-8, only one input variable is varied at a time. Were the other input variables maintained with the values listed in Table 1?

5- How was the regression model chosen in each of the 16 situations listed in figure 9? More specifically: why the Matern 5/2 Gaussian Process Regression was used in 14 situations and another model was used in the remaining two situations.

Author Response

Thank you very much for taking the time to review this manuscript. Please find the detailed responses below and the corresponding revisions/corrections highlighted/in track changes in the re-submitted files.

The reviewer's supportive, critical, and constructive remarks on this article were acknowledged by the coauthors and me. The suggestions were extremely comprehensive and helpful in enhancing the manuscript.  We have exclusively considered them in our rewrite. We are submitting a revised document with the advice implemented. The paper has been changed in response to the reviewer's recommendations, and our replies to all of them are as follows:

1- In lines 130-132, the following statement is found: “Professedly, this research is unprecedented as

the use of supervised machine learning and deep learning for the prediction of parametric analysis

of a turbofan engine has not been researched before.” Here a question naturally arises: has no

turbofan engine other than the F100-PW229 been analyzed in this way before? If so, this issue

should be emphasized; otherwise, it should be better contextualized in the revised manuscript.

Response: Sorry for the confusion, clarification has been added in this statement that both deep learning and supervised machine learning algorithms have not been employed for the parametric analysis of F100-PW229 engine in the lines 161-165.

2- It is important to comment the basic principles and assumptions involved in Equations 2 and 4.

Response:  Thank you for this suggestion, it has been edited in lines 205-208

3- In line 206, reference is made to “five models”. Later, in section 3.3, it becomes clear that “four

models” were actually built, one for each output variable. It is necessary to correct or clarify.

Response:  Sorry for this mistake, it has been corrected to four models in line 238.

4- In Figs. 5-8, only one input variable is varied at a time. Were the other input variables maintained

with the values listed in Table 1?

Response:  The explanation of single and two parameter variation has been added in lines 302-305.

5- How was the regression model chosen in each of the 16 situations listed in figure 9? More

specifically: why the Matern 5/2 Gaussian Process Regression was used in 14 situations and another

model was used in the remaining two situations.

Response:  Thank you for pointing this out, the reason for selecting Gaussian Process Regression and Wide Neural Network has been added in lines 372-376, and 384-386, along with the comparison of all other models.

Author Response File: Author Response.pdf

Reviewer 4 Report

1) The Abstract is too general and only descriptive. In the Abstract the Authors should add some of the most important results obtained in this research (its exact values), which will highlight the novelties of the performed research already in the Abstract.

2) In the paper should be added a Nomenclature inside which will be listed and explained in one place all abbreviations, symbols and markings used throughout the paper.

3) The paper should be much better arranged. In Section 2.1.2 Validation are presented input parameters for the GasTurb software, any validation results are missing in this Section. The validation results are presented in Table 3, which is placed in Section 3. Results, but actually this Table represents validation. The paper should be properly and logically arranged, at the moment paper arrangement is confusing and illogical.

4) Section 2.3 Deep Learning – the Authors should add in a paper a discussion related to the neural network type (MLP) selection. For this problem can be used many neural network types, so it should be described why this neural network type is selected. According to my experience, some other neural network types may give the same (or maybe better) results. So, the neural network selection explanation is required.

Moreover, is it really necessary to present MLP structure in Figure 1? In my opinion, it is sufficient to describe as a text what are exact input and output parameters, other important elements can be found in Table 2. MLP structure is well known so far, so I don’t see a point to present it once more.

5) Table 3 – the Authors should present much more parameters and compare them with data from the literature. Comparison of simulation and published data for thrust and fuel consumption rate only (both in wet and dry conditions) is surely not sufficient for the simulation results validation.

In regards to the error between simulation model results and published results – the Authors have stated that the error less than 10% is satisfying. This statement needs some kind of evidence. In my opinion, accurate and precise numerical simulations have an error in comparison to measured data for no more than 3%. So, it is unknown how the Authors conclude that the obtained errors are satisfactory. The comparison with a literature which deals with the gas turbine numerical models and their errors will also be correct, but some kind of explanation and confirmation related to the acceptable error must be presented in the paper.

6) Figures from 4 to 8 should be much better explained. The reasons which lead to the presented results should be discussed. At the moment, practically each figure is explained by using one sentence, therefore the reasons which lead to the presented results are unknown – they should be added and described.

7) Figures 9 and 14 are actually Tables, not Figures. RMSE from Figure 9 and MSE from Figure 14 should be defined and explained in the paper. Why the Authors use only these two evaluation metrics, why also the other evaluation metrics are not used?

8) Figures from 10 to 13 are practically not explained at all – the explanations are required.

9) In the paper occur two Sections 3.3. The corrections are required.

10) Also the Figure 16 should be properly explained – any exact explanation at the moment is missing.

11) I am quite suspicious in relation to the presented results because both Machine Learning methods (Figures from 10 to 13) and Deep Learning methods (Figure 16) shows practically perfect predictions. It is very rare to see such results, which did not have obvious deviations in any conditions and in any presented parameter. The Authors should briefly describe such perfection during Figures descriptions, in the most of the cases such perfection is the result of the data overfitting. If so, then all presented results and predictions are actually incorrect. Therefore, the explanations are highly required.

12) The List of the References should be notably enlarged and much more recent literature from this research field should be added.

 

Final remarks: This is surely interesting article. However, deep and detail corrections, as well as proper explanations are required. Without them, all the results presented in this paper can be highly questionable or possibly wrong.

Minor editing of English language required.

Author Response

Thank you very much for taking the time to review this manuscript. Please find the detailed responses below and the corresponding revisions/corrections highlighted/in track changes in the re-submitted files.

Clarify Boundary Conditions: In the paper, make sure to provide a more detailed explanation

of the boundary conditions used in your analysis. Include equations, diagrams, or additional text that

elucidates how these conditions were applied and their significance to the study.

Reply: The parametric study variables and their range has been added in Table 1.

2. Rewrite the Abstract: A well-crafted abstract is crucial for conveying the essence of your work.

Ensure that the abstract is clear, concise, and provides a comprehensive summary of your research,

including the key findings and their significance.

Reply: The abstract should has been modified to include relevant details.

3. Grammar and Language: Review and revise the paper for english grammar and language

issues.

Reply: Necessary revisions have been made.

4. New Research and Citations: Update the paper's references section with more recent and

relevant research. Ensure that all sources are properly

Reply: More literature has been cited.

5. Explanation of Figures and Tables: For each figure and table in the paper, provide clear and

concise explanations. Make sure the captions are informative and not just descriptive.

Reply: Explanations have been added for all figures and tables.

Round 2

Reviewer 4 Report

The Authors did not upload proper point-to-point answers to my comments, the answers are related to some other Reviewer comments, not mine.

Without point-to-point answers, I cannot be sure what the Authors have exactly performed and in which way.

It is highly unexpected that the Authors upload wrong point-to-point answers.

Minor editing of English language required.

Author Response

The reviewer's supportive, critical, and constructive remarks on this article were acknowledged by the coauthors and me. The suggestions were extremely comprehensive and helpful in enhancing the manuscript.  We have exclusively considered them in our rewrite. We are submitting a revised document with the advice implemented. The paper has been changed in response to the reviewer's recommendations, and our replies to all of them are as follows:

• The Abstract is too general and only descriptive. In the Abstract the Authors should add some of the most important results obtained in this research (its exact values), which will highlight the novelties of the performed research already in the Abstract.

Reply: Thank you for this suggestion, the abstract has been updated with relevant details and results in lines 19-26.

• In the paper should be added a Nomenclature inside which will be listed and explained in one place all abbreviations, symbols and markings used throughout the paper.

Reply: Apologies for not including it before, a nomenclature has been added at the end in line 471.

• The paper should be much better arranged. In Section 2.1.2 Validation are presented input parameters for the GasTurb software, any validation results are missing in this Section. The validation results are presented in Table 3, which is placed in Section 3. Results, but actually this Table represents validation. The paper should be properly and logically arranged, at the moment paper arrangement is confusing and illogical.

Reply: Sorry for this confusion, the necessary corrections have been made, input parameters for validation have been placed in Section 3.

• Section 2.3 Deep Learning – the Authors should add in a paper a discussion related to the neural network type (MLP) selection. For this problem can be used many neural network types, so it should be described why this neural network type is selected. According to my experience, some other neural network types may give the same (or maybe better) results. So, the neural network selection explanation is required. Moreover, is it really necessary to present MLP structure in Figure 1? In my opinion, it is sufficient to describe as a text what are exact input and output parameters, other important elements can be found in Table 2. MLP structure is well known so far, so I don’t see a point to present it once more.

Reply: Thank you for pointing it out, MLP selection reason is explained in lines 258-262 and Figure 1 has been omitted.

• Table 3 – the Authors should present much more parameters and compare them with data from the literature. Comparison of simulation and published data for thrust and fuel consumption rate only (both in wet and dry conditions) is surely not sufficient for the simulation results validation. In regards to the error between simulation model results and published results – the Authors have stated that the error less than 10% is satisfying. This statement needs some kind of evidence. In my opinion, accurate and precise numerical simulations have an error in comparison to measured data for no more than 3%. So, it is unknown how the Authors conclude that the obtained errors are satisfactory. The comparison with a literature which deals with the gas turbine numerical models and their errors will also be correct, but some kind of explanation and confirmation related to the acceptable error must be presented in the paper.

Reply: Thank you for your comment, due to the lack of complete commercially classified information concerning the F100-PW229 engine, only few parameters were compared. However, total fuel mass flow rate of the engine with afterburner has been added.

The error between published results and simulated results for gas turbine simulation in (Sabzehali et al.) and (Sung) are greater than 9% (cited in the paper). Hence the error of 9.2% is considered in this paper as well.

• Figures from 4 to 8 should be much better explained. The reasons which lead to the presented results should be discussed. At the moment, practically each figure is explained by using one sentence, therefore the reasons which lead to the presented results are unknown – they should be added and described.

Reply: Thank you for this suggestion, discussion for these figures have been added.

• Figures 9 and 14 are actually Tables, not Figures. RMSE from Figure 9 and MSE from Figure 14 should be defined and explained in the paper. Why the Authors use only these two evaluation metrics, why also the other evaluation metrics are not used?

Reply: Sorry for the mistake, necessary corrections have been made and the reason for selection of evaluation metrics is also added in lines 365-372.

• Figures from 10 to 13 are practically not explained at all – the explanations are required.

Reply: Thank you for this suggestion, explanations have been added.

• In the paper occur two Sections 3.3. The corrections are required.

Reply: Apology for the repetition, corrections have been made.

• Also the Figure 16 should be properly explained – any exact explanation at the moment is missing.

Reply: Thank you for this suggestion, explanation has been added.

• I am quite suspicious in relation to the presented results because both Machine Learning methods (Figures from 10 to 13) and Deep Learning methods (Figure 16) shows practically perfect predictions. It is very rare to see such results, which did not have obvious deviations in any conditions and in any presented parameter. The Authors should briefly describe such perfection during Figures descriptions, in the most of the cases such perfection is the result of the data overfitting. If so, then all presented results and predictions are actually incorrect. Therefore, the explanations are highly required.

Reply: Sorry for not clarifying this before, In MATLAB, a cross validation with k-fold technique has been implemented that counters the overfitting of data. Similarly, In Deep Learning, hold out method and k-fold cross validation technique have been used to prevent overfitting of data. The accuracy of the neural network developed on test data is also satisfactory that ensures the predictions of the model are acceptable.

• The List of the References should be notably enlarged and much more recent literature from this research field should be added.

Reply: Thank you for this suggestion, more literature has been cited.

Author Response File: Author Response.pdf

Round 3

Reviewer 4 Report

The Authors have performed all required corrections/improvements.

I still believe that the descriptions of some obtained results can be more detail, but after revision they are at least sufficient.

The paper can be published in a presented (revised) form.