JT9D Engine Thrust Estimation and Model Sensitivity Analysis Using Gradient Boosting Regression Method

Abstract

In recent years, artificial intelligence (AI) technology has been applied in different research fields. In this study, the XGBoost regression model is proposed to estimate JT9D engine thrust. The model performance mean absolute error (MAE) is 0.004845, the mean-squared error (MSE) is 0.000161, and the coefficient of determination (R²) values of the training, validation, and testing subsets are 0.99, 0.99, and 0.98, respectively. Based on a model sensitivity analysis, the four parameters’ optimal values are as follows: the number of estimators is 900; the learning rate is 0.1; the maximum depth is 4, and the random state is 3. In addition, a comparison between the model performance in this study and that in a previous one was conducted. The MSE value is as low as 0.000021.

1. Introduction

The JT9D engine program was introduced in 1965 by Pratt and Whitney, who conducted the first engine test in 1966. Its advanced mechanical design technology and excellent turbofan engine performance were then exploited successfully in the military J58 turbofan engine [1]. In the 1970s and 1980s, the national aeronautics and space administration (NASA) implemented some technologies related to the JT9D turbofan engine. In 1978, two kinds of research were conducted on the JT9D engine in regard to its performance deterioration and the analytical study of the thermal barrier coating of the first-stage blades [2,3]. In 1980, the performance and deterioration of the CF6/JT9D engine were investigated by NASA [4]. Furthermore, NASA presented another report in 1982 investigating flight loads on the Boeing 747 aircraft, which used the JT9D turbofan engine [5]. Since the JT9D engine began to serve on the Boeing 747 aircraft in 1970, it has proven itself as the workhorse for the early 747, 767, A300, A310, and DC-10 aircraft models, with more than 3200 engines delivered in total [6].

An analysis of the JT9D engine’s power was conducted by Onal and Turan using aero-engine momentum and continuity equations [7]. Fang et al. successfully built the FMI-based multidomain simulation of the JT9D engine control system [8]. The estimation of engine thrust is the primary objective of an engine control system. Therefore, Maggiore et al. proposed an estimator design to handle the nonlinear dynamics system of a jet engine [9]. Zhou and Zhang presented a study report on engine thrust estimation using an ensemble of improved wavelet extreme learning machines to satisfy the conditions of direct thrust control [10]. In addition, Oruc and Bakircioglu used a particle swarm optimization algorithm to model JT9D engine thrust; in this study, the investigators found high-accuracy thrust values during cruise flights [11]. A study was proposed to investigate the application of artificial neural networks in accurately predicting aircraft thrust by leveraging the capability of ANNs to learn from large datasets [12].

In recent years, artificial intelligence (AI) technology has been applied in different research fields, such as the prediction of the composition of syngas [13], the emissions of diesel engines [14], and aeroengine thrust estimations [15,16]. Furthermore, the XGBoost algorithm has been widely utilized for detection, disease diagnosis, classification, and predictions [17,18,19,20,21,22,23]. Based on the excellent performances of the XGBoost algorithm in previous research, the XGBoost regression model is proposed in the present study to estimate JT9D engine thrust. Nguyen-Sy et al. studied the XGB model to predict the compressive strength of concrete and perform a sensitivity analysis of the model parameters’ effects [23]. Sensitivity analysis was also applied to this study’s XGBoost model to investigate the effects of the parameters.

2. Materials and Methods

2.1. GT9D Turbofan Engine

In the research conducted by Jackson et al., the authors presented the performance and advanced design of the JT9D turbofan engine [1]. The specifications of the JT9D turbofan engine are listed in

Table 1. Specifications of the JT9D turbofan engine [1].

Parameter	Value
Thrust (N)	186,900
Fan Pressure Ratio	1.55
Overall Compressor Pressure Ratio	24.5
Bypass Ratio	5.0
H.P. Compressor Pressure Ratio	10.0
Total Corrected Airflow (kg/s)	740.02

(Takeoff performance, sea level static, 21.1 °C day).

. The engine stations are as follows: Air flows through the inlet and enters the fan situated in station 2. The exit station of the fan is named 2.5. The bypass airs flow from stations 13 to 16. The exit for the compressor is at station 3, and the exit of the burner is at station 4. The compression process between stations 2.5 and 3 is meant to compress the air to achieve a pressure ratio according to the needs of the burner. The hot, high-pressure gas discharged from the burner (station 4) enters the high-pressure turbine that is connected to the compressor through a shaft. Then, the hot gas enters the low-pressure turbine in station 4.5 and exits the engine at station 5 [24]. Finally, the exhaust gas discharges from the nozzle name station 8. The schematic of the JT9D turbofan engine is shown in Figure 1.

JT9D Turbofan Engine Parametric Cycle Analysis

Parametric cycle analysis, which is a design point of the engine, studies the thermodynamic changes in the working fluid flowing through the engine [25]. NASA uses the numerical propulsion system simulation code to implement the thermodynamic performance analysis of aircraft gas turbine engine cycles [26]. The main focus of this analysis involves the operating characteristics of each module according to the principles of thermodynamics and turbomachinery; these are introduced as follows:

• Fan

For a given fan pressure ratio $\left({\pi }_f\right)$ and isentropic efficiency $\left({\eta }_f\right)$, the relationship between the fan total temperature and total pressure is as follows:

$$P_{t25}=P_{t2}\cdot {\pi }_f$$

(1)

$$T_{t25}=T_{t2}\left[1+\frac{1}{{\eta }_f}\left({\pi }_f^{\raisebox{1ex}{${\gamma }_a-1$}\!\left/ \!\raisebox{-1ex}{${\gamma }_a$}\right.}-1\right)\right]$$

(2)

where $P_{t25}$ is the total pressure at the outlet of the fan, $T_{t25}$ is the total temperature at the outlet of the fan, and ${\gamma }_a$ is the ratio of the specific heat of the cold airflow.

Because the air is compressed by the fan and guided into the bypass channel according to the set ratio, mixed with the core airflow at the outlet of the turbine, and then discharged from the nozzle to generate thrust, the ratio of the bypass airflow to the core airflow is defined as the bypass ratio, as follows:

$$R_{Bypass}=\frac{G_{a_{Bypass}}}{G_{a_{core}}}$$

(3)

where $R_{Bypass}$ is the bypass ratio, $G_{a_{Bypass}}$ is the bypass airflow, and $G_{a_{core}}$ is the core airflow.

2.

Compressor

The primary function of the compressor module is to compress the core airflow entering the engine and to provide combustion in the combustion chamber. The compressor pressure ratio is equal to LPC-Pt times HPC-Pt. When the pressure ratio is given, the total pressure of the outlet airflow can be calculated as follows:

$$P_{t3}={\pi }_C\cdot P_{t25}$$

(4)

where $P_{t3}$ is the total pressure of the outlet airflow of compressor and ${\pi }_C$ is the compressor pressure ratio.

Then, according to the overall adiabatic compressor efficiency, ${\eta }_C=\frac{w_{C_{ideal}}}{w_{C_{actual}}}=\frac{\Delta h_{tC}^{\prime }}{\Delta h_{tC}}$, the axial work required for the airflow to be compressed in the compressor can be calculated as follows:

$$W_C=\frac{C_{pa}{T}_{t25}}{{\eta }_C}\left[{\left(\frac{P_{t3}}{P_{t25}}\right)}^{\frac{r_a-1}{r_a}}-1\right]$$

(5)

$$T_{t3}=\frac{W_C}{C_{pa}}+T_{t25}$$

(6)

where $W_C$ is the compressor’s required work, $C_{pa}$ is the specific heat of the cold airflow, and $T_{t3}$ is the total temperature of the outlet airflow of the compressor.

To prevent the turbine blades from being burned due to high temperature, the compressor usually extracts part of the relatively low-temperature compressed air into the turbine; thus, the airflow entering the combustion chamber should be calculated as follows:

$$G_{agg}=G_{a_{core}}-G_{a_{NGV}}-G_{a_{HPT}}-G_{a_{LPT}}$$

(7)

In Equation (7), $G_{a_{NGV}}$, $G_{a_{HPT}}$, and $G_{a_{LPT}}$ represent the cooling airflow required to be extracted by the first-stage nozzle of the turbine, the high-pressure turbine, and the low-pressure turbine, respectively. Suppose the compressor is an n-stage axial flow compressor, and the cooling airflow is extracted from the outlet of the a-stage compressor. In this case, the pressure ratio of the cooling airflow can be calculated as follows:

$${{\pi }_{Cool}={\left(\sqrt[n]{{\pi }_C}\right)}^a}_{}$$

(8)

where ${\pi }_{Cool}$ is the pressure ratio of the cooling airflow.

3.

Combustor

The primary function of the combustion chamber module is to provide a mixture of fuel and high-pressure gas for combustion to release the chemical energy for power, thereby increasing the temperature and thermal energy of the working airflow. The ideal combustion state in the combustion chamber should be equal-pressure combustion; however, in real conditions, pressure losses may be caused by friction and other factors. By defining the total pressure loss percentage $\Delta P$, the total pressure of the airflow at the exit of the combustion chamber can be calculated as follows:

$$P_{t4}=\left(1-\Delta P\right)\cdot P_{t3}$$

(9)

where $P_{t4}$ is the total pressure of the airflow at the exit of the combustion chamber, and $\Delta P$ is the total pressure loss percentage:

$$T_{t4}=\frac{G_f\cdot HV\cdot {\eta }_B+C_{Pa}{T}_{t3}\left(G_f+G_{a_{core}}\right)}{C_{Pg}\cdot \left(G_f+G_{a_{core}}\right)}+T_{t3}$$

(10)

where $T_{t4}$ is the total temperature of the airflow at the exit of the combustion chamber, $G_f$ is the fuel flow, $HV$ is the heating value, ${\eta }_B$ is the combustion efficiency, and $C_{Pg}$ is the specific heat of the hot airflow.

4.

High-pressure turbine

The primary function of the high-pressure turbine is to convert the outlet airflow of the combustion chamber to shaft work in order to drive the rotation of the compressor. If a small part of the shaft work must be transferred for other purposes (i.e., power takeoff) owing to task requirements, such as driving the accessory gearbox, the shaft power extraction percentage is defined as $PTO$; for the $PTO$, the principle of shaft power matching between the compressor and high-pressure turbine must be satisfied. Assuming that the mechanical efficiency ${\eta }_{mech}$ of the transmission between the compressor and high-pressure turbine is known, the high-pressure shaft work provided by the turbine can be calculated as follows:

$${\dot{W}}_{HPT}=\frac{{\dot{W}}_C}{\left(1-PTO\right){\eta }_{mech}}$$

(11)

where ${\dot{W}}_{HPT}$ is the high-pressure turbine shaft power, $PTO$ is the shaft power extraction percentage, and ${\eta }_{mech}$ is the mechanical efficiency of the drive shaft between the high-pressure turbine and compressor.

Furthermore, according to the definition of the high-pressure turbine output shaft work,

$$\begin{array}{c}{\dot{W}}_{HPT}=\left(G_{agg}+G_{aNGV}+{\dot{G}}_{aHPT}+{\dot{G}}_f\right)\\ \times \left(C_{Pg}{T}_{t4}-C_{Pg}{T}_{t45}\right)\end{array}$$

(12)

where $T_{t45}$ is the total temperature of the high-pressure turbine.

The total temperature of the airflow at the outlet of the high-pressure turbine can then be calculated as follows:

$$T_{t45}=\frac{\left[C_{Pg}{T}_{t4}-\frac{{\dot{W}}_{HPT}}{\left({\dot{G}}_{agg}+G_{aNGV}+{\dot{G}}_{aHPT}+{\dot{G}}_f\right)}\right]}{C_{Pg}}$$

(13)

Assuming that the airflow through the high-pressure turbine follows an adiabatic expansion process, its adiabatic efficiency ${\eta }_{HPT}$ can be defined as follows:

$${\eta }_{HPT}=\frac{∆h_{tHPT}}{∆h_{t{HPT}^{\prime }}}=\frac{C_{Pg}{(T}_{t4}-T_{t45})}{C_{Pg}(T_{t4}-T_{t45}^{\prime })}$$

(14)

where $∆h_{tHPT}$ is the actual enthalpy for the high-pressure turbine, $∆h_{t{HPT}^{\prime }}$ is the ideal enthalpy for the high-pressure turbine, and $T_{t45}^{\prime }$ is the total temperature of the ideal high-pressure turbine.

The total temperature of the outlet airflow of the high-pressure turbine in the ideal process can be deduced through the transposition of Equation (11), as follows:

$$T_{t45}^{\prime }=T_{t4}-\frac{T_{t4}-T_{t45}}{{\eta }_{HPT}}$$

(15)

The total pressure at the outlet airflow of the high-pressure turbine can then be deduced according to the isentropic relationship, as follows:

$$P_{t45}=P_{t4}{\left(\frac{T_{t45}^{\prime }}{T_{t4}}\right)}^{\frac{\gamma }{\gamma -1}}$$

(16)

where $P_{t45}$ is the total pressure of the high-pressure turbine.

5.

Low-pressure turbine

The primary function of the low-pressure turbine is the conversion of the heat energy of the airflow introduced from the high-pressure turbine to shaft power for driving the rotation of the fan. The calculations for the changes in airflow properties are similar to those of the high-pressure turbine, because the shaft work must be matched between the fan and low-pressure turbine (according to the power-matching principle). Hence, on the premise that the mechanical efficiency of the shaft work transfer between the two modules is known, the formula for the shaft work output per unit time of the low-pressure turbine can be deduced as follows:

$${\dot{W}}_{LPT}=\frac{{\dot{W}}_F}{{\eta }_{mech}}$$

(17)

where ${\dot{W}}_{LPT}$ is the low-pressure turbine shaft power.

Then, the low-pressure turbine output shaft work per unit time can be defined as follows:

$$\begin{array}{c}{\dot{W}}_{LPT}=({\dot{G}}_{agg}+{\dot{G}}_{aNGV}+{\dot{G}}_{aHPT}\\ +{\dot{G}}_{aLPT}+{\dot{G}}_f)C_{Pg}\left(T_{t45}-T_{t5}\right)\end{array}$$

(18)

where $T_{t5}$ is the total temperature of the low-pressure turbine.

The total temperature of the airflow at the outlet of the low-pressure turbine is then calculated as follows:

$$T_{t5}=T_{t45}-\frac{W_{LPT}}{\begin{array}{c}{C}_{Pg}({\dot{G}}_{agg}+{\dot{G}}_{aNGV}+{\dot{G}}_{aHPT}\\ +{\dot{G}}_{aLPT}+{\dot{G}}_f)\end{array}}$$

(19)

Suppose that the airflow passing through the low-pressure turbine is assumed to be an adiabatic expansion process. Thus, its adiabatic efficiency ${\eta }_{LPT}$ can be defined as follows:

$${\eta }_{LPT}=\frac{∆h_{LPT}}{∆h_{{LPT}^{\prime }}}=\frac{C_{Pg}{(T}_{t45}-T_{t5})}{C_{Pg}(T_{t45}-T_{t5}^{\prime })}=\frac{T_{t45}-T_{t5}}{T_{t45}-T_{t5}^{\prime }}$$

(20)

where $∆h_{tLPT}$ is the actual enthalpy for the low-pressure turbine, $∆h_{{tLPT}^{\prime }}$ is the ideal enthalpy for the low-pressure turbine, and $T_{t5}^{\prime }$ is the total temperature of the ideal low-pressure turbine.

From Equation (20), the total temperature of the outlet airflow of the low-pressure turbine in the ideal process can be deduced as follows:

$$T_{t5}^{\prime }=T_{t45}-\frac{T_{t45}-T_{t5}}{{\eta }_{LPT}}$$

(21)

The total pressure of the outlet airflow of the low-pressure turbine can then be deduced according to the isentropic relationship, as follows:

$$P_{t5}=P_{t45}{\left(\frac{T_{t5}^{\prime }}{T_{t45}}\right)}^{\frac{\gamma }{\gamma -1}}$$

(22)

where $P_{t5}$ is the total pressure of the low-pressure turbine.

2.2. GT9D Turbofan Engine Data

Simulations are used to collect the dataset at the component level of the JT9D turbofan engine [27]. The 13 features selected for the AI model are listed in

Table 2. Thrust estimator parameters for the JT9D turbofan engine.

Symbol/Unit	Parameter
Wf (pps)	Fuel flowrate
HPC-Pt	High-pressure compressor pressure ratio
LPC-Pt	Low-pressure compressor pressure ratio
P4 (psia)	Combustion chamber exit pressure
T4 (°R)	Combustion chamber exit temperature
HPT-Pt	High-pressure turbine pressure ratio
LPT-Pt	Low-pressure turbine pressure ratio
HPS-Nmech (rpm)	High-pressure rotor speed
LPS-Nmech (rpm)	Low-pressure rotor speed
P45 (psia)	High-pressure turbine exit pressure
T45 (°R)	High-pressure turbine exit temperature
T8 (°R)	Nozzle exit temperature
Fg (lbf)	Engine thrust

. A total of 12 features were chosen for the AI model inputs, including the fuel flowrate (W_f), combustion chamber exit temperature (T₄), and nozzle exit temperature (T₈). The engine thrust (F_g) is the expected target value of the AI model. The selection of the 12 input features selection was based on the research of Wang et al. [16]. Furthermore, the thrust simulation results of Wang and this study could be compared to evaluate the accuracy.

JT9D Turbofan Engine Data

The total number of samples obtained in the data collection was 438. Figure 2a shows the violin plots depicting the summary statistics and densities of HPS-N_mech and F_g. The broader areas of the violin plots represent higher probabilities that the members of the population will take on the given values; conversely, the narrower areas represent lower probabilities. For instance, in Figure 2a, the white dot represents the median and the black bar in the center represents the interquartile range. The top of the black bar represents the third quartile, and the bottom represents the first quartile. Figure 2b–d shows the summary statistics and densities of T₄, LPS-N_mech, T₄₅, T₈, P₄, P₄₅, W_fuel, HPC_Pt, LPC_Pt, HPT_Pt, and LPT_Pt. The exact interquartile range values are listed in

Table 3. The exact interquartile range values.

Symbol	Third Quartile	Median	First Quartile
Wf	2.951835	1.64026	0.889763
HPC-Pt	5.603	5.306	4.82475
LPC-Pt	2.25775	2.117	1.717
P4	186.84425	114.1555	71.088
T4	2459.115	2192.31	1808.205
HPT-Pt	2.77175	2.725	2.69925
LPT-Pt	4.81675	4.5125	3.514
HPS-Nmech	7595.275	7226.1	6802.7
LPS-Nmech	3584.2	3213.65	2675.525
P45	69.066	41.7	26.085
T45	1919.1875	1700.105	1386.7825
T8	1340.6175	1214.885	1049.4125
Fg	22,097.65	10,562.45	5623.125

.

The statistical analysis results between the model input features and target are represented visually in Figure 3. From Figure 3, it can be observed that parts of the input features are proportional to the engine thrust F_g, such as W_fuel, P₄, T₄, HPS_N_mech, P₄₅, and T₈. Figure 3a shows the results of the thrust increasing due to the increase in the fuel flow rate. This means that energy sustainably increases when more fuels are sprayed into the combustion chamber. Furthermore, Figure 3d shows that the pressure loss in the combustion chamber affects the thrust. The most important factor affecting the thrust is the temperature exit from the combustion chamber, shown in Figure 3e. A higher T₄ value means that a higher thrust can be created. This is a similar effect to that of the aforementioned fuel flow rate. It is noted that even at the same temperature values, the thrust also increases because the cooling consideration limits the temperature value in the case of burnout. The relationships between the other input features and F_g also show proportional trends but are limited by the power matching and cooling considerations.

2.3. XGBoost Model

Chen and Guestrin proposed a novel sparsity-aware algorithm, which is an open-source package for approximate tree learning called the XGBoost. The XGBoost algorithm can effectively reduce calculation costs while providing a higher model performance [28]. Figure 4 shows the flowchart of the proposed XGBoost algorithm model for engine thrust prediction. In the research of Wang et al., the percentages of splitting the dataset for different combinations were investigated [16]. There are three ratios for the training and testing datasets: 8:2, 6:4, and 5:5. The ratio 8:2 had the lowest MSE value, which indicated an excellent model performance. In the present study, the dataset was split into three subsets using the Python sklearn library train_test_split code, so that 70% was used for model training needs, 10% was used for the validation of the model’s performance, and the remaining 20% was used for model testing. The primary parameters of the XGBoost model are as follows: for the Max_depth, denoting the maximum depth of the tree, the default value is 6; this was set to 4 in this study to prevent overfitting. N_estimators represents the number of trees used for boosting, whose value was set to 900 herein. The Learning_rate indicates the learning rate that determines the step size at each iteration while reaching a minimum loss function; the lower the value of this parameter is, the better the performance is. Thus, this value was set to 0.1 in this study. Colsample_bytree denotes a family of parameters for subsampling the columns within the range of 0 to 1, and this value was set to 0.8. The subsample represents the training sample’s subsample ratio, which was set to 0.5. The parameter values of Colsample_bytree and the subsample ratio were determined using typical values based on experience.

The model’s performance reveals its accuracy, and the mean absolute error ($MAE$), mean-squared error ($MSE$), and coefficient of determination ($R^2$) are the metrics used in this study to evaluate the model’s performance. The statistical equations for these metrics are as given in Equations (23)–(25), where $P_i$ is the predicted value obtained from the model and $T_i$ is the target value obtained from the dataset. ${\overline{P}}_i$ is the average of the predicted values for the entire dataset.

$$MAE=\frac{1}{n}{\sum }_{i=1}^n\left|T_i-P_i\right|$$

(23)

$$MSE=\frac{1}{n}{\sum }_{i=1}^n{\left(T_i-P_i\right)}^2$$

(24)

$$R^2=1-\frac{{\displaystyle {\sum }_{i=1}^n}{\left(P_i-T_i\right)}^2}{{\displaystyle {\sum }_{\mathrm{i}=1}^n}{\left({\overline{P}}_i-T_i\right)}^2}$$

(25)

3. Results

3.1. Model Performance

The model performances are listed in

Table 4. Performance $MAE$, $MSE$, and $R^2$ statistics results of the XGBoost model.

$\mathit{M}\mathit{A}\mathit{E}$	$\mathit{M}\mathit{S}\mathit{E}$	${\mathit{R}}^{\mathbf{2}}$
		Train	Test	Validation	All
0.004845	0.000161	0.99	0.98	0.99	0.99

. These results were calculated based on Equations (20)–(22). According to Equation (22), if the predicted values are close to the target values, the R² value approaches 1. The $MAE$ and $MSE$ values are calculated based on the min–max normalization values of the dataset. In this model, the $R^2$ value is 0.99, which means that the predicted values are relative to the target values. Only the R² value for the 20% dataset for testing is 0.98. This is unsurprising, because the testing dataset is used to test the model, and it is possible to obtain a lower $R^2$ value for the actual model. It also shows the outstanding performance of the XGBoost regression model. The scatter plot of the engine thrust predictions versus the dataset values is shown in Figure 5.

3.2. Sensitivity Analysis

3.2.1. Effect of the Number of Estimators

In the XGBoost model, the estimators represent the number of trees used for boosting. The range varies from 50 to 1100. The MSE and R² values were observed during the estimator variation in the model training and testing. Figure 6a,b illustrate the MSE values for the model training and testing, respectively. Figure 6c,d represent the R² values for the model training and testing, respectively. The opposite result is observed between the MSE and R² values because of their definitions. While the number of estimators reaches 300, the lowest value of the MSE and the highest value of R² were obtained for the training dataset, as shown in Figure 6a,c. The same trends were observed in Figure 6b,d, except for the lower value compared with the results in Figure 6a,c. The reason for this is that the model testing dataset was selected for testing only. Therefore, the model’s performance for the testing dataset was lower than that for the training dataset.

3.2.2. Effect of the Learning Rate

The learning rate determines the step size at each iteration while reaching a minimum loss function. The range varies from 0.04 to 0.4. The MSE and R² values were also observed during the model training and the testing learning rate variation. Figure 7a,b illustrates the MSE values for the model’s training and testing, respectively. Figure 7c,d represents the R² values for the model’s training and testing, respectively. While the learning rate reached 0.1, the testing dataset’s MSE had the lowest value, and the R² value was the highest. These results determine the whole model’s performance.

3.2.3. Effect of the Max_depth

Max depth denotes the maximum depth of the tree. The range varies from 1 to 7. The MSE and R² values were also observed during the model’s training and testing with max depth variation. Figure 8a,b illustrates the MSE values for the model’s training and testing, respectively. Figure 8c,d represents the R² values for the model’s training and testing, respectively. While the max depth reached 4, the testing dataset’s MSE was the lowest, and the R² value was the highest. These results show a similar trend to that of the learning rate sensitivity analysis.

3.2.4. Effect of the Random State

The random state value for splitting the train and test datasets was also investigated, because it controls the shuffling applied to the data before using the split. The random state value will directly determine the samples of the training, validating, and testing datasets and affect the model’s performance. The range varies from 1 to 7. The MSE and R² values were also observed during the model’s training and testing with max depth variation. Figure 9a,b illustrates the MSE values for the model’s training and testing, respectively. Figure 9c,d represents the R² values for the model’s training and testing, respectively. While the random state value reached 3, the testing dataset’s MSE was the lowest, and the R² value was the highest. These results show a similar trend to that of the max depth sensitivity analysis.

4. Discussion

The deviations in the values between the validation dataset and its predictions are shown in Figure 10. The pink triangles and the blue squares stand for the validation dataset and the prediction of the validation dataset, respectively. Most predictions are close to the original validation dataset, reflecting the model’s outstanding performance. The deviations between the test dataset and its predicted values are shown in Figure 11. Most predictions are close to the original test dataset, especially when the engine thrust values are around 10,000 lbs; however, the deviations are more significant when the engine thrust values are over 30,000 lbs. These deviation numbers are obviously attributable to the fact that test dataset represents only 20% of the total dataset. Because the 20% testing dataset is for testing only, it is reasonable to infer that decreasing the testing dataset percentage would lead to a higher model performance, because the training dataset percentage would increase. Furthermore, Table 3 shows that the third quartile of the thrust dataset is 22,097.65 lbs. This statistic also reveals higher deviations between the actual and predicted thrust after the XGBoost regression model simulation, where the real thrust values are over 30,000 lbs.

Furthermore, the XGBoost regression model performance was validated by comparisons with the results reported by Wang et al. [16]. The MSE value in the present study is less than that of Wang’s model listed in

Table 5. Performance comparison between the XGBoost regression model and that of Wang et al. [16].

	MSE
XGBoost regression model	0.000161
Wang et al. model [16]	0.000182

, which indicates outstanding model performance.

The XGBoost regression model’s performance was investigated through model parameters such as the number of estimators, learning rate, max depth, and random state. These former three parameters directly relate to the model’s training and testing results and determine the final model performance. The last one is associated with the dataset split. The sample qualities of the dataset will also affect the model’s performance. Model sensitivity analyses were investigated by adjusting the range of these model parameters. These results are helpful in fully understanding the model’s training and testing processes. This procedure also helps the researcher to tune the model parameters. Based on the sensitivity analysis, the optimal parameters of the XGBoost model are listed in

Table 6. Optimal parameters of the XGBoost model.

Number of Estimators	Learning Rate	Max Depth	Random State
900	0.1	4	3

.

5. Conclusions

In this study, the statistical analysis of the violin plot and correlation between the input feature and target (thrust) of the JT9D engine data were calculated and plotted first. It is helpful to analyze and visualize the original data patterns and trends and to preprocess the dataset for model training.

Then, the thrust estimation model for the JT9D turbofan engine was successfully built based on the XGBoost regression model. The R² values of the training, validation, and testing subsets were 0.99, 0.99, and 0.98, respectively. In addition, a comparison between the model performance in this study and that in a previous one was conducted. The MSE value was as low as 0.000021, which shows the advantages of the XGBoost regression model.

After model sensitivity analysis, four parameters’ optimal values were determined by adjusting the contents in a particular range, as listed in Table 6. The values were as follows: the number of estimators was 900; learning rate was 0.1; max depth was 4; and random state was 3.