Using eXtreme Gradient BOOSTing to Predict Changes in Tropical Cyclone Intensity over the Western North Pacific

Abstract

Coastal cities in China are frequently hit by tropical cyclones (TCs), which result in tremendous loss of life and property. Even though the capability of numerical weather prediction models to forecast and track TCs has considerably improved in recent years, forecasting the intensity of a TC is still very difficult; thus, it is necessary to improve the accuracy of TC intensity prediction. To this end, we established a series of predictors using the Best Track TC dataset to predict the intensity of TCs in the Western North Pacific with an eXtreme Gradient BOOSTing (XGBOOST) model. The climatology and persistence factors, environmental factors, brainstorm features, intensity categories, and TC months are considered inputs for the models while the output is the TC intensity. The performance of the XGBOOST model was tested for very strong TCs such as Hato (2017), Rammasum (2014), Mujiage (2015), and Hagupit (2014). The results obtained show that the combination of inputs chosen were the optimal predictors for TC intensification with lead times of 6, 12, 18, and 24 h. Furthermore, the mean absolute error (MAE) of the XGBOOST model was much smaller than the MAEs of a back propagation neural network (BPNN) used to predict TC intensity. The MAEs of the forecasts with 6, 12, 18, and 24 h lead times for the test samples used were 1.61, 2.44, 3.10, and 3.70 m/s, respectively, for the XGBOOST model. The results indicate that the XGBOOST model developed in this study can be used to improve TC intensity forecast accuracy and can be considered a better alternative to conventional operational forecast models for TC intensity prediction.

1. Introduction

A tropical cyclone (TC) is an inherent atmospheric feature of tropical and subtropical regions. TCs generate strong winds and waves, and also often include heavy rain and storm surges that result in significant damage to coastal communities. The frequency of TCs in the Western North Pacific (WNP) is very high compared to other regions [1]. As a result, coastal communities around the WNP suffer serious casualties and property losses annually [2,3,4]. The area covered by the Northwest Pacific Basin is to the north of the equator and to the west of 180° E, and includes the South China Sea. The forecast skill of TC intensity in this region has displayed no signs of significant improvement by using numerical simulation methods in recent years [5]. Thus, this study focuses on improving the forecast skill of TC intensity in the WNP.

The characteristics that affect TC intensity are nonlinear and thus difficult to predict [2]. In recent years, many researchers have studied TC intensity prediction primarily using numerical forecasting and statistical methods [6,7,8,9,10]. Numerical forecasting is the main tool used to forecast TCs around the world, and systems such as the European Centre for Medium-Range Weather Forecasts-Integrated Forecasting System (ECMWF-IFS) [11], the Japan Meteorological Agency’s global spectral model (JMA-GSM) [12], and the National Centers for Environmental Prediction-Global Forecast System (NCEP-GFS) [13] have been developed as operational techniques. Statistical intensity forecast methods such as Climatology and Persistence (CLIPER) [14] and the Statistical Hurricane Intensity Prediction Scheme (SHIPS) [15] have also been developed as operational techniques. However, advancements in TC intensity prediction have been relatively slow, although notable work has been done in the past 20 years on predicting TC paths [16]. One of the reasons for this is that the internal structure of storms is not yet sufficiently understood. TCs have an asymmetric structure, which may be caused by thermal and dynamic factors such as uneven distribution of sea surface temperature and humidity, horizontal or vertical shear, and asymmetric distribution of convection [2,17]. Changes in TC intensity are controlled by many environmental and oceanic variables, including oceanic heat, vertical wind shears, and underlying surfaces changes [9]. Because these factors exhibit nonlinear characteristics, TC intensity is difficult to predict accurately using traditional statistical and numerical forecasting methods [2]. Thus, conventional linear statistical methods have difficulties when applied to nonlinear systems (e.g., forecasting TC intensity). At present, artificial neural network (ANN) methods can be used to effectively solve complex nonlinear issues, and numerous studies have concentrated on novel modeling techniques to improve the accuracy of TC intensity predictions [18,19,20].

In recent years, studies have clearly showed the potential of machine learning systems such as back propagation neural networks (BPNNs) to forecast TC intensity [18]. However, the traditional BPNN is unstable and less accurate than a probabilistic neural network (PNN) [2], which is a radial basis function neural (RBFN) network that is strongly fault-tolerant, and has adaptive capabilities. The k-nearest-neighbor (K-NN) machine learning algorithm has also been applied to predict TC intensity based on microwave imager data [21]. A comparison of TC intensity prediction accuracy using multilayer perceptron (MLP), multiple linear regression, RBFN, and ordinary linear regression indicated that MLP had the smallest prediction error [22]. The MLP model may thus also be considered an alternative to the conventional operational forecast models for predicting TC intensity [23]. Other research predicted cyclone wind intensity in the South Pacific using Elman recurrent neural networks [24] and proved that the accuracy of TC intensity forecasts using a double hidden layer neural network is higher than that found by using a single layer neural network [25]. These previous models for predicting TC intensity have therefore been popular with researchers.

Although these models have been used widely by many researchers to forecast TC intensity, each method has unique shortcomings in various areas, including predictive accuracy, model interpretability, and computational efficiency. For example, BPNN performs well for most of these standards; however, the accuracy of BPNN model predictions does not satisfy the needs of operational applications. Therefore, more accurate TC intensity prediction models are needed. Several researchers have stated that the extreme gradient boosting (XGBOOST) machine learning method, which is based on the gradient boosting decision tree (GBDT), is a promising classification model [26]. GBDT is an iterative decision tree algorithm comprising multiple decision trees, with the sum of the results of all decision trees constituting the final result. It is frequently applied to solve classification and regression problems [27]. In the process of gradient boosting machine model operation, the objective of every iteration is to reduce the residuals of the previous iteration. In order to eliminate the residual, the gradient boosting machine model builds a new model in the direction of the gradient of residual reduction. However, the objective function of the GBDT has no penalty term and may therefore be overfitted. The error function of the GBDT is the first derivative, which has a slow convergence speed. Based on the GBDT model, the XGBOOST model has the following changes: (1) Penalty terms are incorporated into the objective function of XGBOOST to prevent overfitting; and (2) the error function of XGBOOST is the second derivative, which can converge faster on the training set. These advantages and the fact that it has a very strong predictive ability and has recently been successful in many machine learning competitions led us to choose the XGBOOST model to forecast TC intensity in the WNP. Sheridan et al. used the XGBOOST model to quantify structure-activity relationships, with fast convergence speed being exhibited [28]. The XGBOOST model has also been used to predict agricultural crop yields [29]. In addition, it provides the following advantages: (1) It utilizes parallelism, is easy to use, and has impressive prediction accuracy; and (2) it has an intrinsic capability to manage the highly diverse and complex features of predictors. The structure of a TC and the factors affecting its development are very complicated; the XGBOOST model is highly suited for such conditions.

In this study, XGBOOST-based frameworks for TC intensity prediction in the WNP were established based on China Meteorological Administration-Shanghai Typhoon Institute (CMA-STI) data for the period 1979–2017. These data were used because they include significant amounts of coastal sounding data, which are useful for analyzing TCs in the WNP. Analyses of the dynamics of the development of TC intensity were conducted by comparing the differences between the predicted and real data. Predicting TC intensity using the XGBOOST model not only results in a series of predictors, but also improves the accuracy of the predictions. The primary objectives of this study were to (1) extract various features for predicting TC intensity from CMA-STI data; (2) establish a TC dataset (1979–2017) based on machine learning and explore XGBOOST frameworks for predicting TC intensity with lead times of 6, 12, 18, and 24 h; and (3) demonstrate that this method outperforms several baseline techniques. The results of this study provide a novel and feasible method with which to improve TC intensity prediction accuracy.

2. Data and XGBOOST Model

In this section, we present the framework for TC intensity prediction in the WNP based on XGBOOST. The ECMWF Re-Analysis (ERA) Interim and CMA-STI best track datasets, which are used to evaluate the performance of our method, are described in Section 2.1. The processing methods and the performance of our framework are reviewed in Section 2.2, and the performance of the XGBOOST model is discussed in Section 2.3.

2.1. Data

The TC intensity information in the CMA TC best-track dataset was determined using meteorological satellite cloud image data and estimated using the Dvorak technique [30]. The CMA-STI Best Track Dataset for the WNP from 1979–2017 was downloaded from the China Meteorological Administration (http://www.typhoon.gov.cn/). The dataset included latitude, longitude, 2-min mean maximum sustained wind near the center of the TC, intensity category, and minimum pressure near the center of the TC. Moreover, the WNP TCs in this study are defined as TCs that either have passed through or were generated in the WNP region. The TC had to have had a minimum life history of 48 h.

The environmental datasets were produced by the European Centre for Medium-Range Weather Forecasts (ECMWF) [31]. The spatial and temporal resolutions of the data are 1° × 1° and four times daily, respectively. From the ERA Interim dataset, we selected variables including the relative humidity, temperature, divergence, and u- and v-wind at 200, 250, 300, 350, 400, 450, 500, 700, 750, 775, 800, 825, and 850 hPa. We selected the sea surface temperature from the ERA Interim dataset to estimate the maximum potential intensity (MPI).

The TC samples from 1979–2005 were selected during the training process, whereas the TC samples from 2006–2017 were selected during the testing phase.

Table 1. Numbers of training samples and testing samples of TC intensity prediction experiment for 1979–2017 over the WNP.

Lead Time (h)	Training Samples (1979–2005)	Testing Samples (2006–2017)
6	20114	7708
12	19269	7383
18	18429	7058
24	17594	6735

lists the number of samples at 6, 12, 18, and 24 h, all of which were divided into two: a training sample and a testing sample. The experimental process was mainly divided into four parts: preprocessing data, building predictors, tuning parameters, and running XGBOOST. Figure 1 illustrates the process of the experiment.

2.2. Potential Predictors

Data preprocessing: We collected data comprising time, latitude, longitude, 2-min mean maximum sustained wind near the TC center, intensity category, and minimum pressure near the TC center from the CMA-STI Best Track Dataset, multiplied the latitude and longitude data by 0.1, and deleted the samples of the nulls in the dataset.

We used four types of potential predictors, including persistence and climatology factors, environmental factors, brainstorm features, intensity categories, and TC months to establish the WNP intensity model.

The CLIPER method, which has been employed extensively in TC tracking and intensity forecasting methods, was preferentially selected based on the climatology and persistence method [32]. The persistence factors were based on the past 24 h motion of the TC, while the climatology factors were from the meteorological elements of the TC. The 20 persistence factors (PFs) are listed in

Table 2. Detailed description of the 20 persistence factors. TC: tropical cyclone.

Predictors	Description
T1, T2, T3, T4	Latitude, longitude, minimum pressure near the TC center, and 2-min mean maximum sustained wind near the TC center at the current time
T5, T6, T7, T8	Latitude, longitude, minimum pressure near the TC center, and 2-min mean maximum sustained wind near the TC center 6 h prior
T9, T10, T11, T12	Latitude, longitude, minimum pressure near the TC center, and 2-min mean maximum sustained wind near the TC center 12 h prior
T13, T14, T15, T16	Latitude, longitude, minimum pressure near the TC center, and 2-min mean maximum sustained wind near the TC center 18 h prior
T17, T18, T19, T20	Latitude, longitude, minimum pressure near the TC center, and 2-min mean maximum sustained wind near the TC center 24 h prior

and the 52 climatology factors (CFs) are listed in

Table 3. Detailed description of the 52 climatology factors.

Predictors	Description
V1, V2, V3, V4	Difference between the latitude at the current time and the latitude 6, 12, 18, and 24 h prior
V5, V6, V7, V8	Difference between the longitude at the current time and the longitude 6, 12, 18, and 24 h prior
V9, V10, V11, V12	Difference between the minimum pressure near the TC center at the current time and the minimum pressure near the TC center 6, 12, 18, and 24 h prior
V13, V14, V15, V16	Difference between the 2-min mean maximum sustained wind near the TC center at the current time and the 2-min mean maximum sustained wind near the TC center 6, 12, 18, and 24 h prior
V17, V18, V19, V20	Zonal velocity between the current time and the 6, 12, 18, and 24 h prior
V21, V22, V23, V24	Meridional velocity between the current time and the 6, 12, 18, and 24 h prior
V25, V26, V27, V28	Synthetic speed 1 of movement between the current time and the 6, 12, 18, and 24 h prior
V29, V30, V31, V32	Zonal acceleration between the current time and the 6, 12, 18, and 24 h prior
V33, V34, V35, V36	Meridional acceleration between the current time and the 6, 12, 18, and 24 h prior
V37, V38, V39, V40	Synthetic acceleration 2 between the current time and the 6, 12, 18, and 24 h prior
V41, V42, V43, V44	Zonal displacement between the current time and the 6, 12, 18, and 24 h prior
V45, V46, V47, V48	Meridional displacement between the current time and the 6, 12, 18, and 24 h prior
V49, V50, V51, V52	Resultant displacement between the current time and the 6, 12, 18, and 24 h prior

¹ For example, in the expression $\mathrm{V}25=\sqrt{\mathrm{V}{17}^2+\mathrm{V}{21}^2}$, V25 represents the synthetic speed of movement between the current time and the 6 h prior, V17 is the zonal velocity between the current time and the 6 h prior, and V21 is the meridional velocity between the current time and the 6 h prior. ² For example, in the expression $\mathrm{V}37=\sqrt{\mathrm{V}{29}^2+\mathrm{V}{33}^2}$, V37 represents the synthetic acceleration between the current time and the 6 h prior, V29 is the zonal acceleration between the current time and the 6 h prior, and V33 is the meridional acceleration between the current time and the 6 h prior.

.

Feature engineering is a superset of activities that include feature extraction and feature selection. Each of these is an important step and none should be ignored. We could generalize the importance; from our experience, the relative importance of the steps would follow the order feature construction > feature extraction > feature selection [33]. To extract features from raw data without considering their importance for the time being, brainstorm features (BFs) correspond to feature construction. Brainstorming refers to a spontaneous group discussion conducted with the objective of solving a problem or formulating good ideas. To accurately predict TC intensity, we extracted several key features from a large number of studies in the literature. Potential predictors in the Statistical Typhoon Intensity Prediction Scheme (STIPS) include several quadratic terms and cosine function [34]. Thus, the BFs in the XGBOOST model include cosine of the latitude at the current time, square of the 2-min mean maximum sustained wind near the TC center at the current time, cube of 2-min mean maximum sustained wind near the TC center at the current time, and so on. The 59 BFs are listed in

Table 4. Detailed description of the 59 brainstorm features.

Predictors	Description
B1	Cosine of the latitude at the current time
B2, B3, B4, B5	Cosine of the latitude 6, 12, 18, and 24 h prior
B6, B7, B8, B9	Differences between B1 and B2, B1 and B3, B1 and B4, and B1 and B5
B10	Square of the 2-min mean maximum sustained wind near the TC center at the current time
B11, B12, B13, B14	Square of the 2-min mean maximum sustained wind near the TC center 6, 12, 18, and 24 h prior
B15	Logarithm value of B10
B16, B17, B18, B19	Logarithm value of B11, B12, B13, and B14
B20	Cube of 2-min mean maximum sustained wind near the TC center at the current time
B21, B22, B23, B24	Cube of 2-min mean maximum sustained wind near the TC center 6, 12, 18, and 24 h prior
B25	Ratio of minimum pressure near the center of the TC center and 2-min mean maximum sustained wind near the TC center at the current time
B26, B27, B28, B29	Ratio of minimum pressure near the TC center and 2-min mean maximum sustained wind near the TC center 6, 12, 18, and 24 h prior
B30	1/B25
B31, B32, B33, B34	1/B26, 1/B27, 1/B28, 1/B29
B35	Ratio of minimum pressure near the TC center and B10
B36, B37, B38, B39	Ratio of minimum pressure near the TC center and B11, B12, B13, B14
B40	1/B35
B41, B42, B43, B44	1/B36, 1/B37, 1/B38, 1/B39
B45	Ratio of minimum pressure near the TC center and B20
B46, B47, B48, B49	Ratio of minimum pressure near the TC center and B21, B22, B23, B24
B50	1/B45
B51, B52, B53, B54	1/B46, 1/B47, 1/B48, 1/B49
B55	Logarithm value of minimum pressure near the TC center at the current time
B56, B57, B58, B59	Logarithm value of minimum pressure near the TC center 6, 12, 18, and 24 h prior

.

To assess the potential impact of moisture, wind fields, and SST on TC intensity, we selected the environmental factors listed in

Table 5. Detailed description of the 24 environmental factors.

Predictor	Description
E1	Maximum potential intensity
E2	E1 squared
E3	E1 cubed
E4	MPI times 2-min mean maximum sustained wind
E5	Area-averaged 850–700-hPa relative humidity within 200–800 km of TC center
E6	Area-averaged 500–300-hPa relative humidity within 200–800 km of TC center
E7	Area-averaged 200-hPa zonal wind within 200–800 km of TC center
E8	Area-averaged 200-hPa temperature within 200–800 km of TC center
E9	Area-averaged 200-hPa divergence within 1000 km of TC center
E10	Area-averaged 500–850-hPa wind shear within 200–800 km of TC center
E11	E10 squared
E12	E10 cubed
E13	Area-averaged 200–850-hPa wind shear within 200–800 km of TC center
E14	E13 squared
E15	E13 cubed
E16	Area-averaged 500–850-hPa zonal wind shear within 200–800 km of TC center
E17	E16 squared
E18	E16 cubed
E19	Area-averaged 200–850-hPa zonal wind shear with 200–800 km of TC center
E20	E19 squared
E21	E19 cubed
E22	E13 times sine of the latitude
E23	E10 times sine of the latitude
E24	Area-averaged 850-hPa relative vorticity within 1000 km of TC center

. The treatment of the environmental factors was similar to those of DeMaria and Knaff [34,35]. The MPI was calculated according to the method described by DeMaria and Kaplan [36]. The SST selected from the ERA Interim dataset was interpolated to the TC center following the best track from 1979–2017 to determine the relationship between the SST and the intensity. Then the exponential MPI function was described by ($\mathrm{MPI}=\mathrm{A}+{\mathrm{B}\mathrm{e}}^{\mathrm{C}\left(\mathrm{T}-{\mathrm{T}}_0\right)}$). The results were A = 18.42 m/s, B = 51.47 m/s, C = 0.09687 °C⁻¹, ${\mathrm{T}}_0$ = 30.0 °C. The maximum value of MPI is 80 m/s. In addition to the environmental factors mentioned by DeMaria and Knaff, we added a cubed term to the environmental factors.

Because the TCs in different seasons have different characteristics, we chose the TC month as a predictor. Intensity category (IC) predictors contain five moments: the current time, past 6 h, past 12 h, past 18 h, and past 24 h. We constructed all the features as shown in

Table 6. Potential climatology, persistence, and environmental factors.

Predictors	Description
PF	Persistence factors (20)
CF	Climatology factors (52)
BF	Brainstorm features (59)
MON	TC month
IC	Intensity category
EF	Environmental factors (24)

.

2.3. Performance for the XGBOOST Model

2.3.1. XGBOOST Model Function

XGBOOST was established by Chen et al. [26]. The XGBOOST model has been applied to predict energy consumption [37] and traffic at intersections [38], and to search for parsimonious solutions [39]. The XGBOOST model combines M Classification and Regression Trees (CART) $\left\{T_1\left(x_i,y_i\right)\cdots T_M\left(x_i,y_i\right)\right\}$, where $x_i$ is a given training set of predictors related to a TC for predicting the intensity ($y_i$) in the future:

$$\stackrel{`}{y_i}={{\displaystyle \sum }}_{m=1}^M{f}_m\left(x_i\right),f_m\in F$$

(1)

where $f_m$ is a tree and $F$ represents the space of the entire CART. The optimization regularization objective is obtained by Equation (2):

$$obj\left(\theta \right)={{\displaystyle \sum }}_i^{n}l\left(y_i,\stackrel{`}{y_i}\right)+{{\displaystyle \sum }}_m^M\tau \left(f_m\right)$$

(2)

where $l$ represents the differentiable loss function that shows the difference between the target $y_i$ and predicted $\stackrel{`}{y_i}$. $\tau$ represents a regularization parameter. This variable is designed to prevent a complex model structure and to avoid overfitting. $\tau \left(S_m\right)=\phi N+\frac{1}{2}\theta {{\displaystyle \sum }}_{j-1}^T{\alpha }_j^2$, where $N$ represents the number of leaf nodes and $\alpha$ is the score of the leaf node. $\phi$ and $\theta$ are used to describe the level of regularization. In addition to using regularization term, predictor subsampling was used to prevent overfitting [26].

The prediction process adds the results of each tree to obtain the final results in the XGBOOST model. We need to determine the parameters of each tree ($f_t$), which includes the structure of the tree and the scores obtained by each leaf node. The additive training method adds the result of a tree to the model at a given time. The predicted value (${\stackrel{`}{y}}_i^{\left(t\right)}$) obtained in step t can be used to obtain the algorithm process:

$${\stackrel{`}{y}}_i^{\left(t\right)}={{\displaystyle \sum }}_{m=1}^M{f}_m\left(x_i\right)={\stackrel{`}{y}}_i^{\left(t-1\right)}+f_t\left(x_i\right).$$

(3)

The optimal tree is selected to achieve the optimization effect in each step:

$$obj{\left(\theta \right)}^{\left(t\right)}={{\displaystyle \sum }}_i^{n}l\left(y_i,{\stackrel{`}{y}}_i^{\left(t-1\right)}+f_t\left(x_i\right)\right)+\tau \left(f_t\right).$$

(4)

Second-order Taylor expansion occurs in the upper form:

$$obj{\left(\theta \right)}^{\left(t\right)}={{\displaystyle \sum }}_i^{n}l\left(y_i,{\stackrel{`}{y}}_i^{\left(t-1\right)}\right)+g_i{f}_t\left(x_i\right)+\frac{1}{2}{h}_i{f}_t^2\left(x_i\right)+\tau \left(f_t\right),$$

(5)

where $g_i={\partial }_{{\stackrel{`}{y}}_i^{(t-1)}}l(y_i,{\stackrel{`}{y}}_i^{(t-1)})$ shows the first derivative on the loss function, and $h_i={\partial }_{{\stackrel{`}{y}}_i^{(t-1)}}^{2}l(y_i,{\stackrel{`}{y}}_i^{(t-1)})$ represents the second derivative on the loss function. By deleting the constants, the equation for step t is obtained as follows:

$$obj{\left(\theta \right)}^{\left(t\right)}={{\displaystyle \sum }}_i^n{g}_i{f}_t\left(x_i\right)+\frac{1}{2}{h}_i{f}_t^2\left(x_i\right)+\tau \left(f_t\right).$$

(6)

The regularization term ($\tau \left(f_t\right)=\phi N+\frac{1}{2}\theta {{\displaystyle \sum }}_{j-1}^T{\alpha }_j^2$) is added to the cost function. Substituted into the objective function, it is sorted out as follows:

$$\begin{array}{l}obj{\left(\theta \right)}^{\left(t\right)}={{\displaystyle \sum }}_{i=1}^n{g}_i{f}_t\left(x_i\right)+\frac{1}{2}{h}_i{f}_t^2\left(x_i\right)+\phi N+\frac{1}{2}\theta {{\displaystyle \sum }}_{j-1}^T{\alpha }_j^2\\ ={{\displaystyle \sum }}_{j=1}^N\left[\left({\displaystyle \sum }_{i\in I_j}{g}_i\right)a_j+\frac{1}{2}\left({\displaystyle \sum }_{i\in I_j}{h}_i+\theta \right){\alpha }_j^2\right]+\phi N\end{array}$$

(7)

The larger the parameters $\theta$ and $\phi$, the better is the tree. $I_j=\left\{i|q\left(x_i\right)=j\right\}$ is the instance dataset of leaf node $j$, $q\left(x\right)$ is the optimal leaf node weight, while ${\alpha }_j^{*}$ is shown as the optimal objective function:

$${\alpha }_j^{*}=-\frac{G_j}{H_j+\theta }$$

(8)

$$ob{j}^{*}=-\frac{1}{2}{{\displaystyle \sum }}_{j=1}^N\frac{G_j}{H_j+\theta }+\phi N$$

(9)

where $G_j={\displaystyle \sum }_{i\in I_j}{g}_i$ and $H_j={\displaystyle \sum }_{i\in I_j}{h}_i$.

It is difficult to achieve this result in practical application. Therefore, we choose one layer of the tree to optimize, calculate the gain before and after node splitting, and select the point with the greatest gain as the splitting point. In the XGBOOST algorithm, if a node is divided into two leaf nodes, the fractional gain is as follows:

$$\mathrm{Gain}=\frac{1}{2}\left[\frac{G_L^2}{H_L+\phi }+\frac{G_R^2}{H_R+\phi }-\frac{{\left(G_L+G_R\right)}^2}{H_L+H_R+\phi }\right]-\phi$$

(10)

where $I_L$ and $I_R$ show the instance dataset of the left and right nodes after the split, $G_L={\displaystyle \sum }_{i\in I_L}{g}_i$, $G_R={\displaystyle \sum }_{i\in I_R}{g}_i$, $H_L={\displaystyle \sum }_{i\in I_L}{h}_i$, $H_R={\displaystyle \sum }_{i\in I_R}{h}_i$, and $I=I_L\cup I_R$.

It is very difficult to choose the best predictor from among the many factors affecting TC intensity. However, the XGBOOST model is capable of this task. It has a wide range of adjustable parameters. We limited the scope of this experiment to using RStudio to execute XGBOOST. The eta parameter reduces the weight of the features to make the calculation process more conservative, prevent overfitting, and use the shrinkage step in the update process. The gamma parameter is the minimum loss reduction required to make a further partition on a leaf node of the tree. The max_depth parameter represents the maximum depth of a child tree. The min_child_weight parameter shows the minimum sum of the instance weight needed in a child. Subsample represents the ratio of the observed subsamples. The colsample_bytree parameter represents the ratio of the variables used to construct each tree.

2.3.2. Training Method

In this study, we selected different input factor combinations to build six TC intensity prediction models and found that the XGBOOST model resulted in the most accurate prediction (

Table 7. List of test cases with datasets.

Models	Input Parameter	Output Parameter
A1	PF, IC, BF	Intensity at lead times of 6, 12, 18, and 24 h.
A2	PF, IC, BF, CF
B1	PF, IC, BF, MON
B2	PF, IC, BF, MON, CF
C1	PF, IC, BF, MON, EF
C2	PF, IC, BF, MON, EF, CF

). In model A1, the input parameters are PF, IC, and BF from 1979–2017. In model A2, the input parameters are PF, IC, BF, and CF predictors from 1979–2017. In model B1, the input parameters are PF, IC, BF, and MON from 1979–2017. In model B2, the input parameters are PF, IC, BF, MON, and CF predictors from 1979–2017. In model C1, the input parameters are PF, IC, BF, MON, and EF from 1979–2017. In model C2, the input parameters are PF, IC, BF, MON, EF, and CF predictors in 1979–2017.

Because parameter settings are very important for running XGBOOST, we used the expand.grid() function to select the best combination of parameters. We obtained the best execution for XGBOOST when eta, gamma, max_depth, min_child_weight, subsample, and colsample_bytree were (0.01, 0.1, 1), (0.1, 0.5, 0.8), (2, 4, 6, 8), (2, 4, 8), 0.8, and 0.95 respectively. This setting resulted in 108 combinations of parameters. For the 1979–2005 training samples, ten-fold cross-validation methods [40] were used to obtain the best combination of parameters.

2.3.3. Evaluating the capability of the XGBOOST models

We selected the correlation coefficient (CC), mean absolute error (MAE), and normalized root mean square error (NRMSE; as a percentage) as the parameters with which to evaluate the capability of the XGBOOST models during the training and testing phases.

$$\mathrm{NRMSE}=\sqrt{\frac{1}{n}{{\displaystyle \sum }}_{i=1}^{i=n}{\left({\theta }_{obs,i}-{\theta }_{fore,i}\right)}^2}/\left({\theta }_{obs,max}-{\theta }_{obs,min}\right)$$

(11)

$$\mathrm{CC}={{\displaystyle \sum }}_{i=1}^{i=n}\left({\theta }_{obs,i}-{\overline{\theta }}_{obs}\right)\left({\theta }_{fore,i}-{\overline{\theta }}_{fore}\right)/\sqrt{{{\displaystyle \sum }}_{i=1}^{i=n}{\left({\theta }_{obs,i}-{\overline{\theta }}_{obs}\right)}^2{{\displaystyle \sum }}_{i=1}^{i=n}{\left({\theta }_{fore,i}-{\overline{\theta }}_{fore}\right)}^2}$$

(12)

$$\mathrm{MAE}=\frac{1}{n}{\displaystyle \sum }_{i=1}^{i=n}\left|{\theta }_{obs,i}-{\theta }_{fore,i}\right|$$

(13)

where ${\theta }_{obs,i}$ is the observed value at the ith sample, ${\theta }_{fore,i}$ is the forecasted value at the ith sample, n is the number of all predicted samples, ${\overline{\theta }}_{obs}$ is the average observation, and ${\overline{\theta }}_{fore}$ is the average value of the forecast.

3. Experimental Results of XGBOOST Models

In the XGBOOST model experiments, the real-time WNP intensities for 6-, 12-, 18-, and 24-h prediction were studied. The TC intensity variable is expressed by the 2-min mean maximum sustained wind near the TC center. The TC intensity in the WNP is considered the prediction object. As WNP TCs located near the coastal regions of China produce negative effects annually, the WNP was given particular consideration. TCs in the WNP include those developed by the South China Sea. All TCs in the WNP were selected and considered as test objects to establish a forecast program for TC intensity. The XGBOOST model was established to provide a new method with which to predict TC intensity using various samples beneficial for improving the robustness of the model.

3.1. Prediction Results of XGBOOST Models

As Figure 2 shows, the learning performance of the XGBOOST model is very good during the testing phase. The CCs between the predicted values and the observed values are greater than 0.89. In the testing phases, the CCs of the A1 and A2 models at the 24 h lead time are minimal compared to those of other models. As the A1 model has only PF, IC, and BF factors, the TC intensity characteristics cannot be fully displayed. The results in Figure 2 generally indicate that all six models successfully predicted TC intensity in the WNP at lead times of 6, 12, 18, and 24 h.

Figure 3 illustrates the MAEs of the TC intensity at 6, 12, 18, and 24 h lead time when the XGBOOST model is used. According to the statistics, the MAEs of 6 to 24 h forecast for the A1-model test samples are 1.77, 2.75, 3.64, and 4.41 m/s, respectively. The MAEs of 6 to 24 h forecast for A2-model test samples are 1.77, 2.71, 3.56, and 4.35 m/s, respectively. The MAEs of 6 to 24 h lead time forecast for B1-model test samples are 1.69, 2.58, 3.38, and 4.09 m/s, respectively. The MAEs of 6 to 24 h lead time prediction for B2-model test samples are 1.67, 2.57, 3.35, and 4.06 m/s, respectively. The MAEs of 6 to 24 h lead time forecast for C1-model test samples are 1.64, 2.50, 3.24, and 3.91 m/s, respectively. The MAEs of 6 to 24 h lead time forecast for C2-model test samples are 1.61, 2.44, 3.10, and 3.70 m/s, respectively. In all models, MAE increases with the prediction time. The MAE ranges of the As, Bs, and Cs models are below 4.50 m/s.

3.2. TC Intensity Prediction Accuracy

The main task of TC intensity prediction is to determine whether future forecasts of intensity will be accurate. According to the CMA-STI, the strength levels for most TCs in the WNP are Super Typhoon (SuperTY; wind speed of TC center ≥ 51.0 m/s), Strong Typhoon (STY; 41.5 m/s ≤ wind speed of TC center ≤ 50.9 m/s), Typhoon (TY; 32.7 m/s ≤ wind speed of TC center ≤ 41.4 m/s), Strong Tropical Storm (STS; 24.5 m/s ≤ wind speed of TC center ≤ 32.6 m/s), and Tropical Storm (TS; 17.2 m/s ≤ wind speed of TC center ≤ 24.4 m/s) while a few TCs are Tropical Depression-level (TD; 10.8 m/s ≤ wind speed of TC center ≤ 17.1 m/s). In general, forecasting TCs that are at the SuperTY, STY, TY, STS, STS, and TS levels are important because they are the strongest TCs in the WNP and result in more serious disasters. Four TCs were selected to represent the TC intensity of the different grades: Hato from 2017, Rammasum from 2014, Mujiage from 2015, and Hagupit from 2014. These TCs resulted in huge economic and property loss.

3.2.1. Six-Hour (6 h) Lead Time

The 6 h lead time intensity predictions of the TCs Hato, Rammasum, Mujiage, and Hagupit are shown in Figure 4. In these figures, models A, B and C predicted trends that are similar to the observations, but the models forecasted the TCs with a slight shift at the peak along the observations. Figure 4 shows that models A1, A2, B1, and B2 forecasted TC intensities with larger oscillations than the observations. Overall, the predicted values of models C1 (trained by the PF, IC, BF, MON, and EF) and C2 (PF, IC, BF, MON, EF, and CF) are more consistent with the observed values than models A and B. For the observed and predicted intensities for the lead times from 6 to 24 h, we found that the results of the forecast not only satisfy the required forecast but are also in agreement with them within the time domain.

3.2.2. Twelve-Hour (12 h) Lead Time

The 12 h lead time intensity predictions of TCs Hato, Rammasum, Mujiage, and Hagupit are shown in Figure 5. In these figures, the results of TC intensity predictions from models C1 and C2 are roughly the same as the observations, but there are slight differences at the peak. Models B1 and B2 predicted TC intensities with slight oscillations along the observations; they forecasted TCs with a slight shift at the peak along the observations. Figure 5 shows that models A1 and A2 forecasted TC intensity with oscillations larger than those in the observations. Overall, the predictions from models C1 (trained by the PF, IC, BF, MON, and EF) and C2 (PF, IC, BF, MON, EF, and CF), are generally more consistent with the observed values when contrasted with the predictions from models A1, A2, B1, and B2.

3.2.3. Eighteen-Hour (18 h) Lead Time

The 18 h lead time predictions of the intensity of TCs Hato, Rammasum, Mujiage, and Hagupit are shown in Figure 6. In this figure, the predictions of TC intensity from models C1 and C2 are roughly coincident with the observations, but there are slight differences at the peak and turnaround. Models B1 and B2 predicted TC intensities with slight oscillations along the observations, and specifically forecasted the TCs with a slight shift at the peak along the observations. Figure 6 shows that models A1 and A2 forecasted TC intensities with larger oscillations than the observations. Overall, the predicted values of models C1 (trained by the PF, IC, BF, MON, and EF) and C2 (PF, IC, BF, MON, EF, and CF) are close to the observed values.

3.2.4. Twenty-Four-Hour (24 h) Lead Time

The predictions of the intensity of TCs Hato, Rammasum, Mujiage, and Hagupit, with a 24 h lead time, are shown in Figure 7. Overall, the forecasted TC intensities oscillated significantly in models A1, A2, B1, and B2. However, models C1 and C2 predicted relatively stable TC intensities compared to the TC intensities predicted by models A1, A2, B1, and B2. The trends in the predicted TC intensities from models C1 (trained by the PF, IC, BF, MON, and EF) and C2 (PF, IC, BF, MON, EF, and CF), are closest to the red line when compared with the predictions from models A1, A2, B1, and B2. Although the predicted values from models C1 and C2 are consistent with the overall trend in the observations, there is turbulence in some places, such as where TC Hato was rapidly declining. In models A and B, we found that the forecasted TC intensities diverged from the observations in the TCs Hato, Rammasum, and Mujiage.

3.3. Comparison of the Different Models

To evaluate the performance of the XGBOOST model with 6 h lead time, we used two parameters: CC and NRMSE. The results of the two indices are shown in Figure 8a. They show that the CC range in models A, B, and C is 0.98–0.99 and that of the NRMSE is 3.00–3.50%. Among these models, that with the largest CC value and the smallest NRMSE value for TC intensity with a 6 h lead time is model C2, which includes the variables PF, IC, BF, MON, EF, and CF. In addition, the results show that the combination dataset increased the accuracy of the XGBOOST model for the 6 h forecast. The results for a 12 h lead time are shown in Figure 8b. We found that the CC values in models A, B, and C ranged from 0.95–0.97 and that their NRMSE values ranged from 4.50–5.50%. Among models A, B, and C, the best combination of predictors for TC intensity with a 12 h lead time is model C2, with PF, IC, BF, MON, EF, and CF. In addition, the results show that the combination dataset should enhance the precision of the XGBOOST model for the 12 h forecast. The performance of the XGBOOST model with an 18 h lead time is shown in Figure 8c. We found that the CC values in models A, B, and C ranged from 0.92–0.95 and that the NRMSE values ranged from 6.00–7.02%. For models A, B, and C, model C2, with PF, IC, BF, MON, EF, and CF, produced the most accurate TC intensity predictions with an 18 h lead time. The results of the XGBOOST model with a 24 h lead time are shown in Figure 8d. The CC values of most of the models are distributed in the range 0.89–0.94 and the NRMSE values in the range 7.00–8.50%. Model C2, which was trained by the TC intensity parameter datasets PF, IC, BF, MON, EF, and CF, indicated the best prediction capability: the CC and NRMSE are 0.93 and 7.18%, respectively.

4. Discussion

In order to determine the impact of different factors on the forecast results, we designed six models: A1 (PF, IC, BF), A2 (A1 + CF), B1 (A1 + MON), B2 (A2 + MON), C1 (B1 + EF), and C2 (B2 + EF). The research results indicate that when climatology and persistence factors, brainstorm and predictive features, intensity category, and TC month are used as model inputs, the best TC intensity forecast accuracy with 6, 12, 18, and 24 h lead times occurs in the six models. We theorize that there are three main reasons for this.

Firstly, we found that the XGBOOST model with the TC month feature is better than one without it because of differing relationships between the predictors and TC intensity in each month, which made it necessary for monthly experiments to be performed. Huang et al. divided all the samples into five parts and built a monthly ANN model (June–October). They found that the results of the monthly ANN model were better than those of the CLIPER method for TC intensity forecasts [41]. Several researchers consider the months from July to November as the period in the year with the most TC activity [42]. In this study, for the first group, the B2 model with the predictor TC month with a 24 h lead time had good MAE, CC, and NRMSE results: 4.06 m/s, 0.91, and 7.84%, respectively. The A2 model without the TC month predictor and 24 h lead time had the following results: MAE of 4.35 m/s, CC of 0.90, and NRMSE of 8.31%. For the second group, when we trained the PF, IC, BF, and MON datasets (B1 model), the MAE of the 24 h forecast for the test samples was 4.09 m/s; when we trained the PF, IC, and BF datasets (A1 model), the MAE of the 24 h forecast for the test samples was 4.41 m/s. Thus, the month feature is very important to process TC predictions. Secondly, our next finding was that an XGBOOST model with environmental factors is better than one without those features. For the first group, when it was trained with the PF, IC, BF, MON, and EF datasets, the MAE of the 24 h forecast for the test samples was 3.91 m/s; when trained with the PF, IC, BF, and MON datasets, the same MAE was 4.09 m/s. For the second group, the MAE of the 24 h forecast in model B2 for the test samples was 4.06 m/s; the MAE of 24 h forecast in model C2 for the same samples was 3.70 m/s. Finally, climatology predictors also affect the forecast results from the XGBOOST model. The climatology predictors developed by Neumann [43] are feasible and have reasonable bases in meteorology [44]. For example, such predictors include: latitude and longitudes at present and 12, 24, 36 and 48 h prior, pressures at present and 12, 24, 36 and 48 h prior, directions of the storm motions at present and 12, 24, 36 and 48 h prior. In addition to the above information, when we established the climatology predictors, we added the difference between the latitude at the current time and the latitude 6, 12, 18, and 24 h prior, the difference between the longitude at the current time and the latitude 6, 12, 18, and 24 h ago, the difference between the 2-min mean maximum sustained wind near the center of the TC at the current time and the latitude 6, 12, 18, and 24 h prior, etc. (Table 3). The MAE for the 24 h forecast in model A1 for the test samples was 4.41 m/s, while that value in model A2 was 4.35 m/s. The MAE of 24 h forecast in model B1 for the test samples was 4.09 m/s, while the MAE of the 24 h forecast in model B2 for the test samples was 4.06 m/s. The MAE of the 24 h forecast in model C1 for the test samples was 3.91 m/s, while that value was 3.70 m/s in model C2.

As the prediction time increases, the accuracy of the TC intensity predicted using the XGBOOST model gradually decreases. The result indicates that the MAE and NRMSE values gradually increase with prediction time, whereas the value of CC gradually decreases. For example, the CCs of the 6, 12, 18, and 24 h forecasts for test samples in model C1 were 0.99, 0.97, 0.95, and 0.93, respectively. The NRMSEs of these forecasts were 3.15, 4.86, 6.25, and 7.59, respectively, and the MAEs were 1.64, 2.50, 3.24, and 3.91 m/s, respectively. However, the reduction/growth amplitude of each model in the CC/NRMSE differs as the prediction time increases. The relationship between the CC and NRMSE for TC intensity forecast with a 6, 12, 18, and 24 h lead time is shown in Figure 8. The results show that the distance between the C models and the A models in the graph increase with the prediction time. On combining Figure 8a–d, it can be seen that the C models approach the lower right-hand corner with increase in lead time. For example, the CCs of the 6, 12, 18, and 24 h forecasts for the test samples in model C2 were 0.99, 0.97, 0.95, and 0.93, respectively. The NRMSEs of these forecasts were 3.09, 4.72, 6.00, and 7.18, respectively. This means that the CCs of the C models approach one and the NRMSEs approach zero with increasing lead time. When all parts of Figure 8 are combined, the A models approach the top left-hand corner with increase in lead time. For example, the CCs of the 6, 12, 18, and 24 h forecasts for the test samples in model A2 were 0.98, 0.96, 0.93, and 0.90, respectively, whereas their NRMSEs were 3.39, 5.28, 6.87, and 8.31, respectively. This means that the CCs of the A models have lower correlations, with values much less than one, whereas the NRMSEs are higher.

The prediction results show that the XGBOOST method is appropriate for predicting TC intensities.

Table 8. Mean absolute error of tropical cyclone intensity for several forecast methods with a 24 h lead time.

Method	Input Parameter	MAE (m/s)
A1 (XGBOOST model)	PF, IC, BF	4.41
A2 (XGBOOST model)	PF, IC, BF, CF	4.35
B1 (XGBOOST model)	PF, IC, BF, MON	4.09
B2 (XGBOOST model)	PF, IC, BF, MON, CF	4.06
C1 (XGBOOST model)	PF, IC, BF, MON, EF	3.91
C2 (XGBOOST model)	PF, IC, BF, MON, CF, EF	3.70
A3 (back propagation neural network (BPNN) model)	PF, IC, BF	5.48
A4 (BPNN model)	PF, IC, BF, CF	5.25
B3 (BPNN model)	PF, IC, BF, MON	5.41
B4 (BPNN model)	PF, IC, BF, MON, CF	5.23
C3 (BPNN model)	PF, IC, BF, MON, EF	5.05
C4 (BPNN model)	PF, IC, BF, MON, CF, EF	4.57

shows the MAEs of the maximum wind speed forecasts with a 24 h lead time using each method. The prediction error of the XGBOOST model is comparable to that of a previously used model for 24 h predictions. The MAEs for a 24 h lead time in seven machine learning methods—k-nearest neighbor [21], neural network [45], fuzzy neural network [46], artificial neural network [5], multilayer feed forward neural nets [25], Elman recurrent network [24], and probabilistic neural network [2]—were determined for test samples and found to be 8.22, 3.44, 3.52, 4.74, 2.98, 3.58, and 2.93 m/s, respectively. Because the predictors used in the previous studies are not exactly the same as the predictors in this paper, we used a BPNN to predict the TC intensity under the same sample input parameter. The MAEs for a 24 h lead time in six BPNN methods—A3 (input parameters the same as model A1), A4 (input parameters the same as model A2), B3 (input parameters the same as model B1), B4 (input parameters the same as model B2), C3 (input parameters the same as model C1), and C4 (input parameters the same as model C2)—were determined for test samples and found to be 5.48, 5.25, 5.41, 5.23, 5.05, and 4.57 m/s, respectively. The MAEs of 24 h lead time in six models (A1, A2, B1, B2, C1, and C2) for the test samples were 4.41, 4.35, 4.09, 4.06, 3.91, and 3.70 m/s, respectively. The prediction results of the XGBOOST models were better than those made by the BPNN model with the same sample requirements. The XGBOOST model has many advantages, including a simple training process, low computer-processing costs, and fast convergence, compared to ANNs [47]. Therefore, using the XGBOOST model is very advantageous for predicting TC intensity. This significant finding thus supports performing predictions within 24 h using the XGBOOST model as a new method for TC intensity prediction.

5. Conclusions

In this study, we established a series of predictors using the Best Track TC dataset to predict the intensity of TCs in the Western North Pacific (WNP) and conducted the following experiments to improve intensity prediction accuracy. First, we designed a feature set using the brainstorming and CLIPER methods. Then, we used the CMA of the WNP near China as a data source and predicted the 6, 12, 18, and 24 h intensities of TCs for the period 1979–2017 under six scenarios using the XGBOOST model. Finally, we tested the performance of the XGBOOST model using the strongest recent TCs—specifically, Hato, Rammasum, Mujiage, and Hagupit. The following results were obtained:

(1) The prediction accuracy of the XGBOOST model improved by climatology and persistence factors, environmental factors, brainstorm features, intensity category, and TC month. We analyzed the prediction of TC intensity using the XGBOOST model under six scenarios; all of them produced a mean absolute error (MAE) < 4.50 m/s, a correlation coefficient (CC) > 0.89, and a normalized root mean square error (NRMSE) < 10.00%. Among models A (A1 and A2), B (B1 and B2), and C (C1 and C2), we determined that model C2 was the most accurate predictor of TC intensity in the six scenarios.

(2) The NRMSE, MAE, and CC parameters were used to evaluate the performance of the XGBOOST model in the WNP. The MAEs of the 6, 12, 18, and 24 h lead times for the test sample forecasts were 1.61, 2.44, 3.10, and 3.70 m/s, respectively; the CCs were 0.99, 0.97, 0.95, and 0.93, respectively; and the NRMSEs were 3.09, 4.72, 6.00, and 7.18%, respectively. The MAE and NRMSE values gradually increased with lead time, whereas the CC value gradually decreased. The prediction accuracy of our XGBOOST model was found to be higher than that of traditional BPNN models for the same predictors and independent prediction samples.