**3. Principles of the XGBoost Algorithm**

XGBoost is an improved Gradient Boosting Decision Tree (GBDT) algorithm, and there is a big difference between them. GBDT uses only the first derivative in optimization, while XGBoost uses both the first and second derivatives. Moreover, XGBoost uses the tree model complexity as a regular term in the objective function to avoid overfitting [50].
