Elastic Modulus Prediction of Ultra-High-Performance Concrete with Different Machine Learning Models

Abstract

Elastic modulus, crucial for assessing material stiffness and structural deformation, has recently gained popularity in predictions using data-driven methods. However, research systematically comparing different machine learning models under the same conditions, especially for ultra-high-performance concrete (UHPC), remains limited. In this study, 10 different machine learning models were evaluated for their capacity to predict the elastic modulus of UHPC. The results showed that XGBoost demonstrated the highest accuracy in predictions with large training datasets, followed by KNNs. For smaller training datasets, Decision Tree exhibited the greatest accuracy, while XGBoost was the second-best performing model. Linear regression displayed the lowest accuracy. XGBoost demonstrated the most potential for accurately predicting the elastic modulus of UHPC, particularly when a comprehensive dataset is available for model training. The optimized XGBoost exhibited better predictive performance than fitting equations for different UHPC formulations. The findings of this study provide valuable insights for researchers and engineers working on the data-driven design and characterization of UHPC.

1. Introduction

Ultra-high-performance concrete (UHPC), as a new construction material, exhibits exceptional mechanical properties, including compressive strength, durability, and impact resistance [1,2]. The elastic modulus, a critical parameter that characterizes the stiffness of materials, is essential for evaluating the stiffness and creep of concrete structures. Measuring the mechanical strength of UHPC is relatively straightforward using standard concrete testing procedures. These tests are well established and are not overly sensitive to minor variations in UHPC mix design or testing conditions. However, determining the elastic modulus of UHPC is a more complex task. The stress–strain response of samples should be precisely measured, typically using advanced instrumentation [3,4]. The challenge arises from the fact that the elastic modulus is heavily influenced by the microstructural composition and packing of UHPC constituents. Subtle changes in the mix proportions or curing conditions can significantly affect the elastic behavior. Additionally, the stress–strain response of UHPC is nonlinear, even at low stress levels, due to its heterogeneous nature [5]. Accurately determining the tangent modulus, which represents the true elastic stiffness, requires careful analysis and interpretation of the stress–strain data.

To address the challenges in predicting the elastic modulus of UHPC, researchers have developed various codes and standards to establish relationships between compressive strength and elastic modulus [6,7,8,9]. Many researchers have proposed empirical models tailored to specific conditions. Ma et al. [10] predicted the elastic modulus of UHPC without and with coarse aggregates and suggested that the elastic modulus of UHPC is proportion to the 1/3 power of the compressive strength, which falls in a range of 150–180 MPa, and Association Française de Génie Civil (AFGC) obtained a similar relationship between elastic modulus and compressive strength for heat-cured UHPC when the compressive strength is larger than 140 MPa. Sritharan et al. [11] and Graybeal et al. [12,13] found that the elastic modulus of UHPC is proportional to the 1/2 power of the compressive strength. However, it is difficult to predict the elastic modulus of UHPC with different mix designs. Zhang et al. [14] proposed a new computational mesoscale model generation to predict the elastic modulus of concrete and achieved good prediction accuracy, but the computational cost of this model is significant, especially when multiple phases should be modeled. Ouyang et al. [15] developed an analytical model to analyze the effective elastic modulus of UHPC at different scales. However, accurately modeling this relationship is challenging, as variations in mix designs, raw materials, and curing methods across different regions can lead to different results.

On the other hand, machine learning has been successfully applied to various aspects of concrete, including mixture design [16,17,18], fiber pullout performance [19,20], mechanical performance [21,22,23,24], and durability [25,26], etc. For predicting the compressive strength of concrete, Naderpour et al. [27] used an artificial neural network (ANN) to assess recycled aggregate concrete (RAC), reporting regression values of 0.903 for training, 0.89 for validation, and 0.829 for testing, with optimal performance at epoch 5. Fan et al. [28] combined the Modified Andreasen and Andersen (MAA) model with a Genetic Algorithm-based ANN (GA-ANN) to study the compressive strength of UHPC. They developed an efficient graphical user interface (GUI), and their results indicated a correlation coefficient (R²) of over 0.95, with a mean squared error (MSE) of 0.004447. Khodaparasti et al. [29] employed an improved random forest method to investigate the compressive strength of concrete, achieving R², mean absolute error (MAE), and root mean square error (RMSE) values of 0.931, 3.21, and 4.71, respectively. Several models have also been developed to predict the elastic modulus of concrete. For example, Demir et al. [30] used artificial neural networks (ANNs) to predict the elastic modulus of normal-strength concrete (NSC) and high-performance concrete (HPC) and proved the validity of ANN compared with those obtained using empirical models. Yan et al. [31] predicted the elastic modulus of NSC and HPC with support vector machine (SVM) and showed a similar level of prediction errors to the ANN model but with a better generalization capability. Golafshani et al. [32] used four types of soft computing models, including ANN, fuzzy Takagi–Sugeno-Kang (TSK), support vector regression (SVR), and radial basis function neural network (RBFNN) to predict the elastic modulus of recycled aggregate concrete (RAC). The results suggested that ANN and SVR better exhibit the performance than that of fuzzy TSK and RBFNN. Golafshani et al. [32] compared three automatic regression models, that is, genetic programming (GP), artificial bee colony programming (ABCP), and biogeography-based programming (BBP), for the prediction of the elastic modulus of RAC and proved that BBP has the highest accuracy. As the model tree (MT) is more accurate than regression-based models, Behnood et al. [33] developed M5’s tree algorithm to analyze the elastic modulus prediction of RAC, demonstrating an 80% improvement in accuracy compared to other empirical models. Additionally, a new prediction model has been proposed to estimate the post-fire elastic modulus of circular recycled aggregate concrete-filled steel tubular stub columns [34]. It uses gene expression programming and experimental data from both heated and non-heated samples. The model improves accuracy in terms of predicting structural behavior after fire exposure and performs better than existing models, considering key factors like steel ratio, temperature, and material strength.

Despite significant advances in the use of machine learning models to predict the elastic modulus of various concrete types, a critical gap remains in understanding the comparative performance of these models, specifically for UHPC. Most existing studies have focused on individual machine learning approaches, with limited research systematically comparing different models under the same conditions, particularly for UHPC. It is important to note, however, that the superiority of machine learning models over traditional forecasting or prediction methods is not a foregone conclusion. The performance of any given method can vary significantly depending on the specific dataset and problem at hand. In this manuscript, we aim to optimize the ML model to achieve good predictive accuracy for various data. This research introduces the development of ten distinct automated machine learning models for predicting the elastic modulus of UHPC. To validate their effectiveness, a comprehensive analysis and comparison of the performance of these models were conducted. Section 2 details the machine learning models employed, while Section 4 offers an in-depth comparison of the prediction results, demonstrating the strengths and weaknesses of each model. The optimized XGBoost model was implemented to predict the elastic modulus of different types of UHPC in Section 5. The findings are then presented in Section 6, where the overall conclusions are drawn.

2. Methods

2.1. Linear Regression

Linear regression is generally used to predict a single dependent variable. It can establish the relationship between variables as either linear or nonlinear, such as concave, convex, or other arbitrary polynomial forms. The equation of linear regression can be expressed as follows [35]:

$$Y=w\cdot x+b+\epsilon$$

(1)

where Y is the sum of a “known” deterministic component, w is the coefficient and b represents an intercept, and $\epsilon$ represents the combined effect of potentially factors of y. As shown in Figure 1, to estimate the minimum error in general, the least squares method [36] is used, which is

$$\min {\displaystyle \sum _i(y-y_{1i}}{)}^2={\displaystyle \sum _i(y_i-(w\cdot x_i}+b){)}^2$$

(2)

To avoid over-fitting, a regularization term is introduced into Equation (2), for example, the LASSO minimization formulation can be expressed by

$$\min {\displaystyle \sum _i(y_i-(w\cdot x_i}+b){)}^2+c{\displaystyle \sum _i\mid w_j^2\mid }$$

(3)

Root mean squared error (RMSE) is used to evaluate the individuals as follows:

$$RMS{E}_i=\sqrt{{\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n{(y_i-y_{1i})}^2}}$$

(4)

2.2. Bayesian Ridge Regression

In the Bayesian Ridge Regression model [37], the minimum can be expressed as follows:

$${\arg }_{\beta }\min {\Vert y-X\beta \Vert }_2^2+\lambda {\Vert \beta \Vert }_2^2$$

(5)

In the Bayesian setting, the posterior distribution was estimated using Bayes theorem [38]. For Ridge regression, normal likelihood and normal prior need to be set as the parameters. After dropping the normalizing constant, the log-density function of the normal distribution is as follows:

$$\begin{gathered} \log \mathrm{p}(x|\mu ,\sigma )=\log \left[{\displaystyle \frac{1}{\sigma \sqrt{2\pi }}}{e}^{-{\displaystyle \frac{1}{2}}{\left({\displaystyle \frac{x-\mu }{\sigma }}\right)}^2}\right] \\ =\log \left[{\displaystyle \frac{1}{\sigma \sqrt{2\pi }}}\right]+\log \left[e^{-{\displaystyle \frac{1}{2}}{\left({\displaystyle \frac{x-\mu }{\sigma }}\right)}^2}\right] \\ \propto -{\displaystyle \frac{1}{2}}{\left({\displaystyle \frac{x-\mu }{\sigma }}\right)}^2 \\ \propto -{\displaystyle \frac{1}{{\sigma }^2}}{\Vert x-\mu \Vert }_2^2 \end{gathered}$$

(6)

Hence, maximizing the normal log likelihood with normal priors is equivalent to minimizing the squared loss with a ridge penalty:

$$\begin{gathered} {\arg }_{\beta }\max \log N(y\mid X\beta ,\sigma )+\log N(0,\tau ) \\ ={\arg }_{\beta }\min -\{\log N(y\mid X\beta ,\sigma )+\log N(0,\tau )\} \\ ={\arg }_{\beta }\min {\displaystyle \frac{1}{{\sigma }^2}}{\Vert y-X\beta \Vert }_2^2+{\displaystyle \frac{1}{{\tau }^2}}{\Vert \beta \Vert }_2^2 \end{gathered}$$

(7)

2.3. ARD Regression

Automatic Relevance Determination (ARD) Regression [39], considering that each weight has an individual variance, is as follows:

$$p(w|\lambda )~N(0,{\Lambda }^{-1})$$

(8)

where $\Lambda =diag({\lambda }_1,\dots ,{\lambda }_H)$, ${\lambda }_H\in R_{+}$ are the features of the parameters. The minimization problem can be expressed as follows:

$${\min }_w\alpha {\Vert {\varphi }_w-t\Vert }^2+w^2\Lambda w$$

(9)

ARD Regression is a sparse Bayesian learning method. In contrast to Bayesian Ridge Regression, each coordinate of $w_i$ has its own individual ${\lambda }_i$. ARD Regression adapts well when few features are relevant.

2.4. RANSAC Regression

Random Sample Consensus (RANSAC) Regression, which was proposed by Fischler et al. [40], is implemented as follows [41]:

(1)

A minimum number of random points are selected to determine the model parameters.

(2)

Solve the parameters of the model.

(3)

Determine the number of points from the set of all points that fit within a predefined tolerance.

(4)

Otherwise, repeat steps 1 through 4 (a maximum of N times). RANSAC estimates the parameters of a mathematical model through iterations to analyze a set of data containing outliers. Therefore, it can also be explained as an approach to outlier detection.

2.5. ANN Regression

The linear regression feature can be expressed by

$$y_i=h(x_i,w)=w^T{x}_i$$

(10)

The least squares error or loss of data is given by

$$L(w)={\displaystyle \sum i{(h(x_i,w)-y_i)}^2}$$

(11)

In ANN regression [42] (as shown in Figure 2), $x_i^1$, $x_i^2$ represent the features, and $w_1$, $w_2$ denote the weights; the prediction can be computed by the sums. Gradient descent is used to minimize the error in the training data, that is,

$${\nabla }_{w}L(w)=({\displaystyle \frac{\partial L(w)}{\partial w_1}},{\displaystyle \frac{\partial L(w)}{\partial w_2}})=({\displaystyle \sum {x_1}^{^}}{\displaystyle \sum i}2x(2)ih(x_i,w))$$

(12)

The update of w is given by

$$w=w-\eta {\nabla }_{w}L(w)$$

(13)

where $\eta$ represents the step size. After a set number of epochs, the weights $w$ tend to convergence and obtain the best fit. Different activation functions, including unit step (threshold), sigmoid, piecewise linear, Gaussian, etc., are used to transform input data to output data.

2.6. Support Vector Regression

In SVR, Vapnik [43] introduced slack variables ξ, ξ* to the constraints of the optimization problem, as shown in Figure 3. The minimization function of SVR is formulated as follows:

$$\min {\displaystyle \frac{1}{2}}\Vert \omega {\Vert }^2+C{\displaystyle \sum _{i=1}^n({\xi }_i+{\xi }_i^{*})}$$

(14)

s.t.

$$\{\begin{array}{c}{y}_i-f(x_i,\omega )\le \epsilon +{\xi }_i^{*}\\ f(x_i,\omega )-y_i\le \epsilon +{\xi }_i\\ {\xi }_i,{\xi }_i^{*}\ge 0,i=1,\dots ,n\end{array}$$

(15)

In addition, a loss function called $\epsilon$-insensitive loss function, which was proposed by Vapnik [43], is used:

$$L_{\epsilon }(y,f(x,\omega ))=\{\begin{array}{cc}0& if\mid y-f(x,\omega )\mid \le \epsilon \\ \mid y-f(x,\omega )\mid -\epsilon & otherwise\end{array}$$

(16)

$$\mathrm{The} \mathrm{empirical} \mathrm{risk}:R_{emp}(\omega )={\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n{L}_{\epsilon }(y_i,f(x_i,\omega ))}$$

(17)

where $\epsilon$ is the insensitive loss that coincides with the least-modulus loss ($\epsilon =0$) [43], which is used to estimate the prediction performance of SVR with various noise densities. This specifies the kernel types, such as “linear”, “poly”, and “gauss rbf”, used to transform the data into linear forms.

2.7. Decision Tree Regression

Decision Tree regression [44] is used to fit a curve with additional noise, and it is similar to a local linear regression approximation model. To implement the Decision Tree regression, based on the data features, the data can be split into decision nodes, which are determined by mathematical entropy. For example, as shown in Figure 4, based on the selection of the critical entropy, the output nodes (R1 to R5) can be obtained based on the Decision Tree nodes.

2.8. Random Forest Regression

To overcome the weakness of the generalization ability of Decision Trees, random forest regression was developed [45], and the process is shown in Figure 5. In step 1, k random data points were selected from the training set x. In step 2, a Decision Tree was built at each of the k data points. In step 3, the number of N-trees to build was chosen, and steps 1 and 2 were repeated. In step 4, after training, the final predictions can be made by averaging the predictions from all individual regression trees.

2.9. XGBoost

XGBoost [46], which is a fast Gradient Boosted Decision Trees algorithm [47], is commonly used in Kaggle competitions. To learn the model with a set of functions, a minimized regularized objective is used, as follows:

$$Ob{j}^{\left(t\right)}={\displaystyle \sum _{i=1}^{n}l\left(y_i,{\widehat{y}}_i^{\left(t-1\right)}+f_t\left(x_i\right)\right)+\Omega \left(f_t\right)}$$

(18)

where l represents a differentiable convex loss function prediction, and ${\widehat{y}}_i,y_i$ are the prediction and target values, respectively. Ω is the complexity of the model, and f_t means the regression tree. According to the Taylor expansion, Equation (11) can be rewritten as follows:

$$Ob{j}^{\left(t\right)}={\displaystyle \sum _{i=1}^n\left[l\left(y_i,{\widehat{y}}_i^{\left(t-1\right)}\right)+g_i{f}_t\left(x_i\right)+\frac{1}{2}{h}_i{f}_t^2\left(x_i\right)\right]+\Omega \left(f_t\right)}$$

(19)

where $g_i={\partial }_{\widehat{y}\left(t-1\right)}l\left(y_i,{\widehat{y}}^{\left(t-1\right)}\right)$ and $h_i={\partial }_{\widehat{y}\left(t-1\right)}^{2}l\left(y_i,{\widehat{y}}^{\left(t-1\right)}\right)$, and by removing the constant term, Equation (12) can be simplified as follows:

$$Ob{j}^{\left(t\right)}={\displaystyle \sum _{i=1}^n\left[g_i{f}_t\left(x_i\right)+\frac{1}{2}{h}_i{f}_t^2\left(x_i\right)\right]+\Omega \left(f_t\right)}$$

(20)

The complexity of the tree is defined as follows:

$$\Omega \left(f_t\right)=\gamma T+{\displaystyle \frac{1}{2}}{\displaystyle \sum _{j=1}^T{w}_j^2}$$

(21)

The final objective function is expressed by

$$Ob{j}^{\left(t\right)}=-{\displaystyle \frac{1}{2}}{\displaystyle \sum _{j=1}^T\frac{G_j^2}{H_j+\lambda }}+\gamma T$$

(22)

where $G={\displaystyle {\sum }_{i\in I_j}{g}_i}$, $H={\displaystyle {\sum }_{i\in I_j}{h}_i}$, and $I_j=\left\{i\mid q\left(x_i\right)=j\right\}$, in which q is the tree structure. A greedy algorithm is used to add branches to the tree.

2.10. KNNs

K-nearest neighbors (KNNs) is a non-parametric regression model [48], and the prediction is the average of the values of the K-nearest neighbors. The KNN algorithm uses similar features to predict the new data points. In detail, at first, the distance between the new point and all training data points is calculated, and then, based on the distance, the closest k data points are selected. Finally, the prediction is computed by the average of these data points. In addition, three distance functions, including the Euclidean distance, Manhattan distances, and Hamming distance, are selected to calculate the distance between the new point and the training data points. These three distance functions are expressed as follows:

$$\mathrm{Euclidean}: \sqrt{{\displaystyle {\sum }_{i=1}^k{\left(x_i-y_i\right)}^2}}$$

(23)

$$\mathrm{Manhattan}: \sqrt{{\displaystyle {\sum }_{i=1}^k\mid x_i-y_i\mid }}$$

(24)

$$\begin{gathered} \mathrm{Hamming}: D_H={\displaystyle {\sum }_{i=1}^k\mid x_i-y_i\mid } \\ x=y\Rightarrow D=0 \\ x\ne y\Rightarrow D=1 \end{gathered}$$

(25)

To obtain the best prediction, it is important to choose an optimal k kernel. In general, a large k value obtains a better prediction due to less noise; however, the compromise is that the distinct boundaries within the feature space are blurred. Instead, cross-validation enables a way to determine a good k value by using an independent dataset to validate the k value.

3. Dataset Preparation and Strategies to Address Overfitting

3.1. Dataset Preparation

All models were implemented as regression models, with the dataset consisting of discrete values for both the features and target variables. The error was quantified using relative errors, and the R² score was employed to evaluate the performance of the optimized XGBoost model. The dataset used for training the machine learning models in this study was sourced from previous works [49,50,51]. The data were collected from multiple experimental studies focused on the compressive strength of concrete. Each data point represents specific concrete samples tested under varying conditions, including different mix proportions, curing times, and testing environments. The compressive strength values in the dataset span a wide range, typically from 100 MPa to 220 MPa. The dataset was normalized by Z-score normalization, which is a technique used to normalize data so that it has a mean (average) of 0 and a standard deviation of 1, and expressed by

$$Z={\displaystyle \frac{x-\mu }{\sigma }}$$

(26)

where, Z, X, $\mu$, and $\sigma$ represent Z-score, original value, mean, and standard deviation of the dataset, respectively. And R² score, also known as the coefficient of determination, was used to evaluate the performance of the optimized XGBoost in Section 5.

3.2. Strategies to Address Overfitting

The mitigation of overfitting in machine learning models is achieved through the application of various techniques, including regularization in linear models, Bayesian Ridge, ARD, SVR, and XGBoost; data sampling in RANSAC and Random Forest; pruning in Decision Trees; and the selection of optimal neighbors in KNNs. Additionally, methodologies such as dropout and early stopping are employed in ANN to enhance model robustness and generalization across different datasets.

4. Results and Discussion

Machine learning models were applied to predict the elastic modulus of UHPC, which was given by [49] based on a sufficient amount of data. The number of training data points was set to 50. Figure 6 and Figure 7 show the elastic modulus predictions and the corresponding relative errors for each model. It can be seen that XGBoost performs the best prediction, and the linear regression shows the worst prediction when the compressive strength is less than 200 MPa, while the Decision Tree has the lowest accuracy compared to other models when the compressive strength is greater than 200 MPa. SVR has a similar prediction to linear regression and lower accuracy than other models. RANSAC regression exhibits better accuracy than the other linear models (Linear, Bayesian Ridge, and ARD regression). Random forest shows a more stable prediction than Decision Tree. Other models, including ANN, KNNs, SVR-poly, and SVR-rbf, exhibited the same level of prediction accuracy. Through machine learning, it can be observed that the increase in the elastic modulus of UHPC tends to stabilize when the compressive strength reaches 197 MPa. In further, in experimental data, the elastic modulus is 62.1 GPa at a compressive strength of 206 MPa, obviously greater than the elastic modulus of 55.9 GPa at a compressive strength of 205 MPa, and the prediction at a compressive strength of 206 MPa shows a large error, as shown in Table S1 (SI).

Another case of using a small data sample given by [50] to predict the elastic modulus of UHPC with machine learning is shown in Figure 8 and Figure 9. In this case, all data were selected as training data, and the results indicated that the Decision Tree showed the lowest relative error because all data were the tree nodes. Next is XGBoost, and the linear models including linear, Bayesian Ridge, ARD, and regression perform with lower prediction accuracy compared with other models. Hence, the linear model is unsuitable for predicting the elastic modulus of UHPC. ANN, SVR-linear, and Random Forest exhibited instability in terms of accuracy. The relative errors of SVR-poly, SVR-rbf, and KNNs were lower than 2%, as listed in Table S2 (SI).

Random forest regression, ANN regression, and the K-nearest neighbor algorithm have unstable prediction errors and large deviations in the prediction results. Among these seven machine learning models, linear regression has the worst prediction accuracy, and the second worst is SVR-linear. This also indicates that the elastic modulus of ultra-high-performance concrete has a nonlinear relationship with compressive strength, and it is difficult to accurately predict the elastic modulus of UHPC using linear regression-like models for small sample data.

Based on sufficient data, the fitting equation for the elastic modulus of UHPC was as follows:

$$\mathrm{Data} \mathrm{fitting}: E_{\exp eri}=-1860f_c^{-1.6}+59.23$$

(27)

$$\mathrm{XGBoost} \mathrm{predicted} \mathrm{fitting}: E_{\exp eri}=-1974f_c^{-1.6}+59.62$$

(28)

Based on small sample data, the fitting equation for the elastic modulus of UHPC is as follows:

$$\mathrm{Data} \mathrm{fitting}: E_{\exp \mathrm{ri}}=3.326\sqrt{f_c}+7.106$$

(29)

$$\mathrm{XGBoost} \mathrm{predicted} \mathrm{fitting}: E_{predi}=3.432\sqrt{f_c}+5.686$$

(30)

The experimental data, XGBoost predicted data, and fitted curves are shown in Figure 10. It can be seen that XGBoost can predict the elastic modulus of different types of UHPC very well. For the prediction of the elastic modulus of UHPC with large sample data, the difference between the predicted and tested values in the range of a compressive strength of 205–208 MPa is large, which is due to the deviation of the test data in this compressive strength range, where the same compressive strength corresponds to several different values of the elastic modulus and thus affects the prediction. The difference in the relationship between the modulus of elasticity and compressive strength of the UHPCs is mainly due to the different compositions of the UHPC.

5. Optimization of XGBoost Prediction

To determine the validation of XGBoost, data were collected from previous works [49,51] (see Table S3 (SI)). Different specimen names were converted to numbers ranging from 0 to 7, as shown in S3 (SI). Hyperopt was implemented to optimize XGBoost. The hyperparameter selection and the best parameters are listed in

Table 1. Hyperparameters’ selection.

Hyperparameter	Range	Best Parameters
n_estimators	1000~3000	4300
learning_rate	0.001~0.1	0.16
max_depth	3~10	6
min_child_weight	1~10	3.86
subsample	0.5~1.0	0.89
colsample_bytree	0.5~1.0	0.96
gamma	0~5	0.01
reg_alpha	0~10	1.35
reg_lambda	0~10	5.19

.

Figure 11 illustrates the performance of the XGBoost model in predicting the elastic modulus (in GPa), showing both prediction results and errors. Figure 11a compares the predicted and true elastic modulus values, with green points for the training set (R² = 0.9858) and blue points for the test set (R² = 0.8925). The red dashed line represents the ideal fit, with most points clustering around it, demonstrating strong prediction accuracy, although the test points exhibit more variability. Figure 11b displays the prediction errors (true—predicted) as a percentage, with the zero-error line in red. Both training and test errors are generally small, but test set errors increase slightly at higher modulus values, reflecting some overfitting on the training data. Overall, the model demonstrates high accuracy, particularly on the training data.

Figure 12 presents the feature importance and residuals distribution for the XGBoost model used to predict the elastic modulus. Figure 12a displays a bar chart of feature importance, where the feature “compressive strength” has a substantially higher F-score (664.0) compared to “specimen number” (220.0), indicating that compressive strength has a greater influence on the predictions of the XGBoost model. Figure 12b illustrates a histogram of residuals (true—predicted) for both the training and testing sets. Green bars represent the training errors, while blue bars denote the testing errors. The red dashed line marks zero error, indicating perfect prediction. The majority of errors are concentrated around zero for both datasets, with a slightly broader distribution in the test set, demonstrating that most predictions are accurate, although a few test residuals deviate beyond ±4.

In addition, Figure 13 illustrates the comparison of relative errors for different types of UHPC, including Traditional UHPC, CNF-enhanced UHPC, and economic UHPC, using the XGBoost model, Ref. Equations (3) and (4), as referenced in [51]. It is evident that the relative error for the three different UHPC types with XGBoost is lower than that of Ref. Equations (3) and (4). Similarly, for Traditional and CNF-enhanced UHPC, XGBoost demonstrates higher accuracy compared to the fitting equation in Ref. [51]. XGBoost exhibits a comparable level of accuracy to the fitting equation in Ref. [51] for economic UHPC. Additionally, the fitting equation in Ref. [51] requires the use of different fitting parameters for various UHPC types. In contrast, XGBoost implements optimized parameters to predict the elastic modulus for different UHPC types based on hyperparameter tuning with hyperopt. As a result, XGBoost demonstrates superior prediction accuracy for different UHPC types compared to the fitting equation. Further research could investigate the use of automated hyperparameter optimization and meta-learning techniques to streamline the process of selecting and tuning machine learning models for predicting UHPC properties, improving efficiency in practical applications.

6. Conclusions

In this work, 10 different machine learning models, including linear regression, ANN, SVR (linear, poly, and rbf), Decision Tree, Random Forest, XGBoost, and KNNs, were compared to predict the elastic modulus of UHPC. The conclusions are summarized as follows:

(1)

XGBoost demonstrated superior predictive performance for the elastic modulus of UHPC, especially when trained on large datasets. XGBoost outperformed other models, including linear regression, ANN, SVR (linear, poly, and rbf), Decision Tree, Random Forest, and KNNs, in terms of accuracy and stability.

(2)

Decision Tree provided the most accurate predictions when dealing with smaller datasets, while linear regression models consistently showed the lowest accuracy across various scenarios, highlighting limitations in capturing the nonlinear relationships present in UHPC data.

(3)

XGBoost can effectively predict the elastic modulus of UHPC regardless of the dataset size, and machine learning models can be used to generate fitting curves for target predictions in complex datasets. The fitting equations demonstrate that the XGBoost model is highly effective at predicting the relationship between compressive strength and elastic modulus in UHPC.