Optimal Dimensions of Post-Tensioned Concrete Cylindrical Walls Using Harmony Search and Ensemble Learning with SHAP

Abstract

The optimal design of prestressed concrete cylindrical walls is greatly beneficial for economic and environmental impact. However, the lack of the available big enough datasets for the training of robust machine learning models is one of the factors that prevents wide adoption of machine learning techniques in structural design. The current study demonstrates the application of the well-established harmony search methodology to create a large database of optimal design configurations. The unit costs of concrete and steel used in the construction, the specific weight of the stored fluid, and the height of the cylindrical wall are the input variables whereas the optimum thicknesses of the wall with and without post-tensioning are the output variables. Based on this database, some of the most efficient ensemble learning techniques like the Extreme Gradient Boosting (XGBoost), Light Gradient Boosting Machine (LightGBM), Categorical Gradient Boosting (CatBoost) and Random Forest algorithms have been trained. An R² score greater than 0.98 could be achieved by all of the ensemble learning models. Furthermore, the impacts of different input features on the predictions of different machine learning models have been analyzed using the SHapley Additive exPlanations (SHAP) methodology. The height of the cylindrical wall was found to have the greatest impact on the optimal wall thickness, followed by the specific weight of the stored fluid. Also, with the help of individual conditional expectation (ICE) plots the variations of predictive model outputs with respect to each input feature have been visualized. By using the genetic programming methodology, predictive equations have been obtained for the optimal wall thickness.

1. Introduction

Prestressing has well-known benefits in terms of increasing the load-carrying capacity and durability of concrete structures like liquid storage tanks, silos, and nuclear facilities [1]. It significantly reduces concrete cracks at the tension side by pre-compressing those parts of the concrete structures under tensile strain before the service loads are applied thereby countering the effects of tension caused by bending. Post-tensioning is a type of prestressing where the tendons are stressed after the concrete has been poured and hardened. Post-tensioning is a particularly favorable technique when the serviceability requirements on a large-scale structure cannot be met using conventional reinforcement techniques while keeping the dimensions of the structure in an economically feasible range [2].

Previous studies of the reinforced concrete structures optimization with and without prestressing have demonstrated the applicability of metaheuristic optimization techniques like the harmony search (HS) methodology to reduce the structural cost and environmental impact [2,3,4,5,6]. Metaheuristic optimization techniques have been applied to a variety of problems in civil and structural engineering including the dimensioning of trusses [7,8,9,10,11,12,13], retaining walls [14,15,16,17,18,19,20,21,22], laminated composite plates [23,24,25,26,27,28,29,30], and steel plate girders [31]. On the other hand, post-tensioning in structures is an area where metaheuristic optimization techniques found relatively few applications. The research in the area of structural post-tensioning can be grouped into studies related to beams [32], slabs [33,34,35], experimental studies [33,34,36], and numerical studies [32,34,35,36]. Elbelbisi et al. [32] performed a parametric study of post-tensioned fiber reinforced polymer (FRP) systems. The flexural behavior of beams externally post-tensioned with FRP tendons was investigated using finite element analysis. The application of external post-tensioning significantly increased the load-carrying capacity of the beams. Elsheshtawy et al. [33] investigated the effect of prestressing force and layout of strands on the punching shear strength of slab-column connections and found that the banded layout of the post-tensioning strands was most favorable to increase the punching shear strength of the post-tensioned flat slabs. Attia et al. [34] examined the load-carrying capacity of two-way flat slabs under post-tensioning with external FRP laminates using finite element analysis. A parametric study was carried out and it was shown that post-tensioning with FRP laminates increase the ductility. Furthermore, strengthening near the supports was shown to be more effective than in the middle of the slab. Tahmasebinia et al. [35] investigated post-tensioned concrete flat slabs under dynamic loading. According to finite element analysis, increasing the slab thickness and damping ratio is favorable towards better vibration serviceability. Vavrus and Kralovanec [36] investigated the application of steel fiber reinforced concrete in the anchorage zones of post-tensioning tendons to increase the load-carrying capacity. Based on numerical analysis, increasing the fiber content near the anchorage plate is favorable for the increase of the load carrying capacity. In some of the more recent studies in the area of post-tensioned structural members, bearing capacity of anchorage zones [37], precast post-tensioned girders [38], and post-tensioned self-compacting concrete beams with recycled coarse aggregate [39] have been experimentally and numerically investigated. Lei et al. [37] proposed a new formula for the bearing capacity of anchorage zones using the stress field approach. Joyklad et al. [38] carried out an experimental program with full-scale pre-cast post-tensioned (PCPT) girders accompanied with numerical simulations. Two PCPT girders of 29.95 m length, 1.8 m depth and 0.69 m width were manufactured. The upward deflection response of these girders, post-tensioning tendon strains, and reinforcement strains were measured with displacement transducers and strain gauges for 120 days. Furthermore, the thermal response of fresh concrete was measured using thermocouple wires. All measured parameters were observed to increase during post-tensioning. Yu et al. [39] carried out four-point bending tests on post-tensioned beams with and without recycled coarse aggregate. The specimens made with recycled coarse aggregate and self-compacting concrete were demonstrated to have similar flexural capacity as the specimens with natural coarse aggregate. To the best of the authors’ knowledge the only research studies involving the application of metaheuristic optimization techniques to post-tensioned reinforced concrete cylindrical containers was carried out by Bekdaş et al. [40]. It was found that the addition of multiple layers of post-tensioning cables reduces the amount of carbon emission associated with the manufacturing process. Harmony search (HS), teaching–learning based optimization (TLBO), and flower pollination algorithm (FPA) were utilized for minimizing the carbon footprint.

The current study aims to develop predictive models using ensemble machine learning techniques and genetic programming in order to predict the optimal value of wall thickness for a post-tensioned reinforced concrete cylindrical wall. In order to train these predictive models a large database of optimal design configurations has been generated using the harmony search algorithm. The statistical analysis of this data set as well as the harmony search methodology have been presented in Section 2 which also contains the details of the genetic programming and ensemble learning algorithms. Using the genetic programming algorithm, closed-form equations have been proposed for the prediction of the optimal wall thickness for cylindrical walls with and without post-tensioning. The performances of the ensemble learning and genetic programming algorithms in predicting the optimal wall thickness have been presented in Section 3. In the same section also, the input feature impact analysis has been carried out using SHAP and individual conditional expectation plots. The major novelty of the current paper is the proposal of a new methodology to overcome the data scarcity for the training of reliable predictive models using machine learning algorithms. Since reliable machine learning models necessitate large data sets which are not readily available in case of post-tensioned cylindrical walls, the current proposed technique can make a valuable contribution to this area.

2. Machine Learning and Optimization Methods

In this section, the procedures of building data sets of optimum post-tensioned cylindrical walls and developing machine learning (ML) models based on these data sets are explained. The database for the training of the ML models is generated using the harmony search optimization algorithm. In this process the serviceability requirements on the cylindrical wall such as the limitation on the crack size according to ACI 318 code [41] are considered as the constraints of the optimization. The structural analysis of the cylindrical walls has been carried out using the superposition method (SPM), which is based on St. Venant’s principle according to which stresses sufficiently far from an applied load are not significantly altered if this load is changed to a statically equivalent load combination [42].

$${\mathrm{M}}_{\mathrm{A}}+\frac{{\mathrm{P}}_{0\mathrm{A}}}{4\mathsf{\beta }}+\frac{{\mathrm{P}}_{0\mathrm{B}}}{4\mathsf{\beta }}{\mathrm{C}}_{\mathsf{\beta }\mathrm{l}}+\frac{{\mathrm{M}}_{0\mathrm{A}}}{2}+\frac{{\mathrm{M}}_{0\mathrm{B}}}{2}{\mathrm{D}}_{\mathsf{\beta }\mathrm{l}}=0,$$

(1)

$${\mathrm{Q}}_{\mathrm{A}}-\frac{{\mathrm{P}}_{0\mathrm{A}}}{2}+\frac{{\mathrm{P}}_{0\mathrm{B}}}{2}{\mathrm{D}}_{\mathsf{\beta }\mathrm{l}}-\frac{{\mathsf{\beta }\mathrm{M}}_{0\mathrm{A}}}{2}+\frac{\mathsf{\beta }{\mathrm{M}}_{0\mathrm{B}}}{2}{\mathrm{A}}_{\mathsf{\beta }\mathrm{l}}=0,$$

(2)

$${\mathrm{M}}_{\mathrm{B}}+\frac{{\mathrm{P}}_{0\mathrm{A}}}{4\mathsf{\beta }}{\mathrm{C}}_{\mathsf{\beta }\mathrm{l}}+\frac{{\mathrm{P}}_{0\mathrm{B}}}{4\mathsf{\beta }}+\frac{{\mathrm{M}}_{0\mathrm{A}}}{2}{\mathrm{D}}_{\mathsf{\beta }\mathrm{l}}+\frac{{\mathrm{M}}_{0\mathrm{B}}}{2}=0,$$

(3)

$${\mathrm{Q}}_{\mathrm{B}}-\frac{{\mathrm{P}}_{0\mathrm{A}}}{2}{\mathrm{D}}_{\mathsf{\beta }\mathrm{l}}+\frac{{\mathrm{P}}_{0\mathrm{B}}}{2}-\frac{\mathsf{\beta }{\mathrm{M}}_{0\mathrm{A}}}{2}{\mathrm{A}}_{\mathsf{\beta }\mathrm{l}}-\frac{\mathsf{\beta }{\mathrm{M}}_{0\mathrm{B}}}{2}=0,$$

(4)

$${\mathrm{A}}_{\mathsf{\beta }\mathrm{l}}={\mathrm{e}}^{-\mathsf{\beta }\mathrm{l}}\left(\cos \mathsf{\beta }\mathrm{l}+\sin \mathsf{\beta }\mathrm{l}\right),{\mathrm{B}}_{\mathsf{\beta }\mathrm{l}}={\mathrm{e}}^{-\mathsf{\beta }\mathrm{l}}\sin \mathsf{\beta }\mathrm{l}$$

(5)

$${\mathrm{C}}_{\mathsf{\beta }\mathrm{l}}={\mathrm{e}}^{-\mathsf{\beta }\mathrm{l}}\left(\cos \mathsf{\beta }\mathrm{l}-\sin \mathsf{\beta }\mathrm{l}\right),{\mathrm{D}}_{\mathsf{\beta }\mathrm{l}}={\mathrm{e}}^{-\mathsf{\beta }\mathrm{l}}\cos \mathsf{\beta }\mathrm{l}$$

(6)

The process of SPM has been visually depicted in Figure 1 where the beam in the top left shows a section of length L from an infinite wall, q is a distributed load representing the forces acting on the wall, ${\mathrm{M}}_{\mathrm{A}},{\mathrm{M}}_{\mathrm{B}},{\mathrm{Q}}_{\mathrm{A}},{\mathrm{Q}}_{\mathrm{B}}$ are the moments and shear forces at the positions A and B, respectively and ${\mathrm{P}}_0,{\mathrm{M}}_0$ at the bottom left portion of Figure 1 are the end conditioning forces and moments, respectively which should be statically equivalent and in opposite direction to ${\mathrm{M}}_{\mathrm{A}},{\mathrm{M}}_{\mathrm{B}},{\mathrm{Q}}_{\mathrm{A}},{\mathrm{Q}}_{\mathrm{B}}$ such that these moments and shear forces at the positions A,B are cancelled and the infinite wall is equivalent to a finite wall with free ends. The end-conditioning forces can be found by solving Equations (1) to (4). Further details of the SPM can be found in Hetenyi [43] and Bekdaş [2].

The post-tensioning forces are denoted with ${\mathrm{P}}_1$ to ${\mathrm{P}}_{\mathrm{n}}$ in the right side of Figure 1. The optimal distances of these forces from the ground as well as their optimal magnitudes were determined using the harmony search technique. The wall thickness, radius of the cylindrical wall and the height of the wall are denoted with t, r, and H, respectively. Using the methodology described in the previous paragraph, data sets of 1925 samples were generated for cylindrical walls with different numbers of post-tensioning loads. The data samples were generated by considering the total cost of the resulting structures and minimizing the objective function given in Equation (7).

$$\mathrm{f}\left(\mathbf{x}\right)={\mathrm{C}}_{\mathrm{c}}{\mathrm{V}}_{\mathrm{c}}+{\mathrm{C}}_{\mathrm{s}}{\mathrm{W}}_{\mathrm{s}}+{\mathrm{C}}_{\mathrm{p}\mathrm{t}}{\mathrm{W}}_{\mathrm{p}\mathrm{t}}+{\mathrm{C}}_{\mathrm{f}\mathrm{w}}{\mathrm{A}}_{\mathrm{f}\mathrm{w}}$$

(7)

In Equation (7), $\mathrm{f}\left(\mathbf{x}\right)$ denotes the function that outputs the total cost of the resulting structure, $\mathbf{x}$ denotes a vector containing the values for the wall dimensions and unit costs, and ${\mathrm{C}}_{\mathrm{c}},{\mathrm{C}}_{\mathrm{s}},{\mathrm{C}}_{\mathrm{p}\mathrm{t}},{\mathrm{C}}_{\mathrm{f}\mathrm{w}}$ are the cost of concrete per unit volume, cost of steel rebars per unit weight, cost of post-tensioning per unit weight and cost of formwork per unit area, respectively. The details of the objective function can be found in [2]. The database consisting of the optimal design configurations has been taken for the prediction of the optimal wall thickness. The height of the wall (H), the unit cost of concrete (${\mathrm{C}}_{\mathrm{c}}$), specific weight of the liquid ($\mathrm{\gamma }$), and unit cost of steel (${\mathrm{C}}_{\mathrm{s}}$), have been used as the input features defining the load cases. The following section shows the details of each data set used in the development of the predictive models.

2.1. Analysis of the Data Set

The data sets used in the prediction of the optimal wall thickness for the case with and without post-tensioning consist of 1908 and 1925 data samples, respectively. The data sets were generated using the harmony search algorithm to find the optimal wall thickness value corresponding to certain combinations of wall height, fluid specific weight and unit costs of concrete and steel. These data sets are split into a training set and a test set in 70% to 30% ratio. The ensemble learning models have been trained on the training set using the 10-fold cross-validation process. Afterward, the performances of these models have been measured on the test set which consists of data samples yet unseen by the models. In the 10-fold cross-validation process the training set is split into 10 disjoint subsets. The models are trained using 9 of these subsets while the tenth subset is being used for model validation. After 10 passes of the training set the best performing model is being selected which is used for performance evaluation on the test set.

The correlation plots in Figure 2 and Figure 3 show the Pearson correlation coefficients between different features in the upper right triangular parts. It can be seen that the wall height is the most correlated input feature with the wall thickness with correlation coefficients of 0.89 and 0.91 followed by the fluid specific weight ($\mathrm{\gamma }$) with correlation coefficients of 0.34 and 0.37. The strength of the correlation is also denoted by stars in Figure 2 and Figure 3. A Pearson correlation coefficient close to 1 would indicate a highly linear relationship between to variables. The computation formula for the Pearson correlation coefficient is given in Equation (8) where ${\mathrm{x}}_{\mathrm{i}}$, ${\mathrm{y}}_{\mathrm{i}}$ denote two data series of equal length, n is the length of these data series, and ${\mathrm{r}}_{\mathrm{x}\mathrm{y}}$ is the Pearson correlation coefficient between these two data series.

$${\mathrm{r}}_{\mathrm{x}\mathrm{y}}=\frac{\mathrm{n}\sum _{\mathrm{i}=1}^{\mathrm{n}}{\mathrm{x}}_{\mathrm{i}}{\mathrm{y}}_{\mathrm{i}}-\sum _{\mathrm{i}=1}^{\mathrm{n}}{\mathrm{x}}_{\mathrm{i}}\sum _{\mathrm{i}=1}^{\mathrm{n}}{\mathrm{y}}_{\mathrm{i}}}{\sqrt{\mathrm{n}\sum _{\mathrm{i}=1}^{\mathrm{n}}{\mathrm{x}}_{\mathrm{i}}^2-{\left(\sum _{\mathrm{i}=1}^{\mathrm{n}}{\mathrm{x}}_{\mathrm{i}}\right)}^2}\sqrt{\mathrm{n}\sum _{\mathrm{i}=1}^{\mathrm{n}}{\mathrm{y}}_{\mathrm{i}}^2-{\left(\sum _{\mathrm{i}=1}^{\mathrm{n}}{\mathrm{y}}_{\mathrm{i}}\right)}^2}}$$

(8)

In addition to the Pearson coefficient, Figure 2 and Figure 3 also display the distributions of different features. The diagonals of each plot contain frequency distributions of the features whereas the lower left triangular area contains bivariate scatter plots with regression lines. For each feature denoted on a diagonal tile, the value range of this feature is shown in both the horizontal and the vertical axis.

2.2. Harmony Search Algorithm

The harmony search (HS) algorithm was developed by Geem et al. [44] and has been used for solving numerous optimization problems in broad areas including the numerical solution of differential equations [45], project scheduling [46], weapon target assignment [47], law enforcement [48], internet of things [49], robotics [50], soil stability analysis [51], and structural design [52,53,54]. The HS method is based on the evolutionary improvement of an initially randomly generated population of optimal solution candidates denoted with $\mathrm{x}$ in Equations (10) and (11). The size of this population is denoted with HMS in Equation (9) which stands for harmony memory size. As the members of the population go through the HS iterations given in Equations (9) to (12), new and better-performing population members replace the members of the previous generations.

$$\mathrm{k}=\mathrm{i}\mathrm{n}\mathrm{t}\left(\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{d}·\mathrm{H}\mathrm{M}\mathrm{S}\right),\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{d}\in (0,1)$$

(9)

$${\mathrm{x}}_{\mathrm{i},\mathrm{n}\mathrm{e}\mathrm{w}}={\mathrm{x}}_{\mathrm{i},\mathrm{m}\mathrm{i}\mathrm{n}}+\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{d}·({\mathrm{x}}_{\mathrm{i},\mathrm{m}\mathrm{a}\mathrm{x}}-{\mathrm{x}}_{\mathrm{i},\mathrm{m}\mathrm{i}\mathrm{n}}),\mathrm{i}\mathrm{f}\mathrm{H}\mathrm{M}\mathrm{C}\mathrm{R}>\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{d}$$

(10)

$${\mathrm{x}}_{\mathrm{i},\mathrm{n}\mathrm{e}\mathrm{w}}={\mathrm{x}}_{\mathrm{i},\mathrm{k}}+\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{d}·\mathrm{P}\mathrm{A}\mathrm{R}·({\mathrm{x}}_{\mathrm{i},\mathrm{m}\mathrm{a}\mathrm{x}}-{\mathrm{x}}_{\mathrm{i},\mathrm{m}\mathrm{i}\mathrm{n}}),\mathrm{i}\mathrm{f}\mathrm{H}\mathrm{M}\mathrm{C}\mathrm{R}\le \mathrm{r}\mathrm{a}\mathrm{n}\mathrm{d}$$

(11)

$$\mathrm{H}\mathrm{M}\mathrm{C}\mathrm{R}=0.5\left(1-\frac{\mathrm{i}}{\max \left(\mathrm{i}\right)}\right),\mathrm{P}\mathrm{A}\mathrm{R}=0.05\left(1-\frac{\mathrm{i}}{\max \left(\mathrm{i}\right)}\right)$$

(12)

2.3. Ensemble Learning Algorithms

Ensemble learning methods are based on the idea of combining the predictions of multiple predictive models to obtain a strong learning algorithm. In this study XGBoost, Random Forest, LightGBM, and Catboost ensemble learning algorithms have been applied. The ensemble learning models have been trained using the scikit-learn library available for the Python programming language. The XGBoost algorithm iteratively generates decision trees while each newly generated tree corrects the errors of the previous trees. In these iterations the XGBoost algorithm aims at minimizing the objective function given in Equation (13) where M is the size of the training set, N is the number of decision trees, $\mathrm{l}$ is a loss function and $\mathrm{\Omega }$ is a penalty function. The XGBoost algorithm is summarized in Equation (14) where $\mathrm{g}$ denotes the strong learner model which is a linear combination of N weak learners denoted with ${\mathrm{g}}_{\mathrm{k}}$ and $\widehat{\mathrm{y}}$ is the model prediction [55].

$$\mathrm{L}=\sum _{\mathrm{i}=1}^{\mathrm{M}}\mathrm{l}\left({\mathrm{y}}_{\mathrm{i}},{\widehat{\mathrm{y}}}_{\mathrm{i}}\right)+\sum _{\mathrm{k}=1}^{\mathrm{N}}\mathrm{\Omega }\left({\mathrm{f}}_{\mathrm{k}}\right)$$

(13)

$$\mathrm{g}\left(\mathbf{x}\right)=\sum _{\mathrm{k}=1}^{\mathrm{N}}{\mathrm{g}}_{\mathrm{k}}\left(\mathbf{x}\right)=\widehat{\mathrm{y}}$$

(14)

The LightGBM, Random Forest, and CatBoost algorithms function on similar principals as the XGBoost algorithm by iteratively adding weak learner trees to the model and combining their output to build a strong learner model. The distinguishing feature of the LightGBM algorithm is its computational speed. The LightGBM algorithm implements techniques such as histogram-based algorithm, and Gradient-based One-Side Sampling (GOSS) to achieve improved performance [56]. On the other hand, the CatBoost algorithm stands out by its ability to work with categorical features more efficiently. The CatBoost algorithm incorporates ordered boosting, greedy method, and L2-regularization to improve model performance [57].

2.4. Genetic Programming

Genetic programming (GP) is a population based evolutionary algorithm. The algorithm generates a population of programs where each program is represented by a tree structure. Each node in these programs represents an operation (such as addition or multiplication) or a numerical value. Figure 4 shows an example of representing a program as a tree structure where nodes containing numerical values, unary operators and binary operators are shown in different colors. The fitness of each program is evaluated by running on a test set, and the programs that perform the best are selected for producing the next generation of programs through crossover and mutation [58,59,60]. The process of producing new programs from the existing ones using crossover and mutation operations is schematically explained in Figure 5. In the crossover part of Figure 5 the identical parts of the programs are shown inside rectangles of the same color whereas in the mutation part the mutated operator is shown inside a red rectangle.

3. Results

This section gives a detailed presentation of the predictive performances of the machine learning models. The predicted and actual optimal wall thickness values are plotted against each other and the percentage deviations of the predicted values from the actual thickness values have been shown. The prediction accuracies of ML models have been quantified using root mean squared error, mean absolute error, and the coefficient of determination (${\mathrm{R}}^2$). ML models have been trained on datasets of wall geometries with and without post-tensioning cables. Figure 6 shows the prediction of the optimal wall thickness as a function of the concrete and steel unit costs, liquid specific weight, and wall height for a cylindrical wall without post-tensioning. The wall radius has been fixed at a constant value in all the databases in this study since the liquid pressure on the wall is not affected by this quantity. It can be observed from Figure 6 that the predicted and actual optimal values for the thickness perfectly overlap.

The performances of four different ML models have been quantified and listed in

Table 1. Model accuracies in predicting the optimal wall thickness without post-tensioning.

Algorithm	R2		MAE		RMSE		Duration [s]
	Train	Test	Train	Test	Train	Test
XGBoost	0.9999	0.9999	0.0001	0.0002	0.0003	0.0003	4.81
Random Forest	0.9999	0.9999	10−5	3 × 10−5	9 × 10−5	0.0002	3.87
LightGBM	0.9999	0.9999	0.0005	0.0006	0.0012	0.0014	4.52
CatBoost	0.9999	0.9999	0.0002	0.0003	0.0002	0.0004	32.82
GP	0.9584	0.9573	0.0465	0.0460	0.0573	0.0570	359

. It can be observed that all of the ML models were able to predict the optimal wall thickness values with near-perfect accuracy since the ${\mathrm{R}}^2$ scores are close to 1 for all ML models. Table 1 also lists the performance values for the genetic programming model (GP). Clearly, in terms of both accuracy and computational efficiency the GP model performed worse than the ensemble learning models. However, the GP algorithm has been included in this study, because this algorithm delivers closed-form equations that could be easily incorporated into practical engineering applications for the prediction of the optimal wall thickness.

The predicted and actual optimal wall thickness values in the case of a single layer of post-tensioning have been plotted in Figure 7. It can be observed that the introduction of post-tensioning into the models caused a slight reduction in the accuracy of the ML models. However,

Table 2. Model accuracies in predicting the optimal wall thickness with post-tensioning.

Algorithm	R2		MAE		RMSE		Duration [s]
	Train	Test	Train	Test	Train	Test
XGBoost	0.9967	0.9825	0.0087	0.021	0.0139	0.0346	5.66
Random Forest	0.9973	0.9850	0.0077	0.019	0.0126	0.0320	5.14
LightGBM	0.9908	0.9864	0.0153	0.019	0.0234	0.0305	4.39
CatBoost	0.9931	0.9863	0.0129	0.018	0.0203	0.0306	25.15
GP	0.9588	0.9581	0.0422	0.045	0.0501	0.0531	150

shows that all ensemble models were capable of predicting the optimal wall thickness with an ${\mathrm{R}}^2$ score greater than 0.98 on the test set.

To have a clear visualization of the model performance, the percentage errors on the training and test sets have been plotted for the CatBoost model in Figure 8. Figure 8a shows the overlap between the actual and predicted optimal wall thickness values. According to Figure 8b, the error percentages fluctuate in the ±10% and ±30% ranges for the training and test sets respectively. Figure 8c,d show the distributions of the error percentages for the training and test sets respectively. Figure 8d, shows that most of the error percentage fluctuations on the test set are accumulated in the ±20% range whereas smaller error percentages are observed in Figure 8c in the ±10% range. The entire data set was split into a training set and a test set in 70% to 30% ratio. In terms of mean absolute error (MAE), the CatBoost model performed best on the test set, whereas the LightGBM model performed best in terms of RMSE and ${\mathrm{R}}^2$. It should be noted that all of the four ensemble learning models tested in this study demonstrated similar performances and the differences in performance are deemed negligible. On the other hand, a significant difference in terms of the computational speed could be observed for the CatBoost model. It was observed that the duration of training and testing the models is significantly longer in the case of CatBoost. The LightGBM model was observed to be the most efficient model in terms of computational speed, although the XGBoost and Random Forest models also have similar performances to LightGBM in terms of computational speed. Finally, the closed-form equation obtained from the genetic programming (GP) algorithm performed less accurately than the ensemble learning models.

Figure 9 shows the first of the decision trees that constitute the XGBoost model developed for the case with one layer of post-tensioning cables. The XGBoost model consists of a total of 100 iteratively added decision trees such that each new decision tree corrects the errors of the trees before itself. The final prediction of the XGBoost model is the sum of all the predictions made by the decision trees. The decision tree in Figure 9 consists of a root node which is split according to the level of wall height, 41 internal nodes, and 43 leaf nodes. The internal nodes are split at different levels of the fluid specific weight, and the unit costs of steel and concrete. The green colored leaf nodes contain the possible contributions of the decision tree to the final model prediction.

3.1. Interpretation of the Ensemble Learning Models Using SHAP Approach

The SHAP summary plots in Figure 10 and Figure 11 visualize the impact of each input feature on the CatBoost model predictions. The load cases with and without post-tensioning are visualized in Figure 10 and Figure 11, respectively. In the SHAP summary plots each dot represents one of the data points in the data set. The SHAP value corresponding to an input feature in a data point is represented on the horizontal axis, whereas the value of the input feature is color coded as shown with a color bar on the right-hand side of the plot. The higher values of an input feature are displayed with shades of red, while blue colors represent the lower values of an input feature. The input features are sorted by the magnitude of their impact, in decreasing order from top to bottom. The SHAP value measures the contribution of a feature to the model prediction such that positive SHAP values indicate an increasing effect of an input feature on the model output and negative SHAP values indicate a decreasing effect on the model output. The computation of the SHAP values can be described as in Equation (15) where F is the set of all input features and S is a subset of F where the feature with the index i has been withheld. In Equation (15), x and ${\mathsf{\varphi }}_{\mathrm{i}}$ represent a vector of input feature values and the corresponding SHAP value respectively [61].

$${\mathsf{\varphi }}_{\mathrm{i}}=\sum _{\mathrm{S}\subseteq \mathrm{F}\backslash \left\{\mathrm{i}\right\}}\frac{\left|\mathrm{S}\right|!\left(\left|\mathrm{F}\right|-\left|\mathrm{S}\right|-1\right)!}{\left|\mathrm{F}\right|!}\left[{\mathrm{f}}_{\mathrm{S}\cup \left\{\mathrm{i}\right\}}\left({\mathbf{x}}_{\mathrm{S}\cup \left\{\mathrm{i}\right\}}\right)-{\mathrm{f}}_{\mathrm{S}}\left({\mathbf{x}}_{\mathrm{S}}\right)\right]$$

(15)

According to Figure 10 and Figure 11 the height of the cylindrical wall has the greatest impact on the predicted optimal wall thickness for both load cases. The fluid specific weight is the second most impactful input feature whereas the concrete and steel unit costs have a relatively minor impact on the model predictions. Particularly, in the load case without post-tensioning, the impacts of the concrete and steel unit costs are significantly less than the fluid specific weight and the wall height.

Figure 12 and Figure 13 show feature dependence plots where the y-axis represents the SHAP value for the feature of interest, and the x-axis represents the actual value of the feature for each point in the dataset for the cases with and without post-tensioning respectively. The color of each dot on the feature dependence plots represents the value of a second feature that is most correlated with the main feature represented in the horizontal axis. The feature dependence plots convey information about how the predictions of the model are affected as the value of an input feature varies. A positive relationship between a feature’s value and its corresponding SHAP value indicates that an increase in that feature’s value will result in an increase in the model predictions. In both Figure 12 and Figure 13, there is a positive relationship between the height of the wall and the corresponding SHAP values which indicates that increased wall height leads to increased predictions of the wall thickness. The coloring of the dots in Figure 12 and Figure 13 indicate that for any given value of the wall height, an increase in the fluid specific weight is associated with a greater increasing impact on the model output wall heights greater than 10 m. On the other hand, for wall heights less than 10 m, in both load cases with and without post-tensioning, the fluid specific weight has the opposite effect on the predicted wall thickness. A similar relationship is also observed between the fluid specific weight and its SHAP value. For values greater than $10\mathrm{k}\mathrm{N}/{\mathrm{m}}^3$ the addition of this variable into the models has an increasing impact on the model output whereas specific weight values less than $10\mathrm{k}\mathrm{N}/{\mathrm{m}}^3$ are associated with a decrease in the predicted wall thickness values. Furthermore, for any given value of the fluid specific weight, increased wall height also increases the impact of $\mathrm{\gamma }$ on the model output as the coloring of the data points shows.

Individual conditional expectation (ICE) plots are helpful for gaining a better understanding of how a single feature affects the predictions made by a machine learning model. The ICE plots in Figure 14 and Figure 15 show the variation of the model predictions concerning a single input feature while the values of the remaining input features are kept constant. Each line in the ICE plot represents one of the data points in the data set and the average value of the predictions for all the data points is plotted with a thick blue line. Figure 14a and Figure 15a clearly show that the predicted optimal wall thickness values increase with the wall height while the range of optimal wall thickness values becomes wider as the wall height increases. A less steep increase in the optimal wall thickness values can be observed with respect to the γ values in Figure 14b and Figure 15b. Also, for any given $\mathrm{\gamma }$ value, the optimal wall thicknesses take values on a much wider range compared to the wall height. Finally, the ICE plots of Cc and Cs show that these features have significantly less impact on the model output compared to H and γ.

3.2. Genetic Programming

Equations (16) and (17) have been obtained by the genetic programming algorithm for the prediction of the optimal wall thickness as a function of the wall height and liquid specific weight for the case of no post-tensioning and one post-tensioning cable, respectively. The tree representations of Equations (16) and (17) are shown in Figure 16. The tree population size was 5000 and mean absolute error (MAE) was used as the error metric. MAE values of 0.041 and 0.046 could be achieved in less than 50 iterations in cases with and without post-tensioning, respectively.

$$\mathrm{t}\left(\mathrm{H},\mathrm{\gamma }\right)=0.095·\mathrm{H}·\log (0.878·\log \left(\mathrm{\gamma }\right))$$

(16)

$$\mathrm{t}\left(\mathrm{H},\mathrm{\gamma }\right)=\mathrm{t}\mathrm{a}\mathrm{n}\left(\mathrm{t}\mathrm{a}\mathrm{n}\left(0.093·\mathrm{H}·\mathrm{t}\mathrm{a}\mathrm{n}\left(\mathrm{l}\mathrm{o}\mathrm{g}\left(\sqrt{\mathrm{l}\mathrm{o}\mathrm{g}\left(\mathrm{\gamma }\right)}\right)\right)\right)\right)$$

(17)

Figure 17 and Figure 18 show the development of the equation length and accuracy throughout the generations. It can be seen from Figure 17 that after 50 iterations an MAE value of 0.046 could be achieved with a predictive equation that consists of 9 components in the load case without post-tensioning. These 9 components consist of the log function which appears 2 times, the multiplication operator which appears 3 times, the coefficients 0.095 and 0.878, and the variables H and $\mathrm{\gamma }$. Figure 18 shows that after 50 iterations an MAE value 0f 0.041 is achieved with a predictive equation consisting of 11 components. These components are the tangent function which appears 3 times, the log function which appears 2 times, the square root function which appears 1 time, the multiplication operator which appears 2 times, the coefficient 0.093, and the variables H and $\mathrm{\gamma }$. The larger MAE value associated with the case without post-tensioning is due to the larger wall thicknesses necessary in the absence of post-tensioning. A complete list of the hyperparameters modified in obtaining Equations (16) and (17) is given in

Table 3. Hyperparameters for the Genetic Programmin (GP) Model of the wall thickness.

Model	Parameter Name	Value
GP	population_size	5000
	p_crossover	0.7
	p_subtree_mutation	0.1
	p_hoist_mutation	0.05
	p_point_mutation	0.1
	tournament_size	150
	function_set	(‘add’, ‘sin’, ‘cos’, ‘tan’, ‘log’, ‘sub’, ‘mul’, ‘div’, ‘sqrt’)

. For the remaining hyperparameters, the default values have been used.

4. Discussion

The current study demonstrates the application of the harmony search optimization algorithm in generating large data sets for training state-of-the-art machine learning models. The problem of predicting the optimal wall thickness of a cylindrical post-tensioned reinforced concrete wall of a liquid container has been investigated. In addition to ensemble learning models such as XGBoost and CatBoost also, the genetic programming methodology has been utilized to obtain closed-form predictive equations. The ensemble learning models have been trained with the height of the wall, specific weight of the liquid, and unit costs of steel and concrete as the input features. It was shown that the inclusion of post-tensioning cables leads to a decrease in the optimal wall thickness. The output of the ensemble learning models has been further analyzed using the SHAP approach and individual conditional expectation plots. The SHAP analysis showed that the wall height is the most impactful input feature affecting the model predictions of the optimal wall thickness followed by the liquid specific weight. An increase in the wall height and liquid specific weight was shown to also increase the optimal wall thickness. Furthermore, the genetic programming methodology provided predictive equations as functions of wall height and liquid specific weight for the load cases with and without post-tensioning. All predictive models were able to provide highly accurate predictions of the optimal wall thickness, with of the ensemble learning models achieving ${\mathrm{R}}^2$ scores greater than 0.98 on the test sets. The LightGBM model delivered the fastest and most accurate predictions in the case of walls with post-tensioning whereas the CatBoost model was the slowest.

5. Conclusions

The application of optimization techniques to generate big data sets is a novel approach to overcome the lack of experimental data points for the training of ML models related to structural engineering. Furthermore, the availability of closed-form equations for the prediction of optimal design configurations has significant practical benefits. The current study demonstrated the application of the genetic programming methodology in obtaining closed-form equations for the prediction of the optimal wall thickness for a liquid containing post-tensioned cylindrical wall. The data set necessary for this procedure was generated using the harmony search optimization technique. The obtained predictive equations could be incorporated into the engineering design process to facilitate structural optimization. However, further research needs to be done with larger data sets for the performance validation of the presented equations. On the other hand, higher ${\mathrm{R}}^2$ scores could be obtained from the ensemble learning algorithms. A limitation of the current study is the range of design variables included in the data set generation process. Future research in this area can incorporate additional input features into the data set such as the mechanical properties of steel and concrete or the positions of post-tensioning cables. Data sets can also be further enhanced with the results of numerical studies.