5.3.2. Influence of Population Number (npop)

average value for statistical criteria and computation time.

5.3.2. Influence of Population Number (npop) The evaluation of statistical criteria in the function of npop for RMSE, MAE, R, and IA is shown in Figure 9a–d, respectively, for both training and testing parts. It can be seen that except for the low value for population size (i.e., npop = 20), all other npop values show good prediction results, especially for npop = 200. However, as introduced in the preliminary analyses for computation time, the higher the number of npop, the more time is consumed. Finally, npop = 50 was chosen as the most appropriate The evaluation of statistical criteria in the function of npop for RMSE, MAE, R, and IA is shown in Figure 9a–d, respectively, for both training and testing parts. It can be seen that except for the low value for population size (i.e., npop = 20), all other npop values show good prediction results, especially for npop = 200. However, as introduced in the preliminary analyses for computation time, the higher the number of npop, the more time is consumed. Finally, npop = 50 was chosen as the most appropriate average value for statistical criteria and computation time.

Materials 2020, 13, x FOR PEER REVIEW 15 of 27

Figure 9. Evaluation of statistical criteria in the function of npop: (a) RMSE, (b) MAE, (c) R, and (d) IA. **Figure 9.** Evaluation of statistical criteria in the function of npop: (**a**) RMSE, (**b**) MAE, (**c**) R, and (**d**) IA. Figure 9. Evaluation of statistical criteria in the function of npop: (a) RMSE, (b) MAE, (c) R, and (d) IA.

#### 5.3.3. Influence of Initial Weight (wini) 5.3.3. Influence of Initial Weight (wini)

value range of between 0.1 and 0.4 was the most appropriate.

5.3.3. Influence of Initial Weight (wini) The evaluation of statistical criteria in the function of wini for RMSE, MAE, R, and IA is shown in Figure 10a–d, respectively, for both training and testing parts. It can be seen that poor prediction performance was obtained when wini was larger than 0.5 (i.e., an increase of RMSE and MAE values and a decrease of R and IA values). Regarding the statistical criteria (RMSE, MAE, R, and IA), a wini The evaluation of statistical criteria in the function of wini for RMSE, MAE, R, and IA is shown in Figure 10a–d, respectively, for both training and testing parts. It can be seen that poor prediction performance was obtained when wini was larger than 0.5 (i.e., an increase of RMSE and MAE values and a decrease of R and IA values). Regarding the statistical criteria (RMSE, MAE, R, and IA), a wini value range of between 0.1 and 0.4 was the most appropriate. The evaluation of statistical criteria in the function of wini for RMSE, MAE, R, and IA is shown in Figure 10a–d, respectively, for both training and testing parts. It can be seen that poor prediction performance was obtained when wini was larger than 0.5 (i.e., an increase of RMSE and MAE values and a decrease of R and IA values). Regarding the statistical criteria (RMSE, MAE, R, and IA), a wini value range of between 0.1 and 0.4 was the most appropriate.

**Figure 10.** *Cont.*

Figure 10. Evaluation of statistical criteria in the function of wini: (a) RMSE, (b) MAE, (c) R, and (d) IA. **Figure 10.** Evaluation of statistical criteria in the function of wini: (**a**) RMSE, (**b**) MAE, (**c**) R, and (**d**) IA. Figure 10. Evaluation of statistical criteria in the function of wini: (a) RMSE, (b) MAE, (c) R, and (d) IA.

5.3.4. Influence of Personal Learning Coefficient (c1) 5.3.4. Influence of Personal Learning Coefficient (c1)

R, and IA). Therefore, c1 = [1, 1.4] was the most appropriate value.

5.3.4. Influence of Personal Learning Coefficient (c1) The evaluation of statistical criteria in the function of c1 for RMSE, MAE, R, and IA is shown in Figure 11a–d, respectively, for both training and testing parts. It can be seen that good prediction performance was obtained when c1 was in the range of [1, 1.4] for all statistical criteria (RMSE, MAE, The evaluation of statistical criteria in the function of c<sup>1</sup> for RMSE, MAE, R, and IA is shown in Figure 11a–d, respectively, for both training and testing parts. It can be seen that good prediction performance was obtained when c<sup>1</sup> was in the range of [1, 1.4] for all statistical criteria (RMSE, MAE, R, and IA). Therefore, c<sup>1</sup> = [1, 1.4] was the most appropriate value. The evaluation of statistical criteria in the function of c1 for RMSE, MAE, R, and IA is shown in Figure 11a–d, respectively, for both training and testing parts. It can be seen that good prediction performance was obtained when c1 was in the range of [1, 1.4] for all statistical criteria (RMSE, MAE, R, and IA). Therefore, c1 = [1, 1.4] was the most appropriate value.

Figure 11. Evaluation of statistical criteria in the function of c1: (a) RMSE, (b) MAE, (c) R, and (d) IA. **Figure 11.** Evaluation of statistical criteria in the function of c<sup>1</sup> : (**a**) RMSE, (**b**) MAE, (**c**) R, and (**d**) IA.

Figure 11. Evaluation of statistical criteria in the function of c1: (a) RMSE, (b) MAE, (c) R, and (d) IA.

#### 5.3.5. Influence of Global Learning Coefficient 5.3.5. Influence of Global Learning Coefficient

The evaluation of statistical criteria in the function of c<sup>2</sup> for RMSE, MAE, R, and IA is shown in Figure 12a–d, respectively, for both training and testing parts. It can be seen that good prediction performance was obtained when c<sup>2</sup> was higher than 1.8 for all statistical criteria (RMSE, MAE, R, and IA). Therefore, c<sup>2</sup> = [1.8, 2] was the most appropriate value. The evaluation of statistical criteria in the function of c2 for RMSE, MAE, R, and IA is shown in Figure 12a–d, respectively, for both training and testing parts. It can be seen that good prediction performance was obtained when c2 was higher than 1.8 for all statistical criteria (RMSE, MAE, R, and IA). Therefore, c2 = [1.8, 2] was the most appropriate value.

Figure 12. Evaluation of statistical criteria in the function of c2: (a) RMSE, (b) MAE, (c) R, and (d) IA. **Figure 12.** Evaluation of statistical criteria in the function of c<sup>2</sup> : (**a**) RMSE, (**b**) MAE, (**c**) R, and (**d**) IA.

5.3.6. Influence of Velocity Limits

chosen.

5.3.6. Influence of Velocity Limits The evaluation of statistical criteria in the function of fv for RMSE, MAE, R, and IA is shown in Figure 13a–d, respectively, for both training and testing parts. It can be seen that no influence could be established regarding all statistical criteria (RMSE, MAE, R, and IA). Therefore, fv = 0.1 was finally The evaluation of statistical criteria in the function of f<sup>v</sup> for RMSE, MAE, R, and IA is shown in Figure 13a–d, respectively, for both training and testing parts. It can be seen that no influence could be established regarding all statistical criteria (RMSE, MAE, R, and IA). Therefore, f<sup>v</sup> = 0.1 was finally chosen.

Figure 13. Evaluation of statistical criteria in the function of fv: (a) RMSE, (b) MAE, (c) R and (d) IA. **Figure 13.** Evaluation of statistical criteria in the function of fv: (**a**) RMSE, (**b**) MAE, (**c**) R and (**d**) IA.

### *5.4. Prediction Capability of the ANFIS-PSO Model using Optimal Configuration*

5.4. Prediction Capability of the ANFIS-PSO Model using Optimal Configuration Table 4 summarizes all of the optimal values, as identified previously. By using the optimal coefficient in Table 4, a regression graph between the real and predicted Pu (kN) is shown in Figure 14. The slope of the ideal fit was then used to measure the angle between the x-axis and the ideal fit, with angles closer than 45° showing better performance. Figure 14a shows the predictability when using the training set, whereas Figure 14b shows the same information applied to the testing set. In both cases, the angles generated by the predicted output had slopes close to that of the ideal fit. This showed that the performance of the proposed model was consistent. Figure 15 shows the error distribution graph using the training part, testing part, and all data. In short, using the selected Table 4 summarizes all of the optimal values, as identified previously. By using the optimal coefficient in Table 4, a regression graph between the real and predicted P<sup>u</sup> (kN) is shown in Figure 14. The slope of the ideal fit was then used to measure the angle between the *x*-axis and the ideal fit, with angles closer than 45◦ showing better performance. Figure 14a shows the predictability when using the training set, whereas Figure 14b shows the same information applied to the testing set. In both cases, the angles generated by the predicted output had slopes close to that of the ideal fit. This showed that the performance of the proposed model was consistent. Figure 15 shows the error distribution graph using the training part, testing part, and all data. In short, using the selected number of fuzzy rules and PSO parameters, the prediction model gave excellent results (Table 5).


fv 0.1 0.1

number of fuzzy rules and PSO parameters, the prediction model gave excellent results (Table 5). **Table 4.** Parameters used as optimum.

Materials 2020, 13, x FOR PEER REVIEW 19 of 27

Figure 14. Graphs of regression plots between actual and predicted Pu (kN) for the (a) training part and (b) testing part. **Figure 14.** Graphs of regression plots between actual and predicted P<sup>u</sup> (kN) for the (**a**) training part and (**b**) testing part. Figure 14. Graphs of regression plots between actual and predicted Pu (kN) for the (a) training part and (b) testing part.

Figure 15. Distribution of errors. **Figure 15.** Distribution of errors.

Figure 15. Distribution of errors. Table 5. Prediction capability. **Table 5.** Prediction capability.


8 min

(b): Testing part

#### Testing 0.037 0.014 0.968 0.977 0.005 0.037 5.5. Sensitivity Analysis *5.5. Sensitivity Analysis*

5.5. Sensitivity Analysis The sensitivity analysis was performed in order to explore the degree of importance of each input variable using the ANFIS-PSO model. For this, quantile values at 21 points (from 0% to 100%, with a step of 5%) of each input variable were collected from the database and served as a new dataset for the calculation of critical buckling load. More precisely, for a given input, its value varied from 0% to 100%, while all other inputs remained at their median (50%). This variation of values following the probability distribution allows the influence of each input variable to be explored based on their statistical behavior. The results of the sensitivity analysis are indicated in Figure 16 in a bar graph (scaled into the range of 0% to 100%). It can be seen that all variables influenced the prediction of critical buckling load through the ANFIS-PSO model. The most important input variables were L, wflange, tweb, and tflange, which gave degree of importance values of 33.9%, 21.7%, 18.6%, and 10.6%, respectively. This information is strongly relevant and in good agreement with the literature, in which the length of the beam and geometrical parameter of the cross-section are the most important parameters [3–5]. However, it can be seen in Figure 16 that the height of the beam does not seriously affect the buckling capacity of the structural members. It should be noted that only three independent values of the section's height were used to generate the database; for example, 420, 560, and 700 mm. The sensitivity analysis was performed in order to explore the degree of importance of each input variable using the ANFIS-PSO model. For this, quantile values at 21 points (from 0% to 100%, with a step of 5%) of each input variable were collected from the database and served as a new dataset for the calculation of critical buckling load. More precisely, for a given input, its value varied from 0% to 100%, while all other inputs remained at their median (50%). This variation of values following the probability distribution allows the influence of each input variable to be explored based on their statistical behavior. The results of the sensitivity analysis are indicated in Figure 16 in a bar graph (scaled into the range of 0% to 100%). It can be seen that all variables influenced the prediction of critical buckling load through the ANFIS-PSO model. The most important input variables were L, wflange, tweb, and tflange, which gave degree of importance values of 33.9%, 21.7%, 18.6%, and 10.6%, respectively. This information is strongly relevant and in good agreement with the literature, in which the length of the beam and geometrical parameter of the cross-section are the most important parameters [3–5]. However, it can be seen in Figure 16 that the height of the beam does not seriously affect the buckling capacity of the structural members. It should be noted that only three independent values of the section's height were used to generate the database; for example, 420, 560, and 700 mm. Consequently, the linear correlation coefficient between the section's height and the buckling The sensitivity analysis was performed in order to explore the degree of importance of each input variable using the ANFIS-PSO model. For this, quantile values at 21 points (from 0% to 100%, with a step of 5%) of each input variable were collected from the database and served as a new dataset for the calculation of critical buckling load. More precisely, for a given input, its value varied from 0% to 100%, while all other inputs remained at their median (50%). This variation of values following the probability distribution allows the influence of each input variable to be explored based on their statistical behavior. The results of the sensitivity analysis are indicated in Figure 16 in a bar graph (scaled into the range of 0% to 100%). It can be seen that all variables influenced the prediction of critical buckling load through the ANFIS-PSO model. The most important input variables were L, wflange, tweb, and tflange, which gave degree of importance values of 33.9%, 21.7%, 18.6%, and 10.6%, respectively. This information is strongly relevant and in good agreement with the literature, in which the length of the beam and geometrical parameter of the cross-section are the most important parameters [3–5]. However, it can be seen in Figure 16 that the height of the beam does not seriously affect the buckling capacity of the structural members. It should be noted that only three independent values of the section's height were used to generate the database; for example, 420, 560, and 700 mm. Consequently, the linear correlation coefficient between the section's height and the buckling capacity was only −0.092.

Consequently, the linear correlation coefficient between the section's height and the buckling

On the contrary, the minimum value of the beam's length was 4000 mm (approximately 5.7 times larger than the maximum section's height) and five independent values were used to generate the database, ranging from 4000 to 8000 mm, with a step of 1000 mm. Thus, the linear correlation coefficient between the beam's length and the buckling capacity was −0.667 (approximately 7.25 times bigger than the linear correlation coefficient between the section's height and the buckling capacity). Consequently, a larger database should be considered in future studies to estimate the degree of importance of the section's height. capacity was only −0.092. On the contrary, the minimum value of the beam's length was 4000 mm (approximately 5.7 times larger than the maximum section's height) and five independent values were used to generate the database, ranging from 4000 to 8000 mm, with a step of 1000 mm. Thus, the linear correlation coefficient between the beam's length and the buckling capacity was −0.667 (approximately 7.25 times bigger than the linear correlation coefficient between the section's height and the buckling capacity). Consequently, a larger database should be considered in future studies to estimate the degree of importance of the section's height. Materials 2020, 13, x FOR PEER REVIEW 20 of 27 capacity was only −0.092. On the contrary, the minimum value of the beam's length was 4000 mm (approximately 5.7 times larger than the maximum section's height) and five independent values

were used to generate the database, ranging from 4000 to 8000 mm, with a step of 1000 mm. Thus,

Materials 2020, 13, x FOR PEER REVIEW 20 of 27

1

0.6 0.6

0.8 0.4 0.2 <sup>1</sup> <sup>0</sup>

Figure 16. Bar graph showing the estimations of degree of importance values. **Figure 16.** Bar graph showing the estimations of degree of importance values. d0dtwebwflangetflange

H

D

L

The sensitivity analysis presented above demonstrates that the ML technique could assist in the design phase for circular opening steel beams. In addition to reliable prediction of the critical buckling load, the ANFIS-PSO model can also assist in the creation of input–output maps, as illustrated in Figure 17. In particular, as L, tweb, wflange, and tflange were the most important variables, they are used for map illustrations in this section. The values of the remaining variables were kept constant. In Figure 17, four maps of critical buckling load are presented (with the same color range), involving the relationship between Pu and L-wflange, L-tflange, L-tweb, and wflange-tflange, respectively. As can be seen from the surface plots, the input–output relationship exhibited nonlinear behavior, which cannot be easily identified from the database. Figure 17a shows that a maximum value for the critical buckling load can be obtained if L reaches its minimum and wflange reaches its maximum value. On the other hand, the critical buckling load reaches its minimum if L reaches its highest value and wflange reaches its lowest value. This map confirms the negative effect of L, as pointed out in the literature [4]. In Figure 17b,c, the same results are obtained as in Figure 17a. This observation again confirms that the geometrical parameters of the cross-section are highly important [1,5]. Such quantitative information allows the design and analysis recommendations to be explored, as well as for new beam configurations to be generated (within the range of variables considered in this present study). The sensitivity analysis presented above demonstrates that the ML technique could assist in the design phase for circular opening steel beams. In addition to reliable prediction of the critical buckling load, the ANFIS-PSO model can also assist in the creation of input–output maps, as illustrated in Figure 17. In particular, as L, tweb, wflange, and tflange were the most important variables, they are used for map illustrations in this section. The values of the remaining variables were kept constant. In Figure 17, four maps of critical buckling load are presented (with the same color range), involving the relationship between P<sup>u</sup> and L-wflange, L-tflange, L-tweb, and wflange-tflange, respectively. As can be seen from the surface plots, the input–output relationship exhibited nonlinear behavior, which cannot be easily identified from the database. Figure 17a shows that a maximum value for the critical buckling load can be obtained if L reaches its minimum and wflange reaches its maximum value. On the other hand, the critical buckling load reaches its minimum if L reaches its highest value and wflange reaches its lowest value. This map confirms the negative effect of L, as pointed out in the literature [4]. In Figure 17b,c, the same results are obtained as in Figure 17a. This observation again confirms that the geometrical parameters of the cross-section are highly important [1,5]. Such quantitative information allows the design and analysis recommendations to be explored, as well as for new beam configurations to be generated (within the range of variables considered in this present study). Figure 16. Bar graph showing the estimations of degree of importance values. The sensitivity analysis presented above demonstrates that the ML technique could assist in the design phase for circular opening steel beams. In addition to reliable prediction of the critical buckling load, the ANFIS-PSO model can also assist in the creation of input–output maps, as illustrated in Figure 17. In particular, as L, tweb, wflange, and tflange were the most important variables, they are used for map illustrations in this section. The values of the remaining variables were kept constant. In Figure 17, four maps of critical buckling load are presented (with the same color range), involving the relationship between Pu and L-wflange, L-tflange, L-tweb, and wflange-tflange, respectively. As can be seen from the surface plots, the input–output relationship exhibited nonlinear behavior, which cannot be easily identified from the database. Figure 17a shows that a maximum value for the critical buckling load can be obtained if L reaches its minimum and wflange reaches its maximum value. On the other hand, the critical buckling load reaches its minimum if L reaches its highest value and wflange reaches its lowest value. This map confirms the negative effect of L, as pointed out in the literature [4]. In Figure 17b,c, the same results are obtained as in Figure 17a. This observation again confirms that the geometrical parameters of the cross-section are highly important [1,5]. Such quantitative information allows the design and analysis recommendations to be explored, as well as for new beam configurations to be generated (within the range of variables considered in this present study).

Scaled input L Scaled input wflange 0.6 0.6 0.8 0.4 0.2 <sup>1</sup> <sup>0</sup> Scaled input tflange Scaled input L **Figure 17.** *Cont.*

Materials 2020, 13, x FOR PEER REVIEW 21 of 27

Figure 17. Three-dimensional scaled input–output maps: (a) L-wflange, (b) L-tflange, (c) L-tweb, and (d) wflange-tflange. **Figure 17.** Three-dimensional scaled input–output maps: (**a**) L-wflange, (**b**) L-tflange, (**c**) L-tweb, and (**d**) wflange-tflange.

#### 6. Conclusions **6. Conclusions**

PSO is one of the most popular optimization techniques used to optimize and improve the performance of machine learning models in terms of classification and regression. However, its effectiveness depends significantly on the selection of parameters used to train this technique. In this paper, investigation and selection of PSO parameters was carried out to improve and optimize the performance of the ANFIS model, which is one of the most popular and effective ML models, for prediction of the buckling capacity of circular opening steel beams. Different parameters (nrule, npop, wini, c1, c2, and fv) of PSO were tuned on 3645 available data samples to determine the best values for optimization of the performance of ANFIS. The results show that the performance of ANFIS optimized by PSO (ANFIS-PSO) is suitable for PSO is one of the most popular optimization techniques used to optimize and improve the performance of machine learning models in terms of classification and regression. However, its effectiveness depends significantly on the selection of parameters used to train this technique. In this paper, investigation and selection of PSO parameters was carried out to improve and optimize the performance of the ANFIS model, which is one of the most popular and effective ML models, for prediction of the buckling capacity of circular opening steel beams. Different parameters (nrule, npop, wini, c1, c2, and fv) of PSO were tuned on 3645 available data samples to determine the best values for optimization of the performance of ANFIS.

determining the buckling capacity of circular opening steel beams, but is very sensitive under different PSO investigation and selection parameters. The results also show that nrule = 10, npop = 50, wini = 0.1 to 0.4, c1 = [1, 1.4], c2 = [1.8, 2], and fv = 0.1 are the most suitable selection settings in order to get the best performance from ANFIS-PSO. The sensitivity analysis shows that L, wflange, tweb, and tflange are the most important input variables used for prediction of the buckling capacity of circular opening steel beams. In short, this study might help in selection of the suitable PSO parameters for optimization of ANFIS in determining the buckling capacity of circular opening steel beams. It also helps in suitable The results show that the performance of ANFIS optimized by PSO (ANFIS-PSO) is suitable for determining the buckling capacity of circular opening steel beams, but is very sensitive under different PSO investigation and selection parameters. The results also show that nrule = 10, npop = 50, wini = 0.1 to 0.4, c<sup>1</sup> = [1, 1.4], c<sup>2</sup> = [1.8, 2], and f<sup>v</sup> = 0.1 are the most suitable selection settings in order to get the best performance from ANFIS-PSO. The sensitivity analysis shows that L, wflange, tweb, and tflange are the most important input variables used for prediction of the buckling capacity of circular opening steel beams.

selection of input variables for better prediction of the buckling capacity of circular opening steel beams. However, it is noted that the optimal values of PSO parameters found in this study are suitable for the ANFIS model in determining the buckling capacity of circular opening steel beams. Thus, it is suggested that these parameters should be validated with other ML models applied in other problems. Finally, variation in the mechanical properties of material used should be investigated in further research, as this is important from a physics perspective. Author Contributions: Conceptualization, Q.H.N., H.-B.L., T.-T.L, and B.T.P; methodology, H.-B.L, T.-T.L, and B.T.P; validation, H.-B.L. and T.-T.L; formal analysis, Q.H.N., V.Q.T., T.-A.N., V.-H.P., and H.-B.L; data curation, V.Q.T., T.-A.N., and V.-H.P.; writing—original draft preparation, all authors; writing—review and editing, H.- In short, this study might help in selection of the suitable PSO parameters for optimization of ANFIS in determining the buckling capacity of circular opening steel beams. It also helps in suitable selection of input variables for better prediction of the buckling capacity of circular opening steel beams. However, it is noted that the optimal values of PSO parameters found in this study are suitable for the ANFIS model in determining the buckling capacity of circular opening steel beams. Thus, it is suggested that these parameters should be validated with other ML models applied in other problems. Finally, variation in the mechanical properties of material used should be investigated in further research, as this is important from a physics perspective.

**Author Contributions:** Conceptualization, Q.H.N., H.-B.L., T.-T.L, and B.T.P; methodology, H.-B.L, T.-T.L, and B.T.P; validation, H.-B.L. and T.-T.L; formal analysis, Q.H.N., V.Q.T., T.-A.N., V.-H.P., and H.-B.L; data curation, V.Q.T., T.-A.N., and V.-H.P.; writing—original draft preparation, all authors; writing—review and editing, H.-B.L, T.-T.L, and B.T.P; visualization, H.-B.L., T.-A.N., V.-H.P. and T.-T.L; supervision, Q.H.N., H.-B.L, T.-T.L, and B.T.P; project administration, H.-B.L., T.-T.L, and B.T.P; funding acquisition, Q.H.N. All authors have read and agreed to the published version of the manuscript.

B.L, T.-T.L, and B.T.P; visualization, H.-B.L., T.-A.N., V.-H.P. and T.-T.L; supervision, Q.H.N., H.-B.L, T.-T.L,

**Funding:** This research received no external funding

**Conflicts of Interest:** The authors declare no conflict of interest.
