Application of Machine Learning in Constitutive Relationship Prediction of Porous Titanium Materials for Artificial Bone

Abstract

Artificial bone porous titanium materials are widely used in orthopedic implants. However, the traditional constitutive model is often limited by the complexity and accuracy of the model, and it is difficult to accurately and efficiently describe the constitutive relationship of porous titanium materials. In this study, structured data were established based on experimental data from published papers, and goodness of fit ($R^2$), mean absolute error ($MAE$) and mean absolute percentage error ($MAPE$) were used to evaluate the model. The prediction effect of random forest (RF), multi-layer perceptron (MLPR) and support vector machine (SVR) on the constitutive relationship of porous titanium materials was discussed. Through comprehensive comparison, it can be seen that the RF model with max_depth of 24 and n_estimators of 160 has the best performance in prediction, and the average absolute percentage error is less than 4.4%, which means it can accurately predict the temperature sensitivity and strain rate sensitivity of porous titanium materials. And its predictive ability is better than that of the traditional constitutive model, which provides a new idea and method for the constitutive modeling of porous titanium materials.

1. Introduction

In today’s engineering field, the research and application of porous materials have been paid more and more attention. In particular, as an important functional material, porous titanium has excellent mechanical properties and biocompatibility, so it is widely used in orthopedic implants [1,2,3]. The research on the mechanical properties of porous titanium materials is the basis for further research and application of porous titanium. At present, the constitutive model of porous titanium materials is mainly based on the traditional theoretical model [4,5].

In recent years, with the rapid development of machine learning, it has been widely used in various fields, such as Reka using machine learning with machine vision to identify the degree of fruit decay [6], Zheng et al. using machine learning to optimize the installation position of hydraulic cylinders on medical robots [7] and so on. For material prediction, machine learning is also widely used in this field because of its characteristics of learning the laws contained in a wide range of data sets [8,9,10,11]. At present, scholars at home and abroad have launched a certain exploration in this field. In 2023, Lei et al. [12] established a fatigue test data set containing Hastelloy material characteristics, and then built a machine learning framework for fatigue life prediction based on the data set. After comparison, it was found that the prediction effect of the model built based on machine learning was greatly improved compared with traditional prediction methods. Some scholars have tested the prediction effect of common machine learning algorithms such as SVM, RF, AdaBoost and DNN on material prediction and found the most suitable machine learning algorithm corresponding to material properties and the optimal parameters of each algorithm through comprehensive comparison [13,14,15,16,17]. For hyperelastic porous materials, Liu Zhentao et al. [18] built a constitutive model based on artificial neural networks to predict their mechanical behavior. Meanwhile, some scholars have improved the algorithm model for different cases. For example, Zhao [19] and Ismail [20] used the genetic algorithm and particle swarm optimization algorithm, respectively, to optimize the neural network and predict the creep fatigue life of metal oxides and the wear rate of nanocomposites. In 2021, Guo et al. [21] combined convolutional neural networks and an improved long short-term memory neural network to propose a deep learning model that can quickly identify constitutive model parameters and has good denoising performance. Some scholars [22,23] combined intelligent optimization algorithms such as Gray Wolf optimizer and the particle swarm optimization algorithm with support vector machine and XGBoost algorithms to form a hybrid model, and obtained a better model by comparing the performance differences between different hybrid models.

To sum up, machine learning has been used to predict the properties of materials. However, the traditional constitutive model is often based on phenomenology or physics, which is limited by its own explicit mathematical description ability, and it is very difficult to find a constitutive model that can accurately describe the complex mechanical behavior of materials under the combined action of multiple effects [24]. Therefore, when describing some materials with complex properties, it is necessary to optimize and modify the classical constitutive model based on the current material properties, such as introducing dislocation dynamics, grain evolution and crystal plasticity theories [25]. In addition, the prediction range of the traditional constitutive model is limited, and it does not have enough predictive properties to describe the properties of materials under certain conditions of high temperature or high strain rate [4,26]. In contrast, the data set of the constitutive model based on machine learning can be continuously expanded, and the training and verification time of the new model is shorter [27]; so, the flexibility and prediction range of the model are more advantageous than those of the traditional constitutive model. However, the construction of a constitutive model using machine learning relies on a large amount of data to capture the nonlinear, complex and multivariate behavior of materials; so, the application of machine learning in this construction model relies heavily on the training database [26,28]. Moreover, the machine learning algorithm has difficulties in parameter adjustment and convergence, and poor algorithm parameter adjustment will lead to problems such as model convergence failure and overfitting. Although machine learning still has some problems in the field of material prediction, it has great prospects for development. At present, the research on porous materials, especially porous titanium materials, is still limited, and the adaptability and prediction accuracy of different models for porous titanium materials have not been systematically studied.

In this study, a structured data set containing four characteristic parameters was established through missing value processing, outlier processing and standardization processing, and the prediction effect of RF, MLPR and SVR algorithms on the constitutive relationship of porous titanium materials was discussed. Through comprehensive comparison, the RF model was found to be the best. Then, the RF model was compared with the traditional constitutive model, and the prediction accuracy of the RF model was found to be higher. The purpose of this study was to provide new ideas and methods for the establishment of the constitutive model of porous titanium materials, promote the development of porous materials, and promote the research of related fields to make new breakthroughs and more progress.

2. Materials and Methods

2.1. Model Principle

Three algorithms, random forest, multi-layer perceptron and support vector machine, are used in this study. The random forest model is built by integrating multiple decision trees, which has good robustness and prediction accuracy, but it is easy to overfit the noisy data. Multi-layer perceptrons capture complex relationships in data through nonlinear changes in activation functions in hidden layers. They can learn complex nonlinear relationships and perform well on large-scale data sets. However, they require a lot of data and computational resources to train the model, and the model structure is complex and difficult to interpret. SVM uses the kernel function to capture complex relationships in data. It performs well in high-dimensional space and is suitable for high-dimensional data. However, it has high computational complexity for large-scale data sets and high-dimensional data.

2.1.1. Random Forest Regression (RF)

The random forest algorithm is a supervised learning algorithm. Its main idea is to build a forest composed of decision trees in a random way, and the samples are classified by Information Gain, Gain Ratio and Gini Index at the nodes of each tree. The random forest algorithm achieves high prediction accuracy through random selection of samples, features and integrated prediction.

The implementation process of the random forest algorithm is as follows:

• N′ (N > N′ > 2N/3) samples are selected each time through N random sampling for the full set D of samples with a capacity of N, so as to form n sample training sets. The samples that are not drawn are divided into test sets.
• n weak learners are generated based on n sample training sets, but only S′ (S′ << S) are randomly selected from S feature attributes of the sample as feature variables.
• Therefore, the test set tests n weak learners and obtains n predicted values. After averaging them, the final predicted value can be obtained.

2.1.2. Multi-Layer Perceptron Regression (MLPR)

The multi-layer perceptron algorithm is a supervised learning algorithm. Its main idea is to build an algorithm model by simulating the connection mechanism of the human brain neural network. The multi-layer perceptron algorithm consists of three layers: input layer, hidden layer and output layer. The input layer has the same number of nodes as the feature variable and does not include the activation function. The output layer has the same number of nodes as the output variable; each layer of the activation function in the hidden layer is nonlinearly transformed to capture complex relationships in the input data.

The implementation process of the multi-layer perceptron algorithm is as follows:

• The weight parameters and threshold parameters of each layer are initialized. The data are passed through the input layer to the first hidden layer. The weighted input is calculated in the hidden layer, the activation function is applied to pass the result to the next layer, and then propagated forward in turn until the result is finally obtained in the output layer.
• According to the gradient descent method, the connection weight parameters and threshold parameters between neurons are updated to minimize the total loss function.
• Steps (1) and (2) are repeated until a preset stop condition such as network convergence or the maximum number of iterations is reached.

2.1.3. Support Vector Regression (SVR)

Support vector regression is a supervised learning algorithm. The main idea is to map the feature variables to the high-dimensional feature space by using the kernel function, and then perform linear regression in the high-dimensional feature space and nonlinear regression in the low-dimensional space.

The implementation process of the support vector machine algorithm is as follows [29]:

• One must estimate the sample data $\left(x_1-y_1\right)\dots \dots (x_N,y_N),x_i,y_i\in R$ which defines the regression function as follows:

  $$\mathrm{f}(x)=w^{T}x+b$$

  (1)

  where f(x) is the predicted value, w is the weight vector and b is the bias term.
• In order to find the minimum w, the relaxation factor ${\xi }_i,{\xi }_i^{*}$ is introduced to avoid underfitting the model. The optimization objectives are as follows:

  $$\min \frac{1}{2}{‖w‖}^2+C{\displaystyle \sum _{i=1}^N({\xi }_i+{\xi }_i^{*})}$$

  (2)

  $$\left\{\begin{array}{c}f(x_i)-y_i\le \epsilon +{\xi }_i\\ y_i-f(x_i)\le \epsilon +{\xi }_i^{*}\\ {\xi }_i,{\xi }_i^{*}\ge 0\end{array}\right.$$

  where C is the penalty coefficient. The Lagrange function is introduced, $\alpha ,{\alpha }^{\mathrm{*}}$ is introduced for each constraint, and then the kernel function $k(x_i,x)$ is introduced in the process of solving $\alpha ,{\alpha }^{\mathrm{*}}$.
• The regression function is as follows [30]:

  $$f(x)={\displaystyle \sum _{i=1}^N({\alpha }_i^{*}-{\alpha }_i)k(x_i,x)}+b$$

  (3)

2.2. Model Building

In this work, a constitutive model of porous titanium materials was constructed according to Figure 1. The input characteristics in the model are strain, porosity, temperature and strain rate, and the response variable is stress.

2.2.1. Data Description

The sample data come from the experimental data in a previously published paper [4]. The experimental data are the stress–strain curves of porous titanium materials at different temperatures and strain rates, which are obtained by conducting orthogonal experiments using the split Hopkinson pressure bar (SHPB). The raw material of the experiment is high-purity titanium powder provided by Shijiazhuang Yida Co., Ltd. (Shijiazhuang, China), and the experimental material is made of cylindrical samples with a diameter of 8 mm and a height of 3 mm, 4 mm and 6 mm by using the powder sintering method. The sintering additive uses a 2 wt% polyvinyl alcohol aqueous solution. Considering the high chemical activity of titanium, it can react with oxygen, hydrogen, nitrogen and other elements, so the vacuum sintering method is used, and the vacuum degree is 10⁻⁴ Pa. The sintering temperature of titanium powder with a particle size less than or equal to 27 $\mathsf{\mu }\mathrm{m}$ is 1200 °C, the holding time is 2 h and natural cooling is achieved. The sintering temperature of titanium powder with a particle size less than or equal to 74 μm is 1100 °C, the holding time is 2 h and natural cooling is achieved. In the split Hopkinson pressure bar experiment, the sample size was adjusted according to different loading strain rates. A pressure bar with a diameter of 15 mm was used in the experiment, and the experimental conditions of different strain rates were achieved by adjusting the speed of the impact bar. The experiment was repeated 2 to 3 times for each group, and the average value of the effective data of the experimental results was taken to reduce the experimental error. The specific parameters of porous titanium materials are shown in

Table 1. Specific parameters of materials.

Porosity	Aperture	Grain Size	Ti	Fe	Cu	C	O	N
26%	15 $\mathsf{\mu }\mathrm{m}$	≤27 $\mathsf{\mu }\mathrm{m}$	≥99.7%	≤0.15%	≤0.005%	≤0.05%	≤0.2%	≤0.03%
36%	250 $\mathsf{\mu }\mathrm{m}$	≤74 $\mathsf{\mu }\mathrm{m}$	≥99.7%	≤0.25%	≤0.003%	≤0.06%	≤0.2%	≤0.03%

, which shows the percentage of each element of the experimental material in the overall mass, particle size, pore size and porosity, and the sample data are shown in

Table 2. Sample data information.

Porosity/%	Temperature/°C	Strain Rate/s−1
26	25	1200, 2000, 3000, 3600, 5200
	100	950, 1200, 2200, 3000, 4200
	200	800, 1500, 1950, 2750, 3800
	300	1200, 2000, 2900,3600, 3700
36	25	1000, 2000, 3000
	100	1380, 2050, 2350, 3400, 3700
	200	1000,1800, 2000, 2400, 3000
	300	1200, 2000, 3000, 3400, 4500

. A JSM-6480 scanning electron microscope (Nihon Technology Co., LTD., Osaka, Japan) was used to observe the surface morphology of the sample, and the results are shown in Figure 2.

Pearson correlation coefficient is used to view the degree of linear correlation between features. Its value ranges from −1 to +1. The larger the value, the stronger the correlation. Its expression [30] is as follows:

$$\mathrm{p}=\frac{\mathrm{cov}(X,Y)}{{\sigma }_{\mathrm{X}}{\sigma }_Y}=\frac{E(X-{\mathsf{\mu }}_X)(Y-{\mathsf{\mu }}_Y)}{{\sigma }_{\mathrm{X}}{\sigma }_Y}$$

(4)

where $\mathrm{cov}(X,Y)$ is the covariance of the data set $X,Y$; ${\mathsf{\mu }}_X,{\mathsf{\mu }}_Y$ are the average value of for $X,Y$; ${\sigma }_{\mathrm{X}},{\sigma }_Y$ are the deviation of $X,Y$, respectively. It can be seen from Figure 3 that most of the features are nonlinear. However, there is a certain positive correlation between stress and strain, which is caused by the proportional increase in stress in the plastic deformation stage and the nonlinear increase in the elastic stage. The porosity will reduce the mechanical strength of the material, so there is a weak negative correlation between porosity and stress.

2.2.2. Data Preprocessing

In order to improve the final prediction quality of the data quality assurance model, it is necessary to carry out data cleaning on the sample set, such as missing value processing, outlier processing and standardization processing. Among them, standardization processing can eliminate the dimension gap between the data and accelerate the convergence rate of the model. Here, the Z-score standardization method is used, and its expression [30] is as follows:

$$X=\frac{\mathrm{X}-\mu }{\sigma }$$

(5)

where $X$ is the value after data standardization; $\mathrm{X}$ is the value before data standardization; $\mu$ is the average value of the sample; $\sigma$ is the standard deviation of the sample. Because the activation function in the MLPR model has scale requirements on the data, if the scale difference between features is large, it will affect the convergence speed of the gradient descent method; so, the data used to construct the MLPR model need to be standardized. Because the calculation of the kernel function in the SVR model depends on the distance between features, the calculation accuracy of the kernel function will be affected if the feature scale is different, so the data used to construct the SVR model needs to be standardized. The prediction result of the RF model is mainly related to the distribution of variables and the conditional probability between variables, and has nothing to do with the size of the value itself; so, the data used to build RF model do not need to be standardized.

2.2.3. Data Set Partitioning and Model Evaluation

On the basis of data preprocessing, the sample set is divided into a training set and a test set. The training set is used to train the model, and the test set is used to test the model’s performance. In general, the more data used to train the model, the higher the prediction accuracy of the model, but excessive training will lead to a decline in the overfitting generalization ability of the model; so, the ratio of the training set to the test set is 8:2.

Goodness of fit $R^2,$ mean absolute error $MAE$ and mean absolute percentage error $MAPE$ were used to evaluate the performance of regression models. The best fit represents the degree to which the regression curve fits the experimental value, and the closer the goodness of fit is to 1, the better the regression curve fits the predicted value. The mean absolute error represents the mean of the absolute error between the predicted value and the experimental value. The calculation formula [31] is as follows:

$$R^2=1-\frac{{\displaystyle \sum _{i=1}^{\mathrm{n}}{(\stackrel{\wedge }{Y_i}-Y_i)}^2}}{{\displaystyle \sum _{i=1}^n{(Y_i-\overline{Y})}^2}}$$

(6)

$$MAE=\frac{1}{n}{\displaystyle \sum _{i=1}^n\left|(Y_i-\stackrel{\wedge }{Y_i})\right|}$$

(7)

$$MAPE=\frac{100\%}{\mathrm{n}}{\displaystyle \sum _{i=1}^{\mathrm{n}}\left|\frac{{\hat{Y}}_i-Y_i}{Y_i}\right|}$$

(8)

where $\stackrel{\wedge }{Y_i}$ represents the predicted value of the model, $Y_i$ represents the experimental value, $\overline{Y}$ represents the average of the experimental value, and $n$ represents the number of samples.

2.2.4. Model Hyperparameter Tuning

Random Forest Model (RF)

The number of weak learners (n_estimators) and the maximum depth (max_depth) are tuned in the RF model. In this study, the number of weak learners ranged from 10 to 235, with steps of 15, and the maximum depth ranged from 10 to 38, with steps of 2, respectively; $R^2$ and $MAE$ were used to evaluate the influence of parameter values on the predicted results. According to Figure 4a,b, it is found that $R^2$ increases with the increase in the number of weak learners and the maximum depth, and $MAE$ decreases with the increase in the number of weak learners and the maximum depth. However, after the number of weak learners reaches 70, $MAE$ fluctuation greatly occurs at 70 and 160, respectively, and 160 is taken as the minimum value after comparison. When the maximum depth is greater than 24, $R^2$ and·$MAE$ converge gradually; so, the optimal parameters of the number of weak learners and the maximum depth are 160 and 24, respectively.

Multi-Layer Perceptron Model (MLPR)

We now discuss how to tune the solver, activation and hidden_layer_sizes in the MLPR model. The solvers used for weight optimization include lbfgs, which is a quasi-Newtonian optimization method; sgd, which is a stochastic gradient descent method; adam, which is an optimization method based on stochastic gradient. Excitation functions include Rectified Linear Unit function (relu), hyperbolic tangent function (tanh), logistic function and identity function. The identity function is suitable for tasks with linear relationships that lack nonlinearity, but it limits the representation of the model. Both tanh and logistic belong to generalized S-type activation functions, both of which have bidirectional saturation, but there are still some problems such as gradient disappearance and large computation in the derivation process. The relu function solves the gradient disappearance problem and is fast to compute, but it may result in the “death” of neurons. The formula for these four activation functions are as follows:

$$relu:f(x)=\max (0,x)$$

(9)

$$\tanh :f(x)=\frac{e^x-e^{-x}}{e^x+e^{-x}}$$

(10)

$$\log istic:f(x)=\frac{1}{1+e^{-x}}$$

(11)

$$\mathrm{identity}:\mathrm{f}(x)=x$$

(12)

where x is the input value.

In order to improve the computational efficiency and generalization ability of the model, the multi-layer perceptron here uses only one hidden layer. As shown in Figure 5a,b, in the combination of 12 solvers and excitation functions, the relu function and lbfgs solver have the best performance; so, the MLPR model should use the relu function and lbfgs solver. According to Figure 5c, it is found that $R^2$ increases with the increase in the number of hidden layer elements, and $MAE$ decreases with the increase in the number of hidden layer elements. When the number of hidden layer elements is greater than 70, $R^2$ and·$MAE$ gradually converge, so the optimal parameter of the number of hidden layer elements is 70.

Support Vector Machine Model (SVR)

In the SVR model, the error penalty coefficient (C) and kernel function are optimized, and the influence of parameter changes on the prediction results is evaluated by using $R^2$ and $MAE$, respectively. C is an important parameter used to balance the complexity of the model and the accuracy of the fitted data. The larger the value, the more accurately the model learns from the sample, but the easier it is to overfit. The kernel includes linear kernel functions (linear), polynomial kernel functions (poly), radial basis kernel functions (rbf) and S-type kernel functions (sigmoid). The linear kernel function is fast to calculate, but its ability to fit nonlinear data is limited. The polynomial kernel function can handle nonlinear data to some extent, but the computational complexity is high. The radial basis kernel function has a strong ability to fit nonlinear data, but it is easy to overfit. Because the error penalty coefficient is too large, the model is easy to overfit, so C is taken from 0.1 to 100. Figure 6a,b shows that the SVR model should use the radial basis kernel function (rbf), and C should take 100.

2.3. Comprehensive Analysis of Model Prediction Ability

Comparing $R^2$ and $MAE$ of the optimized model, the results are shown in Figure 7. After tuning, $R^2$ of the three models is relatively different, and $R^2$ of the RF model is the highest. However, $MAE$ of the three models is quite different, among which the RF model has the smallest $MAE$. Compared with the three optimized models, the RF model has excellent comprehensive performance, so the RF model should be used for the establishment of a porous titanium constitutive model.

Zhu Tianyu et al. [4] used the DP model to build a constitutive model of porous titanium, and verified the accuracy of the DP model by comparing the predicted values of the model with the experimental values. The DP model was proposed by Drucker and Prager in 1952. It was initially applied to the study of plastic mechanics of geological materials. Later, researchers found that the model parameters could be adjusted to describe the plastic deformation process of metals. In this paper, the prediction ability of the DP model and RF model for porous titanium materials is compared and analyzed.

Porous titanium is temperature-sensitive due to its substrate material. As shown in Figure 8, when the temperature is less than 300 °C, the yield limit and flow stress of porous titanium decrease with the increase in temperature, indicating that the material has a temperature-softening effect. The RF model predicted the data well, and MAPE did not exceed 3.5%. The predicted value of the DP model can reflect the temperature sensitivity, but the error is large, and the average MAPE is as high as 11.4%.

Affected by the microstructure strain efficiency and the matrix material, porous titanium has strain rate sensitivity due to the influence of its matrix material [5,32,33,34]. From the perspective of the microstructure, the pore wall of porous titanium has plastic bending during the compression process, so the strain rate effect will be produced. As shown in Figure 9, when the strain rate is less than 3000/s⁻¹, the yield limit and flow stress of the porosity increase with the increase in the strain rate. When the strain rate reaches 3000/s⁻¹, the distribution of the stress–strain curve is irregular. The predicted results of the RF model can reflect the strain sensitivity of porous titanium materials, and the average MAPE is 3.8%. However, the prediction results of RF show small errors in part of the stress–strain curve. The predicted value of the DP model can reflect the strain rate sensitivity of porous titanium materials when the strain rate is less than 3000/s⁻¹, but the error is large, and the average MAPE is as high as 27.5%.

As shown in Figure 10, the presence of pores significantly reduces the mechanical strength of porous titanium materials. The RF model can predict the data well. The prediction results of the DP model can show that the existence of pores significantly reduces the mechanical strength of porous titanium materials, but the error is large.

As shown in Figure 8, Figure 9 and Figure 10, both the RF model and the traditional DP model can predict the strain rate effect and temperature sensitivity of porous titanium materials, but the prediction accuracy of the latter is lower than that of the former. Because the traditional constitutive model is usually based on physics and phenomenology and describes the structure and properties of materials through mathematical equations, its prediction accuracy is limited by its own explicit mathematical description ability. Machine learning models, on the other hand, predict the properties of materials by training large amounts of data to establish patterns and associations. As shown in

Table 3. Error comparison between DP model and RF model.

Model	$\mathbf{Maximum}\mathit{M}\mathit{A}\mathit{E}$	$\mathbf{Maximum}\mathit{M}\mathit{A}\mathit{P}\mathit{E}$	$\mathbf{Mean}\mathit{M}\mathit{A}\mathit{E}$	$\mathbf{Mean}\mathit{M}\mathit{A}\mathit{P}\mathit{E}$
RF	22.5	8.0%	10.9	4.4%
DP	185.9	44.9%	94.4	25.5%

, the average $MAPE$ of the RF model is much smaller than that of the DP model, so the predictive performance of the constitutive model constructed using the RF model is better than that of the DP model.

3. Results

• For the data set obtained by the split Hopkinson pressure bar experiment, the performance of the RF model is optimal when max_depth and n_estimators are 24 and 160, respectively. When the relu function is combined with the lbfgs solver and the number of hidden layer elements are 70, the performance of the MLPR model is optimal. When the rbf function is used and C is 100, the SVR model has the best performance.
• Based on the data set obtained from the split Hopkinson pressure bar experiment, the prediction performance of three machine learning algorithms, RF, MLPR and SVR, on porous titanium materials is discussed. Through comparative analysis, it can be seen that the prediction performance of the RF model with 160 weak learners and a maximum depth of 24 is the best, and $R^2$ is 0.9998, while $MAE$ is 1.413.
• The traditional DP model can predict the strain rate effect and temperature sensitivity of porous titanium materials. However, the prediction accuracy of the latter was lower than that of the former, with an MAPE of 4.4% for the RF model and 25.5% for the DP model.
• In general, the RF model has good predictive performance for the constitutive relationship of porous titanium materials.