Supervised Machine Learning to Predict Drilling Temperature of Bone

Abstract

Surgeons face a significant challenge due to the heat generated during drilling, as excessive temperatures at the bone–tool interface can lead to irreversible damage to the regenerative soft tissue and result in thermal osteonecrosis. While previous studies have explored the use of machine learning to predict the temperature rise during bone drilling, this in vitro study introduces a comprehensive approach by combining the Response Surface Methodology (RSM) with advanced machine learning techniques. The main objective lies in the comprehensive evaluation and comparison of support vector machine (SVM) and random forest (RF) models specifically for the optimization of the bone drilling parameters to prevent thermal bone necrosis. A total of 27 experiments were conducted using a multi-level factorial method, with analysis performed via the Minitab software version 19.1. Performance metrics such as the mean squared error (MSE), mean absolute percentage error (MAPE), and coefficient of determination (R²) were used to assess model accuracy. The RF model emerged as the most effective, with R² values of 94.2% for testing and 97.3% for training data, significantly outperforming other models in predicting temperature fluctuations. This study demonstrates the superior predictive capabilities of the RF model and offers a robust framework for the optimization of surgical procedures to mitigate the risk of thermal damage.

1. Introduction

Bone drilling is an essential procedure in orthopedic surgery, dental implant placement, and other medical interventions that involve the insertion of screws, plates, or prostheses into the bone [1]. However, one of the significant challenges with bone drilling is the generation of excessive heat, which can lead to a phenomenon called thermal osteonecrosis. Thermal osteonecrosis refers to the death of bone cells (osteocytes) due to elevated temperatures [2,3].

The consequences of thermal osteonecrosis are far-reaching. The affected bone areas may become weakened, leading to poor implant integration, prolonged healing times, or increased susceptibility to post-operative infections [4]. For surgeries that demand robust bone healing, like fracture repairs or spinal fusions, thermal osteonecrosis can even lead to treatment failure [5,6]. Thermal osteonecrosis can be managed by optimizing the operation of the surgical drilling system [7]. Achieving this often necessitates precise predictions, such as estimating the bone drilling temperature based on the optimal combination of the drilling parameters.

The field of machine learning provides efficient techniques for the prediction of bone drilling temperatures based on data [8]. As a branch of computer science, machine learning focuses on how computers can learn autonomously without being directly programmed with specific rules. It encompasses two primary areas: supervised learning and unsupervised learning [9,10]. In supervised learning, predictive models are developed using labeled data, involving tasks like predicting drilling temperatures through regression. Conversely, unsupervised learning deals with unlabeled data and is often utilized for tasks such as outlier detection through clustering [11].

Machine learning techniques are increasingly employed in the modern Industry 4.0 to forecast uncertainty and manage vast amounts of data. Pandey and Panda [12] used machine learning models to alleviate drilling-induced damage to bovine bone. Akgundogdu et al. [13] utilized machine learning methods to evaluate the bone characteristics and categorize different specimens. Support vector machine (SVM) techniques have demonstrated significant success in characterizing the rigidity of trabecular bone samples. Lu et al. [14] implemented machine learning models to improve the anticipation of postsurgical facial profiles using video imaging. Agarwal et al. [15] conducted a comparison of various machine learning models to predict the mechanical strength of orthopedic bone screws, while Agarwal et al. [16] confirmed different assumptions regarding the use of machine learning models in forecasting thermal injuries. Torun and Öztürk [17] employed machine learning models for robotic bone drilling, noting the effectiveness of closed-loop signals in detecting drill bit damage and the morphology during sheep femur bone drilling.

The aim of developing predictive models is to create models that can accurately predict new data that are not available during their creation [18]. A model trained on a specific dataset cannot be evaluated on the same dataset because it retains a “memory” of it. Therefore, the entire dataset is typically split into two parts: one for the training of the model, known as the training set, and the other for the evaluation of the model, known as the test set. This approach allows for the estimation of the model’s generalization capabilities, indicating how well it will perform on unseen data. It is crucial to ensure that no information from the test set leaks into the training set when developing predictive models. Models built with training sets containing test set information will produce overly optimistic results.

The experiment was designed and optimized using the Response Surface Methodology (RSM) technique. The collected data were analyzed to develop two predictive models for the temperature increase using the support vector machine (SVM) and random forest (RF) methods. The main purpose of these models was to improve the evaluation of bone workability. Therefore, the research objectives of this study were as follows.

• To investigate the crucial machinability factors affecting the bone drilling process, with a particular emphasis on the temperature distribution. By exploring the thermo-mechanical properties of these factors, we aim to enhance our understanding and improve the efficiency of bone drilling procedures.
• To evaluate the performance of bovine bone under varying machining conditions, specifically focusing on the effects of the spindle speed, feed rate, and drill bit diameter. Additionally, we seek to analyze the thermo-mechanical interactions occurring during the drilling process.
• To enhance the machining parameters and determine the best combinations of the spindle speed, feed rate, and drill bit diameter to reduce thermal osteonecrosis in bone by utilizing the Response Surface Methodology (RSM).
• To deploy ML models, particularly random forest (RF) and support vector machine (SVM), to forecast the highest temperature increase. To assess the model’s accuracy and performance in predicting the machining results based on the input variables and confirm the precision and dependability of the ML models.

2. Materials and Methods

The thermal and mechanical characteristics of fresh bovine bone are uniform and closely mimic those of human bone [19,20]. The preparation process for the experimental bone drilling involving bovine femur specimens is shown in Figure 1. The epiphysis (end) of the fresh femur was removed using a saw. Subsequently, the diaphysis (middle part) was chosen for experimentation [21,22], and the periosteum, which is the soft tissue attached to the outer cortical bone, was eliminated to prevent the obstruction of the drill bit’s flute, as recommended in previous studies [23,24,25,26]. Specimens with a cortical thickness exceeding 8 to 13.2 mm were selected for inclusion in the study [27,28].

This investigation used bovine femur bone as the study material because human bone was not accessible. Bovine bone is considered the closest analog to human bone in terms of its properties. The femoral bone from cattle was obtained promptly after slaughter from a local slaughterhouse. The bone specimens were stored in saline water at −20 °C to preserve their mechanical and thermo-physical properties, and the experiments were conducted within a few days of procurement.

The machinability study was performed with a radial drilling machine, employing a high-speed steel (HSS, Bosch, Petaling Jaya, Malaysia) drill bit with a standard cutting point head geometry of 118°. The temperature readings were recorded using a multi-channel temperature meter connected to a T-type thermocouple [29,30]. The data were stored in an external storage repository. The bone workpiece was firmly fixed in a vice, and various process parameters, as outlined in

Table 1. Process parameters used in the experiment.

Machine: Sainsmart Genmitsu 3018-PROVer CNC (Sainsmart, Las Vegas, NV, USA)
Cutting Tool: HSS 118°
Process parameter	Spindle speed (rpm)	Feed rate (mm/min)	Drill bit diameter (mm)
	1000	20	2
	1500	30	3
	2000	40	4

, were selected. The input variables considered in this research were the spindle speed, feed rate, and drill bit diameter, while the response variable measured was the elevated average maximum temperature.

2.1. Response Surface Methodology

The Response Surface Methodology (RSM) is a set of mathematical and statistical techniques designed for the modeling and analysis of problems where multiple variables influence a response variable, with the primary goal being to optimize this response [31,32]. Minitab, a statistical software program, provides robust tools for the performance of the RSM. The first step in the RSM is to design experiments that yield sufficient data to model the response surface, using designs such as central composite designs (CCD) and Box–Behnken designs (BBD). Once the data from these experiments are collected, the next step is to fit a polynomial model, typically a second-order (quadratic) model, to the data. This model can be expressed as in Equation (1):

$$y={\beta }_0+{\sum }_{i=1}^k{\beta }_i{x}_i+{\sum }_{i=1}^k{\beta }_{ii}{x_i}^2+{\sum }_{i<j}^k{\beta }_{ij}{x}_i{x}_j+ϵ$$

(1)

where $y$ is the response variable, $x_i$ and $x_j$ are the input variables, ${\beta }_0$ is the intercept, ${\beta }_i$ is the linear coefficient, ${\beta }_{ii}$ is the quadratic coefficient, ${\beta }_{ij}$ is the interaction coefficient, and $ϵ$ is the error term. Minitab uses regression analysis to estimate the coefficients $\beta$ of this polynomial model, which includes performing an analysis of variance (ANOVA) to determine the significance of the model and individual predictors, as well as a lack-of-fit test to check the model’s fit to the data. After fitting the model, Minitab generates response surface plots and contour plots to visualize the relationships between the input factors and the response. These plots aid in understanding how changes in the input variables affect the response and help to identify optimal conditions. The final step is to find the optimal settings of the input variables that maximize or minimize the response. Minitab’s optimization tool uses the fitted model to identify these settings, often by solving the optimization problem using Equation (2):

$$Maximize or Minimize:f\left(x\right)={\beta }_0+{\sum }_{i=1}^k{\beta }_i{x}_i+{\sum }_{i=1}^k{\beta }_{ii}{x_i}^2+{\sum }_{i<j}^k{\beta }_{ij}{x}_i{x}_j$$

(2)

2.2. Support Vector Machine for Regression (SVR)

Support vector regression (SVR) is a nonlinear regression method based on support vector machines (SVM) [33,34]. It aims to construct a hyperplane that reduces the prediction errors and maximizes the margin between the hyperplane and the closest data points. An epsilon tube is used to define acceptable error boundaries. The algorithm applies a kernel function to project the input data into a higher-dimensional feature space, allowing for the creation of the hyperplane and the minimization of errors in this new space. The structure of SVM regression is illustrated in Figure 2.

SVR can be expressed as a convex optimization problem according to Equation (3). Convex optimization ensures that any local minimum is a global minimum, simplifying the task of determining the optimal hyperplane. The optimization problem aims to minimize the error within a specified margin of tolerance, often incorporating a regularization term to prevent overfitting.

$$\mathrm{M}\mathrm{i}\mathrm{n}\mathrm{i}\mathrm{m}\mathrm{i}\mathrm{z}\mathrm{e}:{\displaystyle \frac{1}{2}}{\mathrm{W}}^2+\mathrm{C}\sum _{\mathrm{i}=1}^1\left({\mathsf{\xi }}_{\mathrm{i}}+{\mathsf{\xi }}_{\mathrm{i}}^{\mathrm{*}}\right)$$

$$Constraints:\left\{\begin{array}{c}{Y}_i-〈W,X_i〉-b\le \epsilon +{\xi }_i,\\ 〈W,X_i〉+b-Y_i\le \epsilon +{\xi }_i^{\mathrm{*}},\\ {\xi }_i,{\xi }_i^{\mathrm{*}}\ge 0,\end{array}\right.$$

(3)

where $Y_i$ represents the classes, and $X_i$ denotes a set of features. The parameter b corresponds to the width of the hyperplane, and W signifies the margin. To handle situations where the constraints become infeasible, slack variables ${\xi }_i$ and ${\xi }_i^{*}$ are introduced. The parameter C is used to achieve a trade-off between the width of the margin and the degree of misclassification, effectively controlling the balance between achieving a wider margin and allowing some errors in classification. The process of building an SVM can be visualized through the flow chart presented in Figure 3, which outlines the steps involved in training and predicting the model.

SVR finds application in various fields requiring continuous value predictions, such as financial forecasting, time-series prediction, and other domains necessitating robust regression analysis. Understanding SVR requires a solid grasp of the basic concepts of machine learning, optimization, and linear algebra. This technique is powerful because of its ability to handle high-dimensional data and provide accurate predictions.

2.3. Random Forest Regression (RFR)

Random forest regression (RFR) is a supervised learning algorithm that falls within the ensemble learning family and is utilized for tasks such as regression and classification [35]. During the training phase, numerous decision trees are created. Each decision tree in the forest is constructed using a random subset of the training data and features, promoting diversity and reducing overfitting. The final output from RFR is obtained by averaging the predictions made by each tree, which enhances the accuracy and robustness. This method capitalizes on the strengths of multiple models to produce a more reliable and stable prediction compared to a single decision tree, as shown in Figure 4. RFR is widely favored for its simplicity, versatility, and superior performance in handling complex datasets and capturing non-linear relationships.

2.4. Data Pre-Processing and Performance Evaluation Metrics

The experimental data, capturing the average maximum temperatures recorded during bone drilling at three distinct parameter levels, were imported into Google Colab using the Pandas Python library (version 3.10.12). Pandas is widely used for its robust capabilities in data manipulation, enabling the efficient handling and analysis of complex datasets.

The SVM and RF models’ performance was evaluated using standard metrics for model assessment, namely the mean squared error (MSE), mean absolute percentage error (MAPE), and coefficient of determination (R²), described by the following equations:

$$R^2=1-{\displaystyle \frac{\sum _{i=1}^n{\left(y_i-\widehat{y}\right)}^2}{\sum _{i=1}^n{\left(y_i-\overline{y}\right)}^2}},$$

(4)

$$\mathrm{MAPE}={\displaystyle \frac{1}{\mathrm{n}}}\sum _{\mathrm{i}=1}^{\mathrm{n}}{\displaystyle \frac{|{\mathrm{y}}_{\mathrm{i}}-{\widehat{\mathrm{y}}}_{\mathrm{i}}|}{{\mathrm{y}}_{\mathrm{i}}}}\times 100,$$

(5)

$$\mathrm{MSE}={\displaystyle \frac{1}{n}}\sum _{i=1}^n{(y_i-\widehat{y})}^2,$$

(6)

where n denotes the quantity of data points, $y_i$ represents the observed values, $\widehat{y}$ stands for the predicted values, and $\overline{y}$ denotes the mean value of y.

3. Results and Discussion

In optimization, the Design of Experiments (DOE) plays a pivotal role in systematically planning and conducting experiments. By doing so, the DOE ensures that the data collected are reliable and meaningful, which helps in drawing valid conclusions. This methodology reduces the necessity for a large number of experiments, which can be resource-intensive, time-consuming, and costly, while still maintaining high levels of precision in the results.

The accuracy of the results is a key consideration in this context. Ensuring accuracy means that the outcomes of the experiments are not only precise but also reflect the true effects of the variables being studied. This is where the Response Surface Methodology (RSM) becomes particularly valuable. The RSM is a set of statistical and mathematical techniques that are used to model and analyze problems where multiple variables influence the responses. For instance, in medical applications, regarding the drilling temperature of bone during surgery, the RSM can be used to model how different factors (like the spindle speed, feed rate, and drill bit diameter) affect the temperature. By analyzing these factors, researchers can optimize the drilling process to minimize the thermal damage to the bone, thereby improving the surgical outcomes and patient recovery.

A total of 27 experiments were performed utilizing the multi-level factorial method, as shown in

Table 2. Temperature observed in case of drilling bovine bone.

Spindle Speed (rpm)	Feed Rate (mm/min)	Drill Bit Diameter (mm)	Average Tmax (°C)
1000	20	4	36.4
1000	30	3	35.9
1000	40	3	47.7
2000	40	2	45.2
1000	40	4	43.9
1500	40	3	49.5
1000	30	2	36.8
1500	30	2	42.5
1000	20	2	36
1000	30	4	37.5
2000	30	3	54.5
2000	20	3	49.9
2000	40	4	62.5
1500	30	4	41.7
2000	20	2	44
2000	20	4	53.6
1500	20	2	37.5
2000	40	3	63.8
1500	20	3	43.4
1500	40	4	48.6
1500	40	2	37.5
1000	40	2	35.5
1500	20	4	42.9
1500	30	3	46.5
1000	20	3	35.7
2000	30	2	49
2000	30	4	55.6

, with the Minitab software aiding in the analysis. Figure 5a illustrates the drilled holes within the bone and Figure 5b shows the schematic diagram of the thermocouple location. The temperature during the drilling process was continuously monitored using a T-type thermocouple, capable of measuring temperatures from 0 to 260 °C, and recorded with an Appellant AT4808 multi-channel thermometer data logger (Applent Instruments, Changzhou, China). The results from these experiments are summarized in Table 2.

3.1. Optimization Using RSM

The Response Surface Methodology (RSM) optimization technique was implemented to determine the optimal combination of input variables, such as the spindle speed, feed rate, and drill diameter, which resulted in the lowest feasible process temperature. Figure 6 illustrates the optimization procedure that was designed to consider the model’s predicted minimum process temperature and desirability constraints. The red color represents the optimization input, the blue color indicates the predicted outcome of the experiment, and the black dots correspond to the level of each parameter as shown in Figure 6. The results of the validation process, which encompass a thorough comparison with the experimental data, are comprehensively presented in

Table 3. Optimization to achieve the lowest temperature.

Optimization	Spindle Speed (rpm)	Feed Rate (mm/min)	Dill Bit Diameter (mm)	Average Tmax (°C)
Initial drilling parameters	1000	20	2	36
Model prediction	1000	40	2	35.05
Experimentation	1000	40	2	35.50
Error percentage	-	-	-	0.45%

. Notably, the model exhibits a remarkably low error rate of just 0.45%, underscoring its exceptional accuracy and reliability.

Within the studied range of input values, the process temperature can be reduced to a minimum of 35.05 °C. The most favorable outcomes were achieved using a tool diameter of 2 mm, a feed rate of 40 mm/min, and a rotation speed of 1000 rpm. As shown in Figure 6, a comprehensive array of adjustable input parameters is available to ensure that the temperature applied to the bone remains safe, effectively minimizing the risk of damage to the bone tissue.

3.2. Prediction of Response Variables by SVM and RF

In this research, two distinct regression-based machine learning models were employed to forecast the response values derived from a bone drilling procedure. The dataset was divided randomly, with 70% allocated for model training and the remaining 30% for testing and validation purposes. The models utilized three key input variables—the spindle speed, feed rate, and drill bit diameter—to predict the temperature generated during drilling. To mitigate potential errors arising from differences in the units of these input parameters, both the training and testing datasets were normalized using one-hot encoding. Furthermore, cross-validation was conducted to evaluate and optimize the models by employing a grid search CV to guarantee the selection of the most appropriate parameters while simultaneously considering both training and testing errors.

The temperature values for bone drilling were derived from 27 experiments utilized for both training and testing.

Table 4. Predicted bone drilling temperature values from different machine learning algorithms.

Experimental Temperature (°C)	RFR Model Predicted Temperature (°C)	SVR Model Predicted Temperature (°C)
35.7	38.442	42.360
49.9	51.764	45.013
54.5	52.77	45.013
36	36.976	42.360
36.4	38.538	42.360
36.8	37.726	42.360
42.9	43.051	43.585
43.9	44.299	42.361

displays the varied predictions from the support vector regression (SVR) and random forest regression (RFR) models.

Table 5. Performance metrics for different machine learning algorithms.

Bone Drilling Temperature Predictor Model Errors
	Test Errors		Training Errors
	SVR model	RFR model	SVR model	RFR model
R2	0.249	0.942	0.257	0.973
MAPE	0.125	0.033	0.115	0.026
MSE	33.496	2.568	32.349	2.148

lists the testing and training errors, assessed using three performance indicators to gauge the prediction accuracy.

3.3. Comparison of Machine Learning and RSM Models

The precision and resilience of both the machine learning and standard RSM predictive models were assessed utilizing three distinct error metrics: MSE, MAPE, and R². A comparison of these metrics is presented in Figure 7. It was noted that the machine learning predictive model (RFR) demonstrated superior error metrics in comparison to the standard RSM model. Thus, one can deduce that the RFR model is the most resilient and appropriate predictive model for the forecasting of the temperature during bone drilling.

The spindle speed was deemed crucial for all outcomes. It was noted that the rise in the drill bit diameter led to an increase in the temperature caused by drilling as the helicoid formation deepened and expanded. In contrast, the temperature declined as the feed rate rose. Ein-Afshar et al. [36] documented similar discoveries while examining the drilling procedure on bone. The presence of non-uniform structures in bone samples presents obstacles in attaining uniform surface smoothness and a consistent temperature rise along the entire length of a bored hole. The cutting operation had a notable impact on the adjacent tissue of the bone samples. As per the error metrics, the random forest (RF) model demonstrated superior performance compared to the support vector machine (SVM) regression model. The prediction of the response variables by RF and SVM is plotted and compared with the experimental values in Figure 8, showing that the RF predictions were closer to the experimental values, thereby reducing the error and making RF a better-performing model.

4. Conclusions

This experimental study used machine learning models to predict temperature rises during bone drilling. Tests were conducted on bovine femur bone with different drilling settings to develop predictive algorithms. The study examined how various drilling parameters affected the temperature increases throughout the process, leading to several significant findings.

• The RSM method found that the best operating conditions were achieved with a tool diameter of 2 mm, a feed rate of 40 mm/min, and a spindle speed of 1000 rpm.
• This study found that the RFR model outperformed the SVM regression model and RSM, as evidenced by the lower errors in the performance metrics when comparing the three methods.
• The precision in predicting the bone drilling temperatures using the RSM method is 79%, while it is 97.3% with RFR and 25.7% with SVR models.
• The machine learning RFR model enhanced the drilling efficiency and reduced the risk of bone thermal damage by accurately predicting the temperature rises.

4.1. Significance of the Study

This study holds significant theoretical and practical importance for the machining of bone and bone-like materials. It enhances the drilling techniques, leading to more efficient and effective drilling operations by determining the optimal drilling parameters for drilling surgeries. The research highlights the impressive capabilities of machine learning techniques, particularly the RF model, in accurately predicting and optimizing the drilling temperatures. This success suggests a promising future for AI-driven solutions in industrial settings. It encourages the further exploration of machine learning’s potential to enhance the precision and efficiency across various industries. The broader implications of this study extend beyond its specific findings, influencing manufacturing processes, sustainability efforts, and the scope of machine learning in industrial optimization. The insights gained from this research can lead to improved practices and significant advancements, benefiting researchers, companies, and society as a whole.

4.2. Limitations

This study has inherent limitations that need to be acknowledged. The complexity and difficulty of the experimental process resulted in a somewhat limited dataset for the training of the machine learning models. This restriction could impact the accuracy of the predicted temperature values, as models trained on smaller datasets may not perform as well. Additionally, if this research is extended to in vivo real-time medical surgeries, monitoring temperature variations at the drill site presents another challenge. The potential technical solutions include the use of more advanced sensor technologies like real-time temperature sensors placed in the drill bit and the development of new monitoring methods to overcome the identified limitations. The limited duration of surgery and the constraints in using thermocouples for temperature measurement add to the difficulty.