*3.3. Evaluating Optimization and Regression Results*

The EA determined the hyperparameters of the models to maximize the average coefficient of determination for the validation data sets. The best individuals or parameter combinations are shown in Table 7. A large number of neurons, or many estimators in the ensemble methods, increase the adjustable model parameters, the risk of overfitting to the training data, and poor prediction accuracy for test data. However, the hyperparameters causing overfitting were not chosen by the optimization to maximize the number of model parameters to reach high values for the prediction accuracy based on the training data. In general, this is a first indication that a generalization capable model was created by the training and optimization.


**Table 7.** Optimized parameters of the regression models.

The hyperparameters optimized by the EA were used for training the algorithms. Afterwards, the models were evaluated using the coefficient of determination *R*<sup>2</sup> for test and training data, see Figure 10. For the training data, acceptable values for *R*<sup>2</sup> were obtained for all algorithms. The quantile distance for the normalized *x*˜c-coordinate of the cage reached *<sup>R</sup>*<sup>2</sup> ≈ 0.41 in the case of the random forest regressor, which is to be assessed as a medium correlation. The excitation of the cage, as well as the translational center of mass movement, occurs both for the contact of the cage to the rolling elements and to the rib in the radial direction. The relationship between the geometry and load parameters as well as the axial center of mass movement and finally the *R*<sup>2</sup> of the predictions were therefore lower compared to the other center of mass coordinates. Random forest regressor predicted very well for all target values, but reached a slightly lower *R*<sup>2</sup> compared to XGboost and the neural network for training data. The random components in the random forest algorithm (e.g., feature selection) prevent possible overfitting to the training data and led to slightly inferior prediction. The coefficients of determination *<sup>R</sup>*<sup>2</sup> ≈ 1 for XGBoost were very high and indicate a significant fit to the data sets.

The test datasets generally showed a lower coefficient of determination than the training datasets but were within an acceptable range apart from the quantile distance of the normalized *<sup>x</sup>*˜c coordinate of the cage. qd(*x*˜c) exhibited the worst values of *<sup>R</sup>*<sup>2</sup> ≈ 0.41 for the random forest and *<sup>R</sup>*<sup>2</sup> ≈ 0.6 the ANN. Thus, while qd(*x*˜c) is suitable for assessing cage dynamics when derived from calculated time series, there is no strong correlation to bearing load or cage geometry. The difference in prediction accuracy for training and test data was lowest for random forest, which indicates a generalization of the model. However, the difference for XGBoost and the ANN was also in an acceptable range, which is also a sufficient generalization capability. All models reached comparable values for the coefficient of determination *R*<sup>2</sup> based on the test data sets and thus can be used equally for the prediction of cage dynamics. The best prediction values for *R*<sup>2</sup> based on the test data sets were obtained for the quantile distance of the equivalent deformation force *F*e, the median of the Ω-ratio and the median of the friction torque *T*<sup>f</sup> in the range of *<sup>R</sup>*<sup>2</sup> ∈ [0.90 ... 0.94]. For the remaining target parameters, with the exception of qd(*x*˜c), at least one of the models investigated achieved a coefficient of determination *R*<sup>2</sup> > 0.8 and sufficient prediction accuracy.

**Figure 10.** Coefficient of determination *R*<sup>2</sup> for the target variables of the regression algorithms Random Forest, XGBoost, and Neural Network for training (**a**) and test data (**b**).

The scatter plot in Figure 11 shows, representative of the trained models, the predictions of the ANN compared to the true values in the training (blue) and test (red) data. As can be seen from the correlation matrix and the coefficient of determination, the predictions for the quantile distance of the axial coordinate of the cage qd(*x*˜c) were considerably more scattered than the other target variables. For the quantile distance of the omega ratio, the deviation of the predictions from the test data sets was smaller, but a stronger, though still acceptable, scatter was also present here. For the remaining parameters, a good correlation was present, analogous to the *R*2. The deviations are within a tolerable range, as can be seen by the intervals containing 90% of the errors determined for the test data (blue area).

The hyperparameters obtained from the optimization by the EA were used to perform a 10-fold cross-validation. This allowed us to determine how strong the predictions of the algorithm differ depending on the used training and test data set, see Figure 12. Based on this, the sensitivity of the prediction results for different training and test data sets could be investigated. Figure 12 exhibits the distribution of the average prediction of the target values for the training data and a 10-fold cross-validation including (a) and excluding (b) the quantile distance of the cage coordinate *x*˜c as regression target. For all three models, omitting the normalized coordinate improves the average prediction quality, as lower *R*<sup>2</sup> values are obtained for qd(*x*˜c) than for the other values in all iterations of the cross-validation. The minima and maxima of *R*<sup>2</sup> for the three models without considering *x*˜c in the cross-validation were very similar and differ only slightly. As no obvious favorite could be identified based on the prediction accuracies, all three algorithms were suitable for predicting the cage dynamics with a comparable error tolerance.

**Figure 11.** Scatter plot of the target parameters for training (**blue**) and test (**red**) data sets and the predicted values by the neural network. The colored area represents the range where 90% of the errors for the test data sets are located.

**Figure 12.** Distribution of *R*<sup>2</sup> values for all target variables (**a**) and without the normalized *x*˜ccoordinate (**b**) for a 10-fold cross-validation. Besides the minimum and maximum, the distribution of the values is also illustrated.

#### **4. Discussion**

The results of the dynamics simulation illustrate the strong influence of the bearing load and cage geometry on the resulting cage dynamics, see Figure 7. Depending on the geometry, the tendency of a cage to highly dynamic and unstable cage movements varies significantly. However, the relationship between the geometry parameters and the resulting cage dynamics is very complex and difficult to determine using conventional methods of descriptive statistics, as can be seen from the covariance matrix in Figure 9. The complex relationship between the input and output parameters can basically be determined with the help of the investigated algorithms. Analyzing the prediction results for test data, it can be seen that for the quantile distance of the normalized center of mass coordinate *x*˜c of the cage, mediocre prediction values could be obtained. As the frictional forces acting in contact between the cage and the other components accelerate the cage primarily in the bearing plane, the physical relationship between the input parameters of the model and the resulting axial cage motion is less than for the other parameters. The normalized *x*˜c coordinate of the cage is thus less suitable for predicting the cage dynamics. Though, as a component of the multivariate metric CDI, which can be derived from calculated time series representing cage dynamics, *x*˜c is a contribution to improve the classification performance.

The algorithms Random Forest, XGBoost, and ANN achieved similar values for the *R*<sup>2</sup> of the different target variables for the test data sets, see Figure 10. A 10-fold cross-validation exhibited that the differences between the models are small, and thus all algorithms are suitable for the prediction of the cage dynamics. The robustness of the predicted targets for a given cage geometry with respect to deviations from the true values can be improved by a large number of predictions by the regression algorithm with a subsequent statistical analysis. This reduces the influence of single incorrect predictions and improves the comparability of the dynamic behavior of different cage variants.

A transfer of the predictions to other rolling bearing sizes is possible in general. For this purpose, new training data must be generated and the existing database expanded. However, a similar effect on the cage dynamics can be expected, especially for the load conditions as shown, for example, by Schwarz et al. for various bearings [14,21]. Therefore, the amount of training data for the same bearing type and similar cage shapes can probably be lower than for the investigated angular contact ball bearing. In addition to the extension to other bearing types, other parameters can also be added as input variables, so that depending on the existing application, the model can also be designed flexibly. As with the geometry parameters, new data sets must be created for the training, but the database established so far serves as an initial starting point for further investigations.

#### **5. Summary and Conclusions**

The aim of this paper was to present a procedure for predicting the dynamics of cages in an angular contact ball bearing using dynamics simulations and machine learning regression methods. To achieve this aim, the approach in this paper is structured as follows: starting with a comprehensive simulation study, a database was created to represent the relationship between the input (cage geometry and bearing load) and output (cage dynamics and bearing friction) parameters for the regression models. As part of the training, the hyperparameters of the random forest, XGBoost, and artificial neural network models were optimized using an evolutionary algorithm. The optimized hyperparameters were used to train the regression models. The prediction accuracy of the models was compared using the coefficient of determination *R*<sup>2</sup> and regression plots. Based on the models and their predictions, the dynamics of the cage represented by the target variables can be predicted with high accuracy. The following conclusions can be drawn from the results of this paper:

• The cage geometry has a significant influence on the resulting cage dynamics. The occurrence of unstable cage movements can be significantly reduced by changing the geometry of the cage.


**Author Contributions:** Conceptualization, S.S., M.B., H.G., O.G.-G., S.T. and S.W.; methodology, S.S., H.G., M.B., O.G.-G., S.T. and S.W.; software, S.S.; validation, S.S., H.G., M.B., S.T. and S.W.; formal analysis, S.S., H.G. and M.B.; investigation, S.S., H.G. and M.B.; resources, O.G.-G. and S.W.; data curation, S.S.; writing—original draft preparation, S.S., H.G. and M.B.; writing—review and editing, O.G.-G., S.T. and S.W.; visualization, S.S.; supervision, O.G.-G., S.T. and S.W.; project administration, O.G.-G., M.B. and S.W.; and funding acquisition, H.G., S.S., S.T., O.G.-G. and S.W. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was executed within the project Machine Learned Dynamics (MeLD), which is funded by Bayerische Forschungsstiftung (BFS) grant number AZ-1398-19.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** The data presented in this work are available on request from the corresponding author.

**Acknowledgments:** We acknowledge financial support by Deutsche Forschungsgemeinschaft (DFG) and Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU) within the funding programme "Open Access Publication Funding".

**Conflicts of Interest:** The authors declare no conflicts of interest.
