**3. Results**

*3.1. Stability of Stable and Metastable Phases*

From CALPHAD-based calculations, one can observe the OMEGA (ω) phase at cryogenic temperatures as low as 50 K [15]. At these temperatures, phase stability calculations provided a high amount of the OMEGA (ω) phase (above 0.8 mole fraction). With a rise in temperature, the OMEGA (ω) phase decreased and reached zero even below room temperature. In order to stabilize the OMEGA (ω) phase, few phases including HCP\_A3 (α), BCC\_B2 (β) and ALTI3\_D019 (α") were suppressed/removed while performing phase stability calculations. This stabilized the OMEGA (ω) phase, and we were able to stabilize the OMEGA (ω) phase at higher temperatures through the CALPHAD approach. From experiments, it has been confirmed that the OMEGA (ω) phase is present in the same amount after processing/heat treatment [8–15]. Thus, it was important to stabilize the OMEGA (ω) phase for better understanding of its formation and stability over a large temperature range.

Figure 1 shows the relative comparison of the occurrence of the OMEGA (ω) phase over a large range of temperature (0–1500 K). In Figure 1, the entire temperature range was divided into five parts. About 3000 candidate alloy compositions were analyzed in this temperature range. The number of cases was recorded for which the OMEGA (ω) phase was observed in each of these temperature ranges. Regarding legends, "α and β stable" means HCP\_A3 (α) and BCC\_B2 (β) were included in the phase stability calculations and both phases were stable. Legend "α" only" means that in this case, both HCP\_A3 (α) and BCC\_B2 (β) were removed while performing equilibrium calculations for stabilizing ALTI3\_D019 (α") phase. Legend "ω only" means that HCP\_A3 (α), BCC\_B2 (β) and ALTI3\_D019 (α") phases were removed while performing phase stability calculations for stabilizing the OMEGA (ω) phase.

**Figure 1.** Occurrence (%) of the OMEGA (ω) phase for different temperature ranges for three separate equilibrium calculations.

From Figure 1, we can see that the OMEGA (ω) phase can be stabilized at elevated temperatures through the CALPHAD approach as observed through experiments [8–15]. In order to stabilize the OMEGA (ω) phase at elevated temperatures through the CALPHAD approach, one needs to remove HCP\_A3 (α), BCC\_B2 (β) and ALTI3\_D019 (α") phases along with a few other phases while performing phase stability calculations. Through the CALPHAD approach, a user needs to perform separate calculations each time they need to analyze a particular composition or temperature for determining metastable phases. This approach is time consuming as a user needs to have access to the computer on which CALPHAD-based software is installed.

Next, we move forward to application of AI algorithms on phase stability data generated through the CALPHAD approach. AI algorithms will be helpful in developing accurate predictive models that can capture trends and patterns within a large dataset.

#### *3.2. DLANN Model*

As mentioned before, DLANN models were selected on the basis of physical metallurgy of titanium alloys as well as on error metrics. DLANN architecture and error metrics (MSE and MAE) over the validation set are listed in Table 2. From Table 2, one can notice that values of MSE are acceptable, but values of MAE are a bit high. The amount of phase varied between zero and one for each of the stable and metastable phases included in this work, while MAE varies between 0.01783 to 0.03574. Thus, MAE for this work is between approximately 1.8% and 3.6% of the maximum amount of any phase. We have mentioned before that 67% of data were assigned to the training set and 33% of data were included in the testing or validation set. Thus, there is room for improvement in prediction accuracy (error metrics) by increasing the amount of data in the training set. However, while working on accuracy, we must be careful as ANN models are susceptible to "overfitting". Thus, based upon physical metallurgy of titanium alloys, error metrics in the present case and our own experience in handling such problems, we selected the models listed in Table 2 for further analysis.

**Table 2.** Performance metrics for deep learning artificial neural network (DLANN) models for various phases for the Ti-Nb-Zr-Sn system.


DLANN models were used as a predictive tool and can be used on a computer and even on an Android device. As mentioned before, metastable phases are absent in the presence of stable phases while performing phase stability calculations under the framework of the CALPHAD approach [29,33,34,39]. We used the alloy composition and temperatures included in the dataset obtained from initial calculations containing only stable phases and then predicted metastable phases for these alloy compositions and temperatures through DLANN models. Thus, DLANN models were used to obtain an improved dataset for further analysis through SOM.

#### *3.3. Self-Organizing Maps (SOM)*

SOM analysis [25,28,31,42] was performed on the data obtained through the CAL-PHAD approach and DLANN models. From CALPHAD and DLANN analysis, we have a matrix of 3000 rows and nine columns. Here, rows are 3000 candidate alloys. Columns are alloy compositions (Ti, Nb, Zr, Sn), temperature and the phases BCC\_B2, HCP\_A3, ALTI3\_D019, and the OMEGA phase. Thus, we included all the design variables and the objectives. Calculations were performed in batch mode, where all the designs are introduced to the SOM algorithm with value of X unit set at 15 and Y unit assigned a value of 18 [28,31]. Thus, there are 270 map units on a SOM map. Each map unit is in the form of a hexagonal cell and candidate alloys are positioned at the vertices of the hexagonal units. The 3000 candidate alloys along with temperature and concentration of phase values are arranged over 270 units on the SOM maps on the basis of algorithm setting. Other parameters were optimized so that SOM maps are able to capture trends in the dataset [28,31,42].

In this work, we used a commercial software ESTECO-modeFRONTIER for SOM analysis [42]. This software provides a user with two types of error values: quantization error and topological error. Quantization refers to the ability of the SOM algorithm to learn from data distribution. As mentioned, about 3000 candidate alloy compositions and temperatures and amounts of stable and metastable phases are presented in batch mode. These 3000 candidates are positioned at the vertices of 270 hexagonal unit cells on SOM maps. SOM analysis provides these candidates with new prototype positions on the SOM map. Quantization error is an estimate of the average distance between the initial position of a candidate and its prototype position assigned through SOM analysis.

The SOM algorithm is known for preserving the topology of the dataset. As mentioned, there are 270 hexagonal map units and candidates are arranged on each of these units. Through topology error, the algorithm checks for the relative position of a candidate with respect to candidates positioned in adjacent hexagonal map units. Thus, initially all the candidates are positioned on the SOM maps and as per SOM algorithm settings, all of these candidates are assigned new prototype positions. Through topology error, the SOM algorithm determines the relative distance between initial and prototype positions of candidates in the neighboring hexagonal units. This way, all the candidates positioned on the SOM maps are checked.

The SOM model was chosen on the basis of error metrics of a model and capability of a model to mimic trends shown in the literature for Ti-based biomaterials. Physical metallurgy of Ti-based alloys was given a priority while error metrics acted as a guiding tool. SOM error metrics have been reported in Table 3. Here, we can observe that model error for SOM is quite low. Hence, we moved ahead with analyzing the SOM maps for understanding patterns within the dataset.

**Table 3.** Self-organizing maps (SOM) error metrics for the Ti-Nb-Zr-Sn system.


Figure 2 shows the SOM component plot for the Ti-Nb-Zr-Sn system. For SOM analysis, BCC\_B2#2 phase was not included as there were too many missing points and also due to the fact that it is another form of same BCC\_B2 phase included in the TCTI2 [37] database. From Figure 2, we can observe that BCC\_B2(β) and HCP\_A3( α) are positioned together. Components positioned together are correlated in SOM maps. From physical metallurgy of titanium alloys, we know that titanium alloys in practice are either predominantly α or β, or a mixture of both in different proportions [2,28]. Thus, the stability of α and β phases is correlated from a metallurgical point of view. The SOM algorithm was able to determine correlations that can be verified from reported works on titanium alloys, even though the SOM algorithm is an unsupervised machine learning approach and does not work on the principle of Gibbs energy minimization [25,28,31].

ALTI3\_D019 ( α") is close to HCP\_A3 ( α) and can be correlated. The OMEGA ( ω) phase is far enough from other cells so we cannot confirm that it is correlated with the other components. Temperature is below BCC\_B2 (β) and HCP\_A3 ( α) and close to Sn and Zr. Temperature is not close enough to these components and we cannot provide a concluding remark on the correlation between temperature and other components. Elements Ti and Nb are clustered together similar to Zr and Sn. The OMEGA ( ω) phase is close to Ti and Nb, but not close enough to point towards strong correlation. Thus, SOM analysis provided us with vital information on various strong and weak correlations among alloying elements, stable and metastable phases, and temperature for the Ti-Nb-Zr-Sn system. Now, we will proceed further to analyze each of these components.

**Figure 2.** SOM components plot for the Ti-Nb-Zr-Sn system.

Figure 3 shows the SOM maps for HCP\_A3 (α), BCC\_B2 (β), ALTI3\_D019 (α") and OMEGA (ω) phases along with chemical concentrations of Nb, Zr and Sn and temperature. From Figure 3, one can observe that for temperature the lowest value on the color bar is 645 K and the highest value is 1232 K, while in Table 1, the range of temperature was between 50 K and 1526 K. The reason for this is that we have analyzed about 3000 candidate alloys through SOM. As mentioned before, each candidate is placed on the vertices of hexagonal cells on SOM maps. The SOM algorithm is used for pattern recognition in small to large and often multi-dimensional datasets. In SOM maps, various regions are marked on the bases of average values of candidate alloys placed on the vertices of a hexagonal cell. Thus, a region marked 645 K in the figure consists of six candidate alloys for which the average temperature is about 645 K.

**Figure 3.** SOM plot showing chemical concentrations, temperatures and resulting stable and metastable phases for the Ti-Nb-Zr-Sn system.

From literature, we know that HCP\_A3 ( α) is stable at lower temperatures and BCC\_B2 (β) is stable at higher temperatures [2,28]. In Figure 3, we can observe that same pattern for HCP\_A3 ( α) and BCC\_B2 (β) phases. At higher temperatures, one can fully stabilize BCC\_B2 (β) phase, while suppressing formation of HCP\_A3 ( α), ALTI3\_D019 ( α") and OMEGA ( ω) phase. With respect to composition, one needs to design compositions in a way that Nb is between average to low value, Sn is below average value and Zr is average and below average. Here, the average value refers to the color bar for the compositions in Figure 3.

Figure 4 shows the distribution for titanium and temperature along with BCC\_B2 (β), HCP\_A3 ( α), ALTI3\_D019 ( α") and OMEGA ( ω) phase. From Figure 4, one can observe that a user must maintain titanium at average composition as shown through color bar in the figure. At the average titanium composition, along with elevated temperature, a user can design compositions that will be predominantly the BCC\_B2 (β) phase and these candidates are expected to be free from the HCP\_A3 ( α), ALTI3\_D019 ( α") and OMEGA (ω) phase.

**Figure 4.** SOM plot showing composition (Ti), temperature and resulting stable and metastable phases for the Ti-Nb-Zr-Sn system.

From Figures 3 and 4, one can observe that OMEGA ( ω) predicted through the DLANN model is stable for a wide range of temperatures and compositions. Figure 1 shows a similar trend of occurrence of the OMEGA ( ω) phase over a wide temperature range. Figure 1 was plotted using data from CALPHAD-based calculations, where a user needs to perform calculations separately for stabilizing metastable phases. Through AI algorithms, all of this can be achieved at the same instant. AI-based predictions can be performed on a normal computer for free as we have developed our code in Python language, which is free.

From this work, five candidate alloy compositions and temperatures were identified (Table 4). These alloys are expected to have a fully stabilized BCC\_B2 (β) phase and to be free from other phases such as HCP\_A3 ( α), ALTI3\_D019 ( α"), and OMEGA ( ω) phase. For these select alloys, the amount of OMEGA ( ω) phase obtained through phase stability calculations, stabilizing OMEGA ( ω) phase and value of OMEGA ( ω) phase predicted through DLANN models and SOM maps were all zero.


**Table 4.** Candidate alloys predicted through CALculation of PHAse Diagram (CALPHAD), DLANN models and SOM approach, with zero concentrations of HCP\_A3 (α), ALTI3\_D019 (α"), and OMEGA (ω) phases.

## **4. Discussion**

This research problem had the main goal of determining the compositions and temperatures for Ti-Nb-Zr-Sn alloys, which will provide an alloy that is predominantly BCC\_B2 (β) phase and free from other phases such as HCP\_A3 (α), ALTI3\_D019 (α"), and OMEGA (ω) phase. In this work, this task was accomplished through combined CALPHAD and artificial intelligence (AI).

We identified one publication [2] on thermodynamic modeling on Ti-based biomaterials, which can be compared with our current work. In that work [2], the author performed first-principle calculations along with thermodynamic modeling within the framework of the CALPHAD approach for predicting metastable phases in the Ti-Nb-Zr-Sn-Ta system [2]. The author listed as his future work that he will work on the development of models based on first-principle calculations for predicting the α" and ω phase [2] and indicated plans to study the effect of Sn addition in larger amounts in order to study its effect on the stability of the β phase [2]. Another work [16] based on shape memory alloys can also be compared with the present work. In their work [16], the authors performed first-principle calculations for developing ω-phase free Ti-Ta-X systems. Both of these references are thorough works and have included results from first-principle calculations [2,16]. Density functional theory (DFT) or first-principle calculations are computationally expensive, and a user needs to have access to supercomputers for performing DFT-based study. Additionally, one of these works [2] was performed in 2017 when Thermo-Calc did not have a commercially available database for Ti-based alloys. In the last few years, there has been significant development in improving the database of Ti-based alloys [23,36–39]. The Ti-based alloy database now includes several new elements, which means several new equilibriums [37]. Many new models have been included for predicting various stable and metastable phases [37]. Thus, in the current work, it was possible to address a few of the limitations mentioned in these references [2].

In this work, we used Thermo-Calc [23,36] along with the TCTI2 database [23,36–39]. Our objective was to accelerate the process of discovery of new alloy compositions for Ti-based biomaterials and temperatures at which the β phase is fully stabilized. Thus, we relied on existing CALPHAD-based models and generated data for stability of various stable and metastable phases. Thereafter, we chose to develop models for various stable and metastable phases through the application of artificial intelligence algorithms.

Notice that no work was presented on improving the models for α" and ω-phase through first-principle calculations, as this was not within the scope of the present work. The purpose was to develop models that can be used for predicting the concentrations of stable and metastable phases in a few seconds. Consequently, DLANN models developed in this work can be used on a personal computer and even on a normal Android phone.

The SOM algorithm was further helpful in determining various correlations among chemical compositions, temperatures, and concentrations of stable and metastable phases. Determining these correlations were mentioned in the future work of one of the articles that dealt with thermodynamic modeling [2]. The current work demonstrates that it is

possible to efficiently predict a few candidate alloys that are expected to meet requirements regarding the stability of β phase.
