Exploring Multi-Fidelity Data in Materials Science: Challenges, Applications, and Optimized Learning Strategies

Abstract

Machine learning techniques offer tremendous potential for optimizing resource allocation in solving real-world problems. However, the emergence of multi-fidelity data introduces new challenges. This paper offers an overview of the definition, applications, data preprocessing methodologies, and learning approaches associated with multi-fidelity data. To validate the algorithms, we examine three widely-used learning methods relevant to multi-fidelity data through the design of multi-fidelity datasets that encompass various types of noise. As we expected, employing multi-fidelity data learning methods yields better results compared to solely using high-fidelity data learning methods. Additionally, considering the inherent various types of noise within datasets, the comprehensive correction strategy proves to be the most effective. Moreover, multi-fidelity learning methods facilitate effective decision-making processes by enabling the combination of datasets from various sources. They extract knowledge from lower fidelity data, improving model accuracy compared to models solely relying on high-fidelity data.

1. Introduction

Multi-fidelity (MF) often appears in the field of science and engineering. It is often used to describe a variety of data or models with different precision, which are called MF data and MF models. Generally, in MF data, the quality of a large amount of data with a low acquisition cost is coarse (i.e., low-fidelity (LF) data), while there are also small amounts of data with very high accuracy (i.e., high-fidelity (HF) data). This is the so-called “cost-accuracy trade-off”. In practical applications, multi-fidelity data may not be limited to just two levels. The levels of data could be five [1,2] or even more. However, most of the recent works still largely focus on the performance of models trained on single-fidelity data [3,4,5]. These efforts either face issues from insufficient data or problems with poor data quality [6].

To overcome these problems, many methods are designed to exploit the multi-fidelity data efficiently, such as iterative denoising [1], information fusion [7,8], Bayesian optimization [9,10], etc. Iterative denoising enhances the model’s accuracy by iteratively training on MF data with specific permutations and applying denoising operations on specific data after each iteration. Information fusion algorithms fuse data information by learning the relationships between HF and LF data, effectively leveraging MF data. Bayesian optimization uses the correlation between HF and LF data to reduce the computational cost and improve the prediction efficiency when iteratively searching for the best value.

The systematic exposition and validation of multi-fidelity data learning methods in other fields have been extensively reviewed [11,12,13]. However, within the realm of materials science, there remains a notable absence of an overview and validation of multi-fidelity data learning methods. This paper aims to address this gap by presenting an introduction and validation of multi-fidelity data learning methods, with a specific focus on their application in materials science. The paper highlights the widespread presence of multi-fidelity data in the materials domain, emphasizing that employing multi-fidelity data learning methods in the face of such data scenarios is undoubtedly a promising choice. Additionally, the article introduces several typical multi-fidelity datasets within the materials domain, enabling interested readers to swiftly access them.

The partitioning of multi-fidelity data needs a combination between prior knowledge and quantitative evaluation methods. Through the consideration of the application fields and the methodologies for data acquisition, a categorization of the generation of multi-fidelity data are summarized in

Table 1. Different acquisition methods of MF data in different fields, including their respective references and citation counts (As of 3 December 2023).

Materials Science	Different Algoritdms	functional	[14,15,16,17,18] citations (273, 60, 4, 43, 25)
		experiment	[19] citations (18)
		functional and experiment	[1,2] citations (1, 71)
		physical models	[20] citations (14)
		functional and physical models	[9] citations (55)
	Different hyperparameters	mesh size	[21] citations (145)
		time step	[22,23] citations (21, 23)
		different hyper parameters	[24] citations (35)
Aerospace Science	Different Algorithms	CFD methodology	[25,26,27,28,29,30,31,32] citations (52, 19, 12, 40, 3, 6, 5, 20)
		CFD tool	[33] citations (16)
		CFD and experiment	[34] citations (14)
		multicopter dynamics and experiment	[35] citations (35)
	Different hyperparameters	mesh size	[36,37,38] citations (1, 60, 55)
Mechanical Engineering	Different Algorithms	mathematical models	[12,39,40,41,42] citations (17, 5, 40, 5, 35)
		measurement tools	[43,44] citations (43, 11)
	Different hyperparameters	mesh size	[45,46,47] citations (2, 1, 13)
		convergence condition and simulated objects	[10] citations (1)
		iteration limit	[48] citations (25)
Others	Different Algorithms	mathematical models	[49] citations (104)
		mathematical models and simulation tools	[50] citations (27)
	Different hyperparameters	main or simplified numerical model	[51] citations (1)
		mesh size	[52] citations (66)
		physical constraints	[53] citations (1)

. MF data often appear in the fields of materials science, aerospace science, and mechanical engineering, as shown in Figure 1. In every field, there are often a variety of different calculation results for the same object, and these results may have different accuracy and quantity. We will engage in a section-by-section discussion.

1.1. Multi-Fidelity Data from Different Algorithms

In the field of materials science, the existence of multi-fidelity data is primarily attributed to the diverse methodologies. These approaches often range from experimental procedures to theoretical calculations. Experimental methods are generally considered to be more time-consuming than their theoretical counterparts, but they offer a higher degree of accuracy. Here, for instance, some works solely focus on theoretical methods, and their key results are summarized in

Table 2. Distribution of high/low fidelity (HF/LF) data acquisition methods in the materials field.

Reference	LDA [58]	PBE [59]	PBE0  [60]	HSE [61,62]	Potential [63]
[9]		HF			LF
[14,15]		LF		HF
[16]	LF	LF		HF
[17]		LF	HF
[18]		LF		HF

. It is essential to recognize that even within the realm of theoretical computation, fidelity varies significantly. For example, the commonly used empirical potentials are generally considered low-fidelity (LF) methods when compared to density functional theory (DFT), which is often regarded as high-fidelity (HF). However, it is important to note that even within DFT, there exists a ladder of accuracy. The Perdew–Burke–Ernzerhof (PBE) functional, for instance, can sometimes be categorized as high-fidelity depending on the context, while in other cases, it might be considered low-fidelity. There are often systematic errors [54,55,56] and random errors compared to true values when using DFT calculations to determine material properties. For example, it has been reported in the literature that DFT calculations tend to underestimate the band gap width by 30% to 100% [57]. Random errors typically arise from the initial trial charge density used in the DFT calculations. Different trial charge densities can lead to varying random errors. Additionally, different convergence criteria in self-consistency may also introduce random errors. Systematic errors, on the other hand, are often introduced during the calculation of the Hamiltonian in DFT. The Kohn–Sham equations, which are used to compute the exchange–correlation potential and solve for the electronic wave functions, typically employ different approximation methods, in which different approximation methods introduce different errors. All of these errors arise during the DFT calculation process. For the same calculation target, the different errors resulting from different functionals can be viewed as discrepancies between HF and LF. These discrepancies can be corrected using multi-fidelity data learning methods.

In addition to different algorithm applications in materials science and aerospace science, different computational fluid dynamics (CFD) methodologies or different CFD tools (Q3D and MATRICS-V [33]) are commonly used to obtain MF data. Reynolds-Averaged Navier–Stokes (RANS) method is commonly used to obtain HF data [30,31,32], and the potential flow model to obtain LF data [26,29]. In reference [25], RANS is used to obtain LF data, while large-eddy simulations are used to obtain HF data. In reference [27], URANS and Euler models are used to provide LF and HF data, respectively. In reference [28], the doublet-lattice method and CFD-based flutter calculation are used to obtain LF and HF data, respectively. In mechanical engineering, different mathematical models (e.g., different finite element models [39,42] and surrogate models [12,40,41]) or different measuring tools (e.g., different strain sensors [43] and different wind-turbine-specific aero-servo-elastic computer simulators [44]) are commonly used to obtain MF data. For other fields, mainly in biomedical and economic fields, the methods of acquiring multi-fidelity data are listed in Table 1.

In addition to the MF data obtained through theoretical calculations, more accurate data can also be obtained through manual experiments, and the data obtained through experiments are often used as the data with the highest fidelity [1,34,35].

1.2. Multi-Fidelity Data from Different Hyperparameters

Different hyperparameters under the same method may also lead to the generation of MF data. The hyperparameters here are generally some parameters that affect the computational complexity. For example, in the theoretical calculation, different mesh sizes are often used to obtain MF data, in which the smaller mesh corresponds to the HF and the larger mesh corresponds to the LF [21,36,45]. In the field of materials science, by using the same functional but specifying different parameters (such as plane wave truncation energy, k point, atomic relaxation, etc.), the results obtained are also different [24]. In addition, MF data is also obtained using different time steps [22,23] or different convergence conditions and simulated objects [10]. Different iterations of the calculation method are also one of the ways to generate MF data. The data obtained from partial convergence [64] can be regarded as LF data, while the data obtained from full convergence can be regarded as HF data. For example, in reference [48], the results of approximately six hundred iterations and one hundred iterations are regarded as HF and LF, respectively.

The acquisition method of MF data determines the fidelity of the data. The fidelity type of the MF data obtained in different ways needs to be judged by the evaluation method or prior knowledge. The selection of evaluation method or the use of prior knowledge also affects the fidelity of the data.

1.3. Multi-Fidelity Datasets under Materials Science

In the field of materials science, machine learning is mostly used to predict the property value of materials in reality, while the known existing and widely used datasets are as follows: Materials Project (MP) [65], Open Quantum Materials Database (OQMD) [66] and other datasets, which are usually calculated by density functional theory (DFT) or other theoretical methods, and there are different degrees of error with the real attribute values in the real world. In addition, there are even a variety of measurement methods with different results for the properties of some materials [67].

Multi-fidelity data is a prevalent phenomenon within the domain of materials science, extensively manifested in extant datasets. For example, in the MP dataset, there are multiple calculation methods for a certain attribute of the same substance, most of which are theoretical calculations. The attribute value calculated by the corresponding method can be obtained by calling the API in MP. Subsequently, this section introduces two large datasets with a wide range of MF data: the band gap width dataset calculated by different density functional in MP [1,2] and the molecular ultraviolet/visible spectral attribute dataset [19] obtained from different experiments, as shown in Figure 2 and Figure 3.

As can be seen from Figure 2, the data amount calculated by the PBE method is far more than that calculated by HSE, SCAN, and GLLB. PBE datasets are often used as LF data due to their large volume, and there are also articles that divide them and assign weights of different sizes to them as MF data, and on this basis, study the MF data learning method [68].

In the field of materials, there are more than two datasets that are used to construct MF models at the same time. For example, in reference [1], four different datasets in Figure 2 are used as MF datasets at the same time, and the influence of different combination methods on the training results is discussed. The reader can obtain an MF dataset with a band gap in MP from https://doi.org/10.6084/m9.figshare.13040330 (accessed on 3 December 2023).

Figure 3 shows the dataset containing dyes and solvents of optical materials. These data were obtained through different experiments, and contain their quantities, storage forms (e.g., names or SMILES), etc. Figure 3 shows the quantities of dyes and solvents of optical materials in nine common datasets, from largest to smallest, as follows: Deep4Chem [69], ChemDataExtractor [70], DSSCDB [71], ChemFlour [72], Dye aggregation [73], NIST [74], Fluorophores.org [75], PhotochemCAD [76], and UV/Vis+ [77]. Reference [19] adopts the top five datasets for the experiment. The relevant data and their code can be found in reference [19].

2. Multi-Fidelity Data Learning Methods

The key starting point of an MF data learning method is to find the relationship between MF data. The learning methods for MF data mainly include the following two directions [6]: (1) the hierarchical way is used to organize HF and LF model information, which is called the multi-fidelity hierarchical model (MFHM) [78,79,80]. (2) The HF and LF model information is fused through surrogate model construction, which is called the multi-fidelity surrogate model (MFSM) [7]. This article focuses on MFSM. Surrogate models, also known as meta models, response surface models, or approximation models, are simplifications of the true model. The advantage of using surrogate models is that the physical meaning between model input values and response values can be ignored, and one can directly focus on the data themselves for mathematical modeling [81].

MFSM can be divided into deterministic methods (DM) and non-deterministic methods (NDM) according to the estimation methods for parameters of MFSM [6]. A DM finds their parameters by minimizing the difference between data and functions [82,83]. An NDM assumes that the function or parameter is uncertain and uses the sample to reduce the uncertainty [84]. The most popular in NDM is the Bayesian framework [9,10,85], where the posterior distribution of the model parameters depends on the likelihood distribution and prior distribution of unknown parameters. A common implementation method of the Bayesian framework is Gaussian process (GP) [86,87,88]. In the validation part, the NDM of GP is used. The reason for employing GP lies in its established track record of effectively handling multi-fidelity data and delivering favorable outcomes [89,90,91]. Statistics indicate that among the numerous methods used for constructing surrogate models, 37% utilize GP [92].

In this paper, we utilize GP for subsequent experimental validation. The chosen task type is regression, for which we deliberately introduced various types of noise into a small and low-dimensional dataset. In the case of MF datasets with diverse noise types, traditional linear regression struggles to handle them effectively [93]. GP, being an NDM, provides estimates of uncertainty in predictions, which proves highly valuable when dealing with data containing diverse types of noise [94]. On the other hand, references [95,96] found that NDMs tend to be more accurate compared to DMs. Regarding MFSM, whether a DM or NDM, the correction approaches for MFSM can be classified into three major categories: additive correction, multiplicative correction, and comprehensive correction. We describe each of these methodologies in the following sections.

2.1. Additive Correction

Additive correction (AC) [97] corrects the response value of the LF data by constructing a surrogate model of the difference between the HF data and the LF data, which is used as an approximation of the response value of the HF model. The equation can be expressed as Equation (1):

$${\widehat{y}}_h\left(x\right)=y_l\left(x\right)+\delta \left(x\right)$$

(1)

In Equation (1), ${\widehat{y}}_h\left(x\right)$ is the approximation of the HF response value at x, while $y_l\left(x\right)$ is the response value of the LF data at the same position. The term $\delta \left(x\right)$ is a surrogate model that captures the difference between the HF data and the LF data at x. The core idea lies in combining the LF data response value $y_l\left(x\right)$ and the difference function $\delta \left(x\right)$ to approximate the response value of the HF data, a technique termed as the additive correction. $\mathsf{\Delta }$-learning [98,99] is also considered as an additive correction method.

2.2. Multiplicative Correction

Multiplicative correction (MC) [100] corrects the response value of the LF data by constructing a surrogate model of the ratio between the HF data and the LF data. The equation can be expressed as Equation (2):

$${\widehat{y}}_h\left(x\right)={\rho \left(x\right)y}_l\left(x\right)$$

(2)

In Equation (2), ${\widehat{y}}_h\left(x\right)$ is the approximation of the HF response value at x, while $y_l\left(x\right)$ is the response value of the LF data at the same position. The key element here is the surrogate model $\rho \left(x\right)$, which represents the model of the ratio between HF and LF data. Specifically, the product of the LF response value $y_l\left(x\right)$ and the ratio function $\rho \left(x\right)$ forms an approximation of the HF response value.

2.3. Comprehensive Correction

When the HF response value cannot be approximated by a simple additive correction or multiplicative correction, it is better to consider using a comprehensive correction (CC), which integrates additive correction and multiplicative correction. Generally, it can be expressed as Equation (3):

$${\widehat{y}}_h\left(x\right)={\rho \left(x\right)y}_l\left(x\right)+\delta \left(x\right)$$

(3)

where $\rho \left(x\right)$ and $\delta \left(x\right)$ are the multiplicative correction surrogate model and the additive correction surrogate model, respectively, and $\rho \left(x\right)$ is usually replaced by a constant $\rho$ [9,90,95,101]. The replacement equation is as follows:

$${\widehat{y}}_h\left(x\right)={\rho y}_l\left(x\right)+\delta \left(x\right)$$

(4)

In the validation section of this paper, Gaussian process regression (GPR) [102,103] is employed as the surrogate model. GPR is based on Bayesian principles, wherein it characterizes model uncertainty by constructing a prior distribution and updates this distribution based on the observed data to obtain a posterior distribution. Specifically, GPR utilizes a kernel function to compute the correlations between the data and subsequently constructs a covariance matrix. The procedure is as follows:

Assuming we have a dataset $(X,y)$, where X and y represent the input features of the data and their corresponding labels, and we have new data $x_{N^{\prime }}$ that we want to predict using GPR, GPR constructs a covariance matrix K that describes the relationships among all the data (including the white noise):

$$K=\left[\begin{array}{cc}{K}_N+{\sigma }_n^{2}I& K_{N{N}^{\prime }}\\ K_{N^{\prime }N}& K_{N^{\prime }{N}^{\prime }}\end{array}\right]$$

(5)

where $K_N$ represents the covariance matrix of the original data X, $K_{N{N}^{\prime }}$ and $K_{N^{\prime }N}$ represent the covariance matrices between the data points X and the data point to be predicted, and $K_{N^{\prime }{N}^{\prime }}$ represents the covariance matrix of the data point to be predicted. ${\sigma }_n^2$ represents the variance of the white noise added during modeling, and I denotes the identity matrix. For each element within the covariance matrix, it can be computed using a covariance function; in this case, we employ the Gaussian kernel function (also known as the radial basis function):

$$K^{ij}=k(x,x^{\prime })={\sigma }_f^{2}exp\left(-\frac{1}{2{\sigma }_l^2}{\parallel x-x^{\prime }\parallel }^2\right)+{\delta }_{ij}{\sigma }_n^2$$

(6)

where ${\sigma }_f$ and ${\sigma }_l$ are hyperparameters that control the characteristics of the covariance function. ${\delta }_{ij}$ is used to ensure that only elements where $i=j$ are affected by the noise term, representing that the observation errors between different data points are uncorrelated. $\parallel x-x^{\prime }\parallel$ represents the Euclidean distance between two data points. Once the estimation of model hyperparameters is completed, the mean $\left({\mu }_{N^{\prime }}\right)$ and variance $\left({\Sigma }_{N^{\prime }}\right)$ of the prediction result are obtained by maximizing the conditional likelihood:

$${\mu }_{N^{\prime }}=K_{N^{\prime }N}{(K_N+{\sigma }_n^{2}I)}^{-1}y$$

(7)

$${\mathsf{\Sigma }}_{N^{\prime }}=K_{N^{\prime }{N}^{\prime }}-K_{N^{\prime }N}{(K_N+{\sigma }_n^{2}I)}^{-1}{K}_{N{N}^{\prime }}+{\sigma }_n^{2}I$$

(8)

In practical applications, the prediction result is often represented by the mean $\left({\mu }_{N^{\prime }}\right)$.

In the correction methods used in this paper’s validation, GPR is employed as the surrogate model. Specifically, for the comprehensive correction, a constant $\rho$ is used to replace the multiplicative correction term, and the equation is as follows:

$${\widehat{y}}_h\left(x\right)=\rho {\widehat{y}}_l\left(x\right)+\delta \left(x\right)$$

(9)

where ${\widehat{y}}_l\left(x\right)$ and $\delta \left(x\right)$ represent surrogate models for the LF data and discrepancy data, respectively. The value of $\rho$ can be obtained by minimizing the following equation:

$$min:{\left(\delta \left(x_h\right)-d_h\right)}^2$$

(10)

$$d_h=\rho {\widehat{y}}_l\left(x_h\right)-y_h$$

(11)

The solution for $\rho$ can be determined through the method of least squares to minimize Equation (10). For the given Equation (10), it can be treated as a quadratic function with respect to $\rho$. To find the value of $\rho$ that minimizes the function, you can calculate its derivative and set it equal to zero for solving.

MF data learning methods have their own advantages and disadvantages. For example, when calculating $\rho \left(x\right)$ in the multiplicative correction, if $y_l\left(x\right)$ as the denominator is zero, then $\rho \left(x\right)$ has no solution. Additive correction can make up for the deficiency of multiplicative correction, but it is not unexpected that additive correction is not always better than multiplicative correction, which is proven in reference [104,105]. The comprehensive correction method makes up for the shortcomings of additive correction and multiplicative correction to a certain extent, but a major drawback of this method is that it can only deal with two kinds of linearly related fidelity data, while in some cases [8,106], the relationship between LF and HF data follows a nonlinear relationship. For this case, reference [21] decomposed the relationship between HF and LF data into linear and nonlinear parts and solved it using a neural network.

Next, we evaluate the performance of these three correction methods based on GPR.

3. Performance Validation of Existing Learning Algorithms

This section demonstrates the learning methods of the above three MF data through validation. The method of artificially constructing MF data is adopted to apply additive correction, multiplicative correction, and comprehensive correction methods, respectively, on the datasets containing additive noise, multiplicative noise, and mixed noise (i.e., containing both additive and multiplicative noise). This section introduces both MF data acquisition and processing, as well as the validation results.

3.1. Multi-Fidelity Data Acquisition and Processing

In this validation, MF datasets are artificially constructed to obtain datasets. Here, we assumed an ideal function as Equation (12) to compute the true values.

$$Y_{true}\left(x\right)={\mathit{p}}_{\mathit{1}}\ast \mathit{sin}\left({p_2\ast x}^2+p_3\right)\ast \left({p_4\ast x}^2+p_5\right)+p_6\ast \left(p_7\ast x^3+p_8\right)+p_9\ast x+p_{10}$$

(12)

where $p_1$∼$p_{10}$ are random parameters. We conducted a total of one hundred tests to validate our approach. For brevity, we present the details of one with parameters (in Equation (12)) set as $P_{sample}$ to typify the behavior observed across one hundred repetitions. The parameters selected are as follows:

$$P_{sample}=[-1,-5,-4,5,-1,-2,1,0,-4,-2]$$

(13)

where the true value function corresponding to the parameter $P_{sample}$ is shown in Figure 4. We selected the results of one of these for display, as shown in Figure 5, which includes the distribution of HF and LF data points, true value, HF data fitting model, and different correction methods under different noises.

In a certain range, this function can exhibit wave-like behavior. In reality, we can assume that the function curve is obtained from a plane wave. Based on this assumption, we can add additive noise, multiplicative noise, and mixed noise to the function for different MF data learning methods. The LF data input value of the validation was two hundred points equidistant from zero to one, denoted by $X_l$. The input values for the HF data are contained in the LF data input values, which are twenty equidistant points between zero and one, denoted by $X_h$. The response values of HF and LF data are represented by $Y_h$ and $Y_l$, respectively. The equation is as follows:

$$Y_h={E_{h_{\ast }}\ast Y}_{true}+E_{h_{+}}$$

(14)

$$Y_l={E_{l_{\ast }}\ast Y}_{true}+E_{l_{+}}$$

(15)

where $E_{h_{\ast }}$ and $E_{l_{\ast }}$, respectively, represent multiplicative noise under HF and LF; and $E_{h_{+}}$ and $E_{l_{+}}$, respectively, represent additive noise under HF and LF. The additive noise in the HF data is relatively small, while the additive noise in the LF data is relatively large. The multiplicative noise in the HF data is close to one, while the multiplicative noise in the LF data is far from one. The above formula is the construction formula for the MF data in the comprehensive correction method. In Equations (14) and (15), we can set parameters $E_{h_{\ast }}$ and $E_{l_{\ast }}$ to one for generating datasets only containing additive noise and set parameters $E_{h_{+}}$ and $E_{l_{+}}$ to zero to generate datasets with multiplicative noise only. During validation, for the datasets with additive noise, random numbers ranging from 0 to 1 are generated. These numbers are then multiplied by 0.4 and 0.8 to obtain the values for $E_{h_{+}}$ and $E_{l_{+}}$. For multiplicative noise, we randomly select numbers within the range of 0.9 to 1.1 to represent $E_{h_{\ast }}$, and within the range of 0.8 to 1.3 to represent $E_{l_{\ast }}$, respectively.

The above HF and LF datasets with different noise added are randomly shuffled as the training set, while the input values of the test set were an additional twenty points from zero to one that were not included in the HF and LF datasets. The mean square error (MSE) was used as an evaluation method, and the experiment was repeated one hundred times randomly with different parameter arrays P to calculate the mean MSE between each MF data learning method and the true value and the mean MSE between the high-fidelity data model (HFDM) and the true value.

3.2. Validation Results

Through the above MF data learning method, it is not difficult to find from Figure 5 that the results obtained by using the MF data learning method between the range of 0.55 and 0.75 are closer to the ‘True value’ than those obtained by the high fidelity data model (HFDM). This is because the amount of HF data is small, leading to the low accuracy of the fitted model. Conversely, the MF data learning method can simultaneously use HF and LF data, and the HF data in the unknown location can be approximated by the known LF data, so the fitting results are relatively good.

The experiments were repeated one hundred times, randomly utilizing different parameter arrays P. In each model construction, the method of 5-fold cross-validation [107,108] is used to find out the model with the best parameters, and on this basis, the training and testing are carried out. The average MSE between each MF data learning method and the true value, as well as the average MSE between HFDM and the true value, were calculated. The results are presented in

Table 3. Average MSE of different methods on datasets with different noises, the lowest values are highlighted in bold.

Method	Average MSE
	Additive Noise	Multiplicative Noise	Mixed Noise
HFDM	2.545	2.531	2.389
AC	0.065	0.753	0.501
MC	0.608	0.609	1.535
CC	0.057	0.436	0.425

. For MF datasets with arbitrary noise, comprehensive correction (CC) has the lowest MSE and the high-fidelity data model (HFDM) has the highest MSE. For MF datasets with only additive noise, the MSE of additive correction (AC) is lower than that of multiplicative correction (MC). For MF datasets with only multiplicative noise, the MSE of MC is lower than that of AC.

The experimental results show that the results obtained by using the multi-fidelity data learning method are better than those obtained by using only high-fidelity data modeling. Among the three correction methods, the comprehensive correction has the best effect, which is corrected on the basis of additive and multiplicative correction. In addition, we also used RMSE, MAE, $R^2$, and other evaluation methods for testing. We found that the multiplicative correction outperforms the additive correction only in datasets with multiplicative noise. However, in datasets with other types of noise, the multiplicative correction does not outperform the other correction methods. On the contrary, the comprehensive correction consistently yields the best results across all noise types. This further validates the advantage of the comprehensive correction method across various noise datasets. In summary, employing MF data learning methods can enhance model accuracy when faced with limited HF data and abundant LF data. When the noise of multi-fidelity datasets is not known, the comprehensive correction method can obtain better results. The choice of specific correction methods largely depends on the MF dataset. Finally, We also used two statistical test methods, Shapiro–Wilk [109] and Levene [110], to test the mean error (ME) of the four methods under different noises. Readers can find all the relevant codes and results in the data availability statement for this paper.

4. Outlook and Limitations

In this paper, we focus on the application of multi-fidelity (MF) data in materials science, including methods for acquiring MF data, datasets in previous works, and machine learning algorithms. Additionally, we applied different machine learning methods to handle simulated MF data with different types of noise (additive, multiplicative, and mixed) as validation. Our results indicate that it is helpful to introduce a larger amount of lower fidelity data to work together with expensive higher fidelity data.

In an era where materials science is increasingly becoming data driven, the transformation from a single-fidelity-data driven approach to a multi-fidelity-data driven paradigm seems inevitable. As a part of this ongoing evolution, we propose the following outlook and limitations for the future research in the area of multi-fidelity data driven methods:

• Classic methods, such as the Gaussian process regression as demonstrated in this study, will continue to play a pivotal role. For this paper, while classic methods such as Gaussian process regression prove effective, their computational demands might limit their performance on larger datasets or higher-dimensional problems. For researchers, choosing the appropriate model and having a clear understanding of the algorithm and its principles remain indispensable.
• Alongside hardware advancements, newer methods are surfacing that offer innovative ways to address challenges. Techniques such as large language model (LLM) warrant close attention, as they could redefine how we handle multi-fidelity problems. However, concurrently, utilizing LLM or similar complex technologies may introduce challenges associated with interpretability, computational resources, and data requirements.
• Active learning strategies, especially when integrated with both experimental and theoretical multi-fidelity data, hold the potential for groundbreaking contributions. Machine learning models can determine not only material structures or features but also which levels of data fidelity are most informative for exploration. However, a pertinent issue is that active learning strategies necessitate high-quality and representative initial datasets. If the initial data lack quality or do not aptly represent the studied domain, it may impact the effectiveness of learning. Acquiring suitable multi-fidelity datasets is, thus, a critical concern for researchers.