Rubber Material Property Prediction Using Electron Microscope Images of Internal Structures Taken under Multiple Conditions

Abstract

A method for prediction of properties of rubber materials utilizing electron microscope images of internal structures taken under multiple conditions is presented in this paper. Electron microscope images of rubber materials are taken under several conditions, and effective conditions for the prediction of properties are different for each rubber material. Novel approaches for the selection and integration of reliable prediction results are used in the proposed method. The proposed method enables selection of reliable results based on prediction intervals that can be derived by the predictors that are each constructed from electron microscope images taken under each condition. By monitoring the relationship between prediction results and prediction intervals derived from the corresponding predictors, it can be determined whether the target prediction results are reliable. Furthermore, the proposed method integrates the selected reliable results based on Dempster–Shafer (DS) evidence theory, and this integration result is regarded as a final prediction result. The DS evidence theory enables integration of multiple prediction results, even if the results are obtained from different imaging conditions. This means that integration can even be realized if electron microscope images of each material are taken under different conditions and even if these conditions are different for target materials. This nonconventional approach is suitable for our application, i.e., property prediction. Experiments on rubber material data showed that the evaluation index mean absolute percent error (MAPE) was under 10% by the proposed method. The performance of the proposed method outperformed conventional comparative property estimation methods. Consequently, the proposed method can realize accurate prediction of the properties with consideration of the characteristic of electron microscope images described above.

1. Introduction

Materials informatics has been attracting attention recently as an important research area for the development of high-performance materials. Modern materials science and engineering research produce a large amount of heterogeneous data, and analyzing the stored data by computational approaches discovers new knowledge for researchers [1,2,3,4]. Among many existing materials, rubber materials’ ability to resist permanent deformation and fracture is well suited for many applications [5]. We can easily list many applications using rubber materials: tires, vibration isolators, seals, hoses, belts, structural bearings, impact bumpers, medical devices, and footwear. These applications involve large static and time-varying strains over long periods. To enhance the performance of the applications, the materials informatics technology for rubber materials is necessary [6,7].

When a rubber material is used for a long time, it usually hardens and degrades and loses its performance. This deterioration is mainly due to heat caused by hysteretic loss, which affects not only the material properties but also the rubber materials’ service life. The performance of such rubber materials can be evaluated by measuring their physical properties through various experiments. Generally, numerical data for various properties of rubber materials such as friction resistance and tensile strength are needed to design and develop rubber materials; however, measurement of the properties is a time-consuming task [8]. If these properties can be determined without measurement procedures, the time required to design and produce high-performance rubber materials can be shortened. Hence, the prediction of rubber material properties has become an important research topic [9].

Methods for prediction of rubber material properties have been intensively studied [10,11,12]. These popular methods use mix proportions or mixing process information for the prediction of properties. It is well known that properties of rubber materials are strongly related to their mixing process information. For example, Liu et al. proposed methods for prediction of Mooney viscosity of rubber materials using mixing process information [11,12]. Mooney viscosity is an evaluation index for the processability, moldability, and flowability of rubber materials, and it can be measured with a specialized device called a viscometer. Mixing temperature, energy, power, pressure, and duration in the chamber of an internal mixer are used as mixing process information for the prediction of Mooney viscosity in those methods. However, some mixing process information such as the order of addition of raw materials in the chamber cannot be quantified.

The mixing process information is reflected in the internal structure of the material and can be visually confirmed. Hence, observation by electron microscope is one of the most important steps in the development phase. Specifically, images of internal structures taken by an electron microscope contain mixing process information regardless of whether the information can be quantified or not. Thus, material property prediction approaches using images of microstructure have been proposed [13,14,15,16]. Machine learning models such as a neural network [17] and relevance vector machines [18] are used in these models, and prediction of material properties is realized with consideration of the internal structures. The advantage of using image information for property estimation is that it allows us to contrast the performance of a material with the structural information of the material that we can intuitively, or visually, understand from electron microscope images.

Electron microscope images of rubber materials are generally taken under different conditions such as different magnifications and imaging conditions, as shown in Figure 1. Effective imaging conditions for prediction differ depending on the rubber material. However, only electron microscope images taken under the same conditions have been used in previous methods, and differences in conditions have not been considered. For realization of accurate prediction, it is desirable to construct a predictor with many rubber material samples. Therefore, electron microscope images taken from multiple observations should be used for the prediction of rubber material properties. We should focus on electron microscope images taken under different imaging conditions.

In this paper, we propose a method for prediction of rubber material properties using electron microscope images of internal structures taken under multiple imaging conditions. First, the proposed method selects reliable prediction results from multiple results obtained by predictors that are each constructed from electron microscope images taken under the same condition. This selection process utilizes the prediction interval of the prediction results, which can take into account the accuracy of prediction results [19]. As a result, the proposed method uses only prediction results obtained from electron microscope images taken under beneficial conditions for the subsequent integration. Next, the proposed method integrates these reliable results based on a Dempster–Shafer (DS) evidence theory [20], and this integration result is regarded as a final prediction result. The DS evidence theory has been used to integrate data from multiple information sources, and this theory has been used in many studies as a method for decision level fusion [21,22,23,24]. In the proposed method, information sources of the prediction results are different for each material sample since reliable results are different for each material sample. The DS evidence theory enables successful integration of multiple prediction results, even if the information sources from rubber material samples are different. Consequently, accurate prediction of the properties of rubber materials can be realized by the proposed method.

The contributions of this paper are summarized as follows:

• Our new robust material property prediction method considering different imaging conditions has a wide applicability not only to rubber materials but also to other materials.
• Only reliable prediction results are automatically selected and integrated based on the prediction interval and the DS evidence theory.

This paper is organized as follows. In Section 2, we describe our method for prediction of rubber material properties. In Section 3, experimental results are shown to verify the effectiveness of our method.

2. Method for Prediction of Rubber Material Properties

In this section, we describe the details of the proposed method. An overview of the proposed method is shown in Figure 2. Our method consists of three Steps (1–3): training of predictors, selection of reliable results, and integration of prediction results. In Step 1, we construct a predictor for each imaging condition of electron microscope images based on the simplest regression model, support vector regression (SVR) [25] using images and mix proportions to obtain multiple prediction results from the predictors. Next, in Step 2, the proposed method selects reliable results from prediction results obtained by these SVR predictors based on prediction intervals [26]. In step 3, basic probability assignment (BPA) functions [27] of these reliable prediction results are determined, and these BPA functions are integrated on the basis of the DS evidence theory to calculate the pignistic probability from these results. Finally, the final prediction result can be simply calculated based on the DS evidence theory.

This section is organized as follows. In Section 2.1, the property prediction method using SVR is explained as Step 1. The process for selection of reliable prediction results in Step 2 is described in Section 2.2. Finally, integration of the selected results based on the DS evidence theory in Step 3 is explained in Section 2.3.

2.1. Step 1: Property Prediction for Each Imaging Condition

The proposed method constructs the predictors for each imaging condition based on SVR using the electron microscope images and mix proportions to obtain multiple prediction results. First, the proposed method calculates numerical feature vectors from the electron microscope images and mix proportions. In the electron microscope images, the variation and distribution of luminance change according to the mix proportions and mixing process. Imaging conditions affect the luminance and colors in images. In this study, we have two different image conditions: Mode 1 and Mode 2, as shown in Figure 1. It is common in material development for images to be captured in different conditions and at different magnifications. Therefore, it is necessary to construct a robust method for estimating properties using image information. Therefore, the proposed method utilizes three kinds of numerical features—co-occurrence matrix-related features [28], Gabor wavelet-based features [29] and adaptive local binary patterns [30]—to obtain the visual feature vector by concatenating the above visual features. In the field of general image recognition, although deep-neural-network-based features have been proposed [31], handcrafted features are still useful since the number of electron microscope images is limited and their characteristics are different from those of a pretrained network. The mix proportion feature vectors are each represented as a vector whose elements are mix proportion values. In this way, features for property prediction can be extracted from the set of training images and their corresponding mix proportion information. Since obtained features have high-dimensional vectors, this can easily cause the “curse of dimensionality” when available samples are small. To solve this problem, we utilize the RReliefF algorithm [32] as a feature selection method. The RReliefF algorithm is used to calculate the contribution of each feature to the recognition task, and this also contributes to the dimensionality reduction. We obtain the sophisticated features through the RReliefF algorithm feature selection method. The proposed method obtains a feature vector $x$ by concatenating the visual and mix proportion feature vectors.

Next, the proposed method constructs the predictors based on SVR. In the proposed method, accurate prediction of rubber material properties becomes feasible since SVR enables effective prediction by nonlinear regression even if the number of training samples is small. The SVR calculates the prediction result $\widehat{y}$ as follows:

$$\begin{array}{c}\hfill \widehat{y}={\mathbf{\beta }}_{\mathrm{p}}^{\top }\mathsf{\Phi }\left(x\right)+b_{\mathrm{p}},\end{array}$$

(1)

where ${\mathbf{\beta }}_{\mathrm{p}}$ is a weighted vector in the property prediction and $\mathsf{\Phi }(·)$ denotes a nonlinear projection function. In addition, $b_{\mathrm{p}}$ is a bias parameter in the property prediction. Nonlinear regression is an extension of linear regression that can be used with large and more general class classification tasks. Unlike linear regression, it can flexibly represent complicated data via a kernel function adopted in the projection function. The SVR determines the optimal weighted vector ${\mathbf{\beta }}_{\mathrm{p}}$ by solving the following constrained optimization problem:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill {\mathbf{\beta }}_{\mathrm{p}}=& arg\underset{{\mathbf{\beta }}_{\mathrm{p}}}{min}\left\{\frac{1}{2}\left|\right|{\mathbf{\beta }}_{\mathrm{p}}{\left|\right|}^2+C\sum _{i=1}^N({\xi }_i+{\xi }_i^{*})\right\},\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& y_i-{\mathbf{\beta }}_{\mathrm{p}}^{\top }\mathsf{\Phi }\left(x_i\right)-b_{\mathrm{p}}\le ϵ+{\xi }_i,\hfill \\ & {\mathbf{\beta }}_{\mathrm{p}}^{\top }\mathsf{\Phi }\left(x_i\right)+b_{\mathrm{p}}-y_i\le ϵ+{\xi }_i^{*},\hfill \\ & {\xi }_i,{\xi }_i^{*}\ge 0,\hfill \end{array}\end{array}$$

(2)

where $C(\ge$0) is a trade-off parameter and ${\xi }_i$ and ${\xi }_i^{*}$ are slack variables. Furthermore, $y_i$ is an actual property value of the ith ($i=1,2,\dots ,N$; N being the number of training samples) training sample, $ϵ$ is a margin of tolerance, and $x_i$ is a feature vector of the ith training sample. In this way, we train the M SVR predictors for each image condition and magnification that can provide property prediction results. Namely, in the test phase, we can obtain M property prediction results in Step 1.

2.2. Step 2: Selection of Reliable Prediction Results

We can obtain M property prediction results through the processing in Step 1. However, since the SVR is constructed for each imaging condition and magnification, not all the predictions are reliable. In other words, the obtained results include a mixture of reliable and unreliable ones. Therefore, the proposed method adopts the prediction intervals for the selection of reliable prediction results in Step 2. The prediction interval in the regression analysis can take the accuracy of prediction results into account [26]. Specifically, if the prediction interval is small, this means that the variance of the prediction results is also small. Therefore, prediction results for which prediction intervals are smaller than those of other results can be regarded as reliable prediction results.

First, the proposed method calculates the prediction error $e_i$ of the ith training sample as follows:

$$\begin{array}{c}\hfill e_i={\widehat{y}}_i-y_i,\end{array}$$

(3)

where ${\widehat{y}}_i$ is a prediction value of the ith training sample. Next, the proposed method applies the fuzzy c-means algorithm [33] to the above prediction errors. The proposed method constructs an empirical distribution function, as shown in Figure 3, to calculate lower and upper values of the prediction interval in each cluster.

Specifically, let ${\mu }_{ik}$ be the ith training sample’s attribute probability to the cluster k (= $1,2,\dots ,K$; $K$ being the number of clusters). ${\tilde{e}}_i$ and ${\tilde{\mu }}_{ik}$ are obtained by sorting $e_i$ and ${\mu }_{ik}$ in ascending order, respectively. We denote ${\underline{\mathrm{PIC}}}_k$ and ${\overline{\mathrm{PIC}}}_k$ as the lower and upper values, respectively, of the prediction interval in cluster k. ${\underline{\mathrm{PIC}}}_k$ and ${\overline{\mathrm{PIC}}}_k$ under the trust rate $100\times (1-\alpha )\%$ (0 $\le \alpha \le$ 1) are calculated as follows:

$$\begin{array}{c}\hfill {\underline{\mathrm{PIC}}}_k={\tilde{e}}_q,\end{array}$$

(4)

$$\begin{array}{c}\hfill {\overline{\mathrm{PIC}}}_k={\tilde{e}}_r,\end{array}$$

(5)

where q (≤N) and r (≤N) are maximum values to satisfy the following conditions:

$$\begin{array}{c}\hfill \sum _{i=1}^q{\tilde{\mu }}_{ik}<\alpha /2\sum _{i=1}^N{\mu }_{ik},\end{array}$$

(6)

$$\begin{array}{c}\hfill \sum _{i=1}^r{\tilde{\mu }}_{ik}<(1-\alpha /2)\sum _{i=1}^N{\mu }_{ik}.\end{array}$$

(7)

The lower and upper values of the ith training samples’ prediction result ${\underline{\mathrm{PI}}}_i$ and ${\overline{\mathrm{PI}}}_i$ are calculated as follows:

$$\begin{array}{c}\hfill {\underline{\mathrm{PI}}}_i=y_i+\sum _{k=1}^K{\mu }_{ik}{\underline{\mathrm{PIC}}}_k,\end{array}$$

(8)

$$\begin{array}{c}\hfill {\overline{\mathrm{PI}}}_i=y_i+\sum _{k=1}^K{\mu }_{ik}{\overline{\mathrm{PIC}}}_k.\end{array}$$

(9)

The proposed method calculates the prediction interval of the test samples’ prediction result by utilizing the above prediction intervals of the training samples. Specifically, the prediction interval $\widehat{\mathrm{PI}}$ of the test sample is calculated by SVR utilizing the training sample’s prediction intervals ${\mathrm{PI}}_i={\overline{\mathrm{PI}}}_i-{\underline{\mathrm{PI}}}_i$ as follows:

$$\begin{array}{c}\hfill \widehat{\mathrm{PI}}={\mathbf{\beta }}_{\mathrm{PI}}^{\top }\mathsf{\Phi }\left(x\right)+b_{\mathrm{PI}},\end{array}$$

(10)

where ${\mathbf{\beta }}_{\mathrm{PI}}$ is an optimal weighted vector calculated from ${\mathrm{PI}}_i$ and $b_{\mathrm{PI}}$ is a bias parameter. Then, S reliable prediction results for which the prediction intervals are smaller than those of the other results are selected as reliable results from M prediction results in our method.

2.3. Step 3: Integration of Reliable Results Based on DS Evidence Theory

First, the proposed method determines BPA functions of reliable prediction results that have been obtained as described in the previous section. Then, the proposed method integrates the above BPA functions using the DS evidence theory to calculate the pignistic probability from the integrated result. Finally, the final prediction result can be simply estimated by an average of the selected prediction results for which the values of pignistic probability are higher than a predefined threshold.

Xu et al. proposed a method for determining BPA functions for classification problems [27]. In the proposed method, we extend this determination method to a regression version. First, the property values of the training samples are assigned to T clusters using the k-medoids algorithm [34]. We define the range in the property domain of cluster t (= $1,2,\dots ,T$) as $I_t$ = [$L_t-c\times R_t$, $U_t+c\times R_t$], where $L_t$ and $U_t$ are, respectively, the lowest value and the highest value of the training samples’ property values belonging to cluster t, $R_t=U_t-L_t$, and $c\in [0,1]$. Next, the probability density function (PDF) of each cluster is calculated. Specifically, the proposed method discretizes the property domain of each cluster into V segments. Then, let $p_{tv}$ ($v=1,2,\dots ,V$) be the probability density of the vth discretization domain in cluster t, and ${\mathit{p}}_t={[p_{t1},p_{t2},\cdots ,p_{tV}]}^{\top }\in {\mathbb{R}}^V$. The vector ${\mathit{p}}_t$ is calculated by solving the following constrained optimization problem:

$$\begin{array}{c}{\mathit{p}}_t=arg{\displaystyle \underset{{\tilde{\mathit{p}}}_t}{max}}{\displaystyle \prod _{l=1}^{N_t}}f\left(y_{tl}\right|{\tilde{\mathit{p}}}_t),\hfill \\ \mathrm{s}.\mathrm{t}.{\tilde{p}}_{tv}\ge 0,v=1,2,\cdots ,V,\hfill \\ f\left(y\right|{\tilde{\mathit{p}}}_t)\ge 0,\forall y\in I_t,\hfill \\ \int f\left(y\right|{\tilde{\mathit{p}}}_t)dy=1,\hfill \end{array}$$

(11)

where ${\tilde{\mathit{p}}}_t={[{\tilde{p}}_{t1},{\tilde{p}}_{t2},\cdots ,{\tilde{p}}_{tV}]}^{\top }\in {\mathbb{R}}^V$, $y_{tl}$ ($l=1,2,\dots ,N_t$; $N_t$ being the number of training samples belonging to cluster t) is the property value of the lth training sample in cluster t and $f\left(y_{tl}\right|{\tilde{\mathit{p}}}_t)={\tilde{p}}_{tv}$ if $y_{tl}$ belongs to the vth discretization domain in cluster t. We solve this constrained optimization problem using a genetic algorithm in the same manner as [35].

We calculate the PDF of each cluster by utilizing the interpolation based on Gaussian process regression (GPR) [36] using the above probability densities ${\mathit{p}}_t$. In the GPR, it is assumed that the available function values represent a particular realization of a Gaussian process. Let $g_{tv}$ be a representative value of segmentation v into cluster t. The PDF of the tth cluster ${\mathrm{Pr}}_t\left(y\right)$ is constructed as follows:

$$\begin{array}{c}\hfill {\mathrm{Pr}}_t\left(y\right)=\left\{\begin{array}{cc}\frac{1}{\sqrt{2\pi {\sigma }_t^2}}exp\left(-\frac{{(y-e_t)}^2}{2{\sigma }_t^2}\right)\hfill & \mathrm{if}y\in I_t\hfill \\ 0\hfill & \mathrm{otherwise}\hfill \end{array}\right.,\end{array}$$

(12)

where $e_t$ and ${\sigma }_t^2$ are prediction means and variance of cluster t, respectively.

The proposed method calculates the test sample’s probability density using the prediction results. Let ${\rho }_{st}$ be the sth ($s=1,2,\dots ,S$) prediction result’s probability density for cluster t obtained by the PDF. We calculate the probability density ${\rho }_{st}$ as follows:

$$\begin{array}{c}\hfill {\rho }_{st}={\mathrm{Pr}}_r\left({\widehat{y}}_s\right),\end{array}$$

(13)

where ${\widehat{y}}_s$ is the sth prediction result of the test sample.

Finally, we determinate the BPA function of each prediction result obtained by SVR as described in the previous section. Let ${\theta }_t$ denote the tth identifiable object and let $\Theta \in \{{\theta }_1,{\theta }_2,\cdots ,{\theta }_T\}$ be a frame of discernment. Let ${\tilde{\rho }}_{s1},{\tilde{\rho }}_{s2},\cdots ,{\tilde{\rho }}_{sT}$ be sorted results in decreasing order of ${\rho }_{st}$, and $C_{st}\in \Theta$ is the identifiable object that ${\tilde{\rho }}_{st}$ belongs to. If $\exists {\tilde{\rho }}_{st}\ne 0$ ($t=1,2,\dots ,T$), the BPA function of the sth prediction result $m_s(·)$ is determined as follows:

$$\begin{array}{c}\hfill \begin{array}{cc}& m_s\left(\left\{C_{s1}\right\}\right)={\tilde{\rho }}_{s1},\hfill \\ & m_s\left(\{C_{s1},C_{s2}\}\right)={\tilde{\rho }}_{s2},\hfill \\ & ⋮\hfill \\ & m_s\left(\{C_{s1},C_{s2},\cdots ,C_{sT}\}\right)={\tilde{\rho }}_{sT}.\hfill \end{array}\end{array}$$

(14)

On the other hand, if $\forall {\tilde{\rho }}_{st}=0$ ($t=1,2,\dots ,T$), $m_s(·)$ is determined as follows:

$$\begin{array}{c}\hfill m_s\left(\left\{{\theta }_t\right\}\right)=exp(-b_t),\end{array}$$

(15)

$$\begin{array}{c}\hfill b_t=\left\{\begin{array}{cc}{a}_t/\sum _{t^{\prime }=1}^T{a}_{t^{\prime }}\hfill & \mathrm{if}max\left(a_t\right)>1\hfill \\ a_t\hfill & \mathrm{otherwise}\hfill \end{array}\right.,\end{array}$$

(16)

$$\begin{array}{c}\hfill a_t=\frac{1}{S}\sum _{s=1}^S|{\tilde{y}}_s-u_t|,\end{array}$$

(17)

where $u_t$ is the representative value of cluster t. Then, the proposed method normalizes the obtained BPA function $m_s(·)$ to satisfy the following conditions:

$$\begin{array}{c}\hfill \sum _{Q\in \{\left\{C_{s1}\right\},\{C_{s1},C_{s2}\},\cdots ,\{C_{s1},C_{s2},\cdots ,C_{sT}\}\}}{m}_s\left(Q\right)=1,\end{array}$$

(18)

$$\begin{array}{c}\hfill m_s\left(\mathsf{\Phi }\right)=0,\end{array}$$

(19)

where $\mathsf{\Phi }$ denotes a null set. Therefore, the proposed method normalizes the obtained BPA functions.

Next, the proposed method calculates the final prediction result by integrating the above BPA functions based on the DS evidence theory. The power set of $\Theta$ is the set containing all $2^T$ possible subsets of $\Theta$ represented by $P(\Theta )$ defined as

$$\begin{array}{c}\hfill P(\Theta )=\{\mathsf{\Phi },\left\{{\theta }_1\right\},\cdots ,\left\{{\theta }_T\right\},\{{\theta }_1,{\theta }_2\},\cdots ,\Theta \}.\end{array}$$

(20)

According to Dempster’s orthogonal rule [20], we have

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill m\left(Y\right)& =m_1\left(Z_1\right)\oplus m_2\left(Z_2\right)\oplus \cdots \oplus m_S\left(Z_S\right)\hfill \\ & =\frac{{\displaystyle \sum _{Z_1\cap Z_2\cap \cdots \cap Z_S=Y}{m}_1\left(Z_1\right)m_2\left(Z_2\right)\cdots m_S\left(Z_S\right)}}{{\displaystyle 1-\sum _{Z_1\cap Z_2\cap \cdots \cap Z_S=\mathsf{\Phi }}{m}_1\left(Z_1\right)m_2\left(Z_2\right)\cdots m_S\left(Z_S\right)}},\hfill \end{array}\end{array}$$

(21)

where $Z_s$ is the subset of $P(\Theta )$ for the sth information source and ⊕ is an operator that represents the integration of BPA functions.

The proposed method calculates the pignistic probability ${\mathrm{Pr}}_{\mathrm{pig}}\left({\theta }_t\right)$ ($t=1,2,\dots ,T$) based on the above integrated BPA function $m(·)$ as follows:

$$\begin{array}{c}\hfill {\mathrm{Pr}}_{\mathrm{pig}}\left({\theta }_i\right)=\sum _{D\subseteq \Theta }\frac{|\left\{{\theta }_i\right\}\cap D|\times m\left(D\right)}{\left|D\right|},\end{array}$$

(22)

where $\left|D\right|$ is the number of singleton elements in set D.

Then, the final prediction result of the proposed method is calculated as the average of reliable prediction results belonging to the cluster that is assigned maximum pignistic probability. Specifically, the proposed method calculates the average value of prediction results belonging to the cluster for which the pignistic probability is higher than those of the other clusters. Let $y_{s^{\prime }}$ be the $s^{\prime }$th (= $1,2,\dots ,S^{\prime };S^{\prime }$ being the number of prediction results belonging to the cluster having the maximum values of pignistic probability) prediction result obtained by SVR, and ${\gamma }_{s^{\prime }}\in \Theta$ is the cluster that $y_{s^{\prime }}$ belongs to. The final prediction result ${\widehat{y}}_{\mathrm{final}}$ is calculated as follows:

$$\begin{array}{c}\hfill {\widehat{y}}_{\mathrm{final}}=\frac{1}{S^{\prime }}\sum _{s^{\prime }=1}^{S^{\prime }}{y}_{s^{\prime }}.\end{array}$$

(23)

Since the pignistic probability denotes the confidence of the prediction result, the proposed method enables accurate property prediction.

3. Experimental Results

In this section, we show the experimental results to verify the effectiveness of the proposed method. In this experiment, we conducted prediction of rubber material properties using electron microscope images and mix proportions. We used electron microscope images and mix proportions of 75 rubber material samples as the data set. The data set in this experiment consisted of electron microscope images taken under eight different conditions, and the number of samples and electron microscope images in each condition are shown in

Table 1. Numbers of samples and electron microscope images in each condition.

Condition	Magnification & Mode	Number of Samples	Number of Images
Condition 1	× 5000, Mode 1	71	71
Condition 2	× 10,000, Mode 1	72	152
Condition 3	× 20,000, Mode 1	72	156
Condition 4	× 40,000, Mode 1	56	118
Condition 5	× 2500, Mode 2	19	19
Condition 6	× 5000, Mode 2	19	38
Condition 7	× 10,000, Mode 2	19	52
Condition 8	× 20,000, Mode 2	19	57

. As shown in Table 1, each sample is taken under several conditions, but not all of the conditions are satisfied. Hence, the number of samples and the number of images of each condition are different in the experimental data set. Each electron microscope image was eight-bit grayscale, and its size was 1536 × 1024 pixels.

The proposed method is a novel approach for prediction of properties using electron microscope images of internal structures taken under multiple conditions, and this is the first time that a prediction method using multiple kinds of electron microscope images has been proposed. We compared our method with many conventional methods, including a recently proposed convolutional-neural-network (CNN) [37]-based method. The details of comparative methods used in this experiment are shown in

Table 2. Comparative methods used in the experiment.

Method	Overview
CM1	The final prediction result is obtained by integration of reliable results using the DS evidence theory. Reliable results are selected from prediction results obtained by an SVR predictor that do not consider different imaging conditions.
CM2	The final prediction result is calculated by integration of reliable results using the DS evidence theory. Reliable results are selected from prediction results obtained by the predictor of each condition using only visual features.
CM3	The final prediction result is obtained by the SVR predictor using only mix proportion features. Each rubber material sample is assumed to have a single mix proportion. Therefore, CM3 does not apply selection and integration of prediction results.
CM4	The final prediction result is calculated by the average of the multiple prediction results.
CM5	The final prediction result is obtained by weighted average of the prediction results. CM5 utilizes prediction interval as the weight.
CM6	The final prediction result is obtained by selection of a reliable result from multiple prediction results using prediction intervals.
CM7	The final prediction result is calculated by the average of reliable results selected from the prediction results obtained by the predictor of each condition using visual and mix proportions features.
CM8	The final prediction result is calculated by integration of multiple prediction results based on the DS evidence theory.
CM9	The final prediction result is calculated by selection and integration of prediction results obtained by a convolutional neural network (CNN) [37]. Specifically, CM9 utilizes Xception [39] that is fine-tuned for property prediction using electron microscope images.

. According to the proposed method, the kernel function of SVR used in the proposed method and these comparative methods was the Gaussian kernel, and each parameter of the SVR was determined by a practical selection approach in the same manner as the paper [38]. Furthermore, the parameters used in calculation of the prediction interval and the DS evidence theory were determined on the basis of prediction error of the final results. The performance of each method was evaluated via leave-one-out cross-validation. In the leave-one-out cross-validation, a sample was extracted from the 75 samples, the extracted sample was allocated to the test sample, and the other 74 samples were allocated to the training samples. We trained our models using those 74 training samples and estimated the label of the test sample. We repeated this step 75 times to evaluate all samples and calculated the prediction performance.

In order to evaluate the accuracy of the property prediction, we utilized mean absolute error (MAE) and mean absolute percent error (MAPE) shown in the following equations:

$$\begin{array}{c}\hfill \mathrm{MAE}=\frac{1}{H}\sum _{h=1}^H|y_h-{\widehat{y}}_h|,\end{array}$$

(24)

$$\begin{array}{c}\hfill \mathrm{MAPE}=\frac{1}{H}\sum _{h=1}^H\frac{|y_h-{\widehat{y}}_h|}{y_h}\times 100(\%),\end{array}$$

(25)

where H (= 75) is the number of test samples.

The experimental results are shown in

Table 3. Mean absolute error (MAE) and mean absolute percent error (MAPE) of prediction results obtained by the proposed method and the comparative methods.

	PM	CM1	CM2	CM3	CM4	CM5	CM6	CM7	CM8	CM9
MAE	2.68	4.14	3.77	3.17	2.93	3.04	3.17	2.93	2.69	3.80
MAPE	9.64%	15.4%	12.3%	11.7%	11.2%	11.5%	11.7%	11.2%	10.4%	13.7%

. It should be noted that the MAE results provided in Table 3 are the average values of estimated properties for each sample. Namely, the results for 75 test samples are averaged in a single value. From the obtained results, we can confirm that the proposed method realizes the most accurate prediction. Therefore, the effectiveness of the proposed method was verified. A comparison of the results of the proposed method and the results of CM1 shows the effectiveness of using electron microscope images taken under several conditions. We can also confirm that the prediction method using electron microscope images and mix proportions enables accurate prediction of the properties of rubber materials by comparing the results of the proposed method and those of CM2 and CM3. A comparison of the results of the proposed method and those of CM4 and CM5 shows that integration based on the DS evidence theory is effective for prediction of properties of the rubber materials. This is because the pignistic probability denotes the accuracy of each prediction result obtained by the predictor, and successful prediction was realized by integration using these probabilities. Furthermore, we can confirm that the prediction method utilizing selection and integration of reliable results enables accurate prediction of the properties by comparing the results of the proposed method and those of CM6, CM7, and CM8. Finally, we can also confirm that the proposed method enables more accurate prediction than the approaches based on the CNN-based method by comparing the results of the proposed method and those of CM9. This is because the SVR enables accurate prediction even if the number of training sample is small. Considering that the measurement error of the material is always a few percent, it is not too much to say that the proposed method can estimate the properties with high accuracy (less than 10% error).

There are several limitations in this paper. First, although our method can be applied to other materials, we have not confirmed the effectiveness of our method in other materials due to the data restriction. Samples used in this experiment are still small to verify the potential of the proposed method for real-world applications. These concerns should be considered for the next step of our study.

Our rubber material property prediction method is versatile and can be applied not only to rubber materials but also to other materials. In addition, our method is different from the conventional methods in that it can estimate the properties of materials even when the image capturing conditions are not available. In future works, we will improve the accuracy of property estimation by enhancing the image feature extraction methods.

4. Conclusions

In this paper, we have proposed a method for prediction of rubber material properties using electron microscope images of internal structures taken under multiple imaging conditions. The proposed method selects reliable prediction results from multiple results obtained by SVR predictors constructed from electron microscope images taken under each condition. The selection utilizes the prediction interval of the prediction results. Then, the proposed method integrates the reliable results based on the DS evidence theory, and this integration result is regarded as the final prediction result. The results of an experiment showed the effectiveness of the proposed method by comparing the results obtained by comparative methods.