A Novel Model Validation Method Based on Area Metric Disagreement between Accelerated Storage Distributions and Natural Storage Data

Abstract

It has been a challenge to quantify the credibility of the accelerated storage model until now. This paper introduces a quantitative measurement named the CMADT (Creditability Metric of Accelerated Degradation Test), which quantifies the credibility of the accelerated aging model based on available data. The relevant criterion data are obtained from the natural storage test. CMADT is a credibility metric obtained by measuring the difference in the metric area between the probability distribution of the accelerated storage model and its criterion data. In addition, the accelerated aging model might include multiple parameters resulting in several single-parameter CMADTs. This paper proposes a method that integrates several single-parameter CMADT metrics into a single metric to assess the overall credibility of the accelerated storage model. Moreover, CMADT is universal for different scales of sample data. The cases addressed in this paper show that CMADT helps designers and decision-makers judge the credibility of the result obtained by the accelerated storage model intuitively and makes it easier to compare various products horizontally.

1. Introduction

Weapons such as missiles and torpedoes have the characteristics of long-term storage and one-time use, and the majority of their life cycle involves storage. During the long-term storage period, due to factors such as corrosion, aging, and material surface and interface reactions, the material properties and physical parameters of products will gradually change, which ultimately leads to product failure due to failure to meet functional performance requirements.

During the design phase, we need to access the weapon storage lifetime, including components, raw materials, and parts, to ensure that the elements used in the equipment can meet the storage lifetime criterion. At the stereotype stage, the storage lifetime of the overall system needs to be verified, and after delivery, there is still a need for periodic sampling and testing of products in service and prediction of remaining life.

The accelerated storage test (AST) is one of the critical techniques used in the aforementioned storage lifetime assessment. AST is a testing technique that enhances a particular stress quantity to obtain the crucial parameters of the product’s performance degradation during storage while the failure mechanism remains constant [1,2,3]. It is practically useful in engineering due to its high speed and efficiency. Since AST obtains degenerating performance data by rapidly increasing the corresponding stress, it is possible that the failure mechanism will change due to excessive stress [4]. That is, the failure mechanism maintaining invariability is a precondition for AST [5,6]. Once the failure mechanism changes, the assessment results will definitely deviate from the real state of the product. Many factors may affect the credibility of AST results, such as test equipment errors, artificial errors, test ambient fluctuation, and sample dispersion. Even though all mentioned factors could be controlled precisely, there still exists disagreement between the parameters that are inferred from the AST model under normal stress, such as the storage lifetime and model credibility, and the system’s real state. In addition, there is a continuing debate as to whether the AST model is correct and reflects the acceleration process of various products.

In recent decades, extensive research has been carried out both domestically and abroad to investigate the alteration of the failure mechanism that occurs in the accelerated testing process. In general, these studies can be divided into two categories: the first involves using the perspective of failure physics to judge the consistency of the failure mechanism under different stresses. This aim is mainly achieved through a comprehensive analysis of the chemical and microscopic structures, as well as destructive physics, to judge whether samples under different stresses are consistent in terms of microscopic appearance, element distribution, material properties, and other factors [7,8,9,10]. It has a clear physical concept and can only qualitatively judge whether the failure mechanism has changed. However, it is difficult to provide quantitatively consistent test results. Spearman’s rank correlation coefficient [11] and grey theory [12,13] are used to identify the consistency of the failure mechanism based on the shape of the degradation path. From the perspective of the pseudo-life distribution, the literature [14,15] has evaluated the reliability and predicted the lifetime, and it obeys a log-normal or Weibull distribution using the F-statistic and Bartlett statistic. The variation in the shape of the parameters in the Arrhenius model also provides a way to explore the consistency of the failure mechanism [16]. Cai et al. [17] proposed a change-point model for the coefficients of variation to fit the abrupt change behavior of the failure mechanisms with a nonparametric empirical likelihood approach, which was used in the lifetime data of the metal oxide semiconductor transistors in the power distribution system of the Chinese Tiangong space station. Zhai et al. [18] proposed a method for consistency testing of ADT (accelerated degradation test) failure mechanisms based on the activation energy invariance method and the likelihood ratio test, accounting for the degradation dispersion caused by the manufacturing technology.

The aforementioned technique ensures the credibility of the AST result to some extent, even though there are still a few defects. For example, the method based on failure physics only qualitatively determines the change in the failure mechanism. Based on the experimental data, the boundary consistency method and the failure mechanism consistency discrimination only determine the consistency of the failure mechanism instead of assessing the result incredibility caused by factors such as measurement error and accelerated model applicability. For the AST result, the most reliable criterion is to test its consistency with the corresponding natural storage test data. However, how can the degree of consistency be judged and to what extent? There is still a lack of good evaluation metrics.

This paper proposes a new method for evaluating the credibility of accelerated storage test data based on the idea of an area metric with small samples of natural storage test data as the benchmark. The remainder of this paper is organized as follows: Section 2 introduces the theory of the probability distribution area metric. Section 3 defines a credibility metric called CMADT, which is extrapolated from the aforementioned theory and is applied to assess the credibility of the AST results. An engineering use case is conducted to demonstrate the validation of the assessment metric in Section 4, and Section 5 offers conclusions with a summary of the main findings.

2. CMADT Credibility Metric for the Accelerated Aging Test

The main stresses loaded in the accelerated storage test are high-temperature single stress and temperature-humidity double-stress. There are four types of accelerated storage tests: constant stress, step stress, step-down stress, and sequential stress, according to the different methods of stress loading. Regardless of which stress and stress loading methods are used, we can derive the degradation model of key performance parameters under normal stress (e.g., temperature 25 °C) through performance degradation modeling and accelerated model solving [19,20,21,22].

In engineering practice, there are often some natural storage test data in addition to accelerated storage test data. For example, the natural storage test is carried out during the initial sample stage of product development, and the natural storage test data can be obtained during the final evaluation. Natural storage data can also be obtained during the service stage after the equipment is delivered every year. Generally, natural storage data have a higher confidence level, but there are often two problems. First, the storage period is short; for example, it is necessary to assess the reliability of product storage for 20 years, while natural storage data often only have a few years of storage time data. Second, the sample size is often small, and the natural storage data are not sufficient to provide high-confidence assessment results. For the above reasons, storage life assessment is often provided by accelerated storage tests in engineering practice, while natural storage test data are mainly used to verify the correctness of accelerated storage tests.

2.1. Area Metric for a Single Parameter

The area metric was first proposed in 2008 by American scholars Ferson and Oberkampf [23,24], and developed by LI [25], JI [26] and ZHANG [27], et al.; it is a confirmation metric based on the probability distribution distance, which is mainly used in the field of modeling simulations. Figure 1 shows that by calculating the area between the simulation model response and the empirical cumulative distribution function of the experimental observations (shaded part in Figure 1), the accuracy of the simulation model is quantified and evaluated.

Suppose that the accumulated distribution function for the equivalent data of a product’s key performance parameters, which are obtained from the accelerated storage test at time $t_i$, is $F_{t_i}^a\left(x\right)$, and the accumulated distribution function of the r samples tested in the natural storage test at time $t_i$ is $F_{t_i}^b\left(x\right)$, then, the area metric confirmed by the model can be borrowed. $F_{t_i}^a\left(x\right)$ can be seen as the accumulated distribution function of the simulation model response, and $F_{t_i}^b\left(x\right)$ is seen as the accumulated distribution function of the test observations. The area metric of the accelerated storage test $t_i$ momentary confidence evaluation is defined as

$$A(F_{t_i}^a(x),F_{t_i}^b(x))={\displaystyle {\int }_{-\infty }^{+\infty }\left|F_{t_i}^a(x)-F_{t_i}^b(x)\right|dx}$$

(1)

From Equation (1), it can be seen that the area metric is smaller when the probability distribution of the equivalent data is closer to that of the benchmark data, and vice versa. Therefore, the area metric can be used to assess the confidence level of the accelerated storage test.

2.2. Dimensionless Measurement of Area Metrics

The metrics defined in Equation (1) are content-based, and it is not clear how large the gap is between multiple parameters of different scales for the same product or between different products; therefore, the evaluation results cannot be compared. It is also not clear how the large gap indicates “excellent” quality and how a small gap indicates “poor” quality.

To obtain a unified evaluation criterion, the metrics of Equation (1) in this paper are dimensionless, and their mathematical definition at time $t_i$ is

$${\rho }_i=\frac{A(F_{t_i}^a(x),F_{t_i}^b(x))}{A(F_{t_i}^a(x),F_{t_i}^0(x))}\times 100$$

(2)

${\rho }_i$ is called the Credibility Metric of Accelerated Degradation Test (CMADT), where $A(F_{t_i}^a\left(x\right),F_{t_i}^0\left(x\right))$ is the value of the same scale as the area metric $A\left(F_{t_i}^a\right(x),F_{t_i}^b(x\left)\right)$ and is used to characterize the dispersion of the accelerated storage test data, as shown in Figure 2. Its expression is

$$A(F_{t_i}^a(x),F_{t_i}^0(x))={\displaystyle {\int }_{{\mu }_i-3{\sigma }_i}^{{\mu }_i+3{\sigma }_i}{F}_{t_i}^a(x)dx}$$

(3)

where ${\mu }_i$ and ${\sigma }_i$ are the mean and standard deviation of the accelerated storage test degradation data at time $t_i$, respectively.

From Equation (2), it is clear that ${\mathsf{\rho }}_i$ is dimensionless, which is convenient for subsequent accelerated storage tests of multiple key performance parameters in terms of providing a unified plausibility measure.

3. Credibility Metric of the Accelerated Aging Test

3.1. Probability Distributions of Key Performance Parameters Based on Natural Storage Tests with Small Samples

The sample size of the natural storage test is relatively small compared with that of the accelerated aging test. Otherwise, the degradation model can be obtained directly from the empirical data of the natural storage test.

Assume that the sample size of the natural storage test is r and the corresponding test time is $\mathit{t}=\{t_1,t_2,\cdots ,t_L\}$, where L is the number of tests. A set of key parameter degradation data is $\mathit{B}=\{{\mathit{b}}_1,{\mathit{b}}_2,\cdots ,{\mathit{b}}_L\}$, where $b_i=\{b_{i1,}{b}_{i2},\dots ,b_{ir}\}$ is the test data at moment $t_i(i=1,2,\cdots ,L)$. Considering the small sample, the model using the probability distribution easily introduces strong subjective assumptions that affect the accuracy of the assessment. This paper constructs the upper and lower bounds of the p-box of natural storage data based on the belief function and plausibility function of D–S evidence theory, and the processing of this method does not make subjective assumptions about the distribution type, which can effectively retain the statistical characteristics of the original information.

Definition 1.

Let B be the recognition frame and ${\mathit{B}}_i^I$ be the focal element.$m\left({\mathit{B}}_i^I\right)$ is the basic probability assignment (BPA).

First, calculate the data series’ mean value of ${\mathit{b}}_i$

$${\overline{b}}_i={\displaystyle \sum _k{b}_{ik}}/r$$

(4)

Arrange ${\mathit{b}}_i=\{b_{i1},b_{i2},\cdots ,b_{ir}\}$ and ${\overline{b}}_i$ in ascending order

$${\tilde{\mathit{b}}}_i=\{{\tilde{b}}_{i(1)},{\tilde{b}}_i{}_{(2)},\cdots ,{\tilde{b}}_i{}_{(r+1)}\}$$

(5)

The sequence ${\tilde{\mathit{b}}}_i$ can form the set ${\mathit{B}}_i^I$ consisting of r interval numbers.

$$\begin{array}{ll}{\mathit{B}}_i^I& =\left\{B_1^I,B_2^I,\cdots ,B_r^I\right\}\\ & =\left\{[{\tilde{b}}_{i(1)},{\tilde{b}}_{i(2)}],[{\tilde{b}}_{i(2)},{\tilde{b}}_{i(3)}],\cdots ,[{\tilde{b}}_{i(r)},{\tilde{b}}_{i(r+1)}]\right\}\end{array}$$

(6)

The distance from the mean value ${\overline{b}}_i$ to each interval number in ${\mathit{B}}_i^I$ is calculated, and the basic probability distribution of $B_j^I(j=1,2\cdots ,r)$ is obtained. The trust function and likelihood function are constructed to obtain the CDF (Cumulative Distribution Function) of the p-box.

Definition 2.

Let $D=[d_1,d_2]\left(d_1{\le d}_2\right)$ and $E=[e_1,e_2]\left(e_1{\le e}_2\right)$ be two interval numbers and set 

$$L(D,E)=\frac{1}{\sqrt[p]{2}}{\left[{(d_1-e_1)}^p+{(d_2-e_2)}^p\right]}^{1/p}$$

(7)

as the distance between the interval numbers D and E.

When p = 2, note that

$$L_2(D,E)=\frac{1}{\sqrt{2}}{\left[{(d_1-e_1)}^2+{(d_2-e_2)}^2\right]}^{1/2}$$

(8)

$L_2(D,E)$is called the Euclidean distance.

${\overline{b}}_i$ can be viewed as the interval number [${\overline{\mathit{b}}}_{\mathit{i}}$,${\overline{\mathit{b}}}_{\mathit{i}}$], so the Euclidean distance from the interval number $B_j^I(j=1,2\cdots ,r)$ to ${\overline{b}}_i$ is

$$L_2(B_j^I,{\overline{b}}_i)=\frac{1}{\sqrt{2}}{\left[{({\tilde{b}}_{ij}-{\overline{b}}_i)}^2+{({\tilde{b}}_{i(j+1)}-{\overline{b}}_i)}^2\right]}^{1/2}$$

(9)

Normalize $L_2\left(B_j^I,{\overline{b}}_i\right)$ to obtain the distance vector from the interval number $B_j^I$ to ${\overline{\mathit{b}}}_{\mathit{i}}$

$${\theta }_j=L_2(B_j^I,{\overline{b}}_i)/{\displaystyle \sum _{j=1}^r{L}_2(B_j^I,{\overline{b}}_i)}$$

(10)

In turn, the similarity between the interval number $B_j^I$ and ${\overline{b}}_i$ is defined as

$${\xi }_j=1-{\theta }_j$$

(11)

${\xi }_j$ expresses the extent to which the distribution interval of the individual test data is similar to the expected value ${\overline{\mathit{b}}}_{\mathit{i}}$ as a basis for assigning a confidence probability to $B_j^I$, i.e., BPA.

$$m(B_j^I)={\xi }_j/{\displaystyle \sum _{j=1}^r{\xi }_j}$$

(12)

Although the underlying probability distribution of $B_j^I$ has been obtained, there is no evidence to show what distribution the parameter $P$ obeys in the interval $B_j^I$.

$$\{\begin{array}{l}{\overline{F}}_{t_i}^b(x)={\displaystyle \sum _{l=1}^{j}m(B_l^I)(x\in B_j^I)}\\ {\underset{\_}{F}}_{t_i}^b(x)={\displaystyle \sum _{l=1}^{j-1}m(B_l^I)(x\in B_j^I)}\end{array}$$

(13)

Therefore, the p-box of the parameter $P$ is constructed with the upper bound as the likelihood function and the lower bound as the confidence function, such that any possible probability distribution of x falls in this envelope.

3.2. CMADT under the p-Box

The performance degradation model for the parameter P under normal stress, which is derived from the accelerated storage test data, yields a set of time-section data ${\mathit{a}}_{\mathit{i}}=\{a_{i1},a_{i2},\cdots ,a_{iN}\}$ at moment $t_i$, where N is related to the sample size involved in the accelerated storage test, and the performance degradation modeling method is used. If the modeling method of the performance degradation track is used, $N=k_1\times k_2\times \cdots \times k_q$. Generally, the sample size of the equivalent data obtained from the evaluation of accelerated storage test data is relatively large. For example, in a high-temperature accelerated storage test with three stress levels and five samples per stress level, if the performance degradation trajectory method is used, we can obtain $5^3=125$ degradation curves under normal stress. Then, the sample size of the cross-sectional data at moment $t_j$ is 125. Therefore, the probability distribution $F_{t_i}^a\left(x\right)$ of the time-section data $\mathrm{A}$ can be better described by hypothesis testing using commonly used distribution types such as normal and log-normal distributions.

As p-box is used to express the small sample natural storage data in this paper, by using Equation (2) to calculate CMADT, the obtained result will be an interval number ${\rho }_i=[{\underset{¯}{\rho }}_i,{\overline{\rho }}_i]$ with the following upper and lower bounds:

$$\{\begin{array}{l}{\overline{\rho }}_i=\max \left\{\frac{A(F_{t_i}^a(x),{\overline{F}}_{t_i}^b(x))}{A(F_{t_i}^a(x),F_{t_i}^0(x))},\frac{A(F_{t_i}^a(x),{\underset{\_}{F}}_{t_i}^b(x))}{A(F_{t_i}^a(x),F_{t_i}^0(x))}\right\}\times 100\\ {\underset{\_}{\rho }}_i=\min \left\{\frac{A(F_{t_i}^a(x),{\overline{F}}_{t_i}^b(x))}{A(F_{t_i}^a(x),F_{t_i}^0(x))},\frac{A(F_{t_i}^a(x),{\underset{\_}{F}}_{t_i}^b(x))}{A(F_{t_i}^a(x),F_{t_i}^0(x))}\right\}\times 100\end{array}$$

(14)

3.3. Single Key Parameter of the Overall Confidence Evaluation

The previous two sections discussed the credibility evaluation metrics of the equivalent data derived from accelerated storage tests at a single point in time. In general, benchmark data often exist for multiple time points of the test data, so an integrated overall credibility metric that combines the credibility of multiple time points needs to be investigated.

To ensure the normalized characteristics of the overall credibility index, this paper performs probability statistics on the CMADT indices of n test points and calculates the credibility confidence lower limits at confidence level $\gamma$ as the overall credibility index of individual key performance parameters.

According to Equation (17), the CMADT value $\mathit{\rho }=[{\mathit{\rho }}_L,{\mathit{\rho }}_U]$ can be obtained for all n test points, i.e.,

$$\left\{\begin{array}{c}{\mathit{\rho }}_L=\left({\underset{¯}{\rho }}_1,{\underset{¯}{\rho }}_2,\cdots ,{\underset{¯}{\rho }}_n\right)\\ {\mathit{\rho }}_U=\left({\overline{\rho }}_1,{\overline{\rho }}_2,\cdots ,{\overline{\rho }}_n\right)\end{array}\right.$$

(15)

Let the confidence level of the credibility assessment be $\gamma =1-\alpha$ and α be the significance level. For ${\mathit{\rho }}_L$, if the series can pass a normality test (e.g., a K–S test), the confidence intervals for the n CMADT values at confidence level $\gamma$ can be counted according to the normal distribution as follows:

$$\left\{\begin{array}{c}{\rho }_L^{-}={\mu }_{\rho ,L}-\frac{{\sigma }_{\rho ,L}{z}_{1-\frac{\alpha }{2}}}{\sqrt{n}}\\ {\rho }_L^{+}={\mu }_{\rho ,L}+\frac{{\sigma }_{\rho ,L}{z}_{1-\frac{\alpha }{2}}}{\sqrt{n}}\end{array}\right.$$

(16)

where

$$\left\{\begin{array}{c}{\mu }_{\rho ,L}=\frac{1}{n}{\displaystyle \sum _{i=1}^n}{\underset{¯}{\rho }}_i\\ {\sigma }_{\rho ,L}=\frac{1}{n-1}{\displaystyle \sum _{i=1}^n}({\underset{¯}{\rho }}_i-{\mu }_{\rho }{)}^2\end{array}\right.$$

(17)

$Z_{\alpha }$ can be obtained by looking up the normal distribution table, and for the more widely used significance level $\alpha$ used in practice, the values of $Z_{\alpha }$ are provided in

Table 1. Values of Z under common α.

α	0.01	0.05	0.1
$z_{1-\alpha /2}$	2.5758	1.9600	1.6440

.

The calculation process is similar for ${\mathit{\rho }}_U$. The calculation process is not repeated, and finally, ${\rho }_U^{-}$ and ${\rho }_U^{+}$ are obtained.

To be conservative, the credibility evaluation of this accelerated storage test is used as the upper confidence limit, i.e., the overall credibility of the single key performance parameter (CMADT of single parameter, CSP) evaluation at confidence level $\gamma$ is $\eta =[{\eta }_L,{\eta }_U]$, where

$$\left\{\begin{array}{c}{\eta }_L={\mu }_{\rho ,L}+{\sigma }_{\rho ,L}{z}_{1-\frac{\alpha }{2}}\\ {\eta }_U={\mu }_{\rho ,U}+{\sigma }_{\rho ,U}{z}_{1-\frac{\alpha }{2}}\end{array}\right.$$

(18)

The index obtained by Equation (21) is dimensionless, which is convenient for the horizontal comparison of multiple key performance parameters related to the same product or the credibility of accelerated storage test data for different products.

If the data series P cannot pass the normality test, the kernel density estimation (KDE) method [28] can be used to calculate the lower confidence limit of credibility at confidence level γ. The kernel density estimation is a nonparametric estimation method that can be used for distribution parameter estimation when the distribution is nonnormal and nonstandard.

For a set of data $\left\{s\right(1),s(2),\dots ,s(N\left)\right\}$, the probability density function is estimated as

$$\widehat{f}\left(t\right)=\frac{1}{Nh}{\displaystyle \sum _{k=1}^N}K\left(\frac{t-s\left(k\right)}{h}\right)$$

(19)

where $h$ is the window width and $k\left(x\right)$ is the kernel function that satisfies

$$K\left(x\right)\ge 0,{\int }_{-\infty }^{+\infty }K\left(x\right)dx=1$$

(20)

In this paper, we choose the most classical Gaussian kernel function.

$$K\left(u\right)=\frac{1}{\sqrt{2\pi }}\exp \left(-\frac{1}{2}{u}^2\right)$$

(21)

For the data series $\mathit{\rho }=[{\mathit{\rho }}_L,{\mathit{\rho }}_U]$, after obtaining the KDE estimate, the confidence interval under the confidence level $\gamma$ can be obtained as

$$\{\begin{array}{l}{\rho }_L^{-}={\widehat{f}}_L^{{}^{-1}}(\alpha /2)\\ {\rho }_L^{+}={\widehat{f}}_L^{{}^{-1}}(1-\alpha /2)\\ {\rho }_U^{-}={\widehat{f}}_U^{{}^{-1}}(\alpha /2)\\ {\rho }_U^{+}={\widehat{f}}_U^{{}^{-1}}(1-\alpha /2)\end{array}$$

(22)

where ${\widehat{f}}_L^{{}^{-1}}(\alpha /2)$ is the $\alpha /2$-quartile point of the probability distribution of ${\mathit{\rho }}_L$ and ${\widehat{f}}_L^{{}^{-1}}(1-\alpha /2)$ is the $1-\alpha /2$-quartile point of the probability distribution of ${\mathit{\rho }}_L$. The significances of ${\widehat{f}}_U^{{}^{-1}}(\alpha /2)$ and ${\widehat{f}}_U^{{}^{-1}}(1-\alpha /2)$ are similar.

In this case, the single key performance parameter CSP evaluation at confidence level $\gamma$ results in $\eta =[{\eta }_L,{\eta }_U]$, where

$$\left\{\begin{array}{c}{\eta }_L={\widehat{f}}_L^{-1}\left(\frac{\alpha }{2}\right)\\ {\eta }_U={\widehat{f}}_U^{-1}\left(\frac{\alpha }{2}\right)\end{array}\right.$$

(23)

3.4. Overall Credibility Evaluation of Multiple Key Parameters

The above discussion is for a case in which the key parameters of the product are single parameters. However, in practice, multiparameter degradation is also common [29,30,31,32]. Let the product have m key performance parameters, which are denoted as P₁, P₂, …, P_m, and the confidence of the accelerated storage test for m parameters is ${\rho }^{\left(1\right)},{\rho }^{\left(2\right)},\dots ,{\rho }^{\left(m\right)}$ according to the method described in Section 3.3. To obtain the confidence of the whole accelerated storage test, the weights of the m parameters need to be calculated, and the total confidence index is obtained by the weighted average method.

The dynamic time warping (DTW) distance [33,34] is a time series similarity measure with better performance that is suitable for the application scenario of different key performance parameters over time in this paper. Therefore, in this paper, the dynamic time warping distance is used to measure the similarity between individual response quantities and then to determine the contribution (i.e., weight) of each key performance parameter in the calculation of the overall confidence.

Suppose there are two time series $q$ and $c$, which are denoted as

$$\left\{\begin{array}{c}\mathit{q}=\left\{q_1,q_2,\cdots ,q_m\right\}\\ \mathit{c}=\left\{c_1,c_2,\cdots ,c_n\right\}\end{array}\right.$$

(24)

Then, the DTW distance can be recursively defined as

$$\{\begin{array}{l}{D}_{dtw}(<>,<>)=0\\ D_{dtw}(\mathit{q},<>)=D_{dtw}(<>,\mathit{c})=0\\ D_{dtw}(\mathit{q},\mathit{c})=d(q_1,c_1)+\min \{\begin{array}{l}{D}_{dtw}(\mathit{q},\mathrm{Re}\mathrm{st}(\mathit{c}))\\ D_{dtw}(\mathrm{Re}\mathrm{st}(\mathit{q}),\mathit{c})\\ D_{dtw}(\mathrm{Re}\mathrm{st}(\mathit{q}),\mathrm{Re}\mathrm{st}(\mathit{c}))\end{array}\\ d(q_i,c_j)={\Vert q_i-c_j\Vert }_p\end{array}$$

(25)

where $\mathrm{Rest}\left(q\right)=\left\{q_2,\dots ,q_m\right\}$, $\mathrm{Rest}\left(c\right)=\left\{c_2,\dots ,c_m\right\}$.

From the above definition, it can be seen that it calculates the distance between two time series by finding the minimum path of the distance between time series. The DTW distances of two time series X and Y can be calculated recursively directly using Equation (28).

The correlation calculation of the product’s m key parameters based on the DTW distance is provided below.

Step 1: Note that the time series obtained during the degradation trials of the two key parameters $P_i$ and $P_j$ are ${\mathit{y}}_i$ and ${\mathit{y}}_j(i,j=\mathrm{1,2},\cdots ,m)$, respectively, and the DTW distances of ${\mathit{y}}_i$ and ${\mathit{y}}_j(i,j=\mathrm{1,2},\cdots ,m)$ are calculated according to the above method. The results are expressed as $d_{i,j}$.

Step 2: Repeat Step 1 to obtain the DTW distance between two of all m key parameters, denoted as

$$\mathit{d}=\left[\begin{array}{cccc}0& d_{1,2}& \cdots & d_{1,m}\\ d_{2,1}& 0& \cdots & d_{2,m}\\ ⋮& ⋮& ⋮& ⋮\\ d_{m,1}& d_{m,2}& \cdots & 0\end{array}\right]$$

(26)

Step 3: Normalize Equation (26)

$${\tilde{d}}_{i,j}=\frac{d_{i,j}}{\underset{i}{sum}\{\underset{j}{sum}\{d\}\}}$$

(27)

Step 4: The rows of the DTW distance matrix d are summed to obtain the weight coefficient of the $j\mathrm{th}$ key parameter as

$${\varpi }_j={\displaystyle \sum _{i=1}^m}{d}_{i,j}\left(i,j=\mathrm{1,2},\cdots ,m\right)$$

(28)

Step 5: Let the credibility of m key parameters be $\widehat{\eta }$, and use the weighted average method to obtain a uniform credibility measure for the entire product accelerated storage test, that is

$$\widehat{\eta }={\displaystyle \sum _{j=1}^m}{\varpi }_j{\eta }_j$$

(29)

3.5. Expert Systems for Credibility Assessment

To facilitate designers or decision-makers to have a more intuitive judgment about the credibility of accelerated storage test results, this paper establishes the credibility evaluation level of the accelerated storage test shown in

Table 2. The AST credibility degree.

CMADT	$\mathit{\phi }\ge 85$	$70\le \mathit{\phi }<85$	$60\le \mathit{\phi }<70$	$\mathit{\phi }<60$
evaluation level	excellent	good	medium	poor

and divides the evaluation results into four levels: “excellent,” “good,” “medium,” and “poor.” In Table 2,

$$\phi =(1-\rho )*100$$

(30)

The number of intervals corresponding to the four levels is expressed as

$$\mathsf{\Omega }=\left\{\left[85,100\right],\left[70,85\right],\left[60,70\right],\left[0,60\right]\right\}$$

(31)

The confidence level of the accelerated storage test can be determined by calculating the similarity of the intervals $\phi$ and $\mathsf{\Omega }$. $i=1,2,3,4$ correspond to “excellent”, “good”, “medium”, and “poor”, respectively. The similarity is calculated by using the Euclidean distance of Equation (13), which is defined in Section 3.1; the four similarity variables ${\xi }_i(i=1,2,3,4)$ are obtained, and the confidence level of the key performance parameters is

$$j=\left\{j\left|{\xi }_j=max\{{\xi }_i\right\}\right\}$$

(32)

4. Use Case

A quartz accelerometer is a typical inertial device that is widely used in the mathematics field. The quartz plus meter experiences the problem of accuracy drift during long-term storage, which affects its reliability in use. With the aim of evaluating the storage life of quartz plus meters, a high-temperature accelerated storage test was conducted. The specimens were divided into three groups, $T_1,T_2,\cdot \cdot \cdot ,T_{10}$ denote the test cycles, each test cycle was 80 h, and accelerated storage tests were carried out at 60 °C, 72 °C, and 85 °C for approximately 800 h, with sample sizes of three, five, and five, respectively. The key performance parameters of the quartz accelerometer were the amount of output voltage degradation at position 0°; the amount of output voltage degradation at position 180°; the output voltage deviation of the centrifugal test, which were noted as P₁, P₂, and P₃, respectively; and the failure threshold $D_f=1.6\mathrm{mv}$. The performance degradation data for the above three parameters are shown in

Table 3. The amount of output voltage degradation for the whole quartz plus meter’s 0° position.

Accelerated Stress (°C)	Serial Number	Test Cycles and Measured Values
		T0	T1	T2	T3	T4	T5	T6	T7	T8	T9	T10
60	1	0	−0.02730	−0.01030	−0.01020	0.01260	0.05130	0.02880	0.03210	0.08730	0.09860	0.12550
	2	0	0.00050	−0.00440	−0.00070	−0.00510	0.00930	0.01390	0.02070	0.02140	0.03330	0.02470
	3	0	0.06790	0.07230	0.06320	0.09360	0.09750	0.12510	0.11960	0.09110	0.08260	0.14160
72	4	0	0.01880	0.02250	0.04330	0.05670	0.09040	0.07320	0.08100	0.15040	0.15260	0.17140
	5	0	−0.02730	−0.01030	−0.01020	0.01260	0.05130	0.02880	0.03210	0.08730	0.09860	0.12550
	6	0	0.01420	0.05850	0.03450	0.05230	0.08530	0.04560	0.05810	0.02300	0.08520	0.10360
	7	0	0.01420	0.05850	0.03450	0.05230	0.08530	0.04560	0.05810	0.02300	0.08520	0.10360
	8	0	0.00220	0.00210	0.01510	0.02290	0.02990	0.02040	0.03340	0.03340	0.06570	0.08600
85	9	0	0.03710	0.11450	0.02640	0.07070	0.13260	0.12890	0.16090	0.19910	0.16140	0.23130
	10	0	0.05980	0.04400	0.05960	0.10920	0.17010	0.25360	0.32970	0.36860	0.37820	0.25720
	11	0	0.01660	0.09620	0.09480	0.08360	0.07410	0.09560	0.07990	0.11700	0.11140	0.10800
	12	0	0.02190	0.01590	0.02010	0.03700	0.02510	0.04050	0.02790	0.07460	0.05720	0.08060
	13	0	0.06220	0.06440	0.07870	0.06880	0.08350	0.07810	0.11680	0.10340	0.19050	0.18830

,

Table 4. The amount of output voltage degradation for the whole quartz plus the meter’s 180° position.

Accelerated Stress (°C)	Serial Number	Test Cycles and Measured Values
		T0	T1	T2	T3	T4	T5	T6	T7	T8	T9	T10
60	1	0	−0.00760	0.00375	0.00816	0.00377	0.00507	0.00856	0.00251	0.02726	0.04558	0.05843
	2	0	−0.00537	0.03428	0.04603	0.09327	0.07479	0.13421	0.13046	0.13716	0.14078	0.13172
	3	0	0.01132	0.02604	0.04013	0.05384	0.04508	0.07164	0.07365	0.09690	0.07011	0.11637
72	4	0	0.03003	0.03335	0.02223	0.02099	0.03354	0.03607	0.04431	0.04853	0.07397	0.10085
	5	0	−0.00160	0.01510	0.03357	0.02817	0.04932	0.06003	0.07451	0.07866	0.08942	0.09837
	6	0	0.04752	0.05506	0.07672	0.08712	0.09437	0.13868	0.11316	0.11316	0.15462	0.22292
	7	0	0.02564	0.02246	0.06346	0.07203	0.07290	0.10490	0.05314	0.13885	0.08181	0.16473
	8	0	0.04440	0.01781	0.05957	0.05013	0.03349	0.10509	0.12531	0.23292	0.21116	0.26006
85	9	0	0.07157	0.02200	0.03846	0.08515	0.10439	0.11603	0.18147	0.17781	0.14061	0.24526
	10	0	−0.01757	−0.03662	0.00948	0.03310	0.03601	0.10666	0.16083	0.15703	0.11142	0.12234
	11	0	−0.00335	−0.01241	0.03374	0.04888	0.05855	0.09858	0.05904	0.13863	0.13938	0.16274
	12	0	0.01237	0.02353	−0.00120	0.07522	0.09946	0.11484	0.05961	0.18696	0.15179	0.19016
	13	0	0.16208	0.00233	0.03331	0.04404	0.11743	0.11824	0.13408	0.13410	0.24822	0.23834

and

Table 5. The amount of deviation from the output voltage of the whole quartz plus meter centrifugal test (unit: mV).

Accelerated Stress (°C)	Serial Number	Test Cycles and Measured Values
		T0	T1	T2	T3	T4	T5	T6	T7	T8	T9	T10
60	1	0	−1.5563	−0.2101	0.1878	0.1707	0.1284	0.1424	0.1216	0.0913	0.0850	0.0818
	2	0	0.0950	0.3085	0.4611	0.5182	0.4762	−0.1217	0.4338	0.4511	0.4116	0.3457
	3	0	−0.1628	0.0301	0.1907	0.1666	0.2039	0.7925	0.2670	0.2733	2.6573	3.7020
72	4	0	−1.3356	0.2356	0.2771	0.2516	0.2468	0.2369	0.2184	0.1770	0.1925	0.1815
	5	0	−1.5563	−0.2101	0.1878	0.1707	0.1284	0.1424	0.1216	0.0913	0.0850	0.0818
	6	0	−1.4207	0.1534	0.2229	0.2392	0.2138	0.1926	0.1805	0.1552	0.1526	0.1309
	7	0	−0.4945	−0.4664	−0.1641	−0.1808	−0.2290	−0.1516	−0.1347	−0.1376	−0.0864	−0.0976
	8	0	−0.0285	−0.1313	0.2119	0.2053	0.2025	0.2474	0.2350	0.2350	0.1779	0.2212
85	9	0	−0.3584	−0.2772	−0.0997	−0.0401	−0.0412	−0.0105	0.0437	−0.0398	−26.0193	−13.3701
	10	0	−0.2593	−0.0593	0.1532	0.2524	0.3648	0.3937	0.4513	0.5900	0.5699	0.6262
	11	0	−0.1668	−0.1101	0.0274	0.0682	0.0813	0.0574	0.0496	0.0543	0.0480	0.0867
	12	0	−0.2679	−0.2024	0.1163	0.1120	0.1229	0.0914	0.1191	0.0632	0.1188	0.0829
	13	0	−0.0682	−0.2282	0.0850	0.1071	0.1005	0.1235	0.1781	0.1067	0.0648	0.1402

.

As seen from the line graph of the test values under each group of stresses (Figure 3), there is an overall linear decreasing degradation trend for the three key performance parameters.

The degradation curves of the quartz accelerometer are drawn from the degradation data of three key performance parameters, namely, the amount of output voltage degradation at position 0°, the amount of output voltage degradation at position 180°, and the output voltage deviation from the centrifugal test at different stress levels.

Furthermore, the degradation curves of the three key performance parameters under normal stress levels are obtained, as shown in Figure 4.

Additionally, a batch (8) of quartz accelerometers manufactured in 2010 was tested at the factory and tested once a year during the period 2016 to 2021. The test data were expressed as a degraded quantity (i.e., the test value minus the initial value), and the results are shown in

Table 6. Output voltage degradation of the natural storage quartz accelerometer at position 0°.

Specimen/P1	T1	T2	T3	T4	T5	T6
1	4.0655	4.5687	4.9844	5.5893	2.0445	6.9587
2	8.1801	9.3958	10.2235	12.0112	2.6803	14.5245
3	4.8009	5.4314	5.9208	6.7370	5.1540	8.3109
4	5.8082	6.6132	7.2034	8.3092	9.7606	10.1631
5	3.7093	4.1509	4.5310	5.0335	9.3660	6.3039
6	3.5906	4.0116	4.3798	4.8482	0.4079	6.0857
7	6.0118	6.8520	7.4626	8.6269	5.2345	10.5375
8	0.9381	0.8998	1.0024	0.7084	6.4367	1.2084

,

Table 7. Output voltage degradation of the natural storage quartz accelerometer at position 180°.

Specimen/P2	T1	T2	T3	T4	T5	T6
1	4.9382	5.5647	6.0950	6.8372	8.8368	8.4453
2	7.5338	8.6227	9.4521	10.9137	7.7705	13.2957
3	3.0850	3.3814	3.6981	3.9268	8.9304	4.9822
4	3.4794	3.8461	4.2083	4.5463	7.7614	5.7193
5	1.6900	1.7380	1.8938	1.7359	9.8377	2.3754
6	8.6169	9.8988	10.8530	12.6147	7.5265	15.3197
7	5.6253	6.3743	6.9837	7.9164	18.1703	9.7294
8	4.5483	5.1054	5.5907	6.2249	3.8905	7.7167

and

Table 8. Natural storage quartz accelerometer centrifugal test output voltage deviation amount (unit: mV).

Specimen/P3	T1	T2	T3	T4	T5	T6
1	37.9425	44.1437	50.1953	56.5964	42.8070	69.3046
2	16.9875	19.6089	22.2900	24.7813	27.7813	30.5230
3	25.6247	29.7216	33.7919	37.8948	46.4200	46.5079
4	29.3904	34.1306	38.8066	43.6121	50.0673	53.4771
5	64.2759	74.9758	85.2629	96.5773	19.7894	118.0402
6	14.8657	17.1246	19.4644	21.5599	57.3586	26.5962
7	47.8747	55.7728	63.4218	71.6762	31.0873	87.6864
8	36.8833	42.9036	48.7848	54.9883	43.3959	67.3444

.

An accelerated storage test evaluation of the quartz accelerometer was carried out, and the three key performance parameters for 6~11 years of equivalent storage obeyed a normal distribution. The parameters are shown in

Table 9. Results of accelerated storage test evaluation of quartz accelerometers.

Equivalent Storage Time (Years)	Parameter Name	Average Value	Standard Deviation
6	Output voltage degradation amount at position 0°	3.7634	2.1589
	Output voltage degradation amount at position 180°	4.4788	2.5053
	Centrifugal test output voltage deviation	30.7629	15.9182
7	Output voltage degradation amount at position 0°	4.3689	2.5379
	Output voltage degradation amount at position 180°	5.1792	2.9568
	Centrifugal test output voltage deviation	35.8918	18.6427
8	Output voltage degradation amount at position 0°	4.9745	2.9171
	Output voltage degradation amount at position 180°	5.8796	3.4090
	Centrifugal test output voltage deviation	41.0208	21.3679
9	Output voltage degradation amount at position 0°	5.5800	3.2963
	Output voltage degradation amount at position 180°	6.5799	3.8616
	Centrifugal test output voltage deviation	46.1497	24.0935
10	Output voltage degradation amount at position 0°	6.1855	3.6755
	Output voltage degradation amount at position 180°	7.2803	4.3145
	Centrifugal test output voltage deviation	51.2786	26.8194
11	Output voltage degradation amount at position 0°	6.7910	4.0548
	Output voltage degradation amount at position 180°	7.9806	4.7676
	Centrifugal test output voltage deviation	56.4075	29.5456

.

The probability distribution of the key performance parameters related to the quartz accelerometer during natural storage is calculated. For P₁, according to Equations (4)–(12), the p-box for its storage from 6 to 11 years can be constructed, as shown in Figure 5.

Similarly, the p-box of the performance degradation data for P₂ and P₃ at each storage time can also be obtained, which is not listed here due to space limitations.

The confidence indices of individual key performance parameters at each natural storage moment are also calculated. First, the plausibility index of P₁ at t = 6a is calculated. According to the probability distribution of p-box and the equivalent storage 6a of P₁ under natural storage at t = 6a (as shown in Figure 6), from Equations (1) and (2), it is obtained that

$$\begin{gathered} A\left(F_{t_i}^a\right(x),F_{t_i}^0(x\left)\right)=6.4768 \\ A\left(F_{t_i}^a\right(x),{\overline{F}}_{t_i}^b(x\left)\right)=1.2469 \\ A(F_{t_i}^a(x),{\underset{¯}{F}}_{t_i}^b(x))=1.3370 \end{gathered}$$

These expressions lead to the upper and lower bounds for the normalized plausibility metric CMADT as

$${\overline{\rho }}_i=\max \left\{\frac{A\left(F_{t_i}^a\right(x),{\overline{F}}_{t_i}^b(x\left)\right)}{A\left(F_{t_i}^a\right(x),F_{t_i}^0(x\left)\right)},\frac{A\left(F_{t_i}^a\right(x),{\stackrel{̱}{F}}_{t_i}^b(x\left)\right)}{A\left(F_{t_i}^a\right(x),F_{t_i}^0(x\left)\right)}\right\}=0.2064$$

$${\stackrel{̱}{\rho }}_i=\min \left\{\frac{A\left(F_{t_i}^a\right(x),{\overline{F}}_{t_i}^b(x\left)\right)}{A\left(F_{t_i}^a\right(x),F_{t_i}^0(x\left)\right)},\frac{A\left(F_{t_i}^a\right(x),{\stackrel{̱}{F}}_{t_i}^b(x\left)\right)}{A\left(F_{t_i}^a\right(x),F_{t_i}^0(x\left)\right)}\right\}=0.1925$$

Similarly, the plausibility metrics can be calculated for the five-time 00 positions P₁ from t = 7a to t = 11a. The procedure for calculating the plausibility metrics for the two key performance parameters, P₂ and P₃, is similar.

Table 10. Calculation results of the credibility metrics.

Parameters	CMADT
	t = 6a	t = 7a	t = 8a	t = 9a	t = 10a	t = 11a
P1	[0.1925,0.2064]	[0.1768,0.1889]	[0.1584,0.1655]	[0.1596,0.1702]	[0.1179,0.1265]	[0.1674,0.1773]
P2	[0.1355,0.1478]	[0.1271,0.1393]	[0.1225,0.1346]	[0.1182,0.1300]	[0.1999,0.2178]	[0.1237,0.1360]
P3	[0.1323,0.1537]	[0.1304,0.1520]	[0.1280,0.1502]	[0.1281,0.1498]	[0.1816,0.1985]	[0.1292,0.1510]

shows the calculation results of the credibility metrics for the three key performance parameters of the quartz plus table.

The overall confidence evaluation index CSP for a single key performance parameter is calculated. According to the K–S test, the parameter P₁ in Table 10 follows a normal distribution. Otherwise, P₂ and P₃ do not follow a normal distribution, so kernel density estimates are used to calculate their statistical properties. Using the confidence level γ = 0.8, $\alpha =1-\gamma =0.2,Z_{1-\alpha /2}=1.2816$, according to Equations (15) and (16), it is obtained that

$$\begin{gathered} \left\{\begin{array}{c}{\rho }_L^{-}={\mu }_{\rho ,L}-{\sigma }_{\rho ,L}{z}_{1-\alpha /2}=0.1490\\ {\rho }_L^{+}={\mu }_{\rho ,L}+{\sigma }_{\rho ,L}{z}_{1-\alpha /2}=0.1752\end{array}\right. \\ \left\{\begin{array}{c}{\rho }_U^{-}={\mu }_{\rho ,U}-{\sigma }_{\rho ,U}{z}_{1-\alpha /2}=0.1584\\ {\rho }_U^{+}={\mu }_{\rho ,U}+{\sigma }_{\rho ,U}{z}_{1-\alpha /2}=0.1865\end{array}\right. \end{gathered}$$

In turn, the evaluation result $\eta =[{\eta }_L,{\eta }_U]$ is obtained as

$$\left\{\begin{array}{c}{\eta }_L={\mu }_{\rho ,L}-{\sigma }_{\rho ,L}{z}_{1-\alpha /2}=0.1752\\ {\eta }_U={\mu }_{\rho ,U}-{\sigma }_{\rho ,U}{z}_{1-\alpha /2}=0.1865\end{array}\right.$$

P₂ and P₃ use kernel density estimation, and their evaluation results $\eta =[{\eta }_L,{\eta }_U]$ are

$$\begin{gathered} \left\{\begin{array}{c}{\eta }_L={\widehat{f}}_L^{-1}(\alpha /2)=0.1991\\ {\eta }_U={\widehat{f}}_U^{-1}(\alpha /2)=0.2169\end{array}\right. \\ \left\{\begin{array}{c}{\eta }_L={\widehat{f}}_L^{-1}(\alpha /2)=0.1814\\ {\eta }_U={\widehat{f}}_U^{-1}(\alpha /2)=0.1982\end{array}\right. \end{gathered}$$

From Equations (24)–(26), the DTW distance between two of all m key parameters is obtained as

$$\mathit{d}=\left[\begin{array}{ccc}0& 2.2262& 7068.3974\\ 2.3426& 0& 6679.7406\\ 8888.4015& 8406.7690& 0\end{array}\right]$$

Furthermore, from Equations (26)–(28), the weight coefficient of the jth key parameter is obtained as

$${\varpi }_1=0.2277;{\varpi }_2=0.2152;{\varpi }_3=0.5570$$

From Equation (25), the weighted average method is used to obtain a uniform measure of confidence for the entire product accelerated storage test as

$$\widehat{\eta }=[{\widehat{\eta }}_L,{\widehat{\eta }}_U]=\left[0.1838,0.1996\right]$$

Then, from Equation (31), $\phi$ = [80.04, 81.62]. According to Table 2, the confidence level of the accelerated storage test is “good”.

5. Conclusions

In many engineering projects related to AST, although many efforts have been expended and various mathematical methods have been used to strive for the accuracy of the evaluation results each time, users still question whether the results obtained under accelerated stress can truly reflect the life of the product. As a result, the results of AST are often questioned in engineering practice, but there seems to be little discussion in academia.

To evaluate the credibility of ASTs, this paper adopts the idea of the area metric to construct the area metric of ASTs and uses natural storage test data as the benchmark. Therefore, a normalized and dimensionless reliability metric CMADT for ASTs is proposed. The percentage from “100%” to “0%” represents model reliability from best to worst. Based on the concept of performance importance, multiple single-point CMADTs are combined into one metric that can reflect the reliability of the AST results for the overall product.

The normalized, dimensionless index CMADT proposed in this paper is of great significance. It meets the challenge that conventional methods cannot quantitatively assess AST results. The CMADT not only allows designers and decision-makers to judge the credibility of AST intuitively, but also allows horizontal comparison of AST results for different products. The quantitative evaluation of CMADT is transformed into expressions that are suitable for human thought in Table 2, i.e., “excellent,” “good,” “moderate,” and “poor,” which can help senior decision-makers make better judgments. In addition, CMADT is applicable to different situations, such as those with large samples, small samples, and very small samples, and it has good generality. In addition, due to the lack of similar research results, it is difficult to compare the results with those of existing research methods. If there are any inaccuracies in the viewpoints of this article, we hope readers can point them out and explore them.