Optimal Release Time and Sensitivity Analysis Using a New NHPP Software Reliability Model with Probability of Fault Removal Subject to Operating Environments

Abstract

With the latest technological developments, the software industry is at the center of the fourth industrial revolution. In today’s complex and rapidly changing environment, where software applications must be developed quickly and easily, software must be focused on rapidly changing information technology. The basic goal of software engineering is to produce high-quality software at low cost. However, because of the complexity of software systems, software development can be time consuming and expensive. Software reliability models (SRMs) are used to estimate and predict the reliability, number of remaining faults, failure intensity, total and development cost, etc., of software. Additionally, it is very important to decide when, how, and at what cost to release the software to users. In this study, we propose a new nonhomogeneous Poisson process (NHPP) SRM with a fault detection rate function affected by the probability of fault removal on failure subject to operating environments and discuss the optimal release time and software reliability with the new NHPP SRM. The example results show a good fit to the proposed model, and we propose an optimal release time for a given change in the proposed model.

1. Introduction

With the latest technological developments, the software industry is at the center of the fourth industrial revolution. The fourth industrial revolution relies on new and innovative information and communication technologies, cyber–physical systems, network communications, simulation, big data analysis, and cloud computing. Software systems play a vital role in controlling key machines in large industries such as aviation, medical, defense, and energy. In today’s complex and rapidly changing environment, where software applications must be developed quickly and easily, software must be focused on rapidly changing information technology. Software systems improve solutions for immediate problems in a variety of industries and continue to offer customers convenience. The systems should be easy-to-use, error-free, and create a product that gives value to its users. To create good software, functions must be implemented that exactly match user requirements, and guarantee reliability, functionality, and performance. Software reliability, defined as the probability of failure-free operation under certain conditions and for a specific time, is one of the significant attributes of the software system development life cycle. Many software reliability models (SRMs) have been proposed to measure, predict, and ensure reliability. Moreover, the growth of reliability, and the trade-off between cost expenditure and optimal release both depend on the accuracy of the established SRM. The basic goal is to produce high-quality software at low cost in software engineering. However, because of the complexity of software systems, software development can be time-consuming and expensive. Therefore, the main focus of software companies is on improving the reliability and stability of a software system. This has prompted research into software reliability engineering and many software reliability growth models have been proposed in recent decades. SRMs are used to estimate and predict the reliability, number of remaining faults, failure intensity, total development cost, etc., of software. Discovering the reliability confidence intervals is done in the field of software reliability because it can aid the decision of software releases and control the related expenditures for software testing [1]. The purpose of many nonhomogeneous Poisson process (NHPP) software reliability models is to obtain an explicit formula for the mean value function m(t), which is applied to the software testing data and used to make predictions of software failures and reliability in the field environments [2]. The Goel–Okumoto (GO) [3] model is one of the most representative studies of SRMs. The GO model defines the mean value function m(t) and the intensity function λ(t) using an exponential distribution and estimates the reliability during mission time t by estimating the number of failures in the course of removing the remaining defects in the software. Yamada et al. [4] and Ohba [5] developed a model of the software fault detection process to evaluate software reliability based on test results during software development. Since then, many models using environmental changes have been developed; Teng and Pham [6] discussed a generalized model that captures the uncertainty of the environment and its effects upon the software failure rate. Recently, Inoue et al. [7] conducted software reliability modeling with uncertainty of testing environments. In addition, Li and Pham [2,8] proposed NHPP SRMs considering fault removal efficiency and error generation, and the uncertainty of operating environments with imperfect debugging and testing coverage. Song et al. [9,10,11] studied NHPP SRMs with various fault detection rates considering the uncertainty of operating environments.

Achieving the unique intended purpose of the software is more important than anything else. If a software failure occurs, such as when the software runs well and suddenly stops, it causes a huge loss to the enterprise and society. The general and specific requirements of the software are interrelated and require a variety of complex features. The cost of investing in software has increased. Indeed, this phenomenon is evident in the management of many software companies. Many companies today prefer short software release cycles. A short software release cycle has many advantages and disadvantages, but the reason for preferring a short software release cycle is to reduce the user’s waiting time. Therefore, it is very important to decide when, how, and at what cost to release the software to users. Some early works have been conducted on the optimal software release problems [12,13,14,15,16], and new software cost models have been developed used in related works in recent years [17,18,19,20,21].

In this paper, we discuss a new NHPP SRM with a fault detection rate function affected by the probability of fault removal on failure when considering operating environments and discuss the optimal release time and software reliability with the new NHPP SRM. The explicit solution of the mean value function for the new NHPP software reliability model is derived in Section 2. The optimal release policy is discussed in Section 3, and criteria for the model comparison, results of a model analysis, optimal release time, and software reliability are discussed in Section 4. Finally, Section 5 provides some concluding remarks.

2. NHPP SRM

2.1. NHPP SRM

A repairable system means a system capable of switching from an active state to a faulty state and back to an active state. Statistical analysis of repairable systems is a very important part of the reliability field because there are many more repairable products compared to products that are virtually non-repairable. Such a repairable system includes a dummy model, a maintenance model, and a repair model, and the selection of the model may vary depending on how the failure is handled. In this case, if the time required for replacement, maintenance, and repair is negligible, that is, if it can be assumed to be zero, the failure time can be adapted to a point process. The Poisson process, which is one of the most important processes in the count process, shows probabilistic characteristics when the number of occurrences in a given interval follows the Poisson distribution and the number of occurrences of the events are independent of each other. This Poisson process can be classified as homogeneous Poisson process (HPP) and NHPP, which deals only with the NHPP.

If the counting process {$\mathrm{N}\left(\mathrm{t}\right),\mathrm{t}\ge 0$} satisfies three conditions: $\mathrm{N}\left(0\right)=0$, with independent increments, and the average of the number of failures in the interval [${\mathrm{t}}_1,{\mathrm{t}}_2$], it is called an NHPP with an intensity function $\mathsf{\lambda }\left(\mathrm{t}\right)$. $\mathrm{N}\left(\mathrm{t}\right)$ ($\mathrm{t}\ge 0$) follows a Poisson distribution with parameter m(t):

$$\mathrm{Pr}\left\{\mathrm{N}\left(\mathrm{t}\right)=\mathrm{n}\right\}=\frac{{\left\{\mathrm{m}\left(\mathrm{t}\right)\right\}}^{\mathrm{n}}}{\mathrm{n}!}\exp \left\{-\mathrm{m}\left(\mathrm{t}\right)\right\},\mathrm{n}=0,1,2,3,\dots .$$

where m(t) is the mean value function of the NHPP.

The intensity function $\mathsf{\lambda }\left(\mathrm{t}\right)$ is as follows.

$$\frac{\mathrm{dm}\left(\mathrm{t}\right)}{\mathrm{dt}}=\mathsf{\lambda }\left(\mathrm{t}\right).$$

Pham et al. [22] formalized the general framework for the software reliability based on NHPP and provided numerical expressions for the mean value function $\mathrm{m}\left(\mathrm{t}\right)$ using differential equations. The mean value function $\mathrm{m}\left(\mathrm{t}\right)$ of the general NHPP SRM with different values for $\mathrm{a}\left(\mathrm{t}\right)$ and $\mathrm{b}\left(\mathrm{t}\right)$, which reflect various assumptions, can be obtained with the initial condition $\mathrm{N}\left(0\right)=0$.

$$\frac{\mathrm{d}\mathrm{m}\left(\mathrm{t}\right)}{\mathrm{dt}}=\mathrm{b}\left(\mathrm{t}\right)\left[\mathrm{a}\left(\mathrm{t}\right)-\mathrm{m}\left(\mathrm{t}\right)\right].$$

(1)

The general solution of (1) is

$$\mathrm{m}\left(\mathrm{t}\right)={\mathrm{e}}^{-\mathrm{B}\left(\mathrm{t}\right)}[{\mathrm{m}}_0+{{\displaystyle \int }}_{{\mathrm{t}}_0}^{\mathrm{t}}\mathrm{a}\left(\mathrm{s}\right)\mathrm{b}\left(\mathrm{s}\right){\mathrm{e}}^{\mathrm{B}\left(\mathrm{s}\right)}\mathrm{bs}],$$

(2)

where $\mathrm{B}\left(\mathrm{t}\right)={{\displaystyle \int }}_{{\mathrm{t}}_0}^{\mathrm{t}}\mathrm{b}(\mathrm{s})\mathrm{ds}$, and $\mathrm{m}\left({\mathrm{t}}_0\right)={\mathrm{m}}_0$ is the marginal condition of (2).

A generalized NHPP SRM that incorporates uncertainty in the operating environment is formulated as follows [23]:

$$\frac{\mathrm{d}\mathrm{m}\left(\mathrm{t}\right)}{\mathrm{dt}}=\mathsf{\eta }\left[\mathrm{b}\left(\mathrm{t}\right)\right]\left[\mathrm{N}-\mathrm{m}\left(\mathrm{t}\right)\right],$$

(3)

where $\mathsf{\eta }$ is a random variable that represents the uncertainty of the system fault detection rate in the operating environment with a probability density function g; $\mathrm{b}\left(\mathrm{t}\right)$ is the fault detection rate function, which also represents the average failure rate caused by faults; $\mathrm{N}$ is the expected number of faults that exist in the software before testing.

Thus, a generalized mean value function, m(t), where the initial condition m(0) = 0, is given by

$$\mathrm{m}\left(\mathrm{t}\right)={{\displaystyle \int }}_{\mathsf{\eta }}\mathrm{N}(1-{\mathrm{e}}^{-\mathsf{\eta }{{\displaystyle \int }}_0^{\mathrm{t}}\mathrm{b}\left(\mathrm{x}\right)\mathrm{dx}})\mathrm{dg}\left(\mathsf{\eta }\right).$$

(4)

The mean value function [24] from (4) using the random variable η has a generalized probability density function g with two parameters $\mathsf{\alpha }\ge 0$ and $\mathsf{\beta }\ge 0$, and is given by

$$\mathrm{m}\left(\mathrm{t}\right)=\mathrm{N}{(1-\frac{\mathsf{\beta }}{\mathsf{\beta }+{{\displaystyle \int }}_0^{\mathrm{t}}\mathrm{b}\left(\mathrm{s}\right)\mathrm{ds}})}^{\mathsf{\alpha }},$$

(5)

where b(t) is the fault detection rate per fault per unit time.

2.2. New NHPP SRM

We propose a new NHPP SRM using Equations (3)–(5) and add the following assumptions to those of the existing NHPP SRM subject to operating environments [9,10,11]:

• The fault detection rate function will be affected by the probability of fault removal on a failure.

In this study, we consider the fault detection rate function $\mathrm{b}\left(\mathrm{t}\right)$ to be as follows:

$$\mathrm{b}\left(\mathrm{t}\right)=\frac{\mathrm{ap}}{1+{\mathsf{\gamma }\mathrm{e}}^{-\mathrm{bpt}}},\mathrm{a},\mathrm{b},\mathsf{\gamma }>0,0<\mathrm{p}<1,$$

(6)

where b(t) is an S-shaped curve that can capture the learning process of the software testers/developers and this function is affected by the probability of fault removal on a failure. We obtain a new NHPP SRM with a fault detection rate function affected by the probability of fault removal on failure subject to the uncertainty of the environments, m(t), that can be used to determine the expected number of software failures detected by time t by substituting the function b(t) above into Equation (5):

$$\mathrm{m}\left(\mathrm{t}\right)=\mathrm{N}{\left[1-\left(\frac{\mathsf{\beta }}{\mathsf{\beta }-\frac{\mathrm{a}}{\mathrm{b}}\ln \left(\frac{\left(1+\mathsf{\gamma }\right){\mathrm{e}}^{-\mathrm{bpt}}}{1+{\mathsf{\gamma }\mathrm{e}}^{-\mathrm{bpt}}}\right)}\right)\right]}^{\alpha }.$$

(7)

Table 1. Nonhomogeneous Poisson process (NHPP) software reliability models and their mean value functions and intensity functions.

No.	Model	$\mathbf{m}\left(\mathbf{t}\right)$	$\mathsf{\lambda }\left(\mathbf{t}\right)$	Model Type
1	Goel–Okumoto [3] (GO)	$\mathrm{m}\left(\mathrm{t}\right)=\mathrm{a}\left(1-{\mathrm{e}}^{-\mathrm{bt}}\right)$	$\mathsf{\lambda }\left(\mathrm{t}\right)={\mathrm{abe}}^{-\mathrm{bt}}$	Concave
2	Yamada et al. (Delayed S-shaped) [4] (DS)	$\mathrm{m}\left(\mathrm{t}\right)=\mathrm{a}\left(1-\left(1+\mathrm{bt}\right){\mathrm{e}}^{-\mathrm{bt}}\right)$	$\mathsf{\lambda }\left(\mathrm{t}\right)={\mathrm{ab}}^2{\mathrm{te}}^{-\mathrm{bt}}$	S
3	Ohba (Inflection S-shaped) [5] (IS)	$\mathrm{m}\left(\mathrm{t}\right)=\frac{\mathrm{a}\left(1-{\mathrm{e}}^{-\mathrm{bt}}\right)}{1+{\mathsf{\beta }\mathrm{e}}^{-\mathrm{bt}}}$	$\mathsf{\lambda }\left(\mathrm{t}\right)=\frac{\mathrm{ab}\left(1+\mathsf{\beta }\right){\mathrm{e}}^{-\mathrm{bt}}}{{\left(1+{\mathsf{\beta }\mathrm{e}}^{-\mathrm{bt}}\right)}^2}$	S
4	Yamada et al. (Imperfect Debugging 1) [25] (Y1)	$\mathrm{m}\left(\mathrm{t}\right)=\frac{\mathrm{ab}}{\mathsf{\alpha }+\mathrm{b}}\left({\mathrm{e}}^{\mathsf{\alpha }\mathrm{t}}-{\mathrm{e}}^{-\mathrm{bt}}\right)$	$\mathsf{\lambda }\left(\mathrm{t}\right)=\frac{\mathrm{ab}}{\mathsf{\alpha }+\mathrm{b}}\left({\mathsf{\alpha }\mathrm{e}}^{\mathsf{\alpha }\mathrm{t}}+{\mathrm{be}}^{-\mathrm{bt}}\right)$	Concave
5	Yamada et al. (Imperfect Debugging 2) [25] (Y2)	$\mathrm{m}\left(\mathrm{t}\right)=\mathrm{a}\left[1-{\mathrm{e}}^{-\mathrm{bt}}\right]\left[1-\frac{\mathsf{\alpha }}{\mathrm{b}}\right]+\mathsf{\alpha }\mathrm{at}$	$\mathsf{\lambda }\left(\mathrm{t}\right)={\mathrm{abe}}^{-\mathrm{bt}}\left[1-\frac{\mathsf{\alpha }}{\mathrm{b}}\right]+\mathsf{\alpha }\mathrm{a}$	Concave
6	Pham et al. (Generalized Imperfect Debugging 1) [22] (PNZ)	$\mathrm{m}\left(\mathrm{t}\right)=\frac{\mathrm{a}\left[1-{\mathrm{e}}^{-\mathrm{bt}}\right]\left[1-\frac{\mathsf{\alpha }}{\mathrm{b}}\right]+\mathsf{\alpha }\mathrm{at}}{1+{\mathsf{\beta }\mathrm{e}}^{-\mathrm{bt}}}$	$\mathsf{\lambda }\left(\mathrm{t}\right)=\frac{\mathrm{ab}\left[1-\frac{\mathsf{\alpha }}{\mathrm{b}}\right]{\mathrm{e}}^{-\mathrm{bt}}+\mathsf{\alpha }\mathrm{a}}{1+{\mathsf{\beta }\mathrm{e}}^{-\mathrm{bt}}}$ $+\frac{\mathrm{b}\mathsf{\beta }(\mathrm{a}\left[1-{\mathrm{e}}^{-\mathrm{bt}}\right]\left[1-\frac{\mathsf{\alpha }}{\mathrm{b}}\right]+\mathsf{\alpha }\mathrm{at}){\mathrm{e}}^{-\mathrm{bt}}}{{(1+{\mathsf{\beta }\mathrm{e}}^{-\mathrm{bt}})}^2}$	Concave, S
7	Pham–Zhang (Generalized Imperfect Debugging 2) [26] (PZ)	$\mathrm{m}\left(\mathrm{t}\right)=\frac{1}{1+{\mathsf{\beta }\mathrm{e}}^{-\mathrm{bt}}}\ast$ $(\left(\mathrm{c}+\mathrm{a}\right)\left[1-{\mathrm{e}}^{-\mathrm{bt}}\right]$ $-\left[\frac{\mathrm{ab}}{\mathrm{b}-\mathsf{\alpha }}\left({\mathrm{e}}^{-\mathsf{\alpha }\mathrm{t}}-{\mathrm{e}}^{-\mathrm{bt}}\right)\right])$	$\mathsf{\lambda }\left(\mathrm{t}\right)=\frac{\mathrm{b}\left(\mathrm{c}+\mathrm{a}\right){\mathrm{e}}^{-\mathrm{bt}}-\frac{\mathrm{ab}}{\mathrm{b}-\mathsf{\alpha }}({\mathrm{be}}^{-\mathrm{bt}}-{\mathsf{\alpha }\mathrm{e}}^{-\mathsf{\alpha }\mathrm{t}})}{1+{\mathsf{\beta }\mathrm{e}}^{-\mathrm{bt}}}$ +$\frac{{\mathsf{\beta }\mathrm{be}}^{-\mathrm{bt}}[\left(\mathrm{c}+\mathrm{a}\right)\left(1-{\mathrm{e}}^{-\mathrm{bt}}\right)-\frac{\mathrm{ab}}{\mathrm{b}-\mathsf{\alpha }}({\mathrm{e}}^{-\mathsf{\alpha }\mathrm{t}}-{\mathrm{e}}^{-\mathrm{bt}})}{{(1+{\mathsf{\beta }\mathrm{e}}^{-\mathrm{bt}})}^2}$	Concave, S
8	Pham (Dependent Parameter 1) [27] (DP1)	$\mathrm{m}\left(\mathrm{t}\right)=\mathsf{\alpha }\left(\mathsf{\gamma }\mathrm{t}+1\right)(\mathsf{\gamma }\mathrm{t}-1+{\mathrm{e}}^{-\mathsf{\gamma }\mathrm{t}}$)	$\mathsf{\lambda }\left(\mathrm{t}\right)=\mathsf{\alpha }\mathsf{\gamma }[\left(\mathsf{\gamma }\mathrm{t}+1\right)(1-{\mathrm{e}}^{-\mathsf{\gamma }\mathrm{t}}$) $+(\mathsf{\gamma }\mathrm{t}-1+{\mathrm{e}}^{-\mathsf{\gamma }\mathrm{t}}$)]	Convex
9	Pham (Dependent Parameter 2) [27] (DP2)	$\mathrm{m}\left(\mathrm{t}\right)={\mathrm{m}}_0\left(\frac{\mathsf{\gamma }\mathrm{t}+1}{{\mathsf{\gamma }\mathrm{t}}_0+1}\right)\ast$ ${\mathrm{e}}^{-\mathsf{\gamma }\left(\mathrm{t}-{\mathrm{t}}_0\right)}+\mathsf{\alpha }\left(\mathsf{\gamma }\mathrm{t}+1\right)(\mathsf{\gamma }\mathrm{t}-1+\left(1-{\mathsf{\gamma }\mathrm{t}}_0\right){\mathrm{e}}^{-\mathsf{\gamma }\left(\mathrm{t}-{\mathrm{t}}_0\right)}$	$$\begin{gathered} \mathsf{\lambda }\left(\mathrm{t}\right)={\mathsf{\alpha }\mathsf{\gamma }}^2\mathrm{t}[2-\left(1-{\mathsf{\gamma }\mathrm{t}}_0\right){\mathrm{e}}^{-\mathsf{\gamma }\left(\mathrm{t}-{\mathrm{t}}_0\right)} \\ -{\mathrm{m}}_0\left(\frac{{\mathsf{\gamma }}^2\mathrm{t}}{{\mathsf{\gamma }\mathrm{t}}_0+1}\right){\mathrm{e}}^{-\mathsf{\gamma }\left(\mathrm{t}-{\mathrm{t}}_0\right)} \end{gathered}$$	Convex
10	Chang et al. (Testing Coverage) [28] (TC)	$\mathrm{m}\left(\mathrm{t}\right)=\mathrm{N}\left[1-{\left(\frac{\mathsf{\beta }}{\mathsf{\beta }+{\left(\mathrm{at}\right)}^{\mathrm{b}}}\right)}^{\mathsf{\alpha }}\right]$	$\mathsf{\lambda }\left(\mathrm{t}\right)=\frac{{\mathrm{Na}}^{\mathrm{b}}{\mathrm{b}\mathsf{\alpha }\mathsf{\beta }}^{\mathsf{\alpha }}{\mathrm{t}}^{\mathrm{b}-1}}{{\left(\mathsf{\beta }+{\left(\mathrm{at}\right)}^{\mathrm{b}}\right)}^{\mathsf{\alpha }+1}}$	Concave
11	Song et al. (Three parameter) [9] (3PD)	$\mathrm{m}\left(\mathrm{t}\right)=\mathrm{N}\left[1-\left(\frac{\mathsf{\beta }}{\mathsf{\beta }-\frac{\mathrm{a}}{\mathrm{b}}\ln \left(\frac{\left(1+\mathrm{c}\right){\mathrm{e}}^{-\mathrm{bt}}}{1+{\mathrm{ce}}^{-\mathrm{bt}}}\right)}\right)\right]$	$\mathsf{\lambda }\left(\mathrm{t}\right)=\frac{\mathrm{Na}\mathsf{\beta }\left(1-\frac{{\mathrm{ce}}^{-\mathrm{bt}}}{1+{\mathrm{ce}}^{-\mathrm{bt}}}\right)}{{\left(\mathsf{\beta }-\frac{\mathrm{a}}{\mathrm{b}}\ln \left(\frac{\left(1+\mathrm{c}\right){\mathrm{e}}^{-\mathrm{bt}}}{1+{\mathrm{ce}}^{-\mathrm{bt}}}\right)\right)}^2}$	Concave, S
12	Song et al. (Weibull) [10] (NW)	$\mathrm{m}\left(\mathrm{t}\right)=\mathrm{N}{\left(1-\frac{\mathsf{\beta }}{\mathsf{\beta }+{(\mathrm{at})}^{\mathrm{b}}}\right)}^{\mathsf{\alpha }}$	$\mathsf{\lambda }\left(\mathrm{t}\right)=\frac{\mathrm{Nab}\mathsf{\alpha }\mathsf{\beta }{\left(\mathrm{at}\right)}^{\mathrm{b}-1}{\left(1-\frac{\mathsf{\beta }}{\mathsf{\beta }+{\left(\mathrm{at}\right)}^{\mathrm{b}}}\right)}^{\mathsf{\alpha }-1}}{{\left(\mathsf{\beta }+{\left(\mathrm{at}\right)}^{\mathrm{b}}\right)}^2}$	S
13	New Proposed (New)	$\mathrm{m}\left(\mathrm{t}\right)=\mathrm{N}{\left[1-\left(\frac{\mathsf{\beta }}{\mathsf{\beta }-\frac{\mathrm{a}}{\mathrm{b}}\ln \left(\frac{\left(1+\mathsf{\gamma }\right){\mathrm{e}}^{-\mathrm{bpt}}}{1+{\mathsf{\gamma }\mathrm{e}}^{-\mathrm{bpt}}}\right)}\right)\right]}^{\mathsf{\alpha }}$	$\mathsf{\lambda }\left(\mathrm{t}\right)=\mathrm{Na}\mathsf{\alpha }\mathsf{\beta }\mathrm{p}\ast \left(\frac{{\mathsf{\gamma }\mathrm{e}}^{-\mathrm{bpt}}}{1+{\mathsf{\gamma }\mathrm{e}}^{-\mathrm{bpt}}}-1\right)$ $\ast \frac{{\left(1-\left(\frac{\mathsf{\beta }}{\mathsf{\beta }-\frac{\mathrm{a}}{\mathrm{b}}\ln \left(\frac{\left(1+\mathsf{\gamma }\right){\mathrm{e}}^{-\mathrm{bpt}}}{1+{\mathsf{\gamma }\mathrm{e}}^{-\mathrm{bpt}}}\right)}\right)\right)}^{\mathsf{\alpha }-1}}{{\left(\mathsf{\beta }-\frac{\mathrm{a}}{\mathrm{b}}\ln \left(\frac{\left(1+\mathsf{\gamma }\right){\mathrm{e}}^{-\mathrm{bpt}}}{1+{\mathsf{\gamma }\mathrm{e}}^{-\mathrm{bpt}}}\right)\right)}^2}$	S

summarizes the model types and the different mean value functions and intensity functions of the proposed new model and other existing NHPP models. Models 10 through 13 take into account the uncertainty of the operating environment.

3. Optimal Software Release

The basic infrastructure of the software cost is described in Figure 1 from the testing environment to the end of the software field environment. As can be seen from Figure 1, we can deduce the basic cost model with software testing and operating cost, software removal cost, and risk cost when software is released.

The expected total software cost $\mathrm{EC}\left(\mathrm{T}\right)$ can be expressed as

$$\mathrm{EC}\left(\mathrm{T}\right)={\mathrm{C}}_1\mathrm{T}+{\mathrm{C}}_2\mathrm{m}\left(\mathrm{T}\right)+{\mathrm{C}}_3\left(1-\mathrm{R}\left(\mathrm{x}|\mathrm{T}\right)\right)+{\mathrm{C}}_4\left[\mathrm{m}\left(\mathrm{T}+\mathrm{x}\right)-\mathrm{m}\left(\mathrm{T}\right)\right]$$

(8)

where ${\mathrm{C}}_1\mathrm{T}$ is the cost of testing, ${\mathrm{C}}_2\mathrm{m}\left(\mathrm{T}\right)$ and ${\mathrm{C}}_4\left[\mathrm{m}\left(\mathrm{T}+\mathrm{x}\right)-\mathrm{m}\left(\mathrm{T}\right)\right]$ are the cost to remove all errors detected by time $\mathrm{T}$ during the testing and operating phases, ${\mathrm{C}}_3\left(1-\mathrm{R}\left(\mathrm{x}|\mathrm{T}\right)\right)$ is the risk cost owing to failures that occur after the system release time $\mathrm{T}$.

We aim to find the optimal software release time, T*, with the basic reliability requirements and minimum total software cost as follows:

$$\begin{gathered} \mathrm{Minimize}\mathrm{C}\left(\mathrm{T}\right) \\ \mathrm{Subject}\mathrm{to}\mathrm{R}\left(\mathrm{x}\right|\mathrm{T})\ge {\mathrm{R}}_0 \end{gathered}$$

4. Numerical Examples

4.1. Criteria for Model Comparison

The parameters in the mean value function $\mathrm{m}\left(\mathrm{t}\right)$ of models can be estimated using various parameter estimation methods. Here, the parameter for the mean value function m(t) is estimated by the LSE (least squares estimate) method. Eight common criteria for model comparison, i.e., MSE (mean squared error), RMSE (root mean squared error), AIC (Akaike’s information criterion), R² (correlation index of the regression curve equation), Adj R² (adjected R²), SAE (sum of absolute error), PRR (predictive ratio risk), and PP (predictive power), will be used for the goodness-of-fit estimation of the model and to compare the proposed model with other models in Table 1. These criteria are described as follows in

Table 2. List of criteria for model comparisons. MSE (mean squared error), RMSE (root mean squared error), AIC (Akaike’s information criterion), R² (correlation index of the regression curve equation), Adj R² (adjected R²), SAE (sum of absolute error), PRR (predictive ratio risk), and PP (predictive power)

No.	Criteria	Formula
1	MSE	$\frac{{{\displaystyle \sum }}_{\mathrm{i}=0}^{\mathrm{n}}{\left(\widehat{\mathrm{m}}\left({\mathrm{t}}_{\mathrm{i}}\right)-{\mathrm{y}}_{\mathrm{i}}\right)}^2}{\mathrm{n}-\mathrm{m}}$
2	RMSE	$\sqrt{\frac{{{\displaystyle \sum }}_{\mathrm{i}=0}^{\mathrm{n}}{\left(\widehat{\mathrm{m}}\left({\mathrm{t}}_{\mathrm{i}}\right)-{\mathrm{y}}_{\mathrm{i}}\right)}^2}{\mathrm{n}-\mathrm{m}}}$
3	AIC [29]	$-2\log \mathrm{L}+2\mathrm{m}$
4	${\mathrm{R}}^2$ [8]	$1-\frac{{{\displaystyle \sum }}_{\mathrm{i}=0}^{\mathrm{n}}{\left(\widehat{\mathrm{m}}\left({\mathrm{t}}_{\mathrm{i}}\right)-{\mathrm{y}}_{\mathrm{i}}\right)}^2}{{{\displaystyle \sum }}_{\mathrm{i}=0}^{\mathrm{n}}{\left({\mathrm{y}}_{\mathrm{i}}-\overline{{\mathrm{y}}_{\mathrm{i}}}\right)}^2}$
5	Adj ${\mathrm{R}}^2$ [8]	$1-\frac{\left(1-{\mathrm{R}}^2\right)\left(\mathrm{n}-1\right)}{\mathrm{n}-\mathrm{m}-1}$
6	SAE [10]	${{\displaystyle \sum }}_{\mathrm{i}=0}^{\mathrm{n}}\left|\widehat{\mathrm{m}}\left({\mathrm{t}}_{\mathrm{i}}\right)-{\mathrm{y}}_{\mathrm{i}}\right|$
7	PRR [30]	${\displaystyle \sum _{\mathrm{i}=0}^{\mathrm{n}}}{\left(\frac{\widehat{\mathrm{m}}\left({\mathrm{t}}_{\mathrm{i}}\right)-{\mathrm{y}}_{\mathrm{i}}}{\widehat{\mathrm{m}}\left({\mathrm{t}}_{\mathrm{i}}\right)}\right)}^2$
8	PP [30]	${\displaystyle \sum _{\mathrm{i}=0}^{\mathrm{n}}}{\left(\frac{\widehat{\mathrm{m}}\left({\mathrm{t}}_{\mathrm{i}}\right)-{\mathrm{y}}_{\mathrm{i}}}{{\mathrm{y}}_{\mathrm{i}}}\right)}^2$

. For six of these criteria, i.e., MSE, RMSE, AIC, SAE, PRR, and PP, the smaller the value, the closer the model fits relative to other models. ${\mathrm{R}}^2$ and ${\mathrm{Adj}\mathrm{R}}^2$ values should be close to 1 for a good fit.

In Table 2, $\widehat{\mathrm{m}}\left({\mathrm{t}}_{\mathrm{i}}\right)$ is the estimated cumulative number of failures at ${\mathrm{t}}_{\mathrm{i}}$ for $\mathrm{i}=1,2,\cdots ,\mathrm{n}$; ${\mathrm{y}}_{\mathrm{i}}$ is the total number of failures observed at time ${\mathrm{t}}_{\mathrm{i}}$; n is the actual data, which include the total number of observations; and $\mathrm{m}$ is the number of unknown parameters in the model. We use the following Equation (9) to obtain the confidence interval of the NHPP SRM in Table 1.

$$\widehat{\mathrm{m}}\left(\mathrm{t}\right)\pm {\mathrm{Z}}_{\mathsf{\alpha }/2}\sqrt{\widehat{\mathrm{m}}\left(\mathrm{t}\right)},$$

(9)

where ${\mathrm{Z}}_{\mathsf{\alpha }/2}$ is the percentile of the standard normal distribution and $\alpha$ is a significant level [30].

4.2. Results

4.2.1. Model Analysis

Dataset #1, presented in

Table 3. Failure data of large medical record system (LMRS)—Dataset #1 and #2.

Week Index	Dataset #1		Dataset #2
	Failures	Cumulative Failures	Failures	Cumulative Failures
1	28	28	90	90
2	1	29	17	107
3	0	29	19	126
4	0	29	26	145
5	0	29	17	171
6	8	37	1	188
7	26	63	1	189
8	29	92	0	190
9	24	116	0	190
10	9	125	2	190
11	14	139	0	192
12	13	152	0	192
13	12	164	0	192
14	0	164	11	192
15	1	165	0	203
16	3	168	1	203
17	2	170	6	204
18	6	176

, was reported by [31]. The failure data come from two releases of a large medical record system (LMRS). The week index is from 1 week to 18 weeks, and there are 176 cumulative failures at 18 weeks in Dataset #1. In Dataset #2, the week index is from 1 week to 17 weeks, and there are 204 cumulative failures at 17 weeks. Detailed information can be seen in [31].

Dataset #3, given in

Table 4. System test data for a telecommunication system (TS)—Dataset #3.

Week Index	Exposure Time (Cumulative System Test Hours)	Failures	Cumulative Failures
1	416	3	3
2	832	1	4
3	1248	0	4
4	1664	3	7
5	2080	2	9
6	2496	0	9
7	2912	1	10
8	3328	3	13
9	3744	4	17
10	4160	2	19
11	4576	4	23
12	4992	2	25
13	5408	5	30
14	5824	2	32
15	6240	4	36
16	6656	1	37
17	7072	2	39
18	7488	0	39
19	7904	0	39
20	8320	3	42
21	8736	1	43

, was reported by [32] based on system test data for a telecommunication system (TS data set). In Dataset #3, the week index is from 1 week to 21 weeks, and there are 43 cumulative failures at 21 weeks. Detailed information can be seen in [32].

We obtained the estimated parameters and the eight common criteria in Table 2 of all 13 models at $t=1,2,3,\cdots ,18$ from Dataset #1, at $t=1,2,3,\cdots ,17$ from Dataset #2, and at $t=1,2,3,\cdots ,21$ from Dataset #3. We used the LSE method for parameter estimation.

Table 5. Model parameter estimation from datasets.

Model	Dataset #1	Dataset #2	Dataset #3
GO	$\widehat{\mathrm{a}}$ = 984.237, $\widehat{\mathrm{b}}$ = 0.0121	$\widehat{\mathrm{a}}$ = 197.387, $\widehat{\mathrm{b}}$ = 0.399	$\widehat{\mathrm{a}}$ = 1546.477, $\widehat{\mathrm{b}}$ = 0.00139
DS	$\widehat{\mathrm{a}}$ = 226.111, $\widehat{\mathrm{b}}$ = 0.1741	$\widehat{\mathrm{a}}$ = 192.528, $\widehat{\mathrm{b}}$ = 0.882	$\widehat{\mathrm{a}}$ = 62.326, $\widehat{\mathrm{b}}$ = 0.1185
IS	$\widehat{\mathrm{a}}$ = 176.517, $\widehat{\mathrm{b}}$ = 0.423, $\widehat{\mathsf{\beta }}$ = 26.888	$\widehat{\mathrm{a}}$ = 197.354, $\widehat{\mathrm{b}}$ = 0.399, $\widehat{\mathsf{\beta }}$ = 0.000001	$\widehat{\mathrm{a}}$ = 46.539, $\widehat{\mathrm{b}}$ = 0.241, $\widehat{\mathsf{\beta }}$ = 12.233
Y1	$\widehat{\mathrm{a}}$ = 991.78, $\widehat{\mathrm{b}}$ = 0.012, $\widehat{\mathsf{\alpha }}$ = 0.00	$\widehat{\mathrm{a}}$ = 182.550, $\widehat{\mathrm{b}}$ = 0.467, $\widehat{\mathsf{\alpha }}$ = 0.007	$\widehat{\mathrm{a}}$ = 239.618, $\widehat{\mathrm{b}}$ = 0.008, $\widehat{\mathsf{\alpha }}$ = 0.0197
Y2	$\widehat{\mathrm{a}}$ = 15.521, $\widehat{\mathrm{b}}$ = 0.605, $\widehat{\mathsf{\alpha }}$ = 0.722	$\widehat{\mathrm{a}}$ = 182.934, $\widehat{\mathrm{b}}$ = 0.464, $\widehat{\mathsf{\alpha }}$ = 0.0071	$\widehat{\mathrm{a}}$ = 174.77, $\widehat{\mathrm{b}}$ = 0.011, $\widehat{\mathsf{\alpha }}$ = 0.0219
PNZ	$\widehat{\mathrm{a}}$ = 176.517, $\widehat{\mathrm{b}}$ = 0.423, $\widehat{\mathsf{\alpha }}$ = 0.000, $\widehat{\mathsf{\beta }}$ = 26.888	$\widehat{\mathrm{a}}$ = 183.125, $\widehat{\mathrm{b}}$ = 0.463, $\widehat{\mathsf{\alpha }}$ = 0.007, $\widehat{\mathsf{\beta }}$ = 0.0001	$\widehat{\mathrm{a}}$ = 46.540, $\widehat{\mathrm{b}}$ = 0.241 $\widehat{\mathsf{\alpha }}$ = 0.0001, $\widehat{\mathsf{\beta }}$ = 12.233
PZ	$\widehat{\mathrm{a}}$ = 0.310, $\widehat{\mathrm{b}}$ = 0.423, $\widehat{\mathsf{\alpha }}$ = 0.000, $\widehat{\mathsf{\beta }}$ = 26.888, $\widehat{\mathrm{c}}$ = 176.940	$\widehat{\mathrm{a}}$ = 195.990, $\widehat{\mathrm{b}}$ = 0.3987, $\widehat{\mathsf{\alpha }}$ = 1000.0, $\widehat{\mathsf{\beta }}$ = 0.000, $\widehat{\mathrm{c}}$ = 1.390	$\widehat{\mathrm{a}}$ = 0.610, $\widehat{\mathrm{b}}$ = 0.241, $\widehat{\mathsf{\alpha }}$ = 1988.117, $\widehat{\mathsf{\beta }}$ = 12.233, $\widehat{\mathrm{c}}$ = 45.929
DP1	$\widehat{\mathsf{\alpha }}$ = 0.000001, $\widehat{\mathsf{\gamma }}$ = 858.504	$\widehat{\mathsf{\alpha }}$ = 0.000001, $\widehat{\mathsf{\gamma }}$ = 1031.599	$\widehat{\mathsf{\alpha }}$ = 0.000002, $\widehat{\mathsf{\gamma }}$ = 249.70
DP2	$\widehat{\mathsf{\alpha }}$ = 27641, $\widehat{\mathsf{\gamma }}$ = 0.006, ${\mathrm{t}}_0$ = 0.159, ${\mathrm{m}}_0$ = 43.0	$\widehat{\mathsf{\alpha }}$ = 26,124.0, $\widehat{\mathsf{\gamma }}$ = 0.0044, ${\mathrm{t}}_0$ = 0.000, ${\mathrm{m}}_0$ = 147.0	$\widehat{\mathsf{\alpha }}$ = 564.21, $\widehat{\mathsf{\beta }}$ = 0.017 ${\widehat{\mathrm{t}}}_0$ $=3.00,$ ${\widehat{\mathrm{m}}}_0$ = 9.0
TC	$\widehat{\mathrm{a}}$ = 0.05, $\widehat{\mathrm{b}}$ = 2.500, $\widehat{\mathsf{\alpha }}$ = 101.0, $\widehat{\mathsf{\beta }}$ = 14.00, $\widehat{\mathrm{N}}$ = 174.0	$\widehat{\mathrm{a}}$ = 0.053., $\widehat{\mathrm{b}}$ = 0.774, $\widehat{\mathsf{\alpha }}$ = 181.0, $\widehat{\mathsf{\beta }}$ = 38.60, $\widehat{\mathrm{N}}$ = 204.140	$\widehat{\mathrm{a}}$ $=0.02,$ $\widehat{\mathrm{b}}$ = 1.863, $\widehat{\mathsf{\alpha }}$ = 9474.019, $\widehat{\mathsf{\beta }}$ $=866.248,$ $\widehat{\mathrm{N}}$ = 48.983
3PD	$\widehat{\mathrm{a}}$ = 1.430, $\widehat{\mathrm{b}}$ = 0.42, $\widehat{\mathrm{c}}$ = 1042.4, $\widehat{\mathsf{\beta }}$ = 0.09, $\widehat{\mathrm{N}}$ = 178.9	$\widehat{\mathrm{a}}$ = 0.028, $\widehat{\mathrm{b}}$ = 0.21, $\widehat{\mathrm{c}}$ = 9.924, $\widehat{\mathsf{\beta }}$ = 0.005, $\widehat{\mathrm{N}}$ = 206.387	$\widehat{\mathrm{a}}$ $=3.078,$ $\widehat{\mathrm{b}}$ $=0.241,$ $\widehat{\mathrm{c}}$ = 999.493, $\widehat{\mathsf{\beta }}$ = 0.17, $\widehat{\mathrm{N}}$ = 46.843
NW	$\widehat{\mathrm{a}}$ = 0.1363, $\widehat{\mathrm{b}}$ = 10.245, $\widehat{\mathsf{\alpha }}$ = 0.1622, $\widehat{\mathsf{\beta }}$ = 151.41, $\widehat{\mathrm{N}}$ = 171.064	$\widehat{\mathrm{a}}$ = 0.18, $\widehat{\mathrm{b}}$ = 5.714, $\widehat{\mathsf{\alpha }}$ = 0.08, $\widehat{\mathsf{\beta }}$ = 3.77, $\widehat{\mathrm{N}}$ = 196.852	$\widehat{\mathrm{a}}$ = 0.071, $\widehat{\mathrm{b}}$ = 13.102, $\widehat{\mathsf{\alpha }}$ = 0.109, $\widehat{\mathsf{\beta }}$ = 10.203, $\widehat{\mathrm{N}}$ = 41.717
New	$\widehat{\mathrm{a}}$ = 0.488, $\widehat{\mathrm{b}}$ = 0.892, $\widehat{\mathsf{\alpha }}$ = 0.328, $\widehat{\mathsf{\beta }}$ = 0.801, $\widehat{\mathsf{\gamma }}$ = 4644.6, $\widehat{\mathrm{p}}$ = 0.942, $\widehat{\mathrm{N}}$ = 184.23	$\widehat{\mathrm{a}}$ = 0.076, $\widehat{\mathrm{b}}$ = 2.728, $\widehat{\mathsf{\alpha }}$ = 0.082, $\widehat{\mathsf{\beta }}$ = 0.249, $\widehat{\mathsf{\gamma }}$ = 20,125.0, $\widehat{\mathrm{p}}$ = 0.773, $\widehat{\mathrm{N}}$ = 203.545	$\widehat{\mathrm{a}}$ = 0.93, $\widehat{\mathrm{b}}$ = 0.78, $\widehat{\mathsf{\alpha }}$ = 0.44, $\widehat{\mathsf{\beta }}$ = 1.95, $\widehat{\mathsf{\gamma }}$ = 400.001, $\widehat{\mathrm{p}}$ = 0.590, $\widehat{\mathrm{N}}$ = 50.001

shows the estimated parameters and

Table 6. Comparison criteria from LMRS dataset (Dataset #1).

Model	MSE	RMSE	AIC	R2	Adj R2	SAE	PRR	PP
GO	299.3292	17.3011	261.4487	0.9247	0.9146	254.5092	2.7083	2.6703
DS	202.8454	14.2424	286.6365	0.9490	0.9421	204.5240	70.0366	2.3678
IS	116.2935	10.7839	237.2497	0.9726	0.9667	123.8812	65.0591	1.8223
Y1	319.2849	17.8685	263.4846	0.9247	0.9085	254.5864	2.7120	2.6668
Y2	326.5601	18.0710	271.4442	0.9230	0.9065	264.7073	4.2598	2.1294
PNZ	124.6002	11.1624	239.2497	0.9726	0.9641	123.8812	65.0591	1.8223
PZ	134.2981	11.5887	241.2417	0.9725	0.9611	124.7652	64.6969	1.8236
DP1	1482.3642	38.5015	451.7756	0.6270	0.5772	575.2253	1463.9339	4.2365
DP2	773.2907	27.8081	281.3004	0.8297	0.7773	403.4070	1.5855	3.2787
TC	171.2268	13.0854	330.4311	0.9650	0.9504	136.5833	1563.7908	2.4065
3PD	134.3453	11.5907	240.2708	0.9725	0.9611	124.2817	62.8017	1.8147
NW	154.1849	12.4171	275.4208	0.9709	0.9588	128.3625	89.5199	1.8579
New	93.2910	9.6587	182.2178	0.9839	0.9726	99.6157	3.1124	0.9340

,

Table 7. Comparison criteria from LMRS dataset (Dataset #2).

Model	MSE	RMSE	AIC	R2	Adj R2	SAE	PRR	PP
GO	80.6779	8.9821	184.3314	0.9388	0.9300	104.4025	0.1705	0.1013
DS	232.6282	15.2522	331.8567	0.8234	0.7982	142.5442	1.2915	0.3330
IS	86.4395	9.2973	186.3337	0.9388	0.9246	104.3703	0.1706	0.1013
Y1	78.8663	8.8807	157.4475	0.9441	0.9312	100.4119	0.1256	0.0860
Y2	78.8367	8.8790	157.8252	0.9442	0.9313	100.6173	0.1276	0.0866
PNZ	84.9016	9.2142	159.8729	0.9442	0.9255	100.6068	0.1281	0.0867
PZ	100.9894	10.0493	190.3321	0.9387	0.9108	104.3539	0.1719	0.1017
DP1	11,053.0563	105.1335	1208.2816	−7.3888	−8.5872	1516.4369	7828.3035	6.8736
DP2	769.2822	27.7359	480.3412	0.4940	0.3253	334.1283	0.4149	0.7120
TC	72.2825	8.5019	158.9330	0.9561	0.9362	103.1963	0.0521	0.0479
3PD	81.0903	9.0050	163.7968	0.9508	0.9284	106.3409	0.0729	0.0615
NW	46.7395	6.8366	182.3811	0.9759	0.9650	65.5785	0.0293	0.0271
New	25.2102	5.0210	131.1529	0.9872	0.9773	56.3823	0.0073	0.0072

and

Table 8. Comparison criteria from TS dataset (Dataset #3).

Model	MSE	RMSE	AIC	R2	Adj R2	SAE	PRR	PP
GO	6.7541	2.5989	78.3086	0.9714	0.9682	43.8536	0.6938	1.1445
DS	3.2731	1.8092	81.0881	0.9862	0.9846	32.5598	44.3248	1.4297
IS	1.8704	1.3676	76.9490	0.9925	0.9912	21.9615	5.9598	0.8969
Y1	6.2849	2.5070	81.0083	0.9748	0.9704	41.0689	0.6806	0.7548
Y2	31.0054	5.5683	81.1537	0.8757	0.8538	85.5914	1.5928	0.8203
PNZ	1.9834	1.4083	78.9480	0.9925	0.9906	21.9572	5.9580	0.8970
PZ	2.1042	1.4506	80.9490	0.9925	0.9900	21.9616	5.9599	0.8969
DP1	43.6887	6.6097	104.9786	0.8152	0.7946	121.7027	600.8316	4.5285
DP2	22.9775	4.7935	98.8587	0.9130	0.8913	79.7928	1.5190	6.5936
TC	3.3002	1.8167	86.4351	0.9882	0.9843	28.8649	56.9751	1.5764
3PD	2.1046	1.4507	80.9477	0.9925	0.9900	21.9661	5.9567	0.8967
NW	2.0555	1.4337	86.1131	0.9927	0.9902	21.9748	8.2751	0.9430
New	1.1626	1.0783	81.4086	0.9964	0.9944	15.7154	0.2447	0.1851

show the values of the eight common criteria for all 13 models. The closer the values of MSE, RMSE, AIC, SAE, PRR, and PP are to 0 and the closer the values of ${\mathrm{R}}^2$ and ${\mathrm{Adj}\mathrm{R}}^2$ are to 1, the better. As a result, Table 6 shows that for the proposed model, the values of MSE, RMSE, AIC, SAE, and PP are 93.2910, 9.6587, 182.2178, 99.6157, and 0.9340, respectively, which are lower than those of the other models. In addition, the values of ${\mathrm{R}}^2$ and ${\mathrm{Adj}\mathrm{R}}^2$ are 0.9836 and 0.9726, respectively, which are higher than those of the other models. In Table 7, the proposed model for the values of MSE, RMSE, AIC, SAE, PRR and PP are 25.2102, 5.0210, 131.1529, 56.3823, 0.0073, and 0.0072, respectively, which are lower than those of the other models and, the values of ${\mathrm{R}}^2$ and ${\mathrm{Adj}\mathrm{R}}^2$ are 0.9872 and 0.9773, respectively, which are higher than those of the other models. In Table 8, the proposed model for the values of MSE, RMSE, SAE, PRR, and PP are 1.1626, 1.0783, 15.7154, 0.2447, and 0.1851, respectively, which are lower than those of the other models, and the values of ${\mathrm{R}}^2$ and ${\mathrm{Adj}\mathrm{R}}^2$ are 0.9964 and 0.9944, respectively, which are higher than those of the other models. As can be seen from the results, the proposed model is the most suitable when comparing the common criteria with the other models.

Figure 2, Figure 3 and Figure 4 show graphs of the mean value functions for all 13 models for Dataset #1, #2 and #3, respectively. Figure 5, Figure 6 and Figure 7 show graphs of the 95% and 99% confidence limits of the proposed model for Dataset #1, #2 and #3. Figure 8, Figure 9 and Figure 10 show graphs of the relative error of all 13 models for Dataset #1, #2, and #3, respectively, and, the proposed model provides a more accurate prediction because the value of relative error is closer to zero. Figure 11, Figure 12 and Figure 13 compare RMSE and AIC of all 13 models for Dataset #1, #2, and #3, showing that the values of the proposed model are close to zero. In addition,

Table A1. 99% Confidence interval of all 13 NHPP SRMs from Dataset #1.

Model	Time Index	1	2	3	4	5	6	7	8	9
GO	LCL	2.9752	11.0372	19.8294	28.9373	38.2010	47.5388	56.9029	66.2622	75.5959
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	11.8375	23.5326	35.0871	46.5026	57.7808	68.9234	79.9320	90.8081	101.5535
	UCL	20.6998	36.0281	50.3449	64.0679	77.3607	90.3080	102.9611	115.3541	127.5111
DS	LCL	−1.4475	2.3988	9.8762	19.7193	30.9757	42.9376	55.0882	67.0585	78.5932
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	3.0539	10.9049	21.9419	34.9465	49.0079	63.4565	77.8095	91.7285	104.9858
	UCL	7.5552	19.4109	34.0077	50.1736	67.0402	83.9755	100.5308	116.3985	131.3784
IS	LCL	−1.3876	0.7345	4.9084	11.5269	20.9997	33.4459	48.4297	64.8712	81.2785
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	3.2709	8.0368	14.8268	24.1977	36.5783	52.0250	69.9771	89.1986	108.0540
	UCL	7.9295	15.3391	24.7452	36.8685	52.1569	70.6040	91.5245	113.5260	134.8295
Y1	LCL	2.9706	11.0274	19.8152	28.9198	38.1810	47.5174	56.8810	66.2410	75.5762
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	11.8302	23.5194	35.0690	46.4810	57.7568	68.8981	79.9065	90.7836	101.5309
	UCL	20.6898	36.0113	50.3229	64.0422	77.3325	90.2787	102.9319	115.3262	127.4856
Y2	LCL	1.7621	8.6986	16.7396	25.3788	34.3917	43.6577	53.1057	62.6911	72.3848
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	9.8437	20.3058	31.1057	42.0900	53.1750	64.3150	75.4850	86.6714	97.8668
	UCL	17.9252	31.9130	45.4717	58.8011	71.9582	84.9723	97.8644	110.6518	123.3489
PNZ	LCL	−1.3876	0.7345	4.9084	11.5269	20.9997	33.4459	48.4297	64.8712	81.2785
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	3.2709	8.0368	14.8268	24.1977	36.5783	52.0250	69.9771	89.1986	108.0540
	UCL	7.9295	15.3391	24.7452	36.8685	52.1569	70.6040	91.5245	113.5260	134.8295
PZ	LCL	−1.3854	0.7450	4.9321	11.5697	21.0687	33.5484	48.5716	65.0558	81.5054
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	3.2788	8.0561	14.8623	24.2557	36.6659	52.1496	70.1448	89.4123	108.3130
	UCL	7.9429	15.3671	24.7926	36.9416	52.2632	70.7509	91.7180	113.7689	135.1205
DP1	LCL	−1.4743	−1.4746	−0.0008	2.9470	7.3689	13.2649	20.6349	29.4790	39.7971
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	0.7370	2.9481	6.6333	11.7925	18.4257	26.5330	36.1144	47.1699	59.6994
	UCL	2.9484	7.3708	13.2673	20.6379	29.4825	39.8012	51.5939	64.8607	79.6016
	LCL	26.5001	27.7130	29.7569	32.6558	36.4370	41.1292	46.7608	53.3593	60.9505
DP2	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	43.4862	44.9904	47.5118	51.0621	55.6529	61.2957	68.0019	75.7827	84.6494
	UCL	60.4722	62.2677	65.2666	69.4684	74.8688	81.4623	89.2430	98.2062	108.3484
TC	LCL	−1.4553	−1.1784	2.2130	9.2349	19.8279	33.4224	49.0474	65.5033	81.5698
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	0.7003	3.9242	10.5989	21.0540	35.0853	51.9963	70.7069	89.9303	108.3864
	UCL	2.8559	9.0268	18.9847	32.8732	50.3427	70.5702	92.3664	114.3573	135.2031
3PD	LCL	−1.3728	0.7935	5.0173	11.6787	21.1710	33.5985	48.5203	64.8703	81.1846
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	3.3220	8.1446	14.9902	24.4031	36.7959	52.2106	70.0842	89.1976	107.9468
	UCL	8.0167	15.4957	24.9631	37.1276	52.4208	70.8228	91.6481	113.5249	134.7090
NW	LCL	−1.5187	1.1251	6.4796	14.1086	23.7628	35.2697	48.4822	63.2088	79.0574
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	2.7625	8.7404	17.1454	27.6542	40.0675	54.2401	70.0392	87.2721	105.5167
	UCL	7.0437	16.3556	27.8111	41.1998	56.3722	73.2106	91.5961	111.3353	131.9759
New	LCL	2.5514	6.0537	10.3882	16.0978	23.7208	33.8252	46.8738	62.8212	80.4445
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	11.1541	16.5245	22.6460	30.2694	40.0147	52.4864	68.1359	86.8224	107.1017
	UCL	19.7568	26.9953	34.9038	44.4409	56.3087	71.1476	89.3979	110.8236	133.7590
Model	Time Index	10	11	12	13	14	15	16	17	18
GO	LCL	84.8889	94.1304	103.3121	112.4278	121.4725	130.4426	139.3350	148.1476	156.8786
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	112.1696	122.6580	133.0203	143.2580	153.3725	163.3654	173.2381	182.9920	192.6287
	UCL	139.4502	151.1856	162.7285	174.0882	185.2725	196.2882	207.1411	217.8365	228.3788
DS	LCL	89.5235	99.7463	109.2071	117.8870	125.7931	132.9502	139.3951	145.1722	150.3299
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	117.4374	129.0024	139.6461	149.3678	158.1903	166.1527	173.3047	179.7020	185.4031
	UCL	145.3513	158.2584	170.0852	180.8486	190.5874	199.3552	207.2143	214.2317	220.4763
IS	LCL	96.2253	108.7771	118.6274	125.9585	131.2036	134.8518	137.3398	139.0140	140.1303
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	125.0271	139.1636	150.1952	158.3745	164.2116	168.2647	171.0257	172.8822	174.1195
	UCL	153.8288	169.5500	181.7631	190.7905	197.2196	201.6776	204.7116	206.7504	208.1087
Y1	LCL	84.8718	94.1167	103.3028	112.4238	121.4747	130.4518	139.3522	148.1736	156.9141
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	112.1500	122.6425	133.0098	143.2535	153.3750	163.3757	173.2572	183.0207	192.6678
	UCL	139.4283	151.1683	162.7169	174.0832	185.2752	196.2996	207.1621	217.8679	228.4216
Y2	LCL	82.1664	92.0215	101.9394	111.9118	121.9320	131.9949	142.0961	152.2320	162.3994
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	109.0671	120.2701	131.4745	142.6797	153.8853	165.0912	176.2972	187.5033	198.7094
	UCL	135.9678	148.5186	161.0095	173.4476	185.8386	198.1875	210.4983	222.7746	235.0194
PNZ	LCL	96.2253	108.7771	118.6274	125.9585	131.2036	134.8518	137.3398	139.0140	140.1303
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	125.0271	139.1636	150.1952	158.3745	164.2116	168.2647	171.0257	172.8822	174.1195
	UCL	153.8288	169.5500	181.7631	190.7905	197.2196	201.6776	204.7116	206.7504	208.1087
PZ	LCL	96.4904	109.0742	118.9495	126.2992	131.5576	135.2151	137.7094	139.3877	140.5069
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	125.3267	139.4970	150.5551	158.7540	164.6051	168.6679	171.4356	173.2965	174.5367
	UCL	154.1629	169.9199	182.1608	191.2088	197.6527	202.1208	205.1618	207.2052	208.5666
DP1	LCL	51.5893	64.8556	79.5959	95.8102	113.4987	132.6612	153.2977	175.4083	198.9930
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	73.7029	89.1805	106.1322	124.5579	144.4577	165.8316	188.6795	213.0014	238.7974
	UCL	95.8165	113.5055	132.6685	153.3056	175.4167	199.0019	224.0612	250.5945	278.6019
	LCL	69.5581	79.2041	89.9088	101.6907	114.5672	128.5544	143.6675	159.9207	177.3277
DP2	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	94.6130	105.6844	117.8746	131.1943	145.6542	161.2649	178.0369	195.9805	215.1061
	UCL	119.6678	132.1647	145.8404	160.6978	176.7412	193.9754	212.4063	232.0403	252.8845
TC	LCL	96.2056	108.6911	118.6868	126.2064	131.5257	135.0653	137.2814	138.5872	139.3112
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	125.0048	139.0670	150.2617	158.6507	164.5696	168.5016	170.9609	172.4090	173.2117
	UCL	153.8040	169.4429	181.8365	191.0950	197.6135	201.9380	204.6404	206.2308	207.1121
3PD	LCL	96.0649	108.5914	118.4569	125.8343	131.1467	134.8761	137.4559	139.2308	140.4556
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	124.8457	138.9551	150.0047	158.2361	164.1483	168.2916	171.1544	173.1226	174.4800
	UCL	153.6266	169.3188	181.5526	190.6380	197.1499	201.7072	204.8530	207.0143	208.5043
NW	LCL	95.1400	109.8266	121.2536	128.6725	132.8455	135.0189	136.1257	136.6943	136.9930
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	123.8001	140.3413	153.1282	161.3963	166.0364	168.4501	169.6787	170.3096	170.6410
	UCL	152.4602	170.8561	185.0027	194.1201	199.2273	201.8814	203.2316	203.9248	204.2890
New	LCL	97.1777	110.5793	120.0313	126.3680	130.6527	133.6536	135.8425	137.4998	138.7947
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	126.1032	141.1857	151.7636	158.8307	163.5991	166.9341	169.3644	171.2031	172.6390
	UCL	155.0286	171.7921	183.4958	191.2934	196.5454	200.2146	202.8863	204.9064	206.4834

,

Table A2. 99% Confidence interval of all 13 NHPP SRMs from Dataset #2.

Model	Time Index	1	2	3	4	5	6	7	8	9
GO	LCL	44.1844	81.6849	107.5241	125.0619	136.9017	144.8746	150.2361	153.8388	156.2583
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	64.9422	108.5177	137.7566	157.3755	170.5397	179.3727	185.2996	189.2764	191.9449
	UCL	85.6999	135.3506	167.9890	189.6892	204.1776	213.8708	220.3630	224.7141	227.6315
DS	LCL	25.7372	75.4099	111.9612	133.6497	145.3218	151.2622	154.1784	155.5743	156.2301
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	42.5368	101.3403	142.7351	166.9298	179.8675	186.4327	189.6511	191.1907	191.9138
	UCL	59.3364	127.2706	173.5090	200.2099	214.4131	221.6031	225.1239	226.8071	227.5975
IS	LCL	44.1752	81.6689	107.5036	125.0383	136.8760	144.8475	150.2081	153.8101	156.2292
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	64.9313	108.4996	137.7335	157.3492	170.5111	179.3427	185.2686	189.2448	191.9128
	UCL	85.6873	135.3302	167.9634	189.6601	204.1463	213.8379	220.3291	224.6795	227.5964
Y1	LCL	47.0719	84.4861	108.9555	124.8918	135.3779	142.4181	147.2900	150.8015	153.4618
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	68.3705	111.7110	139.3638	157.1859	168.8487	176.6538	182.0440	185.9239	188.8605
	UCL	89.6692	138.9358	169.7721	189.4801	202.3195	210.8894	216.7980	221.0463	224.2593
Y2	LCL	46.9038	84.3161	108.8575	124.8826	135.4480	142.5476	147.4564	150.9836	153.6403
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	68.1714	111.5173	139.2538	157.1757	168.9265	176.7971	182.2280	186.1250	189.0574
	UCL	89.4390	138.7185	169.6501	189.4688	202.4050	211.0466	216.9996	221.2664	224.4746
PNZ	LCL	46.8632	84.2786	108.8405	124.8881	135.4715	142.5829	147.4972	151.0244	153.6770
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	68.1233	111.4746	139.2347	157.1818	168.9526	176.8362	182.2731	186.1701	189.0980
	UCL	89.3834	138.6706	169.6289	189.4755	202.4337	211.0895	217.0490	221.3158	224.5189
PZ	LCL	44.1050	81.6039	107.4509	124.9995	136.8503	144.8330	150.2028	153.8121	156.2368
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	64.8476	108.4253	137.6743	157.3060	170.4827	179.3267	185.2628	189.2470	191.9212
	UCL	85.5903	135.2468	167.8978	189.6125	204.1150	213.8204	220.3227	224.6820	227.6056
DP1	LCL	−1.5930	−1.0577	1.6061	6.3983	13.3188	22.3677	33.5451	46.8508	62.2849
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	1.0642	4.2568	9.5778	17.0271	26.6049	38.3111	52.1456	68.1086	86.1999
	UCL	3.7214	9.5712	17.5494	27.6560	39.8910	54.2544	70.7462	89.3664	110.1149
	LCL	115.9952	116.6742	117.8113	119.4110	121.4782	124.0181	127.0358	130.5368	134.5268
DP2	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	147.2522	148.0118	149.2832	151.0707	153.3789	156.2120	159.5743	163.4702	167.9039
	UCL	178.5093	179.3493	180.7550	182.7305	185.2796	188.4060	192.1129	196.4036	201.2809
TC	LCL	55.3076	86.9606	107.6686	122.0984	132.5186	140.2258	146.0275	150.4553	153.8723
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	78.0664	114.5264	137.9188	154.0710	165.6732	174.2253	180.6481	185.5416	189.3135
	UCL	100.8252	142.0921	168.1690	186.0435	198.8277	208.2248	215.2686	220.6279	224.7546
3PD	LCL	52.2342	86.8857	108.5907	123.1337	133.3574	140.7969	146.3505	150.5778	153.8456
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	74.4613	114.4412	138.9543	155.2258	166.6051	174.8581	181.0052	185.6769	189.2840
	UCL	96.6883	141.9967	169.3178	187.3180	199.8527	208.9193	215.6600	220.7760	224.7224
NW	LCL	57.6766	83.8305	103.7251	120.1147	133.6045	143.9716	151.0377	155.2967	157.6696
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	80.8355	110.9641	133.4851	151.8567	166.8795	178.3735	186.1848	190.8846	193.5005
	UCL	103.9944	138.0978	163.2452	183.5986	200.1545	212.7754	221.3319	226.4725	229.3315
New	LCL	64.4950	79.9166	97.6041	117.6815	135.8098	146.3428	151.6415	154.6909	156.6830
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	88.7630	106.4987	126.5848	149.1380	169.3281	180.9967	186.8515	190.2164	192.4131
	UCL	113.0309	133.0808	155.5654	180.5946	202.8464	215.6506	222.0614	225.7420	228.1432
Model	Time Index	10	11	12	13	14	15	16	17
GO	LCL	157.8827	158.9731	159.7050	160.1961	160.5257	160.7469	160.8953	160.9949
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	193.7354	194.9368	195.7429	196.2839	196.6468	196.8903	197.0537	197.1634
	UCL	229.5881	230.9005	231.7809	232.3716	232.7679	233.0338	233.2122	233.3319
DS	LCL	156.5338	156.6729	156.7360	156.7644	156.7771	156.7828	156.7853	156.7864
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	192.2487	192.4020	192.4715	192.5028	192.5169	192.5231	192.5258	192.5271
	UCL	227.9635	228.1311	228.2071	228.2413	228.2566	228.2634	228.2664	228.2677
IS	LCL	157.8533	158.9436	159.6753	160.1663	160.4959	160.7170	160.8654	160.9650
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	193.7030	194.9042	195.7102	196.2510	196.6139	196.8574	197.0208	197.1304
	UCL	229.5527	230.8649	231.7452	232.3357	232.7320	232.9978	233.1762	233.2959
Y1	LCL	155.5912	157.3908	158.9870	160.4591	161.8567	163.2112	164.5421	165.8618
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	191.2094	193.1933	194.9521	196.5734	198.1121	199.6027	201.0668	202.5181
	UCL	226.8275	228.9958	230.9172	232.6877	234.3675	235.9942	237.5916	239.1745
Y2	LCL	155.7490	157.5131	159.0605	160.4716	161.7971	163.0690	164.3073	165.5246
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	191.3834	193.3281	195.0330	196.5872	198.0465	199.4463	200.8086	202.1473
	UCL	227.0178	229.1431	231.0055	232.7027	234.2959	235.8236	237.3099	238.7701
PNZ	LCL	155.7784	157.5327	159.0686	160.4670	161.7790	163.0367	164.2604	165.4628
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	191.4158	193.3497	195.0419	196.5821	198.0265	199.4107	200.7570	202.0794
	UCL	227.0532	229.1667	231.0153	232.6972	234.2741	235.7848	237.2536	238.6960
PZ	LCL	157.8652	158.9586	159.6927	160.1855	160.5163	160.7384	160.8874	160.9874
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	193.7161	194.9208	195.7294	196.2721	196.6364	196.8809	197.0450	197.1552
	UCL	229.5670	230.8830	231.7661	232.3588	232.7565	233.0235	233.2026	233.3229
DP1	LCL	79.8474	99.5383	121.3576	145.3053	171.3814	199.5859	229.9187	262.3800
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	106.4196	128.7678	153.2443	179.8492	208.5825	239.4442	272.4343	307.5528
	UCL	132.9919	157.9972	185.1310	214.3931	245.7836	279.3026	314.9499	352.7256
	LCL	139.0116	143.9968	149.4885	155.4925	162.0147	169.0611	176.6375	184.7497
DP2	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	172.8795	178.4014	184.4737	191.1005	198.2860	206.0343	214.3493	223.2353
	UCL	206.7475	212.8060	219.4589	226.7086	234.5573	243.0075	252.0612	261.7209
TC	LCL	156.5339	158.6235	160.2755	161.5893	162.6400	163.4841	164.1655	164.7175
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	192.2487	194.5517	196.3712	197.8178	198.9741	199.9030	200.6526	201.2598
	UCL	227.9636	230.4798	232.4670	234.0462	235.3083	236.3219	237.1397	237.8021
3PD	LCL	156.4035	158.4271	160.0431	161.3445	162.4010	163.2653	163.9776	164.5692
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	192.1050	194.3353	196.1153	197.5483	198.7112	199.6622	200.4460	201.0966
	UCL	227.8065	230.2434	232.1875	233.7520	235.0213	236.0592	236.9143	237.6241
NW	LCL	158.9588	159.6668	160.0664	160.2998	160.4407	160.5285	160.5848	160.6220
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	194.9210	195.7008	196.1410	196.3980	196.5531	196.6498	196.7119	196.7528
	UCL	230.8833	231.7349	232.2156	232.4962	232.6656	232.7712	232.8389	232.8836
New	LCL	158.0961	159.1551	159.9802	160.6421	161.1855	161.6398	162.0255	162.3571
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	193.9706	195.1372	196.0460	196.7749	197.3732	197.8733	198.2978	198.6628
	UCL	229.8450	231.1194	232.1118	232.9078	233.5609	234.1069	234.5702	234.9686

and

Table A3. 99% Confidence interval of all 13 SRM models from Dataset #3.

Model	Time Index	1	2	3	4	5	6	7
GO	LCL	−1.6271	−1.0439	−0.0990	1.0319	2.2808	3.6126	5.0067
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	2.1481	4.2932	6.4354	8.5746	10.7108	12.8440	14.9743
	UCL	5.9234	9.6304	12.9698	16.1172	19.1407	22.0754	24.9418
DS	LCL	−1.2338	−1.6546	−1.4302	−0.7010	0.4162	1.8246	3.4442
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	0.4045	1.4969	3.1185	5.1372	7.4441	9.9496	12.5804
	UCL	2.0428	4.6484	7.6671	10.9754	14.4719	18.0745	21.7166
IS	LCL	−1.5571	−1.6348	−1.3334	−0.7248	0.1772	1.3684	2.8391
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	0.9391	2.0807	3.4533	5.0820	6.9847	9.1674	11.6194
	UCL	3.4352	5.7962	8.2399	10.8888	13.7923	16.9664	20.3997
Y1	LCL	−1.6486	−1.1940	−0.3781	0.6349	1.7818	3.0308	4.3631
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	1.9283	3.8795	5.8543	7.8533	9.8771	11.9263	14.0014
	UCL	5.5051	8.9530	12.0867	15.0718	17.9724	20.8217	23.6398
Y2	LCL	−1.6586	−1.3833	−0.7786	0.0033	0.9039	1.8930	2.9526
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	1.6343	3.2861	4.9553	6.6416	8.3448	10.0648	11.8014
	UCL	4.9272	7.9555	10.6892	13.2798	15.7857	18.2367	20.6502
PNZ	LCL	−1.5571	−1.6348	−1.3332	−0.7242	0.1783	1.3704	2.8425
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	0.9392	2.0809	3.4539	5.0833	6.9870	9.1710	11.6248
	UCL	3.4354	5.7967	8.2410	10.8908	13.7956	16.9715	20.4072
PZ	LCL	−1.5571	−1.6348	−1.3334	−0.7248	0.1772	1.3684	2.8391
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	0.9391	2.0807	3.4533	5.0820	6.9847	9.1674	11.6194
	UCL	3.4352	5.7962	8.2399	10.8888	13.7923	16.9664	20.3997
DP1	LCL	−0.7849	−1.3204	−1.6065	−1.6432	−1.4305	−0.9684	−0.2569
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	0.1247	0.4988	1.1223	1.9952	3.1175	4.4892	6.1103
	UCL	1.0343	2.3180	3.8511	5.6336	7.6655	9.9468	12.4775
	LCL	0.8976	1.0355	1.2725	1.6168	2.0776	2.6654	3.3907
DP2	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	8.3334	8.5809	9.0000	9.5960	10.3741	11.3392	12.4963
	UCL	15.7692	16.1263	16.7275	17.5753	18.6705	20.0130	21.6018
TC	LCL	−1.1910	−1.6387	−1.5204	−0.9189	0.0932	1.4453	3.0679
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	0.3649	1.3143	2.7550	4.6141	6.8200	9.3009	11.9855
	UCL	1.9208	4.2672	7.0304	10.1471	13.5468	17.1565	20.9030
3PD	LCL	−1.5571	−1.6348	−1.3332	−0.7245	0.1776	1.3690	2.8398
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	0.9392	2.0810	3.4538	5.0827	6.9856	9.1684	11.6206
	UCL	3.4356	5.7968	8.2408	10.8899	13.7936	16.9679	20.4013
NW	LCL	−1.5127	−1.6433	−1.3007	−0.6009	0.3823	1.6042	3.0345
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	0.8205	1.9940	3.5580	5.3658	7.3796	9.5745	11.9322
	UCL	3.1537	5.6314	8.4167	11.3325	14.3770	17.5448	20.8300
New	LCL	−1.6104	−1.3350	−0.9086	−0.3265	0.4400	1.4274	2.6738
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	2.2732	3.4481	4.6398	5.9641	7.4891	9.2699	11.3527
	UCL	6.1567	8.2312	10.1882	12.2546	14.5381	17.1125	20.0317
Model	Time Index	8	9	10	11	12	13	14
GO	LCL	6.4495	7.9316	9.4462	10.9881	12.5533	14.1386	15.7414
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	17.1016	19.2259	21.3473	23.4658	25.5813	27.6939	29.8035
	UCL	27.7537	30.5203	33.2485	35.9435	38.6093	41.2492	43.8656
DS	LCL	5.2094	7.0666	8.9726	10.8928	12.7995	14.6715	16.4922
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	15.2774	17.9927	20.6888	23.3359	25.9113	28.3980	30.7837
	UCL	25.3453	28.9189	32.4049	35.7790	39.0231	42.1246	45.0752
IS	LCL	4.5662	6.5093	8.6112	10.8022	13.0068	15.1524	17.1769
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	14.3103	17.1884	20.1834	23.2124	26.1886	29.0312	31.6735
	UCL	24.0544	27.8675	31.7555	35.6225	39.3704	42.9099	46.1700
Y1	LCL	5.7667	7.2337	8.7583	10.3364	11.9650	13.6419	15.3652
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	16.1032	18.2323	20.3893	22.5750	24.7900	27.0349	29.3106
	UCL	26.4397	29.2309	32.0204	34.8136	37.6149	40.4280	43.2559
Y2	LCL	4.0712	5.2405	6.4546	7.7089	9.0000	10.3250	11.6816
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	13.5545	15.3237	17.1090	18.9101	20.7270	22.5594	24.4071
	UCL	23.0377	25.4069	27.7634	30.1113	32.4539	34.7937	37.1326
PNZ	LCL	4.5713	6.5167	8.6214	10.8157	13.0240	15.1738	17.2027
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	14.3180	17.1991	20.1976	23.2307	26.2116	29.0592	31.7069
	UCL	24.0648	27.8815	31.7739	35.6458	39.3991	42.9446	46.2111
PZ	LCL	4.5662	6.5093	8.6112	10.8022	13.0068	15.1524	17.1769
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	14.3103	17.1884	20.1834	23.2124	26.1886	29.0312	31.6735
	UCL	24.0543	27.8675	31.7555	35.6225	39.3704	42.9099	46.1700
DP1	LCL	0.7040	1.9143	3.3740	5.0831	7.0416	9.2495	11.7068
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	7.9808	10.1007	12.4700	15.0887	17.9568	21.0743	24.4412
	UCL	15.2576	18.2871	21.5660	25.0943	28.8720	32.8991	37.1756
	LCL	4.2639	5.2952	6.4939	7.8690	9.4287	11.1804	13.1313
DP2	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	13.8500	15.4051	17.1661	19.1373	21.3231	23.7275	26.3548
	UCL	23.4361	25.5151	27.8383	30.4056	33.2174	36.2746	39.5783
TC	LCL	4.8937	6.8584	8.9028	10.9735	13.0240	15.0150	16.9149
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	14.8047	17.6932	20.5914	23.4460	26.2115	28.8504	31.3334
	UCL	24.7156	28.5280	32.2799	35.9184	39.3990	42.6859	45.7519
3PD	LCL	4.5670	6.5101	8.6119	10.8028	13.0072	15.1525	17.1767
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	14.3115	17.1895	20.1844	23.2131	26.1890	29.0313	31.6732
	UCL	24.0560	27.8689	31.7568	35.6235	39.3709	42.9100	46.1697
NW	LCL	4.6512	6.4371	8.3780	10.4599	12.6649	14.9604	17.2800
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	14.4391	17.0837	19.8559	22.7443	25.7309	28.7786	31.8071
	UCL	24.2269	27.7302	31.3337	35.0287	38.7970	42.5968	46.3342
New	LCL	4.2106	6.0509	8.1732	10.5083	12.9377	15.3161	17.5104
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	13.7684	16.5204	19.5674	22.8105	26.0962	29.2462	32.1055
	UCL	23.3263	26.9899	30.9616	35.1128	39.2547	43.1762	46.7006
Model	Time Index	15	16	17	18	19	20	21
GO	LCL	17.3596	18.9914	20.6353	22.2899	23.9543	25.6274	27.3083
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	31.9102	34.0140	36.1149	38.2128	40.3078	42.4000	44.4892
	UCL	46.4609	49.0366	51.5945	54.1357	56.6614	59.1725	61.6700
DS	LCL	18.2495	19.9343	21.5408	23.0652	24.5055	25.8613	27.1334
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	33.0599	35.2212	37.2650	39.1905	40.9985	42.6914	44.2723
	UCL	47.8703	50.5081	52.9892	55.3157	57.4916	59.5216	61.4112
IS	LCL	19.0338	20.6944	22.1468	23.3929	24.4447	25.3206	26.0416
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	34.0684	36.1901	38.0319	39.6028	40.9225	42.0173	42.9159
	UCL	49.1031	51.6858	53.9170	55.8126	57.4002	58.7140	59.7903
Y1	LCL	17.1339	18.9469	20.8036	22.7036	24.6466	26.6326	28.6614
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	31.6176	33.9569	36.3291	38.7349	41.1752	43.6507	46.1623
	UCL	46.1014	48.9669	51.8545	54.7662	57.7038	60.6689	63.6633
Y2	LCL	13.0678	14.4820	15.9228	17.3890	18.8795	20.3933	21.9297
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	26.2700	28.1480	30.0408	31.9483	33.8703	35.8068	37.7574
	UCL	39.4723	41.8140	44.1588	46.5076	48.8612	51.2202	53.5851
PNZ	LCL	19.0643	20.7297	22.1869	23.4379	24.4946	25.3752	26.1009
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	34.1076	36.2350	38.0827	39.6594	40.9849	42.0855	42.9898
	UCL	49.1508	51.7404	53.9784	55.8809	57.4753	58.7957	59.8786
PZ	LCL	19.0338	20.6944	22.1468	23.3929	24.4447	25.3206	26.0416
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	34.0684	36.1901	38.0319	39.6028	40.9225	42.0173	42.9159
	UCL	49.1031	51.6858	53.9170	55.8126	57.4002	58.7140	59.7903
DP1	LCL	14.4135	17.3696	20.5751	24.0301	27.7344	31.6881	35.8912
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	28.0575	31.9232	36.0384	40.4029	45.0168	49.8801	54.9928
	UCL	41.7015	46.4768	51.5016	56.7757	62.2992	68.0721	74.0944
	LCL	15.2877	17.6556	20.2407	23.0483	26.0832	29.3501	32.8534
DP2	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	29.2088	32.2933	35.6123	39.1692	42.9677	47.0112	51.3031
	UCL	43.1299	46.9311	50.9838	55.2901	59.8522	64.6723	69.7527
TC	LCL	18.6995	20.3518	21.8610	23.2220	24.4349	25.5033	26.4341
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	33.6392	35.7538	37.6704	39.3879	40.9101	42.2452	43.4041
	UCL	48.5788	51.1559	53.4799	55.5537	57.3854	58.9871	60.3741
3PD	LCL	19.0334	20.6939	22.1463	23.3926	24.4448	25.3213	26.0433
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	34.0679	36.1895	38.0313	39.6023	40.9226	42.0182	42.9180
	UCL	49.1025	51.6850	53.9163	55.8121	57.4004	58.7151	59.7928
NW	LCL	19.4976	21.4257	22.8915	23.8548	24.4178	24.7248	24.8877
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	34.6629	37.1190	38.9717	40.1830	40.8888	41.2730	41.4767
	UCL	49.8281	52.8123	55.0519	56.5112	57.3598	57.8212	58.0657
New	LCL	19.4345	21.0601	22.4030	23.5015	24.4000	25.1390	25.7523
	$\widehat{\mathrm{m}}\left(\mathrm{t}\right)$	34.5821	36.6551	38.3556	39.7393	40.8664	41.7906	42.5556
	UCL	49.7296	52.2501	54.3082	55.9771	57.3329	58.4422	59.3590

in Appendix A list the 99% confidence intervals of all 13 models for Dataset #1, #2 and #3.

4.2.2. Optimal Software Release

We apply the estimated parameters of the proposed model, which are set as follows: $\widehat{\mathrm{a}}$ = 0.488, $\widehat{\mathrm{b}}$ = 0.892, $\widehat{\mathsf{\alpha }}$ = 0.328, $\widehat{\mathsf{\beta }}$ = 0.801, $\widehat{\mathsf{\gamma }}$ = 4644.6, $\widehat{\mathrm{p}}$ = 0.942, and $\widehat{\mathrm{N}}$ = 184.23, in Section 2 to the cost model using Dataset #1.

The expected total software cost is as follows.

$$\begin{gathered} \begin{array}{l}\mathrm{EC}\left(\mathrm{T}\right)\\ ={\mathrm{C}}_1\mathrm{T}+{\mathrm{C}}_2\mathrm{m}\left(\mathrm{T}\right)+{\mathrm{C}}_3\left(1-\mathrm{R}\left(\mathrm{x}|\mathrm{T}\right)\right)+{\mathrm{C}}_4\left[\mathrm{m}\left(\mathrm{T}+\mathrm{x}\right)-\mathrm{m}\left(\mathrm{T}\right)\right]\\ ={\mathrm{C}}_1\mathrm{T}+{\mathrm{C}}_2\mathrm{N}{\left[1-\left(\frac{\mathsf{\beta }}{\mathsf{\beta }-\frac{\mathrm{a}}{\mathrm{b}}\ln \left(\frac{\left(1+\mathsf{\gamma }\right){\mathrm{e}}^{-\mathrm{bpT}}}{1+{\mathsf{\gamma }\mathrm{e}}^{-\mathrm{bpT}}}\right)}\right)\right]}^{\mathsf{\alpha }}+{\mathrm{C}}_3\left(1-\mathrm{R}\left(\mathrm{x}|\mathrm{T}\right)\right) \\ +{\mathrm{C}}_4\mathrm{N}{\left[1-\left(\frac{\mathsf{\beta }}{\mathsf{\beta }-\frac{\mathrm{a}}{\mathrm{b}}\ln \left(\frac{\left(1+\mathsf{\gamma }\right){\mathrm{e}}^{-\mathrm{bp}\left(\mathrm{T}+\mathrm{x}\right)}}{1+{\mathsf{\gamma }\mathrm{e}}^{-\mathrm{bp}\left(\mathrm{T}+\mathrm{x}\right)}}\right)}\right)\right]}^{\mathsf{\alpha }}\end{array} \end{gathered}$$

We consider coefficients in the cost model for the baseline case, which are set as follows:

$${\mathrm{C}}_1=5, {\mathrm{C}}_2=100, {\mathrm{C}}_3=2000, {\mathrm{C}}_4=300, \mathrm{x}=15, {\mathrm{R}}_0=0.85$$

As shown in

Table 9. Optimal release time with expected total cost EC(T) and the software reliability subject to the operating period.

Operating Period	EC(T)	T*	Reliability
x = 9	19,024.95	88.0	0.8729
x = 12	19,098.35	96.1	0.8641
x = 15 (base)	19,157.92	102.6	0.8564
x = 18	19,211.82	108.1	0.8496
x = 21	19,261.71	112.7	0.8431

, the optimal release time T*, with the expected minimum total cost of 19,157.92 is 102.6. The software reliability at T* = 102.6 is 0.8564, which is larger than ${\mathrm{R}}_0=0.85$ for the baseline case. In addition, we compared some of the operating periods and coefficients to study the impact of the operating period and various coefficients on the optimal release time with the expected minimum total cost and the software reliability. First, we find the impact of the operating period x on the optimal release time with the expected minimum total cost by changing the value of the operating period x and comparing the optimal release times and the software reliability. We change the operating period x from 9 weeks to 12, 18, and 21 weeks. In Table 9, when x = 9, the optimal release time T* is 88.0, the expected minimum total cost is 19,024.95, and the software reliability is 0.8729. When x = 12, the optimal release time T* is 96.1, the expected minimum total cost is 19,098.35, and the software reliability is 0.8641. When x = 18, the optimal release time T* is 108.1, the expected minimum total cost is 19,211.82, and the software reliability is 0.8496. When x = 21, the optimal release time T* is 112.7, the expected minimum total cost is 19,261.71 is, and the software reliability is 0.8431. Figure 14 shows graphs of the expected total cost and the software reliability subject to the operating period. Figure 15 shows graphs of the expected total cost based on different operating periods. As shown in Table 9, as the operating period increases, the optimal release time also increases, and the software reliability decreases.

Second, we find the impact of the cost coefficients, C1, C2, C3, and C4 on the optimal release time with the expected minimum total cost and the software reliability by changing their values. When C1 = 2.5, the optimal release time T* is 129.1, the expected minimum total cost is 18,907.92, and the software reliability is 0.9074. When C1 = 7.5, the optimal release time T* is 89.2, the expected minimum total cost is 195,395.72, and the software reliability is 0.8135. When C2 = 80, the optimal release time T* is 104.3, the expected minimum total cost is 15,496.43, and the software reliability is 0.8608. When C2 = 120, the optimal release time T* is 100.9, the expected minimum total cost is 22,819.4, and the software reliability is 0.8518. When C3 = 1500, the optimal release time T* is 92.6, the expected minimum total cost is 19,078.56, and the software reliability is 0.8260. When C3 = 2500, the optimal release time T* is 111.2, the expected minimum total cost is 19,233.28, and the software reliability is 0.8767. When C4 = 200, the optimal release time T* is 100.4, the expected minimum total cost if 19,141.58, and the software reliability is 0.8504. When C4 = 400, the optimal release time T* is 104.7, the expected minimum total cost is 19,174.25, and the software reliability is 0.8618. As shown in

Table 10. Optimal release time with expected total cost and the software reliability according to cost coefficient.

Cost Coefficient	EC(T)	T*	Reliability
Base	19,157.92	102.6	0.8564
C1 = 2.5	18,907.92	129.1	0.9074
C1 = 7.5	19,395.72	89.2	0.8135
C2 = 80	15,496.43	104.3	0.8608
C2 = 120	22,819.4	100.9	0.8518
C3 = 1500	19,078.56	92.6	0.8260
C3 = 2500	19,233.28	111.2	0.8767
C4 = 200	19,141.58	100.4	0.8504
C4 = 400	19,174.25	104.7	0.8618

, as the values of C1 and C2 increase, the optimal release time decreases, and the software reliability decreases. On the contrary, as the values of C3 and C4 increase, the optimal release time increases, and the software reliability increases.

4.2.3. Sensitivity Analysis

A sensitivity analysis refers to an analysis of the effect of a parameter change by substituting all the possible values of the parameter when it is uncertain in a model. We conducted a sensitivity analysis on the parameters of the proposed model. A sensitivity analysis of parameter $\mathsf{\theta }$ can be performed using the following formula:

$${\mathrm{S}}_{\mathrm{T}}=\frac{\mathrm{T}\left(\left(1+\mathsf{\theta }\right)\mathrm{P}\right)+\mathrm{T}\left(\mathrm{P}\right)}{\mathrm{T}\left(\mathrm{P}\right)}, {\mathrm{S}}_{\mathrm{R}}=\frac{R\left(\left(1+\mathsf{\theta }\right)\mathrm{P}\right)+\mathrm{R}\left(\mathrm{P}\right)}{\mathrm{R}\left(\mathrm{P}\right)}.$$

where ${\mathrm{S}}_{\mathrm{T}}$ is the sensitivity of the optimal release time, ${\mathrm{S}}_{\mathrm{R}}$ is the sensitivity of the software reliability, and P is the parameter of the proposed model. Here, the relative changes in parameters are set as $0\%$, $\pm 10\%$, $\pm 20\%$, and $\pm 30\%$. As a result, as shown in

Table 11. Sensitivity analysis for parameters of the optimal release time.

Parameter	−30%	−20%	−10%	0%	10%	20%	30%
$\mathrm{a}$	0.10039	0.06238	0.02924	0	−0.02534	−0.04873	−0.07018
$\mathrm{b}$	0.04191	0.02437	0.01072	0	−0.00877	−0.01657	−0.02242
$\mathsf{\gamma }$	−0.00390	−0.00195	−0.00097	0	0.00097	0.00195	0.00292
$\mathsf{\alpha }$	−0.09649	−0.06140	−0.02924	0	0.02729	0.05263	0.07602
$\mathsf{\beta }$	−0.09357	−0.05945	−0.02827	0	0.02632	0.05068	0.07407
$\mathrm{p}$	0.14327	0.08674	0.03996	0	−0.03411	−0.06530	−0.09259
$\mathrm{N}$	−0.09747	−0.06140	−0.02924	0	0.02729	0.05263	0.07700

, the variation in the parameter p is greater than the other parameters for the sensitivity of the optimal release time. In

Table 12. Sensitivity analysis for parameters of the software reliability.

Parameter	−30%	−20%	−10%	0%	10%	20%	30%
$\mathrm{a}$	−0.02531	−0.01525	−0.00698	0	0.00622	0.0115	0.01603
$\mathrm{b}$	−0.00002	−0.00004	−0.00005	0	0.00004	−0.0001	0.00006
$\mathsf{\gamma }$	0.00008	0.00020	0.00008	0	−0.00004	−0.0001	−0.00004
$\mathsf{\alpha }$	0.02151	0.01381	0.00673	0	−0.00629	−0.0123	−0.01813
$\mathsf{\beta }$	0.02163	0.01389	0.00678	0	−0.00633	−0.01237	−0.01794
$\mathrm{p}$	−0.02501	−0.01529	−0.00703	0	0.00626	0.01139	0.01608
$\mathrm{N}$	0.02147	0.01398	0.00682	0	−0.00638	−0.01247	−0.01808

, again, the variations in the parameters a and p are greater than the other parameters for the sensitivity of the software reliability. Figure 16 shows the graphs of variations in parameters of the optimal release time.

5. Conclusions

Many NHPP SRMs have been developed in a controlled (testing) environment. The environments in which software runs are very diverse and complex. Moreover, it is very important to decide when, how, and at what cost to release software to users. We proposed a new NHPP SRM with a fault detection rate function affected by the probability of fault removal on failure when considering operating environments and discussed the optimal release time and software reliability with the new NHPP SRM. A comparison of eight common criteria through numerical examples shows that the proposed new NHPP SRM model is more suitable than the other NHPP SRMs. Additionally, we analyzed the software reliability, the optimal release time, and the expected total cost with respect to changes in operating period and cost coefficients. A sensitivity analysis was performed to examine the parameter uncertainty of the parameters for the proposed model. As a result, it was confirmed that the variation of p, the parameter of the defect detection function, is the largest.