On One Approach to Obtaining Estimates of the Rate of Convergence to the Limiting Regime of Markov Chains

Abstract

We revisit the problem of the computation of the limiting characteristics of (in)homogeneous continuous-time Markov chains with the finite state space. In general, it can be performed only numerically. The common rule of thumb is to interrupt calculations after quite some time, hoping that the values at some distant time interval will represent the sought-after solution. Convergence or ergodicity bounds, when available, can be used to answer such questions more accurately; i.e., they can indicate how to choose the position and the length of that distant time interval. The logarithmic norm method is a general technique that may allow one to obtain such bounds. Although it can handle continuous-time Markov chains with both finite and countable state spaces, its downside is the need to guess the proper similarity transformations, which may not exist. In this paper, we introduce a new technique, which broadens the scope of the logarithmic norm method. This is achieved by firstly splitting the generator of a Markov chain and then merging the convergence bounds of each block into a single bound. The proof of concept is illustrated by simple examples of the queueing theory.

1. Introduction

The number of theoretical and application-oriented results available in the literature concerning Markov chains can (and will) never be counted. The task does not become simpler even if one narrows down the search to the continuous-time Markov chains (CTMCs), which are homogeneous with a finite-state space. Nevertheless, the richness of this mathematical model and its widespread applications give rise to new and old research questions (see, for example, the recent review [1]), which require new, improved, or clarified solutions.

In this paper, we revisit the well-known problem of the computation of the limiting distributions of CTMCs with possibly time-varying transition intensities (for a review, one can refer, for example, to [2,3] and the references therein). Such problems arise in various operation research applications (see [4,5]). But here, for illustration purposes, we will make use of simple examples from the queueing theory, as its models (see, for example, refs. [2,6,7,8,9,10,11,12,13]) fit the mathematical framework of such Markov chains excellently.

The computation of limiting distributions usually has to be performed numerically, without knowing the limiting distribution itself. Therefore, an algorithm must be “told” when to stop the computation, i.e., when the limiting regime is reached. One of the methods which may provide such an answer in the form of an ergodicity (convergence) bound for the considered type of CTMC is the logarithmic norm method. (although ergodic properties of Markov chains have been the subject of extensive research (see, for example, refs. [14,15,16,17,18,19]), practically useful general ergodicity bounds are difficult to obtain and it remains, to a large extent, an open problem).

This implies (see, for example, refs. [20,21,22]) that (under certain conditions on the generator of the CTMC, say, $A\left(t\right)=\left(a_{ij}\left(t\right)\right)$) for the vector $\mathbf{x}\left(t\right)$ of state probabilities, which satisfies the system of ordinary differential equations (ODEs), ${\displaystyle \frac{d}{dt}}\mathbf{x}\left(t\right)=A\left(t\right)\mathbf{x}\left(t\right)$, $t\ge 0$, the upper bound. Henceforth, $\parallel ·\parallel$ is the $l_1$-norm; i.e., if $\mathbf{x}$ is an $(S+1)$-dimensional column vector, then $\parallel \mathbf{x}\parallel ={\sum }_{k=0}^S\left|x_k\right|$. In the case of a linear operator, the norm is induced by the $l_1$-norm on column vectors, for example, $\parallel A\left(t\right)\parallel ={sup}_{0\le j\le S}{\sum }_{i=0}^S\left|a_{ij}\left(t\right)\right|$. $\parallel \mathbf{x}\left(t\right)\parallel \le e^{-{\int }_0^t\alpha \left(u\right)du}\parallel \mathbf{x}\left(0\right)\parallel$ holds, where $-\alpha \left(t\right)={sup}_i\left\{a_{ii}\left(t\right)+{\sum }_{j\ne i}\left|a_{ji}\left(t\right)\right|\right\}$ is called the logarithmic norm of $A\left(t\right)$. The pros and cons of the method can be readily seen. On the one hand, it is quite general and the bounds can be simply computed. On the other hand, it can happen (and in the most interesting cases, it is so) that $-\alpha \left(t\right)>0$ and thus the bound is meaningless. Therefore, certain techniques were developed (see, for example, refs. [23,24,25,26]), which allow one to obtain the convergence bounds but in weighted norms. Nevertheless, the success that can be achieved along these lines remains limited due to the fact that the weights need to be guessed for each new CTMC without a “rule of thumb”; they do not have any probabilistic meaning and can be considered as analogues of Lyapunov functions.

In this paper, we present the new technique, which expands the scope of the logarithmic norm method. It implies that in order to obtain the convergence bound for the given CTMC, one firstly partitions its infinitesimal matrix into blocks and the ergodicity properties of the blocks are studied independently. Then, the results are merged into one single ergodicity bound for the original CTMC. Whenever the probability flows between the blocks are not high, the merged bound is good. This technique does not solve the problem of choosing proper weights. But partitioning makes the search for them easier and sometimes unnecessary. Yet it is a matter of pure guessing regarding how to perform the partitioning, and there is no guarantee that the resulting (merged) ergodicity bound will be meaningful (see [27]).

The use of this new technique allows one to study the probabilistic characteristics of quasi-birth–death (QBD) processes, which have become very popular especially in the field of queueing theory. The first works on this class of random processes appeared in the late 1960s, when the first publications of Evans [28] and Wallace [29] appeared and further research was carried out by Neuts [30,31,32,33] and Ost [34]. Ergodicity bounds for QBDs, using easy-to-use techniques like logarithmic norm, usually cannot be found. But, as experiments show, using the technique proposed in this paper, sometimes finite QBD generators can be split into blocks, and the ergodicity condition established for each block can be merged into a single meaningful bound.

In Section 2, we introduce notation and in Section 3 and Section 4, we provide the description of the new technique. Section 5, Section 6 and Section 7 are devoted to special cases, which demonstrate the applicability of the technique and its connections to the known results in the literature.

2. Preliminaries

Let $\left\{X\right(t),t\ge 0\}$ be a (possibly inhomogeneous) CTMC with a finite state space $\mathcal{X}=\{0,1,\cdots ,S\}$. The transition probabilities will be denoted by $p_{ij}(s,t)$, i.e., $p_{ij}(s,t)=Pr\left\{X\left(t\right)=j\left|X\left(s\right)=i\right.\right\}$, $i,j\ge 0,0\le s\le t$. Let the state probability also be denoted by $p_i\left(t\right)=Pr\left\{X\left(t\right)=i\right\}$, and the vector of state probabilities by $\mathbf{p}\left(t\right)={\left(p_0\left(t\right),p_1\left(t\right),\cdots ,p_S\left(t\right)\right)}^T$. In what follows, it is assumed that

$$Pr\left\{X\left(t+h\right)=j|X\left(t\right)=i\right\}=\left\{\begin{array}{cc}{q}_{ij}\left(t\right)h+{\alpha }_{ij}\left(t,h\right),\hfill & \mathrm{if} j\ne i\hfill \\ 1-{\displaystyle \sum _{k\ne i}{q}_{ik}\left(t\right)h+{\alpha }_i\left(t,h\right)},\hfill & \mathrm{if} j=i,\hfill \end{array}\right.$$

(1)

where all ${\alpha }_i(t,h)$ are $o\left(h\right)$ uniformly in i, that is, ${sup}_i\left|{\alpha }_i(t,h)\right|=o\left(h\right)$. Moreover, if $X\left(t\right)$ is inhomogeneous, then all its infinitesimal characteristics (intensity functions) $q_{ij}\left(t\right)$ are integrable in t on any interval $[a,b]$, $0\le a\le b$.

Denote $a_{ij}\left(t\right)=q_{ji}\left(t\right)$ as $j\ne i$ and $a_{ii}\left(t\right)=-{\sum }_{j\ne i}{a}_{ji}\left(t\right)=-{\sum }_{j\ne i}{q}_{ij}\left(t\right)$.

Further, in order to be able to obtain tighter estimates, it is assumed that

$$|a_{ii}\left(t\right)|\le L<\infty$$

(2)

for almost all $t\ge 0$. The state probabilities then satisfy the forward Kolmogorov system:

$${\displaystyle \frac{d}{dt}}\mathbf{p}\left(t\right)=A\left(t\right)\mathbf{p}\left(t\right),t\ge 0,$$

(3)

where the initial distribution $\mathbf{p}\left(0\right)$ is given, $A\left(t\right)=Q^T\left(t\right)$, and $Q\left(t\right)$ is the infinitesimal matrix of the CTMC.

Let $\parallel ·\parallel$ be the usual $l_1$-norm, i.e., $\parallel \mathbf{x}\parallel =\sum |x_i|$, and $\parallel B\parallel ={sup}_j{\sum }_i\left|b_{ij}\right|$ for $B={\left(b_{ij}\right)}_{i,j=0}^{\infty }$. Denote $\Omega =\left\{\mathbf{x}:\mathbf{x}\in l_1^{+}\&\parallel \mathbf{x}\parallel =1\right\}$.

Therefore, $\parallel A\left(t\right)\parallel =2{sup}_k\left|a_{kk}\left(t\right)\right|\le 2L$ for almost all $t\ge 0$, and we can apply the results of [35] to the Equation (3) in the space $l_1$. Namely, in [35] it was shown that the Cauchy problem for the Equation (3) has a unique solution for an arbitrary initial condition. Moreover, if $\mathbf{p}\left(s\right)\in \Omega$, then $\mathbf{p}\left(t\right)\in \Omega$, for any $0\le s\le t$ and any initial condition $\mathbf{p}\left(s\right)$.

In what follows, we will make use of the fact that the subtraction of a given function from any row of the matrix $A\left(t\right)$ does not change the CTMC’s convergence rate (see, for example, ref. [26]). Indeed, let $\overrightarrow{1}$ denote the vector of ones and ⊗ be the Kronecker product. Then, for the vector $\overrightarrow{r}\left(t\right)=(r_0\left(t\right),r_1\left(t\right),\cdots ,r_S{\left(t\right)}^T$ of arbitrary functions $r_i\left(t\right)$, the system (3) can be rewritten in the following form:

$${\displaystyle \frac{d}{dt}}\mathbf{p}\left(t\right)=\left(A\left(t\right)-\overrightarrow{1}\otimes \overrightarrow{r}\left(t\right)\right)\mathbf{p}\left(t\right)+\left(\overrightarrow{1}\otimes \overrightarrow{r}\left(t\right)\right)\mathbf{p}\left(t\right).$$

Since ${\sum }_{i=0}^S{p}_i\left(t\right)=1$ for any $t\ge 0$, one has that $\left(\overrightarrow{1}\otimes \overrightarrow{r}\left(t\right)\right)\mathbf{p}\left(t\right)=\overrightarrow{r}\left(t\right)$. Therefore, for any two solutions ${\mathbf{p}}^{*}\left(t\right)$ and ${\mathbf{p}}^{**}\left(t\right)$ of the system (3), one has that

$${\displaystyle \frac{d}{dt}}{\mathbf{p}}^{*}\left(t\right)=(A\left(t\right)-\overrightarrow{1}\otimes \overrightarrow{r}\left(t\right)){\mathbf{p}}^{*}\left(t\right)+\overrightarrow{r}\left(t\right),$$

$${\displaystyle \frac{d}{dt}}{\mathbf{p}}^{**}\left(t\right)=(A\left(t\right)-\overrightarrow{1}\otimes \overrightarrow{r}\left(t\right)){\mathbf{p}}^{**}\left(t\right)+\overrightarrow{r}\left(t\right),$$

and hence,

$${\displaystyle \frac{d}{dt}}\mathbf{x}\left(t\right)=\tilde{A}\left(t\right)\mathbf{x}\left(t\right),$$

(4)

where $\mathbf{x}\left(t\right)={\mathbf{p}}^{*}\left(t\right)-{\mathbf{p}}^{**}\left(t\right)$ and $\tilde{A}\left(t\right)=A\left(t\right)-\overrightarrow{1}\otimes \overrightarrow{r}\left(t\right)$.

Henceforth, it will be convenient to use the block representation of $\tilde{A}\left(t\right)$ for the matrix (which is always feasible).

$$\tilde{A}\left(t\right)=\left(\begin{array}{cc}\tilde{C}\left(t\right)& \tilde{D}\left(t\right)\\ \tilde{E}\left(t\right)& \tilde{F}\left(t\right)\end{array}\right),$$

(5)

in which the diagonal blocks are square matrices of size $m>0$ and $N>0$, respectively, subject to the constraint $m+N=S+1$.

3. Upper Bounds for the Rate of Convergence

In the previous section, it was shown that the rate of convergence of the system ${\displaystyle \frac{d}{dt}}\mathbf{p}\left(t\right)=A\left(t\right)\mathbf{p}\left(t\right)$ coincides with the rate of convergence of the system ${\displaystyle \frac{d}{dt}}\mathbf{x}\left(t\right)=\tilde{A}\left(t\right)\mathbf{x}\left(t\right)$, where $\tilde{A}\left(t\right)$ is given by (5). By denoting

$$\begin{array}{ccc}\hfill \mathbf{x}\left(t\right)& =& {(x_0\left(t\right),x_1\left(t\right),\dots ,x_{m-1}\left(t\right),x_m\left(t\right),x_{m+1}\left(t\right),\cdots ,x_S\left(t\right))}^T,\hfill \\ \hfill {\mathbf{x}}_0\left(t\right)& =& {(x_0\left(t\right),x_1\left(t\right),\dots ,x_{m-1}\left(t\right))}^T,\hfill \\ \hfill {\mathbf{x}}_1\left(t\right)& =& {(x_m\left(t\right),x_{m+1}\left(t\right),\dots ,x_S\left(t\right))}^T,\hfill \end{array}$$

this new system can be split into the following two coupled systems of equations:

$$\begin{array}{cc}\hfill {\displaystyle \frac{d}{dt}}{\mathbf{x}}_0\left(t\right)& =\tilde{C}\left(t\right){\mathbf{x}}_0\left(t\right)+\tilde{D}\left(t\right){\mathbf{x}}_1\left(t\right),\hfill \end{array}$$

(6)

$$\begin{array}{cc}\hfill {\displaystyle \frac{d}{dt}}{\mathbf{x}}_1\left(t\right)& =\tilde{E}\left(t\right){\mathbf{x}}_0\left(t\right)+\tilde{F}\left(t\right){\mathbf{x}}_1\left(t\right).\hfill \end{array}$$

(7)

The solutions of these systems can be written in the form

$${\mathbf{x}}_0\left(t\right)=U_0(t,0){\mathbf{x}}_0\left(0\right)+{\int }_0^t{U}_0(t,s)\tilde{D}\left(s\right){\mathbf{x}}_1\left(s\right)ds,$$

(8)

$${\mathbf{x}}_1\left(t\right)=U_1(t,0){\mathbf{x}}_1\left(0\right)+{\int }_0^t{U}_1(t,s)\tilde{E}\left(s\right){\mathbf{x}}_0\left(s\right)ds.$$

(9)

Assume without the loss of generality that $\parallel U_0(t,0)\parallel \le \parallel U_1(t,0)\parallel$ and $K={sup}_t\parallel \tilde{D}\left(t\right)\parallel ,$ $L={sup}_t\parallel \tilde{E}\left(t\right)\parallel$. Let ${\mathbf{x}}_1\left(0\right)=\overrightarrow{0}$, where $\overrightarrow{0}$ denotes the vector of zeros. By sequentially substituting the expressions for ${\mathbf{x}}_0\left(t\right)$ and ${\mathbf{x}}_1\left(t\right)$, given by (8) and (9), into the integrals on the right-hand side of (8) and (9), and then taking norms of both sides, one obtains the following upper bound for $\parallel {\mathbf{x}}_0\left(t\right)\parallel$:

$$\begin{array}{c}\parallel {\mathbf{x}}_0\left(t\right)\parallel \le \parallel U_0(t,0)\parallel \parallel {\mathbf{x}}_0\left(0\right)\parallel +{\int }_0^t\parallel U_0(t,s_1)\parallel \parallel \tilde{D}\left(s_1\right)\parallel \parallel U_1(s_1,0)\parallel \parallel {\mathbf{x}}_1\left(0\right)\parallel d{s}_1+\cdots \le \hfill \\ \le \parallel U_1(t,0)\parallel \parallel {\mathbf{x}}_0\left(0\right)\parallel +{\int }_0^t\parallel U_1(t,s_1)\parallel \parallel \tilde{D}\left(s_1\right)\parallel \parallel U_1(s_1,0)\parallel \parallel {\mathbf{x}}_1\left(0\right)\parallel d{s}_1+\cdots \le \\ \hfill \le \parallel U_1(t,0)\parallel \left(1+{\displaystyle \frac{\left(KL\right)t^2}{2!}}+{\displaystyle \frac{{\left(KL\right)}^2{t}^4}{4!}}+\cdots \right)\parallel {\mathbf{x}}_0\left(0\right)\parallel .\end{array}$$

(10)

Since, due to (2), both K and L are finite, from the last inequality, it follows that

$$\parallel {\mathbf{x}}_0\left(t\right)\parallel \le \parallel U_1(t,0)\parallel cosh\left(t\sqrt{KL}\right)\parallel {\mathbf{x}}_0\left(0\right)\parallel .$$

(11)

Accordingly, by putting ${\mathbf{x}}_0\left(0\right)=\overrightarrow{0}$, one has that

$$\parallel {\mathbf{x}}_1\left(t\right)\parallel \le \parallel U_1(t,0)\parallel cosh\left(t\sqrt{KL}\right)\parallel {\mathbf{x}}_1\left(0\right)\parallel .$$

(12)

Now, since the vector $\mathbf{x}\left(t\right)$ can be represented as $\mathbf{x}\left(t\right)=\left({\displaystyle \genfrac{}{}{0pt}{}{1}{0}}\right)\otimes {\mathbf{x}}_0\left(t\right)+\left({\displaystyle \genfrac{}{}{0pt}{}{0}{1}}\right)\otimes {\mathbf{x}}_1\left(t\right)$ and

$$\parallel {\mathbf{x}}_0\left(t\right)\parallel =∥\left({\displaystyle \genfrac{}{}{0pt}{}{1}{0}}\right)\otimes {\mathbf{x}}_0\left(t\right)∥,\parallel {\mathbf{x}}_1\left(t\right)\parallel =∥\left({\displaystyle \genfrac{}{}{0pt}{}{0}{1}}\right)\otimes {\mathbf{x}}_1\left(t\right)∥,$$

we obtain from (11) and (12)

$$\begin{array}{c}\hfill \parallel \mathbf{x}\left(t\right)\parallel \le 2\parallel U_1(t,0)\parallel cosh\left(t\sqrt{KL}\right)\parallel \mathbf{x}\left(0\right)\parallel .\end{array}$$

(13)

The derivations made above can be summarized in the following theorem.

Theorem 1. 

Let there exist a function $\alpha \left(t\right)$ and a positive M such that the rate of convergence for the matrices $\tilde{C}\left(t\right)$ and $\tilde{F}\left(t\right)$ satisfies the inequalities $\parallel U_0(t,0)\parallel \le \parallel U_1(t,0)\parallel \le e^{-\underset{0}{\overset{t}{\int }}\alpha \left(s\right)ds}$. Then, if ${\int }_0^{\infty }(\alpha \left(s\right)-\sqrt{KL})ds=+\infty$ for some $K\ge {sup}_t\parallel \tilde{D}\left(t\right)\parallel$ and $L\ge {sup}_t\parallel \tilde{E}\left(t\right)\parallel$, the CTMC $\left\{X\right(t),t\ge 0\}$ is exponentially ergodic and the bound

$$\begin{array}{c}\hfill \parallel {\mathbf{p}}^{*}\left(t\right)-{\mathbf{p}}^{**}\left(t\right)\parallel \le 2M{e}^{-\alpha t}cosh\left(t\sqrt{KL}\right)\parallel {\mathbf{p}}^{*}\left(0\right)-{\mathbf{p}}^{**}\left(0\right)\parallel \end{array}$$

(14)

holds for any two initial conditions ${\mathbf{p}}^{*}\left(0\right)$ and ${\mathbf{p}}^{**}\left(0\right)$.

A number of corollaries can be drawn from the theorem. For example, the bound (14) is applicable even if the initial distributions are unknown, since $\parallel {\mathbf{p}}^{*}\left(0\right)-{\mathbf{p}}^{**}\left(0\right)\parallel \le 2$. Next, making use of the assumption that the state space of the CTMC is finite, one immediately obtains the bound $4M{S}^n{e}^{-\alpha t}cosh\left(t\sqrt{KL}\right)$ for the rate of convergence of any positive moment n of $\left\{X\right(t),t\ge 0\}$. This enables one to use concentration inequalities to obtain some insight into the the behaviour of the probabilities of events related to $\left\{X\right(t),t\ge 0\}$.

4. Relationship between Cauchy Operators

Denote the Cauchy operator of the ODE system ${\displaystyle \frac{d}{dt}}\mathbf{x}\left(t\right)=A\left(t\right)\mathbf{x}\left(t\right)$, $t\ge 0$, by $U_x(t,s)$. Thus, $\mathbf{x}\left(t\right)=U_x(t,0)\mathbf{x}\left(0\right)$. For $\mathbf{y}\left(t\right)=T\mathbf{x}\left(t\right)$, where T is any linear transform, we have that ${\displaystyle \frac{d}{dt}}\mathbf{y}\left(t\right)=TA\left(t\right)T^{-1}\mathbf{y}\left(t\right)$ and $\mathbf{y}\left(t\right)=U_y(t,0)\mathbf{y}\left(0\right)$, where $U_y(t,s)$ is the corresponding Cauchy operator. Assuming that T has a unique inverse, we have that

$$\mathbf{x}\left(t\right)=T^{-1}\mathbf{y}\left(t\right)=T^{-1}{U}_y(t,0)T{T}^{-1}\mathbf{y}\left(0\right)$$

and therefore,

$$\mathbf{x}\left(t\right)=T^{-1}{U}_y(t,0)T\mathbf{x}\left(0\right).$$

(15)

Due to the uniqueness of the solution of the initial ODE system, we have that $U_x(t,0)=T^{-1}{U}_y(t,0)T$. Therefore, the Cauchy operators of the two systems satisfy the following inequalities:

$$\parallel U_x(t,0)\parallel \le \parallel T^{-1}\parallel \parallel T\parallel \parallel U_y(t,0)\parallel ,$$

(16)

$$\parallel U_y(t,0)\parallel \le \parallel T^{-1}\parallel \parallel T\parallel \parallel U_x(t,0)\parallel .$$

(17)

Since $\parallel U_y(t,0)\parallel \le exp\left(-\gamma \left(TA\left(t\right)T^{-1}\right)\right)$, the following theorem holds.

Theorem 2. 

If the matrix T has a unique inverse, the Cauchy operator of the ODE system ${\displaystyle \frac{d}{dt}}\mathbf{x}\left(t\right)=A\left(t\right)\mathbf{x}\left(t\right)$ satisfies the following inequality:

$$\parallel U(t,0)\parallel \le \parallel T^{-1}\parallel \parallel T\parallel exp\left(-\gamma \left(TA\left(t\right)T^{-1}\right)\right).$$

(18)

5. Homogeneous BDPs

Let us consider a homogeneous BDP process with birth and death rates $\lambda$ and $\mu$, respectively, and $S+1$ states. From its infinitesimal matrix,

$$A=\left(\begin{array}{cccccc}-\lambda & \mu & 0& 0& \cdots & 0\\ \lambda & -(\lambda +\mu )& \mu & 0& \cdots & 0\\ 0& \lambda & -(\lambda +\mu )& \mu & \cdots & 0\\ 0& 0& \lambda & -(\lambda +\mu )& \cdots & 0\\ ⋮& ⋮& ⋮& ⋮& \ddots & \mu \\ 0& 0& 0& 0& \cdots & -\mu \end{array}\right),$$

one can obtain the matrix $\tilde{A}$ by (5), for example, by subtracting $\mu$ from the first row, and $\lambda$ from the second row. Therefore, the blocks of the matrix $\tilde{A}$ will have the following form:

$$\tilde{C}=\begin{array}{c}\hfill -\lambda -\mu \end{array},\tilde{D}=(0,-\mu ,-\mu ,\cdots ,-\mu ),\tilde{E}={(0,0,\cdots ,0)}^T,$$

$$\tilde{F}=\left(\begin{array}{cccc}\hfill -(2\lambda +\mu )\hfill & \mu -\lambda & \hfill \cdots \hfill & -\lambda \\ \hfill \lambda \hfill & -(\lambda +\mu )& \hfill \cdots \hfill & 0\\ \hfill ⋮\hfill & ⋮& \hfill \ddots \hfill & \mu \\ \hfill 0\hfill & 0& \hfill \cdots \hfill & -\mu \end{array}\right).$$

Consider the matrix T of the form

$$T=\left(\begin{array}{ccccc}\hfill 1\hfill & 1& \hfill 1\hfill & \cdots & 1\\ \hfill 0\hfill & \sqrt{{\displaystyle \frac{\mu }{\lambda }}}& \hfill \sqrt{{\displaystyle \frac{\mu }{\lambda }}}\hfill & \cdots & \sqrt{{\displaystyle \frac{\mu }{\lambda }}}\\ \hfill 0\hfill & 0& \hfill {\left(\sqrt{{\displaystyle \frac{\mu }{\lambda }}}\right)}^2\hfill & \cdots & {\left(\sqrt{{\displaystyle \frac{\mu }{\lambda }}}\right)}^2\\ \hfill ⋮\hfill & ⋮& \hfill ⋮\hfill & \ddots & ⋮\\ \hfill 0\hfill & 0& \hfill 0\hfill & \cdots & {\left(\sqrt{{\displaystyle \frac{\mu }{\lambda }}}\right)}^{S-1}\end{array}\right).$$

(19)

Then, the similarity transformation of the matrix $\tilde{F}$ yields

$$T\tilde{F}{T}^{-1}=\left(\begin{array}{ccccc}\hfill -\lambda -\mu \hfill & \sqrt{\mu \lambda }& \hfill 0\hfill & \cdots & 0\\ \hfill \sqrt{\mu \lambda }\hfill & -(2\lambda +\mu )& \hfill \sqrt{\mu \lambda }\hfill & \cdots & 0\\ \hfill 0\hfill & \sqrt{\mu \lambda }& \hfill -(\lambda +\mu )\hfill & \cdots & ⋮\\ \hfill ⋮\hfill & ⋮& \hfill ⋮\hfill & \ddots & \sqrt{\mu \lambda }\\ \hfill 0\hfill & 0& \hfill 0\hfill & \cdots & -\lambda -\mu \end{array}\right).$$

(20)

Since $K={max}_i\parallel \tilde{E}\parallel =0$ and $L={max}_i\parallel \tilde{D}\parallel =\mu$, the logarithmic norm of the matrix $\tilde{F}$ is equal to ${(\sqrt{\mu }-\sqrt{\lambda })}^2$. Therefore, from (14), we obtain the bound

$$\begin{array}{c}\hfill \parallel {\mathbf{p}}^{*}\left(t\right)-{\mathbf{p}}^{**}\left(t\right)\parallel \le 2{\left(\sqrt{{\displaystyle \frac{\mu }{\lambda }}}\right)}^{S-1}max\left(1,2\sqrt{{\displaystyle \frac{\lambda }{\mu }}}\right)e^{-{(\sqrt{\mu }-\sqrt{\lambda })}^{2}t}\parallel {\mathbf{p}}^{*}\left(0\right)-{\mathbf{p}}^{**}\left(0\right)\parallel ,\end{array}$$

for any two initial conditions ${\mathbf{p}}^{*}\left(0\right)$ and ${\mathbf{p}}^{**}\left(0\right)$. We note that the exponential rate of convergence ${(\sqrt{\mu }-\sqrt{\lambda })}^2$ coincides with the well-known results in the literature [36].

6. One Example of the Quasi-Birth–Death Process

Consider the model of a production line (see Model 3 in [37]) as a single server with the finite capacity FIFO-queue, single Poisson flow of customers of intensity $\lambda \left(t\right)$, and Erlang-2 service times with the intensity $\mu \left(t\right)$ at each stage. It gives rise to the QBD on the state space $E=\{(i,j),0\le i\le M_1\le j\le n\}$, with the following infinitesimal generator (the argument t is omitted for brevity):

$$Q=\left(\begin{array}{ccccccc}\hfill {\mathfrak{B}}_0\hfill & \mathfrak{R}& \hfill 0\hfill & 0& \cdots & 0& 0\\ \hfill \mathfrak{A}\hfill & \mathfrak{B}& \hfill \mathfrak{R}\hfill & 0& \cdots & 0& 0\\ \hfill 0\hfill & \mathfrak{A}& \hfill \mathfrak{B}\hfill & \mathfrak{R}& \cdots & 0& 0\\ \hfill ⋮\hfill & ⋮& \hfill ⋮\hfill & \ddots & \ddots & \ddots & ⋮\\ \hfill 0\hfill & 0& \hfill 0\hfill & \cdots & \mathfrak{A}& \mathfrak{B}& \mathfrak{R}\\ \hfill 0\hfill & 0& \hfill 0\hfill & \cdots & 0& \mathfrak{A}& {\mathfrak{B}}_M\end{array}\right),$$

(21)

where

$$\mathfrak{A}=\left(\begin{array}{cc}\hfill 0\hfill & 0\\ \hfill \mu \hfill & 0\end{array}\right),{\mathfrak{B}}_0=\left(\begin{array}{cc}\hfill -(\lambda +\mu )\hfill & \mu \\ \hfill \mu \hfill & -(\lambda +\mu )\end{array}\right),\mathfrak{B}={\mathfrak{B}}_0-\mathfrak{A},$$

(22)

$$\mathfrak{R}=\left(\begin{array}{cc}\hfill \lambda \hfill & 0\\ \hfill 0\hfill & \lambda \end{array}\right),{\mathfrak{B}}_M=\mathfrak{B}+\mathfrak{R}.$$

(23)

Let us split the infinitesimal generator $A\left(t\right)=Q^T\left(t\right)$ into four blocks of the following form:

$$C\left(t\right)=\left(\begin{array}{cc}-\lambda -\mu & \mu \\ \mu & -\lambda -\mu \end{array}\right),$$

(24)

$$D\left(t\right)=\left(\begin{array}{ccccccc}0& \mu & 0& 0& \cdots & 0& 0\\ 0& 0& 0& 0& \cdots & 0& 0\end{array}\right),$$

(25)

$$E\left(t\right)=\left(\begin{array}{cc}\lambda & 0\\ 0& \lambda \\ ⋮& ⋮\\ 0& 0\\ 0& 0\end{array}\right),$$

(26)

$$F\left(t\right)=\left(\begin{array}{ccccccccc}-\lambda -\mu & 0& 0& \mu & 0& 0& \cdots & 0& 0\\ \mu & -\lambda -\mu & 0& 0& 0& 0& \cdots & 0& 0\\ \lambda & 0& -\lambda -\mu & 0& 0& \mu & \cdots & 0& 0\\ 0& \lambda & \mu & -\lambda -\mu & 0& 0& \cdots & 0& 0\\ 0& 0& \lambda & 0& -\lambda -\mu & 0& \cdots & 0& 0\\ 0& 0& 0& \lambda & \mu & -\lambda -\mu & \cdots & 0& 0\\ ⋮& ⋮& ⋮& ⋮& ⋮& ⋮& \ddots & ⋮& ⋮\\ 0& 0& 0& 0& 0& 0& \cdots & -\mu & 0\\ 0& 0& 0& 0& 0& 0& \cdots & \mu & -\mu \end{array}\right).$$

(27)

By subtracting ${\displaystyle \frac{\mu \left(t\right)}{2}}$ from both rows of the matrix $C\left(t\right)$, we obtain

$$\tilde{C}\left(t\right)=\left(\begin{array}{cc}-\lambda -{\displaystyle \frac{3}{2}}\mu & {\displaystyle \frac{1}{2}}\mu \\ {\displaystyle \frac{1}{2}}\mu & -\lambda -{\displaystyle \frac{3}{2}}\mu \end{array}\right),$$

(28)

$$\tilde{D}\left(t\right)=\left(\begin{array}{ccccccc}-{\displaystyle \frac{\mu }{2}}& {\displaystyle \frac{\mu }{2}}& -{\displaystyle \frac{\mu }{2}}& -{\displaystyle \frac{\mu }{2}}& \cdots & -{\displaystyle \frac{\mu }{2}}& -{\displaystyle \frac{\mu }{2}}\\ -{\displaystyle \frac{\mu }{2}}& -{\displaystyle \frac{\mu }{2}}& -{\displaystyle \frac{\mu }{2}}& -{\displaystyle \frac{\mu }{2}}& \cdots & -{\displaystyle \frac{\mu }{2}}& -{\displaystyle \frac{\mu }{2}}\end{array}\right).$$

(29)

$$\tilde{E}\left(t\right)=E\left(t\right),\tilde{F}\left(t\right)=F\left(t\right).$$

(30)

Let us find the rate of convergence to the limiting regime for the block $\tilde{F}\left(t\right)$. Fix $h>0$ and $k>0$, and consider the square matrix T of size N of the following form:

$$T=\left(\begin{array}{ccccccccc}0& 1& 0& 0& 0& 0& \cdots & 0& 0\\ k& 0& 0& 0& 0& 0& \cdots & 0& 0\\ 0& 0& 0& h& 0& 0& \cdots & 0& 0\\ 0& 0& hk& 0& 0& 0& \cdots & 0& 0\\ 0& 0& 0& 0& 0& h^2& \cdots & 0& 0\\ 0& 0& 0& 0& h^{2}k& 0& \cdots & 0& 0\\ ⋮& ⋮& ⋮& ⋮& ⋮& ⋮& \ddots & ⋮& ⋮\\ 0& 0& 0& 0& 0& 0& \cdots & 0& h^2\\ 0& 0& 0& 0& 0& 0& \cdots & h^{2}k& 0\end{array}\right).$$

(31)

Then, the similarity transformation $T\tilde{F}{T}^{-1}$ yields the following matrix:

$$T\tilde{F}{T}^{-1}=\left(\begin{array}{ccccccccc}-\lambda -\mu & {\displaystyle \frac{\mu }{k}}& 0& 0& 0& 0& \cdots & 0& 0\\ \mu & -\lambda -\mu & {\displaystyle \frac{k\mu }{h}}& 0& 0& 0& \cdots & 0& 0\\ h\lambda & 0& -\lambda -\mu & {\displaystyle \frac{\mu }{k}}& 0& 0& \cdots & 0& 0\\ 0& h\lambda & \mu & -\lambda -\mu & {\displaystyle \frac{k\mu }{h}}& 0& \cdots & 0& 0\\ 0& 0& h\lambda & 0& -\lambda -\mu & {\displaystyle \frac{\mu }{k}}& \cdots & 0& 0\\ 0& 0& 0& h\lambda & \mu & -\lambda -\mu & \cdots & 0& 0\\ ⋮& ⋮& ⋮& ⋮& ⋮& ⋮& \ddots & ⋮& ⋮\\ 0& 0& 0& 0& 0& 0& \cdots & -\mu & {\displaystyle \frac{\mu }{k}}\\ 0& 0& 0& 0& 0& 0& \cdots & \mu & -\mu \end{array}\right),$$

(32)

the logarithmic norm of which is equal to

$$\gamma \left(T\tilde{F}{T}^{-1}\right)=max\left(-\lambda -\mu +{\displaystyle \frac{\mu }{k}}+h\lambda ,-\lambda -\mu +{\displaystyle \frac{k\mu }{h}}+h\lambda \right).$$

Now, if one puts $h=k^2,k>1$, then the rate of convergence to the limiting regime is equal to $-\lambda -\mu +{\displaystyle \frac{\mu }{k}}+\lambda k^2$. Therefore, for the norm $\parallel U_1(t,0)\parallel$ of the Cauchy operator, one has the upper bounded (note that the size of the block N coincides with the size of the block $\tilde{F}$) by

$$\parallel U_1(t,0)\parallel \le k^{N-1}exp\left({\int }_0^t\left(-\lambda \left(s\right)-\mu \left(s\right)+{\displaystyle \frac{\mu \left(s\right)}{k}}+\lambda \left(s\right)k^2\right)ds\right).$$

The convergence rate for the block $\tilde{C}\left(t\right)$ is equal to $-\lambda -\mu$ and therefore, the Cauchy operator $U_0(t,0)$ is bounded by

$$\parallel U_0(t,0)\parallel \le exp\left({\int }_0^t\left(-\lambda \left(s\right)-\mu \left(s\right)\right)ds\right).$$

Clearly, $K={sup}_t\parallel \tilde{D}\parallel ={sup}_t\mu \left(t\right)$ and $L={sup}_t\parallel \tilde{E}\parallel ={sup}_t\lambda \left(t\right)$. Since $\parallel U_1(t,0)\parallel \ge \parallel U_0(t,0)\parallel$, the Theorem 1 yields the following bound:

$$\begin{array}{c}\hfill \parallel {\mathbf{p}}^{*}\left(t\right)-{\mathbf{p}}^{**}\left(t\right)\parallel \le 2k^{N-1}{e}^{{\int }_0^t\left(-\lambda \left(s\right)-\mu \left(s\right)+{\displaystyle \frac{\mu \left(s\right)}{k}}+\lambda \left(s\right)k^2\right)ds+\sqrt{KL}t}\parallel {\mathbf{p}}^{*}\left(0\right)-{\mathbf{p}}^{**}\left(0\right)\parallel .\end{array}$$

(33)

7. Illustrative Examples

Let us take the model from the previous section as the basis for the two illustrative numerical examples: one with time-independent intensities and the other with time-dependent intensities.

Let $\lambda =1$, $\mu =16$. It is straightforward to check that in such a case, $K=16$, $L=1$ and

$$\begin{array}{c}\hfill \underset{k}{min}\left(-\lambda -\mu +{\displaystyle \frac{\mu }{k}}+\lambda k^2+\sqrt{KL}\right)=\underset{k}{min}\left(-13+{\displaystyle \frac{16}{k}}+k^2\right)=-1,\end{array}$$

(34)

which is attained when $k=2$. Then, (33) provides us with the following bound:

$$\parallel {\mathbf{p}}^{*}\left(t\right)-{\mathbf{p}}^{**}\left(t\right)\parallel \le 2^N{e}^{-t}\parallel {\mathbf{p}}^{*}\left(0\right)-{\mathbf{p}}^{**}\left(0\right)\parallel .$$

(35)

Now assume that the Poisson arrival process has the periodic intensity $\lambda \left(t\right)=0.5+0.5sin\left(\pi t\right)$, $\mu \left(t\right)=15+sin\left(\pi t\right)$. Then, $K={sup}_t\mu \left(t\right)=16$ and $L={sup}_t\lambda \left(t\right)=1$. Hence, the minimum is again attained at $k=2$, i.e.,

$$\begin{array}{c}\hfill \underset{k>1}{min}\left(\underset{t>0}{sup}\left(-(0.5+0.5sin\left(\pi t\right)))-(15+sin\left(\pi t\right))+{\displaystyle \frac{15+sin\left(\pi t\right)}{k}}+(0.5+0.5sin\left(\pi t\right))k^2+\sqrt{KL}\right)\right)=\underset{k>1}{min}\left(\underset{t>0}{sup}\left(-2+sin\left(\pi t\right)\right)\right)\ge -1,\end{array}$$

(36)

and from (33), we obtain the bound

$$\parallel {\mathbf{p}}^{*}\left(t\right)-{\mathbf{p}}^{**}\left(t\right)\parallel \le 2^N{e}^{-t}\parallel {\mathbf{p}}^{*}\left(0\right)-{\mathbf{p}}^{**}\left(0\right)\parallel .$$

(37)

By plugging $N=100$ into the last inequality, one has that $\parallel {\mathbf{p}}^{*}\left(t\right)-{\mathbf{p}}^{**}\left(t\right)\parallel \le {10}^{-8}$ for $t\ge 90$; i.e., starting from the instant $t=90$, the distribution of the considered CTMC can be regarded as limiting (with the error not greater than ${10}^{-8}$). The behaviour of the (two largest) probabilities $p_0\left(t\right)$ and $p_1\left(t\right)$ under two different initial conditions can be seen in Figure 1 and Figure 2. Since the upper bounds in Theorem 1 are not tight, the value $t=90$ is clearly a pessimistic estimate. The system “forgets” its initial state much earlier and already, beyond $t>10$, the influence of the initial conditions vanishes (see Figure 3 and Figure 4). The limiting distribution of $X\left(t\right)$ is periodic with the period equal to 2 (being the smallest common multiple of the periods of $\lambda \left(t\right)$ and $\mu \left(t\right)$). The behaviour of the probabilities $p_0\left(t\right)$ and $p_1\left(t\right)$ on a single period, which covers the transition to the limiting regime, is depicted in Figure 5 and Figure 6.

8. Conclusions

Although the convergence bounds, if obtained using the proposed technique, are, in general, not tight, they can cover quite general CTMCs, which are beyond the scope of other methods. For CTMCs with countable state spaces, the logarithmic norm method proved to be useful in establishing the truncation bounds for numerical solutions of systems of ODEs with an infinite number of equations. Therefore, the generalization of the proposed technique to the case of the countable state space is a promising direction for further research.