2.1. Basic Information about Multidimensional Continuous-Time Markov Chains under Study
The operation of many queueing systems can be described by a suitably constructed multidimensional continuous-time
The first component of the
has a countable state space,
and corresponds to the current number of customers in the queueing system, buffer, orbit or network. The process
describes the transitions of a finite component of the
. The process
having a finite state space indeed may be a whole finite set of finite components representing various auxiliary processes, e.g., the number of busy or broken servers in multi-server systems, the state of the underlying process of arrivals (if the arrivals occur in the Markov arrival process (
) or batch Markov arrival process (
) or marked Markov arrival process (
), etc.), the state of the underlying process of service (if the service time has a phase type,
distribution, or service is defined by the Markov service process,
), the random environment, which has an impact on the system operation or the number of customers at other stages of a tandem system with finite intermediate buffers, etc. An account of the physical meaning of these components is very important for writing down the generator of the
. However, for the purposes of this paper, we assume that the values of these finite components are enumerated in some order. Thus, without the loss of generality, for the simplicity of denotations, in
Section 2, we consider the case of only one finite component
In
Section 3 and
Section 4 devoted to retrial queues, we consider the case where the finite component
is two-dimensional.
The set of states of the
having the fixed value, say,
i of the first, countable, component is called level
i of the
We suppose that there exists a finite number
such that the cardinalities of all levels
i such that
are equal. In particular, in
Section 2, we will assume that there exists an integer number
N such that the component
of the
admits, for any
and all
, the values in the set
The cardinality of the levels
i for
is equal to
The levels having numbers
can have various dimensions, and we do not impose any specific assumptions about the behavior of the
for these levels except the obvious requirement of the boundedness of the transition rates from these levels.
Let
be the generator of the
Within this paper, we assume that this generator has the upper-Hessenberg structure
where the matrices
consist of the entries
defining, except the diagonal entries
, transition rates from the state
to the state
The mentioned diagonal entries are negative. Their moduli define the rates of the
exit from the corresponding state. For
are square matrices of size
Here,
O denotes zero matrix. By
I, we will denote the identity matrix. If it is necessary, the size of the matrix can be indicated by the subscript.
The popular partial case suggests that the matrix is block tri-diagonal. The s having a generator of a block tri-diagonal structure are called Quasi-Birth-and-Death processes ().
2.2. Level-Independent Quasi-Birth-and-Death Processes and Type Markov Chains
The most well studied in the literature classes of s having the structure of form (1) are the level-independent and type s. These classes assume that the matrices of transition rates are block quasi-Toeplitz, i.e., the value of the block for depends on the difference but does not depend on i and j separately. The name quasi-Toeplitz goes back to the concept of the Toeplitz matrix. The prefix “quasi” reflects the possibility of a violation of the Toeplitz property of the generator for some low levels.
For
type
s, there exist matrices
such that
For level-independent
s,
for
For brevity, we will denote
The classes of the level-independent
and
type
s were investigated in detail by M. Neuts; see, e.g., his seminal books [
10,
11]. Following M. Neuts, we assume that the matrix
is irreducible.
The necessary and sufficient condition for the ergodicity of type s is given by M. Neuts in the following form.
Lemma 1. type is ergodic if and only if the inequalityholds good where the row vector is the unique solution to the systemwhere is the zero row vector and the column vector has all entries equal to 1. The following statement immediately follows from this lemma.
Corollary 1. The level-independent is ergodic if and only if the inequality holds good.
Sometimes, in application to a concrete queueing system, it is possible to analytically obtain the vector and reduce inequalities (2) or (3) to a simple scalar form.
2.3. Level-Dependent Quasi-Birth-and-Death Processes and Type Markov Chains, Asymptotically Quasi-Toeplitz Markov Chains
The classes of the level-independent
and
type
s are extremely useful for the analysis of a variety of queueing models, in particular, various queueing models with an infinite buffer. However, the inherent feature of many important queueing systems is that the
describing the behavior of the system is level-dependent. This implies that the blocks of the corresponding generator describing transition rates from the level
i to the level
j depend not only on the difference
but also on
i and
j separately. It is mentioned in [
12] that level-dependent
s are often more realistic and, while efficient and stable numerical solution techniques are available for level-independent
s, there are only a few approaches that try to exploit the block structure in the level-dependent case.
As the most important queueing models described by the level-dependent
s, retrial queueing models and queues with customer impatience have to be mentioned. More information about the retrial queues, real-world examples and known results can be found, e.g., in the books [
13,
14] and papers [
15,
16,
17,
18,
19].
The importance of retrial queueing models stems from their suitability for modeling various real-world systems, including contact centers, delivery systems and the extremely popular wireless communication networks. The total intensity of retrials of customers staying in the orbit in the overwhelming majority of real systems and networks depends on the number of these customers. This makes the describing the behavior of the system level-dependent.
The impatience (reneging, abandonment, etc.) of customers also makes the M describing system behavior level-dependent.
Due to the practical importance of the analysis of level-dependent
s, the notion of asymptotically quasi-Toeplitz Markov chains (
s) was introduced in the paper [
20]. The rough intuitive definition of
s is as follows. The
is an
with the block upper-Hessenberg structure (1) of a generator that does not possess the quasi-Toeplitz property; however, in asymptotic ones, for very large values of the countable component of the chain, this property appears. A more exact and formal definition of
is given below.
The main motivation for introducing
s is the necessity of considering retrial queues, with the retrial rate proportional to the number of customers in orbit, the
arrival process and the phase-type distribution of service times, see [
21]. The significant difference between a system with an infinite buffer and a similar system with retrials is that in the former system, a new customer is immediately picked up for service from the buffer when some server is released. In an analogous situation in the latter system, there exists an interval during which the server (or servers) remains idle despite the customer’s presence in the orbit. This period finishes via a new primary customer arrival or the retrial of a customer from the orbit. When the number of customers staying in the orbit infinitely increases, such a period becomes shorter and completely disappears in the limit. Therefore, the
behaves in limit exactly as the corresponding limiting quasi-Toeplitz
.
In the case when the generator of the is block tri-diagonal, the is a special case of the level-dependent The difference is that no assumptions about the blocks of a generator are made for the level-dependent , while their asymptotic behavior is suggested for
It is worth noting that, due to the absence of a quasi-Toeplitz property of the s, which describes many queueing systems with customer retrials and (or) impatience, the problem of solving the infinite system of equilibrium equations for the stationary probabilities of the chain is quite difficult. Therefore, many researchers impose, from the early beginning, quite unrealistic assumptions about the considered system, like the orbit capacity is finite, and the rate of retrials from the orbit is constant, independent of the current number of customers in the orbit, etc.
Some other researchers solve the infinite system of equilibrium equations via its truncation, see, e.g., [
22]. The worst case, from a mathematical point of view, is when this is direct (brute force) truncation. Some equations are cut, and the remaining finite system is solved with the use of a computer. The better case is when the researchers use so-called soft truncation. One of the possible ways for soft truncation was offered by M.F. Neuts and B.M. Rao in [
23]. This soft truncation suggests that the blocks of the generator for levels exceeding some fixed threshold become constant, independent of the level. In application to the analysis of a multi-server retrial queue, this means that after the number of customers in the orbit reaches some threshold (and until it drops below this threshold), the retrial rate becomes constant. But this suggestion is definitely not realistic because, usually, the total retrial rate is proportional to the number of customers staying in the orbit. When soft truncation is implemented, the results from [
10] can be applied. If the generator is block tri-diagonal, i.e., the
is the
the vectors of the stationary probabilities of the states that belong to high levels have a matrix geometric form. When the truncation threshold is chosen suitably, the algorithm from [
23] can give satisfactory results.
However, to apply this algorithm, it is necessary first to prove that, under the fixed set of transition rates, the considered
is ergodic and that the computed distribution is indeed the stationary distribution of the
Unfortunately, the paper [
23] does not contain information about the conditions for ergodicity of the
Some researchers derive the ergodicity condition via the use of the results from [
10] for level-independent
. But the obtained condition is the ergodicity condition for the
with other dynamics of the
when the current level of the
is above the truncation threshold. Obviously, it is not the ergodicity condition for the initial level-dependent
The problem of the derivation of ergodicity conditions in the case of level-dependent s is very important but is not sufficiently addressed in the existing literature.
The condition given for the level-dependent
s in [
24] is not a constructive one. It is given as a requirement for the convergence of some matrix series, the terms of which contain the infinite set of matrices (denoted as
in [
24]) that are formally computed recursively. In the case of level-independent
the recursion turns to the quadratic matrix equation. As it is known, see [
10], the solution of this equation with the required properties exists only if the
is ergodic. Therefore, in the more complicated case of level-dependent
considered here, the situation is more difficult; the existence of a solution to the infinite recursion has to be justified, and at least the ergodicity of
has to be postulated. Thus, there is an evident vicious circle. To check the ergodicity, it is required to compute the matrices
which, in turn, may have a chance to be computed only if the
is ergodic.
2.4. Conditions for Ergodicity and Non-Ergodicity of Asymptotically Quasi-Toeplitz Markov Chains
Constructive sufficient conditions for the ergodicity and non-ergodicity of
s, a special case of which is an important class of level-dependent
were presented in [
20]. Here, we briefly reproduce the results relevant to our analysis from [
20].
According to the definition of s, an belongs to the class of s if
- (A)
Its generator has the upper-Hessenberg structure (1);
- (B)
The following matrices
exist:
where
if
and
; otherwise,
is the diagonal matrix with the diagonal entries defined by the moduli of the diagonal entries of the matrix
In other words,
where ∘ is the Hadamard product of matrices symbol, see, e.g., [
25], and the matrix
is the stochastic one;
- (C)
Some technical assumptions related to the requirement of the finiteness of the average size of the jump-up of the level of
(see Theorem 4 in [
20]) are fulfilled. These assumptions are evidently implemented, e.g., under the suggestion that
for
where
K is a finite integer,
Thus, below, we impose this suggestion.
A sufficient condition for the ergodicity of
s proven in [
20] is given as follows.
Let us introduce the matrix generating function Ergodicity conditions are different depending on the irreducibility or reducibility of the matrix It is worth noting that although we supposed above that the matrix is irreducible, the matrix can be (and often is) reducible. Therefore, two variants of the ergodicity condition have to be analysed.
Lemma 2. If the matrix is irreducible, the sufficient condition for the ergodicity of s is the fulfillment of the inequalitywhere the row vector is the unique solution of the system If the matrix
is reducible, then, by means of the coordinated permutation of rows and columns, the matrix
can be represented in the normal form, see [
26],
where
are irreducible stochastic matrices, the matrices
are irreducible matrices, and for each
at least one of the matrices
is non-zero.
Correspondingly to the normal form of the matrix all matrices can also be represented in a similar form. In particular, we denote by the diagonal blocks of the matrix
According to [
20], the following statement is true.
Lemma 3. In the case of the reducible matrix , the sufficient condition for the ergodicity of s is the fulfillment of all m inequalitieswhere the row vector is the unique solution of the system The use of Lemmas 2 and 3 allows for the determination of ergodicity conditions for various queueing systems. Sometimes, these conditions can be easily verified numerically. Finite systems (5) or (7) of the linear algebraic equations are solved, and their solutions are substituted into inequalities (6) or (7). Sometimes, the inequalities can be reduced to a nice scalar form.
However, the application of these conditions to the analysis of concrete queueing models requires preliminary verification that the describing a queueing model indeed belongs to the class of s. To this end, a computation of the blocks of the one-step transition probability matrix for the limiting discrete-time jump is necessary. It may not be easy.
It appears that if the customers arriving at the system can balk (abandon) the system with the probability tending to 1 when queue length upon arrival infinitely increases, or if the customers waiting in the system are impatient or non-persistent, sometimes, this verification and the other steps for the proof that the considered is ergodic are not necessary. It can be stated that the considered belongs to the class of s and is ergodic (the corresponding queueing system has a stationary regime of operation) for any set of the system parameters.
Here, we present the results, which allow us to skip, under quite non-restrictive assumptions, the necessity of proving the affiliation of the considered
to the class of
s, including the computation of the matrices
This justifies the direct use of the algorithms developed for the computation of the stationary distribution of
s in [
20,
27,
28,
29] to compute the stationary distribution of the considered queueing system. The use of the algorithms from [
20,
27] requires certain analytical derivations (calculation of the limits of some matrices) to obtain the explicit form of the blocks
of the one-step transition probability matrix of the limiting discrete-time
for the
. The algorithms proposed in [
28,
29] do not need such derivations because they operate directly only with the blocks of the generator of the
. It should also be stressed that the results of this paper render unnecessary the derivation and control of the fulfillment of an ergodicity condition for
s describing various queueing systems because it is shown here that these
s are always ergodic due to customers’ balking, impatience or non-persistence.