Matrix Algebraic Properties of the Fisher Information Matrix of Stationary Processes

Abstract

In this survey paper, a summary of results which are to be found in a series of papers, is presented. The subject of interest is focused on matrix algebraic properties of the Fisher information matrix (FIM) of stationary processes. The FIM is an ingredient of the Cramér-Rao inequality, and belongs to the basics of asymptotic estimation theory in mathematical statistics. The FIM is interconnected with the Sylvester, Bezout and tensor Sylvester matrices. Through these interconnections it is shown that the FIM of scalar and multiple stationary processes fulfill the resultant matrix property. A statistical distance measure involving entries of the FIM is presented. In quantum information, a different statistical distance measure is set forth. It is related to the Fisher information but where the information about one parameter in a particular measurement procedure is considered. The FIM of scalar stationary processes is also interconnected to the solutions of appropriate Stein equations, conditions for the FIM to verify certain Stein equations are formulated. The presence of Vandermonde matrices is also emphasized. 15A23, 15A24, 15B99, 60G10, 62B10, 62M20.

1. Introduction

In this survey paper, a summary of results derived and described in a series of papers, is presented. It concerns some matrix algebraic properties of the Fisher information matrix (abbreviated as FIM) of stationary processes. An essential property emphasized in this paper concerns the matrix resultant property of the FIM of stationary processes. To be more explicit, consider the coefficients of two monic polynomials p(z) and q(z) of finite degree, as the entries of a matrix such that the matrix becomes singular if and only if the polynomials p(z) and q(z) have at least one common root. Such a matrix is called a resultant matrix and its determinant is called the resultant. The Sylvester, Bezout and tensor Sylvester matrices have such a property and are extensively studied in the literature, see e.g., [1–3]. The FIM associated with various stationary processes will be expressed by these matrices. The derived interconnections are obtained by developing the necessary factorizations of the FIM in terms of the Sylvester, Bezout and tensor Sylvester matrices. These factored forms of the FIM enable us to show that the FIM of scalar and multiple stationary processes fulfill the resultant matrix property. Consequently, the singularity conditions of the appropriate Fisher information matrices and Sylvester, Bezout and tensor Sylvester matrices coincide, these results are described in [4–6].

A statistical distance measure involving entries of the FIM is presented and is based on [7]. In quantum information, a statistical distance measure is set forth, see [8–10], and is related to the Fisher information but where the information about one parameter in a particular measurement procedure is considered. This leads to a challenging question that can be presented as, can the existing distance measure in quantum information be developed at the matrix level?

The matrix Stein equation, see e.g., [11], is associated with the Fisher information matrices of scalar stationary processes through the solutions of the appropriate Stein equations. Conditions for the Fisher information matrices or associated matrices to verify certain Stein equations are formulated and proved in this paper. The presence of Vandermonde matrices is also emphasized. The general and more detailed results are set forth in [12] and [13]. In this survey paper it is shown that the FIM of linear stationary processes form a class of structured matrices. Note that in [14], the authors emphasize that statistical problems related to stationary processes have been treated successfully with the aid of Toeplitz forms. This paper is organized as follows. The various stationary processes, considered in this paper, are presented in Section 2, the Fisher information matrices of the stationary processes are displayed in Section 3. Section 3 sets forth the interconnections between the Fisher information matrices and the Sylvester, Bezout, tensor Sylvester matrices, and solutions to Stein equations. A statistical distance measure is expressed in terms of entries of a FIM.

2. The Linear Stationary Processes

In this section we display the class of linear stationary processes whose corresponding Fisher information matrix shall be investigated in a matrix algebraic context. But first some basic definitions are set forth, see e.g., [15].

If a random variable X is indexed to time, usually denoted by t, the observations {X_t, t ∈ } is called a time series, where is a time index set (for example, = ℤ, the integer set).

Definition 2.1

A stochastic process is a family of random variables {X_t, t ∈ } defined on a probability space {Ω, $ℱ$, ℘}.

Definition 2.2

The Autocovariance function. If {X_t, t ∈ } is a process such that Var(X_t) < ∞ (variance) for each t, then the autocovariance function γ_X (·, ·) of {X_t} is defined by γ_X (r, s) = Cov (X_r, X_s) = [(X_r − X_r) (X_s − X_s)], r, s ∈ ℤ and represents the expected value.

Definition 2.3

Stationarity. The time series {X_t, t ∈ ℤ}, with the index set ℤ ={0,±}1,±}2, . . .}, is said to be stationary if

(i)

|X_t|² < ∞

(ii)

(X_t) = m for all t ∈ ℤ, m is the constant average or mean

(iii)

γ_X (r, s) = γ_X (r + t, s + t) for all r, s, t ∈ ℤ,

From Definition 2.3 can be concluded that the joint probability distributions of the random variables {X₁, X₂, . . . X_tn} and {X_1+k, X_2+k, . . . X_tn+k} are the same for arbitrary times t₁, t₂, . . . , t_n for all n and all lags or leads k = 0, ±}1, ±}2, . . .. The probability distribution of observations of a stationary process is invariant with respect to shifts in time. In the next section the linear stationary processes that will be considered throughout this paper are presented.

2.1. The Vector ARMAX or VARMAX Process

We display one of the most general linear stationary process called the multivariate autoregressive, moving average and exogenous process, the VARMAX process. To be more specific, consider the vector difference equation representation of a linear system {y(t), t ∈ ℤ}, of order (p, r, q),

$$\sum _{j=0}^p{A}_j\hspace{0.17em}y(t-j)=\sum _{j=0}^r{C}_j\hspace{0.17em}x(t-j)+\sum _{j=0}^q{B}_j\hspace{0.17em}\varepsilon (t-j),\hspace{0.17em}t\in \mathbb{Z}$$

(1)

where y(t) are the observable outputs, x(t) the observable inputs and ɛ(t) the unobservable errors, all are n-dimensional. The acronym VARMAX stands for vector autoregressive-moving average with exogenous variables. The left side of (1) is the autoregressive part the second term on the right is the moving average part and x(t) is exogenous. If x(t) does not occur the system is said to be (V)ARMA. Next to exogenous, the input x(t) is also named the control variable, depending on the field of application, in econometrics and time series analysis, e.g., [15], and in signal processing and control, e.g., [16,17]. The matrix coefficients, A_j ∈ ℝ^n×n, C_j ∈ ℝ^n×n, and B_j ∈ ℝ^n×n are the associate parameter matrices. We have the property A₀ ≡ B₀ ≡ C₀ ≡ I_n.

Equation (1) can compactly be written as

$$A(z)\hspace{0.17em}y(t)=C(z)\hspace{0.17em}x(t)+B(z)\hspace{0.17em}\varepsilon (t)$$

(2)

where

$$A(z)=\sum _{j=0}^p{A}_j\hspace{0.17em}{z}^j;\hspace{0.17em}C(z)=\sum _{j=0}^r{C}_j\hspace{0.17em}{z}^j;\hspace{0.17em}B(z)=\sum _{j=0}^q{B}_j\hspace{0.17em}{z}^j$$

we use z to denote the backward shift operator, for example z x_t = x_t−1. The matrix polynomials A(z), B(z) and C(z) are the associated autoregressive, moving average matrix polynomials, and the exogenous matrix polynomial respectively of order p, q and r respectively. Hence the process described by Equation (2) is denoted as a VARMAX(p, r, q) process. Here z ∈ ℂ with a duplicate use of z as an operator and as a complex variable, which is usual in the signal processing and time series literature, e.g., [15,16,18]. The assumptions Det(A(z)) ≠ 0, such that |z| ≤ 1 and Det(B(z)) ≠ 0, such that |z| < 1 for all z ∈ ℂ, is imposed so that the VARMAX(p, r, q) process (2) has exactly one stationary solution and the condition Det(B(z)) ≠ 0 implies the invertibility condition, see e.g., [15] for more details. Under these assumptions, the eigenvalues of the matrix polynomials A(z) and B(z) lie outside the unit circle. The eigenvalues of a matrix polynomial Y (z) are the roots of the equation Det(Y (z)) = 0, Det(X) is the determinant of X. The VARMAX(p, r, q) stationary process (2) is thoroughly discussed in [15,18,19].

The error {ɛ(t), t ∈ ℤ} is a collection of uncorrelated zero mean n-dimensional random variables each having positive definite covariance matrix ∑ and we assume, for all s, t, { x(s) ɛ^T(t)} = 0, where X^T denotes the transposition of matrix X and represents the expected value under the parameter ϑ. The matrix ϑ represents all the VARMAX(p, r, q) parameters, with the total number of parameters being n²(p + q + r). For different purposes which will be specified in the next sections, two choices of the parameter structure are considred. First, the parameter vector ϑ ∈ ℝⁿ²(^p+q+r)×1 is defined by

$$\vartheta =\text{vec}\hspace{0.17em}\{A_1,A_2,\dots ,A_p,C_1,C_2,\dots ,C_r,B_1,B_2,\dots ,B_q\}$$

(3)

The vec operator transforms a matrix into a vector by stacking the columns of the matrix one underneath the other according to vec $X=\text{col}{(\text{col}{(X_{ij})}_{i=1}^n)}_{j=1}^n$, see e.g., [2,20]. A different choice is set forth, when the parameter matrix ϑ ∈ ℝ^n×n(p+q+r) is of the form

$$\vartheta =({\vartheta }_1\hspace{0.17em}{\vartheta }_2\dots {\vartheta }_p\hspace{0.17em}{\vartheta }_{p+1}\hspace{0.17em}{\vartheta }_{p+2}\dots {\vartheta }_{p+r}\hspace{0.17em}{\vartheta }_{p+r+1}\hspace{0.17em}{\vartheta }_{p+r+2}\dots {\vartheta }_{p+r+q})$$

(4)

$$=(A_1\hspace{0.17em}{A}_2\dots A_p\hspace{0.17em}{C}_1\hspace{0.17em}{C}_2\dots C_r\hspace{0.17em}{B}_1\hspace{0.17em}{B}_2\dots B_q)$$

(5)

Representation (5) of the parameter matrix has been used in [21]. The estimation of the matrices A₁, A₂,. . ., A_p, C₁, C₂,. . ., C_r, B₁, B₂, . . ., B_q and ∑ has received considerable attention in the time series and statistical signal processing literature, see e.g., [15,17,19]. In [19], the authors study the asymptotic properties of maximum likelihood estimates of the coefficients of VARMAX(p, r, q) processes, stored in a (ℓ × 1) vector ϑ, where ℓ = n²(p + q + r).

Before describing the control-exogenous variable x(t) used in this survey paper, we shall present the different special cases of the model described in Equations (1) and (2).

2.2. The Vector ARMA or VARMA Process

When the process (2) does not contain the control process x(t) it yields

$$A(z)y(t)=B(z)\varepsilon (t)$$

(6)

which is a vector autoregressive and moving average process, VARMA(p, q) process, see e.g., [15]. The matrix ϑ represents now all the VARMA parameters, with the total number of parameters being n²(p+q). The VARMA(p, q) version of the parameter vector ϑ defined in (3) is then given by

$$\vartheta =\text{vec}\hspace{0.17em}\{A_1,A_2,\dots ,A_p,B_1,B_2,\dots ,B_q\}$$

(7)

A VARMA process equivalent to the parameter matrix (4) is then the n × n(p + q) parameter matrix

$$\vartheta =({\vartheta }_1\hspace{0.17em}{\vartheta }_2\dots {\vartheta }_p\hspace{0.17em}{\vartheta }_{p+1}\hspace{0.17em}{\vartheta }_{p+2}\dots {\vartheta }_{p+q})=(A_1\hspace{0.17em}{A}_2\dots A_p\hspace{0.17em}{B}_1\hspace{0.17em}{B}_2\dots B_q)$$

(8)

A description of the input variable x(t), in Equation (2) follows. Generally, one can assume either that x(t) is non stochastic or that x(t) is stochastic. In the latter case, we assume { x(s) ɛ^T(t)} = 0, for all s, t, and that statistical inference is performed conditionally on the values taken by x(t). In this case it can be interpreted as constant, see [22] for a detailed exposition. However, in the papers referred in this survey, like in [21] and [23], the observed input variable x(t), is assumed to be a stationary VARMA process, of the form

$$\alpha (z)x(t)=\beta (z)\eta (t)$$

(9)

where α(z) and β(z) are the autoregressive and moving average polynomials of appropriate degree and {η(t), t ∈ ℤ} is a collection of uncorrelated zero mean n-dimensional random variables each having positive definite covariance matrix Ω. The spectral density of the VARMA process x(t) is R_x(·)/2π and for a definition, see e.g., [15,16], to obtain

$$R_x(e^{\mathbf{i}\omega })={\alpha }^{-1}(e^{\mathbf{i}\omega })\beta (e^{\mathbf{i}\omega })\mathrm{\Omega }{\beta }^{*}(e^{\mathbf{i}\omega }){\alpha }^{-*}(e^{\mathbf{i}\omega })\mathrm{\hspace{0.17em} }\mathrm{\hspace{0.17em} }\mathrm{\hspace{0.17em} }\omega \in [-\pi ,\pi ]$$

(10)

where i is the imaginary unit with the property i² = −1, ω is the frequency, the spectral density R_x(e^iω) is Hermitian, and we further have, R_x(e^iω) ≥ 0 and ${\int }_{-\pi }^{\pi }{R}_x(e^{\mathbf{i}\omega })d\omega <\infty$. As mentioned above, the basic assumption, x(t) and ɛ(t) are independent or at least uncorrelated processes, which corresponds geometrically with orthogonal processes, holds and X* is the complex conjugate transpose of matrix X.

2.3. The ARMAX and ARMA Processes

The scalar equivalent to the VARMAX(p, r, q) and VARMA(p, q) processes, given by Equations (2) and (6) respectively, shall now be displayed, to obtain for the ARMAX(p, r, q) process

$$a(z)y(t)=c(z)x(t)+b(z)\varepsilon (t)$$

(11)

and for the ARMA(p, q) process

$$a(z)y(t)=b(z)\varepsilon (t)$$

(12)

popularized in, among others, the Box-Jenkins type of time series analysis, see e.g., [15]. Where a(z), b(z) and c(z) are respectively the scalar autoregressive, moving average polynomials and exogenous polynomial, with corresponding scalar coefficients a_j, b_j and c_j,

$$a(z)=\sum _{j=0}^p{a}_j\hspace{0.17em}{z}^j;\hspace{0.17em}c(z)=\sum _{j=0}^r{c}_j\hspace{0.17em}{z}^j;\hspace{0.17em}b(z)=\sum _{j=0}^q{b}_j\hspace{0.17em}{z}^j$$

(13)

Note that as in the multiple case, a₀ = b₀ = 1. The parameter vector, ϑ, for the processes, Equations (11) and (12) is then

$$\vartheta =\{a_1,a_2,\dots ,a_p,c_1,c_2,\dots ,c_r,b_1,b_2,\dots ,b_q\}$$

(14)

and

$$\vartheta =\{a_1,a_2,\dots ,a_p,b_1,b_2,\dots ,b_q\}$$

(15)

respectively.

In the next section the matrix algebraic properties of the Fisher information matrix of the stationary processes (2), (6), (11) and (12) will be verified. Interconnections with various known structured matrices like the Sylvester resultant matrix, the Bezout matrix and Vandermonde matrix are set forth. The Fisher information matrix of the various stationary processes is also expressed in terms of the unique solutions to the appropriate Stein equations.

3. Structured Matrix Properties of the Asymptotic Fisher Information Matrix of Stationary Processes

The Fisher information is an ingredient of the Cramér-Rao inequality, also called by some the Cauchy-Schwarz inequality in mathematical statistics, and belongs to the basics of asymptotic estimation theory in mathematical statistics. The Cramér-Rao theorem [24] is therefore considered. When assuming that the estimators of ϑ, defined in the previuos sections, are asymptotically unbiased, the inverse of the asymptotic information matrix yields the Cramér-Rao bound, and provided that the estimators are asymptotically efficient, the asymptotic covariance matrix then verifies the inequality

$$\text{Cov}\hspace{0.17em}\left(\widehat{\vartheta }\right)\succcurlyeq {ℐ}^{-1}\left(\widehat{\vartheta }\right)$$

here $ℐ$ (ϑ̂) is the FIM, Cov (ϑ̂) is the covariance of ϑ̂, the unbiased estimator of ϑ, for a detailed fundamental statistical analysis, see [25,26]. The FIM equals the Cramér-Rao lower bound, and the subject of the FIM is also of interest in the control theory and signal processing literature, see e.g., [27]. Its quantum analog was introduced immediately after the foundation of mathematical quantum estimation theory in the 1960’s, see [28,29] for a rigorous exposition of the subject. More specifically, the Fisher information is also emphasized in the context of quantum information theory, see e.g., [30,31]. It is clear that the Cramér-Rao inequality takes a lot of attention because it is located on the highly exciting boundary of statistics, information and quantum theory and more recently matrix theory. In the next sections, the Fisher information matrices of linear stationary processes will be presented and its role as a new class of structured matrices will be the subject of study.

When time series models are the subject, using Equation (2) for all t ∈ ℤ to determine the residual ɛ(t) or ɛ_t(ϑ), to emphasize the dependency on the parameter vector ϑ, and assuming that x(t) is stochastic and that (y(t), x(t)) is a Gaussian stationary process, the asymptotic FIM $ℱ$(ϑ) is defined by the following (ℓ × ℓ) matrix which does not depend on t

$$\mathcal{F}(\vartheta )=\mathbb{E}\hspace{0.17em}\left\{{\left(\frac{\partial {\varepsilon }_t(\vartheta )}{\partial {\vartheta }^{\top }}\right)}^{\top }{\mathrm{\Sigma }}^{-1}\hspace{0.17em}\left(\frac{\partial {\varepsilon }_t(\vartheta )}{\partial {\vartheta }^{\top }}\right)\right\}$$

(16)

where the (v × ℓ) matrix ∂(·)/∂ϑ ^T, the derivative with respect to ϑ ^T, for any (v × 1) column vector (·) and ℓ is the total number of parameters. The derivative with respect to ϑ ^T is used for obtaining the appropriate dimensions. Equality (16) is used for computing the FIM of the various time series processes presented in the previous sections and appropriate definitions of the derivatives are used, especially for the multivariate processes (2) and (6), see [21,22].

3.1. The Fisher Information Matrix of an ARMA(p, q) Process

In this section, the focus is on the FIM of the ARMA process (12). When ϑ is given in Equation (15), the derivatives in Equation (16) are at the scalar level

$$\frac{\partial {\varepsilon }_t(\vartheta )}{\partial a_j}=\frac{1}{a(z)}{\varepsilon }_{t-j}\mathrm{\hspace{0.17em} }\mathrm{\hspace{0.17em} }\mathrm{\hspace{0.17em} }\text{for}\hspace{0.17em}j=1,\dots ,p\hspace{0.17em}\text{and}\frac{\partial {\varepsilon }_t(\vartheta )}{\partial b_k}=-\frac{1}{b(z)}{\varepsilon }_{t-k}\mathrm{\hspace{0.17em} }\text{for}\hspace{0.17em}k=1,\dots ,q$$

when combined for all j and k, the FIM of the ARMA process (12) with the variance of the noise process ɛ_t(ϑ) equal to one, yields the block decomposition, see [32]

$$\mathcal{F}(\vartheta )=\left(\begin{array}{cc}{\mathcal{F}}_{aa}(\vartheta )& {\mathcal{F}}_{ab}(\vartheta )\\ {\mathcal{F}}_{ba}(\vartheta )& {\mathcal{F}}_{bb}(\vartheta )\end{array}\right)$$

(17)

The expressions of the different blocks of the matrix $ℱ$(ϑ) are

$${\mathcal{F}}_{aa}(\vartheta )=\frac{1}{2\pi \mathbf{i}}\underset{\mid z\mid =1}{\oint }\frac{u_p(z)u_p^{\top }(z^{-1})}{a(z)a(z^{-1})}\frac{dz}{z}=\frac{1}{2\pi \mathbf{i}}\underset{\mid z\mid =1}{\oint }\frac{u_p(z)v_p^{\top }(z)}{a(z)\widehat{a}(z)}dz$$

(18)

$${\mathcal{F}}_{ab}(\vartheta )=-\frac{1}{2\pi \mathbf{i}}\underset{\mid z\mid =1}{\oint }\frac{u_p(z)u_q^{\top }(z^{-1})}{a(z)b(z^{-1})}\frac{dz}{z}=-\frac{1}{2\pi \mathbf{i}}\underset{\mid z\mid =1}{\oint }\frac{u_p(z)v_q^{\top }(z)}{a(z)\widehat{b}(z)}dz$$

(19)

$${\mathcal{F}}_{ba}(\vartheta )=-\frac{1}{2\pi \mathbf{i}}\underset{\mid z\mid =1}{\oint }\frac{u_q(z)u_p^{\top }(z^{-1})}{a(z^{-1})b(z)}\frac{dz}{z}=-\frac{1}{2\pi \mathbf{i}}\underset{\mid z\mid =1}{\oint }\frac{u_q(z)v_p^{\top }(z)}{\widehat{a}(z)b(z)}dz$$

(20)

$${\mathcal{F}}_{bb}(\vartheta )=\frac{1}{2\pi \mathbf{i}}\underset{\mid z\mid =1}{\oint }\frac{u_q(z)u_q^{\top }(z^{-1})}{b(z)b(z^{-1})}\frac{dz}{z}=\frac{1}{2\pi \mathbf{i}}\underset{\mid z\mid =1}{\oint }\frac{u_q(z)v_q^{\top }(z)}{b(z)\widehat{b}(z)}dz$$

(21)

where the integration above and everywhere below is counterclockwise around the unit circle. The reciprocal monic polynomials â(z) and b̂(z) are defined as â(z) = z^pa(z⁻¹) and b̂ (z) = z^qb(z⁻¹) and ϑ =(a₁, . . . , a_p, b₁, . . . , b_q) ^T introduced in (15). For each positive integer k we have u_k(z) = (1, z, z², . . . , z^k−1) ^T and v_k(z) = z^k−1u_k(z⁻¹). Considering the stability condition of the ARMA(p, q) process implies that all the roots of the monic polynomials a(z) and b(z) lie outside the unit circle. Consequently, the roots of the polynomials â(z) and b̂(z) lie within the unit circle and will be used as the poles for computing the integrals (18)–(21) when Cauchy’s residue theorem is applied. Notice that the FIM $ℱ$(ϑ) is symmetric block Toeplitz so that ${\mathcal{F}}_{ab}(\vartheta )={\mathcal{F}}_{ba}^{\top }(\vartheta )$ and the integrands in (18)–(21) are Hermitian. The computation of the integral expressions, (18)–(21) is easily implementable by using the standard residue theorem. The algorithms displayed in [33] and [22] are suited for numerical computations of among others the FIM of an ARMA(p, q) process.

3.2. The Sylvester Resultant Matrix - The Fisher Information Matrix

The resultant property of a matrix is considered, in order to show that the FIM $ℱ$(ϑ) has the matrix resultant property implies to show that the matrix $ℱ$(ϑ) becomes singular if and only if the appropriate scalar monic polynomials â(z) and b̂(z) have at least one common zero. To illustrate the subject, the following known property of two polynomials is set forth. The greatest common divisor (frequently abbreviated as GCD) of two polynomials is a polynomial, of the highest possible degree, that is a factor of both the two original polynomials, the roots of the GCD of two polynomials are the common roots of the two polynomials. Consider the coefficients of two monic polynomials p(z) and q(z) of finite degree, as the entries of a matrix such that the matrix becomes singular if and only if the polynomials p(z) and q(z) have at least one common root. Such a matrix is called a resultant matrix and its determinant is called the resultant. Therefore we present the known (p + q) × (p + q) Sylvester resultant matrix of the polynomials a and b, see e.g., [2], to obtain

$$\mathcal{S}(a,b)=\left(\begin{array}{ccccccc}1& a_1& \cdots & a_p& 0& \cdots & 0\\ 0& \ddots & \ddots & \hspace{0.17em}& \ddots & \ddots & ⋮\\ ⋮& \ddots & \ddots & \ddots & \hspace{0.17em}& \ddots & 0\\ 0& \cdots & 0& 1& a_1& \cdots & a_p\\ 1& b_1& \cdots & b_q& 0& \cdots & 0\\ 0& \ddots & \ddots & \hspace{0.17em}& \ddots & \ddots & ⋮\\ ⋮& \ddots & \ddots & \ddots & \hspace{0.17em}& \ddots & 0_{n\times n}\\ 0& \cdots & 0& 1& b_1& \cdots & b_q\end{array}\right)$$

(22)

Consider the q ×(p+q) and p×(p+q) upper and lower submatrices (b) and (−a) of the Sylvester resultant matrix (−b, a) such that

$$\mathcal{S}(b,-a)=\left(\begin{array}{c}{\mathcal{S}}_p(b)\\ -{\mathcal{S}}_q(a)\end{array}\right)$$

(23)

The matrix (a, b) becomes singular in the presence of one or more common zeros of the monic polynomials â(z) and b̂(z), this property is assessed by the following equalities

$$\mathcal{R}(a,b)=\prod _{\begin{array}{l}i=1,\dots ,p\\ j=1,\dots ,q\end{array}}({\alpha }_i-{\beta }_j),\mathcal{R}(b,a)={(-1)}^{pq}\prod _{\begin{array}{l}i=1,\dots ,p\\ j=1,\dots ,q\end{array}}({\alpha }_i-{\beta }_j)$$

(24)

and

$$\mathcal{R}(b,-a)={(-1)}^q\prod _{\begin{array}{l}i=1,\dots ,p\\ j=1,\dots ,q\end{array}}({\beta }_j-{\alpha }_i),\hspace{0.17em}\text{and}\hspace{0.17em}\mathcal{R}(-b,a)={(-1)}^p\prod _{\begin{array}{l}i=1,\dots ,p\\ j=1,\dots ,q\end{array}}({\beta }_j-{\alpha }_i)$$

(25)

where $ℛ$(a, b) is the resultant of â(z) and b̂(z), and is equal to Det (a, b). The string of equalities in (24) and (25) hold since $ℛ$(b, a) = (−1)^pq $ℛ$(a, b), $ℛ$(b, −a) = (−1)^q $ℛ$(b, a), and $ℛ$(−b, a) = (−1)^p $ℛ$(b, a), see [34]. The zeros of the scalar monic polynomials â(z) and b̂(z) are α_i and β_j respectively and are assumed to be distinct. By this is meant, when we have (z − α_i)^n_αi and (z − β_j)^n_βj with the powers n_αi and n_βj both greater than one, that only the distinct roots will be considered free from the corresponding powers. The key property of the classical Sylvester resultant matrix (a, b) is that its null space provides a complete description of the common zeros of the polynomials involved. In particular, in the scalar case the polynomials â(z) and b̂(z) are coprime if and only if (a, b) is non-singular. The following key property of the classical Sylvester resultant matrix (a, b), is given by the well known theorem on resultants, to obtain

$$\text{dim\hspace{0.17em}Ker}\hspace{0.17em}\mathcal{S}(a,b)=\nu (a,b)$$

(26)

where ν(a, b) is the number of common roots of the polynomials â(z) and b̂(z), with counting multiplicities, see e.g., [3]. The dimension of a subspace is represented by dim ( ), Ker (X) is the null space or kernel of the matrix X, denoted by Null or Ker. The null space of an n × n matrix A with coefficients in a field K (typically the field of the real numbers or of the complex numbers) is the set Ker A = {x ∈ Kⁿ: Ax = 0}, see e.g., [1,2,20].

In order to prove that the FIM $ℱ$(ϑ) fulfills the resultant matrix property, the following factorization is derived, Lemma 2.1 in [5],

$$\mathcal{F}(\vartheta )=\mathcal{S}(b,-a)\mathcal{P}(\vartheta ){\mathcal{S}}^{\top }(b,-a)$$

(27)

where the matrix ℘(ϑ) ∈ ℝ^(p+q)×(p+q) admits the form

$$\mathcal{P}(\vartheta )=\frac{1}{2\pi \mathbf{i}}\underset{\mid z\mid =1}{\oint }\frac{u_{p+q}(z)u_{p+a}^{\top }(z^{-1})}{a(z)b(z)a(z^{-1})b(z^{-1})}\frac{dz}{z}=\frac{1}{2\pi \mathbf{i}}\underset{\mid z\mid =1}{\oint }\frac{u_{p+q}(z)v_{p+q}^{\top }(z)}{a(z)b(z)\widehat{a}(z)\widehat{b}(z)}dz$$

(28)

It is proved in [5] that the symmetric matrix ℘(ϑ) fulfills the property, ℘(ϑ) ≻ O. The factorization (27) allows us to show the matrix resultant property of the FIM, Corollary 2.2 in [5] states.

The FIM of an ARMA(p, q) process with polynomials a(z) and b(z) of order p, q respectively becomes singular if and only if the polynomials â(z) and b̂(z) have at least one common root. From Corollary 2.2 in [5] can be concluded, the FIM of an ARMA(p, q) process and the Sylvester resultant matrix (−b, a) have the same singularity property. By virtue of (26) and (27) we will specify the dimension of the null space of the FIM $ℱ$(ϑ), this is set forth in the following lemma.

Lemma 3.1

Assume that the polynomials â(z) and b̂(z) have ν(a, b) common roots, counting multiplicities. The factorization (27) of the FIM and the property (26) enable us to prove the equality

$$dim\mathrm{\hspace{0.17em} }(Ker\hspace{0.17em}\mathcal{F}(\vartheta ))=dim\mathrm{\hspace{0.17em} }(Ker\hspace{0.17em}\mathcal{S}(b,-a))=\nu (a,b)$$

(29)

Proof

The matrix ℘(ϑ) ∈ ℝ^(p+q)×(p+q), given in (27), fulfills the property of positive definiteness, as proved in [5]. This implies that a Cholesky decomposition can be applied to ℘(ϑ), see [35] for more details, to obtain ℘(ϑ) =L^T(ϑ)L(ϑ), where L(ϑ) is a ℝ^(p+q)×(p+q) upper triangular matrix that is unique if its diagonal elements are all positive. Consequently, all its eigenvalues are then positive so that the matrix L(ϑ) is also positive definite. Factorization of (27) now admits the representation

$$\mathcal{F}(\vartheta )=\mathcal{S}(b,-a)L^{\top }(\vartheta )L(\vartheta ){\mathcal{S}}^{\top }(b,-a)$$

(30)

and taking the property, if A is an m× n matrix, then Ker (A) = Ker (A^TA), into account, yields when applied to (30)

$$\text{Ker}\hspace{0.17em}\mathcal{F}(\vartheta )=\text{Ker}\hspace{0.17em}\mathcal{S}(b,-a)L^{\top }(\vartheta )L(\vartheta ){\mathcal{S}}^{\top }(b,-a)=\text{Ker}\hspace{0.17em}L(\vartheta ){\mathcal{S}}^{\top }(b,-a)$$

Assume the vector u ∈ Ker L(ϑ) (b, −a), such that L(ϑ) (b, −a)u = 0 and set (b, −a)u = v = ⇒ L(ϑ)v = 0, since the matrix L(ϑ) ≻ O = ⇒ v = 0, this implies (b, −a)u = 0 = ⇒ u ∈ Ker (b, −a). Consequently,

$$\text{Ker}\hspace{0.17em}\mathcal{F}(\vartheta )=\text{Ker}\hspace{0.17em}{\mathcal{S}}^{\top }(b,-a)$$

(31)

We will now consider the Rank-Nullity Theorem, see e.g., [1], if A is an m × n matrix, then

$$\text{dim\hspace{0.28em}}(\text{Ker}\hspace{0.17em}A)+\text{dim\hspace{0.28em}}(\text{Im}\hspace{0.17em}A)=n$$

and the property dim (Im A) = dim (Im A^T). When applied to the (p + q) × (p + q) matrix (b, −a), it yields

$$\text{dim\hspace{0.28em}}(\text{Ker}\hspace{0.17em}\mathcal{S}(b,-a))=\text{dim\hspace{0.28em}}(\text{Ker}\hspace{0.17em}{\mathcal{S}}^{\top }(b,-a))\Rightarrow \text{dim\hspace{0.28em}}(\text{Ker}\hspace{0.17em}\mathcal{F}(\vartheta ))=\text{dim\hspace{0.28em}}(\text{Ker}\hspace{0.17em}\mathcal{S}(b,-a))$$

which completes the proof.

Notice that the dimension of the null space of matrix A is called the nullity of A and the dimension of the image of matrix A, dim (Im A), is termed the rank of matrix A. An alternative proof to the one developed in Corollary 2.2 in [5], is given in a corollary to Lemma 3.1, reconfirming the resultant matrix property of the FIM $ℱ$(ϑ).

Corollary 3.2

The FIM $ℱ$(ϑ) of an ARMA(p, q) process becomes singular if and only if the autoregressive and moving average polynomials â(z) and b̂(z) have at least one common root.

Proof

By virtue of the equality (31) combining with the property Det (b, −a) = Det (b, −a) and the matrix resultant property of the Sylvester matrix (b, −a) yields, Det (b, −a) = 0 ⇔ Ker (b, −a) ≠ {0} if and only if the ARMA(p, q) polynomials â(z) and b̂(z) have at least one common root. Equivalently, Det (b, −a) ≠ 0 ⇔ Ker (b, −a) = {0} if and only if the ARMA(p, q) polynomials â(z) and b̂(z) have no common roots. Consequently, by virtue of the equality Ker $ℱ$(ϑ) =Ker (b, −a) can be concluded, the FIM $ℱ$(ϑ) becomes singular if and only if the ARMA(p, q) polynomials â(z) and b̂(z) have at least one common root. This completes the proof.

3.3. The Statistical Distance Measure and the Fisher Information Matrix

In [7] statistical distance measures are studied. Most multivariate statistical techniques are based upon the concept of distance. For that purpose a statistical distance measure is considered that is a normalized Euclidean distance measure with entries of the FIM as weighting coefficients. The measurements x₁, x₂,. . . , x_n are subject to random fluctuations of different magnitudes and have therefore different variabilities. It is then important to consider a distance that takes the variability of these variables or measurements into account when determining its distance from a fix point. A rotation of the coordinate system through a chosen angle while keeping the scatter of points given by the data fixed, is also applied, see [7] for more details. It is shown that when the FIM is positive definite, the appropriate statistical distance measure is a metric. In case of a singular FIM of an ARMA stationary process, the metric property depends on the rotation angle. The statistical distance measure, is based on m parameters unlike a statistical distance measure introduced in quantum information, see e.g., [8,9], that is also related to the Fisher information but where the information about one parameter in a particular measurement procedure is considered.

The straight-line or Euclidean distance between the stochastic vector $x={\left(\begin{array}{cccc}{x}_1& x_2& \dots & x_n\end{array}\right)}^{\top }$ and fixed vector $y={\left(\begin{array}{cccc}{y}_1& y_2& \dots & y_n\end{array}\right)}^{\top }$ where x, y ∈ ℝⁿ, is given by

$$d(x,y)=\Vert x-y\Vert ={\left({\sum }_{j=1}^n{(x_j-y_j)}^2\right)}^{1/2}$$

(32)

where the metric d(x, y):= ||x−y|| is induced by the standard Euclidean norm || · || on ℝⁿ, see e.g., [2] for the metric conditions.

The observations x₁, x₂, . . . , x_n are used to compute maximum likelihood estimated of the parameters ϑ₁, ϑ₂, . . . , ϑ_m and where m < n. These estimated parameters are random variables, see e.g., [15]. The distance of the estimated vector ϑ ∈ ℝ^m, given in (15), is studied. Entries of the FIM are inserted in the distance measure as weighting coefficients. The linear transformation

$$\tilde{\vartheta }={\mathcal{L}}_i(\phi )\vartheta$$

(33)

is applied, where $ℒ$_i(φ) ∈ ℝ^m×n is the Givens rotation matrix with rotation angle φ, with 0 ≤ φ ≤ 2π and i ∈ {1, . . . , m − 1}, see e.g., [36], and is given by

$${\mathcal{L}}_i(\phi )=\left(\begin{array}{cccc}{I}_{i-1}& 0& 0& 0\\ 0& {(\text{cos}(\phi ))}_{i,i}& {(-\text{sin}(\phi ))}_{i,i+1}& 0\\ 0& {(\text{sin}(\phi ))}_{i+1,i}& {(\text{cos}(\phi ))}_{i+1,i+1}& 0\\ 0& 0& 0& I_{m-i-1}\end{array}\right),\mathrm{\hspace{0.17em} }\mathrm{\hspace{0.17em} }\mathrm{\hspace{0.17em} }0\le \phi \le 2\pi$$

(34)

The following matrix decomposition is applied in order to obtain a transformed FIM

$${\mathcal{F}}_{\phi }(\vartheta )={\mathcal{L}}_i(\phi )\mathcal{F}(\vartheta ){\mathcal{L}}_i^{\top }(\phi )$$

(35)

where $ℱ$_φ(ϑ) and $ℱ$ (ϑ) are respectively the transformed and untransformed Fisher information matrices. It is straightforward to conclude that by virtue of (35), the transformed and untransformed Fisher information matrices $ℱ$_φ(ϑ) and $ℱ$(ϑ), are similar since the rotation matrix $ℒ$_i(φ) is orthogonal. Two matrices A and B are similar if there exists an invertible matrix X such that the equality AX = XB holds. As can be seen, the Givens matrix $ℒ$_i(φ) involves only two coordinates that are affected by the rotation angle φ whereas the other directions, which correspond to eigenvalues of one, are unaffected by the rotation matrix.

By virtue of (35) can be concluded that a positive definite FIM, $ℱ$(ϑ) ≻ 0, implies a positive definite transformed FIM, $ℱ$_φ(ϑ) ≻ 0. Consequently, the elements on the main diagonal of $ℱ$(ϑ), f_1,1, f_2,2, . . . , f_m,m, as well as the elements on the main diagonal of $ℱ$_φ(ϑ), f̃_1,1, f̃_2,2, . . . , f̃_m,m are all positive. However, the elements on the main diagonal of a singular FIM of a stationary ARMA process are also positive.

As developed in [7], combining (33) and (35) yields the distance measure of the estimated parameters ϑ₁, ϑ₂, . . . , ϑ_m accordingly, to obtain

$$d_{{\mathcal{F}}_{\phi }}^2(\vartheta )=\sum _{j=1,j\ne i,i+1}^m\left\{\frac{{\vartheta }_j^2}{f_{j,j}}\right\}+\frac{{\{{\vartheta }_i\hspace{0.17em}\text{cos}(\phi )-{\vartheta }_{i+1}\hspace{0.17em}\text{sin}(\phi )\}}^2}{{\tilde{f}}_{i,i}(\phi )}+\frac{{\{{\vartheta }_{i+1}\hspace{0.17em}\text{cos}(\phi )+{\vartheta }_i\hspace{0.17em}\text{sin}(\phi )\}}^2}{{\tilde{f}}_{i+1,i+1}(\phi )}$$

(36)

where

$${\tilde{f}}_{i,i}(\phi )=f_{i,i}\hspace{0.17em}{\text{cos}}^2(\phi )-f_{i,i+1}\hspace{0.17em}\text{sin}(2\phi )+f_{i+1,i+1}\hspace{0.17em}{\text{sin}}^2(\phi )$$

(37)

$${\tilde{f}}_{i+1,i+1}(\phi )=f_{i+1,i+1}\hspace{0.17em}{\text{cos}}^2(\phi )+f_{i,i+1}\hspace{0.17em}\text{sin}(2\phi )+f_{i,i}\hspace{0.17em}{\text{sin}}^2(\phi )$$

(38)

and f_j,l are entries of the FIM $ℱ$(ϑ) whereas f̃_i,i(φ) and f̃_i+1,i+1(φ) are the transformed components since the rotation affects only the entries, i and i+1, as can be seen in matrix $ℒ$_i(φ). In [7], the existence of the following inequalities is proved

$${\tilde{f}}_{i,i}(\phi )>0\mathrm{\hspace{0.17em} }\mathrm{\hspace{0.17em} }\mathrm{\hspace{0.17em} }\text{and\hspace{0.28em}}\mathrm{\hspace{0.17em} }\mathrm{\hspace{0.17em} }{\tilde{f}}_{i+1,i+1}(\phi )>0$$

this guaratees the metric property of (36). When the FIM of an ARMA(p, q) process is the case, a combination of (27) and (35) for the ARMA(p, q) parameters, given in (15) yields for the transformed FIM,

$${\mathcal{F}}_{\phi }(\vartheta )={\mathcal{S}}_{\phi }(-b,a)\mathcal{P}(\vartheta ){\mathcal{S}}_{\phi }^{\top }(-b,a)$$

(39)

where ℘(ϑ) is given by (28) and the transformed Sylvester resultant matrix is of the form

$${\mathcal{S}}_{\phi }(-b,a)={\mathcal{L}}_i(\phi )\mathcal{S}(-b,a)$$

(40)

Proposition 3.5 in [7], proves that the transformed FIM $ℱ$_φ(ϑ) and the transformed Sylvester matrix (−b, a) fulfill the resultant matrix property by using the equalities (40) and (39). The following property is then set forth.

Proposition 3.3

The properties

$$Ker\hspace{0.17em}{\mathcal{F}}_{\phi }(\vartheta )=Ker\hspace{0.17em}{\mathcal{S}}_{\phi }^{\top }(-b,a)\mathrm{\hspace{0.17em} }and\mathrm{\hspace{0.17em} }Ker\hspace{0.17em}{\mathcal{S}}_{\phi }(-b,a)=Ker\hspace{0.17em}\mathcal{S}(-b,a)$$

hold true.

Proof

By virtue of the equalities (39), (40) and the orthogonality property of the rotation matrix $ℒ$_i(φ) which implies that Ker $ℒ$_i(φ) = {0} combined with the same approach as in Lemma 3.1 completes the proof.

A straightforward conclusion from Proposition 3.3 is then

$$\text{dim\hspace{0.17em}Ker}\hspace{0.17em}{\mathcal{F}}_{\phi }(\vartheta )=\text{dim\hspace{0.17em}Ker}\hspace{0.17em}{\mathcal{S}}_{\phi }(-b,a),\hspace{0.17em}\text{dim\hspace{0.17em}Ker}\hspace{0.17em}{\mathcal{S}}_{\phi }(-b,a)=\text{dim\hspace{0.17em}Ker}\hspace{0.17em}\mathcal{S}(-b,a)$$

In the next section a distance measure introduced in quantum information is discussed.

Statistical Distance Measure - Fisher Information and Quantum Information

In quantum information, the Fisher information, the information about a parameter θ in a particular measurement procedure, is expressed in terms of the statistical distance s, see [8–10]. The statistical distance used is defined as a measure to distinguish two probability distributions on the basis of measurement outcomes, see [37]. The Fisher information and the statistical distance are statistical quantities, and generally refer to many measurements as it is the case in this survey. However, in the quantum information theory and quantum statistics context, the problem set up is presented as follows. There may or may not be a small phase change θ, and the question is whether it is there. In that case you can design quantum experiments that will tell you the answer unambiguously in a single measurement. The equality derived is of the form

$$\mathcal{F}\hspace{0.17em}(\phi )={\left(\frac{ds}{d\theta }\right)}^2$$

(41)

the Fisher information is the square of the derivative of the statistical distance s with respect to θ. Contrary to (36), where the square of the statistical distance measure is expressed in terms of entries of a FIM $ℱ$ (ϑ) which is based on information about m parameters estimated from n measurements, for m < n. A challenging question could therefore be formulated as follows, can a generalization of equality (41) be developed in a quantum information context but at the matrix level ? To be more specific, many observations or measurements that lead to more than one parameter such that the corresponding Fisher information matrix is interconnected to an appropriate statistical distance matrix, a matrix where entries are scalar distance measures. This question could equally be a challenge to algebraic matrix theory and to quantum information.

3.4. The Bezoutian - The Fisher Information Matrix

In this section an additional resultant matrix is presented, it concerns the Bezout matrix or Bezoutian. The notation of Lancaster and Tismenetsky [2] shall be used and the results presented are extracted from [38]. Assume the polynomials a and b given by $a(z)={\sum }_{j=0}^n{a}_j\hspace{0.17em}{z}^j$ and $b(z)={\sum }_{j=0}^n{b}_j\hspace{0.17em}{z}^j$, cfr. (13) but where p = q = n, and we further assume a₀ = b₀ = 1. The Bezout matrix B(a, b) of the polynomials a and b is defined by the relation

$$a(z)b(w)-a(w)b(z)=(z-w)u_n^{\top }(z)B(a,b)u_n(z)$$

This matrix is often referred as the Bezoutian. We will display a decomposition of the Bezout matrix B(a, b) developed in [38]. For that purpose the matrix U_φ and its inverse T_φ are presented, where φ is a given complex number, to obtain

$$U_{\phi }=\left(\begin{array}{ccccc}1& 0& \cdots & \cdots & 0\\ -\phi & 1& \cdots & \cdots & 0\\ 0& \hspace{0.17em}& \ddots & \hspace{0.17em}& ⋮\\ ⋮& \hspace{0.17em}& \hspace{0.17em}& \ddots & ⋮\\ 0& \cdots & 0& -\phi & 1\end{array}\right),\hspace{0.17em}{T}_{\phi }=\left(\begin{array}{ccccc}1& 0& \cdots & \cdots & 0\\ \phi & 1& \cdots & \cdots & 0\\ {\phi }^2& \hspace{0.17em}& \ddots & \hspace{0.17em}& ⋮\\ ⋮& \hspace{0.17em}& \hspace{0.17em}& \ddots & ⋮\\ {\phi }^{n-1}& \cdots & {\phi }^2& \phi & 1\end{array}\right)$$

Let (1 − α₁z) and (1 − β₁z) be a factor of a(z) and b(z) respectively and α₁ and β₁ are zeros of â(z) and b̂(z). Consider the factored form of the nth order polynomials a(z) and b(z) of the form a(z) = (1 − α₁z)a₋₁(z) and b(z) = (1 − β₁z)b₋₁(z) respectively. Proceeding this way, for α₂, . . . , α_n yields the recursion a_−(k−1)(z) = (1 − α_kz)a_−k(z), equivalently for the polynomials b_−k(z) and a₀(z) = a(z) and b₀(z) = b(z). Proposition 3.1 in [38] is presented.

The following non-symmetric decomposition of the Bezoutian is derived, considering the notations above

$$B(a,b)=U_{{\alpha }_1}\hspace{0.17em}\left(\begin{array}{cc}B(a_{-1},b_{-1})& 0\\ 0& 0\end{array}\right)\hspace{0.17em}{U}_{{\beta }_1}^{\top }+({\beta }_1-{\alpha }_1)b_{{\beta }_1}{a}_{{\alpha }_1}^{\top }$$

(42)

with a_α1 such that $a_{{\alpha }_1}^{\top }\hspace{0.17em}{u}_n(z)=a_{-1}$ similarly for b_β1. Iteration gives the following expansion for the Bezout matrix

$$B(a,b)=\sum _{k=1}^n({\beta }_k-{\alpha }_k)U_{{\alpha }_1}\dots U_{{\alpha }_{k-1}}{U}_{{\beta }_{k+1}}\dots U_{{\beta }_n}{e}_1^n\hspace{0.17em}{(e_1^n)}^{\top }\hspace{0.17em}{U}_{{\beta }_1}^{\top }\dots U_{{\beta }_{k-1}}^{\top }{U}_{{\alpha }_{k+1}}^{\top }\dots U_{{\alpha }_n}^{\top }$$

where $e_1^n$ is the first unit standard basis column vector in ℝⁿ, by e_j we denote the jth coordinate vector, e_j = (0, . . . , 1, . . . , 0) ^T, with all its components equal to 0 except the jth component which equals 1. The following corollarys to Proposition 3.1 in [38] are now presented.

Corollary 3.2 in [38] states. Let φ be a common zero of the polynomials â(z) and b̂(z). Then a(z) = (1 − φz)a₋₁(z) and b(z) = (1 − φz)b₋₁(z) and

$$B(a,b)=U_{\phi }\hspace{0.17em}\left(\begin{array}{cc}B(a_{-1},b_{-1})& 0\\ 0& 0\end{array}\right)\hspace{0.17em}{U}_{\phi }^{\top }$$

This a direct consequence of (42) and from which can be concluded that the Bezoutian B(a, b) is non-singular if and only if the polynomials a(z) and b(z) have no common factors. A similar conclusion is drawn for the FIM in (27) so that matrices $ℱ$(ϑ) and B(a, b) have the same singularity property.

Related to Corollary 3.2 in[38], this is where we give a description of the kernel or nullspace of the Bezout matrix.

Corollary 3.3 in [38] is now presented. Let φ₁, . . ., φ_m be all the common zeros of the polynomials â(z) and b̂(z), with multiplicities n₁, . . . , n_m. Let ℓ be the last unit standard basis column vector in ℝⁿ and put

$$w_k^j={\left(T_{{\phi }_k}^j{J}^{j-1}\right)}^{\top }\ell$$

for k = 1, . . . , m and j = 1, . . . , n_k and by J we denote the forward n × n shift matrix, J_ij = 1 if i = j + 1. Consequently, the subspace Ker B(a, b) is the linear span of the vectors $w_k^j$.

An alternative representation to (27) but involving the Bezoutian B(b, a) and derived in Proposition 5.1 in [38] is of the form

$$\mathcal{F}(\vartheta )={\mathcal{M}}^{-1}(b,a)\mathcal{H}(\vartheta ){\mathcal{M}}^{-\top }(b,a)$$

(43)

where

$$\mathcal{H}(\vartheta )=\left(\begin{array}{cc}I& 0\\ 0& B(b,a)\end{array}\right)\hspace{0.17em}Q(\vartheta )\hspace{0.17em}\left(\begin{array}{cc}I& 0\\ 0& B(b,a)\end{array}\right)\mathrm{\hspace{0.17em} }\mathrm{\hspace{0.17em} }\mathrm{\hspace{0.17em} }\text{and}\hspace{0.17em}\mathcal{M}(b,a)=\left(\begin{array}{cc}P& 0\\ PS(\widehat{a})P& PS(\widehat{b})P\end{array}\right)$$

(44)

and

$$P=\left(\begin{array}{cccc}0& \cdots & 0& 1\\ ⋮& ⋰& 1& 0\\ 0& ⋰& ⋰& ⋮\\ 1& 0& \cdots & 0\end{array}\right),\hspace{0.17em}S(\widehat{a})=\left(\begin{array}{cccc}{a}_{n-1}& a_{n-2}& \cdots & a_0\\ a_{n-2}& ⋰& a_0& 0\\ ⋮& ⋰& ⋰& ⋮\\ a_0& 0& \cdots & 0\end{array}\right)\mathrm{\hspace{0.17em} }\mathrm{\hspace{0.17em} }\mathrm{\hspace{0.17em} }\text{and}\hspace{0.17em}Q(\vartheta )\succ 0$$

The matrix S(â) is the symmetrizer of the polynomial â(z), in this paper a₀ = 1, see [2] and P is a permutation matrix. In [38] it is shown that the matrix Q(ϑ) is the unique solution to an appropriate Stein equation and is strictly positive definite. However, in the next section an explicit form of the Stein solution Q(ϑ) is developed. Some comments concerning the property summarized in Corollary 5.2 in [38] follow.

The matrix $\mathscr{H}$(ϑ) is non-singular if and only if the polynomials a(z) and b(z) have no common factors. The proof is straightforward since the matrix Q(ϑ) is non-singular which implies that the matrix $\mathscr{H}$(ϑ) is only non-singular when the Bezoutian B(b, a) is non-singular and this is fulfilled if and only if the polynomials a(z) and b(z) have no common factors.

The matrix $ℳ$(b, a) is non-singular if a₀ ≠ 0 and b₀ ≠ 0, which is the case since we have a₀ = b₀ = 1. From (43) can be concluded that the FIM $ℱ$(ϑ) is non-singular only when the matrix $\mathscr{H}$(ϑ) is non-singular or by virtue of (44) when the Bezoutian B(b, a) is non-singular. Consequently, the singularity conditions of the Bezoutian B(b, a), the FIM $ℱ$(ϑ) and the Sylvester resultant matrix (b, −a) are therefore equivalent. Can be concluded, by virtue of (29) proved in Lemma 3.1 and the equality dim (Ker (a, b)) = dim (Ker B(a, b)) proved in Theorem 21.11 in [1], yields

$$\text{dim\hspace{0.28em}}(\text{Ker}\hspace{0.17em}\mathcal{S}(b,-a))=\text{dim\hspace{0.28em}}(\text{Ker}\hspace{0.17em}\mathcal{F}(\vartheta ))=\text{dim}\mathrm{\hspace{0.17em} }(\text{Ker}\hspace{0.17em}B(b,a))=\nu (a,b)$$

3.5. The Stein Equation - The Fisher Information Matrix of an ARMA(p, q) Process

In [12], a link between the FIM of an ARMA process and an appropriate solution of a Stein equation is set forth. In this survey paper we shall present some of the results and confront some results displayed in the previous sections. However, alternative proofs will be given to some results obtained in [12,38].

The Stein matrix equation is now set forth. Let A ∈ ℂ^m×m, B ∈ ℂ^n×n and Γ ∈ ℂ^n×m and consider the Stein equation

$$S-BS{A}^{\top }=\mathrm{\Gamma }$$

(45)

It has a unique solution if and only if λμ ≠ 1 for any λ ∈ σ(A) and μ ∈ σ(B), the spectrum of D is σ(D) = {λ ∈ ℂ: det(λI_m − D) = 0}, the set of eigenvalues of D. The unique solution will be given in the next theorem [11].

Theorem 3.4

Let A and B be, such that there is a single closed contour C with σ(B) inside C and for each non-zero w ∈ σ(A), w⁻¹ is outside C. Then for an arbitrary Γ the Stein Equation (45) has a unique solution S

$$S=\frac{1}{2\pi \mathbf{i}}\underset{C}{\oint }{(\lambda I_n-B)}^{-1}\mathrm{\Gamma }{(I_m-\lambda A)}^{-\top }d\lambda$$

(46)

In this section an interconnection between the representation (27) of the FIM $ℱ$(ϑ) and an appropriate solution to a Stein equation of the form (45) as developed in [12] is set forth. The distinct roots of the polynomials â(z) and b̂(z) are denoted by α₁, α₂, . . . , α_p and β₁, β₂, . . . , β_q respectively such that the non-singularity of the FIM $ℱ$(ϑ) is guaranteed. The following representation of the integral expression (28) is given when Cauchy’s residue theorem is applied, equation (4.8) in [12]

$$\mathcal{P}(\vartheta )=\mathcal{U}(\vartheta )\mathcal{D}(\vartheta )\widehat{\mathcal{U}}(\vartheta )$$

(47)

where

$$\mathcal{U}(\vartheta )=\{u_{p+q}({\alpha }_1),u_{p+q}({\alpha }_2),\dots ,u_{p+q}({\alpha }_p),u_{p+q}({\beta }_1),u_{p+q}({\beta }_2),\dots ,u_{p+q}({\beta }_q)\}$$

$\mathcal{D}(\vartheta )=\text{diag}\hspace{0.17em}\left\{\left(\frac{1}{\widehat{a}(z;{\alpha }_i)\widehat{b}({\alpha }_i)a({\alpha }_i)b({\alpha }_i)}\right),\left(\frac{1}{\widehat{a}({\beta }_j)\widehat{b}(z;{\beta }_j)a({\beta }_j)b({\beta }_j)}\right)\right\}$, i = 1, ..., p and j = 1, ..., q

and

$$\widehat{\mathcal{U}}(\vartheta )={\{v_{p+q}({\alpha }_1),v_{p+q}({\alpha }_2),\dots ,v_{p+q}({\alpha }_p),v_{p+q}({\beta }_1),v_{p+q}({\beta }_2),\dots ,v_{p+q}({\beta }_q)\}}^{\top }$$

the polynomial p(·; β) is defined accordingly, $p(z;\beta )=\frac{p(z)}{(z-\beta )}$ and (ϑ) is the (p + q) × (p + q) diagonal matrix. The matrices (ϑ) and (ϑ) in (47) are the (p + q)× (p + q) Vandermonde matrices V_αβ and V̂_αβ respectively, given by

$$V_{\alpha \beta }=\left(\begin{array}{ccccc}1& {\alpha }_1& {\alpha }_1^2& \cdots & {\alpha }_1^{p+q-1}\\ 1& {\alpha }_2& {\alpha }_2^2& \cdots & {\alpha }_2^{p+q-1}\\ ⋮& ⋮& ⋮& ⋮& ⋮\\ 1& {\alpha }_p& {\alpha }_p^2& \cdots & {\alpha }_p^{p+q-1}\\ 1& {\beta }_1& {\beta }_1^2& \cdots & {\beta }_1^{p+q-1}\\ 1& {\beta }_2& {\beta }_2^2& \cdots & {\beta }_2^{p+q-1}\\ ⋮& ⋮& ⋮& ⋮& ⋮\\ 1& {\beta }_q& {\beta }_q^2& \cdots & {\beta }_q^{p+q-1}\end{array}\right)\mathrm{\hspace{0.17em} }\mathrm{\hspace{0.17em} }\mathrm{\hspace{0.17em} }\text{and}\hspace{0.17em}{\widehat{V}}_{\alpha \beta }=\left(\begin{array}{ccccc}{\alpha }_1^{p+q-1}& {\alpha }_1^{p+q-2}& \cdots & {\alpha }_1& 1\\ {\alpha }_2^{p+q-1}& {\alpha }_2^{p+q-2}& \cdots & {\alpha }_2& 1\\ ⋮& ⋮& ⋮& ⋮& ⋮\\ {\alpha }_p^{p+q-1}& {\alpha }_p^{p+q-2}& \cdots & {\alpha }_p& 1\\ {\beta }_1^{p+q-1}& {\beta }_1^{p+q-2}& \cdots & {\beta }_1& 1\\ {\beta }_2^{p+q-1}& {\beta }_2^{p+q-2}& \cdots & {\beta }_2& 1\\ ⋮& ⋮& ⋮& ⋮& ⋮\\ {\beta }_q^{p+q-1}& {\beta }_q^{p+q-2}& \cdots & {\beta }_q& 1\end{array}\right)$$

It is clear that the (p + q) × (p + q) Vandermonde matrices V_αβ and V̂_αβ are nonsingular when α_i ≠ α_j, β_k ≠ β_h and α_i ≠ β_k for all i, j = 1, . . . , p and k, h = 1, . . . , q. A rigorous systematic evaluation of the Vandermonde determinants DetV_αβ and DetV̂_αβ, yields

$$\text{Det}{V}_{\alpha \beta }={(-1)}^{(p+q)\hspace{0.17em}(p+q-1)/2}\mathrm{\Phi }\hspace{0.17em}({\alpha }_i,{\beta }_k)$$

where

$$\mathrm{\Phi }\hspace{0.17em}({\alpha }_i,{\beta }_k)=\prod _{1\le i<j\le p}({\alpha }_i-{\alpha }_j)\prod _{1\le k<h\le q}({\beta }_k-{\beta }_h)\prod _{\begin{array}{l}m=1,\dots p\\ n=1,\dots q\end{array}}({\alpha }_m-{\beta }_n)$$

Since $V_{\alpha \beta }=P{\widehat{V}}_{\alpha \beta }^{\top }$ and given the configuration of the permutation matrix, P, this leads to the equalities $\text{Det}{\widehat{V}}_{\alpha \beta }^{\top }=\text{Det}P\hspace{0.17em}\text{Det}{V}_{\alpha \beta }$ and DetP = (−1)^{(p+q)(p+q−1)/2} so that

$$\text{Det}{\widehat{V}}_{\alpha \beta }={(-1)}^{(p+q)\hspace{0.17em}(p+q-1)}\hspace{0.17em}\mathrm{\Phi }\hspace{0.17em}({\alpha }_i,{\beta }_k)\Rightarrow \mid \text{Det}{V}_{\alpha \beta }\mid =\mid \text{Det}{\widehat{V}}_{\alpha \beta }\mid$$

We shall now introduce an appropriate Stein equation of the form (45) such that an interconnection with ℘(ϑ) in (47) can be verified. Therefore the following (p + q)× (p + q) companion matrix is introduced,

$${\mathcal{C}}_g=\left(\begin{array}{cccc}0& 1& \cdots & 0\\ ⋮& \hspace{0.17em}& \ddots & ⋮\\ 0& \cdots & 0& 1\\ -g_{p+q}& -g_{p+q-1}& \cdots & -g_1\end{array}\right)$$

(48)

where the entries g_i are given by $z^{p+q}+{\sum }_{i=1}^{p+q}{g}_i(\vartheta )z^{p+q-i}=\widehat{a}(z)\widehat{b}(z)=\widehat{g}(z,\vartheta )$ and ĝ(ϑ) is the vector ĝ(ϑ) = (g_p+q(ϑ), g_p+q−1(ϑ), . . . , g₁(ϑ)) ^T. Likewise is the vector g(z, ϑ) = a(z)b(z) and g(ϑ) = (g₁(ϑ), g₁(ϑ), . . . , g_p+q(ϑ)) ^T, for investigating the properties of a companion matrix see e.g., [36], [2]. Since all the roots of the polynomials â(z) and b̂(z) are distinct and lie within the unit circle implies that the products α_iβ_j ≠ 1, α_iα_j ≠ 1 and β_iβ_j ≠ 1 hold for all i = 1, 2, . . . , p and j = 1, 2, . . . , q. Consequently, the uniqueness condition of the solution of an appropriate Stein equation is verified. The following Stein equation and its solution, according to (45) and (46), are now presented

$$S-{\mathcal{C}}_{g}S{\mathcal{C}}_g^{\top }=\mathrm{\Gamma }\hspace{0.17em}\text{and}\hspace{0.17em}S=\frac{1}{2\pi \mathbf{i}}\underset{\mid z\mid =1}{\oint }{(z{I}_{p+q}-{\mathcal{C}}_g)}^{-1}\mathrm{\Gamma }{(I_{p+q}-z{\mathcal{C}}_g)}^{-\top }dz$$

where the closed contour is now the unit circle |z| = 1 and the matrix Γ is of size (p + q)× (p + q). A more explicit expression of the solution S is of the form

$$S=\frac{1}{2\pi \mathbf{i}}\underset{\mid z\mid =1}{\oint }\frac{\text{adj}(z{I}_{p+q}-{\mathcal{C}}_g)\mathrm{\Gamma }\hspace{0.17em}\text{adj}{(I_{p+q}-z{\mathcal{C}}_g)}^{\top }}{a(z)b(z)\widehat{a}(z)\widehat{b}(z)}dz$$

(49)

where adj(X) = X⁻¹ Det(X), the adjoint of matrix X. When Cauchy’s residue theorem is applied to the solution S in (49), the following factored form of S is derived, equation (4.9) in [12]

$$S=({\mathcal{C}}_1,{\mathcal{C}}_2)\hspace{0.17em}(I_{p+q}\otimes \mathrm{\Gamma })\hspace{0.17em}(\mathcal{D}(\vartheta )\otimes I_{p+q})\hspace{0.17em}{({\mathcal{C}}_3,{\mathcal{C}}_4)}^{\top }$$

(50)

where

$$\begin{array}{l}{\mathcal{C}}_1=\text{adj}({\alpha }_1{I}_{p+q}-{\mathcal{C}}_g),\text{adj}({\alpha }_2{I}_{p+q}-{\mathcal{C}}_g),\dots ,\text{adj}({\alpha }_p{I}_{p+q}-{\mathcal{C}}_g)\\ {\mathcal{C}}_2=\text{adj}({\beta }_1{I}_{p+q}-{\mathcal{C}}_g),\text{adj}({\beta }_2{I}_{p+q}-{\mathcal{C}}_g),\dots ,\text{adj}({\beta }_p{I}_{p+q}-{\mathcal{C}}_g)\\ {\mathcal{C}}_3=\text{adj}(I_{p+q}-{\alpha }_1{\mathcal{C}}_g),\text{adj}(I_{p+q}-{\alpha }_2{\mathcal{C}}_g),\dots ,\text{adj}(I_{p+q}-{\alpha }_p{\mathcal{C}}_g)\\ {\mathcal{C}}_4=\text{adj}(I_{p+q}-{\beta }_1{\mathcal{C}}_g),\text{adj}(I_{p+q}-{\beta }_2{\mathcal{C}}_g),\dots ,\text{adj}(I_{p+q}-{\beta }_p{\mathcal{C}}_g)\end{array}$$

and (ϑ) is given in (47), the following matrix rule is applied

$$(A\otimes B)\hspace{0.17em}(C\otimes D)=AC\otimes BD$$

and the operator ⊗ is the tensor (Kronecker) product of two matrices, see e.g., [2], [20].

Combining (47) and (50) and taking the assumption, α_i ≠ α_j, β_k ≠ β_h and α_i ≠ β_k, into account implies that the inverse of the (p + q)× (p + q) Vandermonde matrices V_αβ and V̂_αβ exist, as Lemma 4.2 [12] states.

The following equality holds true

$$S=({\mathcal{C}}_1,{\mathcal{C}}_2)\hspace{0.17em}\left(V_{\alpha \beta }^{-1}\mathcal{P}(\vartheta ){\widehat{V}}_{\alpha \beta }^{-1}\otimes \mathrm{\Gamma }\right)\hspace{0.17em}{({\mathcal{C}}_3,{\mathcal{C}}_4)}^{\top }$$

or

$$S=({\mathcal{C}}_1,{\mathcal{C}}_2)\hspace{0.17em}\left(V_{\alpha \beta }^{-1}{\mathcal{S}}^{-1}(b,-a)\mathcal{F}(\vartheta ){\mathcal{S}}^{-\top }(b,-a){\widehat{V}}_{\alpha \beta }^{-1}\otimes \mathrm{\Gamma }\right)\hspace{0.17em}{({\mathcal{C}}_3,{\mathcal{C}}_4)}^{\top }$$

(51)

Consequently, under the condition α_i ≠ α_j, β_k ≠ β_h and α_i ≠ β_k, and by virtue of (27) and (51), an interconnection involving the FIM $ℱ$(ϑ), a solution to an appropriate Stein equation S, the Sylvester matrix (b, −a) and the Vandermonde matrices V_αβ and V̂_αβ is established. It is clear that by using the expression (43), the Bezoutian B (a, b) can be inserted in equality (51).

We will formulate a Stein equation when the matrix $\mathrm{\Gamma }=e_{p+q}{e}_{p+q}^{\top }$,

$$S-{\mathcal{C}}_{g}S{\mathcal{C}}_g^{\top }=e_{p+q}{e}_{p+q}^{\top }$$

(52)

where e_p+q is the last standard basis column vector in ℝ^p+q, $e_i^m$ is the i-th unit standard basis column vector in ℝ^m, with all its components equal to 0 except the i-th component which equals 1. The next lemma is formulated.

Lemma 3.5

The symmetric matrix ℘(ϑ) defined in (28) fulfills the Stein Equation (52).

Proof

The unique solution of (52) is according to (46)

$$S=\frac{1}{2\pi \mathbf{i}}\underset{\mid z\mid =1}{\oint }{(z{I}_{p+q}-{\mathcal{C}}_g)}^{-1}{e}_{p+q}{e}_{p+q}^{\top }{(I_{p+q}-z{\mathcal{C}}_g)}^{-\top }dz$$

more explictely written,

$$S=\frac{1}{2\pi \mathbf{i}}\underset{\mid z\mid =1}{\oint }\frac{\text{adj}(z{I}_{p+q}-{\mathcal{C}}_g)e_{p+q}{e}_{p+q}^{\top }\text{adj}{(I_{p+q}-z{\mathcal{C}}_g)}^{\top }}{a(z)b(z)\widehat{a}(z)\widehat{b}(z)}dz$$

Using the property of the companion matrix , standard computation shows that the last column of adj(zI_p+q − ) is the basic vector u_p+q(z) and consequently the last column of adj(I_p+q − z ) is the basic vector v_p+q(z) = z^p+q−1u_p+q(z⁻¹). This implies that adj(zI_p+q − )e_p+q = u_p+q(z) and $e_{p+q}^{\top }\text{adj}{(I_{p+q}-z{\mathcal{C}}_g)}^{\top }=v_{p+q}^{\top }(z)$ or

$$S=\frac{1}{2\pi \mathbf{i}}\underset{\mid z\mid =1}{\oint }\frac{u_{p+q}(z)v_{p+q}^{\top }(z)}{a(z)b(z)\widehat{a}(z)\widehat{b}(z)}dz=\mathcal{P}(\vartheta )$$

Consequently, the solution S to the Stein Equation (52) coincides with the matrix ℘(ϑ) defined in (28).

The Stein equation that is verified by the FIM $ℱ$(ϑ) will be considered. For that purpose we display the following p × p and q × q companion matrices and of the form,

$${\mathcal{C}}_a=\left(\begin{array}{ccccc}-a_1& -a_2& \cdots & \cdots & -a_p\\ 1& 0& \cdots & \cdots & 0\\ 0& \ddots & \hspace{0.17em}& \hspace{0.17em}& ⋮\\ ⋮& \ddots & \ddots & \hspace{0.17em}& ⋮\\ 0& \cdots & 0& 1& 0\end{array}\right),{\mathcal{C}}_b=\left(\begin{array}{ccccc}-b_1& -b_2& \cdots & \cdots & -b_q\\ 1& 0& \cdots & \cdots & 0\\ 0& \ddots & \hspace{0.17em}& \hspace{0.17em}& ⋮\\ ⋮& \ddots & \ddots & \hspace{0.17em}& ⋮\\ 0& \cdots & 0& 1& 0\end{array}\right)$$

respectively. Introduce the (p + q) × (p + q) matrix $\mathcal{K}(\vartheta )=\left(\begin{array}{cc}{\mathcal{C}}_a& O\\ O& {\mathcal{C}}_b\end{array}\right)$ and the (p + q) × 1 vector $\mathcal{B}=\left(\begin{array}{c}{e}_p^1\\ -e_q^1\end{array}\right)$, where $e_p^1$ and $e_q^1$ are the first standard basis column vectors in ℝ^p and ℝ^q respectively. Consider the Stein equation

$$S-\mathcal{K}(\vartheta )S{\mathcal{K}}^{\top }(\vartheta )=\mathcal{B}{\mathcal{B}}^{\top }$$

(53)

followed by the theorem.

Theorem 3.6

The Fisher information matrix $ℱ$(ϑ) (17) coincides with the solution to the Stein equation (53).

Proof

The eigenvalues of the companion matrices and are respectively the zeros of the polynomials â(z) and b̂(z) which are in absolute value smaller than one. This implies that the unique solution of the Stein Equation (53) exists and is given by

$$S=\frac{1}{2\pi \mathbf{i}}\underset{\mid z\mid =1}{\oint }{(z{I}_{p+q}-\mathcal{K}(\vartheta ))}^{-1}\mathcal{B}{\mathcal{B}}^{\top }{(I_{p+q}-z\mathcal{K}(\vartheta ))}^{-\top }dz$$

developing this integral expression in a more explicit form yields

$$S=\frac{1}{2\pi \mathbf{i}}\underset{\mid z\mid =1}{\oint }\left(\begin{array}{cc}\frac{\text{adj}(z{I}_p-{\mathcal{C}}_a)}{\widehat{a}(z)}& O\\ O& \frac{\text{adj}(z{I}_q-{\mathcal{C}}_b)}{\widehat{b}(z)}\end{array}\right)\hspace{0.17em}\left(\begin{array}{c}{e}_p^1\\ -e_q^1\end{array}\right)\hspace{0.17em}{\left\{\left(\begin{array}{cc}\frac{\text{adj}(I_p-z{\mathcal{C}}_a)}{a(z)}& O\\ O& \frac{\text{adj}(I_q-z{\mathcal{C}}_b)}{b(z)}\end{array}\right)\hspace{0.17em}\left(\begin{array}{c}{e}_p^1\\ -e_q^1\end{array}\right)\right\}}^{\top }dz$$

Considering the form of the companion matrices and leads through straightforward computation to the conclusion, the first column of adj(zI_p − ) is the basic vector v_p(z) and consequently the first column of adj(I_p − z ) is the basic vector u_p(z). Equivalently for the companion matrix , this yields

$$S=\frac{1}{2\pi \mathbf{i}}\underset{\mid z\mid =1}{\oint }\left(\begin{array}{c}\frac{v_p(z)}{\widehat{a}(z)}\\ -\frac{v_q(z)}{\widehat{b}(z)}\end{array}\right)\hspace{0.17em}\left(\begin{array}{cc}\frac{u_p^{\top }(z)}{a(z)}& -\frac{u_q^{\top }(z)}{b(z)}\end{array}\right)\hspace{0.17em}dz$$

(54)

Representation (54) is such that in order to obtain an equivalent representation to the FIM $ℱ$(ϑ) in (17), the transpose of the solution to the Stein Equation (53) is therefore required, to obtain

$$S^{\top }=\frac{1}{2\pi \mathbf{i}}\underset{\mid z\mid =1}{\oint }\left(\begin{array}{cc}\frac{u_p(z)v_p^{\top }(z)}{a(z)\widehat{a}(z)}& -\frac{u_p(z)v_q^{\top }(z)}{a(z)\widehat{b}(z)}\\ -\frac{u_q(z)v_p^{\top }(z)}{\widehat{a}(z)b(z)}& \frac{u_q(z)v_q^{\top }(z)}{b(z)\widehat{b}(z)}\end{array}\right)\hspace{0.17em}dz=\mathcal{F}(\vartheta )$$

(55)

or

$$S^{\top }=\frac{1}{2\pi \mathbf{i}}\underset{\mid z\mid =1}{\oint }{(I_{p+q}-z\mathcal{K}(\vartheta ))}^{-1}\mathcal{B}{\mathcal{B}}^{\top }{(z{I}_{p+q}-\mathcal{K}(\vartheta ))}^{-\top }dz=\mathcal{F}(\vartheta )$$

The symmetry property of the FIM $ℱ$ (ϑ), leads to S = $ℱ$ (ϑ). From the representation (55) can be concluded that the solution S of the Stein Equation (53) coincides with the symmetric block Toeplitz FIM $ℱ$( ϑ) given in (17). This completes the proof.

It is straightforward to verify that the submatrix (1,2) in (55) is the complex conjugate transpose of the submatrix (2,1), whereas each submatrix on the main diagonal is Hermitian, consequently, the integrand is Hermitian. This implies that when the standard residue theorem is applied, it yields $ℱ$( ϑ) = $ℱ$ ^T (ϑ).

An Illustrative Example of Theorem 3.6

To illustrate Theorem 3.6, the case of an ARMA(2, 2) process is considered. We will use the representation (17) for computing the FIM $ℱ$ (ϑ) of an ARMA(2, 2) process. The autoregressive and moving average polynomials are of degree two or p = q = 2 and the ARMA(2, 2) process is described by,

$$y(t)a(z)=b(z)\varepsilon (t)$$

(56)

where y(t) is the stationary process driven by white noise ɛ(t), a(z) = (1 + a₁z + a₂z²) and b(z) = (1+b₁z + b₂z²) and the parameter vector is ϑ = (a₁, a₂, b₁, b₂)^T. The condition, the zeros of the polynomials

$$\widehat{a}(z)=z^{2}a(z^{-1})=z^2+a_{1}z+a_2\hspace{0.17em}\text{and}\hspace{0.17em}\widehat{b}(z)=z^{2}b(z^{-1})=z^2+b_{1}z+b_2$$

are in absolute value smaller than one, is imposed. The FIM $ℱ$ (ϑ) of the ARMA(2, 2) process (56) is of the form

$$\mathcal{F}(\vartheta )=\left(\begin{array}{cc}{\mathcal{F}}_{aa}(\vartheta )& {\mathcal{F}}_{ab}(\vartheta )\\ {\mathcal{F}}_{ab}^{\top }(\vartheta )& {\mathcal{F}}_{bb}(\vartheta )\end{array}\right)$$

(57)

where

$$\begin{array}{c}{\mathcal{F}}_{aa}(\vartheta )=\frac{1}{(1-a_2)\hspace{0.17em}\left[{(1+a_2)}^2-a_1^2\right]}\left(\begin{array}{cc}1+a_2& -a_1\\ -a_1& 1+a_2\end{array}\right)\\ {\mathcal{F}}_{bb}(\vartheta )=\frac{1}{(1-b_2)\hspace{0.17em}\left[{(1+b_2)}^2-b_1^2\right]}\left(\begin{array}{cc}1+b_2& -b_1\\ -b_1& 1+b_2\end{array}\right)\\ {\mathcal{F}}_{ab}(\vartheta )=\frac{1}{{(a_2{b}_2-1)}^2+(a_2{b}_1-a_1)\hspace{0.17em}(b_1-a_1{b}_2)}\left(\begin{array}{cc}{a}_2{b}_2-1& a_1-a_2{b}_1\\ b_1-a_1{b}_2& a_2{b}_2-1\end{array}\right)\end{array}$$

The submatrices $ℱ$_aa(ϑ) and $ℱ$_bb(ϑ) are symmetric and Toeplitz whereas $ℱ$_ab(ϑ) is Toeplitz. One can assert that without any loss of generality, the property, symmetric block Toeplitz, holds for the class of Fisher information matrices of stationary ARMA(p, q) processes, where p and q are arbitrary, finite integers that represent the degrees of the autoregressive and moving average polynomials, respectively. The appropriate companion matrices , , the 4 × 4 matrices (ϑ) and $ℬ$$ℬ$^T are

$${\mathcal{C}}_a=\left(\begin{array}{cc}-a_1& -a_2\\ 1& 0\end{array}\right),{\mathcal{C}}_b=\left(\begin{array}{cc}-b_1& -b_2\\ 1& 0\end{array}\right),\mathcal{K}(\vartheta )=\left(\begin{array}{cccc}-a_1& -a_2& 0& 0\\ 1& 0& 0& 0\\ 0& 0& -b_1& -b_2\\ 0& 0& 1& 0\end{array}\right)\mathrm{\hspace{0.17em} }\mathrm{\hspace{0.17em} }\mathrm{\hspace{0.17em} }\text{and}\hspace{0.17em}\mathcal{B}{\mathcal{B}}^{\top }=\left(\begin{array}{cccc}1& 0& -1& 0\\ 0& 0& 0& 0\\ -1& 0& 1& 0\\ 0& 0& 0& 0\end{array}\right)$$

(58)

where $\mathcal{B}={\left(\begin{array}{cccc}1& 0& -1& 0\end{array}\right)}^{\top }$. It can be verified that the Stein equation

$$\mathcal{F}(\vartheta )-\mathcal{K}(\vartheta )\mathcal{F}(\vartheta ){\mathcal{K}}^{\top }(\vartheta )=\mathcal{B}{\mathcal{B}}^{\top }$$

holds true, when $ℱ$(ϑ) is of the form (57) and the matrices (ϑ) and $ℬ$$ℬ$^T are given in (58).

3.5.1. Some Additional Results

In Proposition 5.1 in [38], the matrix Q(ϑ) in (44) fulfills the Stein Equation (59) and the property Q(ϑ) ≻ 0 is proved. It states that when $e_P^{\top }={\left(e_1^{\top }P,0\right)}^{\top }={\left(e_n,0_n\right)}^{\top }\in {\mathbb{R}}^{2n}$, where e₁ is the first unit standard basis column vector in ℝⁿ and e_n is the last or n-th unit standard basis column vector in ℝⁿ, the following Stein equation admits the form

$$Q(\vartheta )=F_N(\vartheta )Q(\vartheta )F_N^{\top }(\vartheta )+e_P{e}_P^{\top }$$

(59)

where

$$F_N(\vartheta )=\left(\begin{array}{cc}{\widehat{\mathcal{C}}}_a& 0\\ e_1{e}_1^{\top }& {\mathcal{C}}_b\end{array}\right),{\widehat{\mathcal{C}}}_a=\left(\begin{array}{ccccc}0& 1& 0& \cdots & 0\\ 0& 0& 1& \cdots & 0\\ ⋮& \ddots & \hspace{0.17em}& \ddots & ⋮\\ 0& \ddots & \ddots & \hspace{0.17em}& 1\\ -a_p& -a_{p-1}& \cdots & \cdots & -a_1\end{array}\right)$$

A corollary to Proposition 5.1, [38] will be set forth, the involvement of various Vandermonde matrices in the explicit solution to equation (59) is confirmed. For that purpose the following Vandermonde matrices are displayed,

$$V_{\alpha }=\left(\begin{array}{ccc}1& 1& 1\\ {\alpha }_1& {\alpha }_2& {\alpha }_n\\ {\alpha }_1^2& {\alpha }_2^2& {\alpha }_n^2\\ ⋮& ⋮& ⋮\\ {\alpha }_1^{n-1}& {\alpha }_2^{n-1}& {\alpha }_n^{n-1}\end{array}\right),\hspace{0.17em}{\widehat{V}}_{\alpha }=\left(\begin{array}{ccc}{\alpha }_1^{n-1}& {\alpha }_1^{n-2}& 1\\ {\alpha }_2^{n-1}& {\alpha }_2^{n-2}& 1\\ {\alpha }_3^{n-1}& {\alpha }_3^{n-2}& 1\\ ⋮& ⋮& ⋮\\ {\alpha }_n^{n-1}& {\alpha }_n^{n-2}& 1\end{array}\right),\hspace{0.17em}{\widehat{V}}_{\alpha \beta }=\left(\begin{array}{c}{\widehat{V}}_{\alpha }\\ {\widehat{V}}_{\beta }\end{array}\right),\text{and}\hspace{0.17em}{V}_{\alpha \beta }=\left(\begin{array}{cc}{V}_{\alpha }& V_{\beta }\end{array}\right)$$

(60)

where V̂_β and V_β have the same configuration as V̂_α and V_α respectively. A corollary to Proposition 5.1 in [38] is now formulated.

Corollary 3.7

An explicit expression of the solution to the Stein equation (59) is of the form

$$Q(\vartheta )=\left(\begin{array}{cc}{V}_{\alpha }{\mathcal{D}}_{11}(\vartheta ){\widehat{V}}_{\alpha }& V_{\alpha }{\mathcal{D}}_{12}(\vartheta )V_{\alpha }^{\top }\\ {\widehat{V}}_{\alpha \beta }^{\top }{\mathcal{D}}_{21}(\vartheta ){\widehat{V}}_{\alpha \beta }& {\widehat{V}}_{\alpha \beta }^{\top }{\mathcal{D}}_{22}(\vartheta )V_{\alpha \beta }^{\top }\end{array}\right)$$

(61)

where the n × n and 2n × 2n diagonal matrices (ϑ) shall be specified in the proof.

Proof

The condition of a unique solution of the Stein Equation (59) is guaranteed since the eigenvalues of the companions matrices and given respectively by the zeros of the polynomials â (z) and b̂ (z) are in absolute value smaller than one. Consequently, the unique solution to the Stein Equation (59) exists and is given by

$$Q(\vartheta )=\frac{1}{2\pi \mathbf{i}}\underset{\mid z\mid =1}{\oint }{(z{I}_{2n}-F_N(\vartheta ))}^{-1}{e}_P{e}_P^{\top }{(I_{2n}-z{F}_N(\vartheta ))}^{-\top }dz$$

(62)

in order to proceed successfully, the following matrix property is displayed, to obtain

$${\left(\begin{array}{cc}A& O\\ B& C\end{array}\right)}^{-1}=\left(\begin{array}{cc}{A}^{-1}& O\\ -C^{-1}B{A}^{-1}& C^{-1}\end{array}\right)$$

When applied to the Equation (62), it yields

$$\begin{array}{l}Q(\vartheta )=\frac{1}{2\pi \mathbf{i}}\underset{\mid z\mid =1}{\oint }\left(\begin{array}{cc}\frac{\text{adj}(z{I}_p-{\widehat{\mathcal{C}}}_a)}{\widehat{a}(z)}& O\\ \frac{\text{adj}(z{I}_q-{\mathcal{C}}_b)e_1{e}_1^{\top }\text{adj}(z{I}_p-{\widehat{\mathcal{C}}}_a)}{\widehat{a}(z)\widehat{b}(z)}& \frac{\text{adj}(z{I}_q-{\mathcal{C}}_b)}{\widehat{b}(z)}\end{array}\right)\hspace{0.17em}\left(\begin{array}{c}{e}_n\\ 0\end{array}\right)\times \\ {\left\{\left(\begin{array}{cc}\frac{\text{adj}(I_n-z{\widehat{\mathcal{C}}}_a)}{\widehat{a}(z)}& O\\ \frac{\text{adj}(I_n-z{\mathcal{C}}_b)e_1{e}_1^{\top }\text{adj}(I_p-z{\widehat{\mathcal{C}}}_a)}{\widehat{a}(z)\widehat{b}(z)}& \frac{\text{adj}(I_n-z{\mathcal{C}}_b)}{\widehat{b}(z)}\end{array}\right)\hspace{0.17em}\left(\begin{array}{c}{e}_n\\ 0\end{array}\right)\right\}}^{\top }dz\end{array}$$

Considering that the last column vector of the matrices adj(zI_p − ) and adj(I_n − z ) are the vectors u_n(z) and v_n(z) respectively, it then yields

$$\begin{array}{l}Q(\vartheta )=\frac{1}{2\pi \mathbf{i}}\underset{\mid z\mid =1}{\oint }\left(\begin{array}{c}\frac{u_n(z)}{\widehat{a}(z)}\\ \frac{v_n(z)}{\widehat{a}(z)\widehat{b}(z)}\end{array}\right)\hspace{0.17em}\left(\begin{array}{cc}\frac{v_n^{\top }(z)}{a(z)}& \frac{z^{n-1}{u}_n^{\top }(z)}{a(z)b(z)}\end{array}\right)\hspace{0.17em}dz\\ =\frac{1}{2\pi \mathbf{i}}\underset{\mid z\mid =1}{\oint }\left(\begin{array}{cc}\frac{u_n(z)v_n^{\top }(z)}{a(z)\widehat{a}(z)}& \frac{z^{n-1}{u}_n(z)u_n^{\top }(z)}{\widehat{a}(z)a(z)b(z)}\\ \frac{v_n(z)v_n^{\top }(z)}{\widehat{a}(z)\widehat{b}(z)a(z)}& \frac{z^{n-1}{v}_n(z)u_n^{\top }(z)}{\widehat{a}(z)\widehat{b}(z)a(z)b(z)}\end{array}\right)\hspace{0.17em}dz=\left(\begin{array}{cc}{Q}_{11}(\vartheta )& Q_{12}(\vartheta )\\ Q_{21}(\vartheta )& Q_{22}(\vartheta )\end{array}\right)\end{array}$$

Applying the standard residue theorem leads for the respective submatrices

$$\begin{array}{c}{Q}_{11}(\vartheta )=\{u_n({\alpha }_1),\dots ,u_n({\alpha }_n)\}{\mathcal{D}}_{11}(\vartheta )\hspace{0.17em}{\{v_n({\alpha }_1),\dots ,v_n({\alpha }_n)\}}^{\top }\\ Q_{12}(\vartheta )=\{u_n({\alpha }_1),\dots ,u_n({\alpha }_n)\}{\mathcal{D}}_{12}(\vartheta )\hspace{0.17em}{\{u_n({\alpha }_1),\dots ,u_n({\alpha }_n)\}}^{\top }\\ Q_{21}(\vartheta )=\{v_n({\alpha }_1),\dots ,v_n({\alpha }_n),v_n({\beta }_1),\dots ,v_n({\beta }_n)\}{\mathcal{D}}_{21}(\vartheta )\hspace{0.17em}{\{v_n({\alpha }_1),\dots ,v_n({\alpha }_n),v_n({\beta }_1),\dots ,v_n({\beta }_n)\}}^{\top }\\ Q_{22}(\vartheta )=\{v_n({\alpha }_1),\dots ,v_n({\alpha }_n),v_n({\beta }_1),\dots ,v_n({\beta }_n)\}{\mathcal{D}}_{22}(\vartheta )\hspace{0.17em}{\{u_n({\alpha }_1),\dots ,u_n({\alpha }_n),u_n({\beta }_1),\dots ,u_n({\beta }_n)\}}^{\top }\end{array}$$

where the n × n diagonal matrices are

$${\mathcal{D}}_{11}(\vartheta )=\text{diag}\hspace{0.17em}\{1/(a({\alpha }_i)\widehat{a}(z;{\alpha }_i))\},{\mathcal{D}}_{12}(\vartheta )=\text{diag}\hspace{0.17em}\{{\alpha }_i^{n-1}/(a({\alpha }_i)b({\alpha }_i)\widehat{a}(z;{\alpha }_i))\}\mathrm{\hspace{0.17em} }\text{for}\hspace{0.17em}i=1,\dots ,n$$

and the 2n × 2n diagonal matrices are

$$\begin{array}{c}{\mathcal{D}}_{21}(\vartheta )=\text{diag}\hspace{0.17em}\left\{1/\left(a({\alpha }_i)\widehat{b}({\alpha }_i)\widehat{a}(z;{\alpha }_i)\right),1/\left(\widehat{a}({\beta }_j)a({\beta }_j)\widehat{b}(z;{\beta }_j)\right)\right\},\mathrm{\hspace{0.17em} }\text{for}\hspace{0.17em}i,j=1,\dots ,n\\ {\mathcal{D}}_{22}(\vartheta )=\text{diag}\hspace{0.17em}\left\{{\alpha }_i^{n-1}/\left(a({\alpha }_i)b({\alpha }_i)\widehat{b}({\alpha }_i)\widehat{a}(z;{\alpha }_i)\right),{\beta }_j^{n-1}/\left(\widehat{a}({\beta }_j)a({\beta }_j)b({\beta }_j)\widehat{b}(z;{\beta }_j)\right)\right\},\mathrm{\hspace{0.17em} }\text{for}\hspace{0.17em}i,j=1,\dots ,n\end{array}$$

It is clear that the first and third matrices in Q₁₁(ϑ), Q₁₂(ϑ), Q₂₁(ϑ) and Q₂₂(ϑ) are the appropriate Vandermonde matrices displayed in (60), it can be concluded that the representation (61) is verified. This completes the proof.

In this section an explicit form of the solution Q(ϑ), expressed in terms of various Vandermonde matrices, is displayed. Also, an interconnection between the Fisher information $ℱ$(ϑ) and appropriate solutions to Stein equations and related matrices is presented. Proofs are given when the Stein equations are verified by the FIM $ℱ$(ϑ) and the associated matrix ℘(ϑ). These are alternative to the proofs developed in [38]. The presence of various forms of Vandermonde matrices is also emphasized. In the next section some matrix properties of the FIM $ℱ$(ϑ) of an ARMAX process is presented.

3.6. The Fisher Information Matrix of an ARMAX(p, r, q) Process

The FIM of the ARMAX process (11) is set forth according to [4]. The derivatives in the corresponding representation (16) are

$$\frac{\partial {\varepsilon }_t(\vartheta )}{\partial a_j}=\frac{c(z)}{a(z)b(z)}x(t-j)+\frac{1}{a(z)}\varepsilon (t-j),\frac{\partial {\varepsilon }_t(\vartheta )}{\partial c_l}=-\frac{1}{b(z)}\varepsilon (t-l)\hspace{0.17em}\text{and}\frac{\partial {\varepsilon }_t(\vartheta )}{\partial b_k}=-\frac{1}{b(z)}{\varepsilon }_{t-k}$$

where j = 1, . . . , p, l = 1, . . . , r and k = 1, . . . , q. Combining all j, l and k yields the (p + r + q) × (p + r + q) FIM

$$\mathcal{G}(\vartheta )=\left(\begin{array}{ccc}{\mathcal{G}}_{aa}(\vartheta )& {\mathcal{G}}_{ac}(\vartheta )& {\mathcal{G}}_{ab}(\vartheta )\\ {\mathcal{G}}_{ac}^{\top }(\vartheta )& {\mathcal{G}}_{cc}(\vartheta )& {\mathcal{G}}_{cb}(\vartheta )\\ {\mathcal{G}}_{ab}^{\top }(\vartheta )& {\mathcal{G}}_{cb}^{\top }(\vartheta )& {\mathcal{G}}_{bb}(\vartheta )\end{array}\right)$$

(63)

where the submatrices of (ϑ) are given by

$$\begin{array}{l}{\mathcal{G}}_{aa}(\vartheta )=\frac{1}{2\pi \mathbf{i}}\underset{\mid z\mid =1}{\oint }{R}_x(z)\frac{u_p(z)u_p^{\top }(z^{-1})c(z)c(z^{-1})}{a(z)a(z^{-1})b(z)b(z^{-1})}\frac{dz}{z}+\frac{1}{2\pi \mathbf{i}}\underset{\mid z\mid =1}{\oint }\frac{u_p(z)u_p^{\top }(z^{-1})}{a(z)a(z^{-1})}\frac{dz}{z}\\ =\frac{1}{2\pi \mathbf{i}}\underset{\mid z\mid =1}{\oint }{R}_x(z)\frac{u_p(z)v_p^{\top }(z)c(z)\widehat{c}(z)}{a(z)\widehat{a}(z)b(z)\widehat{b}(z)z^{r-q}}dz+\frac{1}{2\pi \mathbf{i}}\underset{\mid z\mid =1}{\oint }\frac{u_p(z)v_p^{\top }(z)}{a(z)\widehat{a}(z)}dz\\ {\mathcal{G}}_{ab}(\vartheta )=-\frac{1}{2\pi \mathbf{i}}\underset{\mid z\mid =1}{\oint }\frac{u_p(z)u_q^{\top }(z^{-1})}{a(z)b(z^{-1})}\frac{dz}{z}=-\frac{1}{2\pi \mathbf{i}}\underset{\mid z\mid =1}{\oint }\frac{u_p(z)v_q^{\top }(z)}{a(z)\widehat{b}(z)}dz\\ {\mathcal{G}}_{ac}(\vartheta )=-\frac{1}{2\pi \mathbf{i}}\underset{\mid z\mid =1}{\oint }{R}_x(z)\frac{u_p(z)u_r^{\top }(z^{-1})c(z)}{a(z)b(z)b(z^{-1})}\frac{dz}{z}=-\frac{1}{2\pi \mathbf{i}}\underset{\mid z\mid =1}{\oint }{R}_x(z)\frac{u_p(z)v_r^{\top }(z)c(z)}{a(z)b(z)\widehat{b}(z)z^{r-q}}dz\\ {\mathcal{G}}_{cc}(\vartheta )=\frac{1}{2\pi \mathbf{i}}\underset{\mid z\mid =1}{\oint }{R}_x(z)\frac{u_r(z)u_r^{\top }(z^{-1})}{b(z)b(z^{-1})}\frac{dz}{z}=\frac{1}{2\pi \mathbf{i}}\underset{\mid z\mid =1}{\oint }{R}_x(z)\frac{u_r(z)v_r^{\top }(z)}{b(z)\widehat{b}(z)z^{r-q}}dz\\ {\mathcal{G}}_{bb}(\vartheta )=\frac{1}{2\pi \mathbf{i}}\underset{\mid z\mid =1}{\oint }\frac{u_q(z)u_q^{\top }(z^{-1})}{b(z)b(z^{-1})}\frac{dz}{z}=-\frac{1}{2\pi \mathbf{i}}\underset{\mid z\mid =1}{\oint }\frac{u_q(z)v_q^{\top }(z)}{b(z)\widehat{b}(z)}dz,\text{and}\hspace{0.17em}{\mathcal{G}}_{cb}(\vartheta )=O\end{array}$$

where R_x(z) is the spectral density of the process x(t) and is defined in (10). Let K(z) = a(z)a(z⁻¹)b(z)b(z⁻¹), combining all the expressions in (63) leads to the following representation of (ϑ) as the sum of two matrices

$$\frac{1}{2\pi \mathbf{i}}\underset{\mid z\mid =1}{\oint }\frac{R_x(z)}{K(z)}\left(\begin{array}{c}c(z)u_p(z)\\ -a(z)u_r(z)\\ O\end{array}\right)\hspace{0.17em}{\left(\begin{array}{c}c(z)u_p(z)\\ -a(z)u_r(z)\\ O\end{array}\right)}^{*}\frac{dz}{z}+\frac{1}{2\pi \mathbf{i}}\underset{\mid z\mid =1}{\oint }\frac{1}{K(z)}\left(\begin{array}{c}b(z)u_p(z)\\ O\\ -a(z)u_q(z)\end{array}\right)\hspace{0.17em}{\left(\begin{array}{c}b(z)u_p(z)\\ O\\ -a(z)u_q(z)\end{array}\right)}^{*}\frac{dz}{z}$$

(64)

where (X)^* is the complex conjugate transpose of the matrix X ∈ ℂ^m×n. Like in (23) we set forth

$$\mathcal{S}(-c,a)=\left(\begin{array}{c}-{\mathcal{S}}_p(c)\\ {\mathcal{S}}_r(a)\end{array}\right)$$

here (c) is formed by the top p rows of (−c, a). In a similar way we decompose

$$\mathcal{S}(-b,a)=\left(\begin{array}{c}-{\mathcal{S}}_p(b)\\ {\mathcal{S}}_q(a)\end{array}\right)$$

The representation (64) can be expressed by the appropriate block representations of the Sylvester resultant matrices, to obtain

$$\mathcal{G}(\vartheta )=\left(\begin{array}{c}-{\mathcal{S}}_p(c)\\ {\mathcal{S}}_r(a)\\ O\end{array}\right)\hspace{0.17em}\mathcal{W}(\vartheta )\hspace{0.17em}{\left(\begin{array}{c}-{\mathcal{S}}_p(c)\\ {\mathcal{S}}_r(a)\\ O\end{array}\right)}^{\top }+\left(\begin{array}{c}-{\mathcal{S}}_p(b)\\ O\\ {\mathcal{S}}_q(a)\end{array}\right)\hspace{0.17em}\mathcal{P}(\vartheta )\hspace{0.17em}{\left(\begin{array}{c}-{\mathcal{S}}_p(b)\\ O\\ {\mathcal{S}}_q(a)\end{array}\right)}^{\top }$$

(65)

where the matrix ℘(ϑ) is given in (28) and the matrix (ϑ) ∈ ℝ^(p+r)×(p+r) is of the form

$$\mathcal{W}(\vartheta )=\frac{1}{2\pi \mathbf{i}}\underset{\mid z\mid =1}{\oint }{R}_x(z)\frac{u_{p+r}(z)u_{p+r}^{\top }(z^{-1})}{a(z)a(z^{-1})b(z)b(z^{-1})}\frac{dz}{z}=\frac{1}{2\pi \mathbf{i}}\underset{\mid z\mid =1}{\oint }{R}_x(z)\frac{u_{p+r}(z)v_{p+r}^{\top }(z)}{a(z)b(z)\widehat{a}(z)\widehat{b}(z)}dz$$

(66)

It is shown in [4] that (ϑ) ≻ O. As can be seen in (65), the ARMAX part is explained by the first term, whereas the ARMA part is described by the second term, the combination of both terms is a summary of the Fisher information of a ARMAX(p, r, q) process. The FIM (ϑ) under form (65) allows us to prove the following property, Theorem 3.1 in [4]. The FIM (ϑ) of the ARMAX(p, r, q) process with polynomials a(z), c(z) and b(z) of order p, r, q respectively becomes singular if and only if these polynomials have at least one common root. Consequently, the class of resultant matrices is extended by the FIM (ϑ).

3.7. The Stein Equation - The Fisher Information Matrix of an ARMAX(p, r, q) Process

In Lemma 3.5 it is proved that the matrix ℘(ϑ) (28) fulfills the Stein Equation (52). We will now consider the conditions under which the matrix (ϑ) (66) verifies an appropriate Stein equation. For that purpose we consider the spectral density to be of the form R_x(z) = (1/h(z)h(z⁻¹)). The degree of the polynomial h(z) is ℓ and we assume the distinct roots of the polynomial h(z) to lie outside the unit circle, consequently, the roots of the polynomial ĥ(z) lie within the unit circle. We therefore rewrite (ϑ) accordingly

$$\mathcal{W}(\vartheta )=\frac{1}{2\pi \mathbf{i}}\underset{\mid z\mid =1}{\oint }\frac{u_{p+r}(z)u_{p+r}^{\top }(z^{-1})}{h(z)h(z^{-1})a(z)a(z^{-1})b(z)b(z^{-1})}\frac{dz}{z}$$

We consider a companion matrix of the form (48) and with size p + q + ℓ, it is denoted by and the entries f_i are given by $z^{p+q+\ell }+{\sum }_{i=1}^{p+q+\ell }{f}_i(\vartheta )z^{p+q+q\ell -i}=\widehat{a}(z)\widehat{b}(z)\widehat{h}(z)=\widehat{f}(z,\vartheta )$ and f̂(ϑ) is the vector f̂(ϑ) = (f_p+q+ℓ(ϑ), f_p+q+ℓ−1(ϑ), . . . , f₁(ϑ))^T. Likewise for the vector f(z, ϑ) = a(z)b(z)h(z) and f(ϑ) = (f₁(ϑ), f₁(ϑ), . . . , f_p+q+ℓ(ϑ))^T. The property Det(zI_p+q+ℓ − ) = â(z)b̂(z)ĥ(z) and Det(I_p+q+ℓ − z ) = a(z)b(z)h(z) holds and assume

$$r=q+\ell \hspace{0.17em}\text{or}\hspace{0.17em}p+q+\ell =p+r\hspace{0.17em}\text{and}\hspace{0.17em}r>q$$

(67)

(ϑ) is then of the form

$$\mathcal{W}(\vartheta )=\frac{1}{2\pi \mathbf{i}}\underset{\mid z\mid =1}{\oint }\frac{u_{p+r}(z)v_{p+r}^{\top }(z)}{h(z)\widehat{h}(z)a(z)\widehat{a}(z)b(z)\widehat{b}(z)}dz$$

(68)

We will formulate a Stein equation when the matrix $\mathrm{\Gamma }=e_{p+r}{e}_{p+r}^{\top }$ and which is of the form

$$S-{\mathcal{C}}_f\mathcal{S}{\mathcal{C}}_f^{\top }=e_{p+r}{e}_{p+r}^{\top }$$

(69)

where e_p+r is the last standard basis column vector in ℝ^p+r. The next lemma is formulated.

Lemma 3.8

The matrix (ϑ) given in (68) fulfills the Stein Equation (69).

Proof

The unique solution of (69) is assured since the product of all the eigenvalues of are different from one, the solution is of the form

$$S=\frac{1}{2\pi \mathbf{i}}\underset{\mid z\mid =1}{\oint }{(z{I}_{p+r}-{\mathcal{C}}_f)}^{-1}{e}_{p+r}{e}_{p+r}^{\top }{(I_{p+r}-z{\mathcal{C}}_f)}^{-\top }dz$$

or

$$S=\frac{1}{2\pi \mathbf{i}}\underset{\mid z\mid =1}{\oint }\frac{\text{adj}(z{I}_{p+r}-{\mathcal{C}}_f)e_{p+r}{e}_{p+r}^{\top }\text{adj}{(I_{p+r}-z{\mathcal{C}}_f)}^{\top }}{\widehat{a}(z)\widehat{b}(z)\widehat{h}(z)a(z)b(z)h(z)}dz$$

taking the property of the companion matrix into account implies that the last column vector of adj(zI_p+r − ) is the basic vector u_p+r(z), consequently the last column of adj(I_p+r − z ) is the basic vector v_p+r(z), this yields

$$S=\frac{1}{2\pi \mathbf{i}}\underset{\mid z\mid =1}{\oint }\frac{u_{p+r}(z)v_{p+r}^{\top }(z)}{\widehat{a}(z)\widehat{b}(z)\widehat{h}(z)a(z)b(z)h(z)}dz=\mathcal{W}(\vartheta )$$

Consequently, the matrix (ϑ) defined in (68) verifies the Stein Equation (69). This completes the proof.

The matrices, ℘(ϑ) and (ϑ), in (65), verify under specific conditions appropriate Stein equations, as has been shown in Lemma 3.5 and Lemma 3.8, respectively. We will now confirm the presence of Vandermonde matrices by applying the standard residue theorem to (ϑ) in (68), to obtain

$$\mathcal{W}(\vartheta )=V_{\alpha \beta \xi }\mathcal{R}\hspace{0.17em}(\vartheta )\hspace{0.17em}{\widehat{V}}_{\alpha \beta \xi }$$

(70)

The (p + r) × (p + r) diagonal matrix $ℛ$(ϑ) is of the form

$$\mathcal{R}\hspace{0.17em}(\vartheta )=\text{diag}\hspace{0.17em}\left\{\left(1/\widehat{a}(z;{\alpha }_i)\widehat{b}({\alpha }_i)\widehat{h}({\alpha }_i)\phi ({\alpha }_i)\right),\left(1/\widehat{a}({\beta }_j)\widehat{b}(z;{\beta }_j)\widehat{h}({\beta }_j)\phi ({\beta }_j)\right),\left(1/\widehat{a}({\xi }_k)\widehat{b}({\xi }_k)\widehat{h}(z;{\xi }_k)\phi ({\xi }_k)\right)\right\}$$

where φ(z) = a(z)b(z)h(z) and i = 1, . . . , p, j = 1, . . . , q and k = 1, . . . , ℓ. Whereas the (p + r) × (p + r) matrices V_αβξ and V̂_αβξ are of the form

$$V_{\alpha \beta \xi }={\left(\begin{array}{ccccc}1& {\alpha }_1& {\alpha }_1^2& \cdots & {\alpha }_1^{p+r-1}\\ ⋮& ⋮& ⋮& ⋮& ⋮\\ 1& {\alpha }_p& {\alpha }_p^2& \cdots & {\alpha }_p^{p+r-1}\\ 1& {\beta }_1& {\beta }_1^2& \cdots & {\beta }_1^{p+r-1}\\ ⋮& ⋮& ⋮& ⋮& ⋮\\ 1& {\beta }_q& {\beta }_q^2& \cdots & {\beta }_q^{p+r-1}\\ 1& {\xi }_1& {\xi }_1^2& \cdots & {\xi }_1^{p+r-1}\\ ⋮& ⋮& ⋮& ⋮& ⋮\\ 1& {\xi }_{\ell }& {\xi }_{\ell }^2& \cdots & {\xi }_{\ell }^{p+r-1}\end{array}\right)}^{\top },\hspace{0.17em}{\widehat{V}}_{\alpha \beta \xi }=\left(\begin{array}{ccccc}{\alpha }_1^{p+r-1}& {\alpha }_1^{p+r-2}& \cdots & {\alpha }_1& 1\\ ⋮& ⋮& \hspace{0.17em}& ⋮& ⋮\\ {\alpha }_p^{p+r-1}& {\alpha }_p^{p+r-2}& \cdots & {\alpha }_p& 1\\ {\beta }_1^{p+r-1}& {\beta }_1^{p+r-2}& \cdots & {\beta }_1& 1\\ ⋮& ⋮& \hspace{0.17em}& ⋮& ⋮\\ {\beta }_q^{p+r-1}& {\beta }_q^{p+r-2}& \cdots & {\beta }_q& 1\\ {\xi }_1^{p+r-1}& {\xi }_1^{p+r-2}& \cdots & {\xi }_1& 1\\ ⋮& ⋮& \hspace{0.17em}& ⋮& ⋮\\ {\xi }_{\ell }^{p+r-1}& {\xi }_{\ell }^{p+r-2}& \cdots & {\xi }_{\ell }& 1\end{array}\right)$$

The (p + r) × (p + r) Vandermonde matrices V_αβξ and V̂_αβξ are nonsingular when α_i ≠ α_j , β_k ≠ β_h, ξ_m ≠ ξ_n, α_i ≠ β_k, α_i ≠ ξ_m, β_k ≠ ξ_m for all i, j = 1, . . . , p, k, h = 1, . . . , q and m,n = 1, . . . , ℓ. The Vandermonde determinants DetV_αβξ and Det V̂_αβξ, are

$$\text{Det}{V}_{\alpha \beta \xi }={(-1)}^{(p+r)\hspace{0.17em}(p+r-1)/2}\hspace{0.17em}\mathrm{\Psi }\hspace{0.17em}({\alpha }_i,{\beta }_k,{\xi }_m)$$

where

$$\begin{array}{c}\mathrm{\Psi }\hspace{0.17em}({\alpha }_i,{\beta }_k,{\xi }_m)=\\ \prod _{1\le i<j\le p}({\alpha }_i-{\alpha }_j)\prod _{1\le k<h\le q}({\beta }_k-{\beta }_h)\prod _{1\le m<n\le \ell }({\xi }_m-{\xi }_n)\prod _{\begin{array}{l}r=1,\dots ,p\\ s=1,\dots ,q\end{array}}({\alpha }_r-{\beta }_s)\prod _{\begin{array}{l}r=1,\dots ,p\\ w=1,\dots ,\ell \end{array}}({\alpha }_r-{\xi }_w)\prod _{\begin{array}{l}s=1,\dots ,q\\ w=1,\dots ,\ell \end{array}}({\beta }_s-{\xi }_w)\end{array}$$

Like for the Vandermonde matrices V_αβ and ${\widehat{V}}_{\alpha \beta }^{\top }$,

$$\text{Det}{\widehat{V}}_{\alpha \beta \xi }={(-1)}^{(p+r)\hspace{0.17em}(p+r-1)}\hspace{0.17em}\mathrm{\Psi }\hspace{0.17em}({\alpha }_i,{\beta }_k,{\xi }_m)\Rightarrow \mid \text{Det}{V}_{\alpha \beta \xi }\mid =\mid \text{Det}{\widehat{V}}_{\alpha \beta \xi }\mid$$

Equation (70) is the ARMAX equivalent to (47). A combination of both equations generates a new representation of the FIM (ϑ), this is set forth in the following lemma.

Lemma 3.9

Assume the conditions (67) to hold and consider the representations of ℘(ϑ) and (ϑ) in (47) and (70) respectively, leads to an alternative form to (65), it is given by

$$\mathcal{G}(\vartheta )=\left(\begin{array}{c}-{\mathcal{S}}_p(c)\\ {\mathcal{S}}_r(a)\\ O\end{array}\right)\hspace{0.17em}{V}_{\alpha \beta \xi }\mathcal{R}\hspace{0.17em}(\vartheta )\hspace{0.17em}{\widehat{V}}_{\alpha \beta \xi }\hspace{0.17em}{\left(\begin{array}{c}-{\mathcal{S}}_p(c)\\ {\mathcal{S}}_r(a)\\ O\end{array}\right)}^{\top }+\left(\begin{array}{c}-{\mathcal{S}}_p(b)\\ O\\ {\mathcal{S}}_q(a)\end{array}\right)\hspace{0.17em}{V}_{\alpha \beta }\mathcal{D}(\vartheta ){\widehat{V}}_{\alpha \beta }\hspace{0.17em}{\left(\begin{array}{c}-{\mathcal{S}}_p(b)\\ O\\ {\mathcal{S}}_q(a)\end{array}\right)}^{\top }$$

In Lemma 3.9, the FIM (ϑ) is expressed by submatrices of two Sylvester matrices and various Vandermonde matrices, both type of matrices become singular if and only if the appropriate polynomials have at least one common root.

3.8. The Fisher Information Matrix of a Vector ARMA(p, q) Process

The process (6) is summarized as,

$$A(z)y(t)=B(z)\varepsilon (t)$$

and we assume that {y(t), t ∈ ℕ}, is a zero mean Gaussian time series and {ɛ(t), t ∈ ℕ} is a n-dimensional vector random variable, such that {ɛ(t)} = 0 and {ɛ(t)ɛ^T (t)} = ∑ and the parameter vector ϑ is of the form (7). In [6] it is shown that representation (16) for the n²(p+q)×n²(p+q) asymptotic FIM of the VARMA process (6) is

$$\mathbf{F}(\vartheta )={\mathbb{E}}_{\vartheta }\hspace{0.17em}\left\{{\left(\frac{\partial \varepsilon }{\partial {\vartheta }^{\top }}\right)}^{\top }{\mathrm{\Sigma }}^{-1}\hspace{0.17em}\left(\frac{\partial \varepsilon }{\partial {\vartheta }^{\top }}\right)\right\}$$

(71)

where ∂ɛ/∂ϑ^T is of size n×n²(p+q) and for convenience t is omitted from ɛ(t). Using the differential rules outlined in [6], yields

$$\frac{\partial \varepsilon }{\partial {\vartheta }^{\top }}=\left\{{(A^{-1}(z)B(z)\varepsilon )}^{\top }\otimes B^{-1}(z)\right\}\frac{\partial \text{vec}\hspace{0.17em}A(z)}{\partial {\vartheta }^{\top }}-({\varepsilon }^{\top }\otimes B^{-1}(z))\frac{\partial \text{vec}\hspace{0.17em}B(z)}{\partial {\vartheta }^{\top }}$$

(72)

The substitution of representation (72) of ∂ɛ/∂ϑ ^T in (71) yields the FIM of a VARMA process. The purpose is to construct a factorization of the FIM F(ϑ) that should be a multiple variant of the factorization (27), so that a multiple resultant matrix property can be proved for F(ϑ). As illustrated in [6], the multiple version of the Sylvester resultant matrix (22) does not fulfill the multiple resultant matrix property. In that case even when the matrix polynomials A(z) and B(z) have a common zero or a common eigenvalue, the multiple Sylvester matrix is not neccessarily singular. This has also been illustrated in [3]. In order to consider a multiple equivalent to the resultant matrix (−b, a), Gohberg and Lerer set forth the n²(p + q) × n²(p + q) tensor Sylvester matrix

$${\mathcal{S}}^{\otimes }(-B,A):=\left(\begin{array}{ccccccc}(-I_n)\otimes I_n& (-B_1)\otimes I_n& \cdots & (-B_q)\otimes I_n& O_{n^2\times n^2}& \cdots & O_{n^2\times n^2}\\ O_{n^2\times n^2}& \ddots & \ddots & \hspace{0.17em}& \ddots & \ddots & ⋮\\ ⋮& \ddots & \ddots & \ddots & \hspace{0.17em}& \ddots & O_{n^2\times n^2}\\ O_{n^2\times n^2}& \cdots & O_{n^2\times n^2}& (-I_n)\otimes I_n& (-B_1)\otimes I_n& \cdots & (-B_q)\otimes I_n\\ I_n\otimes I_n& I_n\otimes A_1& \cdots & I_n\otimes A_p& O_{n^2\times n^2}& \cdots & O_{n^2\times n^2}\\ O_{n^2\times n^2}& \ddots & \ddots & \hspace{0.17em}& \ddots & \ddots & ⋮\\ ⋮& \ddots & \ddots & \ddots & \hspace{0.17em}& \ddots & O_{n^2\times n^2}\\ O_{n^2\times n^2}& \cdots & O_{n^2\times n^2}& I_n\otimes I_n& I_n\otimes A_1& \cdots & I_n\otimes A_p\end{array}\right)$$

(73)

In [3], the authors prove that the tensor Sylvester matrix (−B,A) fulfills the multiple resultant property, it becomes singular if and only if the appropriate matrix polynomials A(z) and B(z) have at least one common zero. In Proposition 2.2 in [6], the following factorized form of the Fisher information F(ϑ) is developed

$$\mathbf{F}(\vartheta )=\frac{1}{2\pi \mathbf{i}}\underset{\mid z\mid =1}{\oint }\mathrm{\Phi }(z)\mathrm{\Theta }(z){\mathrm{\Phi }}^{*}(z)\frac{dz}{z}$$

(74)

where

$$\mathrm{\Phi }(z)=\left(\begin{array}{cc}{I}_p\otimes A^{-1}(z)\otimes I_n& O_{p{n}^2\times q{n}^2}\\ O_{q{n}^2\times p{n}^2}& I_q\otimes I_n\otimes A^{-1}(z)\end{array}\right)\hspace{0.17em}{\mathcal{S}}^{\otimes }(-B,A)\hspace{0.17em}(u_{p+q}(z)\otimes I_{n^2})$$

and

$$\mathrm{\Theta }(z)=\mathrm{\Sigma }\otimes \sigma (z),\sigma (z)=B^{-\top }(z){\mathrm{\Sigma }}^{-1}{B}^{-1}(z^{-1})$$

(75)

In order to obtain a multiple variant of (27), the following matrix is introduced,

$$\mathbf{M}(\vartheta )=\frac{1}{2\pi \mathbf{i}}\underset{\mid z\mid =1}{\oint }\mathrm{\Lambda }(z)\mathcal{J}(z){\mathrm{\Lambda }}^{*}(z)\frac{dz}{z}={\mathcal{S}}^{\otimes }(-B,A)\mathbf{P}(\vartheta )\hspace{0.17em}{({\mathcal{S}}^{\otimes }(-B,A))}^{\top }$$

(76)

where

$$\mathcal{J}(z)=\mathrm{\Phi }(z)\mathrm{\Theta }(z){\mathrm{\Phi }}^{*}(z)\mathrm{\hspace{0.17em} }\text{and}\mathrm{\hspace{0.17em} }\mathrm{\Lambda }(z)=\left(\begin{array}{cc}{I}_p\otimes A(z)\otimes I_n& O_{p{n}^2\times q{n}^2}\\ O_{q{n}^2\times p{n}^2}& I_q\otimes I_n\otimes A(z)\end{array}\right)$$

and the matrix P(ϑ) is a multiple variant of the matrix ℘(ϑ) in (28), it is of the form

$$\mathbf{P}(\vartheta )=\frac{1}{2\pi \mathbf{i}}\underset{\mid z\mid =1}{\oint }(u_{p+q}(z)\otimes I_{n^2})\hspace{0.17em}\mathrm{\Theta }(z)\hspace{0.17em}{(u_{p+q}(z)\otimes I_{n^2})}^{*}\frac{dz}{z}$$

(77)

In Lemma 2.3 in [6], it is proved that the matrix M(ϑ) in (76) becomes singular if and only if the matrix polynomials A(z) and B(z) have at least one common eigenvalue-zero. The proof is a multiple equivalent of the proof of Corollary 2.2 in [5], since the equality (76) is a multiple version of (27). Consequently, the matrix M(ϑ) like the tensor Sylvester matrix (−B,A), fulfills the multiple resultant matrix property. Since the matrix M(ϑ) is derived from the FIM F(ϑ), this enables us to prove that the matrix F(ϑ) fulfills the multiple resultant matrix property by showing that it becomes singular if and only if the matrix M(ϑ) is singular, this is done in Proposition 2.4 in [6]. Consequently, it can be concluded from [6] that the FIM of a VARMA process F(ϑ) and the tensor Sylvester matrix (−B,A) have the same singularity conditions. The FIM of a VARMA process F(ϑ) can therefore be added to the class of multiple resultant matrices.

A brief summary of the contribution of [6] follows, in order to show that the FIM of a VARMA process F(ϑ) is a multiple resultant matrix two new representations of the FIM are derived. To construct such representations appropriate matrix differential rules are applied. The newly obtained representations are expressed in terms of the multiple Sylvester matrix and the tensor Sylvester matrix. The representation of the FIM expressed by the tensor Sylvester matrix is used to prove that the FIM becomes singular if and only if the autoregressive and moving average matrix polynomials have at least one common eigenvalue. It then follows that the FIM and the tensor Sylvester matrix have equivalent singularity conditions. In a numerical example it is shown, however, that the FIM fails to detect common eigenvalues due to some kind of numerical instability. The tensor Sylvester matrix reveals it clearly, proving the usefulness of the results derived in this paper.

3.9. The Fisher Information Matrix of a Vector ARMAX(p, r, q) Process

The n²(p + q + r) × n²(p + q + r) asymptotic FIM of the VARMAX(p, r, q) process (2)

$$A(z)y(t)=C(z)x(t)+B(z)\varepsilon (t)$$

is displayed according to [23] and is an extension of the FIM of the VARMA(p, q) process (6). Representation (16) of the FIM of the VARMAX(p, r, q) process is then

$$\mathbf{G}(\vartheta )={\mathbb{E}}_{\vartheta }\hspace{0.17em}\left\{{\left(\frac{\partial \varepsilon }{\partial {\vartheta }^{\top }}\right)}^{\top }{\mathrm{\Sigma }}^{-1}\hspace{0.17em}\left(\frac{\partial \varepsilon }{\partial {\vartheta }^{\top }}\right)\right\}$$

where

$$\begin{array}{l}\frac{\partial \varepsilon }{\partial {\vartheta }^{\top }}=\left\{{(A^{-1}(z)C(z)x)}^{\top }\otimes B^{-1}(z)\right\}\frac{\partial \text{vec}\hspace{0.17em}A(z)}{\partial {\vartheta }^{\top }}\\ +\left\{{(A^{-1}(z)B(z)\varepsilon )}^{\top }\otimes B^{-1}(z)\right\}\frac{\partial \text{vec}\hspace{0.17em}A(z)}{\partial {\vartheta }^{\top }}\\ -\{x^{\top }\otimes B^{-1}(z)\}\frac{\partial \text{vec}\hspace{0.17em}C(z)}{\partial {\vartheta }^{\top }}\\ -({\varepsilon }^{\top }\otimes B^{-1}(z))\frac{\partial \text{vec}\hspace{0.17em}B(z)}{\partial {\vartheta }^{\top }}\end{array}$$

(78)

To obtain the term ∂ɛ/∂ϑ ^T, of size n × n²(p + q + r), the same differential rules are applied as for the VARMA(p, q) process. In Proposition 2.3 in [23], the representation of the FIM of a VARMAX process is expressed in terms of tensor Sylvester matrices, this obtained when ∂ɛ/∂ϑ ^T in (78) is substituted in (16), to obtain

$$\mathbf{G}(\vartheta )=\frac{1}{2\pi i}{\int }_{\mid z\mid =1}{\mathrm{\Phi }}_x(z)\mathrm{\Theta }(z){\mathrm{\Phi }}_x^{*}(z)\frac{dz}{z}+\frac{1}{2\pi i}{\int }_{\mid z\mid =1}{\mathrm{\Lambda }}_x(z)\mathrm{\Psi }(z){\mathrm{\Lambda }}_x^{*}(z)\frac{dz}{z}$$

(79)

The matrices in (79) are of the form

$$\begin{array}{c}{\mathrm{\Phi }}_x(z)=\left(\begin{array}{ccc}{I}_p\otimes A^{-1}(z)\otimes I_n& O_{p{n}^2\times r{n}^2}& O_{p{n}^2\times q{n}^2}\\ O_{r{n}^2\times p{n}^2}& O_{r{n}^2\times r{n}^2}& O_{r{n}^2\times q{n}^2}\\ O_{q{n}^2\times p{n}^2}& O_{q{n}^2\times r{n}^2}& I_q\otimes I_n\otimes A^{-1}(z)\end{array}\right)\hspace{0.17em}\left(\begin{array}{c}-{\mathcal{S}}_p^{\otimes }(B)\\ O_{r{n}^2\times n^2(p+q)}\\ {\mathcal{S}}_q^{\otimes }(A)\end{array}\right)\hspace{0.17em}(u_{p+q}(z)\otimes I_{n^2})\\ {\mathrm{\Lambda }}_x(z)=\left(\begin{array}{ccc}{I}_p\otimes A^{-1}(z)\otimes I_n& O_{p{n}^2\times r{n}^2}& O_{p{n}^2\times q{n}^2}\\ O_{r{n}^2\times p{n}^2}& I_r\otimes I_n\otimes A^{-1}(z)& O_{r{n}^2\times q{n}^2}\\ O_{q{n}^2\times p{n}^2}& O_{q{n}^2\times r{n}^2}& O_{q{n}^2\times q{n}^2}\end{array}\right)\hspace{0.17em}\left(\begin{array}{c}-{\mathcal{S}}_p^{\otimes }(C)\\ {\mathcal{S}}_r^{\otimes }(A)\\ O_{q{n}^2\times n^2(p+r)}\end{array}\right)\hspace{0.17em}(u_{p+r}(z)\otimes I_{n^2})\\ {\mathcal{S}}_{p,q}^{\otimes }(-B,A)=\left(\begin{array}{c}-{\mathcal{S}}_p^{\otimes }(B)\\ {\mathcal{S}}_q^{\otimes }(A)\end{array}\right),\mathrm{\hspace{0.17em} }{\mathcal{S}}_{p,r}^{\otimes }(-C,A)=\left(\begin{array}{c}-{\mathcal{S}}_p^{\otimes }(C)\\ {\mathcal{S}}_r^{\otimes }(A)\end{array}\right)\end{array}$$

(80)

additionally we have Ψ(z) = R_x(z) ⊗ σ(z) and the Hermitian spectral density matrix R_x(z) is defined in (10), whereas the matrix polynomials Θ(z) and σ(z) are presented in (75). In (80), we have the pn² × (p + q)n² and qn² × (p + q)n² submatrices ${\mathcal{S}}_p^{\otimes }(-B)$ and ${\mathcal{S}}_q^{\otimes }(A)$ of the tensor Sylvester resultant matrix ${\mathcal{S}}_{p,q}^{\otimes }(-B,A)$. Whereas the matrices ${\mathcal{S}}_p^{\otimes }(-C)$ and ${\mathcal{S}}_r^{\otimes }(A)$ are the upper and lower blocks of the (p+r)n²×(p+r)n² tensor Sylvester resultant matrix ${\mathcal{S}}_{p,r}^{\otimes }(-C,A)$. As for the FIM of the VARMA(p, q) process, the objective is to construct a multiple version of (65), this done in [23], to obtain

$$\begin{array}{c}{\mathit{M}}_x(\vartheta )=\frac{1}{2\pi i}{\oint }_{\mid z\mid =1}\mathcal{L}(z)\mathcal{A}(z){\mathcal{L}}^{*}(z)\frac{dz}{z}+\frac{1}{2\pi i}{\oint }_{\mid z\mid =1}\mathcal{W}(z)\mathcal{B}(z){\mathcal{W}}^{*}(z)\frac{dz}{z}\\ =\left(\begin{array}{c}-{\mathcal{S}}_p^{\otimes }(B)\\ O_{r{n}^2\times n^2(p+q)}\\ {\mathcal{S}}_q^{\otimes }(A)\end{array}\right)\hspace{0.17em}\mathit{P}(\vartheta )\hspace{0.17em}{\left(\begin{array}{c}-{\mathcal{S}}_p^{\otimes }(B)\\ O_{r{n}^2\times n^2(p+q)}\\ {\mathcal{S}}_q^{\otimes }(A)\end{array}\right)}^{\top }+\left(\begin{array}{c}-{\mathcal{S}}_p^{\otimes }(C)\\ {\mathcal{S}}_r^{\otimes }(A)\\ O_{q{n}^2\times n^2(p+r)}\end{array}\right)\hspace{0.17em}\mathit{T}(\vartheta )\hspace{0.17em}{\left(\begin{array}{c}-{\mathcal{S}}_p^{\otimes }(C)\\ {\mathcal{S}}_r^{\otimes }(A)\\ O_{q{n}^2\times n^2(p+r)}\end{array}\right)}^{\top }\end{array}$$

(81)

The matrices involved are of the form

$$\begin{array}{c}\mathcal{L}(z)=\left(\begin{array}{ccc}{I}_p\otimes A(z)\otimes I_n& O_{p{n}^2\times r{n}^2}& O_{p{n}^2\times q{n}^2}\\ O_{r{n}^2\times p{n}^2}& O_{r{n}^2\times r{n}^2}& O_{r{n}^2\times q{n}^2}\\ O_{q{n}^2\times p{n}^2}& O_{q{n}^2\times r{n}^2}& I_q\otimes I_n\otimes A(z)\end{array}\right)\hspace{0.17em}\text{and}\hspace{0.17em}\mathcal{A}(z):={\mathrm{\Phi }}_x(z)\mathrm{\Theta }(z){\mathrm{\Phi }}_x^{*}(z)\\ \mathcal{W}(z)=\left(\begin{array}{ccc}{I}_p\otimes A(z)\otimes I_n& O_{p{n}^2\times r{n}^2}& O_{p{n}^2\times q{n}^2}\\ O_{r{n}^2\times p{n}^2}& I_r\otimes I_n\otimes A(z)& O_{r{n}^2\times q{n}^2}\\ O_{q{n}^2\times p{n}^2}& O_{q{n}^2\times r{n}^2}& O_{q{n}^2\times q{n}^2}\end{array}\right)\hspace{0.17em}\text{and}\hspace{0.17em}\mathcal{B}(z):={\mathrm{\Lambda }}_x(z)\mathrm{\Psi }(z){\mathrm{\Lambda }}_x^{*}(z)\\ \mathit{T}(\vartheta )=\frac{1}{2\pi i}{\oint }_{\mid z\mid =1}(u_{p+r}(z)\otimes I_{n^2})\hspace{0.17em}\mathrm{\Psi }(z)\hspace{0.17em}{(u_{p+r}(z)\otimes I_{n^2})}^{*}\frac{dz}{z}\end{array}$$

and P(ϑ) is given in (77). Note, the matrices Φ_x(z), Λ_x(z), $ℒ$(z) and (z) are the corrected versions of the corresponding matrices in [23].

A parallel between the scalar and multiple structures is straightforward. This is best illustrated by comparing the representations (27) and (28) with (76) and (77) respectively, confronting the FIM for scalar and vector ARMA(p, q) processes. The FIM of the scalar ARMAX(p, r, q) process contains an ARMA(p, q) part, this is confirmed by (65), through the presence of the matrix ℘(ϑ) which is originally displayed in (28). The multiple resultant matrices M(ϑ) and M_x(ϑ) derived from the FIM of the VARMA(p, q) and VARMAX(p, r, q) processes respectively both contain P(ϑ), whereas the first matrix term of the matrices Φ(z) and Φ_x(z), which are of different size, consist of the same nonzero submatrices. To summarize, in [23] compact forms of the FIM of a VARMAX process expressed in terms of multiple and tensor Sylvester matrices are developed. The tensor Sylvester matrices allow us to investigate the multiple resultant matrix property of the FIM of VARMAX(p, r, q) processes. However, since no proof of the multiple resultant matrix property of the FIM G(ϑ) has been done yet, justifies the consideration of a conjecture. A conjecture that states, the FIMG(ϑ) of a VARMAX(p, r, q) process becomes singular if and only if the matrix polynomials A(z), B(z) and C(z) have at least one common eigenvalue. A multiple equivalent to Theorem 3.1 in [4] and combined with Proposition 2.4 in [6], but based on the representations (79) and (81), can be envisaged to formulate a proof which will be a subject for future study.

4. Conclusions

In this survey paper, matrix algebraic properties of the FIM of stationary processes are discussed. The presented material is a summary of papers where several matrix structural aspects of the FIM are investigated. The FIM of scalar and multiple processes like the (V)ARMA(X) are set forth with appropriate factorized forms involving (tensor) Sylvester matrices. These representations enable us to prove the resultant matrix property of the corresponding FIM. This has been done for (V)ARMA(p, q) and ARMAX(p, r, q) processes in the papers [4–6]. The development of the stages that lead to the appropriate factorized form of the FIM G(ϑ) (79) is set forth in [23]. However, there is no proof done yet that confirms the multiple resultant matrix property of the FIM G(ϑ) of a VARMAX(p, r, q) process. This justifies the consideration of a conjecture which is formulated in the former section, this can be a subject for future study.

The statistical distance measure derived in [7], involves entries of the FIM. This distance measure can be a challenge to its quantum information counterpart (41). Because (36) involves information about m parameters estimated from n measurements. Whereas in quantum information, like in e.g., [8–10], the information about one parameter in a particular measurement procedure is considered for establishing an interconnection with the appropriate statistical distance measure. A possible approach, by combining matrix algebra and quantum information, for developing a statistical distance measure in quantum information or quantum statistics but at the matrix level, can be a subject of future research. Some results concerning interconnections between the FIM of ARMA(X) models and appropriate solutions to Stein matrix equations are discussed, the material is extracted from the papers, [12] and [13]. However, in this paper, some alternative and new proofs that emphasize the conditions under which the FIM fulfills appropriate Stein equations, are set forth. The presence of various types of Vandermonde matrices is also emphasized when an explicit expansion of the FIM is computed. These Vandermonde matrices are inserted in interconnections with appropriate solutions to Stein equations. This explains, when the matrix algebraic structures of the FIM of stationary processes are investigated, the involvement of structured matrices like the (tensor) Sylvester, Bezoutian and Vandermonde matrices is essential.