Probabilistic Perturbation Bounds for Invariant, Deflating and Singular Subspaces

Abstract

In this paper, we derive new probabilistic bounds on the sensitivity of invariant subspaces, deflation subspaces and singular subspaces of matrices. The analysis exploits a unified method for deriving asymptotic perturbation bounds of the subspaces under interest and utilizes probabilistic approximations of the entries of random perturbation matrices implementing the Markoff inequality. As a result of the analysis, we determine with a prescribed probability asymptotic perturbation bounds on the angles between the corresponding perturbed and unperturbed subspaces. It is shown that the probabilistic asymptotic bounds proposed are significantly less conservative than the corresponding deterministic perturbation bounds. The results obtained are illustrated by examples comparing the known deterministic perturbation bounds with the new probabilistic bounds.

1. Introduction

In this paper, we are concerned with the derivation of realistic perturbation bounds on the sensitivity of various important subspaces arising in matrix analysis. Such bounds are especially needed in the perturbation analysis of high dimension subspaces when the known bounds may produce very pessimistic results. We show that much tighter bounds on the subspace sensitivity can be obtained by using a probabilistic approach based on the Markoff inequality.

The sensitivity of invariant, deflation and singular subspaces of matrices is considered in detail in the fundamental book of Stewart and Sun [1], as well as in the surveys of Bhatia [2] and Li [3]. In particular, perturbation analysis of the eigenvectors and invariant subspaces of matrices affected by deterministic and random perturbations is presented in several papers and books, see for instance [4,5,6,7,8,9,10,11,12]. Survey [13] is entirely devoted to the asymptotic (first-order) perturbation analysis of eigenvalues and eigenvectors. The algorithmic and software problems in computing invariant subspaces are discussed in [14]. The sensitivity of deflating subspaces arising in the generalized Schur decomposition is considered in [15,16,17,18], and numerical algorithms for analyzing this sensitivity are presented in [19]. Bounds on the sensitivity of singular values and singular spaces of matrices that are subject to random perturbations are derived in [20,21,22,23,24,25,26], to name a few. The stochastic matrix theory that can be used in case of stochastic perturbations is developed in the papers of Stewart [27], and Edelman and Rao [28].

In [29], the author proposed new componentwise perturbation bounds of unitary and orthogonal matrix decomposition based on probabilistic approximations of the entries of random perturbation matrices implementing the Markoff inequality. It was shown that using such bounds it is possible to decrease significantly the asymptotic perturbation bounds of the corresponding similarity or equivalence transformation matrices. Based on the probabilistic asymptotic estimates of the entries of random perturbation matrices, presented in [29], in this paper we derive new new probabilistic bounds on the sensitivity of invariant subspaces, deflation subspaces and singular subspaces of matrices. The analysis and the examples given demonstrate that, in contrast to the known deterministic bounds, the probabilistic bounds are much tighter with a sufficiently high probability. The analysis performed exploits a unified method for deriving asymptotic perturbation bounds of the subspaces under interest, developed in [30,31,32,33,34], and utilizes probabilistic approximations of the entries of random perturbation matrices implementing the Markoff inequality. As a result of the analysis, we determine, with a prescribed probability, asymptotic perturbation bounds on the angles between the perturbed and unperturbed subspaces. It is proved that the new probabilistic asymptotic bounds are significantly less conservative than the corresponding deterministic perturbation bounds. The results obtained are illustrated by examples comparing the deterministic perturbation bounds derived by Stewart [16,35] and Sun [11,18] with the probabilistic bounds derived in this paper.

The paper is structured as follows. In Section 2, we briefly present the main results concerning the derivation of lower magnitude bounds on the entries of a random matrix using only its Frobenius norm. In the next three sections, Section 3, Section 4 and Section 5, we show the application of this approach to derive probabilistic perturbation bounds for the invariant, deflating and singular subspaces of matrices, respectively. We illustrate the theoretical results by examples demonstrating that the probability bounds of such subspaces are much tighter than the corresponding deterministic asymptotic bounds. We note that the known deterministic bounds for the invariant, deflating and singular subspaces are presented briefly as theorems without proof, only with the purpose to compare them with the new bounds.

All computations in the paper are performed with MATLAB^® Version 9.9 (R2020b) [36] using IEEE double-precision arithmetic. M-files implementing the perturbation bounds described in the paper can be obtained from the author.

2. Probabilistic Bounds for Random Matrices

Consider an $m\times n$ random matrix, $\delta A$, with uncorrelated elements. In the componentwise perturbation analysis of matrix decompositions, we have to use a matrix bound $\Delta A=[\Delta a_{ij}],\Delta a_{ij}>0$, so that $\left|\delta A\right|⪯\Delta A$, i.e.,

$$|\delta a_{ij}|\le \Delta a_{ij},i=1,2,\dots ,m,j=1,2,\dots ,n,$$

(1)

where $\Delta a_{ij}=\parallel \delta A\parallel$ and $\parallel .\parallel$ is some matrix norm. However, if for instance we use the Frobenius norm of $\delta A$, we have that $\Delta a_{ij}={\parallel \delta A\parallel }_F$ and

$${\parallel \Delta A\parallel }_F=\sqrt{mn}{\parallel \delta A\parallel }_F,$$

which produces very pessimistic results for a large m and n. To reduce ${\parallel \Delta A\parallel }_F$, in [29] it is proposed to decrease the entries of $\Delta A$, taking a bound with entries $\Delta a_{ij}={\parallel \delta A\parallel }_F/\Xi$, where $\Xi >1$. Of course, in the general case, such a bound will not satisfy (1) for all i and j. However, we can allow to exist some entries, $\delta a_{ij}$, of the perturbation $\delta A$ that exceed in magnitude, with some prescribed probability, the corresponding bound, $\Delta a_{ij}$. This probability can be determined by the Markoff inequality ([37], Section 5-4)

$$\mathcal{P}\{\xi \ge a\}\le {\displaystyle \frac{E\left\{\xi \right\}}{a}},$$

(2)

where $\mathcal{P}\{\xi \ge a\}$ is the probability that the random variable $\xi$ is greater or equal to a given number, a, and $E\left\{\xi \right\}$ is the average (or mean value) of $\xi$. Note that this inequality is valid for arbitrary distribution of $\xi$, which makes it conservative for a specific probability distribution. Applying the Markoff inequality, with $\xi$ equal to the entry $|\delta a_{ij}|$ and a equal to the corresponding bound $\Delta a_{ij}$, we obtain the following result [29].

Theorem 1. 

For an $m\times n$ random perturbation, $\delta A$, and a desired probability $0<{\mathcal{P}}^{ref}<1$, the estimate $\Delta A=[\Delta a_{ij}]$, where

$$\Delta a_{ij}={\displaystyle \frac{{\parallel \delta A\parallel }_F}{\Xi }},$$

and

$$\Xi =(1-{\mathcal{P}}^{ref})\sqrt{mn},$$

(3)

satisfies the inequality

$$\mathcal{P}\left\{\right|\delta a_{ij}|<\Delta a_{ij}\}\ge {\mathcal{P}}^{ref},i=1,2,\dots ,m,j=1,2,\dots ,n.$$

Theorem 1 allows the decrease of the mean value of the bound, $\Delta A$, and hence the magnitude of its entries by the quantity $\Xi$, choosing the desired probability, ${\mathcal{P}}^{ref}$, less than 1. The value ${\mathcal{P}}^{ref}=1$ corresponds to the case of the deterministic bound, $\Delta A$, with entries $\Delta a_{ij}={\parallel \delta A\parallel }_F$, when it is fulfilled that all entries of $\Delta A$ are larger than or equal to the corresponding entries of the perturbation, $\delta A$. The value ${\mathcal{P}}^{ref}<1$ corresponds to $\Delta a_{ij}={\parallel \delta A\parallel }_F/\Xi$, where $\Xi >1$. As mentioned above, the probability bound produced by the Markoff inequality is very conservative, with the actual results being much better than the results predicted by the probability, ${\mathcal{P}}^{ref}$. This is due to the fact that the Markoff inequality is valid for the worst possible distribution of the random variable $\xi$.

According to Theorem 1, the using of the scaling factor (3) guarantees that the inequality

$$|\delta a_{ij}|<\Delta a_{ij}$$

holds for each i and j with a probability no less than ${\mathcal{P}}^{ref}$. Since the entries of $\delta A$ are uncorrelated, this means that, for sufficiently large m and n, the number ${\mathcal{P}}^{ref}\%$ also gives a lower bound on the relative number of the entries that satisfy the above inequality.

In some cases, for instance in the perturbation analysis of the Singular Value Decomposition, tighter perturbation bounds are obtained if instead of the norm ${\parallel \delta A\parallel }_F$ we use the spectral norm ${\parallel \delta A\parallel }_2$. The following result is an analogue of Theorem 1 that allows us to use ${\parallel \delta A\parallel }_2$ at the price of producing smaller values of $\Xi$.

Theorem 2. 

For an $m\times n$ random perturbation, $\delta A$, and a desired probability $0<{\mathcal{P}}^{ref}<1$, the estimate $\Delta A=[\Delta a_{ij}]$, where

$$\Delta a_{ij}={\displaystyle \frac{{\parallel \delta A\parallel }_2}{{\Xi }_2}},$$

and

$${\Xi }_2=(1-{\mathcal{P}}^{ref})\sqrt{m},$$

(4)

satisfies the inequality

$$\mathcal{P}\left\{\right|\delta a_{ij}|<\Delta a_{ij}\}\ge {\mathcal{P}}^{ref},i=1,2,\dots ,m,j=1,2,\dots ,n.$$

(5)

This result follows directly from Theorem 1, replacing ${\parallel \delta A\parallel }_F$ by its upper bound $\sqrt{n}{\parallel \delta A\parallel }_2$.

Since, frequently, ${\parallel \delta A\parallel }_2$ is of the order of ${\parallel \delta A\parallel }_F$, for a large n, Theorem 2 may produce pessimistic results in the sense that the actual probability of fulfilling the inequality $|\delta a_{ij}|<\Delta a_{ij}$ is much larger than the value predicted by (5).

In several instances of the perturbation analysis, we have to determine a bound on the elements of the vector

$$x=Mf,$$

(6)

where M is a given matrix and $f\in {\mathbb{R}}^p$ is a random vector with a known probabilistic bound on the elements. In accordance with (6), we have that the following deterministic asymptotic (linear) componentwise bound is valid,

$$|x_{\ell }|\le x_{\ell }^{lin}:=\parallel M_{\ell ,1:p}{\parallel }_2{\parallel f\parallel }_2,\ell =1,2,\dots ,p.$$

(7)

A probability bound on $|x_{\ell }|,\ell =1,2,\dots ,p$ can be determined by the following theorem [29].

Theorem 3. 

If the estimate of the parameter vector x is chosen as $x^{est}=\left|Mf\right|/\Xi$, where Ξ is determined according to

$$\Xi =\sqrt{p}(1-{\mathcal{P}}^{ref}),$$

(8)

then

$$\mathcal{P}\left\{\right|x_{\ell }|\le \parallel x^{est}{\parallel }_2\}\le {\mathcal{P}}^{ref}.$$

(9)

Since

$$\parallel x^{est}{\parallel }_2\le {\parallel M\parallel }_2{\parallel f\parallel }_2/\Xi ,$$

the inequality (9) shows that the probability estimate of the component $|x_{\ell }|$ can be determined if in the linear estimate (7) we replace the perturbation norm ${\parallel f\parallel }_2$ by the probability estimate ${\parallel f\parallel }_2/\Xi$, where the scaling factor, $\Xi$, is taken as shown in (8) for a specified probability, ${\mathcal{P}}^{ref}$. In this way, instead of the linear estimate, $x_{\ell }^{lin}$, we obtain the probabilistic estimate

$$x_{\ell }^{est}=x_{\ell }^{lin}/\Xi =\parallel M_{\ell ,1:p}{\parallel }_2{\parallel f\parallel }_2/\Xi ,\ell =1,2,\dots ,p.$$

3. Perturbation Bounds for Invariant Subspaces

3.1. Problem Statement

Let

$$U^{H}AU=\left[\begin{array}{cc}{T}_{11}& T_{12}\\ 0& T_{22}\end{array}\right]$$

(10)

be the Schur decomposition of the matrix $A\in {\mathbb{C}}^{n\times n}$, where $T_{11}\in {\mathbb{C}}^{k\times k}$ contains a given group of the eigenvalues of A ([38], Section 2.3). The matrix, U, of the unitary similarity transformation can be partitioned as

$$U=[U_1,U_2],U_1\in {\mathbb{C}}^{n\times k},$$

where the columns of $U_1$ are the basis vectors of the invariant subspace, $\mathcal{X}$, associated with the eigenvalues of the block $T_{11}$, and $U_2\in {\mathbb{C}}^{n\times (n-k)}$ is the unitary complement of $U_1$, $U_2^H{U}_1=0$. The invariant subspace satisfies the relation $A\mathcal{X}\subset \mathcal{X}$. Note that the eigenvalues of A can be reordered in the desired way on the diagonal of T (and hence of $T_{11}$) using unitary similarity transformations ([39], Chapter 7).

The invariant subspace, $\mathcal{X}$, is called simple if the matrices $T_{11}$ and $T_{22}$ have no eigenvalues in common.

If matrix A is a subject to a perturbation, $\delta A$, then, instead of the decomposition (10), we have the decomposition

$${\tilde{U}}^H\tilde{A}\tilde{U}=\left[\begin{array}{cc}{\tilde{T}}_{11}& {\tilde{T}}_{12}\\ 0& {\tilde{T}}_{22}\end{array}\right],{\tilde{T}}_{11}\in {\mathbb{C}}^{k\times k},\tilde{A}=A+\delta A$$

(11)

with a perturbed matrix of the unitary transformation

$$\tilde{U}=[{\tilde{U}}_1,{\tilde{U}}_2].$$

The columns of matrix ${\tilde{U}}_1$ are basis vectors of the perturbed invariant subspace, $\tilde{\mathcal{X}}$. We shall assume that matrix A has distinct eigenvalues, i.e., $\mathcal{X}$ is a simple invariant subspace that ensures finite perturbations $\delta U=\tilde{U}-U$ and $\delta T=\tilde{T}-T$ for small perturbations of A.

Let $\mathcal{X}=\mathcal{R}\left(U_X\right)$ and $\mathcal{Y}=\mathcal{R}\left(U_Y\right)$ be two subspaces of dimension k, where $\mathrm{rank}\left(U_X\right)=\mathrm{rank}\left(U_Y\right)=k$. The distance between $\mathcal{X}$ and $\mathcal{Y}$ can be characterized by the gap between these subspaces, defined as [11]

$${\mathrm{gap}}_F(\mathcal{X},\mathcal{Y})={\displaystyle \frac{1}{\sqrt{2}}}{\parallel P_X-P_Y\parallel }_F,$$

(12)

where $P_X$ and $P_Y$ are the orthogonal projections onto $\mathcal{X}$ and $\mathcal{Y}$, respectively. Further on, we shall measure the sensitivity of an invariant subspace of dimension k by the canonical angles

$${\Theta }_1\ge {\Theta }_2\ge \cdots \ge {\Theta }_k\ge 0$$

between the perturbed and unperturbed subspaces ([35], Chapter 4). The maximum angle ${\Theta }_{\max }={\Theta }_1$ is related to the value of the gap between $\tilde{\mathcal{X}}$ and $\mathcal{X}$ by the relationship [40]

$${\Theta }_{\max }\le \arcsin \left({\mathrm{gap}}_F(\tilde{\mathcal{X}},\mathcal{X})\right).$$

(13)

We note that the maximum angle between $\tilde{\mathcal{X}}$ and $\mathcal{X}$ can be computed efficiently from [41]

$${\Theta }_{\max }(\tilde{\mathcal{X}},\mathcal{X})=\arcsin (\parallel U_2^H{\tilde{U}}_1{\parallel }_2).$$

(14)

3.2. $sep$-Based Global Bound

Define

$$E=U^H\delta AU=\left[\begin{array}{cc}{E}_{11}& E_{12}\\ E_{21}& E_{22}\end{array}\right],E_{11}\in {\mathbb{C}}^{k\times k},E_{22}\in {\mathbb{C}}^{(n-k)\times (n-k)}$$

and

$$M=I_k\otimes T_{22}-T_{11}^T\otimes I_{n-k},$$

where ⊗ denotes the Kronecker product ([42], Chapter 4). The norm of the matrix $M^{-1}$ is closely related to the quantity separation between two matrices. The separation between given matrices $A\in {\mathbb{C}}^{n\times n}$ and $B\in {\mathbb{C}}^{m\times m}$ characterizes the distance between the spectra $\lambda \left(A\right)$ and $\lambda \left(B\right)$ and is defined as

$$\mathrm{sep}(A,B)={\min }_{X\ne 0}{\displaystyle \frac{{\parallel AX-XB\parallel }_F}{{\parallel X\parallel }_F}}.$$

(15)

Note that $\mathrm{sep}(A,B)=0$, if and only if A and B have eigenvalues in common.

In the given case, the separation

$$\mathrm{sep}(T_{11},T_{22})={\min }_{X\ne 0}{\displaystyle \frac{\parallel T_{22}X-X{T}_{11}{\parallel }_F}{{\parallel X\parallel }_F}}$$

(16)

between the two blocks $T_{11}$ and $T_{22}$ can be determined from

$$\mathrm{sep}(T_{11},T_{22})={\sigma }_{\min }\left(M\right)$$

(17)

which is equivalent to

$$\parallel M^{-1}{\parallel }_2=1/\mathrm{sep}(T_{11},T_{22}).$$

Assume that the spectra of $T_{11}$ and $T_{22}$ are disjoint, so that $\mathrm{sep}(T_{11},T_{22})>0$. Then the following theorem gives an estimate of the sensitivity of an invariant subspace of A.

Theorem 4 

([16]). Let matrix A be decomposed, as in (10). Given the perturbation, E, set

$$\begin{array}{ccc}\hfill \gamma & =& \parallel E_{21}{\parallel }_2,\hfill \\ \hfill \eta & =& \parallel T_{12}+E_{12}{\parallel }_2,\hfill \\ \hfill \delta & =& \mathrm{sep}(T_{11},T_{22})-\parallel E_{11}{\parallel }_2-{\parallel E_{22}\parallel }_2.\hfill \end{array}$$

If $\delta >0$ and

$${\displaystyle \frac{\gamma \eta }{{\delta }^2}}<{\displaystyle \frac{1}{4}},$$

(18)

then there is a unique matrix, P, satisfying

$${\parallel P\parallel }_2\le {\displaystyle \frac{2\gamma }{\delta +\sqrt{{\delta }^2-4\gamma \eta }}}<2{\displaystyle \frac{\gamma }{\delta }},$$

(19)

such that the columns of ${\tilde{U}}_1=(U_1+U_{2}P){(I_k+P^{H}P)}^{-1/2}$ span a right invariant subspace of $A+\delta A$.

It may be shown that the singular values of the matrix

$$U_2^H{\tilde{U}}_1=P{(I_k+P^{H}P)}^{-1/2}$$

are the sines of the canonical angles ${\Theta }_1,{\Theta }_2,\dots ,{\Theta }_k$ between the invariant subspaces $\tilde{X}$ and $\mathcal{X}$. That is why, if P has singular values ${\sigma }_1,{\sigma }_2,\dots {\sigma }_k$, then the singular values of $I_k+P^{H}P$ are

$$1+{\sigma }_1^2,1+{\sigma }_2^2,\dots ,1+{\sigma }_k^2$$

and

$${\displaystyle \frac{{\sigma }_i}{\sqrt{1+{\sigma }_i^2}}}=\sin \left({\Theta }_i\right).$$

Hence

$${\sigma }_i=\tan \left({\Theta }_i\right),i=1,2,\dots ,k.$$

Thus, Theorem 4 bounds the tangents of the canonical angles between the perturbed, $\tilde{\mathcal{X}}$, and unperturbed, $\mathcal{X}$, invariant subspace of A. The maximum canonical angle fulfils

$${\Theta }_{\max }(\tilde{\mathcal{X}},\mathcal{X})\le \arctan \left({\displaystyle \frac{2\gamma }{\delta +\sqrt{{\delta }^2-4\gamma \eta }}}\right).$$

(20)

3.3. Perturbation Expansion Bound

A global perturbation bound for invariant subspaces is derived by Sun [11,40] using the perturbation expansion method. The essence of this method is to expand the perturbed basis ${\tilde{U}}_X$ in infinite series in the powers of $E_{11},E_{12},E_{21},E_{22}$ and then estimate the series sum.

Theorem 5 

([11]). Let matrix A be decomposed, as in (10). Given the perturbation $E=U^H\delta AU$, set

$$\begin{array}{ccc}\hfill \beta & =& \parallel E_{11}{\parallel }_2+\parallel E_{22}{\parallel }_2,{\beta }_1=\parallel E_{12}{\parallel }_F,{\beta }_2={\parallel E_{21}\parallel }_F,\hfill \\ \hfill {\alpha }_{12}& =& \parallel T_{12}{\parallel }_2,\delta =\mathrm{sep}(T_{11},T_{22}),\hfill \\ \hfill {\gamma }_1& =& {\displaystyle \frac{{\beta }_2}{\delta }},{\gamma }_2={\displaystyle \frac{\beta {\gamma }_1+{\alpha }_{12}{\gamma }_1^2}{\delta }}\hfill \end{array}$$

and

$${\gamma }_m={\displaystyle \frac{\beta {\gamma }_{m-1}+{\beta }_1{\sum }_{j=1}^{m-2}{\gamma }_{m-1-j}{\gamma }_j+{\alpha }_{12}{\sum }_{j=1}^{m-1}{\gamma }_{m-j}{\gamma }_j}{\delta }}.$$

(21)

Then the simple invariant subspace, $\mathcal{X}$, of the matrix $A+\delta A$ has the following qth-order perturbation estimation for any natural number, q:

$$d_F(\tilde{\mathcal{X}},\mathcal{X})\le \sum _{m=1}^q{\gamma }_m+{O(\parallel \delta A\parallel }_F^{q+1}).$$

(22)

Theorem 5 can be used to estimate the canonical angles between the perturbed and unperturbed invariant subspaces. Taking into account (13), we obtain the bound

$${\Theta }_{\max }(\tilde{\mathcal{X}},\mathcal{X})=\arcsin \left(\sum _{m=1}^q{\gamma }_m\right).$$

(23)

The implementation of Theorem 5 to estimate the sensitivity of an invariant subspace shows that the bound (23) tends to overestimate severely the true value of the angle ${\Theta }_{\max }$ for $q>1$ and large n. In practice, it is possible to obtain reasonable results if we use only the first-order term ($q=1$) in the expansion (23), i.e., if we use the linear bound

$${\Theta }_{\max }(\tilde{\mathcal{X}},\mathcal{X})=\arcsin \left({\gamma }_1\right)=\arcsin \left({\displaystyle \frac{\parallel E_{21}{\parallel }_F}{\delta }}\right).$$

(24)

3.4. Bound by the Splitting Operator Method

The essence of the splitting operator method for perturbation analysis of matrix problems [31] consists in the separate deriving of perturbation bounds on $\left|\delta U\right|$ and $\left|\delta T\right|$. For this aim, we introduce the perturbation parameter vector

$$x=\mathrm{vec}\left(\mathrm{Low}\left(\delta W\right)\right)\in {\mathbb{C}}^p,p=n(n-1)/2,$$

where the components of x are the entries of the strictly lower triangular part of the matrix $\delta W=U^H\delta U$. This vector is then used to find bounds on the various elements of the Schur decomposition.

Let

$$F=-U^H\left(\delta A\right)U$$

and construct the vector

$$f=\mathrm{vec}\left(\mathrm{Low}\left(F\right)\right)\in {\mathbb{C}}^p.$$

The equation for the perturbation parameters represents a linear system of equations [32]

$$Mx=f+{\Delta }^x,$$

(25)

where $M\in {\mathbb{C}}^{p\times p}$ is a matrix whose elements are determined from the entries of T, and the components of the vector ${\Delta }^x\in {\mathbb{C}}^p$ contain higher-order terms in the perturbations $\delta u_i,i=1,2,\dots ,n$. Specifically, matrix M is determined by

$$M=\Omega (I_n\otimes T-T^T\otimes I_n){\Omega }^T\in {\mathbb{C}}^{p\times p},$$

(26)

where

$$\begin{array}{ccc}\hfill \Omega & :=& \left[\mathrm{diag}({\Omega }_1,{\Omega }_2,\dots ,{\Omega }_{n-1}),0_{p\times n}\right]\in {\mathbb{R}}^{p\times n^2},\hfill \\ \hfill {\Omega }_k& :=& \left[0_{(n-k)\times k},I_{n-k}\right]\in {\mathbb{R}}^{(n-k)\times n},k=1,2,\dots ,n-1.\hfill \end{array}$$

(27)

Note that matrix M is non-singular although the matrix $I_n\otimes T-T^T\otimes I_n$ is not of full rank.

Equation (25) is independent from the equations that determine the perturbations of the elements of the Schur form T. This first allows us to solve (25) and estimate $\left|\delta U\right|$ and then to use the solution obtained to determine bounds on the elements of $\left|\delta T\right|$.

Neglecting the second-order term ${\Delta }^x$ in (25), we obtain the first-order (linear) approximation of x,

$$x=M^{-1}f.$$

(28)

Since ${\parallel f\parallel }_2\le {\parallel \delta A\parallel }_F$, we have that

$$|x_{\ell }|⪯x_{\ell }^{lin},\ell =1,2,\dots ,p,$$

where

$$x_{\ell }^{lin}=\parallel M_{\ell ,1:p}^{-1}{\parallel }_2{\parallel \delta A\parallel }_F$$

(29)

is the asymptotic bound on $|x_{\ell }|$.

The matrix $\left|\delta W\right|$ can be estimated as

$$\left|\delta W\right|⪯\delta W^{lin}+{\Delta }^W,$$

(30)

where

$$\delta W^{lin}=\left[\begin{array}{ccccc}0& x_1^{lin}& x_2^{lin}& \dots & x_{n-1}^{lin}\\ x_1^{lin}& 0& x_n^{lin}& \dots & x_{2n-3}^{lin}\\ x_2& x_n^{lin}& 0& \dots & x_{3n-6}^{lin}\\ ⋮& ⋮& ⋮& \ddots & ⋮\\ x_{n-1^{lin}}& x_{2n-3}^{lin}& x_{3n-6}^{lin}& \dots & 0\end{array}\right]\in {\mathbb{R}}^{n\times n}$$

is a first-order approximation of $\left|\delta W\right|$, and ${\Delta }^W$ contains higher-order terms in x. Thus, an asymptotic (linear) approximation of the matrix $\left|\delta U\right|$ can be determined as

$$\left|\delta U\right|⪯\delta U^{lin}=\left|U\right||U^H\delta U|=|U|\delta W^{lin}.$$

(31)

Since

$${\tilde{U}}_1=U_1+\delta U_1,U_2^H{U}_1=0,$$

one has that

$$\sin \left({\Theta }_{\max }(\tilde{\mathcal{X}},\mathcal{X})\right)=\parallel U_2^H\delta U_1{\parallel }_2={\parallel \delta W_{k+1:n,1:k}\parallel }_2.$$

(32)

Equation (32) shows that the sensitivity of the invariant subspace, $\mathcal{X}$, of dimension k is connected to the values of the perturbation parameters $x_{\ell }=u_i^H\delta u_j,\ell =i+(j-1)n-{\displaystyle \frac{j(j+1)}{2}},i>k,j=1,2,\dots ,k$. Consequently, if the perturbation parameters are known, it is possible to find at once sensitivity estimates for all invariant subspaces with dimension $k=1,2,\dots ,n-1$. More specifically, let

$$\delta W=\left[\begin{array}{ccccc}\ast & \ast & \ast & \dots & \ast \\ x_1& \ast & \ast & \dots & \ast \\ x_2& x_n& \ast & \dots & \\ ⋮& ⋮& ⋮& \ddots & ⋮\\ x_{n-1}& x_{2n-3}& x_{3n-6}& \dots & \ast \end{array}\right],$$

where * is an unspecified entry. Then we have that the maximum angle between the perturbed and unperturbed invariant subspaces of dimension k is

$${\Theta }_{\max }(\tilde{\mathcal{X}},\mathcal{X})=\arcsin (\parallel \delta W_{k+1:n,1:k}{\parallel }_2).$$

(33)

In this way, we obtain the following result.

Theorem 6. 

Let matrix A be decomposed, as in (10), and assume that the Frobenius norm of the perturbation $\delta A$ is known. Set

$$L=\left[\begin{array}{ccccc}\ast & \ast & \ast & \dots & \ast \\ M_{1,1:p}^{-1}& \ast & \ast & \dots & \ast \\ M_{2,1:p}^{-1}& M_{n,1:p}^{-1}& \ast & \dots & \ast \\ ⋮& ⋮& ⋮& \ddots & ⋮\\ M_{n-1,1:p}^{-1}& M_{2n-3,1:p}^{-1}& M_{3n-6,1:p}^{-1}& \dots & \ast \end{array}\right]\in {\mathbb{C}}^{n\times n\times p},$$

where matrix M is determined by (26). Then the following asymptotic estimate holds,

$${\Theta }_{\max }^{lin}(\tilde{\mathcal{X}},\mathcal{X})\le \arcsin (\parallel L_{k+1:n,1:kp}{\parallel }_2{\parallel f\parallel }_2)\le \arcsin (\parallel L_{k+1:n,1:kp}{\parallel }_2{\parallel \delta A\parallel }_F).$$

(34)

The proof of Theorem 6 follows directly from (33), replacing the matrix $\delta W$ by its linear approximation, $\delta W^{lin}$, and substituting each $x_{\ell }^{lin}$ by its approximation (29). Note that, as always in the case of perturbation bounds, the equality can be achieved only for specially constructed perturbation matrices.

Denote by

$$\delta \lambda =\left[\begin{array}{c}\delta t_{11}\\ \delta t_{22}\\ ⋮\\ \delta t_{nn}\end{array}\right]\in {\mathbb{C}}^n$$

the changes of the diagonal elements of T, i.e., the perturbations of the eigenvalues ${\lambda }_i$ of A. Then the first-order eigenvalue perturbations satisfy

$$|\delta {\lambda }_i|\le \delta {\lambda }_i^{lin},i=1,2,\dots ,n,$$

where

$$\delta {\lambda }_i^{lin}=\parallel Z_{i,1:p+n}{\parallel }_2{\parallel \delta A\parallel }_F,$$

(35)

and

$$Z=[N{M}^{-1},I_n]\in {\mathbb{C}}^{n\times (p+n)},$$

$$N=\Pi (I_n\otimes T-T^T\otimes I_n){\Omega }^T\in {\mathbb{C}}^{n\times p},$$

$$\Pi =\mathrm{diag}\left(e_1^T,e_2^T,\cdots ,e_n^T\right)=\left[e_1{e}_1^T,e_2{e}_2^T,\cdots ,e_n{e}_n^T\right]\in {\mathbb{R}}^{n\times n^2}.$$

The obtained linear bound (35) coincides numerically with the well-known asymptotic bounds from the literature [12,35,39]. The quantity $\parallel Z_{i,1:p+n}{\parallel }_2$ is equal to the condition number of the eigenvalue ${\lambda }_i$.

3.5. Probabilistic Perturbation Bound

The idea of determining tighter perturbation bounds of matrix subspaces consists in replacing the Frobenius or 2-norm of the matrix perturbation by a much smaller probabilistic estimate of the perturbation entries, obtained using Theorems 1 and 3. This allows us to decrease, with a specified probability, the perturbation bounds for the different subspaces, achieving better results for higher dimensional problems. In simple terms, we replace $\parallel \delta A\parallel$ in the corresponding asymptotic estimate by the ratio $\parallel \delta A\parallel /\Xi$, thus decreasing the perturbation bound by the quantity $\Xi >1$, which is determined by the desired probability, ${\mathcal{P}}^{ref}$. We shall illustrate this idea considering first the case of invariant subspaces.

Using Theorem 3, the probabilistic perturbation bounds of x and $\delta U$ in the case of the Schur decomposition can be found from (29) and (31), respectively, replacing in (29) the perturbation norm ${\parallel \delta A\parallel }_F$ by the quantity ${\parallel \delta A\parallel }_F/\Xi$, where $\Xi$ is determined according to (3) from the desired probability, ${\mathcal{P}}^{ref}$, and the problem order, n. In this way, we obtain the probabilistic asymptotic estimate

$${\Theta }_{\max }^{est}(\tilde{\mathcal{X}},\mathcal{X})=\arcsin (\parallel L_{k+1:n,1:kp}{\parallel }_2{\parallel \delta A\parallel }_F/\Xi )$$

(36)

of the maximum angle between the perturbed and unperturbed invariant subspaces of dimension k. In the same way, from (35), we obtain a probabilistic asymptotic estimate

$$\delta {\lambda }_i^{est}=\parallel Z_{i,1:p+n}{\parallel }_2{\parallel \delta A\parallel }_F/\Xi ,i=1,2,\dots ,n,$$

(37)

of the eigenvalue perturbations.

3.6. Bound Comparison

In the next example, we compare the invariant subspace deterministic perturbation bounds, obtained by the $sep$-based approach, the perturbation expansion method and the splitting operator method, with the probabilistic bound obtained by using the Markoff inequality.

Example 1. 

Consider a $100\times 100$ matrix A, taken as

$$A=Q_0{J}_0{Q}_0^{-1},$$

where

$$J_0=\left[\begin{array}{ccccccc}0.1& 0.1& \tau & \tau & \dots & \tau & \tau \\ -0.1& 0.1& \tau & \tau & \dots & \tau & \tau \\ & & 0.2& 0.2& \dots & \tau & \tau \\ & & -0.2& 0.2& \dots & \tau & \tau \\ & & & & \ddots & ⋮& ⋮\\ & & & & & 5.0& 5.0\\ & & & & & -5.0& 5.0\end{array}\right],\tau =0.4$$

and the matrix $Q_0$ is constructed as [43]

$$\begin{array}{ccc}\hfill Q_0& =& H_2\Sigma H_1,Q_0^{-1}=H_1{\Sigma }^{-1}{H}_2,\hfill \\ \hfill H_1& =& I_n-2u{u}^T/n,H_2=I_n-2v{v}^T/n,\hfill \\ \hfill u& =& {[1,1,1,\dots ,1]}^T,v={[1,-1,1,\dots ,{(-1)}^{n-1}]}^T,\hfill \\ \hfill \Sigma & =& \mathrm{diag}(1,\sigma ,{\sigma }^2,\dots ,{\sigma }^{n-1}),\hfill \end{array}$$

where $H_1,H_2$ are elementary reflections, σ is taken equal to $1.065$ and $\mathrm{cond}\left(Q_0\right)=510.0481$. The eigenvalues of A,

$${\lambda }_i=0.1\pm 0.1j_0,0.2\pm 0.2j_0,\dots ,5.0\pm 5.0j_0,$$

are complex conjugated. The perturbation of A is taken as $\delta A={10}^{-12}\times A_0$, where $A_0$ is a matrix with random entries with normal distribution and ${\parallel \delta A\parallel }_F=9.91429\times {10}^{-11}$. The matrix M in (25) is of order $n=4950$ and its inverse satisfies $\parallel M^{-1}{\parallel }_2=8.0431\times {10}^4$, which shows that the eigenvalue problem for A is ill-conditioned since the perturbations of A can be “amplified” ${10}^4$ times in x and, consequently, in $\delta U$ and $\Theta (\tilde{\mathcal{X}},\mathcal{X})$.

In Figure 1, we show the mean value of the matrix $[\Delta a_{ij}/|\delta a_{ij}\left|\right]$ and the relative number $N\{\Delta a_{ij}\ge |\delta a_{ij}\left|\right\}/\left(n^2\right)$ of the entries of the matrix $\left|\delta A\right|$ for which $|\delta a_{ij}|\le |\Delta a_{ij}|$, obtained for normal and uniform distribution of the entries of $\delta A$ and for different values of the desired probability, ${\mathcal{P}}^{ref}$. For the case of normal distribution and ${\mathcal{P}}^{ref}=90\%$, the size of the probability entry bound, $\Delta a_{ij}$, decreases 10 times in comparison with the size of the entry bound ${\parallel \delta A\parallel }_F$, which allows the decrease of the mean value of the ratio $\Delta a_{ij}/\left|\delta a_{ij}\right|$ from $638.38$ to $63.838$ (

Table 1. The mean value of the ratios $[\Delta a_{ij}/|\delta a_{ij}\left|\right]$ and the relative number of entries for which $N\{\Delta a_{ij}>|\delta a_{ij}\left|\right\}/n^2$, obtained for five values of ${\mathcal{P}}^{ref}$, $n=100$.

${\mathcal{P}}^{\mathit{ref}}$	$\mathbf{\Xi }$	$\mathit{E}\left\{\right[\mathbf{\Delta }{\mathit{a}}_{\mathit{ij}}/|\mathit{\delta }{\mathit{a}}_{\mathit{ij}}\left|\right]\}$	$\mathit{N}\{\mathbf{\Delta }{\mathit{a}}_{\mathit{ij}}>|\mathit{\delta }{\mathit{a}}_{\mathit{ij}}\left|\right\}/{\mathit{n}}^2$
%			%
$100.0$	$1.0000\times {10}^0$	$6.3838\times {10}^2$	$100.0000$
$90.0$	$1.0000\times {10}^1$	$6.3838\times {10}^1$	$100.0000$
$80.0$	$2.0000\times {10}^1$	$3.1919\times {10}^1$	$100.0000$
$60.0$	$4.0000\times {10}^1$	$1.5959\times {10}^1$	$98.8200$
$40.0$	$6.0000\times {10}^1$	$1.0640\times {10}^1$	$90.3400$

). For ${\mathcal{P}}^{ref}=40\%$, the probability bound, $\Delta a_{ij}$, is 60 times smaller than the bound ${\parallel \delta A\parallel }_F$, and even for this small desired probability the number of entries for which $|\delta a_{ij}|\le |\Delta a_{ij}|$ is still $90.34\%$.

In Figure 2, we compare the asymptotic bound, $\delta {\lambda }_i^{lin}$, and the probabilistic estimate, $\delta {\lambda }_i^{est}$, with the actual eigenvalue perturbations $|\delta {\lambda }_i|$ for normal distribution of perturbation entries and probabilities ${\mathcal{P}}^{ref}=90\%,80\%$ and $60\%$. (For clarity, the perturbations of the superdiagonal elements of T are hidden). The probabilistic bound, $\delta {\lambda }_i^{est}$, is much tighter than the linear bound, $\delta {\lambda }_i^{lin}$, and the inequality $|\delta {\lambda }_i|\le \delta {\lambda }_i^{est}$ is satisfied for all eigenvalues and all chosen probabilities. In particular, the size of the estimate, $\delta {\lambda }_i^{est}$, is 10 times smaller than the linear estimate, $\delta {\lambda }_i^{lin}$, for ${\mathcal{P}}^{ref}=90\%$, 20 times for ${\mathcal{P}}^{ref}=80\%$ and 40 times for ${\mathcal{P}}^{ref}=60\%$.

In Figure 3, we show the asymptotic bound, $\delta {\Theta }_k^{lin}$, and the probabilistic estimate, $\delta {\Theta }_k^{est}$, along with the actual value $|\delta {\Theta }_k|$ of the maximum angle between the perturbed and unperturbed invariant subspace of dimensions $k=1,2,\dots ,n-1$ for the same probabilities ${\mathcal{P}}^{ref}=90\%,80\%$ and $60\%$. The probability estimate satisfies $|\delta {\Theta }_k|<\delta {\Theta }_k^{est}$ for all $1\le k<100$. For comparison, we give the global bound on the maximum angle between the perturbed and unperturbed invariant subspaces computed by (20) and the first-order bound determined by (24). (The computation of the $sep$-based estimate is performed by using the numerical algorithm, presented in [14].) The global bound is slightly larger than the asymptotic bounds, and the asymptotic bounds (24) and (34) coincide.

4. Perturbation Bounds for Deflating Subspaces

4.1. Problem Statement

Let $A-\lambda B$ be a square matrix pencil [35]. Then there exist unitary matrices U and V, such that $T=U^{H}AV$ and $R=U^{H}BV$ are both upper triangular. The pair $(T,R)$ constitute the generalized Schur form of the pair $(A,B)$. The eigenvalues of $A-\lambda B$ are then equal to $t_{ii}/r_{ii}$, the ratios of the diagonal entries of T and R. Further on, we consider the case of regular pencils for which $\det (A-\lambda B)\ne 0$ and in addition the matrix B is non-singular, i.e., the generalized eigenvalues are finite.

Suppose that the generalized Schur form of the pencil $A-\lambda B$ is reordered as

$$U^{H}AV=\left[\begin{array}{cc}{T}_{11}& T_{12}\\ 0& T_{22}\end{array}\right],U^{H}BV=\left[\begin{array}{cc}{R}_{11}& R_{12}\\ 0& R_{22}\end{array}\right],$$

(38)

where $T_{11}-\lambda R_{11}$ contains k specified eigenvalues. Partitioning conformally the unitary matrices of the equivalence transformation as $U=[U_1,U_2],V=[V_1,V_2]$, we obtain orthonormal bases $U_1\in {\mathbb{C}}^{n\times k}$ and $V_1\in {\mathbb{C}}^{n\times k}$ of the left, $\mathcal{Y}$, and right, $\mathcal{X}$, deflating subspace, respectively. The deflating subspaces satisfy the relations $A\mathcal{X}\subset \mathcal{Y}$ and $B\mathcal{X}\subset \mathcal{Y}$.

Further on, we are interested in the sensitivity of the deflating subspaces corresponding to specified generalized eigenvalues of the pair $(T_{11},R_{11})$. Suppose that the matrices of the pencil $A-\lambda B$ are perturbed as $\tilde{A}=A+\delta A$ and $\tilde{B}=B+\delta B$. The upper triangular matrices in the generalized Schur decomposition are represented as

$${\tilde{U}}^H\tilde{A}\tilde{V}=\left[\begin{array}{cc}{\tilde{T}}_{11}& {\tilde{T}}_{12}\\ 0& {\tilde{T}}_{22}\end{array}\right],{\tilde{U}}^H\tilde{B}\tilde{V}=\left[\begin{array}{cc}{\tilde{R}}_{11}& {\tilde{R}}_{12}\\ 0& {\tilde{R}}_{22}\end{array}\right],$$

where

$$\tilde{U}=\left[\begin{array}{cc}{\tilde{U}}_1,& {\tilde{U}}_2\end{array}\right],\tilde{V}=\left[\begin{array}{cc}{\tilde{V}}_1,& {\tilde{V}}_2\end{array}\right]$$

are the modified equivalence transformation matrices.

4.2. $dif$-Based Global Bound

Let the unperturbed left and right deflation subspaces corresponding to the first k eigenvalues be denoted by $\mathcal{Y}=\mathcal{R}\left(U_1\right)$ and $\mathcal{X}=\mathcal{R}\left(V_1\right)$, respectively, and their perturbed counterparts as $\tilde{\mathcal{X}}$ and $\tilde{\mathcal{Y}}$.

If $\parallel \delta A\parallel$ and $\parallel \delta B\parallel$ are small, we may expect that the spaces $\tilde{\mathcal{X}}$ and $\tilde{\mathcal{Y}}$ will be close to the spaces $\mathcal{X}$ and $\mathcal{Y}$, respectively.

Define the matrices

$$E=U^H\delta AV=\left[\begin{array}{cc}{E}_{11}& E_{12}\\ E_{21}& E_{22}\end{array}\right],F=U^H\delta BV=\left[\begin{array}{cc}{F}_{11}& F_{12}\\ F_{21}& F_{22}\end{array}\right]$$

and

$$M=\left[\begin{array}{cc}{I}_k\otimes T_{22}& -T_{11}^T\otimes I_{n-k},\\ I_k\otimes R_{22}& -R_{11}^T\otimes I_{n-k}\end{array}\right].$$

Note that matrix M is invertible if and only if the pencils $T_{11}-\lambda R_{11}$ and $T_{22}-\lambda R_{22}$ have no eigenvalues in common.

Define the difference between the spectra of $T_{11}-\lambda R_{11}$ and $T_{22}-\lambda R_{22}$ as [15]

$$\mathrm{dif}(T_{11},R_{11};T_{22},R_{22})={\min }_{[S,W]\ne 0}{\displaystyle \frac{{\left|\left|\left[\begin{array}{c}{T}_{22}S-W{T}_{11}\\ R_{22}S-W{R}_{11}\end{array}\right]\right|\right|}_F}{{\parallel [S,W]\parallel }_F}}.$$

It is possible to prove the relationship

$$\mathrm{dif}(T_{11},R_{11};T_{22},R_{22})={\sigma }_{\min }\left(M\right),$$

so that

$$\parallel M^{-1}{\parallel }_2=1/\mathrm{dif}(T_{11},R_{11};T_{22},R_{22}).$$

Assume that the spectra of $T_{11}-\lambda R_{11}$ and $T_{22}-\lambda R_{22}$ are disjoint so that

$$\mathrm{dif}(T_{11},R_{11};T_{22},R_{22})>0.$$

The following theorem gives an estimate of the sensitivity of a deflating subspace of a regular pencil.

Theorem 7 

([1,15]). Let the regular pencil $A-\lambda B$ be represented as in (38). Given the perturbation $(E,F)$, set

$$\begin{array}{ccc}\hfill \gamma & =& \parallel (E_{21},F_{21}){\parallel }_F,\hfill \\ \hfill \eta & =& \parallel (T_{12}+E_{12},R_{12}+F_{12}){\parallel }_F,\hfill \\ \hfill \delta & =& \mathrm{dif}(T_{11},R_{11};T_{22},R_{22})\hfill \\ & & -\max \{\parallel E_{11}{\parallel }_F+\parallel E_{22}{\parallel }_F,\parallel F_{11}{\parallel }_F+\parallel F_{22}{\parallel }_F\}.\hfill \end{array}$$

If $\delta >0$ and

$${\displaystyle \frac{\eta \gamma }{{\delta }^2}}<{\displaystyle \frac{1}{4}},$$

there are matrices P and Q satisfying

$${\parallel (P,Q)\parallel }_F\le {\displaystyle \frac{2\gamma }{\delta +\sqrt{{\delta }^2-4\gamma \eta }}},$$

(39)

such that the columns of ${\tilde{V}}_1=V_1+V_{2}P$ and ${\tilde{U}}_1=U_1+U_{2}Q$ span right and left deflating subspaces for $A+\delta A-\lambda (B+\delta B)$. Note that

$${\parallel (P,Q)\parallel }_F=\sqrt{{\parallel P\parallel }_F^2+{\parallel Q\parallel }_F^2}.$$

It is possible to show that Equation (39) bounds the tangents of the canonical angles between $\tilde{\mathcal{X}}$ and $\mathcal{X}$ or $\tilde{Y}$ and $\mathcal{Y}$, similarly to the ordinary eigenvalue problem. Specifically, we have that

$$\max \{{\Phi }_{\max }(\tilde{\mathcal{X}},\mathcal{X}),{\Theta }_{\max }(\tilde{\mathcal{Y}},\mathcal{Y})\}\le \arctan \left({\displaystyle \frac{2\gamma }{\delta +\sqrt{{\delta }^2-4\gamma \eta }}}\right).$$

(40)

Theorem 7 can be considered as a generalization of Theorem 4 for the ordinary eigenvalue problem.

4.3. Perturbation Expansion Bound

Global perturbation bounds for deflating subspaces that produce individual perturbation bounds for each subspace in a pair of deflating subspaces are presented in [18].

Theorem 8. 

Let $(A,B)$ be an $n\times n$ regular matrix pair represented as in (38), and let $\mathcal{X}=\mathcal{R}\left(V_1\right)$ and $\mathcal{Y}=\mathcal{R}\left(U_1\right)$. Moreover, let ${\kappa }_{\mathcal{X}},{\kappa }_{\mathcal{Y}}$ be defined by

$${\kappa }_{\mathcal{X}}=\parallel P_{\mathcal{X}}{\parallel }_2,{\kappa }_{\mathcal{Y}}={\parallel P_{\mathcal{Y}}\parallel }_2,$$

where

$$\begin{array}{ccc}\hfill P_{\mathcal{X}}& =& [(R_{11}^T\otimes I_{n-k})S^{-1},(-T_{11}^T\otimes I_{n-k})S^{-1}],\hfill \\ \hfill P_{\mathcal{Y}}& =& [(I_k\otimes R_{22})S^{-1},(-I_k\otimes T_{22})S^{-1}],\hfill \\ \hfill S& =& T_{11}\otimes R_{22}-R_{11}^T\otimes T_{22}\hfill \end{array}$$

and let

$$\kappa =\sqrt{{\kappa }_{\mathcal{X}}^2+{\kappa }_{\mathcal{Y}}^2},\gamma ={\left|\left|\left[\begin{array}{c}{E}_{21}\\ F_{21}\end{array}\right]\right|\right|}_F,\eta ={\left|\left|\left[\begin{array}{c}{T}_{12}+E_{12}\\ R_{12}+F_{12}\end{array}\right]\right|\right|}_F,$$

(41)

$$\epsilon =\sqrt{\parallel (E_{11},F_{11}){\parallel }_F^2+{\parallel (E_{22},F_{22})\parallel }_F^2}.$$

(42)

If

$$\kappa (2\sqrt{\gamma \eta }+\epsilon )<1,$$

then there exists a pair $\tilde{\mathcal{X}}=\mathcal{R}\left({\tilde{V}}_1\right)$ and $\tilde{\mathcal{Y}}=\mathcal{R}\left({\tilde{U}}_1\right)$ of k-dimensional deflating subspaces of $(A+\delta A,B+\delta B)$ such that

$$\begin{array}{c}\hfill {\Phi }_{\max }(\tilde{\mathcal{X}},\mathcal{X})\le \arctan \left({\displaystyle \frac{2{\kappa }_{\mathcal{X}}\gamma }{1-\kappa \epsilon +\sqrt{{(1-\kappa \epsilon )}^2-4{\kappa }^2\gamma \eta }}}\right),\end{array}$$

(43)

$$\begin{array}{c}\hfill {\Theta }_{\max }(\tilde{\mathcal{X}},\mathcal{X})\le \arctan \left({\displaystyle \frac{2{\kappa }_{\mathcal{Y}}\gamma }{1-\kappa \epsilon +\sqrt{{(1-\kappa \epsilon )}^2-4{\kappa }^2\gamma \eta }}}\right).\end{array}$$

(44)

4.4. Bound by the Splitting Operator Method

The application of the splitting operator method to the perturbation analysis of the generalized Schur decomposition is performed in [34].

Consider again the generalized Schur decomposition (38). To derive perturbation bounds of the matrices $\delta U_1={\tilde{U}}_1-U_1$ and $\delta V_1={\tilde{V}}_1-V_1$, we introduce the perturbation parameter vectors

$$x=\mathrm{vec}\left(\mathrm{Low}\left(\delta W_U\right)\right)\in {\mathbb{C}}^p,p=n(n-1)/2,$$

and

$$y=\mathrm{vec}\left(\mathrm{Low}\left(\delta W_V\right)\right)\in {\mathbb{C}}^p,$$

where $\delta W_U=U^H\delta U$ and $\delta W_V=V^H\delta V$. Let

$$F=-U^H\left(\delta A\right)V,G=-U^H\left(\delta B\right)V$$

and construct the vectors

$$f=\mathrm{vec}\left(\mathrm{Low}\left(F\right)\right)\in {\mathbb{C}}^p,g=\mathrm{vec}\left(\mathrm{Low}\left(G\right)\right)\in {\mathbb{C}}^p.$$

Then, asymptotic bounds of x and y can be found by solving the linear system of equations

$$\left[\begin{array}{cc}{M}_{xf}& -M_{xg}\\ M_{yf}& -M_{yg}\end{array}\right]\left[\begin{array}{c}x\\ y\end{array}\right]=\left[\begin{array}{c}f\\ g\end{array}\right],$$

(45)

where

$$\begin{array}{cc}\hfill M_{xf}=\Omega (T^T\otimes I_n){\Omega }^T,& M_{xg}=\Omega (I_n\otimes T){\Omega }^T,\end{array}$$

(46)

$$\begin{array}{cc}\hfill M_{yf}=\Omega (R^T\otimes I_n){\Omega }^T,& M_{yg}=\Omega (I_n\otimes R){\Omega }^T\end{array}$$

(47)

and

$$\begin{array}{ccc}\hfill \Omega & :=& \left[\mathrm{diag}({\Omega }_1,{\Omega }_2,\dots ,{\Omega }_{n-1}),0_{p\times n}\right]\in {\mathbb{R}}^{p\times n^2},\hfill \\ \hfill {\Omega }_k& :=& \left[0_{(n-k)\times k},I_{n-k}\right]\in {\mathbb{R}}^{(n-k)\times n},k=1,2,\dots ,n-1.\hfill \end{array}$$

(48)

Hence, linear approximations of the elements of x and y can be determined from

$$\begin{array}{ccc}\hfill x_{\ell }^{lin}& =& \parallel M_{\ell ,1:2p}^{-1}{\parallel }_2\sqrt{{\parallel \delta A\parallel }_F^2+{\parallel \delta B\parallel }_F^2},\hfill \end{array}$$

(49)

$$\begin{array}{ccc}\hfill y_{\ell }^{lin}& =& \parallel M_{\ell +p,1:2p}^{-1}{\parallel }_2\sqrt{{\parallel \delta A\parallel }_F^2+{\parallel \delta B\parallel }_F^2},\hfill \end{array}$$

(50)

$$\begin{array}{ccc}& & \ell =1,2,\dots ,p,\hfill \end{array}$$

(51)

where

$$M=\left[\begin{array}{cc}{M}_{xf}& -M_{xg}\\ M_{yf}& -M_{yg}\end{array}\right].$$

The matrices $|\delta W_U|$ and $|\delta W_V|$ can be estimated as

$$\begin{array}{ccc}\hfill |\delta W_U|& ⪯& \delta W_U^{lin}+{\Delta }^{W_U},\hfill \end{array}$$

(52)

$$\begin{array}{ccc}\hfill |\delta W_V|& ⪯& \delta W_V^{lin}+{\Delta }^{W_V},\hfill \end{array}$$

(53)

where

$$\delta W_U^{lin}=\left[\begin{array}{ccccc}0& x_1^{lin}& x_2^{lin}& \dots & x_{n-1}^{lin}\\ x_1^{lin}& 0& x_n^{lin}& \dots & x_{2n-3}^{lin}\\ x_2& x_n^{lin}& 0& \dots & x_{3n-6}^{lin}\\ ⋮& ⋮& ⋮& \ddots & ⋮\\ x_{n-1^{lin}}& x_{2n-3}^{lin}& x_{3n-6}^{lin}& \dots & 0\end{array}\right]\in {\mathbb{R}}^{n\times n},$$

$$\delta W_V^{lin}=\left[\begin{array}{ccccc}0& y_1^{lin}& y_2^{lin}& \dots & y_{n-1}^{lin}\\ y_1^{lin}& 0& y_n^{lin}& \dots & y_{2n-3}^{lin}\\ y_2& y_n^{lin}& 0& \dots & y_{3n-6}^{lin}\\ ⋮& ⋮& ⋮& \ddots & ⋮\\ y_{n-1^{lin}}& y_{2n-3}^{lin}& y_{3n-6}^{lin}& \dots & 0\end{array}\right]\in {\mathbb{R}}^{n\times n}$$

and ${\Delta }^{W_U},{\Delta }^{W_V}$ contain higher-order terms in $x,y$. Thus, asymptotic approximation of the matrices $\left|\delta U\right|$ and $\left|\delta V\right|$ can be determined as

$$\begin{array}{ccc}\hfill \left|\delta U\right|& ⪯& \delta U^{lin}=\left|U\right||U^H\delta U|=|U|\delta W_U^{lin},\hfill \end{array}$$

(54)

$$\begin{array}{ccc}\hfill \left|\delta V\right|& ⪯& \delta V^{lin}=\left|V\right||V^H\delta V|=|V|\delta W_V^{lin}.\hfill \end{array}$$

(55)

Let ${\tilde{V}}_1$ and $V_1$ be the orthonormal bases of the perturbed and unperturbed right deflation subspace, $\mathcal{X}$, of dimension k, and ${\tilde{U}}_1$ and $U_1$ be, respectively, the orthonormal bases of the perturbed and unperturbed left deflation subspace, $\mathcal{Y}$, of the same dimension. Since

$$\begin{array}{ccc}\hfill {\tilde{V}}_1& =& V_1+\delta V_1,V_2^H{V}_1=0,\hfill \\ \hfill {\tilde{U}}_1& =& U_1+\delta U_1,U_2^H{U}_1=0,\hfill \end{array}$$

we have that

$$\begin{array}{ccc}\hfill \sin \left({\Phi }_{\max }(\tilde{\mathcal{X}},\mathcal{X})\right)& =& \parallel V_2^H\delta V_1{\parallel }_2,\hfill \\ \hfill \sin \left({\Theta }_{\max }(\tilde{\mathcal{Y}},\mathcal{Y})\right)& =& \parallel U_2^H\delta U_1{\parallel }_2.\hfill \end{array}$$

Using these expressions, it is possible to show that

$$\begin{array}{ccc}\hfill {\Phi }_{\max }(\tilde{\mathcal{X}},\mathcal{X})& =& \arcsin (\parallel \delta {W_V}_{k+1:n,1:kp}{\parallel }_2),\hfill \\ \hfill {\Theta }_{\max }(\tilde{\mathcal{Y}},\mathcal{Y})& =& \arcsin \left(\parallel \delta {W_U}_{k+1:n,1:kp}{\parallel }_2\right).\hfill \end{array}$$

Implementing the asymptotic approximations of the elements of the vectors x and y, we obtain the following result.

Theorem 9. 

Let the pair $(A,B)$ be decomposed, as in (38), and assume that the Frobenius norms of the perturbations $\delta A$ and $\delta B$ are known. Set

$$L_X=\left[\begin{array}{ccccc}\ast & \ast & \ast & \dots & \ast \\ M_{1+p,1:2p}^{-1}& \ast & \ast & \dots & \ast \\ M_{2+p,1:2p}^{-1}& M_{n+p,1:2p}^{-1}& \ast & \dots & \ast \\ ⋮& ⋮& ⋮& \ddots & ⋮\\ M_{n-1+p,1:2p}^{-1}& M_{2n-3+p,1:2p}^{-1}& M_{3n-6+p,1:2p}^{-1}& \dots & \ast \end{array}\right]\in {\mathbb{C}}^{n\times 2np},$$

$$L_Y=\left[\begin{array}{ccccc}\ast & \ast & \ast & \dots & \ast \\ M_{1,1:2p}^{-1}& \ast & \ast & \dots & \ast \\ M_{2,1:2p}^{-1}& M_{n,1:2p}^{-1}& \ast & \dots & \ast \\ ⋮& ⋮& ⋮& \ddots & ⋮\\ M_{n-1,1:2p}^{-1}& M_{2n-3,1:2p}^{-1}& M_{3n-6,1:2p}^{-1}& \dots & \ast \end{array}\right]\in {\mathbb{C}}^{n\times 2np}.$$

Then, the following asymptotic bounds of the angles between the perturbed and unperturbed deflation subspaces of dimension $1\le k<n$ are valid,

$$\begin{array}{ccc}\hfill {\Phi }_{\max }^{lin}(\tilde{\mathcal{X}},\mathcal{X})& =& \arcsin \left(\parallel {L_X}_{k+1:p,1:2kp}{\parallel }_2\sqrt{{\parallel \delta A\parallel }_F^2+{\parallel \delta B\parallel }_F^2}\right),\hfill \end{array}$$

(56)

$$\begin{array}{ccc}\hfill {\Theta }_{\max }^{lin}(\tilde{\mathcal{Y}},\mathcal{Y})& =& \arcsin \left(\parallel {L_Y}_{k+1:p,1:2kp}{\parallel }_2\sqrt{{\parallel \delta A\parallel }_F^2+{\parallel \delta B\parallel }_F^2}\right).\hfill \end{array}$$

(57)

Consider the sensitivity of the generalized eigenvalues, i.e., the sensitivity of a simple finite generalized eigenvalue, ${\lambda }_i$, under perturbations of the matrices A and B. If the perturbed pencil is denoted by

$$\tilde{A}+\tilde{\lambda }\tilde{B}=A+\delta A+\tilde{\lambda }(B+\delta B),$$

we want to know how the difference between $\tilde{\lambda }$ and $\lambda$ depends on the size of the perturbation measured by the quantity

$$\epsilon =\sqrt{{\parallel \delta A\parallel }_F^2+{\parallel \delta B\parallel }_F^2}.$$

The distance between two generalized eigenvalues $\lambda =\alpha /\beta$ and $\mu =\gamma /\delta$ is measured by the so-called chordal distance, defined as [15]

$$\chi (\lambda ,\mu )={\displaystyle \frac{|\alpha \delta -\beta \gamma |}{\sqrt{{\left|\alpha \right|}^2+{\left|\beta \right|}^2}\sqrt{{\left|\gamma \right|}^2+{\left|\delta \right|}^2}}}.$$

Substituting

$$\alpha =t_{ii},\tilde{\alpha }={\tilde{t}}_{ii},\beta =r_{ii},\tilde{\beta }={\tilde{r}}_{ii},$$

we find that

$$\chi (\lambda ,\tilde{\lambda })={\displaystyle \frac{|t_{ii}{\tilde{r}}_{ii}-r_{ii}{\tilde{t}}_{ii}|}{\sqrt{|t_{ii}{|}^2+{|r_{ii}|}^2}\sqrt{|{\tilde{t}}_{ii}{|}^2+{|{\tilde{r}}_{ii}|}^2}}}$$

or, in a first order-approximation,

$${\chi }^{lin}(\lambda ,\tilde{\lambda })={\displaystyle \frac{|r_{ii}\delta t_{ii}+t_{ii}\delta r_{ii}|}{|t_{ii}{|}^2+{|r_{ii}|}^2}}$$

(58)

The asymptotic approximations of the diagonal element perturbations of the matrices R and T satisfy

$$\begin{array}{ccc}\hfill \delta t_{ii}& =& \mathrm{cond}\left(t_{ii}\right)\sqrt{{\parallel \delta A\parallel }_F^2+{\parallel \delta B\parallel }_F^2},\hfill \end{array}$$

(59)

$$\begin{array}{ccc}\hfill \delta r_{ii}& =& \mathrm{cond}\left(r_{ii}\right)\sqrt{{\parallel \delta A\parallel }_F^2+{\parallel \delta B\parallel }_F^2},\hfill \end{array}$$

(60)

where the numbers $\mathrm{cond}\left(t_{ii}\right)$ and $\mathrm{cond}\left(r_{ii}\right)$ are determined from

$$\mathrm{cond}\left(t_{ii}\right)=\parallel Z_{i,1:2n}{\parallel }_2,\mathrm{cond}\left(r_{ii}\right)={\parallel Z_{i+n,1:2n}\parallel }_2,$$

$$Z=[Q{M}^{-1},I_{2n}],Q=\left[\begin{array}{cc}{N}_{a1}& N_{a2}\\ N_{b1}& N_{b2}\end{array}\right],$$

$$\begin{array}{ccc}\hfill N_{a1}& =& -\Pi (T^T\otimes I_n){\Omega }^T,N_{a2}=\Pi (I_n\otimes T){\Omega }^T,\hfill \\ \hfill N_{b1}& =& -\Pi (R^T\otimes I_n){\Omega }^T,N_{b2}=\Pi (I_n\otimes R){\Omega }^T.\hfill \end{array}$$

Replacing the expressions for the perturbations $\delta t_{ii}$ and $\delta r_{ii}$ in (58), we find that

$$\chi (\tilde{\lambda },\lambda )\le {\chi }^{lin}(\tilde{\lambda },\lambda )=\mathrm{cond}\left({\lambda }_i\right)\sqrt{{\parallel \delta A\parallel }_F^2+{\parallel \delta B\parallel }_F^2}$$

(61)

where the number

$$\mathrm{cond}\left({\lambda }_i\right)={\displaystyle \frac{|r_{ii}|\mathrm{cond}\left(t_{ii}\right)+\left|t_{ii}\right|\mathrm{cond}\left(r_{ii}\right)}{|t_{ii}{|}^2+{\left|r_{ii}\right|}^2}}$$

can be considered as a condition number of the generalized eigenvalue, ${\lambda }_i$.

4.5. Probabilistic Perturbation Bound

Implementing again Theorem 3, the probabilistic perturbation bounds of the perturbation parameter vectors x and y can be found from (49) and (50), respectively, replacing the quantity $\sqrt{{\parallel \delta A\parallel }_F^2+{\parallel \delta B\parallel }_F^2}$ by the ratio $\sqrt{{\parallel \delta A\parallel }_F^2+{\parallel \delta B\parallel }_F^2}/\Xi$, where $\Xi$ is determined according to (3) from the desired probability, ${\mathcal{P}}^{ref}$, and the problem order, n. This means that the probabilistic asymptotic estimates of $\left|\delta U\right|$ and $\left|\delta V\right|$ can be obtained from (54) and (55), replacing the linear estimates $x^{lin}$, $y^{lin}$ by

$$x^{est}=x^{lin}/\Xi ,y^{est}=y^{lin}/\Xi ,$$

respectively. As a result, we obtain the probabilistic bounds on the angles between the perturbed and unperturbed deflating subspaces as

$$\begin{array}{ccc}\hfill {\Phi }_{\max }^{est}(\tilde{\mathcal{X}},\mathcal{X})& =& \arcsin \left(\parallel {L_X}_{k+1:p,1:2kp}{\parallel }_2\sqrt{{\parallel \delta A\parallel }_F^2+{\parallel \delta B\parallel }_F^2}/\Xi \right),\hfill \end{array}$$

(62)

$$\begin{array}{ccc}\hfill {\Theta }_{\max }^{est}(\tilde{\mathcal{Y}},\mathcal{Y})& =& \arcsin \left(\parallel {L_Y}_{k+1:p,1:2kp}{\parallel }_2\sqrt{{\parallel \delta A\parallel }_F^2+{\parallel \delta B\parallel }_F^2}/\Xi \right),\hfill \\ & & k=1,2,\dots ,n-1.\hfill \end{array}$$

(63)

In the same way, we may obtain a probabilistic asymptotic estimate of the eigenvalue perturbations, replacing in (61) the expression $\sqrt{{\parallel \delta A\parallel }_F^2+{\parallel \delta B\parallel }_F^2}$ by

$\sqrt{{\parallel \delta A\parallel }_F^2+{\parallel \delta B\parallel }_F^2}/\Xi$. This yields

$$\chi (\tilde{\lambda },\lambda )\le {\chi }^{est}(\tilde{\lambda },\lambda )=\mathrm{cond}\left({\lambda }_i\right)\sqrt{{\parallel \delta A\parallel }_F^2+{\parallel \delta B\parallel }_F^2}/\Xi .$$

(64)

4.6. Bound Comparison

Example 2. 

Consider a $80\times 80$ matrix pencil $A-\lambda B$, where

$$A=Q_A{J}_A{Q}_A^{-1},B=Q_B{J}_B{Q}_B^{-1},$$

$$\begin{array}{ccc}& J_A=\left[\begin{array}{ccccccc}0.1& 0.1& \tau & \tau & \dots & \tau & \tau \\ -0.1& 0.1& \tau & \tau & \dots & \tau & \tau \\ & & 0.2& 0.2& \dots & \tau & \tau \\ & & -0.2& 0.2& \dots & \tau & \tau \\ & & & & \ddots & ⋮& ⋮\\ & & & & & 5.0& 5.0\\ & & & & & -5.0& 5.0\end{array}\right],& J_B=\left[\begin{array}{ccccccc}1& \rho & \rho & \rho & \dots & \rho & \rho \\ & 1& \rho & \rho & \dots & \rho & \rho \\ & & 1& \rho & \dots & \rho & \rho \\ & & & 1& \dots & \rho & \rho \\ & & & & \ddots & ⋮& ⋮\\ & & & & & 1& \rho \\ & & & & & & 1\end{array}\right],\hfill \\ & \tau =0.05,& \rho =0.1,\hfill \end{array}$$

the matrices $Q_A$ and $Q_B$ are constructed as in Example 1,

$$\begin{array}{ccc}\hfill Q_A& =& H_2{\Sigma }_A{H}_1,Q_B=H_2{\Sigma }_B{H}_1,\hfill \\ \hfill H_1& =& I_n-2u{u}^T/n,H_2=I_n-2v{v}^T/n,\hfill \\ \hfill u& =& {[1,1,1,\dots ,1]}^T,v={[1,-1,1,\dots ,{(-1)}^{n-1}]}^T,\hfill \\ \hfill {\Sigma }_A& =& \mathrm{diag}(1,{\sigma }_A,{\sigma }_A^2,\dots ,{\sigma }_A^{n-1}),\hfill \\ \hfill {\Sigma }_B& =& \mathrm{diag}(1,{\sigma }_B,{\sigma }_B^2,\dots ,{\sigma }_B^{n-1}),\hfill \end{array}$$

$H_1,H_2$ are elementary reflections, ${\sigma }_A$ is taken equal to $1.02$ and ${\sigma }_B$ is taken equal to $1.01$.

In Figure 4, we give the mean value of the ratios $\Delta a_{ij}/\left|\delta a_{ij}\right|$ and $\Delta b_{ij}/\left|\delta b_{ij}\right|$ obtained for two random distributions and different values of ${\mathcal{P}}^{ref}$ between 100% and 0%, where $\Delta A$ is the probabilistic bound of $\left|\delta A\right|$ and $\Delta B$ is the probabilistic bound of $\left|\delta B\right|$. For each ${\mathcal{P}}^{ref}$, the scaling factor, Ξ, is determined from (3). The same ratios are represented numerically in

Table 2. The mean value of the ratios $[\Delta a_{ij}/|\delta a_{ij}\left|\right]$ and $[\Delta b_{ij}/|\delta b_{ij}\left|\right]$, obtained for three values of ${\mathcal{P}}^{ref}$, $n=80$.

${\mathcal{P}}^{\mathit{ref}}$	$\mathbf{\Xi }$	$\mathit{E}\left\{\right[\mathbf{\Delta }{\mathit{a}}_{\mathit{ij}}/|\mathit{\delta }{\mathit{a}}_{\mathit{ij}}\left|\right]\}$	$\mathit{E}\left\{\right[\mathbf{\Delta }{\mathit{b}}_{\mathit{ij}}/|\mathit{\delta }{\mathit{b}}_{\mathit{ij}}\left|\right]\}$
%
$100.0$	$1.0000\times {10}^0$	$5.5561\times {10}^2$	$7.9274\times {10}^2$
$90.0$	$8.0000\times {10}^0$	$6.9451\times {10}^1$	$9.9093\times {10}^1$
$80.0$	$1.6000\times {10}^1$	$3.4725\times {10}^1$	$4.9546\times {10}^1$

for the perturbation with normal distribution and three values of ${\mathcal{P}}^{ref}$. The results show that if, instead of the deterministic estimates of $\delta A$ and $\delta B$, we use the corresponding probabilistic estimates $\Delta A$ and $\Delta B$ with ${\mathcal{P}}^{ref}=90$ %, then the mean values of $[\Delta a_{ij}/|\delta a_{ij}\left|\right]$ and $[\Delta b_{ij}/|\delta b_{ij}\left|\right]$ decrease by a factor of eight. A further reduction in ${\mathcal{P}}^{ref}$ to 80 % leads to the decreasing of the mean values by a factor of 16.

In Figure 5, we show the relative numbers $N\{\Delta a_{ij}\ge |\delta a_{ij}\left|\right\}/\left(n^2\right)$ and $N\{\Delta b_{ij}\ge |\delta b_{ij}\left|\right\}/\left(n^2\right)$ of the entries for which $\Delta a_{ij}\ge \left|\delta a_{ij}\right|$ and $\Delta b_{ij}\ge \left|\delta b_{ij}\right|$, respectively. In both cases, the relative number of entries for which $\Delta a_{ij}\ge \left|\delta a_{ij}\right|$ and $\Delta b_{ij}\ge \left|\delta b_{ij}\right|$ remains 100 %, which shows that the decreasing of the estimates of $\delta A$ and $\delta B$ can be done safely.

In Figure 6, we show the asymptotic chordal metric bound, ${\chi }_i^{lin}$, of the generalized eigenvalue perturbations along with the probabilistic estimates, ${\chi }_i^{est}$, and the actual eigenvalue perturbations $|{\chi }_i|$ for normal distribution of perturbation entries and probabilities ${\mathcal{P}}^{ref}=90\%,80\%$ and $60\%$. The probabilistic bound, ${\chi }_i^{est}$, is much tighter than the linear bound, ${\chi }_i^{lin}$, and the inequality $|{\chi }_i|\le {\chi }_i^{est}$ is satisfied for all eigenvalues. In particular, the size of the estimate, ${\chi }_i^{est}$, is 8 times smaller than the linear estimate, ${\chi }_i^{lin}$, for ${\mathcal{P}}^{ref}=90\%$, 16 times for ${\mathcal{P}}^{ref}=80\%$ and 32 times for ${\mathcal{P}}^{ref}=60\%$.

In Figure 7 and Figure 8, we show the asymptotic bounds, $\delta {\Phi }_k^{lin}$ and $\delta {\Theta }_k^{lin}$, of the maximum angles between the perturbed and unperturbed deflating subspaces of dimension $k=1,2,\dots ,n-1$ along with the probabilistic estimates, $\delta {\Phi }_k^{est}$, $\delta {\Theta }_k^{est}$, and the actual values of these angles for the same probabilities ${\mathcal{P}}^{ref}=90\%,80\%$ and $60\%$. The probability estimates satisfy $|\delta {\Phi }_k|<\delta {\Phi }_k^{est}$, $|\delta {\Theta }_k|<\delta {\Theta }_k^{est}$ for all $1\le k<100$. For comparison, we give also the global bounds on the maximum angles between the perturbed and unperturbed invariant subspaces computed by (40) and (43), (44). Since the determination of the corresponding bounds requires us to know the norms of parts of the perturbations E and F, which are unknown, these norms are replaced by the Frobenius norms of the whole corresponding matrices. The global bounds practically coincide with the corresponding asymptotic bounds obtained by the splitting operator method.

5. Perturbation Bounds for Singular Subspaces

5.1. Problem Statement

Let $A\in {\mathbb{R}}^{m\times n}$. The factorization

$$U^{H}AV=\Sigma ,$$

(65)

where $U\in {\mathbb{R}}^{m\times m},V\in {\mathbb{R}}^{n\times n}$ are orthogonal matrices and $\Sigma$ is a diagonal matrix, is called the singular value decomposition of A ([38], Section 2.6). If $m\ge n$, matrix $\Sigma$ has the form

$$\Sigma =\left[\begin{array}{c}{\Sigma }_n\\ 0_{(m-n)\times n)}\end{array}\right],{\Sigma }_n=\mathrm{diag}\left({\sigma }_1,{\sigma }_2,\dots ,{\sigma }_n\right).$$

(66)

where the numbers ${\sigma }_i\ge 0$ are the singular values of A. If $U=[U_1,U_2]$ and $[V_1,V_2]$ are partitioned such that $U_1\in {\mathbb{R}}^{m\times k},V_1\in {\mathbb{R}}^{n\times k},k<n$, then $\mathcal{X}=\mathcal{R}\left(V_1\right)$, $\mathcal{Y}=\mathcal{R}\left(U_1\right)$ form a pair of singular subspaces for A. These subspaces satisfy $A\mathcal{X}\subset \mathcal{Y}$ and $A^T\mathcal{Y}\subset \mathcal{X}$.

If matrix A is subject to a perturbation, $\delta A$, then there exists another pair of orthogonal matrices, $\tilde{U}=[{\tilde{U}}_1,{\tilde{U}}_2]$, $\tilde{V}=[{\tilde{V}}_1,{\tilde{V}}_2]$, and a diagonal matrix, $\tilde{\Sigma }$, such that

$${\tilde{U}}^H(A+\delta A)\tilde{V}=\tilde{\Sigma }$$

(67)

where

$$\begin{array}{ccc}\hfill \tilde{\Sigma }& =& \left[\begin{array}{c}{\tilde{\Sigma }}_n\\ 0_{(m-n)\times n)}\end{array}\right],{\tilde{\Sigma }}_n=\mathrm{diag}\left({\tilde{\sigma }}_1,{\tilde{\sigma }}_2,\dots ,{\tilde{\sigma }}_n\right),\hfill \\ \hfill {\tilde{\Sigma }}_n& =& {\Sigma }_n+\delta {\Sigma }_n,\delta {\Sigma }_n=\mathrm{diag}\left(\delta {\sigma }_1,\delta {\sigma }_2,\dots ,\delta {\sigma }_n\right).\hfill \end{array}$$

The aim of the perturbation analysis of singular subspaces consists in bounding the angles between the perturbed, $\tilde{\mathcal{X}}=\mathcal{R}\left({\tilde{V}}_1\right)$, and unperturbed, $\mathcal{X}=\mathcal{R}\left(V_1\right)$, right singular subspaces, and the angles between the perturbed, $\tilde{\mathcal{Y}}=\mathcal{R}\left({\tilde{U}}_1\right)$, and unperturbed, $\mathcal{Y}=\mathcal{R}\left(U_1\right)$, left singular subspaces.

5.2. Global Bound

Consider first the global perturbation bound on the singular subspaces derived by Stewart [16].

Theorem 10. 

Let matrix A be decomposed, as in (65), and let $\mathcal{X}$ and $\mathcal{Y}$ form a pair of singular subspaces for A. Let $\delta A\in {\mathbb{R}}^{m\times n}$ be given and partition $U^T\Delta AV$ conformingly with U and V as

$$E=U^T\delta AV=\left[\begin{array}{cc}{E}_{11}& E_{12}\\ E_{21}& E_{22}\end{array}\right],E_{11}\in {\mathbb{C}}^{k\times k}.$$

Let

$$\eta =\gamma =\parallel (E_{21},E_{12}^T){\parallel }_F$$

and let

$$\delta ={\min }_{\begin{array}{c}1\le j<k\\ k+1\le q\le m\end{array}}|{\sigma }_j-{\sigma }_q|-\parallel E_{11}{\parallel }_2-{\parallel E_{22}\parallel }_2,$$

where, if $m\ge n$, Σ is understood to have $m-n$ zero singular values. Set

$$\kappa ={\displaystyle \frac{2k_2}{1-2k_2+\sqrt{1-4k_2}}},k_2={\displaystyle \frac{\gamma \eta }{\delta }}.$$

If

$${\displaystyle \frac{\gamma }{\delta }}<{\displaystyle \frac{1}{2}},$$

then there are matrices $P\in {\mathbb{R}}^{(m-k)\times k}$ and $Q\in {\mathbb{R}}^{(n-k)\times k}$ satisfying

$${\parallel (P,Q)\parallel }_F\le (1+\kappa ){\displaystyle \frac{\gamma }{\delta }}<2{\displaystyle \frac{\gamma }{\delta }}$$

(68)

such that $\tilde{\mathcal{X}}=\mathcal{R}(V_1+V_{2}P)$ and $\tilde{\mathcal{Y}}=\mathcal{R}(U_1+U_{2}Q)$ form a pair of singular subspaces of $A+\Delta A$.

Theorem 10 bounds the tangents of the angles between the perturbed, $\tilde{\mathcal{X}}$, and unperturbed, $\mathcal{X}$, right singular subspaces and the angles between the perturbed, $\tilde{\mathcal{Y}}$, and unperturbed, $\mathcal{Y}$, left singular subspaces of A. Thus, the maximum angles between perturbed and unperturbed singular subspaces fulfil

$$\begin{array}{c}\hfill {\Phi }_{\max }(\tilde{\mathcal{X}},\mathcal{X})\le \arctan \left((1+\kappa ){\displaystyle \frac{\gamma }{\delta }}\right),\end{array}$$

(69)

$$\begin{array}{c}\hfill {\Theta }_{\max }(\tilde{\mathcal{Y}},\mathcal{Y})\le \arctan \left((1+\kappa ){\displaystyle \frac{\gamma }{\delta }}\right).\end{array}$$

(70)

Note that Theorem 10 produces equal bounds on the maximum angles between the perturbed and unperturbed right and left singular subspaces.

5.3. Perturbation Expansion Bound

Global perturbation bounds for singular subspaces that produce individual perturbation bounds for each subspace in a pair of singular subspaces are presented in [18].

Theorem 11. 

Let $A\in {\mathbb{R}}^{m\times n}(m\ge n)$ and let $U=[U_1,U_2]\in {\mathbb{R}}^{m\times m},V=[V_1,V_2]\in {\mathbb{R}}^{n\times n}$ be orthogonal matrices with $U_1\in {\mathbb{R}}^{m\times k},V_1\in {\mathbb{R}}^{n\times k}$ such that

$$U^{H}AV=\left[\begin{array}{cc}{\Sigma }_1& 0\\ 0& {\Sigma }_2\end{array}\right],$$

(71)

$${\Sigma }_1=\mathrm{diag}({\sigma }_1,\dots ,{\sigma }_k),{\Sigma }_2=\left[\begin{array}{c}{\Sigma }_{21}\\ 0\end{array}\right],{\Sigma }_{21}=\mathrm{diag}({\sigma }_{k+1},\dots ,{\sigma }_n).$$

Assume that ${\sigma }_j\ge 0$ for each j and the singular values of ${\Sigma }_1$ are different from the singular values of ${\Sigma }_2$. Let $\mathcal{X}=\mathcal{R}\left(V_1\right)$ and $\mathcal{Y}=\mathcal{R}\left(U_1\right)$. For $\delta A\in {\mathbb{R}}^{m\times n}$, let

$$E=U^T\delta AV=\left[\begin{array}{cc}{E}_{11}& E_{12}\\ E_{21}& E_{22}\end{array}\right],E_{11}\in {\mathbb{R}}^{k\times k}.$$

Moreover, let

$$\begin{array}{ccc}\hfill {\kappa }_{\mathcal{X}}& =& \underset{\begin{array}{c}1\le j<k\\ k+1\le q\le n\end{array}}{\max }{\displaystyle \frac{\sqrt{{\sigma }_j^2+{\sigma }_q^2}}{|{\sigma }_j^2-{\sigma }_q^2|}},\hfill \\ \hfill {\kappa }_{\mathcal{Y}}& =& \underset{\begin{array}{c}1\le j<k\\ k+1\le q\le m\end{array}}{\max }{\displaystyle \frac{\sqrt{{\sigma }_j^2+{\sigma }_q^2}}{|{\sigma }_j^2-{\sigma }_q^2|}},\hfill \end{array}$$

where we define ${\sigma }_{n+1}=\dots ={\sigma }_m=0$ if $m>n$ and let

$$\kappa =\sqrt{{\kappa }_{\mathcal{X}}^2+{\kappa }_{\mathcal{Y}}^2},\gamma ={\left|\left|\left[\begin{array}{c}{E}_{12}^T\\ F_{21}\end{array}\right]\right|\right|}_F,\epsilon =\parallel E_{11}{\parallel }_2+{\parallel E_{22}\parallel }_2.$$

(72)

If

$$\kappa (2\gamma +\epsilon )<1,$$

then there exists a pair, $\tilde{\mathcal{X}}=\mathcal{R}\left({\tilde{V}}_1\right)$ and $\tilde{\mathcal{Y}}=\mathcal{R}\left({\tilde{U}}_1\right)$, of k-dimensional singular subspaces of $A+\delta A$, such that

$$\begin{array}{c}\hfill {\Phi }_{\max }(\tilde{\mathcal{X}},\mathcal{X})\le \arctan \left({\displaystyle \frac{2{\kappa }_{\mathcal{X}}\gamma }{1-\kappa \epsilon +\sqrt{{(1-\kappa \epsilon )}^2-4{\kappa }^2{\gamma }^2}}}\right),\end{array}$$

(73)

$$\begin{array}{c}\hfill {\Theta }_{\max }(\tilde{\mathcal{Y}},\mathcal{Y})\le \arctan \left({\displaystyle \frac{2{\kappa }_{\mathcal{Y}}\gamma }{1-\kappa \epsilon +\sqrt{{(1-\kappa \epsilon )}^2-4{\kappa }^2{\gamma }^2}}}\right).\end{array}$$

(74)

5.4. Bound by the Splitting Operator Method

Similarly to the perturbation analysis of the generalized Schur decomposition, performed by using the splitting operator method, it is appropriate first to find bounds on the entries of the matrices $\delta W_U:=U^T\delta U_n$ and $\delta W_V:=V^T\delta V$, where $U_n$ is a matrix that consists of the first n columns of U. The matrices $\delta W_U$ and $\delta W_V$ are related to the corresponding perturbations $\delta U_n=U\delta W_U$ and $\delta V=V\delta W_V$ by orthogonal transformations.

Let us define the vectors of the subdiagonal entries of the matrices $\delta W_U$ and $\delta W_V$,

$$x=\mathrm{vec}\left(\mathrm{Low}\left(\delta W_U\right)\right),y=\mathrm{vec}\left(\mathrm{Low}\left(\delta W_V\right)\right).$$

We have that

$$\begin{array}{ccc}\hfill x& =& {[\underset{m-1}{\underbrace{u_2^T\delta u_1,u_3^T\delta u_1,\dots ,u_m^T\delta u_1}},\underset{m-2}{\underbrace{u_3^T\delta u_2,\dots ,u_m^T\delta u_2}},\dots ,\underset{m-n}{\underbrace{u_{n+1}^T\delta u_n,\dots ,u_m^T\delta u_n}}]}^T,\hfill \\ & :=& {[x_1,x_2,\dots ,x_p]}^T\in {\mathbb{R}}^p,\hfill \\ \hfill y& :=& {[\underset{n-1}{\underbrace{v_2^T\delta v_1,v_3^T\delta v_1,\dots ,v_n^T\delta v_1}},\underset{n-2}{\underbrace{v_3^T\delta v_2,\dots ,v_n^T\delta v_2}},\dots ,\underset{1}{\underbrace{v_n^T\delta v_{n-1}}}]}^T,\hfill \\ & :=& {[y_1,y_2,\dots ,y_q]}^T\in {\mathbb{R}}^q,\hfill \end{array}$$

where

$$p=\sum _{i=1}^n(m-i)=n(n-1)/2+(m-n)n=n(2m-n-1)/2,q=n(n-1)/2.$$

It is fulfilled that

$$\begin{array}{ccc}\hfill x_k& =& u_i^T\delta u_j,k=i+(j-1)m-{\displaystyle \frac{j(j+1)}{2}},\hfill \\ & & 1\le j\le n,j<i\le m,1\le k\le p,\hfill \\ \hfill y_{\ell }& =& v_i^T\delta v_j,\ell =i+(j-1)n-{\displaystyle \frac{j(j+1)}{2}},\hfill \\ & & 1\le j\le n,j<i\le n-1,1\le \ell \le q.\hfill \end{array}$$

Further on the quantities $x_k,k=1,2,\dots ,p$ and $y_{\ell },\ell =1,2,\dots ,q$ will be considered as perturbation parameters since they determine the perturbations $\delta U_1$ and $\delta V$ of the singular vectors.

Let us represent the matrix $\delta E=U^T\delta AV$ as

$$\begin{array}{ccc}\hfill \delta E& =& \left[\begin{array}{c}\delta E_1\\ \delta E_2\end{array}\right],\delta E_1\in {\mathbb{R}}^{n\times n},\delta E_2\in {\mathbb{R}}^{(m-n)\times n}.\hfill \end{array}$$

Define the vectors

$$x_{\left(1\right)}={\Omega }_{1}x,and{x}_{\left(2\right)}={\Omega }_{2}x,$$

where

$$\begin{array}{ccc}\hfill {\Omega }_1& :=& \left[\mathrm{diag}({\omega }_1,{\omega }_2,\dots ,{\omega }_n)\right]\in {\mathbb{R}}^{q\times p},\hfill \\ \hfill {\omega }_k& :=& \left[I_{n-k},0_{(n-k)|\times (m-n)}\right]\in {\mathbb{R}}^{(n-k)\times (m-k)},k=1,2,\dots ,n,\hfill \\ & & {\Omega }_1{\Omega }_1^T=I_q,{\parallel {\Omega }_1\parallel }_2=1,\hfill \\ \hfill {\Omega }_2& :=& \left[\mathrm{diag}({\omega }_1,{\omega }_2,\dots ,{\omega }_n)\right]\in {\mathbb{R}}^{(m-n)n\times p},\hfill \\ \hfill {\omega }_k& :=& \left[0_{(m-n)\times (n-k)},I_{m-n}\right]\in {\mathbb{R}}^{(m-n)\times (m-k)},k=1,2,\dots ,n,\hfill \\ & & {\Omega }_2{\Omega }_2^T=I_{(m-n)n},{\parallel {\Omega }_2\parallel }_2=1.\hfill \end{array}$$

It is possible to show that

$$x={\Omega }_1^T{x}_{\left(1\right)}+{\Omega }_2^T{x}_{\left(2\right)}.$$

Following the analysis performed in [30], it can be shown that the unknown vectors $x_{\left(1\right)}$ and y satisfy the system of linear equations

$$\left[\begin{array}{cc}-S_1& S_2\\ S_2& -S_1\end{array}\right]\left[\begin{array}{c}{x}_{\left(1\right)}\\ y\end{array}\right]=-\left[\begin{array}{c}f\\ g\end{array}\right]+\left[\begin{array}{c}{\Delta }_1\\ {\Delta }_2\end{array}\right].$$

(75)

where

$$\begin{array}{ccc}\hfill S_1& =& \mathrm{diag}(\underset{n-1}{\underbrace{{\sigma }_1,{\sigma }_1,\dots ,{\sigma }_1}},\underset{n-2}{\underbrace{{\sigma }_2,\dots ,{\sigma }_2}},\dots ,\underset{1}{\underbrace{{\sigma }_{n-1}}}),\hfill \end{array}$$

(76)

$$\begin{array}{ccc}\hfill S_2& =& \mathrm{diag}(\underset{n-1}{\underbrace{{\sigma }_2,{\sigma }_3,\dots ,{\sigma }_n}},\underset{n-2}{\underbrace{{\sigma }_3,\dots ,{\sigma }_n}},\dots ,\underset{1}{\underbrace{{\sigma }_n}}),\hfill \\ & & S_i\in {\mathbb{R}}^{q\times q},i=1,2,\hfill \end{array}$$

(77)

$$f=\mathrm{vec}\left(\mathrm{Low}\left(\delta E_1\right)\right)\in {\mathbb{R}}^q,g=\mathrm{vec}\left({\left(\mathrm{Up}\left(\delta E_1\right)\right)}^T\right)\in {\mathbb{R}}^q,$$

and ${\Delta }_1,{\Delta }_2$ contain higher-order terms in $\delta U_n,\delta A$ and $\delta V$.

In this way, the determining of the vectors $x_{\left(1\right)}$ and y reduces to the solution of the system of symmetric coupled equations (75) with diagonal matrices of size $q\times q$. The vector $x_{\left(2\right)}$ can be found from the separate equation

$$x_{\left(2\right)}=\mathrm{vec}\left((\delta E_2+{\Delta }_3){\Sigma }_n^{-1}\right),$$

(78)

where ${\Delta }_3$ contains higher-order terms in $\delta u_j,\delta v_j,\delta A$ and ${\Sigma }_n$, defined in (66).

Neglecting the higher-order terms ${\Delta }_1,{\Delta }_2$ in (75), we obtain

$$\left[\begin{array}{c}{x}_{\left(1\right)}\\ y\end{array}\right]=-\left[\begin{array}{cc}{S}_{xf}& S_{xg}\\ S_{yf}& S_{yg}\end{array}\right]\left[\begin{array}{c}f\\ g\end{array}\right],$$

where, taking into account that $S_1$ and $S_2$ commute, we have that

$$\begin{array}{ccc}\hfill S_{xf}& =& {(S_2^2-S_1^2)}^{-1}{S}_1\in {\mathbb{R}}^{q\times q},S_{yg}=S_{xf},\hfill \\ \hfill S_{xg}& =& {(S_2^2-S_1^2)}^{-1}{S}_2\in {\mathbb{R}}^{q\times q},S_{yf}=S_{xg}.\hfill \end{array}$$

The matrices $S_{xf}$ and $S_{xg}$ are diagonal matrices whose nontrivial entries are determined by the singular values ${\sigma }_1,\dots ,{\sigma }_n$.

Hence, the components of the vectors $x_{\left(1\right)}$ and y satisfy

$$\begin{array}{ccc}\hfill |{x_{\left(1\right)}}_{\ell }|& \le & \parallel {S_{xf}}_{\ell ,1:q},{S_{xg}}_{\ell ,1:q}{\parallel }_2{∥\left[\begin{array}{c}f\\ g\end{array}\right]∥}_2,\hfill \\ \hfill |y_{\ell }|& \le & \parallel {S_{yf}}_{\ell ,1:q},{S_{yg}}_{\ell ,1:q}{\parallel }_2{∥\left[\begin{array}{c}f\\ g\end{array}\right]∥}_2,\hfill \\ & & \ell =1,2,\dots ,q.\hfill \end{array}$$

Taking into account the diagonal form of $S_{xf},S_{xg},S_{yf},S_{yg}$, we obtain that

$$\begin{array}{ccc}\hfill |{x_{\left(1\right)}}_{\ell }|& \le & |{S_{xf}}_{\ell ,\ell }\left|\right|f_{\ell }|+|{S_{xg}}_{\ell ,\ell }\left|\right|g_{\ell }|,\hfill \end{array}$$

(79)

$$\begin{array}{ccc}\hfill |y_{\ell }|& \le & |{S_{yf}}_{\ell ,\ell }\left|\right|f_{\ell }|+|{S_{yg}}_{\ell ,\ell }\left|\right|g_{\ell }|,\hfill \\ & & \ell =1,2,\dots ,q.\hfill \end{array}$$

(80)

Since only one element of f and g participates in (79) and (80), these elements can be replaced by ${\parallel \delta A\parallel }_2$, and we find that the linear approximations of the vectors $x_{\left(1\right)}$ and y fulfil

$$\begin{array}{ccc}\hfill |x_{\left(1\right)}|& ⪯& x_{\left(1\right)}^{lin},\left|y\right|⪯y^{lin},\hfill \end{array}$$

(81)

$$\begin{array}{ccc}\hfill {x_{\left(1\right)}^{lin}}_{\ell }& =& \left(\right|{S_{xf}}_{\ell ,\ell }|+|{S_{xg}}_{\ell ,\ell }{\left|\right)\parallel \delta A\parallel }_2,\hfill \end{array}$$

(82)

$$\begin{array}{ccc}\hfill {y^{lin}}_{\ell }& =& \left(\right|{S_{yf}}_{\ell ,\ell }|+|{S_{yg}}_{\ell ,\ell }{\left|\right)\parallel \delta A\parallel }_2,\hfill \\ & & \ell =1,2,\dots ,q.\hfill \end{array}$$

(83)

where $S_{yg}=S_{xf}$, $S_{yf}=S_{xg}$.

An asymptotic estimate of the vector $x_{\left(2\right)}$ is obtained from (78), neglecting the higher-order term ${\Delta }_3$. Since each element of $x_{\left(2\right)}$ depends only on one element of $\delta E_2$, we have that

$$\begin{array}{ccc}\hfill x_{\left(2\right)}^{lin}& =& \mathrm{vec}\left(Z\right),\hfill \\ \hfill Z& =& {\parallel \delta A\parallel }_2\times \left[\begin{array}{cccc}1/{\sigma }_1& 1/{\sigma }_2& \dots & 1/{\sigma }_n\\ 1/{\sigma }_1& 1/{\sigma }_2& \dots & 1/{\sigma }_n\\ ⋮& ⋮& ⋮& ⋮\\ 1/{\sigma }_1& 1/{\sigma }_2& \dots & 1/{\sigma }_n\end{array}\right].\hfill \end{array}$$

(84)

As a result of determining the linear estimates (82)–(84), we obtain an asymptotic approximation of the vector x as

$$\left|x\right|⪯x^{lin},$$

where

$$x^{lin}={\Omega }_1^T{x}_{\left(1\right)}^{lin}+{\Omega }_2^T{x}_{\left(2\right)}^{lin}.$$

(85)

The matrices $|\delta W_U|$ and $|\delta W_V|$ can be estimated as

$$\begin{array}{ccc}\hfill |\delta W_U|& ⪯& \delta W_U^{lin}+{\Delta }^{W_U},\hfill \end{array}$$

(86)

$$\begin{array}{ccc}\hfill |\delta W_V|& ⪯& \delta W_V^{lin}+{\Delta }^{W_V},\hfill \end{array}$$

(87)

where

$$\begin{array}{ccc}\hfill \delta W_U^{lin}& =& \left[\begin{array}{ccccc}0& x_1^{lin}& x_2^{lin}& \dots & x_{n-1}^{lin}\\ x_1^{lin}& 0& x_m^{lin}& \dots & x_{m+n-3}^{lin}\\ x_2^{lin}& x_m^{lin}& 0& \dots & x_{2m+n-6}^{lin}\\ ⋮& ⋮& ⋮& \ddots & ⋮\\ x_{n-1}^{lin}& x_{m+n-3}^{lin}& x_{2m+n-6}^{lin}& \dots & 0\\ x_n^{lin}& x_{m+n-2}^{lin}& x_{2m+n-5}^{lin}& \dots & x_{(n-1)(2m-n)/2+1}^{lin}\\ ⋮& ⋮& ⋮& ⋮& ⋮\\ x_{m-1}^{lin}& x_{2m-3}^{lin}& x_{3m-6}^{lin}& \dots & x_p^{lin}\end{array}\right],\hfill \end{array}$$

(88)

$$\begin{array}{ccc}\hfill \delta W_V^{lin}& =& \left[\begin{array}{ccccc}0& y_1^{lin}& y_2^{lin}& \dots & y_{n-1}^{lin}\\ y_1^{lin}& 0& y_n^{lin}& \dots & y_{2n-3}^{lin}\\ y_2^{lin}& y_n^{lin}& 0& \dots & y_{3n-6}^{lin}\\ ⋮& ⋮& ⋮& \ddots & ⋮\\ y_{n-1}^{lin}& y_{2n-3}^{lin}& y_{3n-6}^{lin}& \dots & 0\end{array}\right]\hfill \end{array}$$

(89)

and the matrices ${\Delta }^{W_U},{\Delta }^{W_V}$ contain higher-order terms in $\delta U,\delta V$. The matrices $\delta W_U^{lin}$ and $\delta W_V^{lin}$ are asymptotic approximations of the matrices $\delta W_U$ and $\delta W_V$, respectively.

We have that

$$\begin{array}{c}\hfill |\delta U_n|⪯|U\left|\right|U^T\delta U_n|⪯\delta U_n^{lin}=\left|U\right|\delta W_U^{lin},\end{array}$$

(90)

$$\begin{array}{c}\hfill \left|\delta V\right|⪯\left|V\right||V^T\delta V|⪯\delta V^{lin}=\left|V\right|\delta W_V^{lin},\end{array}$$

(91)

Assume that the singular value decomposition of A is reordered as

$$U^{T}AV=\left[\begin{array}{cc}{\Sigma }_1& 0\\ 0& {\Sigma }_2\end{array}\right],$$

(92)

where $U=[U_1,U_2]$ and $V=[V_1,V_2]$ are orthogonal matrices with $U_1\in {\mathbb{R}}^{m\times k}$ and $V_1\in {\mathbb{R}}^{n\times k},k<n$, and ${\Sigma }_1\in {\mathbb{R}}^{k\times k}$ contains the desired singular values. The matrices ${\tilde{U}}_1$ and $U_1$ are the orthonormal bases of the perturbed and unperturbed left singular subspace, $\mathcal{X}$, of dimension $k<n$, and ${\tilde{V}}_1$ and $V_1$ are the orthonormal bases of the perturbed and unperturbed right deflation subspace, $\mathcal{Y}$, of the same dimension. We have that

$$\begin{array}{ccc}\hfill \sin \left({\Phi }_{\max }(\tilde{\mathcal{X}},\mathcal{X})\right)& =& \parallel V_2^T\delta V_1{\parallel }_2={\parallel V_{1:n,k+1:n}^T\delta V_{1:n,1:k}\parallel }_2,\hfill \\ \hfill \sin \left({\Theta }_{\max }(\tilde{\mathcal{Y}},\mathcal{Y})\right)& =& \parallel U_2^T\delta U_1{\parallel }_2={\parallel U_{1:m,k+1:m}^T\delta U_{1:m,1:k}\parallel }_2,\hfill \\ & & k=1,\dots ,n-1.\hfill \end{array}$$

Using the asymptotic approximations of the elements of the vectors x and y, we obtain the following result.

Theorem 12. 

Let matrix A be decomposed, as in (92). Given the spectral norm of the perturbation $\delta A$, set the matrices $\delta W_U^{lin}$ and $\delta W_V^{lin}$ in (88), (89) using the linear estimates of the perturbation parameters x and y determined from (81)–(84). Then, the asymptotic bounds of the angles between the perturbed and unperturbed singular subspaces of dimension k are given by

$$\begin{array}{ccc}\hfill {\Phi }_{\max }^{lin}(\tilde{\mathcal{X}},\mathcal{X})& =& \arcsin \left(\parallel {\delta W_V^{lin}}_{k+1:n,1:k}{\parallel }_2\right),\hfill \end{array}$$

(93)

$$\begin{array}{ccc}\hfill {\Theta }_{\max }^{lin}(\tilde{\mathcal{Y}},\mathcal{Y})& =& \arcsin \left(\parallel {\delta W_U^{lin}}_{k+1:m,1:k}{\parallel }_2\right),\hfill \\ & & k=1,\dots ,n-1.\hfill \end{array}$$

(94)

The perturbed matrix of the singular values satisfies

$$\delta {\Sigma }_n=\mathrm{diag}\left(E\right)+\mathrm{diag}\left({\Delta }_4\right),$$

where $E=U^T\delta AV$ and ${\Delta }_4$ contains higher-order terms in $\delta U,\delta V$ and $\delta A$. Neglecting the higher-order terms, we obtain for the singular value perturbation the asymptotic bound

$$|\delta {\sigma }_i|=|\delta E_{ii}|,i=1,2,\dots ,n.$$

(95)

Bounding each diagonal element $|\delta E_{ii}|$ by ${\parallel \delta A\parallel }_2$, we find the normwise estimate

$$|\delta {\sigma }_i{|\le \parallel \delta A\parallel }_2,$$

which is in accordance with Weyl’s theorem ([44], Chapter 1). We have in a first-order approximation that

$$|\delta {\sigma }_i|\le \delta {\sigma }_i^{lin},\delta {\sigma }_i^{lin}:={\parallel \delta A\parallel }_2,i=1,2,\dots ,n.$$

(96)

5.5. Probabilistic Perturbation Bound

Implementing a derivation similar to the one used in the proof of Theorem 3, the probabilistic estimates $x^{est}$ and $y^{est}$ of the parameter vectors x and y can be obtained from the deterministic estimates $x^{lin}$ and $y^{lin}$. For this aim, the value of ${\parallel \delta A\parallel }_2$ in the expressions (82) and (83) for $x^{lin}$ and $y^{lin}$, respectively, is replaced by the value of ${\parallel \delta A\parallel }_2/{\Xi }_2$, where ${\Xi }_2$ is determined from (4) for the specified value of ${\mathcal{P}}^{ref}$. According to (85), the probabilistic perturbation bound of x fulfils

$$x^{est}={\Omega }_1^T{x}_{\left(1\right)}^{lin}/{\Xi }_2+{\Omega }_2^T{x}_{\left(2\right)}^{lin}/{\Xi }_2,$$

where the estimates $x_{\left(1\right)}^{lin}$ and $x_{\left(2\right)}^{lin}$ satisfy (81) and (84), respectively. The bound of y is found from

$$y^{est}=y^{lin}/{\Xi }_2,$$

where $y^{lin}$ satisfies (83).

The bounds on $|\delta U_n|$ and $\left|\delta V\right|$ are determined from (90) and (91), respectively. According to (96), the probability bound on the singular value perturbations is found from

$$\delta {\sigma }_i^{est}={\parallel \delta A\parallel }_2/{\Xi }_2,i=1,2,\dots ,n.$$

(97)

5.6. Bound Comparison

Example 3. 

Consider a $400\times 200$ matrix, taken as

$$A=U_0\left[\begin{array}{c}{\Sigma }_0\\ 0_{(m-n)\times n}\end{array}\right]V_0^T,$$

where

$$\begin{array}{ccc}\hfill {\Sigma }_0& =& \mathrm{diag}({{\sigma }_0}_1,{{\sigma }_0}_2,\dots ,{{\sigma }_0}_n),\hfill \\ & & {{\sigma }_0}_i=(n-i+1)/30000,i=1,2,\dots ,n,\hfill \end{array}$$

the matrices $U_0$ and $V_0$ are constructed as in Example 2,

$$\begin{array}{ccc}\hfill U_0& =& M_2{S}_U{M}_1,\hfill \\ \hfill M_1& =& I_m-2e{e}^T/m,M_2=I_m-2f{f}^T/m,\hfill \\ \hfill e& =& {[1,1,1,\dots ,1]}^T,f={[1,-1,1,\dots ,{(-1)}^{m-1}]}^T,\hfill \\ \hfill S_U& =& \mathrm{diag}(1,\sigma ,{\sigma }^2,\dots ,{\sigma }^{m-1}),\hfill \\ & & \\ \hfill V_0& =& N_2{S}_V{N}_1,\hfill \\ \hfill N_1& =& I_n-2g{g}^T/n,N_2=I_n-2h{h}^T/n,\hfill \\ \hfill g& =& {[1,1,1,\dots ,1]}^T,h={[1,-1,1,\dots ,{(-1)}^{n-1}]}^T,\hfill \\ \hfill S_V& =& \mathrm{diag}(1,\tau ,{\tau }^2,\dots ,{\tau }^{n-1}),\hfill \end{array}$$

and the matrices $M_1,M_2,N_1,N_2$ are elementary reflections. The condition numbers of $U_0$ and $V_0$ with respect to the inversion are controlled by the variables σ and τ and are equal to ${\sigma }^{m-1}$ and ${\tau }^{n-1}$, respectively. In the given case, $\sigma =1.015$, $\tau =1.03$ and $\mathrm{cond}\left(U_0\right)=380.1464$, $\mathrm{cond}\left(V_0\right)=358.5979$.

The perturbation of A is taken as $\delta A={10}^{-9}\times A_0$, where $A_0$ is a matrix with random entries with normal distribution generated by the MATLAB®function randn. This perturbation satisfies ${\parallel \delta A\parallel }_2=3.3800\times {10}^{-8}$. The linear estimates $x^{lin}$ and $y^{lin}$, which are of size 19900, are found by using (49) and (50), respectively, computing in advance the diagonal matrices $S_{xf}$ and $S_{xg}$. These matrices satisfy

$$\parallel S_{xf}{\parallel }_2=9.0330\times {10}^3,{\parallel S_{xg}\parallel }_2=8.3205\times {10}^3,$$

which means that the perturbations in A can be increased nearly ${10}^4$ times in x and y.

In Figure 9, we represent the mean value of the matrix $[\Delta a_{ij}/|\delta a_{ij}\left|\right]$ and the scaling factor ${\Xi }_2$ as a function of ${\mathcal{P}}_{ref}$. Since, in the given case we use ${\parallel \delta A\parallel }_2$ instead of ${\parallel \delta A\parallel }_F$, the value of ${\Xi }_2=(1-{\mathcal{P}}_{ref})\sqrt{m}$ for a given probability ${\mathcal{P}}_{ref}$ is relatively small. For instance, if ${\mathcal{P}}_{ref}=50$ %, then we have that ${\Xi }_2=10$ and the mean value of $[\Delta a_{ij}/|\delta a_{ij}\left|\right]$ is equal to 285.23 (

Table 3. The mean value of the ratios $[\Delta a_{ij}/|\delta a_{ij}\left|\right]$ and the relative number of entries for which $N\{\Delta a_{ij}>|\delta a_{ij}\left|\right\}/n^2$, obtained for five values of ${\mathcal{P}}^{ref}$, $n=100$.

${\mathcal{P}}^{\mathit{ref}}$	${\mathbf{\Xi }}_2$	$\mathit{E}\left\{\right[\mathbf{\Delta }{\mathit{a}}_{\mathit{ij}}/|\mathit{\delta }{\mathit{a}}_{\mathit{ij}}\left|\right]\}$	$\mathit{N}\{\mathbf{\Delta }{\mathit{a}}_{\mathit{ij}}>|\mathit{\delta }{\mathit{a}}_{\mathit{ij}}\left|\right\}/{\mathit{n}}^2$
%			%
$100.0$	$1.0000\times {10}^0$	$2.8523\times {10}^3$	$100.0000$
$90.0$	$2.0000\times {10}^0$	$1.4262\times {10}^3$	$100.0000$
$80.0$	$4.0000\times {10}^0$	$7.1309\times {10}^2$	$100.0000$
$50.0$	$1.0000\times {10}^1$	$2.8523\times {10}^2$	$100.0000$
$20.0$	$1.6000\times {10}^1$	$1.7827\times {10}^1$	$100.0000$

).

In Figure 10, we compare the actual perturbations, $\delta {\sigma }_i$, of the singular values with the normwise bound (96) and the probabilistic bound (97) of the singular value perturbations for ${\mathcal{P}}^{ref}=90\%,80\%$ and $50\%$. The probabilistic perturbation bound $\delta {\sigma }_i^{est}={\parallel \delta A\parallel }_2/{\Xi }_2$ is tighter than the normwise bound ${\parallel \delta A\parallel }_2$. Specifically, $\delta {\sigma }_i^{est}$ is 2 times smaller than ${\parallel \delta A\parallel }_2$ for ${\mathcal{P}}^{ref}=0.9$, 4 times for ${\mathcal{P}}^{ref}=0.8$ and 10 times for ${\mathcal{P}}^{ref}=0.5$. The inequality $|\delta {\sigma }_i|\le \delta {\sigma }_i^{est}$ is satisfied for all singular values and probabilities due to the small values of ${\Xi }_2$. Note that tighter probability estimates can be obtained if instead of ${\parallel \delta A\parallel }_2$ we use ${\parallel \delta A\parallel }_F$ and the scaling parameter $\Xi =(1-{\mathcal{P}}^{ref})\sqrt{mn}$.

In Figure 11 and Figure 12, we show the actual values of the angles between the perturbed and unperturbed right and left singular subspaces, respectively, along with the corresponding linear bounds and probability bounds. For comparison, we give also the global bounds (69), (70) and (73), (74). As in the case of determining the deflation subspace global bounds, since the norms of parts of the perturbation matrix E are unknown, these norms are approximated by the 2-norms of the whole corresponding matrices. Clearly, the probabilistic bounds outperformall deterministic bounds. For instance, if ${\mathcal{P}}^{ref}=0.5$, the probabilistic bounds are 10 times smaller than the deterministic asymptotic bound, as predicted by the analysis.

6. Conclusions

The splitting operator approach used in this paper allows us to derive unified asymptotic perturbation bounds of invariant, deflation and singular matrix subspaces that are comparable with the known perturbation bounds. These unified bounds make it possible to find easily probabilistic perturbation estimates of the subspaces that are considerably less conservative than the corresponding deterministic bounds, especially for high-order problems.

The proposed probability estimates have two disadvantages. First, they can be conservative due to the properties of the Markoff inequality so that the actual probability of the results obtained can be much better than those predicted by these estimates. Secondly, in the case of high-order problems, their computation requires much more memory than the known bounds due to the use of the Kronecker products.