1. Introduction
As it is known ([
1], Ch. 2), ([
2], Ch. 1), ([
3], Ch, 2), each
matrix
can be represented by the
singular value decomposition (SVD) in the factorized form
where the
matrix
U and the
matrix
V are orthogonal and the
matrix
is diagonal:
The numbers
are called
singular values of the matrix
A. The columns of
are called
left singular vectors and the columns of
are the
right singular vectors. The subspaces spanned by sets of left and right singular vectors are called
left and right singular subspaces, respectively.
The singular value decomposition has a long and interesting history described in [
4].
The SVD has many properties, making it an invaluable tool in matrix analysis and matrix computations; see the references cited above. Among them is that the rank
r of
A equals the number of its nonzero singular values and the equality
. The usual assumption is that by an appropriate ordering of the columns
and
, the singular values appear in the order
If some of the singular values are equal to zero, then
A has
linearly independent columns, and the matrix
in (
1) can be represented as
Further, we shall consider the case , i.e., assume that the matrix A is of full column rank.
This paper concerns the case when the matrix
A is subject to an additive perturbation
. In such a case, there exists another pair of orthogonal matrices
and
and a diagonal matrix
, such that
The perturbation analysis of the singular value decomposition consists in determining the changes in the quantities related to the elements of the decomposition due to the perturbation
. This includes determining bounds on the changes of the entries of the orthogonal matrices that reduce the original matrix to diagonal form and bounds on the perturbations of the singular values. Hence, the analysis aims to find bounds on the sizes of
,
, and
as functions of the size of
. It should be emphasized that problems of such a kind arise in most singular value decomposition applications. The most important application of the perturbation analysis is to assess the accuracy of the computed SVD since each algorithm for its determination produces the SVD of
, not of
A, where the size of
depends on the properties of the floating point arithmetic used and the corresponding algorithm. Knowing the size of
, we may use the perturbation bounds to estimate the difference between the actual and computed elements of the decomposition. For a deep and systematic presentation of the matrix perturbation theory and its use in accuracy estimation, the reader is referred to the books of Stewart [
2], Stewart and Sun [
5], and Wilkinson [
6].
According to Weyl’s theorem ([
2], Ch. 1), we have that
which shows that the singular values are perturbed by no more than the 2-norm of the perturbation of
A; i.e., the singular values are always well conditioned. The SVD perturbation analysis is well defined if the matrix
A is of full column rank
n; i.e.,
, since otherwise, the corresponding left singular vector is undetermined.
The size of the perturbations
,
,
, and
is usually measured by using some of the matrix norms, which leads to the so-called
normwise perturbation analysis. In several cases, we are interested in the size of the perturbations of the individual entries of
,
, and
, so that it is necessary to implement a
componentwise perturbation analysis [
7]. This analysis has an advantage when the individual components of
,
, and
differ significantly in magnitude, and the normwise estimates do not produce tight bounds on the perturbations.
The literature on the perturbation theory of the singular value decomposition is significant. The first results on this topic are obtained by Wedin [
8] and Stewart [
9], who developed estimates of the sensitivity of pairs of singular subspaces (see also ([
5], Ch. V)). Other results concerning the sensitivity of the singular vectors and singular subspaces are obtained in [
10,
11], and a survey on the perturbation theory of the SVD until 1990 can be found in [
12]. Using the technique of perturbation expansions for invariant subspaces, Sun [
13] derived perturbation bounds for a pair of singular subspaces that improved the bounds obtained in [
9] and contained as a special case the bounds presented in [
11]. A perturbation theory for the singular values and singular subspaces of diagonally dominant matrices, including graded matrices, is presented in [
14], and optimal perturbation bounds for the case of structured perturbations are derived in [
15]. The seminal paper by Demmel and his co-authors [
16] contains a perturbation theory that provides relative perturbation bounds for the singular values and singular subspaces for different classes of matrices. The problem of backward error analysis of SVD algorithms is discussed in [
17]. High-accuracy algorithms for computing SVD are proposed by Drmač and Veselić in [
18,
19] and implemented in the LAPACK package [
20]. An improvement in the results of [
8] is presented in [
21]. Several results concerning the sensitivity of the SVD are summarized in the survey [
22], and a reach bibliography on the accurate computation of SVD can be found in [
23]. Finally, some recent applications of SVD are described in [
24,
25]. It should be pointed out that the available SVD perturbation theory provides bounds on the sensitivity of the singular vectors and singular subspaces but does not provide perturbation bounds on the individual entries of the matrices
U and
V. Such bounds are important in several applications, and this fact justifies the opinion that apart from the large number of results about the sensitivity of the singular values and singular vectors, a complete componentwise perturbation analysis of the SVD is not available up to the moment.
This paper presents a rigorous perturbation analysis of the orthogonal matrices, singular subspaces, and singular values of a real matrix of full column rank. It is proved that the SVD perturbation problem is well posed only in the case of distinct (simple) singular values. The analysis produces asymptotic (local) componentwise perturbation bounds of the entries of the orthogonal matrices
U and
V and the singular values of the given matrix. Local bounds are derived for the sensitivity of a pair of singular subspaces measured by the angles between the unperturbed and perturbed subspaces. An iterative scheme is described to find global bounds on the respective perturbations, and the results of numerical experiments are presented. The analysis performed in the paper implements the same methodology as the one used previously in [
26] to determine componentwise perturbation bounds of the QR decomposition of a matrix. However, the SVD perturbation analysis has some distinctive features, making it a self-dependent problem.
The paper is organized as follows. In
Section 2, we derive the basic nonlinear algebraic equations used to perform the perturbation analysis of the SVD. After introducing in
Section 3 the perturbation parameters that determine the perturbations of the matrices
U and
V, we derive a symmetric system of coupled equations for these parameters in
Section 4. The solution to the equations for the first-order terms of the perturbation parameters allows us to find asymptotic bounds on the parameters in
Section 5, on the singular values in
Section 6, and on the perturbations of the matrices
U and
V in
Section 7. Using the bounds on the perturbation parameters in
Section 8, we derive bounds on the sensitivity of the singular subspaces. In
Section 9, we develop an iterative scheme for finding global bounds on the perturbations, and in
Section 10 we present the results of some numerical experiments illustrating the proposed analysis. Some conclusions are drawn in
Section 11.
3. Perturbed Orthogonal Matrices and Perturbation Parameters
In the perturbation analysis of the SVD, it is convenient to find first componentwise bounds on the entries of the matrices
and
, which are related to the corresponding perturbations
and
by orthogonal transformations. The implementation of the matrices
and
allows us to find bounds on
and
using orthogonal transformations without increasing the norms of
and
. This helps to determine bounds on
and
, which are as tight as possible.
First, consider the matrix
Further on, we shall use the vector of the subdiagonal entries of the matrix
,
where
As it will become clear later on, together with the orthogonality condition
the vector
x contains the whole information necessary to find the perturbation
. This vector may be expressed as
or, equivalently,
where
is a matrix that “pulls out” the
p elements of
x from the
elements of
(we consider
as an empty matrix). If, for instance,
and
, then
and the matrix
gives the relationship between the subdiagonal entries of the matrix
and the parameter vector
x,
In a similar way, we introduce the vector of the subdiagonal entries of the matrix
(note that
V is a square matrix),
where
. It is fulfilled that
or, equivalently,
where
Further on, the quantities and will be referred to as perturbation parameters since they determine the perturbations and as well as the sensitivity of the singular values and singular subspaces.
First, consider the matrix
Lemma 1. The matrixis the linear (asymptotic) approximation of the matrix ; i.e., for sufficiently small , it is fulfilled that . Proof. Using the vector
x of the perturbation parameters, the matrix
is written in the form
where the diagonal and superdiagonal entries have to be determined.
First, determine the entries of the superdiagonal part of
. Since
, it follows that
and
According to the orthogonality condition (
10), the entries of the strictly upper triangular part of
can be represented as
Now consider how to determine the diagonal entries of the matrix
,
by the elements of
x. Since
, according to (
10) we have that
or
The above expression shows that
is always negative and the entries
depend quadratically on the entries of
. On the other hand, in a linear setting, we have for the superdiagonal elements of
that
From (
12) and (
11), we obtain the quadratic equation
From the two possible solutions to this equation, we take the root
since in this case
with
. The expression (
14) allows us to find an approximation of
from the entries of the matrix
. For small perturbation
(small values of
), we have the estimate (also following from (
11))
So, for small perturbations the quantity depends quadratically on .
Thus, the matrix
can be represented as the sum
where according to (
8), the matrix
has entries depending only on the perturbation parameters
x,
and the matrix
contains second-order terms in
.
Since the matrices
and
contain only second-order terms in the entries of
, taking into account that
and consequently
with
, it follows from (
15) that the matrix
is the linear approximation of
. □
Similarly, for the matrix
like the case of
, it is possible to show that
where
has elements depending only on the perturbation parameters
y,
and the matrix
contains second-order terms in
. The diagonal entries of
are determined as in the case of the matrix
.
4. Equations for the Perturbation Parameters
In this section, we derive exact nonlinear equations for the perturbation parameters
x and
y and equations for their linear approximations. At this stage, we assume that the perturbation
is known, but in
Section 5 we show how to use these equations to find asymptotic approximations of
x and
y knowing only the perturbation norm.
The elements of the perturbation parameter vectors
x and
y can be determined from Equation (
5). For this aim, it is appropriate to transform this equation as follows. Taking into account that
and
, the equation is represented as
where
After transposing (
9), we obtain that
Substituting in (
17) the term
with the expression on the right hand side of (
18), we obtain
where
contains higher-order terms in the entries of
,
, and
.
Replacing the matrices
and
by
, and
, respectively, (
19) is rewritten as
or
Note that the matrices and contain only higher-order terms in the entries of and .
Equation (
21) is the basic equation in the perturbation analysis of the SVD performed in this paper. This equation represents a diagonal system of linear equations with respect to the unknown entries of
and
, allowing us to solve it efficiently even for high-order matrices. Neglecting the higher-order terms in this equation, we obtain in
Section 5 asymptotic bounds for the elements of the SVD. The approximation of these terms makes it possible to determine global perturbation bounds in
Section 9.
The entries of the matrices
and
can be substituted by the corresponding elements
and
of the vectors
x and
y as shown in the previous section. This leads to the representation of Equation (
21) as two matrix equations with respect to two groups of the entries of
x,
and
where
We note that the estimation of requires knowing an estimate of , which is undetermined.
Equations (
22) and (
23) are the basic equations of the SVD perturbation analysis. They can be used to obtain asymptotic as well as global perturbation bounds on the elements of the vectors
x and
y.
Let us introduce the vectors
where
The vector
contains the elements of the unknown vector
x participating in (
22), while
contains the elements of
x participating in (
23). It is easy to prove that
Taking into account that
the strictly lower part of (
22) can be represented column-wise as the following system of linear equations with respect to the unknown vectors
and
y,
where
and
Similarly, the strictly upper part of (
22) is represented row-wise as the system of equations
where
It should be noted that the operators
and
, used in (
26) and (
29), respectively, take only the entries of the strict lower and strict upper part of the corresponding matrix. These entries are then arranged by the operator
column by column, excluding the zeros above or below the diagonal. For instance, if
, the elements of the vectors
f and
g satisfy
In this way, the solution to (
22) reduces to the solution to the two symmetric coupled Equations (
26) and (
29) with diagonal matrices of size
. The Equation (
23) can be solved independently yielding
Note that the elements of depend on the elements of y and vice versa, while depends neither on nor on y.
5. Asymptotic Bounds on the Perturbation Parameters
In this section, we determine linear (asymptotic) bounds on the perturbation parameter vectors x and y using information only for the norm of the perturbation .
Equations (
26) and (
29) can be used to determine asymptotic approximations of the vectors
and
y. The exact solution to these equations satisfies
where taking into account that
and
commute, we have that
Exploiting these expressions, it is possible to show that
Let us consider the conditions for the existence of a solution to the Equations (
26) and (
29).
Theorem 1. The Equations (26) and (29) have a unique solution if and only if the singular values of A are distinct. Proof. The Equations (
26) and (
29) have a unique solution for
and
y if and only if the square symmetric block matrix
is nonsingular, or equivalently, the matrices
,
, and
are nonsingular. The matrices
and
are nonsingular since the matrix
A has nonzero singular values. (For the solution of linear systems of equations with block matrices, see ([
27], Ch. II.)
In turn, a condition for nonsingularity of the matrix
can be found taking into account the structure of the matrices
and
. Clearly, the denominators of the first
diagonal entries of
and
will be different from zero if
is distinct from
. Similarly, the denominators of the next group of
diagonal entries will be different from zero if
is distinct from
, and so on. Finally,
should be different from
. Thus, we conclude that the matrices
and
will exist and the Equations (
26) and (
29) will have a unique solution, if and only if the singular values of
A are distinct. □
We note that Theorem 1 is in accordance with the results obtained in [
14,
28]. Such a result should not come as a surprise since
U is the matrix of the transformation of
to Schur (diagonal) form
and
V is the matrix of the transformation of
to diagonal form
. On the other hand, the perturbation problem for the Schur form is well posed only when the matrix eigenvalues (the diagonal elements of
or
) are distinct.
Neglecting the higher-order terms in (
31) and (
32) and approximating each element of
f and
g by the perturbation norm
, we obtain the following result.
Lemma 2. The linear approximations of the vectors and y satisfywhere and are given by (33) and (34), respectively, and Clearly, if the matrices
have large diagonal entries, then the estimates of the perturbation parameters will be large. Using the expressions for
and
, we may show that
Note that the norms of and can be considered as condition numbers of the vectors and y with respect to the changes in .
An asymptotic estimate of the vector
is obtained, neglecting the higher-order term
and approximating its elements according to (
30).
Lemma 3. The linear approximation of the vector satisfies Equation (
38) shows that a group of
n elements of
will be large if the singular value associated with the corresponding column of
Z is small. The presence of large elements in the vector
x leads to large entries in
and consequently in the estimate of
. This observation aligns with the well known fact that the sensitivity of a singular subspace is inversely proportional to the smallest singular value associated with that subspace.
As a result of determining the linear estimates (
36)–(
38), we obtain an asymptotic approximation of the vector
x as
It should be emphasized that the determination of the linear bounds and requires knowing only the norm of the perturbation .
Thanks to the diagonal structure of the matrices
, and
, the solutions to the Equations (
26)–(
30) are computed efficiently with high accuracy. In fact, the computing of the diagonal elements of the matrices
requires
floating point operations (flops), the determining of
and
according to (
36) and (
37) requires
flops, and the obtaining of
by (
38) needs
flops. Thus, the computing of
and
requires
flops. Also, the solution of the diagonal systems of equations is performed very accurately in a floating point arithmetic.
Example 1. Consider the matrixand assume that it is perturbed bywhere c is a varying parameter. (For convenience, the entries of the matrix are taken as integers). The singular value decompositions of the matrices A and are computed by the function svd
of MATLAB®[29]. The singular values of A are In the given case, the matrices and in Equations (27) and (28) areand the matrices participating in (31) and (32) are The matrixwhich determines the solution for and y, has a condition number with respect to the 2-norm equal to 26.9234179. The exact parameters and their linear approximations computed by using (36) and (38) are shown to eight decimal digits for two perturbation sizes in Table 1. The differences between the values of and are due to the bounding of the elements of the vectors f and g by the value of and taking the terms in (36)–(38) with positive signs. Both approximations are necessary to ensure that for arbitrary small size perturbation. Similarly, in Table 2, we show for the same perturbations of A the exact perturbation parameters and their linear approximations obtained from (37). 7. Asymptotic Bounds on the Perturbations of and V
Having componentwise estimates for the elements of x and y, it is possible to find easily asymptotic bounds on the entries of the matrices and .
Theorem 2. The asymptotic bound on the matrix is given bywhere and the parameters are determined by (39). Proof. The proof follows directly from Lemma 1 and Equation (
6). □
The linear approximation
gives bounds on the perturbations of the individual elements of the orthogonal transformation matrix
U. Note that (
42) is strictly valid only for infinitesimally small perturbation
.
Similarly, we have that the linear approximation of the matrix
is given by
Hence, the matrix entries give asymptotic bounds on the perturbations of the entries of V.
Consider the volume of operations necessary to determine the perturbation bounds
and
provided that the SVD of the matrix
A is already computed. According to (
42) and (
43), the computation of the bounds
and
requires
and
flops, respectively. Adding the
flops necessary to determine
and
, we find that the obtaining of the asymptotic componentwise estimates of
and
requires altogether
flops. Thus, the perturbation analysis requires
operations.
For the matrix
A of Example 1, we obtain that the absolute values of the exact changes of the entries of the matrix
for the perturbation
satisfy
and their asymptotic componentwise estimates found by using (
42) are
Also, for the same
, we have that
and, according to (
43),
It is seen that the magnitude of the entries of and correctly reflect the magnitude of the corresponding entries of and , respectively. Note that the perturbations of the columns of and V tend to increase with increasing of the column number.
8. Sensitivity of Singular Subspaces
The sensitivity of the left
or the right
singular subspace of dimension
r is measured by the canonical angles between the corresponding unperturbed and perturbed subspaces ([
2], Ch. 4), ([
5], Ch. V), [
30].
Let the unperturbed left singular subspace corresponding to the first
r singular values be denoted by
and its perturbed counterpart as
, and let
and
be the orthonormal bases for
and
, respectively. Further on, the sensitivity of the singular subspace
will be characterized by the maximum canonical angle between
and
, defined as
The expression (
44) has the disadvantage that if
is small, then
and the angle
is not well determined. To avoid this difficulty, instead of
, it is preferable to work with
. Let
be the orthogonal complement of
,
. Then, it is possible to show that [
31]
Equation (
46) shows that the sensitivity of the left singular subspace
is related to the values of the perturbation parameters
. In particular, for
, the sensitivity of the first column of
U (the left singular vector, corresponding to
) is determined by
for
one has
and so on (see
Figure 1), where the matrices
for different values of
r are highlighted in boxes.
Similarly, utilizing the matrix
, it is possible to find the sine of the maximum angle between the unperturbed
and the perturbed
right singular subspace
(see
Figure 2). Hence, if the perturbation parameters are determined, it is possible to find sensitivity estimates of the nested singular subspaces
and
Specifically, as
we have that the exact maximum angle between the unperturbed and perturbed left singular subspace of dimension
r is given by
Similarly, the maximum angle between the unperturbed and perturbed right singular subspace of dimension
r is
Thus, we obtain the following result.
Theorem 3. The asymptotic estimate of the angle between the unperturbed and perturbed singular subspace of dimension r satisfieswhereand the parameters are determined from (39). In particular, for the sensitivity of the range
of
A, we obtain that
Similarly, for the angles between the unperturbed and perturbed right singular subspaces of dimension
r, we obtain the linear estimates
where
and
are determined from (37).
We note that using separate x and y parameters decouples the SVD perturbation problem and makes it possible to determine the sensitivity estimates of the left and right singular subspaces independently. This is important when the left or right subspace in a pair of singular subspaces is much more sensitive than its counterpart.
Consider as an example the same perturbed matrix
A as in Example 1. Computing the matrices
and
, it is possible to estimate the sensitivity of all four pairs of singular subspaces of dimensions
, and 4 corresponding to the chosen ordering of the singular values. In
Table 4, we show the actual values of the left and right singular subspaces sensitivity and the computed asymptotic estimates (
49) and (
50) of this sensitivity. To determine the sensitivity of other singular subspaces, reordering the singular values in the initial decomposition so that the desired subspace appears in the set of nested singular subspaces is possible. Note that asymptotic estimates of the canonical angles between an arbitrary unperturbed singular subspace
spanned by specific singular vectors combined in the matrix
and its perturbed counterpart can be determined by computing the singular values of the matrix
, where
is the orthogonal complement of
, consisting of all singular vectors that do not participate in
, and
is the linear estimate obtained by using (
42). Similarly, it is possible to obtain asymptotic perturbation bounds for a desired right singular subspace.
10. Numerical Experiments
In this section, we present the results of some numerical experiments illustrating the properties of the asymptotic and global estimates obtained in the paper. The computations are performed with MATLAB
®Version 9.9 (R2020b) [
29] using IEEE double-precision arithmetic and are verified by using GNU Octave, v. 5.2.0. The M-files implementing the linear and nonlinear SVD perturbation estimates along the example files are available from the authors.
Example 2. This example illustrates the ill-conditioning of the singular subspaces in the case of close singular values of the given matrix.
Consider a matrix A obtained aswhereare orthogonal ad symmetric matrices (elementary reflections [Ch. 4] [2]) andwhere the parameter τ varies between and . In Figure 5, we present the actual values of and the angle along with the asymptotic normwise estimate as functions of the difference . Similarly, in Figure 6, we show the values of and the angle together with the asymptotic estimate as functions of τ. Note that is the larger angle between the subspaces and , while .
The results shown in Figure 5 and Figure 6 confirm that the sensitivity of SVD increases with the decreasing distance between the singular values. According to (33), (34), (36) and (37), with the decreasing of τ the conditioning of the singular subspaces worsens and the norms of the perturbations increase. Hence, a potential source of ill-conditioning of the singular subspaces is the closeness between the singular values of the matrix. Another cause of ill-conditioning may be the presence of small singular values which, according to (38), leads to large elements of the vector and large norms of . The separation of the parameter vector x in two parts and reveals the independent importance of these two causes.
Example 3. In this example, we compare the perturbation bounds of the singular subspaces derived in this paper with some known bounds from the literature.
For the matrix A and the perturbation E given in the previous example, compare first the sensitivity of the singular vector associated with the minimum singular value of A with the respective estimate of presented in [Example 2, p. 267] [5]. In Figure 7, we show the exact value of , the estimate given in [5], and the linear estimate derived in this paper as functions of the parameter τ. (The estimate from [5] is valid for .) Both estimates are very close for all values of τ. Compare now the sensitivity of the left and right singular subspaces for the same matrix, with the bounds derived in [13] and the nonlinear estimates obtained in this paper. Since the estimates in [13] require knowing the norms of parts of the exact perturbation , these norms are substituted by for a fair comparison. In Figure 8, we show the exact value of , the respective estimate from [13], and the estimate , where and are the canonical angles between and . The corresponding comparison of the exact value of with the estimate from [13] and the estimate is given in Figure 9. It is seen from the figures that for values of τ between and the estimates from [13] produce slightly better results, but for values of τ less than , these estimates exceed significantly the nonlinear estimates obtained in this paper. (Note that the estimates derived in [13] are valid for The comparison of the estimates shows that in case of ill conditioned problems (small values of τ) the bound presented in this paper is less conservative than the estimate given in [13]. Example 4. This example illustrates the properties of the linear and nonlinear perturbation bounds obtained in the paper.
Consider a matrix, taken aswhere , the matrices and are constructed as proposed in [34],and the matrices are Householder reflections. The condition numbers of and with respect to the inversion are controlled by the variables σ and τ and are equal to and , respectively. In the given case, , and . The minimum singular value of the matrix A is . The perturbation of A is taken as , where c is a negative number and is a matrix with random entries generated by the MATLAB®functionrand.
In Figure 10, Figure 11, Figure 12, Figure 13, Figure 14 and Figure 15, we show several results related to the perturbations of the singular value decomposition of A for 30 values of c between and . As particular examples, in Figure 10 we display the perturbations of the entry , which is an element of the matrix , and in Figure 11, we display the perturbations of the entry , both as functions of . The componentwise linear bounds correctly reflect the behavior of the actual perturbations and are valid for wide changes in the perturbation size. Note that this holds for all elements of U and V. The global (nonlinear) bounds practically coincide with the linear bounds but do not exist for perturbations whose size is larger than . In Figure 12 and Figure 13, we show the angles between the perturbed and unperturbed leftand rightsingular subspaces of dimension 50. Again, the linear bounds on the angles and are valid for perturbation magnitudes from to and this also holds for singular subspaces of other dimensions. Note that for sufficiently large , the linear estimate also become non-valid. In Figure 14 and Figure 15, we show the perturbations of the singular values and their nonlinear bounds for two different perturbations with and . While in the first case, the nonlinear bound is close to the actual change of the singular values, in the second case, the bound becomes significantly greater than the actual change due to the overestimating of the higher-order term . From the results shown in Figure 10, Figure 11, Figure 12 and Figure 13, it follows that for the matrix in the example under consideration, the overestimates of the perturbations of U and V are approximately 200 times, while the overestimates of and are approximately 100 and 170 times, respectively. In Figure 16 and Figure 17, we show the ratios and , respectively, along with the corresponding mean values (averages) and of the ratio for . The overestimates are mainly due to overestimating the perturbation parameters and in the Equations (36)–(38). In these equations, each entry is replaced by the quantity , leading to an overestimate approximately proportional to m in case of random perturbations. To illustrate this point, in Figure 18 and Figure 19 we show the entries of the matrices and , respectively, along with the corresponding componentwise estimates and for , where the asymptotic estimates are computed for the exact (non approximated) values of the perturbation parameters and , obtained by using the exact . Obviously, the estimates are very close to the exact values, which confirms their good quality. The small differences between the actual quantities and their estimates are caused by taking the absolute values of and in finding the asymptotic bounds and . We note that the overestimates are inevitable if we want to obtain guaranteed asymptotic perturbation bounds of the SVD elements, and they can be significantly reduced only if we use probability perturbation bounds [28]. Example 5. This example visualizes the componentwise estimates of the orthogonal matrices and the singular values for a matrix.
Consider a matrix A with , constructed as in the previous example for and . The perturbation of A is taken as , where is a matrix with random entries.
In Figure 20, Figure 21 and Figure 22, we show the entries of the matrix , the absolute values of the exact changes of all 150 singular values, and the entries of the matrix , respectively, along with the corresponding componentwise estimates , , and . The computation of the linear bounds requires the solution of a system of linear equations for and y and equations for . The nonlinear bounds of and are found only for 7 iterations and are visually indistinguishable from the corresponding linear bounds. The perturbation bounds derived were also tested with higher-order examples, including a example.
The examples presented in this section confirm that the new componentwise perturbation bounds of the singular value decomposition can be used efficiently in the analysis of high-order problems and may compare favorably with the known bounds for some particular cases, for instance in the sensitivity analysis of the singular subspaces. The componentwise bounds of the perturbations in the orthogonal matrices U and V in the last example clearly show that for some specific entries the perturbation magnitudes differ times and the implementation of normwise perturbation bounds for such examples is not informative. It should also be emphasized that the asymptotic estimates, although not global, ensure valid perturbation bounds for sufficiently large perturbations of the given matrix.
11. Conclusions
The paper presents new results related to the perturbation analysis of the singular value decomposition of a real rectangular matrix of full rank. New asymptotic componentwise perturbation bounds are derived for the orthogonal matrices participating in the decomposition, and an alternative method for computing the sensitivity of the singular subspaces is proposed. The possibility to find non-local bounds is illustrated by using a simple iterative procedure.
A potential disadvantage of the proposed perturbation bounds is their conservatism, i.e., the large difference between the bounds and the corresponding perturbations, especially for large values of m and n. This is due to the necessity to replace the entries of the actual perturbation in the derived bounds by its 2-norm, which leads to pessimistic estimates. This conservatism can be removed using probability perturbation bounds, a matter of further research.
The singular value decomposition perturbation analysis presented in this paper has some peculiarities, making it a challenging problem. On the one hand, the SVD analysis is simpler than other problems, like the perturbation analysis of the orthogonal decomposition to triangular form (the QR decomposition). This is due to the diagonal form of the decomposed matrix, which, among the others, allows the equations for the perturbation parameters to be solved easily, avoiding using the Kronecker product. On the other hand, the presence of two matrices in the decomposition requires the introduction of two different parameter vectors, which are mutually dependent due to the relationship between the perturbations of the two orthogonal matrices. This makes it necessary to solve a coupled system of equations about the parameter vectors, complicating the analysis.
The analysis performed in the paper reveals two reasons for the ill-conditioning of the singular subspaces of a matrix. The first cause is the closeness of some singular values of A, which leads to large elements of the vector and consequently to large entries of the perturbation . The second reason is the presence of small singular values of A, which is reflected by large elements of the vector and also leads to large values of the respective entries of . Significantly, these two reasons are independent of each other.