3.2.2. Pseudoinverses

Decomposing *W* into its SVD by *W* = *PSZ* gives the preimage problem in the form *PSZv* = *b* where *P* ∈ R*l*×*<sup>l</sup>* and *Z* ∈ R*L*×*<sup>L</sup>* are orthogonal and *S* ∈ R*l*×*<sup>L</sup>* is diagonal (i.e., ∀*i* = *j*: *sij* = 0) with singular values of *W*, *σt* = *stt*, *t* ∈ *Y*, on its diagonal. The equation at hand is thus

$$\forall t \in \mathcal{Y}: \sum\_{q \in \mathcal{X}} \sum\_{r \in \mathcal{Y}} \sigma\_r p\_{tr} z^\top\_{tq} v\_q = b\_t,$$

where *<sup>z</sup>tq* = *zqt* is the element of the matrix *Z* corresponding to the index *t* in its *q*-th column.

**Theorem 1** (Explicit Solution)**.** *Let W* ∈ R*l*×*<sup>L</sup> be the closeness matrix given by* (8) *and* (9)*, its SVD W* = *PSZ be given by matrices P* ∈ R*l*×*l, S* ∈ R*l*×*<sup>L</sup> and Z* ∈ R*L*×*L, s.t. rows of P, columns of P and rows of S are indexed by Y, Y*<sup>+</sup> = {*t* ∈ *Y* | *σt* > <sup>0</sup>}*. Let, moreover, b* ∈ *B. Then, an explicit solution to the matrix equation Wv* = *b with respect to the given SVD is the vector*

$$v = \sum\_{t \in Y^+} \frac{b^\top p\_t}{\sigma\_t} z\_t \in A\_\prime$$

*where pt is the t-th column of the matrix P and zt is the t-th column of the matrix Z.*

**Proof.** Let 1*t* denote the vector containing all zeros but 1 on the *t*-th coordinate and (*σ*<sup>−</sup><sup>1</sup> *t* )*t* denote the vector containing all zeros but *σ*<sup>−</sup><sup>1</sup> *t* on the *t*-th coordinate. Following the orthogonality of *P*, we have:

$$\begin{split} \mathcal{W}v &= \operatorname{PSZ}^{\top} \sum\_{t \in Y^{+}} \frac{b^{\top} p\_{t}}{\sigma\_{t}} z\_{t} = \sum\_{t \in Y^{+}} \frac{b^{\top} p\_{t}}{\sigma\_{t}} \operatorname{PSZ}^{\top} z\_{t} = \sum\_{t \in Y^{+}} \frac{b^{\top} p\_{t}}{\sigma\_{t}} \operatorname{PS1}\_{t} \\ &= \sum\_{t \in Y^{+}} b^{\top} p\_{t} \operatorname{PS}(\sigma\_{t}^{-1})\_{t} = \sum\_{t \in Y^{+}} b^{\top} p\_{t} p \mathbf{1}\_{t} = \sum\_{t \in Y^{+}} b^{\top} p\_{t} p\_{t} = b \cdot \end{split}$$

**Remark 5.** *Note that the matrices P and Z in Theorem 1 are not determined uniquely, and so there is generally more than one vector v* ∈ *A solving Wv* = *b. Following Lemma 6, the solution to the corresponding preimage problem is given by the set* {*v* ∈ *A* | *v* ≡ *v*} *formed by all vectors equivalent with the explicit solution v.*

Another general description of the solution is expressed using the notion of right inverse matrix, also called (right) pseudoinverse. Any matrix *W*−<sup>1</sup> ∈ R*L*×*<sup>l</sup>* satisfying *WW*−<sup>1</sup> = *I* where *I* ∈ R*l*×*<sup>l</sup>* is the identity matrix, is the *right inverse matrix* of *W* ∈ <sup>R</sup>*l*×*L*; if we demanded that *W*−1*W* were symmetric, *W*−<sup>1</sup> would be unique, otherwise there can be multiple right inverse matrices.

**Lemma 7.** *Let W*−<sup>1</sup> ∈ R*L*×*<sup>l</sup> be any right inverse matrix of the closeness matrix W and b* ∈ *B, then the vector v* = *W*−1*b is a solution to the preimage problem Wv* = *b with respect to the given right inverse matrix W*−1*.*

**Proof.** *Wv* = *WW*−1*b* = *Ib* = *b*.

> The following corollary is a special case of Theorem 1 written in a matrix form.

**Corollary 1.** *Let W* ∈ R*l*×*<sup>L</sup> be the closeness matrix and* rank *W* = *l. If W* = *PSZ is its fixed SVD, then its right inverse is obtained as W*−<sup>1</sup> = *ZS*+*P where S*<sup>+</sup> ∈ R*L*×*<sup>l</sup> is the right inverse of S, i.e., a diagonal matrix with inverted singular values of S on its diagonal. Let b* ∈ *B, then v* = *ZS*+*Pb is a solution to the preimage problem Wv* = *b.*

**Proof.** Following the orthogonality of matrices *P* and *Z*, we have *WW*−<sup>1</sup> = *WZS*+*P* = *PSZZS*+*P* = *PSS*+*P* = *PP* = *I*, hence, *Wv* = *WW*−1*b* = *b*.

**Lemma 8.** *If a matrix W* ∈ R*l*×*L,* 0 < *l* ≤ *L, has linearly independent rows, then W*−<sup>1</sup> = *<sup>W</sup>*(*WW*)−<sup>1</sup> *is its right inverse.*

**Proof.** Following SVD of *W* used in the Corollary 1, full rank of *W* ensures the existence of (*WW*)−1. Then, *WW*−<sup>1</sup> = *WW*(*WW*)−<sup>1</sup> = *I*.

Note that the condition that no singular value of *W* is zero is equivalent to the condition in Lemma 8 that *W* has linearly independent rows.

Example A1 in the Appendix A illustrates the preimage problem and shows one vector that solves it.
