3.3.2. Rotation Invariance

From (30), the pivotal matrices **F** and **G** can be found. So, let

$$\mathbf{U}\_{s2} = \mathbf{U}\_{s1}\mathbf{\varproj}\tag{32}$$

in the same way,

$$\mathbf{U}\_{s3} = \mathbf{U}\_{s1}\mathbf{\varPsi}\_y \tag{33}$$

It is discovered that the eigenvalues of **Ψ***x* and **<sup>Ψ</sup>***y* are diagonal elements of **F** and **G**, respectively.

Equations (32) and (33) are equations themselves, and are usually solved by the Least Squares (LS) method [29,36,38,39]; however, LS only takes the error on the left side of the equation into account, it ignores that the coefficient matrix also has an error; therefore, in order to reduce the error caused by solving the equation as much as possible, this paper considers a more accurate method—TLS [13]. Next, the solution of the equation is obtained by taking (32) as an example.

Combining the idea of TLS with the orthogonal property of subspace, we define a new matrix **U***s*<sup>12</sup> = [**<sup>U</sup>***s*<sup>1</sup> **<sup>U</sup>***s*<sup>2</sup>]. In fact, the main aim is to seek a unitary matrix **D** ∈ G*M*×2*<sup>K</sup>* 3 , which is orthogonal to **U***s*12. In other words, the space formed by **D** is orthogonal to the space formed by the column vectors of **U***s*<sup>1</sup> or **U***s*2. So the **D** can be obtained from the ED of **<sup>U</sup>***Hs*12**U***s*<sup>12</sup> [40]

$$\mathbf{U}\_{s12}^H \mathbf{U}\_{s12} = \mathbf{E} \mathbf{A} \mathbf{E}^H,\tag{34}$$

where **Λ** is the diagonal matrix whose diagonal elements are composed by *K* multi-vectors that only have 0-grade-vector (can regard as non-zero real number) and 3*K* multi-vectors that equal to 0. **E** can be written as

$$\mathbf{E} = \begin{bmatrix} \mathbf{E}\_{11} & \mathbf{E}\_{12} \\ \mathbf{E}\_{21} & \mathbf{E}\_{22} \end{bmatrix}. \tag{35}$$

Let **E***N* = **E**12 **E**22 , which is composed by eigenvectors whose eigenvalues are 0 and form the noise subspace. Since **U***s*<sup>12</sup>is signal subspace, we find that **D** = **E***N*, i.e.,

$$\mathbf{U}\_{s12}\mathbf{D} = \begin{bmatrix} \mathbf{U}\_{s1} & \mathbf{U}\_{s2} \end{bmatrix} \begin{bmatrix} \mathbf{E}\_{12} \\ \mathbf{E}\_{22} \end{bmatrix} = 0. \tag{36}$$

Then,

$$
\mathbf{Y}\_x = -\mathbf{E}\_{12}\mathbf{E}\_{22}^+ \tag{37}
$$

The pseudo-inverse of G3 matrix **E**22 can be found in (15).
