Tensor Eigenvalue and SVD from the Viewpoint of Linear Transformation

Abstract

A linear transformation from vector space to another vector space can be represented as a matrix. This close relationship between the matrix and the linear transformation is helpful for the study of matrices. In this paper, the tensor is regarded as a generalization of the matrix from the viewpoint of the linear transformation instead of the quadratic form in matrix theory; we discuss some operations and present some definitions and theorems related to tensors. For example, we provide the definitions of the triangular form and the eigenvalue of a tensor, and the theorems of the tensor QR decomposition and the tensor singular value decomposition. Furthermore, we explain the significance of our definitions and their differences from existing definitions.

1. Introduction

Unfolding is an important approach in tensor research, and one of the common methods of unfolding is mode-n matricization. Based on this, the mode-n product and n-rank are defined. This helps to extend the concepts of the eigenvalue and the singular value decomposition of matrices to tensors. For example, the eigenvalues of a real supersymmetric tensor are presented in [1], a multilinear singular value decomposition (HOSVD) is presented in [2], and the restricted singular value decomposition (RSVD) for three quaternion tensors is presented in [3]. The singular values of the general tensor are introduced in [4]. In [5,6], the singular values of a tensor are connected with its symmetric embedding. In [7], the authors presented the definition of the singular value of a real rectangular tensor and discussed its properties. For a tensor $T\in {\mathbb{R}}^{I\times J\times I\times J}$, exploiting the matricization of the tensor, a singular value decomposition is presented in [8]. On the other hand, in matrix theory, many studies of matrices are inseparable from linear transformations. Due to the one-to-one correspondence between matrices and linear transformations, we always believe that it is more natural to provide definitions and theorems related to tensors from the viewpoint of linear transformation. This fact also proves that when we think from this perspective, some unresolved tensor problems have already been solved. Similar studies have been considered in [9]. We provide some supplements to the article presented in [9]; for example, we give a more general definition of the multiplication between tensor and tensor and present the form of a triangular tensor by proposing tensor QR decomposition. In addition, in matrix theory, the one-dimensional linear subspace made up of the eigenvectors of one eigenvalue of a linear transformation is stable under the action of this linear transformation. However, this property cannot be generalized for the existing tensor eigenvalue and the corresponding eigenvector. From this point of view, we present a new concept of the eigenvalue and the corresponding eigenvector of an even-order tensor. Then, the concepts of the eigenvalue decomposition, singular value, and singular value decomposition are naturally obtained.

This paper is organized as follows. In Section 2, regarding the tensor as a linear transformation, we present some definitions. For example, we present the identity tensor and a new multiplication for two tensors, and further explain the partitioning of the indices of a tensor. In Section 3, starting from the orthogonalization process of a set of tensors, we present the QR decomposition of a tensor and the form of a triangular tensor. In Section 4, motivated by the invariant eigenspace, we derive the definitions of the tensor eigenvalue and the corresponding tensor eigenvector, and the spectral decomposition theorem of a Hermitian tensor. The singular value decomposition of a tensor is defined in Section 5, and numerical examples are used to illustrate the advantages of our singular value decomposition in tensor compression. The application of the tensor singular value decomposition is described in Section 6.

2. Basic Definitions

For the convenience of writing, we illustrate the symbols to be used, which are similar to those in [9]. The set of integers $\{1,\cdots ,p\}$ will be abbreviated to the symbol $\left[\right[p\left]\right]$ henceforth. Let $(\alpha ,\beta )$ denote a partitioning of the set $\left[\right[p\left]\right]$, where $\alpha =\{{\alpha }_1,\cdots ,{\alpha }_m\}$ and $\beta =\{{\beta }_1,\cdots ,{\beta }_n\}$ are two disjointed nonempty subsets that satisfy $\alpha \cup \beta =\left[\right[p\left]\right]$. Additionally, let $♯\alpha$ denote the number of elements in the set $\alpha$. Under a given partitioning $(\alpha ,\beta )$, an element in the tensor T will be marked as ${\tau }_{\left[\mathcal{I}\right|\mathcal{J}]}^{(\alpha ,\beta )}$, where $\mathcal{I}:=(i_1,\cdots ,i_m)$, $\mathcal{J}:=(j_1,\cdots ,j_n)$ and $i_k\in \left[\left[I_{{\alpha }_k}\right]\right]$, $j_l\in \left[\left[I_{{\beta }_l}\right]\right]$ for all $1\le k\le m$ and $1\le l\le n$.

Given a partitioning $(\alpha ,\beta )$, an order-p tensor $T\in {\mathbb{C}}^{I_1\times \cdots \times I_p}$ can be regarded as a linear map

$${\mathfrak{T}}_{\beta }:{\mathbb{C}}^{I_{{\beta }_1}\times \cdots \times I_{{\beta }_n}}\to {\mathbb{C}}^{I_{{\alpha }_1}\times \cdots \times I_{{\alpha }_m}}.$$

In particular, an identity map is a linear transformation

$${\mathfrak{E}}_{\beta }:{\mathbb{C}}^{I_{{\beta }_1}\times \cdots \times I_{{\beta }_n}}\to {\mathbb{C}}^{I_{{\beta }_1}\times \cdots \times I_{{\beta }_n}}$$

that can map each tensor to itself, in other words, for arbitrary $A\in {\mathbb{C}}^{I_{{\beta }_1}\times \cdots \times I_{{\beta }_n}}$, ${\mathfrak{E}}_{\beta }\left(A\right)=A$. Because the relationship between a tensor and the corresponding linear transformation depends on the partitioning $(\alpha ,\beta )$, in order to avoid confusion, we note the tensor discussed under the partitioning $(\alpha ,\beta )$ as $T_{\beta }^{\alpha }$. When T and $(\alpha ,\beta )$ are known, we can use a command in MATLAB to determine $T_{\beta }^{\alpha }$ directly [10],

T_alpha_beta = permute(T,[alpha,beta]),

where [alpha,beta] indicates the order of the subscripts to be accessed when performing identification. In fact, for a fixed partitioning $(\alpha ,\beta )$, the tensor can be restored as a matrix. Precisely speaking, the element ${\tau }_{i_1,\cdots ,i_p}$ of an order-p tensor $T\in {\mathbb{C}}^{I_1\times I_2\times \cdots \times I_p}$ is saved at location $(i,j)$ of a matrix, where

$$\begin{array}{c}\hfill \begin{array}{ccc}\hfill i& =& (i_{{\alpha }_m}-1)I_{{\alpha }_{m-1}}\cdots I_{{\alpha }_1}+\cdots +(i_{{\alpha }_2}-1)I_{{\alpha }_1}+i_{{\alpha }_1},\hfill \\ \hfill j& =& (i_{{\beta }_n}-1)I_{{\beta }_{n-1}}\cdots I_{{\beta }_1}+\cdots +(i_{{\beta }_2}-1)I_{{\beta }_1}+i_{{\beta }_1}.\hfill \end{array}\end{array}$$

(1)

This storage method can also be regarded as a special case of blocking in [11].

Definition 1 

([11]). The set $M=\{{\mathbf{m}}^{\left(1\right)},\dots ,{\mathbf{m}}^{\left(d\right)}\}$ is a blocking for $T\in {\mathbb{C}}^{I_1\times \cdots \times I_p}$ if ${\mathbf{m}}^{\left(k\right)}=[m_1^{\left(k\right)},\dots ,m_{b_k}^{\left(k\right)}]$ is a vector of positive integers that sums to $I_k$ for $k=1,\cdots ,p$.

When ${\mathbf{m}}^{\left({\alpha }_k\right)}=[1,\cdots ,1]$ is a vector with ones in all elements and of length $I_{{\alpha }_k}$ for all ${\alpha }_k\in \alpha$, and ${\mathbf{m}}^{\left({\beta }_k\right)}=\left[I_{{\beta }_k}\right]$ for all ${\beta }_k\in \beta$, the blocking M is the row blocking of $T_{\beta }^{\alpha }$ under the partitioning $(\alpha ,\beta )$. Similarly, we can obtain the column blockings. Specifically, the entry ${\tau }_{i_1,\cdots ,i_p}$ of an order-p tensor $T\in {\mathbb{C}}^{I_1\times I_2\times \cdots \times I_p}$ can also be saved at the location

$$\begin{array}{c}\hfill (i_p-1)I_{p-1}{I}_{p-2}\cdots I_1+(i_{p-1}-1)I_{p-2}\cdots I_1+\cdots +(i_2-1)I_1+i_1\end{array}$$

(2)

of the linear array. In the subsequent discussion, we shall comply with the rules in (1) or (2) whenever we want to unfold a tensor to a matrix or a vector.

There are many ways to define multiplication between tensors [12]. Based on the relationship between a tensor T and the corresponding map ${\mathfrak{T}}_{\beta }$, in [9], the authors defined the tensor multiplication between the tensor T and any lower-order tensor $A\in {\mathbb{C}}^{I_{{\beta }_1}\times \cdots \times I_{{\beta }_n}}$ as follows

$$\begin{array}{c}\hfill {\mathfrak{T}}_{\beta }\left(A\right):=T{⊛}_{\beta }A=\sum _{j_1=1}^{I_{{\beta }_1}}\cdots \sum _{j_n=1}^{I_{{\beta }_n}}{\tau }_{\left[\mathcal{I}\right|j_1,\cdots ,j_n]}{a}_{j_1,\cdots ,j_n}\in {\mathbb{C}}^{I_{{\alpha }_1}\times \cdots I_{{\alpha }_m}}.\end{array}$$

(3)

The adjoint of ${\mathfrak{T}}_{\beta }$, denoted by ${\mathfrak{T}}_{\beta }^{*}$, is a linear transformation

$$\begin{array}{c}\hfill {\mathfrak{T}}_{\beta }^{*}:{\mathbb{C}}^{I_{{\alpha }_1}\times \cdots \times I_{{\alpha }_m}}\to {\mathbb{C}}^{I_{{\beta }_1}\times \cdots \times I_{{\beta }_n}}\end{array}$$

such that the Lagrange identity [13]

$$\begin{array}{c}\hfill \langle {\mathfrak{T}}_{\beta }\left(A\right),B\rangle =\langle A,{\mathfrak{T}}_{\beta }^{*}\left(B\right)\rangle \end{array}$$

is satisfied for all $A\in {\mathbb{C}}^{I_{{\beta }_1}\times \cdots \times I_{{\beta }_n}}$ and $B\in {\mathbb{C}}^{I_{{\alpha }_1}\times \cdots \times I_{{\alpha }_m}}$. The tensor representation of the linear transformation ${\mathfrak{T}}_{\beta }^{*}$ is ${\left(T_{\beta }^{\alpha }\right)}^{*}$, which is the conjugate transposition of $T_{\beta }^{\alpha }$. See [9] for more details.

Similar to the derivation of the multiplication ${⊛}_{\beta }$ in (3), by means of linear transformation, we define another multiplication between two tensors G and T when there exists $(\gamma ,\eta )$ and $(\alpha ,\beta )$ satisfying $♯\eta =♯\alpha$ and $I_{{\eta }_i}=I_{{\alpha }_i},i=1,\dots ,♯\alpha$. This definition is a generalization of the existing tensor contraction definition.

Definition 2. 

Suppose the linear transformations corresponding to G and T are

$$\begin{array}{c}\hfill {\mathfrak{G}}_{\eta }:{\mathbb{C}}^{I_{{\alpha }_1}\times \cdots \times I_{{\alpha }_m}}\to {\mathbb{C}}^{I_{{\gamma }_1}\times \cdots \times I_{{\gamma }_q}},{\mathfrak{T}}_{\beta }:{\mathbb{C}}^{I_{{\beta }_1}\times \cdots \times I_{{\beta }_n}}\to {\mathbb{C}}^{I_{{\alpha }_1}\times \cdots \times I_{{\alpha }_m}};\end{array}$$

(4)

then, the composition of $\mathfrak{T}$ and $\mathfrak{G}$ is a linear transformation

$$\begin{array}{c}\hfill {\mathfrak{G}}_{\eta }\circ {\mathfrak{T}}_{\beta }:{\mathbb{C}}^{I_{{\beta }_1}\times \cdots \times I_{{\beta }_n}}\to {\mathbb{C}}^{I_{{\gamma }_1}\times \cdots \times I_{{\gamma }_q}}.\end{array}$$

The rule of action is

$$\begin{array}{c}\hfill {({\mathfrak{G}}_{\eta }\circ {\mathfrak{T}}_{\beta }\left(A\right))}_{\mathcal{K}}=({\mathfrak{G}}_{\eta }{\left({\mathfrak{T}}_{\beta }\left(A\right)\right)}_{\mathcal{K}}=\sum _{\mathcal{I}}{g}_{\left[\mathcal{K}\right|\mathcal{I}]}\sum _{\mathcal{J}}{\tau }_{\left[\mathcal{I}\right|\mathcal{J}]}{a}_{\mathcal{J}}=\sum _{\mathcal{J}}\left(\sum _{\mathcal{I}}{g}_{\left[\mathcal{K}\right|\mathcal{I}]}{\tau }_{\left[\mathcal{I}\right|\mathcal{J}]}\right)a_{\mathcal{J}},\end{array}$$

(5)

where $A\in {\mathbb{C}}^{I_{{\beta }_1}\times \cdots \times I_{{\beta }_n}}$, $\mathcal{K}\in \left[\left[I_{{\gamma }_1}\right]\right]\times \cdots \times \left[\left[I_{{\gamma }_q}\right]\right]$. The corresponding tensor form is $H=G_{\eta }^{\gamma }⊛T_{\beta }^{\alpha }$, where $H\in {\mathbb{C}}^{I_{{\gamma }_1}\times \cdots \times I_{{\gamma }_q}\times I_{{\beta }_1}\times \cdots \times I_{{\beta }_n}}$, and

$$h_{\left[\mathcal{K}\right|\mathcal{J}]}=\sum _{\mathcal{I}}{g}_{\left[\mathcal{K}\right|\mathcal{I}]}{\tau }_{\left[\mathcal{I}\right|\mathcal{J}]}.$$

From this, we can present the following theorem:

Theorem 1. 

Let ${\mathfrak{T}}_{\beta }$ and ${\mathfrak{G}}_{\eta }$ be the linear transformations defined in (4) and let $T_{\beta }^{\alpha }$ and $G_{\eta }^{\gamma }$ be their tensor representations. Denote the adjoints of ${\mathfrak{T}}_{\beta }$ and ${\mathfrak{G}}_{\eta }$ by ${\mathfrak{T}}_{\beta }^{*}$ and ${\mathfrak{G}}_{\eta }^{*}$ and the conjugate transposes of $T_{\beta }^{\alpha }$ and $G_{\eta }^{\gamma }$ by ${\left(T_{\beta }^{\alpha }\right)}^{*}$ and ${\left(G_{\eta }^{\gamma }\right)}^{*}$, respectively. Then,

$${({\mathfrak{G}}_{\eta }\circ {\mathfrak{T}}_{\beta })}^{*}={\mathfrak{T}}_{\beta }^{*}\circ {\mathfrak{G}}_{\eta }^{*},{(G_{\eta }^{\gamma }⊛T_{\beta }^{\alpha })}^{*}={\left(T_{\beta }^{\alpha }\right)}^{*}⊛{\left(G_{\eta }^{\gamma }\right)}^{*}.$$

Proof. 

For any $A\in {\mathbb{C}}^{I_{{\gamma }_1}\times \cdots \times I_{{\gamma }_q}}$, $B\in {\mathbb{C}}^{I_{{\beta }_1}\times \cdots \times I_{{\beta }_n}}$, we have

$$\begin{array}{ccc}\hfill \langle {\mathfrak{T}}_{\beta }^{*}\circ {\mathfrak{G}}_{\eta }^{*}\left(A\right),B\rangle & =& \langle {\mathfrak{T}}_{\beta }^{*}\left({\mathfrak{G}}_{\eta }^{*}\left(A\right)\right),B\rangle =\langle {\mathfrak{G}}_{\eta }^{*}\left(A\right),{\mathfrak{T}}_{\beta }\left(B\right)\rangle \hfill \\ & =& \langle A,{\mathfrak{G}}_{\eta }\left({\mathfrak{T}}_{\beta }\left(B\right)\right)\rangle =\langle A,{\mathfrak{G}}_{\eta }\circ {\mathfrak{T}}_{\beta }\left(B\right)\rangle \hfill \\ & =& \langle {({\mathfrak{G}}_{\eta }\circ {\mathfrak{T}}_{\beta })}^{*}\left(A\right),B\rangle .\hfill \end{array}$$

From the arbitrariness of A and B, we can obtain ${({\mathfrak{G}}_{\eta }\circ {\mathfrak{T}}_{\beta })}^{*}={\mathfrak{T}}_{\beta }^{*}\circ {\mathfrak{G}}_{\eta }^{*}$. Because the tensors $G_{\eta }^{\gamma }$ and $T_{\beta }^{\alpha }$ are the tensor representations of the linear transformations ${\mathfrak{G}}_{\eta }$ and ${\mathfrak{T}}_{\beta }$, respectively, the equation ${(G_{\eta }^{\gamma }⊛T_{\beta }^{\alpha })}^{*}={(T_{\beta }^{\alpha })}^{*}⊛{\left(G_{\eta }^{\gamma }\right)}^{*}$ can be established immediately. □

3. QR Decomposition of the Tensor

QR decomposition is a fundamental tool in matrix theory and plays an important role in the design of algorithms. In this section, we aim to present the tensor QR decomposition of a given tensor from the perspective of linear transformation and describe the form of the triangular tensor based on it.

For the given tensor T, the partitioning $(\alpha ,\beta )$ and the multi-index $\mathcal{J}\in \left[\left[I_{{\beta }_1}\right]\right]\times \cdots \times \left[\left[I_{{\beta }_n}\right]\right]$, the process of orthogonalizing the tensors ${\tau }_{[:|\mathcal{J}]}$ is similar to the Gram–Schmidt orthogonalization process of the column vectors of a matrix. In this orthogonalization process, the tensor T and the tensors ${\tau }_{[:|\mathcal{J}]}$ play the roles of the matrix and column vectors, respectively. Suppose the only way to obtain ${\displaystyle \sum _{\mathcal{J}}}{k}_{\mathcal{J}}{\tau }_{[:|\mathcal{J}]}=0$ is for all the scalars $k_{\mathcal{J}}$ to be zero. The orthogonalization process of the tensors ${\tau }_{[:|\mathcal{J}]}$ then works as follows:

$$\begin{array}{c}\hfill \begin{array}{ccc}\hfill p_{[:|1,1,\dots ,1]}& =& {\tau }_{[:|1,1,\dots ,1]},\hfill \\ \hfill q_{[1,1,\dots ,1]}& =& p_{[:|1,1,\dots ,1]}/\parallel p_{[:|1,1,\dots ,1]}\parallel ,\hfill \\ \hfill p_{[:|2,1,\dots ,1]}& =& {\tau }_{[:|2,1,\dots ,1]}-\langle {\tau }_{[:|2,1,\dots ,1]},q_{[1,1,\dots ,1]}\rangle q_{[1,1,\dots ,1]},\hfill \\ \hfill q_{[2,1,\dots ,1]}& =& p_{[:|2,1,\dots ,1]}/\parallel p_{[:|2,1,\dots ,1]}\parallel ,\hfill \\ & \cdots & \end{array}\end{array}$$

where $\parallel U\parallel ={\langle U,U\rangle }^{1/2}$. In general, we have

$$\begin{array}{c}\hfill \begin{array}{ccc}{p}_{[:|l_1,l_2,\dots ,l_n]}& =& {\tau }_{[:|l_1,l_2,\dots ,l_n]}\hfill \\ & & -{\displaystyle \sum _{k_1=1}^{I_{{\beta }_1}}}\cdots {\displaystyle \sum _{k_{n-1}=1}^{I_{{\beta }_{n-1}}}}{\displaystyle \sum _{k_n=1}^{l_n-1}}{r}_{k_1,k_2,\dots ,k_n}{q}_{[k_1,k_2,\dots ,k_n]}\hfill \\ & & -{\displaystyle \sum _{k_1=1}^{I_{{\beta }_1}}}\cdots {\displaystyle \sum _{k_{n-2}=1}^{I_{{\beta }_{n-2}}}}{\displaystyle \sum _{k_{n-1}=1}^{l_{n-1}-1}}{r}_{k_1,\dots ,k_{n-1},l_n}{q}_{[k_1,\dots ,k_{n-1},l_n]}\hfill \\ & & -\cdots -{\displaystyle \sum _{k_1=1}^{l_1-1}}{r}_{k_1,l_2,\dots ,l_n}{q}_{[k_1,l_2,\dots ,l_n]},\hfill \\ q_{[l_1,l_2,\dots ,l_n]}& =& p_{[:|l_1,l_2,\dots ,l_n]}/\parallel p_{[:|l_1,l_2,\dots ,l_n]}\parallel ,\hfill \end{array}\end{array}$$

(6)

where $r_{s_1,s_2,\dots ,s_n}=\langle {\tau }_{[:|l_1,l_2,\dots ,l_n]},q_{[s_1,s_2,\dots ,s_n]}\rangle$. Equation (6) can be rewritten as

$$\begin{array}{c}\hfill \begin{array}{ccc}{\tau }_{[:|l_1,l_2,\dots ,l_n]}& =& {\displaystyle \sum _{k_1=1}^{I_{{\beta }_1}}}\cdots {\displaystyle \sum _{k_{n-1}=1}^{I_{{\beta }_{n-1}}}}{\displaystyle \sum _{k_n=1}^{l_n-1}}{r}_{k_1,k_2,\dots ,k_n}{q}_{[k_1,k_2,\dots ,k_n]}\hfill \\ & & +{\displaystyle \sum _{k_1=1}^{I_{{\beta }_1}}}\cdots {\displaystyle \sum _{k_{n-2}=1}^{I_{{\beta }_{n-2}}}}{\displaystyle \sum _{k_{n-1}=1}^{l_{n-1}-1}}{r}_{k_1,\dots ,k_{n-1},l_n}{q}_{[k_1,\dots ,k_{n-1},l_n]}\hfill \\ & & +\cdots +{\displaystyle \sum _{k_1=1}^{l_1-1}}{r}_{k_1,l_2,\dots ,l_n}{q}_{[k_1,l_2,\dots ,l_n]}+r_{l_1,l_2,\dots ,l_n}{q}_{[l_1,l_2,\dots ,l_n]},\hfill \end{array}\end{array}$$

where $r_{l_1,l_2,\dots ,l_n}=\parallel p_{[:|l_1,l_2,\dots ,l_n]}\parallel$. This expression leads us to obtain a tensor decomposition

$$\begin{array}{c}\hfill T_{\beta }^{\alpha }=Q⊛R,\end{array}$$

(7)

where $Q\in {\mathbb{C}}^{I_{{\alpha }_1}\times \cdots \times I_{{\alpha }_m}\times I_{{\beta }_1}\times \cdots I_{{\beta }_n}}$ satisfies

$$\begin{array}{c}\hfill Q_{[:|k_1,k_2,\dots ,k_n]}=q_{[k_1,k_2,\dots ,k_n]},\end{array}$$

$R\in {\mathbb{C}}^{I_{{\beta }_1}\times \cdots I_{{\beta }_n}\times I_{{\beta }_1}\times \cdots I_{{\beta }_n}}$ satisfies

$$\begin{array}{c}\hfill R_{[t_1,t_2,\dots ,t_n|l_1,l_2,\dots ,l_n]}=\left\{\begin{array}{cc}{r}_{t_1,t_2,\dots ,t_n},\hfill & (t_1,t_2,\dots ,t_n)\le (l_1,l_2,\dots ,l_n),\hfill \\ 0,\hfill & \mathrm{otherwise},\hfill \end{array}\right.\end{array}$$

and $(t_1,t_2,\dots ,t_n)\le (l_1,l_2,\dots ,l_n)$ means

$$\begin{array}{c}\hfill \begin{array}{c}\hfill (t_n-1)I_{{\beta }_{n-1}}\cdots I_{{\beta }_1}+\cdots +(t_2-1)I_{{\beta }_1}+t_1\\ \hfill \le (l_n-1)I_{{\beta }_{n-1}}\cdots I_{{\beta }_1}+\cdots +(l_2-1)I_{{\beta }_1}+l_1.\end{array}\end{array}$$

(8)

Example 1. 

Consider the order-4 tensor $T\in {\mathbb{C}}^{2\times 3\times 2\times 3}$ with the partitioning $\left(\right\{1,2\},\{3,4\left\}\right)$. Applying the orthogonalization process, we can obtain the QR decomposition of T. The tensors $Q,R\in {\mathbb{C}}^{2\times 3\times 2\times 3}$ can be laid out, respectively, as the "matrix" of $2\times 3$ blocks of $2\times 3$ matrices

$$\begin{array}{c}\hfill Q=\left(\begin{array}{ccc}\begin{array}{|ccc|}\hline q_{1111}& q_{1112}& q_{1113}\\ q_{1121}& q_{1122}& q_{1123}\\ \hline\end{array}& \begin{array}{|ccc|}\hline q_{1211}& q_{1212}& q_{1213}\\ q_{1221}& q_{1222}& q_{1223}\\ \hline\end{array}& \begin{array}{|ccc|}\hline q_{1311}& q_{1312}& q_{1313}\\ q_{1321}& q_{1322}& q_{1323}\\ \hline\end{array}\\ \begin{array}{|ccc|}\hline q_{2111}& q_{2112}& q_{2113}\\ q_{2121}& q_{2122}& q_{2123}\\ \hline\end{array}& \begin{array}{|ccc|}\hline q_{2211}& q_{2212}& q_{2213}\\ q_{2221}& q_{2222}& q_{2223}\\ \hline\end{array}& \begin{array}{|ccc|}\hline q_{2311}& q_{2312}& q_{2313}\\ q_{2321}& q_{2322}& q_{2323}\\ \hline\end{array}\end{array}\right)\end{array}$$

and

$$\begin{array}{c}\hfill R=\left(\begin{array}{ccc}\begin{array}{|ccc|}\hline r_{1111}& r_{1112}& r_{1113}\\ r_{1121}& r_{1122}& r_{1123}\\ \hline\end{array}& \begin{array}{|ccc|}\hline 0& r_{1212}& r_{1213}\\ 0& r_{1222}& r_{1223}\\ \hline\end{array}& \begin{array}{|ccc|}\hline 0& 0& r_{1313}\\ 0& 0& r_{1323}\\ \hline\end{array}\\ \begin{array}{|ccc|}\hline 0& r_{2112}& r_{2113}\\ r_{2121}& r_{2122}& r_{2123}\\ \hline\end{array}& \begin{array}{|ccc|}\hline 0& 0& r_{2213}\\ 0& r_{2222}& r_{2223}\\ \hline\end{array}& \begin{array}{|ccc|}\hline 0& 0& 0\\ 0& 0& r_{2323}\\ \hline\end{array}\end{array}\right).\end{array}$$

When the tensor is an order-2 tensor, the QR decomposition of the tensor degenerates into the QR decomposition of the matrix. This is also in line with the fact that tensors are higher-order generalizations of matrices.

Based on the above QR decomposition, we can obtain the following definitions of triangular tensors, which are generalizations of those in matrix theory and different from those in [14].

Definition 3. 

For a given tensor $T\in {\mathbb{C}}^{I_1\times \cdots \times I_p}$ and a partitioning $(\alpha ,\beta )$, we call T upper triangular (or, lower triangular, diagonal) under the partitioning $(\alpha ,\beta )$ if ${\tau }_{[i_{{\alpha }_1},\dots ,i_{{\alpha }_m}|j_{{\beta }_1},\dots ,j_{{\beta }_n}]}=0$ whenever

$$\begin{array}{c}\hfill (i_{{\alpha }_1},\dots ,i_{{\alpha }_m})>(j_{{\beta }_1},\dots ,j_{{\beta }_n}).(\mathrm{or}<,\ne )\end{array}$$

The elements ${\tau }_{[i_{{\alpha }_1},\dots ,i_{{\alpha }_m}|j_{{\beta }_1},\dots ,j_{{\beta }_n}]}$ satisfying

$$\begin{array}{c}\hfill (i_{{\alpha }_1},\dots ,i_{{\alpha }_m})=(j_{{\beta }_1},\dots ,j_{{\beta }_n})\end{array}$$

are the diagonal elements, where $(i_{{\alpha }_1},\dots ,i_{{\alpha }_m})>(\mathrm{or}<,=)(j_{{\beta }_1},\dots ,j_{{\beta }_n})$ means the index $(i_{{\alpha }_1},\dots ,i_{{\alpha }_m},j_{{\beta }_1},\dots ,j_{{\beta }_n})$ satisfies

$$\begin{array}{c}\hfill \begin{array}{cc}& (i_{{\alpha }_m}-1)I_{{\alpha }_{m-1}}\cdots I_{{\alpha }_1}+\cdots +(i_{{\alpha }_2}-1)I_{{\alpha }_1}+i_{{\alpha }_1}>(\mathrm{or}<,=)\hfill \\ & (j_{{\beta }_n}-1)I_{{\beta }_{n-1}}\cdots I_{{\beta }_1}+\cdots +(j_{{\beta }_2}-1)I_{{\beta }_1}+j_{{\beta }_1}.\hfill \end{array}\end{array}$$

Example 2. 

For a tensor $T\in {\mathbb{C}}^{4\times 3\times 10}$ with the partitioning $\left(\right\{1,2\},\{3\left\}\right)$, we mark its diagonal elements and upper triangular elements at the first row of Figure 1 for observation convenience, where the blue dots represent diagonal elements, and the red dots represent the strictly upper triangular elements. Additionally, we mark the upper triangular elements and the strictly upper triangular elements defined by [14] at the second row of Figure 1, where the red dots represent the strictly upper triangular elements, and the green and red dots together form the upper triangle elements. The slices in the figure are lateral slices [12], and the indices of the elements of T are denoted as $(i_1,i_2,i_3)$.

4. Eigenvalue of the Tensor

The eigeninformation of a matrix is a fundamental concept in the field of matrix analysis and plays an important role in many practical applications. Therefore, it is necessary to consider the definition and existence of the eigeninformation of a tensor.

In linear algebra, if there exists a constant $\lambda$ and a nonzero vector $\mathbf{x}$ such that $A\mathbf{x}=\lambda \mathbf{x}$, we call $\lambda$ the eigenvalue of A, and call $\mathbf{x}$ the eigenvector associated with $\lambda$. The equation $A\mathbf{x}=\lambda \mathbf{x}$ is bilinear and implies that the one-dimensional linear subspace expanded by eigenvectors associated with the same eigenvalue remains stable under the corresponding linear transformation. However, H-eigenvalues and Z-eigenvalues do not keep this property. With the benefit of the linear transformation, we present a new definition of the eigeninformation of an even-order tensor.

Firstly, we propose a new concept of square tensors from the viewpoint of an identity map, which is different from the definitions in [9,12].

Definition 4. 

For the tensor $T\in {\mathbb{C}}^{I_1\times \cdots \times I_{2m}}$, if there exists a partitioning $(\alpha ,\beta )$ satisfying $♯\alpha =♯\beta =m$ and $I_{{\alpha }_k}=I_{{\beta }_k},k=1,\cdots ,m$, then we call the tensor T a square tensor under the partitioning $(\alpha ,\beta )$.

Definition 5. 

Suppose $T\in {\mathbb{C}}^{I_1\times \cdots \times I_{2m}}$ is a square tensor under the partitioning $(\alpha ,\beta )$, then we have a transformation ${\mathfrak{T}}_{\beta }:{\mathbb{C}}^{I_{{\beta }_1}\times \cdots \times I_{{\beta }_m}}\to {\mathbb{C}}^{I_{{\alpha }_1}\times \cdots \times I_{{\alpha }_m}}$. We call $\lambda \in \mathbb{C}$ an $(\alpha ,\beta )$-eigenvalue of T, or an $(\alpha ,\beta )$-eigenvalue of ${\mathfrak{T}}_{\beta }$, if when partnered with a nonzero tensor $X\in {\mathbb{C}}^{I_{{\beta }_1}\times \cdots \times I_{{\beta }_m}}$ it satisfies

$$\begin{array}{c}\hfill {\mathfrak{T}}_{\beta }\left(X\right)=T{⊛}_{\beta }X=\lambda X,\end{array}$$

and the tensor X is called an $(\alpha ,\beta )$-eigentensor of T associated with the eigenvalue λ.

This definition is similar to that in [15], which conforms to our original intention of proposing the new definition of the tensor eigenvalue. The linear invariance of matrix eigensubspace is generalized.

Theorem 2. 

Suppose $X_1,X_2,\cdots ,X_r$ are all eigentensors associated with the eigenvalue λ of the tensor T and $k_1,k_2,\dots ,k_r$ are complex numbers. Then, $k_1{X}_1+k_2{X}_2+\cdots +k_r{X}_r$ is also the eigentensor associated with the eigenvalue λ of the tensor T.

Proof. 

Because $X_1,X_2,\cdots ,X_r$ are all eigentensors associated with the eigenvalue $\lambda$, $T{⊛}_{\beta }{X}_i=\lambda X_i,i=1,2,\dots ,r$. Then,

$$\begin{array}{c}\hfill \begin{array}{cc}& T{⊛}_{\beta }(k_1{X}_1+k_2{X}_2+\cdots +k_r{X}_r)\hfill \\ =& k_{1}T{⊛}_{\beta }{X}_1+k_{2}T{⊛}_{\beta }{X}_2+\cdots +k_{r}T{⊛}_{\beta }{X}_r\hfill \\ =& k_1\lambda X_1+k_2\lambda X_2+\cdots +k_r\lambda X_r\hfill \\ =& \lambda (k_1{X}_1+k_2{X}_2+\cdots +k_r{X}_r).\hfill \end{array}\end{array}$$

Thus, the theorem has been proven. □

It is worth noting that for an order-4 tensor, if the elements of T satisfy $T_{ijkl}=T_{ikjl}$, the eigenvalues of T under the partitioning $\left(\right\{1,2\},\{3,4\left\}\right)$ and the partitioning $\left(\right\{1,3\},\{2,4\left\}\right)$ are the same. Extending the conclusion to the general situation, we obtain that if $T\in {\mathbb{C}}^{I_1\times \cdots \times I_{2m}}$ is square under the partitioning $(\alpha ,\beta )$ and $(\xi ,\zeta )$ and satisfies $T_{\beta }^{\alpha }=T_{\zeta }^{\xi }$, then the eigenvalues of T under the partitioning $(\alpha ,\beta )$ and the partitioning $(\xi ,\zeta )$ are the same. Importantly, if T is a supersymmetric tensor, the eigenvalues under arbitrary partitioning $(\alpha ,\beta )$ that satisfy $♯\alpha =♯\beta$ are the same.

For a square tensor $T\in {\mathbb{C}}^{I_1\times \cdots \times I_{2m}}$ under the partitioning $(\alpha ,\beta )$, if its corresponding transformation satisfies ${\mathfrak{T}}_{\beta }={\mathfrak{T}}_{\beta }^{*}$, we call the transformation a self-adjoint operator and call the tensor a Hermitian tensor under the partitioning $(\alpha ,\beta )$ [9]. Based on this concept, and similar to the spectral theorem of a symmetric matrix, we present and prove the following lemma and theorem.

Lemma 1. 

Suppose the tensor $T\in {\mathbb{C}}^{I_1\times \cdots \times I_{2m}}$ is Hermitian under the partitioning $(\alpha ,\beta )$, then the $(\alpha ,\beta )$-eigentensors corresponding to different $(\alpha ,\beta )$-eigenvalues of T are orthogonal.

Proof. 

Let ${\lambda }_1$ and ${\lambda }_2$ be two different $(\alpha ,\beta )$-eigenvalues of T, where $X_1$ and $X_2$ are $(\alpha ,\beta )$-eigentensors corresponding to ${\lambda }_1$ and ${\lambda }_2$, respectively. Then, ${\mathfrak{T}}_{\beta }\left(X_1\right)=T{⊛}_{\beta }{X}_1={\lambda }_1{X}_1$, ${\mathfrak{T}}_{\beta }\left(X_2\right)=T{⊛}_{\beta }{X}_2={\lambda }_2{X}_2$. Therefore,

$$\begin{array}{c}\hfill {\lambda }_1〈X_1,X_2〉=〈{\mathfrak{T}}_{\beta }\left(X_1\right),X_2〉=〈X_1,{\mathfrak{T}}_{\beta }^{*}\left(X_2\right)〉=〈X_1,{\mathfrak{T}}_{\beta }\left(X_2\right)〉={\lambda }_2〈X_1,X_2〉.\end{array}$$

Because ${\lambda }_1\ne {\lambda }_2$, $〈X_1,X_2〉=0$. □

Theorem 3. 

Suppose the tensor $T\in {\mathbb{C}}^{I_1\times \cdots \times I_{2m}}$ is Hermitian under the partitioning $(\alpha ,\beta )$. Then, it has the decomposition

$$\begin{array}{c}\hfill T_{\beta }^{\alpha }=U⊛\Lambda ⊛U^{*},\end{array}$$

where $U\in {\mathbb{C}}^{I_{{\alpha }_1}\times \cdots \times I_{{\alpha }_m}\times I_{{\alpha }_1}\times \cdots \times I_{{\alpha }_m}}$ is a unitary tensor [9], and $\Lambda \in {\mathbb{C}}^{I_{{\alpha }_1}\times \cdots \times I_{{\alpha }_m}\times I_{{\alpha }_1}\times \cdots \times I_{{\alpha }_m}}$ is a diagonal tensor whose diagonal elements are the eigenvalues under the partitioning $(\alpha ,\beta )$ of T.

Proof. 

Let ${\lambda }_{i_1,i_2,\dots ,i_m,i_1,i_2,\dots ,i_m}$ be the $(\alpha ,\beta )$-eigenvalues of T, and let $u_{[:|i_1,i_2,\dots ,i_m]}$ be the corresponding $(\alpha ,\beta )$-eigentensors, where $i_k\in \left[\left[I_{{\alpha }_k}\right]\right],k=1,\dots ,m$. Then,

$$\begin{array}{c}\hfill T_{\beta }^{\alpha }⊛u_{[:|i_1,i_2,\dots ,i_m]}={\lambda }_{i_1,i_2,\dots ,i_m,i_1,i_2,\dots ,i_m}{u}_{[:|i_1,i_2,\dots ,i_m]}.\end{array}$$

Therefore,

$$\begin{array}{c}\hfill \begin{array}{cc}& {\left(U^{*}⊛T_{\beta }^{\alpha }⊛U\right)}_{j_1,j_2,\dots ,j_m,i_1,i_2,\dots ,i_m}\hfill \\ =& u_{[j_1,j_2,\dots ,j_m|:]}^{*}⊛T_{\beta }^{\alpha }⊛u_{[:|i_1,i_2,\dots ,i_m]}\hfill \\ =& {\lambda }_{i_1,i_2,\dots ,i_m,i_1,i_2,\dots ,i_m}〈u_{[:|i_1,i_2,\dots ,i_m]},u_{[:|j_1,j_2,\dots ,j_m]}〉.\hfill \end{array}\end{array}$$

The results from Lemma 1 demonstrate that, when $i_k=j_k,k=1,\dots m$, the above equation is equal to ${\lambda }_{i_1,i_2,\dots ,i_m,i_1,i_2,\dots ,i_m}$; otherwise, it is equal to 0. Furthermore, it can be concluded that $U^{*}⊛T_{\beta }^{\alpha }⊛U=\Lambda$; thus, the theorem has been proven. □

5. Singular Value Decomposition of the Tensor

Based on the eigenvalue of a tensor in Section 4, we want to offer a new definition of tensor singular value decomposition, which is a generalization of the decomposition in [8]. To sufficiently demonstrate the feasibility of this decomposition, we first propose the following theorem and definition.

Theorem 4. 

All the eigenvalues of ${\left(T_{\beta }^{\alpha }\right)}^{*}⊛T_{\beta }^{\alpha }$ or ${\mathfrak{T}}_{\beta }^{*}\circ {\mathfrak{T}}_{\beta }$ are non-negative.

Proof. 

Suppose that $(\lambda ,X)$ is an arbitrary eigenpair of ${\left(T_{\beta }^{\alpha }\right)}^{*}⊛T_{\beta }^{\alpha }$, so it is also an eigenpair of ${\mathfrak{T}}_{\beta }^{*}\circ {\mathfrak{T}}_{\beta }$. Then, we can obtain

$$\begin{array}{c}\hfill \langle {\mathfrak{T}}_{\beta }\left(X\right),{\mathfrak{T}}_{\beta }\left(X\right)\rangle =\langle X,{\mathfrak{T}}_{\beta }^{*}\left({\mathfrak{T}}_{\beta }\left(X\right)\right)\rangle =\langle X,{\mathfrak{T}}_{\beta }^{*}\circ {\mathfrak{T}}_{\beta }\left(X\right)\rangle =\langle X,\lambda X\rangle =\lambda \langle X,X\rangle .\end{array}$$

Because $\langle {\mathfrak{T}}_{\beta }\left(X\right),{\mathfrak{T}}_{\beta }\left(X\right)\rangle \ge 0$ and $\langle X,X\rangle >0$, we have $\lambda \ge 0$. From the arbitrariness of the eigenpair, we can see that all the eigenvalues of ${\left(T_{\beta }^{\alpha }\right)}^{*}⊛T_{\beta }^{\alpha }$ are non-negative. □

Definition 6. 

For a tensor $T\in {\mathbb{C}}^{I_1\times \cdots \times I_p}$ with the partitioning $(\alpha ,\beta )$, we call the non-negative square root of an eigenvalue of ${(T_{\beta }^{\alpha })}^{*}⊛T_{\beta }^{\alpha }\in {\mathbb{C}}^{I_{{\beta }_1}\times \cdots \times I_{{\beta }_n}\times I_{{\beta }_1}\times \cdots \times I_{{\beta }_n}}$ a singular value under the partitioning $(\alpha ,\beta )$ of the tensor T.

Theorem 5. 

Each tensor $T\in {\mathbb{C}}^{I_1\times \cdots \times I_p}$ with the partitioning $(\alpha ,\beta )$ has a singular value decomposition under the partitioning (SVDUP)

$$\begin{array}{c}\hfill T_{\beta }^{\alpha }=U⊛\Sigma ⊛V^{*},\end{array}$$

(9)

in which $U\in {\mathbb{C}}^{I_{{\alpha }_1}\times \cdots \times I_{{\alpha }_m}\times I_{{\alpha }_1}\times \cdots \times I_{{\alpha }_m}}$ and $V\in {\mathbb{C}}^{I_{{\beta }_1}\times \cdots \times I_{{\beta }_n}\times I_{{\beta }_1}\times \cdots \times I_{{\beta }_n}}$ are the unitary tensors, and $\Sigma \in {\mathbb{C}}^{I_{{\alpha }_1}\times \cdots \times I_{{\alpha }_m}\times I_{{\beta }_1}\times \cdots \times I_{{\beta }_n}}$ is a diagonal tensor whose diagonal elements are the singular values under the partitioning $(\alpha ,\beta )$ of T.

Proof. 

Without loss of generality, we only prove the case when ${\displaystyle \prod _{k=1}^{♯\alpha }}{I}_{{\alpha }_k}\ge {\displaystyle \prod _{k=1}^{♯\beta }}{I}_{{\beta }_k}$. ${(T_{\beta }^{\alpha })}^{*}⊛T_{\beta }^{\alpha }\in {\mathbb{C}}^{I_{{\beta }_1}\times \cdots \times I_{{\beta }_n}\times I_{{\beta }_1}\times \cdots \times I_{{\beta }_n}}$ is a Hermitian tensor. From Theorem 3, it can be seen that there is a unitary tensor $V\in {\mathbb{C}}^{I_{{\beta }_1}\times \cdots \times I_{{\beta }_n}\times I_{{\beta }_1}\times \cdots \times I_{{\beta }_n}}$ and diagonal tensor $\Lambda \in {\mathbb{C}}^{I_{{\beta }_1}\times \cdots \times I_{{\beta }_n}\times I_{{\beta }_1}\times \cdots \times I_{{\beta }_n}}$ satisfying

$$\begin{array}{c}\hfill {(T_{\beta }^{\alpha })}^{*}⊛T_{\beta }^{\alpha }=V⊛\Lambda ⊛V^{*}.\end{array}$$

Let ${\Sigma }_1\in {\mathbb{C}}^{I_{{\beta }_1}\times \cdots \times I_{{\beta }_n}\times I_{{\beta }_1}\times \cdots \times I_{{\beta }_n}}$ be a diagonal tensor whose diagonal elements are the square roots of the diagonal elements of $\Lambda$. Let $\Sigma \in {\mathbb{C}}^{I_{{\alpha }_1}\times \cdots \times I_{{\alpha }_m}\times I_{{\beta }_1}\times \cdots \times I_{{\beta }_n}}$ be the tensor by adding or deleting zeros in the tensor ${\Sigma }_1$. $U_1\in {\mathbb{C}}^{I_{{\alpha }_1}\times \cdots \times I_{{\alpha }_m}\times I_{{\beta }_1}\times \cdots \times I_{{\beta }_n}}$ is a tensor satisfying the linear equation $U_1⊛{\Sigma }_1=T_{\beta }^{\alpha }⊛V$, that is, $U_1^{*}⊛T_{\beta }^{\alpha }⊛V={\Sigma }_1$. Finally, we use the appropriate tensors that constitute the unitary tensor $U\in {\mathbb{C}}^{I_{{\alpha }_1}\times \cdots \times I_{{\alpha }_m}\times I_{{\alpha }_1}\times \cdots \times I_{{\alpha }_m}}$ together with $U_1$. □

When $p=2$, T is a second-order tensor, in decomposition (9), U and V are matrices. It is clear that SVDUP is a generalization of matrix singular value decomposition in order. Therefore, we unify the concepts of a tensor singular value and unique left and right singular tensors. For the singular value decomposition of a matrix, the core matrix is a diagonal matrix, but the core tensor of HOSVD [2] is not. Additionally, it is impossible to transform higher-order tensors into a pseudodiagonal tensor by performing orthogonal transformations [2]. From this perspective, our definition would be a better way to generalize the singular value decomposition of matrices. In addition, to demonstrate the advantages of our SVDUP method in data compression, we also provide the following example.

Example 3. 

We randomly generate tensors using the command “create_problem” in the Tensor Toolbox, and compress them using truncated SVDUP and truncated HOSVD, respectively. Then, we compare the accuracy while ensuring that the compression ratio is almost the same in advance. In order to guarantee the generality of the experiment, we test 1000 data points of each size. The metric [16]

$$\begin{array}{c}\hfill \Delta \varphi =1-\frac{\parallel T-\widehat{T}{\parallel }_F}{\parallel T-\tilde{T}{\parallel }_F},\end{array}$$

(10)

where $\widehat{T}$ and $\tilde{T}$ are the approximations of T computed by truncated SVDUP and truncated HOSVD, respectively, is introduced to describe the accuracy of the approximations. $\Delta \varphi >0$ means $\widehat{T}$ can provide a more accurate approximation. The compression ratio is equal to the number of elements contained in the approximate tensor divided by the number of elements in the original tensor. p represents order, and I represents dimension of each order.

Table 1. Comparison between T-SVDUP and T-HOSVD with $I=5$.

Order	Compression Ratio	$\mathbf{\Delta }\mathit{\varphi }>0$	$\mathbf{\Delta }\mathit{\varphi }<0$
$p=4$	$9\%$	6	994
$p=6$	$5\%$	872	128
$p=8$	$1.7\%$	1000	0

and

Table 2. Comparison between T-SVDUP and T-HOSVD with $p=6$.

Dimension	Compression Ratio	$\mathbf{\Delta }\mathit{\varphi }>0$	$\mathbf{\Delta }\mathit{\varphi }<0$
$I=5$	$5\%$	872	128
$I=6$	$9\%$	1000	0
$I=7$	$14\%$	1000	0

illustrate that as the tensor scale increases, our T-SVDUP can obtain higher accuracy approximations than T-HOSVD under a similar compression ratio.

6. Application

In both academic research and practical applications, there are many examples of mapping elements in one high-dimensional space to another. For example, a color image is mapped to a blurred image in 3D space during image processing, one discrete matrix is mapped to another in the process of the discretization of Poisson’s equation, and so on. The existing processing method is to stretch data into vectors, but the structure of the data is destroyed this way. In this section, we take two-dimensional image deblurring as an example to describe the application of SVDUP without the destruction of data structure.

In order to calculate the $(i,j)$ pixels of the blurred pixel matrix Y, we should rotate the PSF (point spread function) array 180 degrees and match it with the pixels in the source pixel matrix X by placing the center of the rotated PSF array on the $(i,j)$ pixels of X [17]. Then, the products of the corresponding components are summed to be $y_{ij}$. For example, let $F={\left(f_{ij}\right)}_{3\times 3}$, $X={\left(x_{ij}\right)}_{3\times 3}$ and $Y={\left(y_{ij}\right)}_{3\times 3}$ be the PSF array, the source pixel matrix and the blurred pixel matrix, respectively. Then,

$$\begin{array}{c}\hfill y_{22}=f_{33}{x}_{11}+f_{32}{x}_{12}+f_{31}{x}_{13}+f_{23}{x}_{21}+f_{22}{x}_{22}+f_{21}{x}_{23}+f_{13}{x}_{31}+f_{12}{x}_{32}+f_{11}{x}_{33}.\end{array}$$

Particularly, if we assume zero boundary conditions, the elements $y_{ij}$ at the border of Y are given by

$$\begin{array}{c}\hfill \begin{array}{c}{y}_{11}=f_{22}{x}_{11}+f_{21}{x}_{12}+f_{12}{x}_{21}+f_{11}{x}_{22},\hfill \\ y_{21}=f_{32}{x}_{11}+f_{31}{x}_{12}+f_{22}{x}_{21}+f_{21}{x}_{22}+f_{12}{x}_{31}+f_{11}{x}_{32},\hfill \\ y_{31}=f_{32}{x}_{21}+f_{31}{x}_{22}+f_{22}{x}_{31}+f_{21}{x}_{32},\hfill \\ y_{12}=f_{23}{x}_{11}+f_{22}{x}_{12}+f_{21}{x}_{13}+f_{13}{x}_{21}+f_{12}{x}_{22}+f_{11}{x}_{23},\hfill \\ \cdots \hfill \\ y_{33}=f_{33}{x}_{22}+f_{32}{x}_{23}+f_{23}{x}_{32}+f_{22}{x}_{33}.\hfill \end{array}\end{array}$$

We assume ${\mathfrak{T}}_{\{3,4\}}:{\mathbb{R}}^{3\times 3}\to {\mathbb{R}}^{3\times 3}$ to be the map from the source pixel matrix to the blurred pixel matrix. Then, the element of its tensor representation T is

$$\begin{array}{c}\hfill T_{ijkl}=\left\{\begin{array}{cccc}{f}_{2-k+i,2-l+j},\hfill & & & -1\le k-i,l-j\le 1,\hfill \\ 0,\hfill & & & \mathrm{otherwise}.\hfill \end{array}\right.\end{array}$$

Then, the transformation can be summarized as $Y=T{⊛}_{\{3,4\}}X$. The image deblurring problem is recovering the source pixel matrix X from the blurred pixel matrix $Y=T{⊛}_{\{3,4\}}X+\Theta$, where $\Theta$ is a kind of noise.

We use the SVDUP $T_{\{3,4\}}^{\{1,2\}}=U⊛\Sigma ⊛V^{*}$ to analyze this problem. The solution can be written as

$$\begin{array}{c}\hfill X=V⊛{\Sigma }^{-1}⊛U^{*}⊛B=\sum _{i=1}^{I_1\times I_2}\frac{\langle U_{·i},B\rangle }{{\sigma }_i}{V}_{·i},\end{array}$$

where $X\in {\mathbb{R}}^{I_1\times I_2}$, $B=Y-\Theta$, $U_{·i}=U_{:,:,l,j},V_{·i}=V_{:,:,l,j}$, satisfying $i=l+I_1(j-1)$, and ${\sigma }_i$ is the singular value satisfying ${\sigma }_1\ge {\sigma }_2\ge \dots \ge {\sigma }_{I_1\times I_2}\ge 0$. In general, for the blurring tensor, ${\sigma }_{I_1\times I_2}\ne 0$, and the number ${\sigma }_1/{\sigma }_{I_1\times I_2}$ is large. The approach that is frequently used to dampen the effects of small singular values is to discard all SVDUP components with small singular values, that is, constructing the approximation

$$\begin{array}{c}\hfill X_k=\sum _{i=1}^k\frac{\langle U_{·i},B\rangle }{{\sigma }_i}{V}_{·i},k<I_1\times I_2,\end{array}$$

where ${\sigma }_j(j\ge k+1)$ is smaller than the given tolerance. This method is called the truncated SVDUP, which is a particular case of the spectral filtering methods

$$\begin{array}{c}\hfill X_{filt}=\sum _{i=1}^{I_1\times I_2}{\varphi }_i\frac{\langle U_{·i},B\rangle }{{\sigma }_i}{V}_{·i},\end{array}$$

where the filter factors ${\varphi }_i$ are chosen such that ${\varphi }_i\approx 1$ for large singular values, and ${\varphi }_i\approx 0$ for small singular values [17]. To some extent, this method can compensate for the errors caused by the truncation of singular values.

Suppose the original image is blurred with a circular averaging blur kernel, and perturbed by Gaussian white noise. Figure 2 shows the image recovered using truncated SVDUP with different truncations. All of our computations were performed in MATLAB 2016a running on a PC Intel(R) Core(TM) i7-7500 of 2.7 GHZ CPU and 8 GB of RAM.

7. Conclusions

In this paper, from the perspective of linear transformation, we present some basic concepts and provide three tensor decompositions: QR decomposition, spectral decomposition and singular value decomposition. These concepts and decompositions are different from the existing results and are more natural generalizations of the corresponding matrix theory. The numerical results also show the effectiveness of our new decomposition method.