Robust Tensor Learning for Multi-View Spectral Clustering

Abstract

Tensor-based multi-view spectral clustering methods are promising in practical clustering applications. However, most of the existing methods adopt the ${\ell }_{2,1}$ norm to depict the sparsity of the error matrix, and they usually ignore the global structure embedded in each single view, compromising the clustering performance. Here, we design a robust tensor learning method for multi-view spectral clustering (RTL-MSC), which employs the weighted tensor nuclear norm to regularize the essential tensor for exploiting the high-order correlations underlying multiple views and adopts the nuclear norm to constrain each frontal slice of the essential tensor as the block diagonal matrix. Simultaneously, a novel column-wise sparse norm, namely, ${\ell }_{2,p}$, is defined in RTL-MSC to measure the error tensor, making it sparser than the one derived by the ${\ell }_{2,1}$ norm. We design an effective optimization algorithm to solve the proposed model. Experiments on three widely used datasets demonstrate the superiority of our method.

1. Introduction

Spectral clustering (SC), which clusters data points using eigenvectors of matrices, is one of the most widely used methods for data clustering. SC and its variants have achieved empirical success in many applications [1,2,3]. Generally, these existing methods investigate how to improve clustering performance with a fixed affinity matrix [4,5] or how to construct a more robust affinity matrix so as to improve clustering performance [6,7]. When accessing multi-view data, many multi-view spectral clustering (MSC) methods have been developed by pursuing a robust affinity matrix [8,9,10] or designing an MSC algorithm with a fixed affinity matrix [11,12,13,14]. Here, we mainly focus on how to learn a more robust affinity matrix from multiple views.

To learn an affinity matrix from multiple views, Nie et al. [15] developed an MSC method with adaptive neighbors (MSCAN) by performing clustering and local structure learning simultaneously. Based on this, a graph learning method is proposed by introducing a rank constraint on the Laplacian matrix in two steps [16]. Although these methods achieve a promising clustering performance, they directly learn the similarity matrix from original multi-view data, which usually contain noise. To address this problem, the subspace-based methods are widely investigated. For example, Xie et al. [17] propose a method by joint latent representation and similarity learning, which learns the common similarity matrix from the adaptive consistent latent representation. By using the Frobenius norm to measure the reconstruction error, the latent representation of each view, and the consistence between the common similarity matrix and the latent representations, a multi-graph fusion method is developed for MSC [18]. Although these methods learn the class indicator matrix directly, they usually perform k-means on this obtained class indicator matrix to obtain the final predicted label. To deal with this issue, some unified clustering methods are designed. For example, Zhong et al. [19] provide an MSC method by simultaneous consensus graph learning and discretization, which learns the similarity matrix and the discrete cluster label matrix in a unified framework. Tang et al. [20] made a design to directly capture the discrete clustering indicator matrix from the unified graph. Meanwhile, a multi-view kernel spectral clustering method [21] is proposed by formulating a weighted kernel canonical correlation analysis in a primal–dual optimization setting typical of least squares support vector machines, where the clustering scores are calculated directly.

Different from these traditional shallow methods, some deep MSC methods are also widely investigated. For example, an MSC network [22] is designed by incorporating the local invariance within every single view and the consistency across different views into a unified framework, where the local invariance is defined by a deep metric learning network. Zhao et al. [23] propose a deep MSC via an ensemble model to fuse the similarity graphs from different views by ensemble clustering. They adopt a graph auto-encoder to learn the common indicator matrix directly. Wang et al. [24] use a set of encoders to obtain the latent representation of each view, and then combine the local structure, global structure, and discriminative constraint into a unified framework to learn a shared affinity matrix. Although these deep MSC methods achieved promising clustering methods by exploring deep features, they are usually limited by ignoring the high-order correlations underlying multiple views.

With the aiming of exploring high-order correlations between multiple views, many tensor-based MSC methods are investigated. These methods can be roughly categorized into two types according to the construction of a tensor. One is to construct rank-1 tensors by calculating the outer products of all view-specific feature maps, which incorporate with mth-order tensors of interconnection weights to perform calculations of all views simultaneously, leading to tensor-based high-order couplings and information fusion [25]. The other way is to construct a third-order tensor by merging a set of the representation matrix from each single view. These works mainly focus on how to approximate the tensor low-rank constraint and how to learn the common affinity matrix. One way to depict the tensor low-rank property is by using the Tucker decomposition [26]. Based on this, Chen et al. [27] recovered the latent representation of each view, and adaptively learned the affinity matrix from these representations. Meanwhile, the tensor singular value decomposition (t-SVD)-based tensor nuclear norm (TNN) is also a widely used approximation of tensor low-rank constraint. Based on this, Wu et al. [28] propose an essential tensor learning (ETL) method for MSC (ETL-MSC). Inspired by the low-rank and sparse decomposition, they recovered a latent tensor from the input tensor that was stacked by the transition probability matrix (TPM) of each view. With the obtained tensor, the affinity matrix is computed by calculating the average of all frontal slices of this tensor. By introducing a probability constraint, Zhang et al. [29] improved the clustering performance of ETL-MSC. Although these t-SVD-TNN-based methods achieved a promising clustering performance, they are still limited by the fact that the TNN is only the sum of singular values, causing a poor rank approximation. To address this problem, the nonconvex Laplace function [30], t-SVD-based weighted TNN (WTNN) [31], tailored tensor low-rank norm [32], and weighted tensor Schatten-p norm [33]-based MSC methods are developed. Unlike these methods that pursue the tensor low-rank approximate, Chen et al. [34] committed to learn a common non-negative affinity matrix directly. Unlike this, Wang et al. [35] aimed to emphasize the types of errors that can vary behavior inconsistency in each view by introducing a group ${\ell }_1$ norm to the error tensor.

Inspired by the fact that t-SVD WTNN shrinks different singular values with different weights, leading to a satisfying clustering performance, we propose a novel robust tensor learning method for MSC (RTL-MSC). The proposed RTL-MSC aims to learn the latent tensor from the input transition probability tensor. Then, the affinity matrix, which would be input to the Markov chain SC algorithm to obtain the final clustering result, is computed by each frontal slice of the derived latent tensor. Unlike the aforementioned tensor-based methods, we impose the low-rank constraint on each frontal slice of the latent tensor to emphasize the low-rank property of each latent TPM. Simultaneously, we adopt the ${\ell }_{2,p}$ norm to regularize the error tensor to depict the noise corresponding to each sample. The contributions of this work can be summarized as follows:

• With a novel integrated strategy, the weighted tensor nuclear norm-based tensor low-rank constraint, the matrix nuclear norm-based low-rank regularization, and the ${\ell }_{2,p}$ regularization are integrated into a unified framework, where the WTNN regularization depicts the information among different samples and different views, and the matrix nuclear norm regularization makes each frontal slice of the learned tensor approximately block-diagonal, capturing the geometry of each single view. Thus, the affinity matrix calculated from the latent tensor depicts the intrinsic clustering structure of the multi-view data.
• A column-wise sparse norm, namely, ${\ell }_{2,p}$ norm, is introduced to enhance the robustness of our model. Compared with the ${\ell }_{2,1}$ norm, the ${\ell }_{2,p}$ norm, which is invariant, continuous, and differentiable, can be better in restricting the sparsity property of noised samples.

2. Background and Motivation

The Markov chain method is one of the classical clustering approaches, which takes a transition probability matrix as the input to perform a Markov chain-based SC algorithm. Based on this, a robust multi-view spectral clustering (RMSC) method [36] is proposed to recover the consistent TPM by low-rank and sparse decomposition. Let ${\left\{{\mathbf{X}}^v\right\}}_{v=1}^m$ be the multi-view data including m different features. ${\mathbf{X}}^v=\{{\mathbf{x}}_1^v,{\mathbf{x}}_2^v,\cdots ,{\mathbf{x}}_n^v\}\in {\Re }^{d^v\times n}$($v=1,2,\cdots ,m$) denotes the features of n samples in the v-th view, and $d^v$ is the feature dimension of the v-th view. RMSC first computes the similarity matrix ${\mathbf{A}}^{\mathbf{v}}\in {\Re }^{\mathbf{n}\times \mathbf{n}}$ by $A_{i,j}^v=exp(-{∥{\mathbf{x}}_i^v-{\mathbf{x}}_j^v∥}_2^2/\phantom{{∥{\mathbf{x}}_i^v-{\mathbf{x}}_j^v∥}_2^2\tau }\tau )$, and then calculates the TPM ${\mathbf{P}}^v$ by ${\mathbf{P}}^v={\left({\mathbf{D}}^v\right)}^{-1}{\mathbf{A}}^v$, where ${\mathbf{D}}^v$ is a diagonal matrix with the i-th element $D_{i,i}^v={\sum }_{j=1}^n{A}_{i,j}^v$. Considering that the features in each view might be corrupted by noise, RMSC separates the transition probability matrices ${\mathbf{P}}^v$ into two parts by th low-rank and sparse decomposition: the common transition probability matrix $\mathbf{P}$, which is shared by all views, and the error matrices ${\mathbf{E}}^v,v=1,\cdots ,m$, which encode the noise of the transition probability matrices. By assuming that the common TPM $\mathbf{P}$ tends to be low rank, while the error matrices are sparse, RMSC is formulated as follows:

$$\begin{array}{c}\underset{\mathbf{P},{\mathbf{E}}^v}{min}{∥\mathbf{P}∥}_{*}+\lambda {\displaystyle \sum _{v=1}^m}{∥{\mathbf{E}}^v∥}_1\\ s.t.{\mathbf{P}}^v=\mathbf{P}+{\mathbf{E}}^v,v=1,2,\cdots ,m\\ \mathbf{P}\ge 0,\mathbf{P}\mathbf{1}=1\hfill \end{array}$$

(1)

where $\lambda$ is a non-negative parameter. RMSC explores the consistent information among multiple views while ignoring the unique information of each single view.

To emphasize this issue, a tensor-based TPM learning method, termed as ETL-MSC, is proposed by extending RMSC to a third-order tensor form. ETL-MSC stacks ${\mathbf{P}}^v$ as the v-th frontal slice of the tensor $\mathcal{P}\in {\Re }^{n\times n\times m}$, and takes $\mathcal{P}$ as the input tensor. By assuming that the essential tensor satisfies tensor low-rank constraint and that the noises in transition probability vectors of noised samples are not sparse while the noisy samples should be sparse, ETL-MSC formulates the following problem:

$$\begin{array}{c}\underset{\mathcal{C},\mathcal{E}}{min}{∥\widehat{\mathcal{C}}∥}_{⊛}+\lambda {∥\mathcal{E}∥}_{2,1}\hfill \\ s.t.\mathcal{P}=\mathcal{C}+\mathcal{E}\hfill \\ \widehat{\mathcal{C}}=\phi \left(\mathcal{C}\right)\in {\Re }^{n\times m\times n}\hfill \end{array}$$

(2)

where the function $\phi \left(\mathcal{C}\right)$ rotates the tensor $\mathcal{C}\in {\Re }^{n\times n\times m}$ to obtain a new tensor $\widehat{\mathcal{C}}$ with the size of $n\times m\times n$. Correspondingly, $\mathcal{C}={\phi }^{-1}\left(\widehat{\mathcal{C}}\right)$. ${∥\widehat{\mathcal{C}}∥}_{⊛}={\displaystyle \sum _{j=1}^n}{∥{\widehat{\mathbf{C}}}_f^{\left(j\right)}∥}_{*}={\displaystyle \sum _{j=1}^n}{\displaystyle \sum _{i=1}^{min(n,m)}}{\sigma }_i\left({\widehat{\mathbf{C}}}_f^{\left(j\right)}\right)$ is the t-SVD-based TNN of $\widehat{\mathcal{C}}$, ${\widehat{\mathcal{C}}}_f=\mathrm{fft}(\widehat{\mathcal{C}},\left[\right],3)$ represents the discrete Fourier transform (DFT) along the third dimension of $\widehat{\mathcal{C}}$, ${\widehat{\mathbf{C}}}_f^{\left(j\right)}$ is the j-th frontal slice of ${\widehat{\mathcal{C}}}_f$, and ${\sigma }_i\left({\widehat{\mathbf{C}}}_f^{\left(j\right)}\right)$ denotes the i-th largest singular values of ${\widehat{\mathbf{C}}}_f^{\left(j\right)}$. The ${\ell }_{2,1}$-norm of $\mathcal{E}$ is defined as ${∥\mathcal{E}∥}_{2,1}\stackrel{\Delta }{=}{∥\mathrm{unfold}\left(\mathcal{E}\right)∥}_{2,1}$, resulting in jointly consistent magnitudes for the column of each frontal slice of the error tensor, $\mathrm{unfolding}\left(\mathcal{E}\right)=[{\mathbf{E}}^{\left(1\right)};{\mathbf{E}}^{\left(2\right)};\cdots ;{\mathbf{E}}^{\left(m\right)}]$.

Two advantages are brought by the t-SVD-based TNN regularization and the rotation operation. First, ETL-MSC could preserve the relationship among all views. Second, the computation complexity can be reduced largely. However, the TNN minimization problem shrinks each singular value with the same parameter. That is, each singular value is considered to be equally important, which contradicts the following facts. As we know, there is a great difference between the non-zero singular values of an arbitrary image. Generally, the larger singular values are considered to be more important than the smaller ones because the larger ones are associated with the salient parts in the image. Thus, to preserve the salient parts, the larger singular values should be shrunk less. Unusually, for a noised image, the larger singular values may carry undesirable information in the existence of noise (such as illumination). In either case, different singular values are suggested to be shrunk differently. To overcome this limit, a t-SVD-based WTNN is proposed and used to formulate the following model to cluster multi-view data [37] by the following:

$$\begin{array}{c}min{∥\widehat{\mathcal{C}}∥}_{\mathbf{w},⊛}+\lambda {∥\mathcal{E}∥}_{2,1}\hfill \\ s.t.\mathcal{P}=\mathcal{C}+\mathcal{E}\hfill \end{array}$$

(3)

where ${∥\widehat{\mathcal{C}}∥}_{\mathbf{w},⊛}={\displaystyle \sum _{j=1}^n}{∥{\widehat{\mathbf{C}}}_f^{\left(j\right)}∥}_{\mathbf{w},*}={\displaystyle \sum _{j=1}^n}{\displaystyle \sum _{i=1}^{min(n,m)}}{w}_i\times {\sigma }_i\left({\widehat{\mathbf{C}}}_f^{\left(j\right)}\right)$ is the t-SVD -based WTNN of $\widehat{\mathcal{C}}$. $\mathbf{w}$ is the weighted vector, with which different singular values could be shrunk differently.

After the essential tensor $\mathcal{C}$ is derived, the common TPM ${\mathbf{P}}^{*}$, which is input to the Markov Chain-based SC algorithm to obtain the final clustering result, is often computed by ${\mathbf{P}}^{*}={\displaystyle \sum _{v=1}^m}{\mathbf{C}}^{\left(v\right)}$. In other words, each frontal slice of the recovered tensor $\mathcal{C}$ is considered to be the intrinsic TPM of each single view, which tends to be a block-diagonal matrix. Such information is generally ignored by the aforementioned tensor methods.

3. The Proposed Method

With the aim of learning a robust TPM in a third-order tensor form, we design a novel robust tensor learning method for multi-view spectral clustering (RTL-MSC). Similar to MGL-WTNN, we also separate the input tensor $\mathcal{P}$ into two parts: the intrinsic tensor $\mathcal{C}$ and the error tensor $\mathcal{E}$. For the intrinsic tensor $\mathcal{C}$, we adopt the t-SVD-based WTNN to regularize its rotated tensor to capture the high-order correlations underlying multiple views. Meanwhile, we introduce the low-rank constraint to each frontal slice of $\mathcal{C}$ to emphasize its block-diagonal structure. Moreover, we utilize a novel ${\ell }_{2,p}$ norm to emphasize the sparsity of an error tensor. Accordingly, the mathematical model is designed as follows:

$$\begin{array}{c}\underset{\mathcal{C},\mathcal{E}}{min}{∥\widehat{\mathcal{C}}∥}_{\mathbf{w},⊛}+\alpha {\displaystyle \sum _{v=1}^m}{∥{\mathbf{E}}^{\left(v\right)}∥}_{2,p}+\beta {\displaystyle \sum _{v=1}^m}{∥{\mathbf{C}}^{\left(v\right)}∥}_{*}\hfill \\ s.t.\mathcal{P}=\mathcal{C}+\mathcal{E}\hfill \\ \widehat{\mathcal{C}}=\phi \left(\mathcal{C}\right)\in {\Re }^{n\times m\times n}\hfill \end{array}$$

(4)

where $\alpha >0$ and $\beta >0$ are two balanced parameters. The function $\phi \left(\mathcal{C}\right)$ rotates the tensor $\mathcal{C}\in {\Re }^{n\times n\times m}$ to a size of $n\times m\times n$. Correspondingly, $\mathcal{C}={\phi }^{-1}\left(\widehat{\mathcal{C}}\right)$, where ${\phi }^{-1}(•)$ represents the inverse operation of $\phi (•)$. ${∥•∥}_{2,p}$ denotes the ${\ell }_{2,p}$ norm, with the definition as follows: ${∥\mathbf{M}∥}_{2,p}={\displaystyle \sum _{i=1}^n}{∥{\mathbf{M}}_{:,i}∥}_2^p={\displaystyle \sum _{i=1}^n}{\left({\displaystyle \sum _{j=1}^n}{\left({\mathbf{M}}_{j,i}\right)}^2\right)}^{\frac{p}{2}}$. Here, $0<p\le 1$. Obviously, when $p=1$, the ${\ell }_{2,p}$ norm is reduced to the ${\ell }_{2,1}$ norm. The solution obtained by the ${\ell }_{2,p}$ norm is sparser than the one obtained by the ${\ell }_{2,1}$ norm. Thus, the second term makes it more robust to noised samples. The third term in Equation (4) makes each frontal slice of the essential tensor a block-diagonal matrix, which will captures the intrinsic clustering structure of the multi-view data.

To solve Equation (4), the alternating direction method of multipliers (ADMM) is used. ADMM reformulates the constrained problem as a unconstrained one by constructing an augmented Lagrangian function. By introducing an intermediate tensor $\mathcal{T}$, the Lagrangian function is written as follows:

$$\begin{array}{c}\mathcal{L}(\mathcal{C},\mathcal{E},\mathcal{T},\mathcal{Y},\mathcal{W})={∥\widehat{\mathcal{T}}∥}_{\mathbf{w},⊛}+\alpha {\displaystyle \sum _{v=1}^m}{∥{\mathbf{E}}^{\left(v\right)}∥}_{2,p}\hfill \\ +\beta {\displaystyle \sum _{v=1}^m}{∥C^{\left(v\right)}∥}_{*}+\frac{{\mu }_1}{2}{∥\mathcal{P}-\mathcal{C}-\mathcal{E}+\frac{\mathcal{Y}}{{\mu }_1}∥}_F^2+\frac{{\mu }_2}{2}{∥\mathcal{T}-\mathcal{C}+\frac{\mathcal{W}}{{\mu }_2}∥}_F^2\hfill \end{array}$$

(5)

where ${\mu }_1>0$ and ${\mu }_2>0$ are two penalty parameters, and $\mathcal{Y}$ and $\mathcal{W}$ are two Lagrange multipliers. Then, each variable is updated by fixing other ones.

Solving $\mathcal{T}$, by fixing $\mathcal{C}$, $\mathcal{E}$, and $\mathcal{W}$, the sub-problem of $\mathcal{T}$ can be reformulated as follows:

$$\begin{array}{c}\underset{\mathcal{T}}{min}{∥\widehat{\mathcal{T}}∥}_{\mathbf{w},⊛}+\frac{{\mu }_2}{2}{∥\mathcal{T}-\mathcal{C}+\frac{\mathcal{W}}{{\mu }_2}∥}_F^2\hfill \\ \Rightarrow \underset{\mathcal{T}}{min}{∥\widehat{\mathcal{T}}∥}_{\mathbf{w},⊛}+\frac{{\mu }_2}{2}{∥\tilde{\mathcal{T}}-\varphi (\mathcal{C}-\frac{\mathcal{W}}{{\mu }_2})∥}_F^2\hfill \end{array}$$

(6)

Denoting the tensor singular value decomposition of $\varphi (\mathcal{C}-\frac{\mathcal{W}}{{\mu }_2})$ as $\mathcal{U}*\mathcal{S}*{\mathcal{V}}^T$, ${\mathcal{S}}_f=\mathrm{fft}(\mathcal{S},\left[\right],3)$, the optimal solution of Equation (6) is ${\mathcal{T}}^{*}=\mathcal{U}*\mathrm{ifft}\left({\mathsf{\Gamma }}_{\frac{\mathbf{w}}{{\mu }_1}}\left({\mathcal{S}}_f\right)\right)*{\mathcal{V}}^T$, where ${\mathsf{\Gamma }}_{\frac{\mathbf{w}}{{\mu }_1}}\left({\mathcal{S}}_f\right)$ is a tensor stacked by ${\mathsf{\Gamma }}_{\frac{\mathbf{w}}{{\mu }_1}}\left({\mathcal{S}}_f^{\left(1\right)}\right),{\mathsf{\Gamma }}_{\frac{\mathbf{w}}{{\mu }_1}}\left({\mathcal{S}}_f^{\left(2\right)}\right),\cdots ,{\mathsf{\Gamma }}_{\frac{\mathbf{w}}{{\mu }_1}}\left({\mathcal{S}}_f^{\left(n\right)}\right)$, and ${\mathsf{\Gamma }}_{\frac{\mathbf{w}}{{\mu }_1}}\left({\mathcal{S}}_f^{\left(k\right)}\right),k=1,2,\cdots ,n$ is defined as ${\mathsf{\Gamma }}_{\frac{\mathbf{w}}{{\mu }_1}}\left({\mathcal{S}}_f^{\left(k\right)}\right)=diag({\theta }_1,{\theta }_2,\cdots ,{\theta }_{min(n,m)})$, and ${\theta }_i=max({S_f}_{ii}^{\left(k\right)}-\frac{{\omega }_i}{{\mu }_1},0)$ [37]. Following [37], the weighted vector $\mathbf{w}$ is adaptively computed by $\mathbf{w}=\sqrt{mn}\times {({\mathbf{S}}_f^{\left(k\right)}+ϵ)}^{-1}$, where $ϵ$ is set to be very small to avoid the singular value of 0.

Solving $\mathcal{E}$, for each view, ${\mathbf{E}}^{\left(v\right)},v=1,2,\cdots ,m$ is independent. By fixing $\mathcal{T}$, $\mathcal{C}$, and $\mathcal{Y}$, the sub-problem of ${\mathbf{E}}^{\left(v\right)}$ is written as follows:

$$\begin{array}{c}\underset{{\mathbf{E}}^{\left(v\right)}}{min}\alpha {∥{\mathbf{E}}^{\left(v\right)}∥}_{2,p}+\frac{{\mu }_1}{2}{∥{\mathbf{P}}^{\left(v\right)}-{\mathbf{C}}^{\left(v\right)}-{\mathbf{E}}^{\left(v\right)}+\frac{{\mathbf{Y}}^{\left(v\right)}}{{\mu }_1}∥}_F^2\hfill \\ \stackrel{\Delta }{=}\underset{\mathbf{E}}{min}\frac{\alpha }{{\mu }_1}{∥\mathbf{E}∥}_{2,p}+\frac{1}{2}{∥\mathbf{E}-\mathbf{Q}∥}_F^2\hfill \\ =\underset{\mathbf{E}}{min}\frac{\alpha }{{\mu }_1}{\displaystyle \sum _{i=1}^n}{∥{\mathbf{E}}_{:,i}∥}_2^p+\frac{1}{2}{\displaystyle \sum _{i=1}^n}{∥{\mathbf{E}}_{:,i}-{\mathbf{Q}}_{:,i}∥}_2^2\hfill \\ =\underset{\mathbf{E}}{min}{\displaystyle \sum _{i=1}^n}(\frac{\alpha }{{\mu }_1}{∥{\mathbf{E}}_{:,i}∥}_2^p+\frac{1}{2}{∥{\mathbf{E}}_{:,i}-{\mathbf{Q}}_{:,i}∥}_2^2)\hfill \end{array}$$

(7)

where $\mathbf{Q}=[{\mathbf{Q}}^{\left(1\right)};{\mathbf{Q}}^{\left(2\right)};\cdots ;{\mathbf{Q}}^{\left(m\right)}]$, ${\mathbf{Q}}^{\left(i\right)}$ is the i-th frontal slice of the tensor $\mathcal{P}-\mathcal{C}+\frac{\mathcal{Y}}{{\mu }_1}$. Then, $\mathbf{E}$ is solved column by column. Thus, Equation (7) is rewritten as follows:

$$\underset{{\mathbf{E}}_{:,i}}{min}\frac{\alpha }{{\mu }_1}{∥{\mathbf{E}}_{:,i}∥}_2^p+\frac{1}{2}{∥{\mathbf{E}}_{:,i}-{\mathbf{Q}}_{:,i}∥}_2^2$$

(8)

here, ${\mathbf{E}}_{:,i}$ can be treated as a special matrix. Then, we can perform a thin SVD on it. As observed, ${\mathbf{E}}_{:,i}$ has exactly one singular value, given by the following:

$$\sigma \left({\mathbf{E}}_{:,i}\right)=\sqrt{{\mathbf{E}}_{:,i}^T{\mathbf{E}}_{:,i}}={∥{\mathbf{E}}_{:,i}∥}_2$$

(9)

Thus, Equation (8) can be rewritten as follows:

$$\underset{{\mathbf{E}}_{:,i}}{min}\frac{\alpha }{{\mu }_1}{\left(\sigma \left({\mathbf{E}}_{:,i}\right)\right)}^p+\frac{1}{2}{∥{\mathbf{E}}_{:,i}-{\mathbf{Q}}_{:,i}∥}_2^2$$

(10)

According to the work in [38], the solution of Equation (10) is as follows:

$${\mathbf{E}}_{:,i}={\mathbf{u}}_i{\sigma }^{*}\left({\mathbf{E}}_{:,i}\right){\mathbf{v}}_i^T$$

(11)

where ${\mathbf{u}}_{\mathbf{i}}$ and ${\mathbf{v}}_i$ are the left and right singular vectors of ${\mathbf{Q}}_{:,i}$, respectively. Additionally, 

$${\sigma }^{*}\left({\mathbf{E}}_{:,i}\right)=\underset{x\ge 0}{argmin}\tau x^p+\frac{1}{2}{(x-\sigma \left({\mathbf{Q}}_{:,i}\right))}^2$$

(12)

where $\frac{\alpha }{{\mu }_1}\stackrel{\Delta }{=}\tau$, $\sigma \left({\mathbf{Q}}_{:,i}\right)={∥{\mathbf{Q}}_{:,i}∥}_2$ [39], and Equation (12) can be solved by Lemma 1 in [33]. Correspondingly, ${\mathbf{E}}_{:,\mathbf{j}}$ is derived.

Solving ${\mathbf{C}}^{\left(v\right)}$, by fixing $\mathcal{T}$, $\mathcal{E}$, $\mathcal{W}$, and $\mathcal{Y}$, Equation (5), w.r.t. $\mathcal{C}$, can be written as as follows:

$$\underset{{\mathbf{C}}^{\left(v\right)}}{argmin}\beta {\displaystyle \sum _{v=1}^m}{∥{\mathbf{C}}^{\left(v\right)}∥}_{*}+\frac{{\mu }_1}{2}{∥\mathcal{C}-(\mathcal{P}-\mathcal{E}+\frac{\mathcal{Y}}{{\mu }_1})∥}_F^2+\frac{{\mu }_2}{2}{∥\mathcal{C}-(\mathcal{T}+\frac{\mathcal{W}}{{\mu }_2})∥}_F^2$$

(13)

For each $v,v=1,2,\cdots ,m$, ${\mathbf{C}}^{\left(\mathbf{v}\right)}$ is independent, so Equation (13) can be rewritten as follows:

$$\begin{array}{c}\underset{{\mathbf{C}}^{\left(v\right)}}{argmin}\beta {∥{\mathbf{C}}^{\left(v\right)}∥}_{*}+\frac{{\mu }_1}{2}{∥{\mathbf{C}}^{\left(v\right)}-{\mathbf{M}}_1^{\left(v\right)}∥}_F^2+\frac{{\mu }_2}{2}{∥\mathbf{C}-{\mathbf{M}}_2^{\left(v\right)}∥}_F^2\hfill \\ =\underset{{\mathbf{C}}^{\left(v\right)}}{argmin}\beta {∥{\mathbf{C}}^{\left(v\right)}∥}_{*}+\frac{{\mu }_1+{\mu }_2}{2}{∥{\mathbf{C}}^{\left(v\right)}-\frac{{\mu }_1{\mathbf{M}}_1^{\left(v\right)}+{\mu }_2{\mathbf{M}}_2^{\left(v\right)}}{{\mu }_1+{\mu }_2}∥}_F^2\hfill \end{array}$$

(14)

where ${\mathbf{M}}_{\mathbf{1}}^{\left(\mathbf{v}\right)}$ is the v-th frontal slice of ${\mathcal{M}}_1=\mathcal{P}-\mathcal{E}+\frac{\mathcal{Y}}{{\mu }_1}$ and ${\mathbf{M}}_{\mathbf{2}}^{\left(\mathbf{v}\right)}$ is the v-th frontal slice of ${\mathcal{M}}_2=\mathcal{T}+\frac{\mathcal{W}}{{\mu }_2}$. According to [40,41], the solution is ${\mathbf{C}}^{\left(v\right)}=\mathbf{U}{\mathsf{\Gamma }}_{\frac{\beta }{{\mu }_1+{\mu }_2}}\left(\Sigma \right){\mathbf{V}}^T$, where $\mathbf{U}\Sigma {\mathbf{V}}^T$ is the singular value decomposition of $\frac{{\mu }_1{\mathbf{M}}_1^{\left(v\right)}+{\mu }_2{\mathbf{M}}_2^{\left(v\right)}}{{\mu }_1+{\mu }_2}$, and ${\mathsf{\Gamma }}_{\theta }\left(x\right)=max(x-\theta ,0)+min(x+\theta ,0)$ is the shrinkage operator.

Summarily, Equation (4) can be solved by Algorithm 1.

Algorithm 1 RTL-MSC Algorithm

Input: Multi-view data ${\mathsf{X}}^j,j=1,2,\dots ,m,\alpha ,\beta ,$ sample number n, and cluster number c   1: Calculate the similarity matrix ${\mathbf{A}}^j$ of the j-th each view by $A_{i,j}^v=exp(-{∥{\mathbf{x}}_i^v-{\mathbf{x}}_j^v∥}_2^2/\phantom{{∥{\mathbf{x}}_i^v-{\mathbf{x}}_j^v∥}_2^2\tau }\tau )$.   2: Compute the degree matrix ${\mathbf{D}}^j$ with the $(i,i)$-th element as $D_{i,i}^j={\sum }_{k=1}^n{A}_{i,k}^j$, $i=1,2,\cdots ,n$, and the other elements are all set to 0.   3: Compute the TPM of the j-th view by ${\mathbf{P}}^j={\left({\mathbf{D}}^j\right)}^{-1}{\mathbf{A}}^j$.   4: Construct the input tensor $\mathcal{P}$ by stacking ${\mathbf{P}}^j$ as its j-th frontal slice.   5: Initialize $\mathcal{T}=\mathcal{C}=\mathcal{E}=\mathcal{Y}=\mathcal{W}=\mathbf{0}$; ${\mu }_1={\mu }_2={10}^{-5},\eta =2,{\mu }_{max}={10}^{10},ϵ={10}^{-7}$   6: repeat   7:    Update $\mathcal{T}$ by solving Equation (6).   8:    Update $\mathcal{C}$ by solving Equation (14).   9:    Update the Lagrange multiplier $\mathcal{W}$ using $\mathcal{W}=\mathcal{W}+{\mu }_2(\mathcal{T}-\mathcal{C})$. 10:    Update $\mathcal{E}$ by solving Equation (8). 11:    Update the Lagrange multiplier $\mathcal{Y}$ using $\mathcal{Y}=\mathcal{Y}+{\mu }_1(\mathcal{P}-\mathcal{C}-\mathcal{E})$. 12:    Update parameters ${\mu }_1$ and ${\mu }_2$ by ${\mu }_1={\mu }_2=min(\eta {\mu }_1,{\mu }_{max})$; 13:    Check the convergence conditions: 14:    ${\parallel \mathcal{P}-\mathcal{C}-\mathcal{E}\parallel }_{\infty }<ϵ$. 15: until converge 16: Obtain the affinity matrix ${\mathbf{P}}^{*}$ as ${\mathbf{P}}^{*}=\left({\sum }_{v=1}^m|{\mathbf{C}}^{\left(v\right)}|+|{\mathbf{C}}^{\left(v\right)\mathrm{T}}|\right)/\left(2m\right)$; 17: ${\mathbf{P}}^{*}$ and c are input to the Markov chain-based SC algorithm. Output: The predicted label.

For Algorithm 1, the computation is mainly consumed by the optimization of $\mathcal{T}$, $\mathcal{C}$, and $\mathcal{E}$. To solve $\mathcal{T}\in {\Re }^{n\times m\times n}$, the FFT and inverse FFT of an $n\times m\times n$ tensor along the third dimension and the SVD of each frontal slice of an $n\times m\times n$ tensor are involved, leading to a complexity of $\mathcal{O}(m^2{n}^2+m{n}^{2}log\left(n\right))$ at each iteration. $\mathcal{C}\in {\Re }^{n\times n\times m}$ is solved slice by slice. For each frontal slice ${\mathbf{C}}^{\left(v\right)}$, the SVD of an $n\times n$ matrix is calculated, leading to a complexity of $\mathcal{O}\left(n^3\right)$. Thus, the solving of $\mathcal{C}$ takes a complexity of $\mathcal{O}\left(m{n}^3\right)$ at each iteration. The ${\ell }_{2,p}$ norm minimization problem requires a complexity of $\mathcal{O}\left(m{n}^2\right)$ at each iteration. Consequently, by denoting the number of iterations as K, the proposed algorithm takes a complexity of $\mathcal{O}\left(Km{n}^2(m+n+log\left(n\right))\right)$ in total.

4. Experiments

To evaluate the performance of the proposed methods, we conduct experiments on three widely used datasets. All experiments are performed in Matlab R2021a.

4.1. Competitors

The proposed methods are compared with the spectral clustering method and six multi-view state-of-the-art clustering methods. (1) ${\mathrm{S}\mathrm{C}}_{\mathrm{best}}$ is achieved by performing the spectral clustering on each single view and obtaining the best result among all views. (2) RMSC learns a common low-rank transition probability matrix via low-rank and sparse matrix decomposition. (3) MSCAN incorporates the common similarity matrix learning and the predicted label matrix learning into a unified framework, which allocates weight for each view automatically without any additional parameters. (4) ETL-MSC recovers an essential tensor via a tensor RPCA, which utilizes the t-SVD-based TNN to approximate the tensor low rank. (5) ATPML-MSC [34] learns a consistent TPM directly under the tensor RPCA framework. The obtained TPM is non-negative and symmetric without any postprocessing. (6) MGL-WTNN [37] recovers an low-rank tensor by using the t-SVD-based WTNN to approximate the tensor low rank.

4.2. Datasets

We take Flower (https://www.robots.ox.ac.uk/~vgg/data/flowers ), Coil20 (https://www.cs.columbia.edu/CAVE/software/softlib), and NUS-WIDE (https://lms.comp.nus.edu.sg/wp-content/uploads/2019/research/nuswide/NUS-WIDE.html) to evaluate the clustering performance. The Flower dataset consists of 1360 samples from 17 different flower categories. These images have large scale, pose, and light variations. Following [34], three different features, 1360d-color, 1360d-texture, and 1360d-shape, are extracted. Coil20 is an object dataset containing 1440 images from 20 object categories. We employ the 4096d intensity features and 3304d LBP features as the two views. NUS-WIDE is a real-world web image dataset from the National University of Singapore. We adopt six types of low-level features extracted from 2400 images of 12 concepts, including a 64-D color histogram, 144-D color correlogram, 75-D edge direction histogram, 128-D wavelet texture, 225-D block-wise color moments, and 500-D bag of words based on SIFT descriptions.

4.3. Experimental Process

To quantitatively evaluate the performance of all these clustering methods, we adopt five widely used metrics, accuracy (ACC), normalized mutual information (NMI), Purity, F-score, and average entropy (AVE). For ACC, NMI, Purity, and F-score, larger values represent better a clustering performance. For AVE, lower values indicate better a clustering performance.

For the compared methods, we download their codes and perform experiments following the relative papers. For RMSC, ETL-MSC, and MGL-WTNN, one parameter is involved and is selected from $[0.0001, 1]$. Two parameters in ATPML-MSC are changed in the range of $[0.0001, 0.5]$, respectively. For each method, we fix the parameters to the ones corresponding to the better accuracy and repeat experiments ten times to obtain the mean value and standard deviation.

4.4. Experimental Results and Analysis

The clustering results on all the three datasets are shown in

Table 1. Clustering results (mean ± standard deviation) on Flower.

Methods	ACC	NMI	Purity	F-score	AVE
${\mathrm{S}\mathrm{C}}_{\mathrm{best}}$	0.3737 ± 0.020	0.3975 ± 0.017	0.3916 ± 0.017	0.2637 ± 0.008	2.4604 ± 0.027
RMSC	0.3796 ± 0.015	0.4167 ± 0.008	0.3982 ± 0.015	0.2685 ± 0.008	2.3890 ± 0.032
MSCAN	0.4585 ± 0.001	0.5070 ± 0.001	0.4852 ± 0.002	0.3164 ± 0.002	2.1405 ± 0.009
ETL-MSC	0.9108 ± 0.034	0.9114 ± 0.016	0.9167 ± 0.028	0.8626 ± 0.033	0.3693 ± 0.068
ATPML-MSC	0.9772 ± 0.001	0.9661 ± 0.002	0.9772 ± 0.001	0.9554 ± 0.004	0.1395 ± 0.010
MGL-WTNN	0.9811 ± 0.001	0.9703 ± 0.001	0.9811 ± 0.001	0.9628 ± 0.001	0.1215 ± 0.003
RTL-WTNN	0.9874 ± 0.001	0.9800 ± 0.001	0.9874 ± 0.001	0.9753 ± 0.001	0.0818 ± 0.003

,

Table 2. Clustering results (mean ± standard deviation) on Coil20.

Methods	ACC	NMI	Purity	F-score	AVE
${\mathrm{S}\mathrm{C}}_{\mathrm{best}}$	0.6886 ± 0.026	0.8131 ± 0.010	0.7276 ± 0.019	0.6609 ± 0.025	0.8388 ± 0.048
RMSC	0.7026 ± 0.020	0.8022 ± 0.006	0.7101 ± 0.011	0.6628 ± 0.015	0.8668 ± 0.031
MSCAN	0.8061 ± 0.022	0.9299 ± 0.005	0.8481 ± 0.016	0.7441 ± 0.030	0.4266 ± 0.044
ETL-MSC	0.8730 ± 0.021	0.9204 ± 0.009	0.8811 ± 0.015	0.8510 ± 0.020	0.3518 ± 0.040
ATPML-MSC	0.9513 ± 0.038	0.9795 ± 0.016	0.9616 ± 0.031	0.9481 ± 0.041	0.0990 ± 0.078
MGL-WTNN	0.8853 ± 0.023	0.9300 ± 0.006	0.8919 ± 0.015	0.8645 ± 0.016	0.3098 ± 0.032
RTL-WTNN	0.9769 ± 0.028	0.9866 ± 0.011	0.9806 ± 0.020	0.9695 ± 0.027	0.0627 ± 0.053

and

Table 3. Clustering results (mean ± standard deviation) on NUS-WIDE.

Methods	ACC	NMI	Purity	F-score	AVE
${\mathrm{S}\mathrm{C}}_{\mathrm{best}}$	0.2338 ± 0.008	0.1095 ± 0.004	0.2457 ± 0.007	0.1361 ± 0.002	3.1950 ± 0.016
RMSC	0.2962 ± 0.004	0.1556 ± 0.004	0.3035 ± 0.004	0.1758 ± 0.003	3.029 ± 0.013
MSCAN	0.2493 ± 0.005	0.1972 ± 0.004	0.2643 ± 0.005	0.1739 ± 0.001	3.0376 ± 0.012
ETL-MSC	0.6906 ± 0.001	0.7007 ± 0.001	0.6927 ± 0.001	0.6243 ± 0.001	1.078 ± 0.004
ATPML-MSC	0.9286 ± 0.001	0.8844 ± 0.001	0.9286 ± 0.001	0.8658 ± 0.002	0.4062 ± 0.005
MGL-WTNN	0.8787 ± 0.001	0.7774 ± 0.001	0.8787 ± 0.001	0.7791 ± 0.001	0.7990 ± 0.003
RTL-WTNN	0.9441 ± 0.001	0.8976 ± 0.001	0.9441 ± 0.001	0.8933 ± 0.001	0.3650 ± 0.001

, where the best results are highlighted in bold and the second ones are underlined.

As can be seen from Table 1, Table 2 and Table 3, the proposed RTL-MSC method almost outperforms the other methods including the the recently proposed ATPML-MSC and MGL-WTNN. For example, RTL-WTNN improves by about $2.56\%$, $1.90\%$, and $2.14\%$ for ACC, Purity, and F-score over the most competitive method ATPML-MSC on the Coil20 dataset, respectively. This represents that the affinity matrix is more effective for a multi-view spectral clustering task. Compared with MGL-WTNN, RTL-MSC improves by $11.16\%$, $5.66\%$, and $8.87\%$ in terms of ACC, NMI, and Purity on the Coil20 dataset. This means that the global geometric structure in each single view and the ${\ell }_{2,p}$ norm-based error constraint are useful for multi-view clustering. ATPML-MSC outperforms ETL-MSC on all these three datasets. This is mainly because the affinity matrix is learned in a one-step process with a probability constraint. MGL-WTNN outperforms ETL-MSC on all these three datasets for almost all metrics. This demonstrates that shrinking each singular value with an adaptive learned weight improves the clustering performance efficiently. On the NUS-WIDE dataset, the SC based on a single view, RMSC and MSCAN, obtains an unsatisfied clustering performance. The major reason may be that the intrinsic structure of each view and the noise are not well depicted by these three methods.

There are three parameters, $\alpha$, $\beta$, and p, involved in Equation (4), which directly affect the performance of the proposed RTL-MSC. To investigate the influence of $\alpha$ and $\beta$, we set $p=1$ and select $\alpha$ and $\beta$ from $[0.01,0.5]$ and $[0.0001,0.01]$, respectively. We report the ACC and NMI versus different combinations of $\alpha$ and $\beta$ on the Flower and Coil20 datasets in Figure 1. It is shown that when $\alpha$ changes in $[0.04,0.5]$, both ACC and NMI fluctuate slightly via $\beta$, and RTL-WTNN achieves promising clustering results on the Flower and Coil20 datasets. This means that the proposed RTL-WTNN is insensitive to $\alpha$ and $\beta$ when they are changed in a suitable range.

Moreover, we fix $\alpha$ and $\beta$ and change p in the range of $[0.9,1]$ to investigate the influence of the ${\ell }_{2,p}$ norm. As shown in Figure 2, the highest values of ACC, NMI, and Purity is achieved at $p=0.98$, which illustrates that $the{\ell }_{2,p}$ norm improves the clustering performance.

4.5. Ablation Study

To investigate the effect of exploiting the low-rank structure of the each frontal slice of the essential tensor and the ${\ell }_{2,p}$ norm-based error constraint, we perform an ablation study. First, we remove the third term in Equation (4) and mark the derived model as RTL-WNN-1. Then, we also perform experiments by using the ${\ell }_{2,1}$ norm to regularize the error tensor, which is termed as RTL-WTNN-2. The results are shown in Figure 3. It can be seen that RTL-WTNN outperforms RTL-WTNN-1 and RTL-WTNN-2 on all the three datasets.

5. Conclusions

We develop a novel tensor learning method for multi-view spectral clustering, which can depict well the noise embedded in each view by using the ${\ell }_{2,p}$ norm to constrain the error tensor. Additionally, by adopting the tensor low-rank constraint and low-rank regularization of each frontal slice of the essential tensor, the common affinity matrix computed by the essential tensor can capture well the intrinsic clustering structure of the multi-view data. The experiments on three widely used datasets exhibit that the proposed RTL-MSC improves by about 1.02–11.16%, 1.39–12.02%, 1.02–8.87%, and 1.99–11.42% in terms of ACC, NMI, Purity, and F-score over the baseline MGL-WTNN, which illustrates the superiority of the proposed model. We plan to investigate a fast multi-view spectral clustering algorithm for clustering big data in our next work.