Enhanced Similarity Matrix Learning for Multi-View Clustering

Abstract

Graph-based multi-view clustering is a fundamental analysis method that learns the similarity matrix of multi-view data. Despite its success, it has two main limitations: (1) complementary information is not fully utilized by directly combining graphs from different views; (2) existing multi-view clustering methods do not adequately address redundancy and noise in the data, significantly affecting performance. To address these issues, we propose the Enhanced Similarity Matrix Learning (ES-MVC) for multi-view clustering, which dynamically integrates global graphs from all views with local graphs from each view to create an improved similarity matrix. Specifically, the global graph captures cross-view consistency, while the local graph preserves view-specific geometric patterns. The balance between global and local graphs is controlled through an adaptive weighting strategy, where hyperparameters adjust the relative importance of each graph, effectively capturing complementary information. In this way, our method can learn the clustering structure that contains fully complementary information, leveraging both global and local graphs. Meanwhile, we utilize a robust similarity matrix initialization to reduce the negative effects caused by noisy data. For model optimization, we derive an effective optimization algorithm that converges quickly, typically requiring fewer than five iterations for most datasets. Extensive experimental results on diverse real-world datasets demonstrate the superiority of our method over state-of-the-art multi-view clustering methods. In our experiments on datasets such as MSRC-v1, Caltech101, and HW, our proposed method achieves superior clustering performance with average accuracy (ACC) values of 0.7643, 0.6097, and 0.9745, respectively, outperforming the most advanced multi-view clustering methods such as OMVFC-LICAG, which yield ACC values of 0.7284, 0.4512, and 0.8372 on the same datasets.

1. Introduction

Multi-view data obtained from multiple sources [1] allow us to represent the same instance from different perspectives. For example, an image can be described by its SIFT feature, HOG feature, and LBP feature, and even its text feature. Typically, different views of the same data provide consistent and complementary information. By exploiting the consistent structure among multi-view data, multi-view learning obtains better performance when compared with traditional learning methods, which only use single-view learning [2,3,4]. Consequently, multi-view learning has found extensive applications in various real-world scenarios, including but not limited to image recognition, data mining, and multimedia [5,6,7], for example, generating pseudo-labels for data before large-scale model training to improve training efficiency or clustering users in social networks to enhance accuracy of recommendation systems.

Multi-view clustering [8,9,10] is a basic tool of multi-view learning. Its goal is to explore consistent information from multiple features and partition data into different groups. Many works have been proposed in recent years and have achieved promising results. Generally, they can be roughly divided into four classes. (1) Co-training/Co-regularized-based methods [11,12,13] are precursor techniques that focus on two views within a clustering framework. These methods work toward reducing the disparity between the two views to enhance clustering performance. The traditional co-training and co-regularized algorithm lacks robustness and can be significantly hampered by just a few inaccurate outliers. (2) Subspace-based methods [14,15,16] learn a common low-dimensional representation for multiple views. They leverage the self-expression characteristic of the data to identify the 34 underlying subspaces and subsequently classify the data points accordingly. For example, Yin et al. [17] learned a shared coefficient matrix by ensuring that the coefficient matrices from each pair of views were as similar as possible. (3) Non-negative matrix factorization based methods [18,19,20] aim to learn shared indicator matrices across multiple perspectives. For example, Akata et al. [21] enforces a shared indicator matrix among different views to perform multi-view clustering in the NMF framework. (4) Graph-based methods [22,23,24,25] learn a common similarity matrix across the multiple views, which can further reveal data cluster structures. They utilize the properties of the Laplacian graph, whose edges denote the similarities between each pair of data points and solve a relaxation of the normalized min-cut problem on the graph. Compared with other methods, graph-based methods can be applied to arbitrarily shaped clustering scenarios and achieve high performance.

Among the diverse multi-view clustering methods, graph-based methods achieve extraordinary performance since they can fuse heterogeneous features and learn a more distinctive clustering structure. Currently, numerous graph-based multi-view clustering methods have been proposed in order to effectively capture the manifold information from each view.

However, most of these methods directly employ original data as input to learn the common similarity matrix for multi-view clustering [22,23,26,27,28,29]. They construct the initial similarity matrix by using manifold learning on the original data for clustering. For example, MV-KM [30] can automatically determine the number of clusters without initialization; MV-FCM [31] extends single-view fuzzy c-means clustering to the multi-view setting and automatically identifies the optimal number of clusters; and MVKM-L2 [32], combines feature weights and view weights to achieve superior performance across multiple datasets. Despite their impressive performance, their effectiveness relies on the assumption that the original data are clean and trusted. To eliminate this assumption, low-rank based methods [24,33] have been proposed, focusing on capturing the global structure from entire samples of each view. This method removes the assumption of clean original data but also ignores the local manifold structure across neighboring samples.

Furthermore, these methods have not adequately explored the structural information across different views. Cross-view structural information is crucial for enhancing the consistency of the shared similarity matrix and achieving a better clustering structure. Nevertheless, existing methods have not effectively utilized this key information, resulting in suboptimal clustering structures. In other words, during data collection steps, even though different views describe the same object, there inevitably exists view-specific information. If such information cannot be properly utilized in cross-view learning, it may become undesirable noise.

Therefore, inspired by the ease of obtaining a robust similarity matrix rather than removing noise data, and recognizing that the elements of a similarity matrix can reveal data correlations, we propose an Enhanced Similarity Matrix Learning method for multi-view clustering (ES-MVC). As illustrated in Figure 1, our method operates on the premise that the correlation information within each view aligns with the correlation information across all views. ES-MVC integrates both global (global similarity matrix) and local (local similarity matrix) graphs to harness complementary information, enhancing clustering performance. More specifically, we firstly employ robust initial similarity matrices as input to alleviate the effect of the redundancy and noise. Next, we construct a global graph and multiple local graphs from training data under the same rank constraint. Therefore, both noise information and complementary information can be well respected. For the model optimization, we propose to employ an alternating direction strategy to solve our ES-MVC method. Finally, we conduct several experiments on real-world datasets to justify the effectiveness of our ES-MVC method. The key contributions of our work are outlined as follows:

• A novel multi-view clustering method is proposed to obtain more complementary information from all the multi-view data, named Enhanced Similarity Matrix Learning for multi-view clustering (ES-MVC).
• Our method leverages both local and global structures across multiple views to construct a consistent graph for enhanced clustering performance.
• Furthermore, we apply a rank constraint to the similarity matrix in order to dynamically optimize neighbor assignment for improved clustering results.
• Additionally, we develop a robust optimization algorithm to efficiently solve our objective function.

Experimental results demonstrate that our method significantly outperforms traditional multi-view clustering methods. In our experiments on datasets such as MSRC-v1, Caltech101, and HW, our proposed method achieves superior clustering performance with average accuracy (ACC) values of 0.7643, 0.6097, and 0.9745, respectively, outperforming the most advanced multi-view clustering methods such as OMVFC-LICAG [34], which yield ACC values of 0.7284, 0.4512, and 0.8372 on the same datasets. These findings suggest that our method, which combines both global and local structures, leads to more consistent and accurate clustering.

The remaining sections of this paper are as follows: Section 2 reviews related work, Section 3 explains the proposed ES-MVC method in detail, Section 4 presents experimental results comparing its performance with state-of-the-art techniques, Section 5 provides a convergence analysis, and Section 5 concludes with future research directions.

2. Related Works

More and more multi-view data are becoming available because of information explosion. This is spurring a lot of studies on multi-view learning, an important part of which is multi-view clustering. This paper primarily emphasizes graph-based multi-view clustering. In this part, we will introduce some multi-view clustering methods based on graphs from the following two aspects.

2.1. Similarity Matrix Learning for Multi-View Clustering

The key point of graph-based multi-view clustering methods is how to construct an ideal similarity matrix for clustering. Many studies have studied it and achieved some promising results. For example, Yu et al. [22] leveraged a co-training technique to learn a Bayesian undirected graph for multiple-view data. Kumar et al. [23] proposed co-regularized spectral multi-view clustering by minimizing the disagreement across two views and adopted the eigenvectors of the Laplacian graph for clustering. With the same technique, Cai et al. [26] integrated multi-view features by developing a multi-modal spectral clustering method. However, these methods are based on the pairwise concept, which is neither efficient nor optimal for multi-view data. To tackle this problem, Tang et al. [27] proposed a way to fuse information from multiple graph sources called LMF. Wang et al. [35] enforced a common eigenvector matrix and formulated a multi-objective problem for clustering. Although these methods improve the clustering performance, their effectiveness hinges on the assumption of the original data’s cleanliness and trustworthiness. All of them directly use the original data to learn a graph for clustering, and the performance is sensitive to noisy data.

2.2. Robust Multi-View Clustering

To enhance the robustness of multi-view clustering, researchers have sought solutions to mitigate the impact of noise. For instance, Xia et al. [33] developed a method called robust multi-view spectral clustering (RMSC) that uses low-rank and sparse decomposition to construct a transition probability matrix, effectively isolating noise from each graph. In order to automatically learn a set of weights for all graphs and assign small weights to data containing noise, Nie et al. [28,29] introduced an auto-weighted multiple graph learning model (AMGL) and an adaptive neighbor method through learning a Laplacian rank-constrained graph (MLAN). Li et al. [36] presented a Multi-view-based Parameter-Free (MPF) method for detecting groups in crowd scenes using an adaptive neighbors strategy. Motivated by the robust graphs learned by the above methods, some works use a robust similarity matrix as input to learn an optimal similarity matrix. These methods can release the influence of noisy data to some extent. For example, Nie et al. [37] introduced a novel multi-view clustering approach that takes graphs as input. Although these methods improve robustness, they primarily use a low rank or an $l1$-norm to separate noise from the original data, resulting in a robust similarity matrix that mainly captures the global structure from all samples in each view. However, low-rank methods, such as those mentioned in [24,33], focus on capturing the global structure from entire samples of each view and ignore the local manifold structure across neighboring samples. Additionally, these methods fail to adequately consider and integrate the overall global structure among all views.

Compared to existing methods, ES-MVC introduces a novel design by using robust similarity matrices to reduce the impact of noise, jointly capturing both global and local structural information to enhance representation and automatically fusing multiple graphs without manual parameter tuning. We summarize methods that we have mentioned in the following

Table 1. The difference between existing methods.

Methods	Robust Similarity Matrix	Global Structure	Local Structure	Weighted Fusion
Co-training [22]	No	No	Yes	No
Co-regularized [26]	No	No	Yes	No
LMF [27]	No	Yes	No	No
RMSC [33]	Yes	Yes	No	No
AMGL [28]	No	Yes	No	Yes
MLAN [29]	No	Yes	No	Yes
MPF [36]	No	Yes	Yes	Yes
PwMC [37]	Yes	Yes	No	No
ES-MVC	Yes	Yes	Yes	Yes

.

3. Our Method

3.1. Motivation

From the above analysis, it is clear that most multi-view clustering methods that rely on original features are vulnerable to noise and outliers. Additionally, methods that use graphs as input fail to fully exploit the local structure within neighboring samples and the global structure across all views. Therefore, the similarity matrices generated by these methods are not optimal for clustering. To this end, we propose a new multi-view clustering method called Enhanced Similarity Matrix Learning for multi-view clustering (ES-MVC). It adaptively learns a shared similarity matrix by extracting the local as well as the global structure information from several robust initial graphs pre-defined from all views. A comparison of the general graph-based framework and our ES-MVC method is shown in Figure 2.

3.2. ES-MVC Method

Consider multi-view data $\{{\mathbf{X}}^{\left(1\right)},\dots ,{\mathbf{X}}^{\left(m\right)}\}\in {\mathbf{R}}^{n\times d^{\left(v\right)}\times m}$, where ${\mathbf{X}}^{\left(v\right)}=[{\mathit{x}}_1^v,{\mathit{x}}_2^v,\dots ,{\mathit{x}}_n^v]\in {\mathbf{R}}^{n\times d^{\left(v\right)}}(v=1,\dots ,m)$ is v-th view matrix, n is the number of data points, m is the number of views, and $d^{\left(v\right)}$ is the feature dimension of the v-th view.

3.2.1. Local Structure in a Single View

Let ${\mathbf{A}}^v=[{\mathit{a}}_1^v,{\mathit{a}}_2^v,\dots ,{\mathit{a}}_n^v]\in {\mathbb{R}}^{n\times n}$ denote the v-th initial local similarity matrix constructed from the v-th view ${\mathbf{X}}^{\left(v\right)}$. We can obtain the local structure based on the initial local similarity matrix by solving the following problem:

$$\begin{array}{c}\underset{\mathbf{W}}{min}{\displaystyle \sum _{v=1}^m}{\displaystyle \sum _{i,j}}∥{{\mathit{a}}_i^v-{\mathit{a}}_j^v∥}_2^2{\mathit{w}}_{ij}^2,\hfill \\ \mathrm{s}.\mathrm{t}.\forall i,{\mathit{w}}_i^{\top }\mathbf{1}=1,0\le {\mathit{w}}_{ij}\le 1,\hfill \end{array}$$

(1)

where $\mathbf{W}=\left\{{\mathit{w}}_{ij}\right\}\in {\mathbb{R}}^{n\times n}$, $\forall i,j\in 1,\dots ,n$ is a novel similarity matrix that we need to learn.

3.2.2. Global Structure Across Each Different View

Let $\mathbf{Y}=[{\mathbf{X}}^{\left(1\right)},\dots ,{\mathbf{X}}^{\left(m\right)}]=[{\mathit{y}}_1,{\mathit{y}}_2,\dots ,{\mathit{y}}_n]\in {\mathbb{R}}^{n\times {\sum }_{v=1}^m{d}_v}$ denote the global data matrix, which is concatenated by all the views, where ${\mathit{y}}_i\in {\mathbb{R}}^{{\sum }_{v=1}^m{d}_v}$ and $\mathbf{G}=[{\mathit{g}}_1,{\mathit{g}}_2,\dots ,{\mathit{g}}_n]\in {\mathbb{R}}^{n\times n}$ is the initial global similarity matrix constructed from the data matrix $\mathbf{Y}$.

Then, we can learn the global structure, which lies among all views, based on the initial global similarity matrix, by the following function:

$$\begin{array}{c}\underset{\mathbf{W}}{min}{\displaystyle \sum _{i,j}}∥{{\mathit{g}}_i-{\mathit{g}}_j∥}_2^2{\mathit{w}}_{ij}^2,\hfill \\ \mathrm{s}.\mathrm{t}.\forall i,{\mathit{w}}_i^{\top }\mathbf{1}=1,0\le {\mathit{w}}_{ij}\le 1.\hfill \end{array}$$

(2)

As shown in Figure 2, we adopt the graph construction method from [38] to create the robust initial graphs, including the local similarity matrices $\mathbf{A}$ and the global similarity matrix $\mathbf{G}$, as inputs for the ES-MVC.

3.2.3. ES-MVC Objective Function

We integrate both local and global structures into the multi-view clustering objective function and utilize an irrational form to automatically determine the optimal weight for each graph, eliminating the need for manual parameter setting for each view. However, most existing graph-based methods require an additional clustering step (such as k-means) to obtain the final clustering result. The two-step strategy will inevitably lose some information, reducing the clustering performance. Therefore, inspired by [39], we constrain the structure of the graph and directly obtain clustering partitions from the connected components of the structured graph.

Consequently, we derive our objective function as follows.

$$\begin{array}{c}J\left(\mathbf{W}\right)=\underset{\mathbf{W}}{\arg min}\underset{\mathbf{Local}\mathbf{structure}}{\underbrace{{\displaystyle \sum _{v=1}^m}\sqrt{{\displaystyle \sum _{i,j}}{∥{\mathit{a}}_i^v-{\mathit{a}}_j^v∥}_2^2{\mathit{w}}_{ij}^2}}}\hfill \\ +\underset{\mathbf{Global}\mathbf{structure}}{\underbrace{\alpha \sqrt{{\displaystyle \sum _{i,j}}{∥{\mathit{g}}_i-{\mathit{g}}_j∥}_2^2{\mathit{w}}_{ij}^2}}},\hfill \\ \mathrm{s}.\mathrm{t}.\forall i,{\mathit{w}}_i^{\top }\mathbf{1}=1,0\le {\mathit{w}}_{ij}\le 1,\underset{\mathbf{Rank}\mathbf{constraint}}{\underbrace{\mathrm{rank}\left({\mathbf{L}}_{\mathbf{W}}\right)=n-c}}.\hfill \end{array}$$

(3)

where $\alpha$ is a parameter to balance the impact of local structure and global structure. Let $\mathbf{D}$ be the degree matrix whose i-th diagonal element is ${\mathbf{D}}_{ii}={\sum }_{j=1}^n{\mathit{w}}_{ij}$, and let ${\mathbf{L}}_{\mathbf{W}}$ denote the normalized graph Laplacian matrix, which is defined as ${\mathbf{L}}_{\mathbf{W}}=\mathbf{D}-({\mathbf{W}}^{\top }+\mathbf{W})/2$. To achieve the ideal neighbor assignment in the learned similarity matrix, we add a rank constraint to the Laplacian matrix.

The rank constraint ensures that the number of connected components in the graph matches the number of clusters c, with each connected component corresponding to one cluster. This allows us to directly derive the clustering partitions from the graph, bypassing the need for an additional clustering step like k-means, which could lead to information loss.

The primary advantage of the rank constraint is that it enforces a structured graph where the connected components represent meaningful cluster assignments. This is particularly effective in multi-view clustering tasks, where maintaining consistency across multiple views is essential. By directly constraining the graph’s rank, the clustering process becomes more efficient, and the method avoids the potential loss of information inherent in two-step clustering strategies.

3.2.4. Robust Initial Graph Construction

While we have thoroughly examined the process of deriving the final optimal graph matrix $\mathbf{W}$ from global and local graphs, it is important to recognize that the quality of these initial graphs significantly influences the final clustering performance. Accordingly, this section focuses specifically on the initialization methodology for each view’s graph structure. Motivated by [38,39], the initial local graph for the v-th view can be obtained by solving the following problem.

$$\underset{\mathbf{A}⪰\mathbf{0},\mathbf{A}\mathbf{1}=1}{min}\sum _{i=1}^n\sum _{j=1}^m{∥x_i^v-x_j^v∥}_2^2{a}_{ij}^v+\gamma {\left(a_{ij}^v\right)}^2,$$

(4)

The first term in Equation (4) enables the affinity matrix $A^v$ to preserve local geometric relationships between samples and anchors, where the affinity values increase inversely with distance. The second term acts as a regularizer that promotes dense connections between anchors and samples. Through the hyperparameter $\gamma$, we control the sparsity of these connections by limiting the number of anchors assigned to each sample. Formally, let $d_{i,\left[j\right]}$ denote the distance between the i-th sample and its j-th nearest anchor, with distances ordered such that $d_{i,\left[1\right]}\le d_{i,\left[2\right]}\le \dots \le d_{i,\left[m\right]}$. The closed-form solution for Equation (4) is as follows.

$$a_{ij}^v=max\left({\displaystyle \frac{d_{i,[k+1]}-d_{i,\left[j\right]}}{k{d}_{i,[k+1]}-{\sum }_{l=1}^k{d}_{i,\left[l\right]}}},0\right),$$

(5)

where k is the number each sample connects to and $\gamma$ can be computed as $\gamma ={\displaystyle \frac{k}{2}}{d}_{i,[k=1]}-{\displaystyle \frac{1}{2}}{\sum }_{l=1}^k{d}_{i,\left[l\right]}$.

The initial graph construction process described here effectively leverages both global and local data characteristics to enhance clustering performance while mitigating noise. Specifically, the first term in the optimization function preserves local geometric relationships by maintaining affinities between samples that are close in space, helping to reduce noise due to local inconsistencies. Meanwhile, the regularization term incorporates global features, ensuring that the affinity matrix is sparsified and capturing global patterns in the data. This balance of local and global features plays a crucial role in addressing view-specific noise, which can arise due to variations in view angles, lighting, or partial occlusions during the data collection phase. Even though different views describe the same object, they often contain view-specific information that, if not properly handled, can introduce undesirable noise. By optimizing the graph structure to balance local and global features, we reduce the impact of such noise, resulting in a clearer and more robust affinity matrix $A^v$, which accurately reflects the underlying clusters. More details on this process can be found in [39].

3.3. Optimization

For the convenience of optimization, we first modify the formulation of the objective function. Denote

$$\begin{array}{c}{\displaystyle {\tilde{\lambda }}_v=\frac{1}{\sqrt{{\displaystyle \sum _{i,j}}∥{{\mathit{a}}_i^v-{\mathit{a}}_j^v∥}_2^2{\mathit{w}}_{ij}^2}}(v=1,\dots ,m),}\hfill \\ {\displaystyle \phi =\frac{1}{\sqrt{{\displaystyle \sum _{i,j}}∥{{\mathit{g}}_i-{\mathit{g}}_j∥}_2^2{\mathit{w}}_{ij}^2}}.}\hfill \end{array}$$

(6)

Through simple algebra and by substituting ${\tilde{\lambda }}_v$ and $\phi$ into objective function (3), we can get

$$\begin{array}{c}J\left(\mathbf{W}\right)=\underset{\mathbf{W}}{\arg min}{\displaystyle \sum _{v=1}^m}{\tilde{\lambda }}_v{\displaystyle \sum _{i,j}}∥{{\mathit{a}}_i^v-{\mathit{a}}_j^v∥}_2^2{\mathit{w}}_{ij}^2\hfill \\ +\phi {\displaystyle \sum _{i,j}}∥{{\mathit{g}}_i-{\mathit{g}}_j∥}_2^2{\mathit{w}}_{ij}^2,\hfill \\ \mathrm{s}.\mathrm{t}.\forall i,{\mathit{w}}_i^{\top }\mathbf{1}=1,0\le {\mathit{w}}_{ij}\le 1,\mathrm{rank}\left({\mathbf{L}}_{\mathbf{W}}\right)=n-c.\hfill \end{array}$$

(7)

We regard the data matrix $\mathbf{Y}$ as the ($m+1$)-th view matrix and the initial global similarity matrix as the ($m+1$)-th similarity matrix. Let ${\lambda }_v=\left\{\begin{array}{c}{\tilde{\lambda }}_v,v\le m\hfill \\ \phi ,v=m+1\hfill \end{array}\right.$. So we can combine the first term and the second term in Equation (7) and use the total initial similarity matrix $\mathbf{A}$ to replace the combination of all initial similarity matrices,

$$\begin{array}{c}J\left(\mathbf{W}\right)=\underset{\mathbf{W}}{\arg min}{\displaystyle \sum _{v=1}^{m+1}}{\lambda }_v{\displaystyle \sum _{i,j}}∥{{\mathit{b}}_i^v-{\mathit{b}}_j^v∥}_2^2{\mathit{w}}_{ij}^2,\hfill \\ \mathrm{s}.\mathrm{t}.\forall i,{\mathit{w}}_i^{\top }\mathbf{1}=1,0\le {\mathit{w}}_{ij}\le 1,\mathrm{rank}\left({\mathbf{L}}_{\mathbf{W}}\right)=n-c.\hfill \end{array}$$

(8)

where $\mathbf{A}=\left\{{\mathbf{A}}^v\right\}=[{\mathbf{A}}^1,{\mathbf{A}}^2,\dots ,{\mathbf{A}}^m,\mathbf{G}]\in {\mathbb{R}}^{n\times n\times (m+1)}$ is the total initial similarity matrix, which consists of all the initial local similarity matrices and an initial global similarity matrix, ${\mathbf{A}}^v=[{\mathit{b}}_1^v,{\mathit{b}}_2^v,\dots ,{\mathit{b}}_n^v]\in {\mathbb{R}}^{n\times n},v=1,\dots ,m+1$.

Assume that the class indicator matrix $\mathbf{H}=[{\mathit{h}}_1,{\mathit{h}}_2,\dots ,{\mathit{h}}_n]\in {\mathbb{R}}^{n\times c}$ is unified across all the views and set aside the normalization to different graphs. Each row of $\mathbf{H}$ represents a data point, and each column of $\mathbf{H}$ represents a cluster, which means that ${\mathit{h}}_i$ indicates the cluster that data point i belongs to.

Let ${\sigma }_i(i=1,2,\dots ,n)$ be the i-th eigenvalue of ${\mathbf{L}}_{\mathbf{W}}$; then, we have $0\le {\sigma }_i\le {\sigma }_{i+1}$. As a result, the constraint $\mathrm{rank}({\mathbf{L}}_{\mathbf{W}})=n-c$ can be satisfied if ${\sum }_{i=1}^c{\sigma }_i=0$. According to the theorem in [40], we transform the problem of summing the smallest c eigenvalues of ${\mathbf{L}}_{\mathbf{W}}$ into minimizing the trace of $H^{\top }{L}_{W}H$, and then we have the following equivalent formulation:

$$\sum _{i=1}^c{\sigma }_i=\underset{{\mathbf{H}}^{\top }\mathbf{H}=\mathbf{I}}{min}\mathrm{tr}\left({\mathbf{H}}^{\top }{\mathbf{L}}_{\mathbf{W}}\mathbf{H}\right).$$

(9)

Therefore, Equation (8) can be converted to

$$\begin{array}{c}J(\mathbf{W},\mathbf{H},{\lambda }_v)=\underset{\mathbf{W},{\mathbf{H}}^{\top }\mathbf{H}=\mathbf{I}}{\arg min}\mu \mathrm{tr}\left({\mathbf{H}}^{\top }{\mathbf{L}}_{\mathbf{W}}\mathbf{H}\right)\hfill \\ +{\displaystyle \sum _{v=1}^{m+1}}{\lambda }_v{\displaystyle \sum _{i,j}}∥{{\mathit{b}}_i^v-{\mathit{b}}_j^v∥}_2^2{\mathit{w}}_{ij}^2,\hfill \\ \mathrm{s}.\mathrm{t}.\forall i,{\mathit{w}}_i^{\top }\mathbf{1}=1,0\le {\mathit{w}}_{ij}\le 1.\hfill \end{array}$$

(10)

where $\mu$ is a parameter that needs to be a large number so that the optimal solution to the objective function (10) will ensure that the equation ${\sum }_{i=1}^c{\sigma }_i=0$ holds. In our method, the parameter $\mu$ is used to enforce that the graph has exactly c connected components. Instead of manually adjusting $\mu$, we use an adaptive rule to update it during the optimization process. Initially, $\mu$ is set to a smaller value, and after each iteration, it is updated based on the number of zero eigenvalues of the Laplacian matrix. If the number of zero eigenvalues is less than c, $\mu$ is multiplied by 2; if the number is greater than $c+1$, $\mu$ is divided by 2; otherwise, the iteration ends. This adaptive updating ensures that the final graph has exactly c connected components. This approach eliminates the need for manually tuning $\mu$, and its adjustment is naturally linked to the structure of the graph. The number of zero eigenvalues of the Laplacian matrix provides a direct indication of how well the graph captures the underlying clusters. This dynamic adjustment makes the method robust to different dataset sizes, as $\mu$ adapts to the data’s complexity. Additionally, in practice, the value of $\mu$ does not need to be large from the outset, as the adaptive strategy allows it to grow or shrink as needed to guide the graph structure toward the correct number of clusters.

In the objective function (10), we have three unknown variables $\mathbf{W}$, ${\lambda }_v$ and $\mathbf{H}$, where ${\lambda }_v$ is related to $\mathbf{W}$. Thus, it is difficult to directly solve the solution of the objective function Equation (10). Here we develop a more efficient algorithm that alternatively updates $\mathbf{W}$ (while fixing $\mathbf{H}$ and ${\lambda }_v$) and updates $\mathbf{H}$ and ${\lambda }_v$ (while fixing $\mathbf{W}$). The detailed process is described as follows:

• Update $\mathbf{H}$ and ${\lambda }_v$, while fixing $\mathbf{W}$
  To be specific, $\mathbf{W}$ is known in the ($t+1$)-th iteration, so we can easily calculate ${\lambda }_v$ by Equation (6), and update $\mathbf{H}$ by minimizing the objective function Equation (10). In this case, the second term and the third term in the objective function Equation (10) become constant. Thus, the objective function Equation (10) is converted to solve the following objective function

  $$\mathbf{H}=\underset{{\mathbf{H}}^{\top }\mathbf{H}=\mathbf{I}}{\arg min}\mathrm{tr}\left({\mathbf{H}}^{\top }{\mathbf{L}}_{\mathbf{W}}\mathbf{H}\right).$$

  (11)
  According to the matrix theory, the column vectors of the optimal indicator matrix $\mathbf{H}$ of Equation (11) are composed of the c eigenvectors of ${\mathbf{L}}_{\mathbf{W}}$ corresponding to the c smallest eigenvalues.
• Update $\mathbf{W}$ while fixing $\mathbf{H}$ and ${\lambda }_v$
  After that, we can update $\mathbf{W}$ while fixing $\mathbf{H}$ and ${\lambda }_v$. Denote ${\mathbf{z}}_{ij}={\sum }_{v=1}^{m+1}{\lambda }_v{∥{\mathit{b}}_i^v-{\mathit{b}}_j^v∥}_2^2$, ${\mathit{y}}_{ij}={∥{\mathit{h}}_i-{\mathit{h}}_j∥}_2^2$.
  According to the equation of $2\mathrm{tr}\left({\mathbf{H}}^{\top }{\mathbf{L}}_{\mathbf{W}}\mathbf{H}\right)={\sum }_{i,j}{∥{\mathit{h}}_i-{\mathit{h}}_j∥}_2^2{\mathit{w}}_{ij}={\sum }_{i,j}{\mathit{y}}_{ij}{\mathit{w}}_{ij}$, the objective function (10) can be converted to Equation (12)

  $$\begin{array}{c}\mathbf{W}=\underset{\mathbf{W}}{\arg min}{\displaystyle \sum _{i,j}}({\mathbf{z}}_{ij}{\mathit{w}}_{ij}^2+\mu {\mathit{y}}_{ij}{\mathit{w}}_{ij}),\hfill \\ \mathrm{s}.\mathrm{t}.\forall i,{\mathit{w}}_i^{\top }\mathbf{1}=1,0\le {\mathit{w}}_{ij}\le 1.\hfill \end{array}$$

  (12)
  Through simple algebra, for each i, we have Equation (13)

  $$\begin{array}{c}\mathbf{W}=\underset{{\mathit{w}}_i}{\arg min}{∥{\mathit{w}}_i+{\mathit{u}}_i∥}_2^2,\hfill \\ \mathrm{s}.\mathrm{t}.\forall i,{\mathit{w}}_i^{\top }\mathbf{1}=1,0\le {\mathit{w}}_{ij}\le 1,\hfill \end{array}$$

  (13)

  where ${\mathit{u}}_i\in {\mathbb{R}}^{n\times 1}$ is a vector and the j-th element is ${\mathit{u}}_{ij}={\displaystyle \frac{1}{2{\mathit{z}}_{ij}}}\mu {\mathit{y}}_{ij}$.

We can calculate a closed-form solution of the problem easily with the method presented in [39]. As a result, the objective function can update the similarity matrix until it achieves ideal neighbors. In addition, $\mathbf{W}$ can be used for clustering directly, which greatly improves the stability of clustering performance. Parameter $\mu$ is first initialized to a smaller value, and after each iteration, it is updated according to the number of zero eigenvalues of the Laplacian matrix. If the number of zero eigenvalues is less than c, multiply $\mu$ by 2; if the number of zero eigenvalues is greater than $c+1$, divide $\mu$ by 2; otherwise, end the iteration. This iterative procedure repeats until the algorithm converges. The convergence proof is given in the following section. Finally, we summarize the pseudocode of solving the objective function (10) in Algorithm 1.

Algorithm 1 ES-MVC

Input: ${\mathbf{X}}^{\left(1\right)},\dots ,{\mathbf{X}}^{\left(m\right)}$, ${\mathbf{X}}^{\left(v\right)}\in {\mathbf{R}}^{n\times d^{\left(v\right)}}(v=1,\dots ,m)$, class number c. Initialize: $t=1$, ${\mathbf{W}}_0\in R^{n\times n}$ which is initial graph. while not converge do    1. Calculate the matrix ${\mathbf{A}}^v$ and ${\lambda }_v$ for each view.    2. Calculate ${\mathbf{L}}_{\mathbf{W}}$.    3. Update the cluster indicator matrix      $\mathbf{H}=\underset{{\mathbf{H}}^{\top }\mathbf{H}=\mathbf{I}}{\arg min}\mathrm{tr}\left({\mathbf{H}}^{\top }{\mathbf{L}}_{\mathbf{W}}\mathbf{H}\right)$    4. For each i, update ${\mathit{w}}_i$.    5. Update t: $t\leftarrow t+1$. end while Compute the clustering labels by running K-means on $\mathbf{H}$ or spectral clustering on $\mathbf{W}$. Output: The final clustering result.

3.4. Convergence Analysis

In this subsection, we provide the convergence analysis of our proposed algorithm. We adopt an alternating iteration approach for solving each subproblem. Although the non-convex nature of the problem allows for the possibility of local optima, the penalty parameter is gradually increased with each iteration. This adaptive strategy ensures that the iteration will continue until it converges to the global optimum, which is a common practice in optimization. The details of the convergence analysis are as follows.

Theorem 1.

In each iteration of Algorithm 1, we have

$$\begin{array}{c}{\displaystyle \sum _{v=1}^{m+1}}{\displaystyle \frac{{\displaystyle \sum _{i,j}}{∥{\mathbf{b}}_i^v-{\mathbf{b}}_j^v∥}_2^2{\left({\mathbf{w}}_{ij}^2\right)}^{t+1}}{2\sqrt{{\displaystyle \sum _{i,j}}{∥{\mathbf{b}}_i^v-{\mathbf{b}}_j^v∥}_2^2{\left({\mathbf{w}}_{ij}^2\right)}^{t+1}}}}\hfill \\ \le {\displaystyle \sum _{v=1}^{m+1}}{\displaystyle \frac{{\displaystyle \sum _{i,j}}{∥{\mathbf{b}}_i^v-{\mathbf{b}}_j^v∥}_2^2{\left({\mathbf{w}}_{ij}^2\right)}^t}{2\sqrt{{\displaystyle \sum _{i,j}}{∥{\mathbf{b}}_i^v-{\mathbf{b}}_j^v∥}_2^2{\left({\mathbf{w}}_{ij}^2\right)}^t}}},\hfill \end{array}$$

(14)

i.e., ${\sum }_{v=1}^{m+1}\sqrt{{\sum }_{i,j}{∥{\mathbf{b}}_i^v-{\mathbf{b}}_j^v∥}_2^2{\left({\mathbf{w}}_{ij}^2\right)}^{t+1}}\le {\sum }_{v=1}^{m+1}\sqrt{{\sum }_{i,j}{∥{\mathbf{b}}_i^v-{\mathbf{b}}_j^v∥}_2^2{\left({\mathbf{w}}_{ij}^2\right)}^t}$.

Proof. 

For each iteration t, according to step 3 in Algorithm 1, we have the following inequality:

$$\begin{array}{c}{\displaystyle \sum _{v=1}^{m+1}}{\displaystyle \frac{{\displaystyle \sum _{i,j}}{∥{\mathit{b}}_i^v-{\mathit{b}}_j^v∥}_2^2{\left({\mathit{w}}_{ij}^2\right)}^{t+1}}{2\sqrt{{\displaystyle \sum _{i,j}}{∥{\mathit{b}}_i^v-{\mathit{b}}_j^v∥}_2^2{\left({\mathit{w}}_{ij}^2\right)}^t}}}\hfill \\ \le {\displaystyle \sum _{v=1}^{m+1}}{\displaystyle \frac{{\displaystyle \sum _{i,j}}{∥{\mathit{b}}_i^v-{\mathit{b}}_j^v∥}_2^2{\left({\mathit{w}}_{ij}^2\right)}^t}{2\sqrt{{\displaystyle \sum _{i,j}}{∥{\mathit{b}}_i^v-{\mathit{b}}_j^v∥}_2^2{\left({\mathit{w}}_{ij}^2\right)}^t}}}.\hfill \end{array}$$

(15)

According to the inequality $\mathit{a}+\mathit{b}\ge 2\sqrt{\mathit{ab}}$, we can easily get ${\displaystyle \sqrt{\mathit{a}}-\frac{\mathit{a}}{2\sqrt{\mathit{b}}}\le \sqrt{\mathit{b}}-\frac{\mathit{b}}{2\sqrt{\mathit{b}}}}$. Let $\mathit{a}={\sum }_{i,j}{∥{\mathit{b}}_i^v-{\mathit{b}}_j^v∥}_2^2{\left({\mathit{w}}_{ij}^2\right)}^{t+1}$, $\mathit{b}={\sum }_{i,j}{∥{\mathit{b}}_i^v-{\mathit{b}}_j^v∥}_2^2{\left({\mathit{w}}_{ij}^2\right)}^t$, then we have

$$\begin{array}{c}({\displaystyle \sum _{v=1}^{m+1}}\sqrt{{\displaystyle \sum _{i,j}}{∥{\mathit{b}}_i^v-{\mathit{b}}_j^v∥}_2^2{\left({\mathit{w}}_{ij}^2\right)}^{t+1}}\hfill \\ -{\displaystyle \sum _{v=1}^{m+1}}{\displaystyle \frac{{\displaystyle \sum _{i,j}}{∥{\mathit{b}}_i^v-{\mathit{b}}_j^v∥}_2^2{\left({\mathit{w}}_{ij}^2\right)}^{t+1}}{2\sqrt{{\displaystyle \sum _{i,j}}{∥{\mathit{b}}_i^v-{\mathit{b}}_j^v∥}_2^2{\left({\mathit{w}}_{ij}^2\right)}^t}}})\hfill \\ \le ({\displaystyle \sum _{v=1}^{m+1}}\sqrt{{\displaystyle \sum _{i,j}}{∥{\mathit{b}}_i^v-{\mathit{b}}_j^v∥}_2^2{\left({\mathit{w}}_{ij}^2\right)}^t}\hfill \\ -{\displaystyle \sum _{v=1}^{m+1}}{\displaystyle \frac{{\displaystyle \sum _{i,j}}{∥{\mathit{b}}_i^v-{\mathit{b}}_j^v∥}_2^2{\left({\mathit{w}}_{ij}^2\right)}^t}{2\sqrt{{\displaystyle \sum _{i,j}}{∥{\mathit{b}}_i^v-{\mathit{b}}_j^v∥}_2^2{\left({\mathit{w}}_{ij}^2\right)}^t}}}).\hfill \end{array}$$

(16)

By adding Equations (15) and (16) to both sides, we have

$$\begin{array}{c}{\displaystyle \sum _{v=1}^{m+1}}\sqrt{{\displaystyle \sum _{i,j}}{∥{\mathit{b}}_i^v-{\mathit{b}}_j^v∥}_2^2{\left({\mathit{w}}_{ij}^2\right)}^{t+1}}\hfill \\ \le {\displaystyle \sum _{v=1}^{m+1}}\sqrt{{\displaystyle \sum _{i,j}}{∥{\mathit{b}}_i^v-{\mathit{b}}_j^v∥}_2^2{\left({\mathit{w}}_{ij}^2\right)}^t}.\hfill \end{array}$$

(17)

Thus, Algorithm 1 will converge to a minimum of the objective function Equation (10). □

4. Experiments

In this section, we validate our method using six multi-view datasets: five widely used ones (Handwritten numerals, MSRC-v1, Caltech101, BDGP, and BBCSport) and one created by us (Volunteer). We compare it with spectral clustering (SC) [3] and five other multi-view clustering methods: Feature Concatenation Spectral Clustering (ConSC) [23], Auto-weighted Multiple Graph Learning (AMGL) [28], Multi-view Learning with Adaptive Neighbors (MLAN) [29], Robust Multi-view Spectral Clustering (RMSC) [33], Exclusivity-Consistency Regularized Multi-view Subspace Clustering (ECMSC) [41] and latent-information-guided one-step multi-view fuzzy clustering based on cross-view anchor graph (OMVFC-LICAG) [34]. Following [29], we set the parameter $\mu$ for the Laplacian matrix rank constraint.

To preliminarily evaluate the effectiveness of the proposed method in manifold structural multi-view scenarios, we add a three-ring dataset to our experiment. We firstly apply a synthetic three-ring toy dataset to preliminarily evaluate the effectiveness of our proposed method in manifold structural multi-view scenarios. The three-ring dataset consists of 600 samples from 3 non-linearly separable clusters, each represented by a 2D coordinate. The second view is obtained by performing FFT on the first view. The result (ACC, NMI, PUR) of the preliminary experiment is 1.0000 ± 0.00, 1.0000 ± 0.00, 1.0000 ± 0.00, and the cluster visualization is shown in Figure 3, which preliminarily demonstrates that our algorithm achieved good results on multi-view scenarios.

Then we evaluated clustering performance using three standard metrics: accuracy (ACC) [26], normalized mutual information (NMI) [42], and purity [43]. All three metrics range from 0 to 1, with higher values indicating better clustering performance. Since all methods involve a k-means procedure sensitive to initial centroids, we repeat the experiments 40 times and report the average performance and standard deviation (Std) for the ACC, NMI, and purity metrics.

We extracted various features from the datasets to perform multi-view clustering. Brief introductions are as follows:

Color Moment (CMT) [44]: extracts the feature that describes the global color distribution of an image.

HOG [45]: describes the information of local or global shape by capturing edge or gradient structure of an image.

GIST [46]: obtains global non-spatial features and their statistics of an image.

LBP [47]: describes the texture feature of the image.

SIFT [48]: is the feature constructed by key points from an image and allows local shape distortion and illumination difference.

Doc2Vec [49]: features embedding documents into a fixed-dimensional vector space, capturing semantic information and contextual relationships of the document.

TF-IDF [50]: features evaluate the importance of words in a document by measuring the product of term frequency and inverse document frequency, reducing the impact of common words on text representation.

LSA [51]: features extract latent semantic structures and concepts by applying singular value decomposition to reduce the dimensionality of text data to a lower-dimensional space.

Finally, it should be noticed that we assume that all views are complete and well aligned across view-specific samples.

The experiments were conducted using MATLAB 2023 on a machine equipped with an Intel Xeon Platinum 8352V CPU (2 processors, 2.10 GHz base frequency, 3.50 GHz turbo frequency) and 128 GB RAM. No GPU was used in the experiments, demonstrating the efficiency of our method even in a CPU-only environment.

4.1. Experiments on the MSRC-v1 Dataset

The MSRC-v1 dataset [52] includes 240 images from 8 classes, with 30 images per class. For our experiment, we selected seven classes and extracted five features: 24-dim CM, 576-dim HOG, 512-dim GIST, 256-dim LBP, and 254-dim CENTRIST.

We evaluated our method on this dataset, as shown in

Table 2. Clustering performance on the MSRC-v1 dataset.

Methods	ACC	NMI	Purity
SC1	0.3244 ± 0.04	0.2740 ± 0.03	0.3878 ± 0.05
SC2	0.5608 ± 0.05	0.5077 ± 0.04	0.5995 ± 0.06
SC3	0.6311 ± 0.08	0.5954 ± 0.07	0.6949 ± 0.05
SC4	0.4699 ± 0.07	0.4225 ± 0.04	0.5196 ± 0.03
SC5	0.5611 ± 0.03	0.4747 ± 0.02	0.5914 ± 0.05
ConcatSC	0.6769 ± 0.05	0.6539 ± 0.05	0.7219 ± 0.08
ESMSC	0.6953 ± 0.02	0.6867 ± 0.01	0.7575 ± 0.03
RMSC	0.6751 ± 0.06	0.6187 ± 0.05	0.7147 ± 0.07
AMGL	0.6875 ± 0.06	0.6587 ± 0.08	0.7212 ± 0.05
MLAN	0.6811 ± 0.02	0.6297 ± 0.02	0.7331 ± 0.02
OMVFC-LICAG	0.7284 ± 0.03	0.5907 ± 0.02	0.7384 ± 0.03
Ours	0.7643 ± 0.08	0.7275 ± 0.06	0.8125 ± 0.04

, where SC1 to SC5 represent the five features. Our method consistently outperformed the others.

4.2. Experiments on the Caltech101 Dataset

The Caltech101 dataset [53] contains images from 101 classes. Figure 4 shows samples from three classes. We used a subset of 441 images from 7 selected classes, with 3 features: 620-dim HOG, 2560-dim SIFT, and 1160-dim LBP.

Table 3. Clustering performance on the Caltech101 dataset.

Methods	ACC	NMI	Purity
SC1	0.4821 ± 0.04	0.3660 ± 0.04	0.5577 ± 0.04
SC2	0.5601 ± 0.04	0.4378 ± 0.05	0.6347 ± 0.04
SC3	0.4611 ± 0.03	0.2755 ± 0.03	0.4915 ± 0.03
ConcatSC	0.5357 ± 0.03	0.4525 ± 0.04	0.6178 ± 0.05
ESMSC	0.5551 ± 0.03	0.4687 ± 0.03	0.6377 ± 0.05
RMSC	0.5005 ± 0.04	0.3416 ± 0.02	0.5544 ± 0.05
AMGL	0.4615 ± 0.05	0.3274 ± 0.06	0.5255 ± 0.02
MLAN	0.5531 ± 0.02	0.4787 ± 0.02	0.6487 ± 0.01
OMVFC-LICAG	0.4512 ± 0.02	0.3612 ± 0.04	0.5283 ± 0.06
Ours	0.6097 ± 0.04	0.5132 ± 0.03	0.6902 ± 0.03

presents the clustering performance, with our method consistently outperforming the others.

4.3. Experiments on the HW Dataset

The HW [54] dataset contains 2000 images across 10 digit classes. We extracted six features: 76-dim FOU, 216-dim FAC, 64-dim KAR, 240-dim PIX, 47-dim ZER, and 6-dim MOR. Figure 5 shows sample images from the HW dataset.

Table 4. Clustering performance on the HW dataset.

Methods	ACC	NMI	Purity
SC1	0.6315 ± 0.07	0.6626 ± 0.04	0.6815 ± 0.06
SC2	0.6152 ± 0.10	0.6847 ± 0.05	0.6824 ± 0.08
SC3	0.7505 ± 0.08	0.8180 ± 0.06	0.8131 ± 0.09
SC4	0.8495 ± 0.05	0.8567 ± 0.08	0.8725 ± 0.08
SC5	0.6109 ± 0.07	0.6068 ± 0.05	0.6601 ± 0.05
SC6	0.4049 ± 0.04	0.4771 ± 0.05	0.4541 ± 0.08
ConcatSC	0.827 ± 0.15	0.8690 ± 0.07	0.8649 ± 0.09
ESMSC	0.7399 ± 0.01	0.7968 ± 0.02	0.7769 ± 0.02
RMSC	0.8625 ± 0.02	0.8105 ± 0.04	0.8715 ± 0.04
AMGL	0.8336 ± 0.03	0.8568 ± 0.03	0.8405 ± 0.05
MLAN	0.9731 ± 0.01	0.9387 ± 0.01	0.9731 ± 0.02
OMVFC-LICAG	0.8372 ± 0.03	0.7981 ± 0.02	0.8381 ± 0.04
Ours	0.9745 ± 0.04	0.9407 ± 0.04	0.9739 ± 0.02

lists the clustering performance of each method on the HW dataset. The results demonstrate the high accuracy of our method.

4.4. Experiments on a Visual and Text Dataset

BDGP [55] is a multi-modal dataset with visual and textual features. It contains 2500 images of drosophila embryos in 5 categories. Each image is represented by a 1750-dim visual vector and a 79-dim textual vector. We evaluate our method using both views.

Table 5. Clustering performance on the BDGP dataset.

Methods	ACC	NMI	Purity
SC1	0.4694 ± 0.02	0.2757 ± 0.02	0.4740 ± 0.02
SC2	0.3893 ± 0.02	0.2057 ± 0.09	0.3907 ± 0.02
ConcatSC	0.4747 ± 0.08	0.2962 ±  0.03	0.4871 ± 0.02
ESMSC	0.9191 ± 0.02	0.8535 ± 0.05	0.9192 ± 0.02
RMSC	0.4585 ± 0.01	0.2672 ± 0.03	0.4584 ± 0.04
AMGL	0.6032 ± 0.02	0.6102 ± 0.05	0.6855 ± 0.06
MLAN	0.9501 ± 0.02	0.8874 ± 0.01	9.9505 ± 0.02
OMVFC-LICAG	0.7754 ± 0.05	0.6941 ± 0.04	0.7752 ± 0.03
Ours	0.9688 ± 0.02	0.9132 ± 0.03	0.9687 ± 0.03

shows the clustering performance for each method. The results demonstrate the high accuracy of our method.

4.5. Experiments on Two Text Datasets

The BBCSport [56] dataset consists of 737 sports news articles across five categories. The Volunteer dataset consists of 2038 volunteers, both male and female, aged 22 to 60, categorized into 5 classes: alleviation, emergency, environment, neighborhood, and training. Each volunteer’s information is described by two distinct text files: one file contains the volunteer’s subjective intentions, while the other file provides objective details about the volunteer. In our experiment, we extracted 3 kinds of features from these datasets, i.e., 100-dim Doc2Vec, 100-dim TF-IDF, and 100-dim LSA. Therefore, we have 3 views for the BBCSport datasets, and 6 views for the Volunteer dataset.

Table 6. The average ACC, NMI, purity, and std of each method on the BBCSport dataset.

Methods	ACC	NMI	Purity
SC1	0.4325 ± 0.02	0.1485 ± 0.04	0.4642 ± 0.03
SC2	0.5573 ± 0.01	0.3444 ± 0.05	0.6230 ± 0.09
SC3	0.7094 ± 0.03	0.6204 ± 0.08	0.7299 ± 0.05
ConcatSC	0.7132 ± 0.07	0.6215 ±  0.03	0.7228 ± 0.05
ESMSC	0.7375 ± 0.05	0.6420 ± 0.08	0.7387 ± 0.08
RMSC	0.6788 ± 0.09	0.5672 ± 0.05	0.7275 ± 0.05
AMGL	0.7165 ± 0.07	0.5825 ± 0.06	0.7460 ± 0.03
MLAN	0.7365 ± 0.04	0.6277 ± 0.02	0.7555 ± 0.05
OMVFC-LICAG	0.5172 ± 0.07	0.5590 ± 0.02	0.3101 ± 0.06
Ours	0.7855 ± 0.09	0.6686 ± 0.07	0.7997 ± 0.08

and

Table 7. Clustering performance on the Volunteer dataset.

Methods	ACC	NMI	Purity
SC1	0.3695 ± 0.02	0.0123 ± 0.02	0.3864 ± 0.05
SC2	0.3652 ± 0.02	0.0080 ± 0.03	0.3768 ± 0.04
SC3	0.3745 ± 0.05	0.0167 ± 0.06	0.3909 ± 0.06
SC4	0.3751 ± 0.06	0.0208 ± 0.03	0.3785 ± 0.08
SC5	0.3731 ± 0.02	0.0254 ± 0.02	0.3925 ± 0.05
SC6	0.3774 ± 0.03	0.0121 ± 0.03	0.3774 ± 0.05
ConcatSC	0.3757 ± 0.06	0.0382 ±  0.04	0.3908 ± 0.07
ESMSC	0.3973 ± 0.02	0.0381 ± 0.02	0.4013 ± 0.05
RMSC	0.3755 ± 0.02	0.0373 ± 0.02	0.3961 ± 0.08
AMGL	0.3843 ± 0.08	0.0199 ± 0.04	0.3852 ± 0.04
MLAN	0.3852 ± 0.02	0.0211 ± 0.01	0.3877 ± 0.02
OMVFC-LICAG	0.3011 ± 0.03	0.2552 ± 0.02	0.2984 ± 0.03
Ours	0.4533 ± 0.02	0.0456 ± 0.05	0.4767 ± 0.04

list the clustering performance of each method. The results indicate that our method outperforms other methods.

4.6. Experimental Analysis

We further analyze our model’s results in comparison with the previously mentioned experimental outcomes. Our observations are as follows:

• From Table 2, Table 3, Table 4, Table 5, Table 6 and Table 7, it is evident that nearly all multi-view clustering methods outperform the best single-view method based on SC [3]. This demonstrates the superior effectiveness of multi-view methods in these scenarios. The results further suggest that multi-view data offers richer information, and the different views complement each other.
• Table 2, Table 3, Table 4, Table 5, Table 6 and Table 7 show that our method outperforms state-of-the-art techniques. It captures both local and global structures for a consistent similarity matrix and uses a robust graph, improving resilience to noise and outliers.
• The results in Table 5 also prove that our method can be applied to multi-model datasets and achieve a promising performance.
• The results of Table 6 and Table 7 also prove that our method can be applied to text dataset, and our method outperforms other state-of-the-art methods.
  Figure 6 visualizes the similarity matrices for the MSRC-v1 dataset. The first three matrices are the initial local similarity matrices for views 1, 2, and 3. The last three show matrices learned by our method using global structure, local structure, and both. Each matrix has seven diagonal blocks representing seven classes, with values indicating affinities within clusters. A clearer diagonal block with more nonzero points signifies better clustering performance. Our method shows a clear block diagonal structure, with more affinity points on the diagonal, resulting in superior clustering. The sixth block, highlighted in red, demonstrates the effectiveness of our method compared to those using only global or local structure.
• Figure 7 shows the value of parameter $\alpha$ vs. ACC results on the MSRC-v1 dataset. From Figure 7, we observe that when $\alpha$ is zero or infinite, the clustering performance is not the best. Figure 6 also proves this. We notice from Figure 6 that the diagonal block of the learned similarity matrix from our method is more complete than others. This illustrates that using global structure or local structure separately cannot obtain the desired similarity structure. It further shows that the local manifold structure within each view alone is not enough for clustering tasks.
• Figure 8 shows the convergence curve of our proposed method vs. the number of iterations on four datasets. It demonstrates that the solution of our method is efficient. In particular, we can see that our method converges in fewer than five iterations.

In summary, our proposed method demonstrates superior performance by adaptively fusing cross-view information through adaptive weighting techniques, effectively uncovering intrinsic data correlations to achieve optimal results. From an algorithmic perspective, it enables information sharing across views by constructing both single-view and cross-view graphs. Furthermore, by imposing constraints on the graph structure, it directly obtains clustering results, eliminating performance degradation caused by additional steps to acquire cluster labels. Finally, the algorithm’s convergence is rigorously validated through both theoretical derivation and experimental verification. Notably, the algorithm converges with remarkably few iterations, highlighting its computational efficiency.

4.7. Advantages and Limitations

In this section, we primarily discuss the advantages and disadvantages of the proposed method. Based on the experimental results, our method consistently outperforms other comparison methods, which directly demonstrates its effectiveness. The underlying reason for this performance lies in its ability to enable information sharing across views by constructing both single-view and cross-view graphs. Moreover, by imposing structural constraints on the graphs, the method directly produces clustering results, avoiding the performance degradation typically caused by additional steps required to obtain cluster labels.

Despite its strong performance, the biggest limitation of the proposed method is its high time complexity. This is because it requires the construction of a pairwise graph. Meanwhile, solving the rank-constraint problem on this graph is also computationally expensive. A potential solution to address this issue is to introduce the anchor strategy. By constructing a bipartite graph between the data points and anchors, the computational complexity of the method could be significantly reduced.

5. Conclusions and Future Works

In this paper, we propose a novel graph-based multi-view clustering method named ES-MVC. Unlike traditional multi-view clustering methods, ES-MVC leverages robust initial similarity matrices to mitigate the adverse effects of noisy data. By uniquely integrating both global and local manifold structures, it captures more complementary information. Furthermore, we introduce a dynamic adjustment of the graph structure through a rank constraint, which best reflects the underlying data distribution and facilitates adaptive and optimal neighbor assignments. To efficiently solve the objective function, we have developed an iterative algorithm and proven its convergence. Notably, this algorithm demonstrates exceptionally quick convergence, typically settling within fewer than five iterations on various datasets, which is a clear advantage for handling large-scale datasets. The ultimate significance of our method lies in enabling scalable unsupervised learning for real-world data fusion scenarios, where multi-view data are pervasive but often suffer from noise, incomplete structures, and high computational costs. By effectively combining both local and global information, ES-MVC not only improves clustering accuracy but also offers a practical and robust solution for multi-view data analysis. Extensive experimental results further demonstrate that ES-MVC outperforms existing state-of-the-art multi-view clustering methods.

In future practical applications, data in real-world scenarios are often multi-source and heterogeneous, where the same object can have multi-view or multi-modal representations. Our proposed method is well-suited for such settings, enabling robust data fusion and improved clustering performance. For example, it could be applied in generating pseudo-labels for large-scale model training, enhancing training efficiency by leveraging complementary information from different data sources. Additionally, it could be useful for clustering users in social networks, improving recommendation system accuracy by better capturing the underlying structure of user preferences across various views. We also plan to extend the proposed method to achieve new research breakthroughs on non-specific datasets, particularly in the context of ubiquitous computing and real-time processing. By addressing challenges such as scalability and real-time performance, we aim to enhance the practicality and applicability of our method in real-world settings. Furthermore, we will explore potential optimizations, such as leveraging anchor-based techniques or distributed computing frameworks, to reduce computational complexity and facilitate its deployment in large-scale applications.

While ES-MVC exhibits promising performance, several avenues remain open for future exploration.

First, the current framework assumes linear relationships within each view. Extending it to capture non-linear manifold structures—such as through deep graph learning or kernel-based methods—could further improve clustering accuracy on complex datasets.

Second, we plan to broaden the applicability of our approach to multi-modal and heterogeneous data, adapting it according to diverse data collection methods. Specifically, in real-world applications, ensuring that image data collected from different views meets the clustering requirements, particularly in terms of viewing angles and sizes, is crucial. To address this, we will explore methods such as aligning data from different views at either the feature or sample level to mitigate inter-view heterogeneity. Additionally, we will investigate data augmentation techniques (e.g., rotation, scaling, cropping) to overcome issues related to insufficient data and improve clustering performance. These techniques will be systematically evaluated in future experiments to determine their effectiveness in enhancing the robustness and accuracy of our clustering model.

Third, since our model is designed for clustering based on batch data by analyzing structural characteristics, it is currently unsuitable for real-time processing of individual samples. Batch processing inevitably incurs certain time costs, limiting its use in time-sensitive applications. To overcome this limitation, future work will focus on optimizing the model’s time complexity to better support real-time analysis scenarios.