Community Detection in Multilayer Networks Based on Matrix Factorization and Spectral Embedding Method

Abstract

Community detection remains a challenging research hotspot in network analysis. With the complexity of the network data structures increasing, multilayer networks, in which entities interact through multiple types of connections, prove to be effective in describing complex networks. The layers in a multilayer network may not share a common community structure. In this paper, we propose a joint method based on matrix factorization and spectral embedding to recover the groups not only for the layers but also for nodes. Specifically, the layers are grouped via the matrix factorization method with layer similarity-based regularization in the perspective of a mixture multilayer stochastic block model, and then the node communities within a layer group are revealed by clustering a combination of the spectral embedding derived from the adjacency matrices and the shared approximation matrix. Numerical studies show that the proposed method achieves competitive clustering results as the number of nodes and/or number of layers vary, together with different topologies of network layers. Additionally, we apply the proposed method on two real-world multilayer networks and obtain interesting findings which again highlight the effectiveness of our method.

1. Introduction

Network analysis has emerged in many areas of research and applications, from sociology and ecology to economics [1,2]. For networks, nodes represent the entities of interest, and connecting edges represent the specific relationship that exists among nodes. As the complexity of real datasets increases, various interactions among the elementary components of these systems have emerged. For example, people may contact each other as workmates in a company and as friends on Facebook; different countries may trade in a variety of products. Traditional monolayer networks, which record only one type of interaction between nodes, are less capable to describe the multiple modes of interactions encountered and to understand the topology of the real datasets precisely. Fortunately, several network models have been in existence as various extensions of monolayer networks. One such example are the multilayer networks in which multiple modes of interactions are considered and each specific interaction forms a layer [3]. In this paper, we focus on the multilayer networks with layers sharing a common set of nodes and without inter-layer edges [4].

Community detection is a fundamental issue when analyzing network data. For monolayer network, community detection aims to cluster nodes into communities, and thus there exist more edges connecting nodes in the same community and fewer edges joining nodes across communities [5]. The community detection problem has significant applications due to the fact that entities in a community are likely to share common properties; for example, people in a community may come from the same place and/or they may share other similar interests [6]. Thus, revealing the community structure in a network can provide a better understanding of the overall functioning of the network. There exists a substantial volume of research about community detection techniques on monolayers, such as probabilistic methods [7,8] and criteria-based approaches [9,10]. For a comprehensive review of these works we refer to [11].

The community detection methods originally proposed for monolayer networks can be used directly to analyze multilayer networks in two ways. One way is to aggregate the layers into a graph and then apply traditional community detection methods on the aggregated network [12,13]. The other way is to use the traditional community detection methods on each layer independently and the clustering results are then merged [14,15]. Several other researchers have also proposed community detection methods for multilayer networks, including probability models [16,17,18,19], spectral methods based on various versions of aggregation adjacency matrices or Laplacian matrices [20,21], matrix factorization methods [22] and criteria based methods such as minimizing a squared objective function or maximizing a modularity function [23,24,25]. Most of these methods aim to detect communities that are common across layers and ignore the inhomogeneous nature of network layers. However, for many real multilayer networks, different layers may have different community structures, and some of the layers are interdependent while others are not. For instance, in an international trade multilayer network in which products represent layers and countries are nodes, the trade patterns among countries related to processed foods may be different from those related to unprocessed foods; on the other hand, the trade activities among countries related to different processed foods may be similar. These findings suggest that we should jointly find groups for layers and assign community memberships to nodes for each layer group.

A few recent works have focused on addressing the community detection problem in inhomogeneous multilayer networks. For example, Ref. [26] introduce a multilayer SBM which clusters layers into groups, the so-called strata, and assume that layers in each strata share a common community structure; Ref. [27] introduce a common subspace independent-edge multiple random graph model allowing a shared latent structure across layers, being layer-specific in each layer, and propose a joint spectral embedding of adjacency matrices to simultaneously estimate underlying parameters for each layer; Ref. [28] investigate the so called “Mixture MultiLayer Stochastic Block Model” (MMLSBM) and propose a regularized tensor decomposition to reveal the communities for the layers and nodes. Ref. [29] propose an alternating minimization algorithm(ALMA) that simultaneously recovers the layer partitions and node partitions. Although the methods for finding layer groups in the mentioned works are a bit different, given the layer groups, these works apply classical cluster methods either on the aggregated adjacency matrices or an estimated matrix of that layer group to find the node assignments. More related works we refer to are [30,31,32].

In this work, we are interested in inhomogeneous multilayer networks under the mixture multilayer stochastic block model (MMLSBM) setting. We consider a two-step procedure to detect the layer groups (between-layer clustering) as well as the node communities within a layer group (within-layer clustering). Unlike commonly used methods that describe the multilayer networks using tensor-wise data in the literature, we instead use the vectorization of the model. This technique can reduce the systematic errors due to the improper setting of parameters in the estimation of the tensor approximations. A combination of the Matrix Factorization and Spectral Embedding method (MFSE) is proposed to handle the joint community detection problem. The present paper makes the following contributions. First, under a vectorized perspective, we propose a matrix factorization based method to recover the layer communities. Moreover, since layers that are very structurally similar are more likely to belong to the same group, we integrate layer similarity regularization into the matrix factorization based objective function. Under the MMLSBM setting, each layer can be described as a stochastic block model (SBM) model and two layers have the same connection probability matrix once they belong to the same layer community. From the least squares perspective, the underlying membership matrix C of the layers is optimized by minimizing the objective function ${\parallel \mathbf{A}-CQ\parallel }_F^2+Pen$, where $\mathbf{A}$ is a vectorized version of the adjacency tensor and the l-th row of matrix Q can be treated as the potential representation of the l-th layer community and $Pen$ is the regularization term. Subsequently, we employ the spectral embedding method to recover the node communities in each layer group. For a given layer group, we simultaneously apply the spectral embedding on the adjacency matrices and on the shared structural matrix derived from Q in that group. The final node latent representations are a combination of two types of spectral embeddings. Then, classical clustering methods can be carried out on the joint spectral embedding to search the node assignments. In addition, an efficient singular value decomposition is used to give a recommended value of the number of layer groups as well as the number of node communities in each layer group.

The rest of the paper is organized as follows. The framework is illustrated in Section 2. Section 3 gives the theoretical results of the proposed method. Section 4 provides several experimental evaluations to demonstrate the effectiveness of the proposed method and Section 5 summarizes the paper.

2. Model Framework

2.1. Notations

For a matrix A, let ${\parallel A\parallel }_F$ be the Frobenius norm and denote its transpose by $A^T$. Let $A(i;·)$ and $A(·;j)$ be the i-th row and j-th column of a matrix A, respectively. Denote the Kronecker product of the matrices of matrices A and B by $A\otimes B$. Some more notations used throughout this paper are collected in

Table 1. Notation definitions.

Notations	Explanations
$\left[n\right]$	$\left[n\right]=\{1,2,\cdots ,n\}$
$n;L$	number of nodes, number of network layers
$G^{\left(l\right)};A^{\left(l\right)}\in {\mathbb{R}}^{n\times n}$	the l-th layer; the corresponding adjacency matrix
$\mathcal{A}\in {\mathbb{R}}^{n\times n\times L},\mathcal{A}(·;·;l)=A^{\left(l\right)}$	the adjacency tensor
$M;L_m(\sum _{m=1}^M{L}_m=L)$	number of layer groups; number of layers in the m-th layer group
$K_m(m=1,2,\cdots ,M)$	number of node communities in the m-th layer group
${\mathcal{G}}_1,\cdots ,{\mathcal{G}}_M$	the network groups such that $\left[L\right]={\displaystyle \bigcup _{m=1}^M}{\mathcal{G}}_m$
${\mathcal{V}}_k^{\left(m\right)},m=1,\cdots ,M,k=1,\cdots ,K_m$	the k-th node community in the m-th layer group
$L_{max};L_{min}$	$L_{max}=\underset{m}{max}{L}_m;L_{min}=\underset{m}{min}{L}_m$
$n_{max}^{\left(m\right)};n_{min}^{\left(m\right)}$	$\begin{array}{c}\hfill n_{max}^{\left(m\right)}\end{array}=\underset{k\in \{1,\cdots ,K_m\}}{max}|{\mathcal{V}}_k^{\left(m\right)}|;\begin{array}{c}\hfill n_{min}^{\left(m\right)}\end{array}=\underset{k\in \{1,\cdots ,K_m\}}{min}|{\mathcal{V}}_k^{\left(m\right)}|$
$Z^{\left(m\right)}\in {\{0,1\}}^{n\times K_m}$	the membership (clustering) matrix of the m-th layer group
$B^{\left(m\right)}\in {[0,1]}^{K_m\times K_m}$	link probability matrix of the m-th layer group
$P^{\left(m\right)}=\mathbb{E}\left(A^{\left(m\right)}\right)=\left(P_{ij}^{\left(m\right)}\right)\in {\mathbb{R}}^{n\times n}$	the expected matrix of the m-th layer
$C\in {\{0,1\}}^{L\times M}$	the membership matrix of layers
$a^{\left(m\right)}$	the vectoried version of matrix $A^{\left(m\right)}$ by sequentially stacking its columns
$\begin{array}{c}\hfill b^{\left(m\right)},p^{\left(m\right)}\end{array}$	the vectoried versions of matrices $B^{\left(m\right)},P^{\left(m\right)}$, respectively
$\mathbf{A}\in {\mathbb{R}}^{L\times n^2},\mathbf{P}\in {\mathbb{R}}^{L\times n^2}$	matrices with rows $a^{\left(m\right)}$ and $p^{\left(m\right)}$, respectively
$A(i;·)$, $A(·;j)$	the i-th row and j-th column of a matrix A, respectively
$A(r;i:j)$	the i-to-jth entities in the r-th row of matrix A
$L_Q$	the Laplacian matrix of Q
$D_Q$	a diagonal-matrix with $(i,i)$-th element $D_Q(i,i)={\sum }_{j}Q(i,j)$
$d_{Q_m}$	$d_{Q_m}=\underset{i}{min}{D}_{Q_m}(i,i)$
${\widehat{U}}_Q=[{\widehat{U}}_{Q1},\cdots ,{\widehat{U}}_{Q{K}_m}];{\tilde{U}}_Q$	eigenvector matrix corresponding to $L_Q$; normalized version of ${\widehat{U}}_Q$
$\widehat{A};A^{*}$	an estimator of matrix A; the population version of A
${\lambda }_k\left(W\right)$	the k-th absolute large eigenvalue of matrix W
$\Pi$	an orthonormal rotation whose rank varies from one line to another

.

2.2. Discussion of the Framework

We consider a multilayer network $\mathcal{G}=\{G^{\left(1\right)},\cdots ,G^{\left(L\right)}\}$ with L layers on the same set of n nodes. For the l-th layer, denote its adjacency matrix by $A^{\left(l\right)}\in {\mathbb{R}}^{n\times n}$, where $A_{ij}^{\left(l\right)}=1$ if the nodes i and j are connected and $A^{\left(l\right)}(i,j)=0$ otherwise, further, $A_{ij}^{\left(l\right)}=A_{ji}^{\left(l\right)}$ if the l-th layer is undirected and $A_{ij}^{\left(l\right)}\ne A_{ji}^{\left(l\right)}$ otherwise.

In the MMLSBM setting, $A_{ij}^{\left(l\right)}$ are independent Bernoulli random variables with $P(A_{ij}^{(l)}=1)=P_{ij}^{\left(l\right)}$ and $P_{ii}^{\left(l\right)}=0$. The probability matrices $P^{\left(l\right)}$ give a tensor form $\mathcal{P}\in {\mathbb{R}}^{n\times n\times L}$. In this paper, we assume that the probability connection tensor $\mathcal{P}$ can be partitioned into $M(M<L)$ groups. In particular, there exists a label function $c:\left[L\right]\to \left[M\right]$ and a corresponding clustering matrix $C\in {\{0,1\}}^{L\times M}$ such that

$$\begin{array}{c}\hfill \mathbb{E}[A^{\left(l\right)}|Z]=P^{\left(l\right)}=Z^{\left(m\right)}{B}^{\left(m\right)}{(Z^{\left(m\right)})}^T,m=c\left(l\right),1\le l\le L,1\le m\le M,\end{array}$$

(1)

where $B^{\left(m\right)}\in {[0,1]}^{K_m\times K_m}$ is the connectivity matrix satisfying $B^{\left(m\right)}(i,j)=B^{\left(m\right)}(j,i)$. We aim to recover the network groups ${\mathcal{G}}_1,\cdots ,{\mathcal{G}}_M$ as well as the within-layer community assignments $\{{\mathcal{V}}_k^{\left(m\right)};k=1,\cdots ,K_m,m=1,\cdots ,M\}$.

2.2.1. Between-Layer Cluster

Methods

Adopting the vectorized versions of $P^{\left(l\right)}$ and $B^{\left(l\right)}$, it follows from [33] that

$$\begin{array}{c}\hfill p^{\left(l\right)}=(Z^{\left(c\right(l\left)\right)}\otimes Z^{\left(c\right(l\left)\right)})b^{\left(c\right(l\left)\right)},l=1,\cdots ,L.\end{array}$$

(2)

Denote ${\Theta }^{\left(c\right(l\left)\right)}=Z^{\left(c\right(l\left)\right)}\otimes Z^{\left(c\right(l\left)\right)}$ and recall that C is the membership matrix of the layers, i.e., for $l=1,\cdots ,L;m=1,\cdots ,M$, $C(l,m)=1$ if $c\left(l\right)=m$ and 0 otherwise. We thus rewrite Equation (1) as

$$\begin{array}{c}\hfill \mathbf{P}=\left(\begin{array}{c}{p}^{\left(1\right)}\\ ⋮\\ p^{\left(L\right)}\end{array}\right)=\left(\begin{array}{c}{\Theta }^{\left(c\right(1\left)\right)}{b}^{\left(c\right(1\left)\right)}\\ ⋮\\ {\Theta }^{\left(c\right(L\left)\right)}{b}^{\left(c\right(L\left)\right)}\end{array}\right)=CQ\end{array}$$

(3)

where $Q={({\Theta }^{\left(1\right)}{b}^{\left(1\right)},\cdots ,{\Theta }^{\left(M\right)}{b}^{\left(M\right)})}^T\in {\mathbb{R}}^{M\times n^2}$ can be considered as the shared structure matrix of M layer groups.

Similarly, we introduce the matrix $\mathbf{A}={(a^{\left(1\right)},a^{\left(2\right)},\cdots ,a^{\left(L\right)})}^T\in {\mathbb{R}}^{L\times n^2}$. Note that $\mathbf{P}$ can not be observed directly but $\mathbf{A}$ can be; combining Equations (1)–(3) and from the perspective of the least-squares method, we propose to estimate the potential layer cluster matrix via the following objective function

$$\begin{array}{c}\hfill \underset{C{C}^T=I;Q}{min}{\parallel \mathbf{A}-CQ\parallel }_F^2.\end{array}$$

(4)

In the estimation step, we relax the restrictions on C, i.e., $C(l,m)=1$ if layer l belongs to group m, and impose the non-negativity together with $C{C}^T=I$ on C. For the non-negative feature of the matrices ${\Theta }^{\left(l\right)}$ and $b^{\left(l\right)}$, we can solve the optimization problem of Equation (4) by a nonnegative matrix factorization method.

Intuitively, the layers with high pairwise structural similarity tend to be partitioned into a group. Some effort has been devoted to designing various kinds of indices to measure the similarity between layers [34,35]. In this paper, we use the Pearson linear correlation coefficient $S(l,m)$ to quantify the degree of correlation between two layers l and m [36]. The Pearson correlation coefficient is defined as

$$\begin{array}{cc}\hfill & S(l,m)=\frac{\sum _{ij}(A_{ij}^{(m)}-\overline{A^{(m)}})(A_{ij}^{(l)}-\overline{A^{(l)}})}{\sigma (\overline{A^{(m)}})\sigma (\overline{A^{(l)}})}\hfill \end{array}$$

where $\overline{A^{(·)}}$ represents the mean degree and $\sigma (\overline{A^{(·)}})$ is the standard deviation of the layer $A^{(·)}$. There is no doubt that $S(l,m)\in [-1,1]$. The closer $S(l,m)$ is to 1 ($-1$), the more positively (negatively) correlated the two layers are. Additionally, layers l and m are almost linearly independent if $S(l,m)$ closes to 0.

Above all, a layer similarity based regularization is defined as

$$\begin{array}{c}\hfill {S(l,m)\parallel C(l;·)-C(m;·)\parallel }_F^2\end{array}$$

(5)

where $S(l,m)$ is the Pearson linear correlation and $C(l;·)$ represents the l-th row of matrix C. A simple calculation leads to the following matrix format of Equation (5)

$$\begin{array}{c}\hfill \frac{1}{2}\sum _l\sum _{m}S(l,m){\parallel C(l;·)-C(m;·)\parallel }_F^2=tr\left(C^T{L}_{S}C\right)\end{array}$$

(6)

where $tr(·)$ is the trace function, $S=\left(S\right(l,m\left)\right)$ is the layer similarity matrix and $L_S=D_S-S$ stands for the Laplacian matrix of the layer similarity matrix. $D_S$ is a diagonal matrix representing the degree matrix of S.

By integrating the regularization Equation (6) into Equation (4), we now formalize the between-layer cluster problem as a non-negative matrix factorization problem with layer similarity regularization:

$$\begin{array}{c}\hfill \underset{C{C}^T=I,Q}{min}\mathcal{L}(C,Q)=\underset{C{C}^T=I,Q}{min}{\parallel \mathbf{A}-CQ\parallel }_F^2+\alpha tr\left(C^T{L}_{S}C\right)\end{array}$$

(7)

where $\alpha$ is a trade-off parameter for the regularization term. In this paper, for $l\ne m$, we set the entities $S(l,m)$ which are less than 0 to 0, and choose the parameter $\alpha$ in Equation (7) by the cross validation method.

Updating Rules

Note that $\mathcal{L}(C,Q)$ is not convex with respect to C and Q together; it is thus difficult to find the global minimum of the loss function. To this end, an iterative algorithm is adopted to pursue the solution of Equation (7). In addition, we introduce Lagrange multipliers $\Gamma \in {\mathbb{R}}^{L\times M}$ and $\Xi \in {\mathbb{R}}^{M\times n^2}$ for the non-negative constraints on C and Q, respectively. Then the objective function Equation (7) is rewritten as

$$\begin{array}{c}\hfill \underset{C{C}^T=I,Q}{min}\mathcal{L}(C,Q)=\underset{C{C}^T=I,Q}{min}{\parallel \mathbf{A}-CQ\parallel }_F^2+\alpha tr\left(C^T{L}_{S}C\right)+tr\left(\Gamma C^T\right)+tr\left(\Xi Q^T\right).\end{array}$$

(8)

The partial derivatives of Equation (8) with respect to C and Q are

$$\begin{array}{c}\hfill \frac{\partial \mathcal{L}(C,Q)}{\partial C}=2CQ{Q}^T-2\mathbf{A}{Q}^T+2\alpha L_{S}C+\Gamma .\end{array}$$

(9)

and

$$\begin{array}{c}\hfill \frac{\partial \mathcal{L}(C,Q)}{\partial Q}=2C^{T}CQ-2C^T\mathbf{A}+\Xi .\end{array}$$

(10)

A local minimum of Equation (8) is achieved by respectively setting

$$\begin{array}{c}\hfill \frac{\partial \mathcal{L}(C,Q)}{\partial C}=0\end{array}$$

(11)

and

$$\begin{array}{c}\hfill \frac{\partial \mathcal{L}(C,Q)}{\partial Q}=0.\end{array}$$

(12)

After that, for $l\in \{1,2,\cdots ,L\},m\in \{1,2,\cdots ,M\}$ and $i\in \{1,2,\cdots ,n^2\}$, we multiply by $C(l,m)$ and $Q(m,i)$ on both sides of Equations (11) and (12), respectively. Since the KKT conditions imply that $C(l,m)\Gamma (l,m)=0$ and $Q(m,i)\Xi (m,i)=0$, we obtain the optimal solutions of $C(l,m)$ and $Q(m,i)$ as follows:

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \widehat{C}(l,m)\end{array}\leftarrow \begin{array}{c}\hfill C(l,m)\end{array}\frac{\left(\mathbf{A}{Q}^T\right)\begin{array}{c}\hfill (l,m)\end{array}}{(CQ{Q}^T+\alpha L_{S}C)\begin{array}{c}\hfill (l,m)\end{array}}\end{array}$$

(13)

and

$$\begin{array}{c}\hfill \widehat{Q}\begin{array}{c}\hfill (m,i)\end{array}\leftarrow \begin{array}{c}\hfill Q(m,i)\end{array}\frac{\left(C^T\mathbf{A}\right)\begin{array}{c}\hfill (m,i)\end{array}}{\left(C^{T}CQ\right)\begin{array}{c}\hfill (m,i)\end{array}}.\end{array}$$

(14)

Now, we can recover the communities of network layers by applying K-means algorithm on the rows of $\widehat{C}$. For each m, the estimated $\widehat{Q}(m;·)$ represents the shared structural information of layers in the m-th layer group; we transform $\widehat{Q}$ into M separate matrices. Specifically, for $m=1,\cdots ,M$, we consider matrices ${\widehat{Q}}_m\in {\mathbb{R}}^{n\times n}$ given by

$$\begin{array}{c}\hfill {\widehat{Q}}_m(i;)=\widehat{Q}(m;((i-1)n+1):in).\end{array}$$

(15)

We stress that ${\widehat{Q}}_m(i;·)$ represents the shared behavior of node i in the m-th layer group.

A toy example is further used to describe the procedure of Equation (7). As shown in Figure 1, the simple multilayer network consists of four layers and five nodes. Each row of the matrix $\mathbf{A}\in {\mathbb{R}}^{4\times 25}$ represents the vectorized version of the adjacency matrix related to the layer. Applying the updating rules in Equations (13) and (14), we obtain the estimated layer membership matrix $C\in {\mathbb{R}}^{4\times 2}$ and the estimated shared structure information matrix $Q\in {\mathbb{R}}^{2\times 25}$. After that, we use K-means on $\widehat{C}$ to obtain the layer groups. Then, by Equation (15), we construct two $5\times 5$ matrices ${\widehat{Q}}_1$ and ${\widehat{Q}}_2$ based on two rows of $\widehat{Q}$, respectively. Each constructed matrix is treated as the potential representation of the related layer group.

2.2.2. Within-Layer Community Detection

Given the layer membership matrix $\widehat{C}$ and the shared structure matrix $\widehat{Q}$, the issue that we focus on is how to use the structural information within a layer to detect node assignments. Refs. [28,29] applied a classical clustering method either on the aggregated adjacency matrices or an estimated embedding matrix associated to that layer group. To better describe the consistent topology structural across layers within a layer group, we propose to use an aggregated spectral embedding derived from both the observed adjacency matrices in that layer group and of the shared ${\widehat{Q}}_m$ on the task of clustering nodes within a layer group.

Once the group layers are fixed, we average the adjacency matrices in the m-th group as ${\overline{A}}_m=\frac{{\displaystyle \sum _{l:c\left(l\right)=m}}{A}^{\left(l\right)}}{L_m}$. Then, a standard spectral embedding method is applied on ${\overline{A}}_m$. We list the detailed steps below.

(Sa):

Calculate the graph Laplacian $L_{{\overline{A}}_m}=D_{{\overline{A}}_m}^{1/2}{\overline{A}}_m{D}_{{\overline{A}}_m}^{1/2}.$

(Sb):

Find orthonormal eigenvectors ${\widehat{U}}_{{\overline{A}}_{m}1},{\widehat{U}}_{{\overline{A}}_{m}2},\cdots ,{\widehat{U}}_{{\overline{A}}_m{K}_m}$ corresponding to the $K_m$ eigenvalues that are largest in absolute value of $L_{{\overline{A}}_m}$ and put them in a matrix ${\widehat{U}}_{{\overline{A}}_m}=[{\widehat{U}}_{{\overline{A}}_{m}1},{\widehat{U}}_{{\overline{A}}_{m}2},\cdots ,{\widehat{U}}_{{\overline{A}}_m{K}_m}]\in {\mathbb{R}}^{n\times K_m}$.

(Sc):

Form the normalized version ${\tilde{U}}_{{\overline{A}}_m}$ by normalizing each row of ${\widehat{U}}_{{\overline{A}}_m}$ to have unit length. Then, ${\tilde{U}}_{{\overline{A}}_m}$ is considered to be the spectral embedding derived from ${\overline{A}}_m$.

Following the steps (Sa)–(Sc) of the spectral embedding, we are coming to find the embedding representations of nodes from ${\widehat{Q}}_m$. Denote the graph Laplacian by $L_{{\widehat{Q}}_m}=D_{{\widehat{Q}}_m}^{1/2}{\widehat{Q}}_m{D}_{{\widehat{Q}}_m}^{1/2}$ and the corresponding normalized embedding matrix by ${\tilde{U}}_{{\widehat{Q}}_m}$.

Based on the above, for the m-th layer group, we add the node representations obtained in ${\tilde{U}}_{{\overline{A}}_m}$ with the ${\tilde{U}}_{{\widehat{Q}}_m}$ to obtain extended node representations:

$$\begin{array}{c}\hfill {\widehat{Y}}_m=\frac{{\tilde{U}}_{{\overline{A}}_m}+{\tilde{U}}_{{\widehat{Q}}_m}}{2}.\end{array}$$

(16)

The within-layer community detection is then carried out by applying a K-means algorithm on ${\widehat{Y}}_m$. Up to now, we formalize a two-step community detection method for multilayer networks based on Matrix Factorization and joint Spectral Embedding (MFSE) which is outlined as Algorithm 1.

Remark 1

(Initializations of C and Q). It is known that random initializations may lead the algorithms into non-informative local minimals. The Pearson correlation matrix S illustrates the structural similarity among layers and thus it can provide a glancing estimation of the membership of the layers. Here, we apply K-means on the layer similarity matrix S and take the centers returned by the K-means algorithm as the initialization of C. In our proposed objective function, each row of Q stands for the potential representations of nodes in a specific layer group, following the way of initializing C; after carrying out K-means on $\mathbf{A}$, the initialization of Q is given by the centers returned by the K-means algorithm.

Remark 2

(Pre-determination of M and $K_m(m=1,\cdots ,M)$). Much work has been carried out for determining the number of communities in the community detection problem ([37,38]), either by maximizing specific objective function or searching all candidates. Among these, the eigengap heuristic method proves to be effective in applications ([39,40]). In this paper, we select the number of layer groups M and number of node communities $K_m$ of the m-th layer group by the eigengap heuristic method. For M, we compute the eigendecomposition of the layer similarity matrix S and then rank the eigenvalues ${\lambda }_1\left(S\right)>{\lambda }_2\left(S\right)>\dots$. In practice, there is usually a significant drop between ${\lambda }_M\left(S\right)$ and ${\lambda }_{M+1}\left(S\right)$ and we set the number of layer groups as M.

Once the layer groups are fixed, we first obtain the averaged adjacency matrix ${\overline{A}}_m$ in the m-th layer group and then apply eigen-decomposition on ${\overline{A}}_m$. After that, $K_m$ is chosen as the number of communities if the eigenvalue ${\lambda }_{K_m}\left({\overline{A}}_m\right)$ is relatively larger than ${\lambda }_{K_m+1}\left({\overline{A}}_m\right)$, i.e., the gap between ${\lambda }_{K_m}\left({\overline{A}}_m\right)$ and ${\lambda }_{K_m+1}\left({\overline{A}}_m\right)$ is much greater than those of other adjacent eigenvalues.

Algorithm 1 MFSE Algorithm

Input: The adjacency matrices $A^{\left(l\right)},l=1,2,\cdots ,L$; number of layer groups M; number of the node groups ${\left\{K_m\right\}}_{m=1}^M$; tuning parameter $\alpha$. Output: The network groups $\{{\mathcal{G}}_1,\cdots ,{\mathcal{G}}_M\}$ and the within-layer node communities $\{{\mathcal{V}}_k^{\left(m\right)};m=1,\cdots ,M,k=1,\cdots ,K_m\}$. Algorithm: 1.  Initialize C and Q; 2.  Iterate until convergence. (a2).  Update $\widehat{C}$ according to Equation (13); (b2).  Update $\widehat{Q}$ according to Equation (14); 3.  Applying K-means on the rows of $\widehat{C}$ to get the layer groups ${\mathcal{G}}_1,\cdots ,{\mathcal{G}}_M$. 4.  For each layer group ${\mathcal{G}}_m$. (a4).  Applying the spectral embedding method, i.e., steps (Sa)–(Sc) on ${\overline{A}}_m$ to obtain the latent node representation matrix ${\tilde{U}}_{{\overline{A}}_m}$; (b4).  Similar to (a4), estimate the node representation matrix ${\tilde{U}}_{{\widehat{Q}}_m}$ based on ${\widehat{Q}}_m$; (c4).  The layer groups $\{{\mathcal{V}}_k^{\left(m\right)}\}$ are obtained by applying K-means on the rows of the final node embedding ${\widehat{Y}}_m$ in Equation (16).

3. Theoretical Guarantees

Some additional assumptions are required to analyze the convergence issues. We collect them as follows:

(A1)

$d_{Q_m}>3log\frac{8n}{ϵ}$ for $ϵ>0$;

(A2)

$24d_{Q_m}^{1/2}{\lambda }_{K_m}\ge {\left(3log\frac{8n}{ϵ}\right)}^{1/2}$ with ${\lambda }_{K_m}$ representing the $K_m$-th largest eigenvalue of $L_{Q_m}{L}_{Q_m}$;

(A3)

cluster of layers and local communities balance condition: there exist positive constants $\underline{\gamma }<\overline{\gamma }$ such that

$$\underline{\gamma }M<\frac{L}{L_{max}}\le \frac{L}{L_{min}}<\overline{\gamma }M,\underline{\gamma }{K}_m<\frac{n}{n_{max}^{\left(m\right)}}\le \frac{n}{n_{min}^{\left(m\right)}}<\overline{\gamma }{K}_m.$$

Note that Assumption (A1) restricts the sparsity of the network, Assumption (A2) requires that the $K_m$-th eigenvalue of $L_{Q_m}{L}_{Q_m}$ is not too small and Assumption (A3) is the so-called balanced network condition and is very common in the problem of community detection ([41]).

We need to obtain the estimators $\widehat{C}$ and $\widehat{Q}$ of $C^{*}\in {\mathbb{R}}^{L\times M}$ and $Q^{*}\in {\mathbb{R}}^{M\times n^2}$ based on the observed $\left\{A^{\left(l\right)}\right\}$. First of all, we give a theoretical result on the consistency of the estimators from the objective function $\mathcal{L}(C,Q)$. We denote the estimators in the iteration by ${\{{\widehat{\Delta }}^{\left[t\right]}=({\widehat{C}}^{\left[t\right]},{\widehat{Q}}^{\left[t\right]})\}}_{t=1}^{\infty }$.

Theorem 1.

${\{{\widehat{\Delta }}^{\left[t\right]}\}}_{t=1}^{\infty }$ converges to a critical point of $\mathcal{L}(C,Q)$.

The proof of Theorem 1 is given in Appendix A.

We denote by $\pi \left(m\right)$ the permutation function of $\left[m\right]$. Given the true partition of the network layers $\left[L\right]={\displaystyle \bigcup _{m=1}^M}{\mathcal{G}}_m$ and the estimated partition $\left[L\right]={\displaystyle \bigcup _{m=1}^M}{\widehat{\mathcal{G}}}_m$, the misclassfication error rate of between-layer clustering can be defined as

$$\begin{array}{c}\hfill {Err}_{BL}=\underset{\tau \in \pi \left(m\right)}{min}\sum _{m=1}^M|{\mathcal{G}}_m\setminus {\widehat{\mathcal{G}}}_{\tau \left(m\right)}|/L.\end{array}$$

(17)

Theorem 2.

Suppose that Assumption (A3) holds, there exists a sufficiently small function $\epsilon (n,L)$ depending on n and L such that the between-layer clustering error satisfies ${Err}_{BL}\le \frac{{\zeta }_{\epsilon }\epsilon (n,L)\underset{m}{max}{L}_m}{4L}$, where ${\zeta }_{\epsilon }$ is a positive constant depending on ε.

The proof of Theorem 2 is given in Appendix B.

For the m-th layer group, the cluster centroids sought by K-means algorithm are defined as $\mathcal{O}=\{O_m\in {\mathbb{R}}^{n\times d}:O_{m}has{K}_m\mathrm{unique}\mathrm{rows}\}$, satisfying ${\widehat{O}}_m=\underset{{\widehat{X}}_m\in \mathcal{O}}{arg\; min}{\parallel {\widehat{Y}}_m-O_m\parallel }_F$. Let

$$\begin{array}{c}\hfill {\Gamma }_m=\left\{i:\parallel {\widehat{O}}_m(i;·)-Y_m^{*}{(i;·)\Pi \parallel }_2>{\parallel {\widehat{O}}_m(j;·)-Y_m^{*}(j;·)\Pi \parallel }_2\right\}\end{array}$$

(18)

be the set of misclustered nodes, where $Y_m^{*}(i;·)$ represents the parameter centroid corresponding to the i-th node while ${\widehat{O}}_m(i;·)$ is the estimated centroid, and $\Pi$ is an orthonormal rotation.

Theorem 3.

Assume that the smallest eigenvalue of $E{\overline{A}}_m$ is larger than ${\alpha }_n$ and Assumptions (A1)- (A2) hold. There exist constants $C\left(ϵ\right)>0,C_0>0,0<\xi \le 1$ and for any $ϵ>0$, with probability at least $1-ϵ$; the proportion of misclustered nodes in the m-th layer group satisfies $\frac{|{\Gamma }_m|}{n}\le \frac{\xi }{2}\left(\frac{144C_0{K}_{m}log(8n/ϵ)}{n{\lambda }_{K_m}^2}+\frac{C\left(ϵ\right)K_{m}logn}{n{\alpha }_n^2}\right)$.

The proof of Theorem 3 is given in Appendix C.

4. Experiments

In this section, we assess the performance of the proposed MFSE method through simulated and real-world multi-layer networks. For synthetic multilayer networks, we compare our method with the TWIST method ([28]) and the ALMA method ([29]). Specifically, TWIST focuses on the MMSBM model and applies Tucker decomposition on the observed adjacency tensor to obtain its low-rank approximations. TWIST finds both the global and the local memberships of nodes, together with the memberships of layers. ALMA is an alternating minimization algorithm which aims at revealing the layer partition as well as the node partitions. We remark that both MFSE and ALMA aim at recovering the community assignment of layers together with the nodes. To make the comparison fair, all three competitive methods are only dedicated to detecting the layer partition and the node partition which are also named as “between-layer clustering” and “within-layer clustering”, respectively. For both synthetic multilayer networks and real-word networks, the between-layer clustering error is assessed by ${Err}_{BL}$(Equation (17)) and the proportion of misclassified nodes within layers (the within-layer clustering error) is evaluated by $\frac{{\displaystyle \sum _{m=1}^M}{\Gamma }_m}{Mn}$ (Equation (18)).

4.1. Simulation Study

In this part, we test our method on synthetic multilayer networks. We carry out three simulations to evaluate the performance of the proposed method by varying the number of layers L, the number of nodes n and network sparsity parameter $\gamma$, respectively. The clustering results of all methods in three simulations averaged over 500 replications. Below, we elaborate on each of the simulation scenarios.

The purpose of the first simulation is to examine the ability of the proposed method to discover both layer and node clusters as the number of nodes n varies. Imagine a MMLSBM with $L=60$ layers, $M=3$ layer groups and $K_1=K_2=K=2$ node communities for each layer group. The underlying community assignments of layers and nodes are generated by the multinomial distributions with equal class probabilities $1/M$ for the layers, and $1/K$ for the nodes within a layer group. For completeness, we take into account both assortative and disassortative SBM models ([42]) in the tests. A SBM model is assortative if $B(i,j)<B(i,i)$ and disassortative if $B(i,j)>B(i,i)$ for $i\ne j$, with $B(i,j)$ representing the connection probability ([41]). We set the connection matrices for three layer groups as

$$\begin{array}{c}\hfill {\mathbf{B}}_m=\left(\begin{array}{ccc}{p}_m& {\gamma }_m{p}_m& {\gamma }_m{p}_m\\ {\gamma }_m{p}_m& p_m& {\gamma }_m{p}_m\\ {\gamma }_m{p}_m& {\gamma }_m{p}_m& p_m\end{array}\right),m=1,2,3\end{array}$$

(19)

where $p=(p_1,p_2,p_3)=(0.4,0.4,0.5)$ and $\gamma =({\gamma }_1,{\gamma }_2,{\gamma }_3)=(1.2,0.8,0.6)$. Notice that values of ${\gamma }_m$ govern the strength of the assortative (disassortative) feature of the network. Roughly speaking, the network is assortative if ${\gamma }_m<1$ and disassortative if ${\gamma }_m>1$, and the community structure becomes obscure as ${\gamma }_m$ closes to 1. We increase the number of nodes n from 40 to 400 and record the between-layer clustering error results and within-layer clustering error results of three methods in Figure 2a,b. As expected, both types of errors rates for all three methods decrease substantially as n becomes large, implying the effect of number of nodes on clustering errors. As $n\ge 80$, the plots demonstrate that MFSE always achieves superior or comparative performances in both between-layer and within-layer clustering.

In the second simulation, we evaluate the performances of three methods as the number of layers varies. The generated MMLSBM is similar to that in the first simulation except $n=100$ is fixed. We set the number of layers L to increase from 40 to 200. The clustering errors are reported in Figure 2c,d. For all methods, both types of errors decrease as L increases, which is consistent with the theoretical results of these methods. Specifically, for between-layer clustering, MFSE outperforms TWIST and ALMA once $L\ge 80$, and MFSE continues to obtain superior results for the within-layer clustering.

The last simulation studies the impact of parameter $\gamma$ which controls the ratio of the connection probabilities of the inter-groups of nodes versus intra-groups of nodes. The settings are the same as previous simulation scenarios except $n=100$ and $L=120$ are fixed. We set $\gamma =\{{\gamma }_1,{\gamma }_1,{\gamma }_1\}$, and the performances of all methods on two types of clustering with ${\gamma }_1$ increasing from 0.1 to 1.9 are investigated. The results of this comparison are presented in Figure 2e,f. Notice that the smaller the ${\gamma }_1$, the more pronounced the within-group and between-group distinction becomes. Additionally, the larger ${\gamma }_1$(>1) indicates the stronger disassortative feature of the network. Further, the within-layer clustering becomes harder when ${\gamma }_1$ closes to 1, which also makes the between-layer clustering difficult. The performances of the three methods in Figure 2e,f agree with those findings.

In summary, for between-layer clustering, the proposed method MFSE achieves better results in all parameter settings and ALMA almost gives the second best results. For within-layer clustering, MFSE still gives superior or comparable performances in different settings. We claim that TWIST works notably even in the cases where the between-layer errors are higher than those of ALAM. This could be due to the fact that TWIST recovers the node partitions by clustering the adjacency matrices in that layer group, while ALAM clusters the low-rank approximation of the adjacency tensor in that layer group which may leave out some direct structure information. We use both direct adjacency matrices and the low-rank approximations to reveal the node memberships within a layer group, which makes our method more effective in various model settings.

4.2. Real-World Network Data

In this section, we demonstrate the merits of MFSE in community detection on two real-world multilayer networks.

4.2.1. FAO data

The data are collected from the Worldwide Food Trading Networks data that are available at https://www.fao.org/faostat/en/#data/TM. The data was collected since 1986 and the last update was given in 23 December 2022. In our analysis, the data was accessed on 1 January 2010, before 31 December 2010. The dataset contains export/import trading among 245 countries/regions for more than 300 products. In the multilayer network, layers represent different types of commodities and nodes represent countries. Two countries have a link if there exist import/export relationships of a product between them. We convert the original directed and weighted networks to undirected and unweighted ones by ignoring the directions and weights. We first remove the layers with links less than 1135 (the third quartile) and retain the layers with average degree greater than 10 (the second quartile). Finally, we choose the countries that are active in trading at least 90% of the selected products, reducing a multilayer network consisting of 46 layers and 100 nodes.

Subsequently, we use Algorithm 1 to recover the partitions of layers (products) and nodes (countries) of the multilayer network. By applying the eigengap heuristic method, we choose $M=2$ and record the between-layer clustering results in

Table 2. Between-layer clustering results obtained by MFSE.

Layer Groups	Products
Group 1	Cheese_whole_cow_milk, Bread, Chocolate_products_nes, Fatty_acids, Anise_badian_fennel_coriander, Beer_of_barley, Chillies_and_peppers_dry, Beverages_distilled_alcoholic, Coffee_extracts, Coffee_roasted, Fruit_dried_nes, Juice_fruit
	_nes, Pepper_(piper_spp.), Sugar_nes, Tea_mate_extracts, Vegetables_frozen
Group 2	Beverages_non_alcoholic, Macaroni, Food_prep_nes, Cigarettes, Beans_dry, Flour_wheat, Crude_materials, Food_preparations_flour_malt_extract, Food_wastes,
	Fruit_prepared_nes, Cereals_breakfast, Coffee_green, Cocoa_powder_and_cake, Pastry, Pet_food, Tea, Nuts_prepared_(exc._groundnuts), Mixes_and_doughs, Oil_vegetable_origin_nes, Oil_essential_nes, Rice_milled, Sugar_refined, Sugar_confectionery, Wine, Spices_nes, Vegetables_dehydrated, Vegetables_preserved_nes, Waters ice_etc, Vegetables_in_vinegar, Tobacco_unmanufactured

.

From Table 2, we see that the first layer group consists mostly of solid and liquid beverage products as well as processed food products with a certain shelf life, and the second group contains mostly cereals and raw materials for food products, together with processed food products with a long shelf life.

Applying the eigengap heuristic method again, we choose $K_1=K_2=5$, which is consistent with the number of continents (Africa, Americas, Asia, Europe, and Oceania) and can be considered as a natural partition of countries. Figure 3 shows the node communities recovered by MFSE. We have the following comments: (1) for both layer groups, there exist close trading activities among some developed and developing countries in Asia and Oceania, for example, China, Japan, Australia, New Zealand, etc, see Community 1 in Figure 3a,b, respectively; (2) for the products in group 1, since North America and Europe are not far apart by ocean transportation, countries in these two regions have more continual demands for trade than the countries in the Americas which have a longer shipping distance, please refer to Communities 2 and 3 in Figure 3a; (3) for the products in group 2, the considerable shelf life of some products allows for close trade among countries with long marine transportation distance, such as among countries in North America and South America (Community 3 in Figure 3b); (4) the countries in Southeast Europe (Community 4 in Figure 3a,b, respectively) have their independent trade patterns regardless of the types of products, probably due to the Free Trade Agreement which was carried out in 2006.

4.2.2. AU-CS Network

The network is constructed from a dataset that describes five online and offline relationships including Facebook, leisure, work, coauthor and lunch among 61 employees of a university research department ([43]). The employees are represented by nodes and one specific relationship is represented by one layer, together with links standing for an existing relation between nodes. Thus, a five-layer multilayer network with 620 edges is formed. As a pre-processing step, we computed the eigenvalues of the graph Laplacian based on the layer-similarity matrix and chose $M=2$ by eigengap heuristic. After that, by using the eigengap heuristic method again, we chose $K_1=K_2=3$ as the numbers of node communities in two layer groups, respectively. We apply Algorithm 1 on the data and list the between-layer clustering result in

Table 3. Between-layer clustering results obtained by MFSE on AU-CS network.

Layer Groups	Layers
Group 1	Coauthor, Leisure, Facebook
Group 2	Lunch, Work

. The intuition of these layer partitions is that the employees have a natural co-occurrence of work and lunch interactions as colleagues; for example, they may work together and have lunch together in the canteen. Simultaneously, it is natural to group together the online/offline relationships based on personal/research interests.

Then, we investigate the within-layer clustering results. Unlike the FAO data, we may not explain intuitively the node partitions and have no idea about the groundtruth communities. For completeness, we use the density metric to evaluate the quality of the clustering method. The density at the l-th network layer is defined as follows:

$$density=\sum _{k=1}^{K_l}\frac{|\{(v_i^{\left(l\right)},v_j^{\left(l\right)})|v_i^{\left(l\right)},v_j^{\left(l\right)}\in C_k^{\left(l\right)},(v_i^{\left(l\right)},v_j^{\left(l\right)})\in E^{\left(l\right)}\}|}{|E^{\left(l\right)}|}$$

where, for the l-th layer, $v_i^{\left(l\right)},C_k^{\left(l\right)}$ and $E^{\left(l\right)}$ represent the i-th node, the k-th node community and the edge set, respectively. Additionally, to make the results comparable, we introduce a standard Spectral Clustering based method (SPC-Mean) in the literature which uses the Mean adjacency matrix $\overline{A}=\frac{{\displaystyle \sum _{l=1}^L}{A}^{\left(l\right)}}{L}$ to recover the node communities. We record the densities of each layer returned by the proposed MFSE method and SPC-Mean method in

Table 4. Densities obtained by MFSE and SPC-Mean on each layer of AU-CS network.

Layers	MFSE	SPC-Mean
Coauthor	0.95	0.76
Facebook	0.76	0.70
Leisure	0.83	0.76
Lunch	0.72	0.62
Work	0.77	0.71

. We further plot the adjacency matrix of each layer returned by MFSE (Figure 4) where nodes are arranged according to the community labels. Table 4 shows that MFSE gives highly competitive densities for all layers on the AU-CS network and Figure 4 implies that the connection patterns of the first three layers are significantly different from those of the remaining two layers.

5. Conclusions

In this paper, we propose a matrix factorization and spectral embedding based community detection method for multilayer networks. The proposed method recovers the layer groups by clustering the low-rank approximate of a vectorized version matrix derived by the adjacency matrices. After that, the node communities within a layer group can be recovered by both the observed adjacency matrices and a shared structural matrix in that layer group. We give the theoretical guarantees of the method and the simulation results confirm the fact that the proposed method performs efficiently under different model settings. Notice that the layers of networks may have various degrees of sparsity in many real applications; it is thus of interest to extent our method to more sparse scenarios in future studies. Another direction worth exploring is where one can generalize the matrix factorization-based representation incorporating node attributes to extract communities in multilayer networks.