Studying Asymmetric Structure in Directed Networks by Overlapping and Non-Overlapping Models

Abstract

We consider the problem of modeling and estimating communities in directed networks. Models to this problem in the previous literature always assume that the sending clusters and the receiving clusters have non-overlapping property or overlapping property simultaneously. However, previous models cannot model the directed network in which nodes in sending clusters have overlapping property, while nodes in receiving clusters have non-overlapping property, especially for the case when the number of sending clusters is no larger than that of the receiving clusters. This kind of directed network exists in the real world for its randomness, and by the fact that we have little prior knowledge of the community structure for some real-world directed networks. To study the asymmetric structure for such directed networks, we propose a flexible and identifiable Overlapping and Non-overlapping model (ONM). We also provide one model as an extension of ONM to model the directed network, with a variation in node degree. Two spectral clustering algorithms are designed to fit the models. We establish a theoretical guarantee on the estimation consistency for the algorithms under the proposed models. A small scale computer-generated directed networks are designed and conducted to support our theoretical results. Four real-world directed networks are used to illustrate the algorithms, and the results reveal the existence of highly mixed nodes and the asymmetric structure for these networks.

1. Introduction

Community detection is a powerful tool in studying social networks with a latent structure of community [1,2,3,4]. The goal of community detection is to estimate a node’s community information from the network. In the study of social networks, various models have been proposed for community detection to model different networks with different community structures [5]. Due to the extremely intensive studies on community detection, we only focus on identifiable models that are closely relevant to our study in this paper.

The Stochastic Blockmodel (SBM) [6] is a classical and widely used model for an undirected network. SBM assumes that the probability of an edge between two nodes only depends on the clusters they belong to, and this assumption is not realistic because nodes have various degrees in real-world networks. To model real-world un-directed networks in which nodes degrees vary, the Degree-Corrected Stochastic Blockmodel (DCSBM) [7] extends SBM by introducing degree heterogeneities. Under SBM and DCSBM, all nodes are pure, such that each node only belongs to one community. However, in real cases, some nodes may belong to multiple communities, and such nodes have overlapping (also known as mixed membership) property. To model undirected networks in which nodes have an overlapping property, Ref. [8] designs the Mixed Membership Stochastic Blockmodel (MMSB). Ref. [9] introduces the Degree-Corrected Mixed Membership model (DCMM), which extends MMSB by considering degree heterogeneities. Ref. [10] designs the Overlapping Continuous Community Assignment model (OCCAM), which equals DCMM actually. Spectral methods with consistent estimations under the above models are provided in [9,11,12,13,14,15,16,17].

For directed networks in which all nodes have a non-overlapping property, Ref. [18] proposes a model called Stochastic co-Blockmodel, (ScBM) and its extension, the Degree-Corrected Stochastic co-Blockmodel (DCScBM), by considering the degree heterogeneity, where ScBM (DCScBM) is an extension of SBM (DCSBM) from an un-directed network to a directed network. ScBM and DCScBM can model non-overlapping directed networks in which row nodes belong to $K_r$ sending clusters (we also use community to denote cluster occasionally) and column nodes belong to $K_c$ receiving clusters, where row nodes can differ from column nodes, and $K_r$ can differ from $K_c$. Ref. [19] studies the consistency of some adjacency-based spectral algorithms under ScBM. Ref. [20] studies the consistency of the spectral method D-SCORE under DCScBM when $K_r=K_c$. Ref. [21] designs the Directed Mixed Membership Stochastic Blockmodel (DiMMSB) as an extension of ScBM and MMSB to model directed networks in which all nodes have overlapping property. Meanwhile, DiMMSB can also be seen as an extension of the two-way blockmodels with a Bernoulli distribution of [22]. All of the above models are identifiable under certain conditions. The identifiability of ScBM and DCScBM holds even for the case when $K_r\ne K_c$. DiMMSB is identifiable only when $K_r=K_c$. Sure, SBM, DCSBM, MMSB, DCMM, and OCCAM are identifiable when $K_r=K_c$, since they model undirected networks. For all the above models, row nodes and column nodes have symmetric structural information such that they always have non-overlapping property or overlapping property simultaneously. As shown by the identifiability of DiMMSB, to model a directed network in which all nodes have overlapping property, the identifiability of the model requires $K_r=K_c$. Naturally, there is a bridge model from ScBM to DiMMSB such that the bride model can model a directed network in which the row nodes and column nodes have asymmetric structural information such that they have different overlapping property. In this paper, we introduce this model and name it the Overlapping and Non-overlapping model.

Our contributions in this paper are as follows. We propose an identifiable model for directed networks, the Overlapping and Non-overlapping model (ONM for short). ONM allows that nodes in a directed network can have different overlapping properties. Without a loss of generality, in a directed network, we let the row nodes have overlapping property while the column nodes do not. The proposed model is identifiable when $K_r\le K_c$. Recall that the identifiability of ScBM modeling non-overlapping directed networks holds even for the case $K_r\ne K_c$, and that DiMMSB modeling overlapping directed networks is identifiable only when $K_r=K_c$, this is the reason for why we call ONM modeling directed networks, in which row nodes have different overlapping properties to column nodes, as a bridge model from ScBM to DiMMSB. We also propose an identifiable model, Overlapping and Degree-Corrected Non-overlapping model (ODCNM), as an extension of ONM, by considering the degree heterogeneity. We construct two spectral algorithms to fit ONM and ODCNM. We show that our methods enjoy consistent estimations under mild conditions. Especially, our theoretical results under ODCNM match those under ONM when ODCNM reduces to ONM. The numerical results of simulated directed networks generated under ONM and ODCNM support our theoretical findings, and the results on four real-world directed networks demonstrate the advantages of our algorithms in studying the asymmetric structure between the sending and receiving clusters.

Notations. We take the following general notations in this paper. For any positive integer m, let $[m]:=\{1,2,\dots ,m\}$, and let $I_m$ denote the $m\times m$ identity matrix. For a vector x and fixed $q>0$, ${\parallel x\parallel }_q$ denotes its $l_q$-norm. For a matrix M, $M^{\prime }$ denotes the transpose of the matrix M, $\parallel M\parallel$ denotes the spectral norm, ${\parallel M\parallel }_F$ denotes the Frobenius norm, and ${\parallel M\parallel }_{2\to \infty }$ denotes the maximum $l_2$-norm of all the rows of M. Let ${\sigma }_i(M)$ be the i-th largest singular value of matrix M, and let ${\lambda }_i(M)$ denote the i-th largest eigenvalue of the matrix M ordered by the magnitude. $M(i,:)$ and $M(:,j)$ denote the i-th row and the j-th column of matrix M, respectively. $M(S_r,:)$ and $M(:,S_c)$ denote the rows and columns in the index sets $S_r$ and $S_c$ of matrix M, respectively. For any matrix M, we simply use $Y=\max (0,M)$ to represent $Y_{ij}=\max (0,M_{ij})$ for any $i,j$. For any matrix $M\in {\mathbb{R}}^{m\times m}$, let $\mathrm{diag}(M)$ be the $m\times m$ diagonal matrix whose i-th diagonal entry is $M(i,i)$, and let $\mathrm{rank}(M)$ be M’s rank. $\mathbf{1}$ is a column vector with all entries being the value 1. $e_i$ is a column vector whose i-th entry is 1, while other entries are zero. In this paper, C is a positive constant which may occasionally be different.

2. The Overlapping and Non-Overlapping Model

Consider a directed network $\mathcal{N}=(V_r,V_c,E)$, where $V_r=\{1,2,\dots ,n_r\}$ is the set of row nodes, $V_c=\{1,2,\dots ,n_c\}$ is the set of column nodes, and E is the set of edges from the row nodes to the column nodes. Note that since the row nodes can be different from the column nodes, we may have $V_r\cap V_c=⌀$ (i.e., there are no common nodes between $V_r$ and $V_c$), and $V_r$ may not be equal to $V_c$ (i.e., the row nodes are different from the column nodes), which is a more general case than $V_r=V_c$ (i.e., all row nodes are same as column nodes), where ⌀ denotes the null set, and such a directed network $\mathcal{N}$ is also known as a bipartite graph (or bipartite network) in [18,19]. In this paper, we use the subscript r and c to distinguish the terms for the row nodes and column nodes, where works in [18,19,23,24,25,26] also consider the general bipartite setting, such that the row nodes may differ from the column nodes. Let $A\in {\{0,1\}}^{n_r\times n_c}$ be the bi-adjacency matrix of directed network $\mathcal{N}$, such that $A(i_r,i_c)=1$ if there is a directional edge from row node $i_r$ to column node $i_c$, and $A(i_r,i_c)=0$ otherwise. For convenience, we call the community that the row nodes belong to as the row community (or sending cluster occasionally), and the community that the column nodes belong to as the column community (or receiving cluster occasionally).

We propose a new blockmodel which we call the Overlapping and Non-overlapping model (ONM for short). ONM can model directed networks whose row nodes belong to $K_r$ overlapping row communities, while the column nodes belong to $K_c$ non-overlapping column communities. For row nodes, let ${\Pi }_r\in {\mathbb{R}}^{n_r\times K_r}$ be the membership matrix, such that

$$\begin{array}{c}\hfill {\Pi }_r(i_r,)\ge 0,{\parallel {\Pi }_r(i_r,:)\parallel }_1=1\mathrm{for}{i}_r\in [n_r].\end{array}$$

(1)

Call row node $i_r$ pure if ${\Pi }_r(i_r,:)$ degenerates (i.e., one entry is 1, all others $K_r-1$ entries are 0), and mixed otherwise. From such a definition, row node $i_r$ has mixed membership and may belong to more than one row communities for $i_r\in [n_r]$.

For column nodes, let ℓ be the $n_c\times 1$ vector whose $i_c$-th entry $\ell (i_c)=k$ if column node $i_c$ belongs to the k-th column community, and $\ell (i_c)$ takes value from $\{1,2,\dots ,K_c\}$ for $i_c\in [n_c]$. Let ${\Pi }_c\in {\mathbb{R}}^{n_c\times K_c}$ be the membership matrix of column nodes, such that for $i_c\in [n_c],k\in [K_c]$,

$$\begin{array}{c}\hfill {\Pi }_c(i_c,k)=1\mathrm{when}\ell (i_c)=k,\mathrm{and}0\mathrm{otherwise},\mathrm{and}{\parallel {\Pi }_c(i_c,:)\parallel }_1=1.\end{array}$$

(2)

From such a definition, column node $i_c$ belongs to exactly one of the $K_c$ column communities for $i_c\in [n_c]$. Sure, all of the column nodes are pure nodes.

In this paper, we assume that

$$\begin{array}{c}\hfill K_r\le K_c.\end{array}$$

(3)

Equation (3) is required for the identifiability of ONM. Let $P\in {\mathbb{R}}^{K_r\times K_c}$ be the probability matrix, such that

$$\begin{array}{c}\hfill 0\le P(k,l)\le \rho \le 1\mathrm{for}k\in [K_r],l\in [K_c],\end{array}$$

(4)

where $\rho$ controls the network sparsity and is called the sparsity parameter in this paper. For convenience, set $P=\rho \tilde{P}$, where $\tilde{P}(k,l)\in [0,1]$ for $k\in [K_r],l\in [K_c]$, and ${\max }_{k\in [K_r],l\in [K_c]}\tilde{P}(k,l)=1$ for model identifiability. For all pairs of $(i_r,i_c)$ with $i_r\in [n_r],i_c\in [n_c]$, our model assumes that $A(i_r,i_c)$ are independent Bernoulli random variables satisfying

$$\begin{array}{c}\hfill \Omega :={\Pi }_{r}P{\Pi }_c^{\prime },A(i_r,i_c)\sim \mathrm{Bernoulli}(\Omega (i_r,i_c)),\end{array}$$

(5)

where $\Omega =\mathbb{E}[A]$, and we call it the population adjacency matrix in this paper.

Definition 1.

Call model (1)–(5) the Overlapping and Non-overlapping model (ONM), and denote it with $ON{M}_{n_r,n_c}(K_r,K_c,P,{\Pi }_r,{\Pi }_c)$.

Remark 1.

Under $ON{M}_{n_r,n_c}(K_r,K_c,P,{\Pi }_r,{\Pi }_c)$, for $i_r\in [n_r],j_c\in [n_c]$, since $\mathbb{P}(A(i_r,j_c)=1)=\Omega (i_r,j_c)=\rho \Pi (i_r,:)\tilde{P}{\Pi }_c^{\prime }(j_c,:)$, we see that increasing ρ increases the probability to generate an edge from row node $i_r$ to column node $j_c$, i.e., the sparsity of the network is governed by ρ.

The following conditions are sufficient for the identifiability of ONM:

• (I1) $\mathrm{rank}(P)=K_r,\mathrm{rank}({\Pi }_r)=K_r$, and $\mathrm{rank}({\Pi }_c)=K_c$.
• (I2) There is at least one pure row node for each of the $K_r$ row communities.

For $k\in [K_r]$, let ${\mathcal{I}}_r^{(k)}=\{i\in [n_r]\}:{\Pi }_r(i,k)=1\}$. By condition (I2), $I_r^{(k)}$ is non-empty for all $k\in [K_r]$. For $k\in [K_r]$, select one row node from ${\mathcal{I}}_r^{(k)}$ to construct the index set ${\mathcal{I}}_r$; i.e., ${\mathcal{I}}_r$ is the indices of row nodes corresponding to $K_r$ pure row nodes, one from each row community. Without loss of generality, let ${\Pi }_r({\mathcal{I}}_r,:)=I_{K_r}$ (Lemma 2.1 [17] also has a similar setting to design their spectral algorithm under MMSB). ${\mathcal{I}}_c$ is defined similarly for the column nodes, such that ${\Pi }_c({\mathcal{I}}_c,:)=I_{K_c}$. The next proposition guarantees the identifiability of ONM.

Proposition 1.

If conditions (I1) and (I2) hold, ONM is identifiable: For eligible $(P,{\Pi }_r,{\Pi }_c)$ and $(\check{P},{\check{\Pi }}_r,{\check{\Pi }}_c)$, if ${\Pi }_{r}P{\Pi }_c^{\prime }={\check{\Pi }}_r\check{P}{\check{\Pi }}_c^{\prime }$, then $P=\check{P},{\Pi }_r={\check{\Pi }}_r$, and ${\Pi }_c={\check{\Pi }}_c$.

All proofs of propositions, lemmas, and theorems are provided in Appendix B and Appendix C of this paper. Compared to some previous models, ONM models different directed networks.

• When the row nodes are the same as the column nodes, $K_r=K_c$, and all nodes are pure, ONM degenerates to SBM. However, ONM can model directed networks where row nodes enjoy mixed memberships, while SBM only models un-directed networks.
• When all row nodes are pure, our ONM reduces to ScBM with $K_r$ row clusters and $K_c$ column clusters [18]. However, ONM allows for row nodes to have overlapping memberships, while ScBM does not. Meanwhile, for model identifiability, ScBM does not require $\mathrm{rank}(P)=K_r$ that ONM requires, and this can be seen as the cost of ONM when modeling the overlapping row nodes.
• Though DiMMSB [21] can model directed networks whose row and column nodes have overlapping memberships, DiMMSB requires $K_r=K_c$ for model identifiability. For comparison, our ONM allows $K_r\le K_c$ at the cost of losing the overlapping property of the column nodes.

2.1. A Spectral Algorithm for Fitting ONM

The primary goal of the proposed algorithm is to estimate the row membership matrix ${\Pi }_r$ and the column membership matrix ${\Pi }_c$ from the observed adjacency matrix A with a given $K_r$ and $K_c$. We now discuss our intuition for the design of our algorithm to fit ONM.

Under conditions (I1) and (I2), by basic algebra, we have $\mathrm{rank}(\Omega )=K_r$. Let $\Omega =U_r\Lambda U_c^{\prime }$ be the compact singular value decomposition of $\Omega$, where $U_r\in {\mathbb{R}}^{n_r\times K_r},\Lambda \in {\mathbb{R}}^{K_r\times K_r},U_c\in {\mathbb{R}}^{n_c\times K_r}$, $U_r^{\prime }{U}_r=I_{K_r},U_c^{\prime }{U}_c=I_{K_r}$, and $I_{K_r}$ is a $K_r\times K_r$ identity matrix. Let $n_{c,k}=\left|\{i_c:\ell (i_c)=k\}\right|$ be the size of the k-th column community for $k\in [K_c]$. Let $n_{c,\max }={\max }_{k\in [K_c]}{n}_{c,k}$ and $n_{c,\min }={\min }_{k\in [K_c]}{n}_{c,k}$. Meanwhile, without causing confusion, let $n_{c,K_r}$ be the $K_r$-th largest size among all column communities. The following lemma guarantees that $U_r$ enjoys an ideal simplex structure and $U_c$ has $K_c$ distinct rows.

Lemma 1.

Under$ON{M}_{n_r,n_c}(K_r,K_c,P,{\Pi }_r,{\Pi }_c)$, there exists a unique$K_r\times K_r$matrix$B_r$and a unique$K_c\times K_r$matrix$B_c$, such that

• $U_r={\Pi }_r{B}_r$, where $B_r=U_r({\mathcal{I}}_r,:)$. Meanwhile, $U_r(i_r,:)=U_r({\overline{i}}_r,:)$ when ${\Pi }_r(i_r,:)={\Pi }_r({\overline{i}}_r,:)$ for $i_r,{\overline{i}}_r\in [n_r]$.
• $U_c={\Pi }_c{B}_c$. Meanwhile,$U_c(i_c,:)=U_c({\overline{i}}_c,:)$when$\ell (i_c)=\ell ({\overline{i}}_c)$for$i_c,{\overline{i}}_c\in [n_c]$, i.e.,$U_c$has$K_c$distinct rows. Furthermore, when$K_r=K_c=K$, we have$\parallel B_c(k,:)-B_c{(l,:)\parallel }_F=\sqrt{\frac{1}{n_{c,k}}+\frac{1}{n_{c,l}}}$for all$1\le k<l\le K$.

Lemma 1 says that the rows of $U_c$ form a $K_r$-simplex in ${\mathbb{R}}^{K_r}$, which we call the Ideal Simplex (IS), with the $K_r$ rows of $B_r$ being the vertices. This IS is also found in [9,17,21]. Meanwhile, Lemma 1 says that $U_c$ has $K_c$ distinct rows, and if two column nodes $i_c$ and ${\overline{i}}_c$ are from the same column community, then $U_c(i_c,:)=U_c({\overline{i}}_c,:)$.

Under ONM, to recover ${\Pi }_c$ from $U_c$, since $U_c$ has $K_c$ distinct rows, applying the k-means algorithm on all rows of $U_c$ returns true column communities by Lemma 1. Since $U_c$ has $K_c$ distinct rows, we can set ${\delta }_c={\min }_{k\ne l}{\parallel B_c(k,:)-B_c(l,:)\parallel }_F$ to measure the minimum center separation of $B_c$. By Lemma 1, ${\delta }_c\ge \sqrt{\frac{2}{n_{c,\max }}}$ when $K_r=K_c=K$ under $ON{M}_{n_r,n_c}(K_r,K_c,P,{\Pi }_r,{\Pi }_c)$. However, when $K_r<K_c$, it is a challenge to obtain a positive lower bound of ${\delta }_c$; see the proof of Lemma 1 for details.

Under ONM, to recover ${\Pi }_r$ from $U_r$, since $B_r$ is full rank, if $U_r$ and $B_r$ are known in advance ideally, we can exactly recover ${\Pi }_r$ by setting ${\Pi }_r=U_r{B}_r^{\prime }{(B_r{B}_r^{\prime })}^{-1}$ via Lemma 1. Set $Y_r=U_r{B}_r^{\prime }{(B_r{B}_r^{\prime })}^{-1}$. Since $Y_r\equiv {\Pi }_r$ and $\parallel {\Pi }_r(i_r,:){\parallel }_1=1$ for $i_r\in [n_r]$, we have

$$\begin{array}{c}\hfill {\Pi }_r(i_r,:)=\frac{Y_r(i_r,:)}{\parallel Y_r(i_r,:){\parallel }_1},i_r\in [n_r].\end{array}$$

With a given $U_r$, since it enjoys IS structure $U_r={\Pi }_r{B}_r\equiv {\Pi }_r{U}_r({\mathcal{I}}_r,:)$, as long as we can obtain the row corner matrix $U_r({\mathcal{I}}_r,:)$ (i.e., $B_r$), we can recover ${\Pi }_r$ exactly. As mentioned in [9,17,21], for such an ideal simplex, the successive projection (SP) algorithm [27] (for details of SP, see Algorithm A1) can be applied to $U_r$ with $K_r$ row communities to find $U_r({\mathcal{I}}_r,:)$.

Based on the above analysis, we are now ready to give the following algorithm which we call Ideal ONA. Input $\Omega ,K_r$, and $K_c$ with $K_r\le K_c$. Outputs: ${\Pi }_r$ and ℓ.

• Let $\Omega =U_r\Lambda U_c^{\prime }$ be the compact SVD of $\Omega$, such that $U_r\in {\mathbb{R}}^{n_r\times K_r},U_c\in {\mathbb{R}}^{n_c\times K_r},\Lambda \in {\mathbb{R}}^{K_r\times K_r},U_r^{\prime }{U}_r=I_{K_r},and U_c^{\prime }{U}_c=I_{K_r}$.
• For the row nodes,

  -

  Run the SP algorithm on all rows of $U_r$, assuming there are $K_r$ row communities to obtain $U_r({\mathcal{I}}_r,:)$. Set $B_r=U_r({\mathcal{I}}_r,:)$.

  -

  Set $Y_r=U_r{B}_r^{\prime }{(B_r{B}_r^{\prime })}^{-1}$. Recover ${\Pi }_r$ by setting ${\Pi }_r(i_r,:)=\frac{Y_r(i_r,:)}{\parallel Y_r(i_r,:){\parallel }_1}$ for $i_r\in [n_r]$.

  For the column nodes,

  -

  Run k-means on $U_c$ assuming that there are $K_c$ column communities, i.e., find the solution to the following optimization problem

  $$\begin{array}{c}\hfill M^{\ast }={\mathrm{argmin}}_{M\in M_{n_c,K_r,K_c}}{\parallel M-U_c\parallel }_F^2,\end{array}$$

  where $M_{n_c,K_r,K_c}$ denotes the set of $n_c\times K_r$ matrices with only $K_c$ different rows.

  -

  Use $M^{\ast }$ to obtain the labels vector ℓ of the column nodes. Note that since $M^{\ast }$ has $K_c$ distinct rows, two different column nodes, $i_c,{\overline{i}}_c\in [n_c]$, are in the same column community if $M^{\ast }(i_c,:)=M^{\ast }({\overline{i}}_c,:)$.

Following a similar proof of Theorem 1 of [21], the Ideal ONA exactly recovers row nodes memberships and column nodes labels, and this also verifies the identifiability of ONM in turn. For convenience, call the two steps for column nodes “run k-means on $U_c$ assuming there are $K_c$ column communities to obtain ℓ”.

We now extend the ideal case to the real case. Set $\tilde{A}={\widehat{U}}_r\widehat{\Lambda }{\widehat{U}}_c^{\prime }$ be the top-$K_r$-dimensional SVD of A, such that ${\widehat{U}}_r\in {\mathbb{R}}^{n_r\times K_r},{\widehat{U}}_c\in {\mathbb{R}}^{n_c\times K_r},\widehat{\Lambda }\in {\mathbb{R}}^{K_r\times K_r},{\widehat{U}}_r^{\prime }{\widehat{U}}_r=I_{K_r},{\widehat{U}}_c^{\prime }{\widehat{U}}_c=I_{K_r}$, and $\widehat{\Lambda }$ contains the top $K_r$ singular values of A. For the real case, we use ${\widehat{B}}_r,{\widehat{B}}_c,{\widehat{Y}}_r,{\widehat{\Pi }}_r,{\widehat{\Pi }}_c$ given in Algorithm 1 to estimate $B_r,B_c,Y_r,{\Pi }_r,{\Pi }_c$, respectively. Algorithm 1, called the Overlapping and Non-overlapping algorithm (ONA for short), is a natural extension of the Ideal ONA to the real case. In ONA, we set the negative entries of ${\widehat{Y}}_r$ as 0 by setting ${\widehat{Y}}_r=\max (0,{\widehat{Y}}_r)$, for the reason that the weights for any row node should be non-negative while there may exist some negative entries of ${\widehat{U}}_r{\widehat{B}}_r^{\prime }{({\widehat{B}}_r{\widehat{B}}_r^{\prime })}^{-1}$. Note that in a directed network, if the column nodes have an overlapping property while row nodes do not, to perform community detection for such a directed network, the transpose of the adjacency matrix should be set as input when applying our algorithm.

Algorithm 1Overlapping and Non-overlapping Algorithm (ONA)

Require: The adjacency matrix $A\in {\mathbb{R}}^{n_r\times n_c}$ of a directed network, the number of row communities $K_r$, and the number of column communities $K_c$ with $K_r\le K_c$. Ensure: The estimated $n_r\times K_r$ membership matrix ${\widehat{\Pi }}_r$ for row nodes, and the estimated $n_c\times 1$ labels vector $\widehat{\ell }$ for column nodes. 1: Compute ${\widehat{U}}_r\in {\mathbb{R}}^{n_r\times K_r}$ and ${\widehat{U}}_c\in {\mathbb{R}}^{n_c\times K_r}$ from the top-$K_r$-dimensional SVD of A. 2: For row nodes: Apply SP algorithm (i.e., Algorithm 2) on the rows of ${\widehat{U}}_r$ assuming there are $K_r$ row clusters to obtain the near-corners matrix ${\widehat{U}}_r({\widehat{\mathcal{I}}}_r,:)\in {\mathbb{R}}^{K_r\times K_r}$, where ${\widehat{\mathcal{I}}}_r$ is the index set returned by SP algorithm. Set ${\widehat{B}}_r={\widehat{U}}_r({\widehat{\mathcal{I}}}_r,:)$. Compute the $n_r\times K_r$ matrix ${\widehat{Y}}_r$ such that ${\widehat{Y}}_r={\widehat{U}}_r{\widehat{B}}_r^{\prime }{({\widehat{B}}_r{\widehat{B}}_r^{\prime })}^{-1}$. Set ${\widehat{Y}}_r=\max (0,{\widehat{Y}}_r)$ and estimate ${\Pi }_r(i_r,:)$ by ${\widehat{\Pi }}_r(i_r,:)=\frac{{\widehat{Y}}_r(i_r,:)}{\parallel {\widehat{Y}}_r(i_r,:){\parallel }_1},i_r\in [n_r]$.

For column nodes: run k-means on ${\widehat{U}}_c$ assuming there are $K_c$ column communities to obtain $\widehat{\ell }$.

2.2. Main Results for ONA

In this section, we show the consistency of our algorithm for fitting the ONM as the number of row nodes $n_r$ and the number of column nodes $n_c$ increases. Throughout this paper, $K_r$ and $K_c$ are two known integers. First, we assume that:

Assumption 1.

$\rho \max (n_r,n_c)\ge \log (n_r+n_c)$.

Assumption 1 controls the sparsity of the directed network considered for theoretical study. When building an estimation consistency of the spectral clustering methods in community detection, the sparsity assumption is common; see [13,14,17,18,20,21]. Especially, when ONM reduces to SBM, the sparsity requirement in Assumption 1 is consistent with that of Theorem 3.1 in [13], which guarantees the theoretical optimality on the sparsity condition of this paper. To measure the performance of ONA for row nodes memberships, since row nodes have mixed memberships, naturally, we use the $l_1$ norm difference between ${\Pi }_r$ and ${\widehat{\Pi }}_r$. Since the column nodes are all pure nodes, we consider the performance criterion defined in [15] to measure the estimation error of ONA on the column nodes. We introduce this measurement of estimation error below.

Let ${\mathcal{T}}_c=\{{\mathcal{T}}_{c,1},{\mathcal{T}}_{c,2},\dots ,{\mathcal{T}}_{c,K_c}\}$ be the true partition of column nodes $\{1,2,\dots ,n_c\}$ obtained from ℓ, such that ${\mathcal{T}}_{c,k}=\{i_c\in [n_c]:\ell (i_c)=k\}$ for $k\in [K_c]$. Let ${\widehat{\mathcal{T}}}_c=\{{\widehat{\mathcal{T}}}_{c,1},{\widehat{\mathcal{T}}}_{c,2},\dots ,{\widehat{\mathcal{T}}}_{c,K_c}\}$ be the estimated partition of column nodes $\{1,2,\dots ,n_c\}$ obtained from $\widehat{\ell }$ of ONA, such that ${\widehat{\mathcal{T}}}_{c,k}=\{i_c\in [n_c]:\widehat{\ell }(i_c)=k\}$ for $k\in [K_c]$. The criterion is defined as

$$\begin{array}{c}\hfill {\widehat{f}}_c={\min }_{\pi \in S_{K_c}}{\max }_{k\in [K_c]}\frac{|{\mathcal{T}}_{c,k}\cap {\widehat{\mathcal{T}}}_{c,\pi (k)}^c|+|{\mathcal{T}}_{c,k}^c\cap {\widehat{\mathcal{T}}}_{c,\pi (k)}|}{n_{c,k}},\end{array}$$

where $S_{K_c}$ is the set of all permutations of $\{1,2,\dots ,K_c\}$, and the superscript c denotes the complementary set. As mentioned in [15], ${\widehat{f}}_c$ measures the maximum proportion of column nodes in the symmetric difference of ${\mathcal{T}}_{c,k}$ and ${\widehat{\mathcal{T}}}_{c,\pi (k)}$.

The next theorem gives the theoretical bounds on the estimations of memberships for both the row and column nodes, which is the main theoretical result for ONA.

Theorem 1.

Under $ON{M}_{n_r,n_c}(K_r,K_c,P,{\Pi }_r,{\Pi }_c)$, when Assumption 1 holds, suppose that ${\sigma }_{K_r}(\Omega )\ge C\sqrt{\rho (n_r+n_c)\log (n_r+n_c)}$, with a probability of at least $1-o({(n_r+n_c)}^{-\alpha })$ for any $\alpha >0$,

• For row nodes, there exists a permutation matrix ${\mathcal{P}}_r$ such that

  $$\begin{array}{cc}\hfill & {\max }_{i_r\in [n_r]}{\parallel e_{i_r}^{\prime }({\widehat{\Pi }}_r-{\Pi }_r{\mathcal{P}}_r)\parallel }_1=O(\varpi \kappa ({\Pi }_r^{\prime }{\Pi }_r)K_r\sqrt{{\lambda }_1({\Pi }_r^{\prime }{\Pi }_r)}),\hfill \end{array}$$

  where $\varpi =\parallel {\widehat{U}}_r{\widehat{U}}_r^{\prime }-U_r{U}_r^{\prime }{\parallel }_{2\to \infty }$ is the row-wise singular eigenvector error.
• For column nodes, ${\widehat{f}}_c=O(\frac{K_r{K}_c\max (n_r,n_c)\log (n_r+n_c)}{{\sigma }_{K_r}^2(\tilde{P})\rho {\delta }_c^2{\sigma }_{K_r}^2({\Pi }_r)n_{c,K_r}{n}_{c,\min }})$. Especially, when $K_r=K_c=K$,

  $$\begin{array}{c}\hfill {\widehat{f}}_c=O(\frac{K^2\max (n_r,n_c)n_{c,\max }\log (n_r+n_c)}{{\sigma }_K^2(\tilde{P})\rho {\sigma }_K^2({\Pi }_r)n_{c,\min }^2}).\end{array}$$

Adding conditions similar to Corollary 3.1 in [17], we have the following corollary.

Corollary 1.

Under $ON{M}_{n_r,n_c}(K_r,K_c,P,{\Pi }_r,{\Pi }_c)$, suppose conditions in Theorem 1 hold, and further, suppose that ${\lambda }_{K_r}({\Pi }_r^{\prime }{\Pi }_r)=O(\frac{n_r}{K_r}),n_{c,\min }=O(\frac{n_c}{K_c})$, with a probability of at least $1-o({(n_r+n_c)}^{-\alpha })$,

• For row nodes, when $K_r=K_c=K$,

  $$\begin{array}{cc}\hfill & {\max }_{i_r\in [n_r]}{\parallel e_{i_r}^{\prime }({\widehat{\Pi }}_r-{\Pi }_r{\mathcal{P}}_r)\parallel }_1=O(\frac{K^2(\sqrt{\frac{C\max (n_r,n_c)}{\min (n_r,n_c)}}+\sqrt{\log (n_r+n_c)})}{{\sigma }_K(\tilde{P})\sqrt{\rho n_c}}).\hfill \end{array}$$
• For column nodes, ${\widehat{f}}_c=O(\frac{K_r^2{K}_c^3\max (n_r,n_c)\log (n_r+n_c)}{{\sigma }_{K_r}^2(\tilde{P})\rho {\delta }_c^2{n}_r{n}_c^2})$. When $K_r=K_c=K$,

  $$\begin{array}{c}\hfill {\widehat{f}}_c=O(\frac{K^4\max (n_r,n_c)\log (n_r+n_c)}{{\sigma }_K^2(\tilde{P})\rho n_r{n}_c}).\end{array}$$

Especially, when$n_r=O(n),n_c=O(n),K_r=O(1)$, and$K_c=O(1)$,

• For row nodes, when $K_r=K_c=K$,

  $$\begin{array}{c}\hfill {\max }_{i_r\in [n_r]}{\parallel e_{i_r}^{\prime }({\widehat{\Pi }}_r-{\Pi }_r{\mathcal{P}}_r)\parallel }_1=O(\frac{\sqrt{\log (n)}}{{\sigma }_K(\tilde{P})\sqrt{\rho n}}).\end{array}$$
• For column nodes, ${\widehat{f}}_c=O(\frac{\log (n)}{{\sigma }_{K_r}^2(\tilde{P})\rho {\delta }_c^2{n}^2})$. When $K_r=K_c=K$,

  $$\begin{array}{c}\hfill {\widehat{f}}_c=O(\frac{\log (n)}{{\sigma }_K^2(\tilde{P})\rho n}).\end{array}$$

When $n_r=O(n),n_c=O(n),K_r=K_c=K=O(1)$ in Corollary 1, the bounds for the row and column nodes are $O(\frac{1}{{\sigma }_K(\tilde{P})}\sqrt{\frac{\log (n)}{n}})$ and $O(\frac{1}{{\sigma }_K^2(\tilde{P})}\frac{\log (n)}{\rho n})$, respectively, and we see that ONA yields a stable and consistent community detection for both the row and column nodes, since the error rates go to zero as $n\to \infty$ when $\tilde{P}$ is fixed. Especially, for the row nodes with mixed memberships, when the DCMM proposed in [9] reduces to MMSB and $K=O(1)$, the error bound of the Mixed-SCORE in Theorem 2.2 of [9] is also $O(\frac{1}{{\sigma }_K(\tilde{P})}\sqrt{\frac{\log (n)}{n}})$, which guarantees the theoretical optimality of our analysis for the row nodes. For the column nodes, when every column community enjoys similar sizes and $K=O(1)$, our bound $O(\frac{1}{{\sigma }_K^2(\tilde{P})}\frac{\log (n)}{\rho n})$ matches Corollary 3.2 in [13] up to a logarithmic factor, which guarantees the theoretical optimality of our analysis for column nodes. Furthermore, the optimality of our requirement on network sparsity and the theoretical upper bounds of ONA’s error rates is also supported by using the separation condition and sharp threshold criterion developed in [28].

3. The Overlapping and Degree-Corrected Non-Overlapping Model

In this section, we propose an extension of ONM by considering the degree heterogeneity, and we build theoretical guarantees for algorithm fitting our model.

Let ${\theta }_c$ be an $n_c\times 1$ vector whose $i_c$-th entry is the degree heterogeneity of column node $i_c$, for $i_c\in [n_c]$. Let ${\Theta }_c$ be an $n_c\times n_c$ diagonal matrix whose $i_c$-th diagonal element is ${\theta }_c(i_c)$. For $i_r\in [n_r],i_c\in [n_c]$, the extended model for generating A is:

$$\begin{array}{c}\hfill \Omega :={\Pi }_{r}P{\Pi }_c^{\prime }{\Theta }_c,A(i_r,i_c)\sim \mathrm{Bernoulli}(\Omega (i_r,i_c)).\end{array}$$

(6)

Definition 2.

Call model (1)–(4), (6) the Overlapping and Degree-Corrected Non-overlapping model (ODCNM), and denote it by $ODCN{M}_{n_r,n_c}(K_r,K_c,P,{\Pi }_r,{\Pi }_c,{\Theta }_c)$.

Note that, under ODCNM, the maximum element of P can be larger than 1, since ${\max }_{i_c\in [n_c]}{\theta }_c(i_c)$ also controls the sparsity of directed network $\mathcal{N}$. The following proposition guarantees that ODCNM is identifiable in terms of $P,{\Pi }_r$, and ${\Pi }_c$, and such identifiability is similar to that of DCSBM.

Proposition 2.

If conditions (I1) and (I2) hold, ODCNM is identifiable for the membership matrices: For eligible $(P,{\Pi }_r,{\Pi }_c,{\Theta }_c)$ and $(\check{P},{\check{\Pi }}_r,{\check{\Pi }}_c,{\check{\Theta }}_c)$, if ${\Pi }_{r}P{\Pi }_c^{\prime }{\Theta }_c={\check{\Pi }}_r\check{P}{\check{\Pi }}_c^{\prime }{\check{\Theta }}_c$, then ${\Pi }_r={\check{\Pi }}_r$ and ${\Pi }_c={\check{\Pi }}_c$.

Remark 2.

By setting ${\theta }_c(i_c)=\rho$ for $i_c\in [n_c]$, ODCNM reduces to ONM, and this is the reason for why ODCNM can be seen as an extension of ONM. Meanwhile, though DCScBM [18] can model directed networks with degree heterogeneities for both row and column nodes, DCScBM does not allow the overlapping property for row nodes. For comparison, our ODCNM allows row nodes to have an overlapping property at the cost of losing the degree heterogeneities and requiring $K_r\le K_c$ for model identifiability.

3.1. A Spectral Algorithm for Fitting ODCNM

We now discuss our intuition for the design of our algorithm to fit ODCNM. Without causing confusion, we also use $U_r,U_c,B_r,B_c,{\delta }_c,Y_r$ under ODCNM. Let $U_{c,\ast }\in {\mathbb{R}}^{n_c\times K_r}$ be the row-normalized version of $U_c$, such that $U_{c,\ast }(i_c,:)=\frac{U_c(i_c,:)}{\parallel U_c(i_c,:){\parallel }_F}$ for $i_c\in [n_c]$. Then, clustering the rows of $U_{c,\ast }$ using the k-means algorithm can return perfect clustering for column nodes, and this is guaranteed by the following lemma.

Lemma 2.

Under $ODCN{M}_{n_r,n_c}(K_r,K_c,P,{\Pi }_r,{\Pi }_c,{\Theta }_c)$, there exists a unique $K_r\times K_r$ matrix $B_r$ and a unique $K_c\times K_r$ matrix $B_c$, such that

• $U_r={\Pi }_r{B}_r$, where $B_r=U_r({\mathcal{I}}_r,:)$. Meanwhile, $U_r(i_r,:)=U_r({\overline{i}}_r,:)$ when ${\Pi }_r(i_r,:)={\Pi }_r({\overline{i}}_r,:)$ for $i_r,{\overline{i}}_r\in [n_r]$.
• $U_{c,\ast }={\Pi }_c{B}_c$. Meanwhile, $U_{c,\ast }(i_c,:)=U_{c,\ast }({\overline{i}}_c,:)$ when $\ell (i_c)=\ell ({\overline{i}}_c)$ for $i_c,{\overline{i}}_c\in [n_c]$. Furthermore, when $K_r=K_c=K$, we have $\parallel B_c(k,:)-B_c{(l,:)\parallel }_F=\sqrt{2}$ for all $1\le k<l\le K$.

Recall that we set ${\delta }_c={\min }_{k\ne l}{\parallel B_c(k,:)-B_c(l,:)\parallel }_F$ by Lemma 2; ${\delta }_c=\sqrt{2}$ when $K_r=K_c=K$ under $ODCN{M}_{n_r,n_c}(K_r,K_c,P,{\Pi }_r,{\Pi }_c,{\Theta }_c)$. However, when $K_r<K_c$, it is a challenge to obtain a positive lower bound of ${\delta }_c$; see the proof of Lemma 2 for details.

Under ODCNM, to recover ${\Pi }_c$ from $U_c$, since $U_{c,\ast }$ has $K_c$ distinct rows, applying the k-means algorithm on all rows of $U_{c,\ast }$ returns true column communities by Lemma 2. To recover ${\Pi }_r$ from $U_r$, the same idea as that of under ONM can be followed.

Based on the above analysis, we are now ready to present the following algorithm, which we call Ideal ODCNA. Input $\Omega ,K_r,K_c$ with $K_r\le K_c$. Output: ${\Pi }_r$ and ℓ.

• Let $\Omega =U_r\Lambda U_c^{\prime }$ be the compact SVD of $\Omega$, such that $U_r\in {\mathbb{R}}^{n_r\times K_r},U_c\in {\mathbb{R}}^{n_c\times K_r},\Lambda \in {\mathbb{R}}^{K_r\times K_r},U_r^{\prime }{U}_r=I_{K_r},U_c^{\prime }{U}_c=I_{K_r}$. Let $U_{c,\ast }$ be the row-normalization of $U_c$.
• For row nodes, they are the same as that of Ideal ONA.
  For column nodes: run k-means on $U_{c,\ast }$ assuming there are $K_c$ column communities to obtain ℓ.

We now extend the ideal case to the real case. Let ${\widehat{U}}_{c,\ast }\in {\mathbb{R}}^{n_c\times K_r}$ be the row-normalized version of ${\widehat{U}}_c$, such that ${\widehat{U}}_{c,\ast }(i_c,:)=\frac{{\widehat{U}}_c(i_c,:)}{\parallel {\widehat{U}}_c(i_c,:){\parallel }_F}$ for $i_c\in [n_c]$. The Overlapping and Degree-Corrected Non-overlapping Algorithm (ODCNA for short) is a natural extension of the Ideal ODCNA to the real case, where all steps of ODCNA are the same as ONA except for those for column nodes. ODCNA applies k-means on ${\widehat{U}}_{c,\ast }$ to obtain $\widehat{\ell }$.

3.2. Main Results for ODCNA

Set ${\theta }_{c,\max }={\max }_{i_c\in [n_c]}{\theta }_c(i_c),{\theta }_{c,\min }={\min }_{i_c\in [n_c]}{\theta }_c(i_c)$, and $P_{\max }={\max }_{k\in [K_r],l\in [n_c]}P(k,l)$. Assume that

Assumption 2.

$P_{\max }\max ({\theta }_{c,\max }{n}_r,\parallel {\theta }_c{\parallel }_1)\ge \log (n_r+n_c)$.

The next theorem is the main theoretical result for ODCNA, where we also use the same measurements as ONA to measure the performances of ODCNA.

Theorem 2.

Under $ODCN{M}_{n_r,n_c}(K_r,K_c,P,{\Pi }_r,{\Pi }_c,{\Theta }_c)$, when Assumption 2 holds, suppose ${\sigma }_{K_r}(\Omega )\ge C\sqrt{{\theta }_{c,\max }(n_r+n_c)\log (n_r+n_c)}$, with a probability at least $1-o({(n_r+n_c)}^{-\alpha })$,

• For the row nodes,

  $$\begin{array}{cc}\hfill & {\max }_{i_r\in [n_r]}{\parallel e_{i_r}^{\prime }({\widehat{\Pi }}_r-{\Pi }_r{\mathcal{P}}_r)\parallel }_1=O(\varpi \kappa ({\Pi }_r^{\prime }{\Pi }_r)K_r\sqrt{{\lambda }_1({\Pi }_r^{\prime }{\Pi }_r)}).\hfill \end{array}$$
• For the column nodes,

  $$\begin{array}{c}\hfill {\widehat{f}}_c=O(\frac{{\theta }_{c,\max }^2{K}_r{K}_c\max ({\theta }_{c,\max }{n}_r,\parallel {\theta }_c{\parallel }_1)n_{c,\max }\log (n_r+n_c)}{{\sigma }_{K_r}^2(P){\theta }_{c,\min }^4{\delta }_c^2{m}_{V_c}^2{\sigma }_{K_r}^2({\Pi }_r)n_{c,K_r}{n}_{c,\min }}),\end{array}$$

  where $m_{V_c}$ is a parameter defined in the proof of this theorem, and it is 1 when $K_r=K_c$. Especially, when $K_r=K_c=K$,

  $$\begin{array}{c}\hfill {\widehat{f}}_c=O(\frac{{\theta }_{c,\max }^2{K}^2\max ({\theta }_{c,\max }{n}_r,\parallel {\theta }_c{\parallel }_1)n_{c,\max }\log (n_r+n_c)}{{\sigma }_K^2(P){\theta }_{c,\min }^4{\sigma }_K^2({\Pi }_r)n_{c,\min }^2}).\end{array}$$

Adding some conditions on model parameters, we have the following corollary.

Corollary 2.

Under $ODCN{M}_{n_r,n_c}(K_r,K_c,P,{\Pi }_r,{\Pi }_c,{\Theta }_c)$, suppose that conditions in Theorem 2 hold, and further, suppose that ${\lambda }_{K_r}({\Pi }_r^{\prime }{\Pi }_r)=O(\frac{n_r}{K_r}),n_{c,\min }=O(\frac{n_c}{K_c})$, with a probability of at least $1-o({(n_r+n_c)}^{-\alpha })$,

• For row nodes, when $K_r=K_c=K$,

  $$\begin{array}{cc}\hfill & {\max }_{i_r\in [n_r]}{\parallel e_{i_r}^{\prime }({\widehat{\Pi }}_r-{\Pi }_r{\mathcal{P}}_r)\parallel }_1=O(\frac{K^2\sqrt{{\theta }_{c,\max }}(\sqrt{\frac{C\max (n_r,n_c)}{\min (n_r,n_c)}}+\sqrt{\log (n_r+n_c)})}{{\theta }_{c,\min }{\sigma }_K(P)\sqrt{n_c}}).\hfill \end{array}$$
• For column nodes, ${\widehat{f}}_c=O(\frac{{\theta }_{c,\max }^2{K}_r^2{K}_c^2\max ({\theta }_{c,\max }{n}_r,\parallel {\theta }_c{\parallel }_1)\log (n_r+n_c)}{{\sigma }_{K_r}^2(P){\theta }_{c,\min }^4{\delta }_c^2{m}_{V_c}^2{n}_r{n}_c})$. When $K_r=K_c=K$,

  $$\begin{array}{c}\hfill {\widehat{f}}_c=O(\frac{{\theta }_{c,\max }^2{K}^4\max ({\theta }_{c,\max }{n}_r,\parallel {\theta }_c{\parallel }_1)\log (n_r+n_c)}{{\sigma }_K^2(P){\theta }_{c,\min }^4{n}_r{n}_c}).\end{array}$$

Especially, when$n_r=O(n),n_c=O(n),K_r=O(1)$and$K_c=O(1)$,

• For row nodes, when $K_r=K_c$,

  $$\begin{array}{c}\hfill {\max }_{i_r\in [n_r]}{\parallel e_{i_r}^{\prime }({\widehat{\Pi }}_r-{\Pi }_r{\mathcal{P}}_r)\parallel }_1=O(\frac{\sqrt{{\theta }_{c,\max }\log (n)}}{{\theta }_{c,\min }{\sigma }_K(P)\sqrt{n}}).\end{array}$$
• For column nodes,${\widehat{f}}_c=O(\frac{{\theta }_{c,\max }^2\max ({\theta }_{c,\max }{n}_r,\parallel {\theta }_c{\parallel }_1)\log (n)}{{\sigma }_{K_r}^2(P){\theta }_{c,\min }^4{\delta }_c^2{m}_{V_c}^2{n}^2})$. When $K_r=K_c=K$,

  $$\begin{array}{c}\hfill {\widehat{f}}_c=O(\frac{{\theta }_{c,\max }^2\max ({\theta }_{c,\max }{n}_r,\parallel {\theta }_c{\parallel }_1)\log (n)}{{\sigma }_K^2(P){\theta }_{c,\min }^4{n}^2}).\end{array}$$

If we further set ${\theta }_{c,\max }=O(\rho )$ and ${\theta }_{c,\min }=O(\rho )$, we have the below corollary.

Corollary 3.

Under $ODCN{M}_{n_r,n_c}(K_r,K_c,P,{\Pi }_r,{\Pi }_c,{\Theta }_c)$, suppose that the conditions in Theorem 2 hold, and further, suppose that ${\lambda }_{K_r}({\Pi }_r^{\prime }{\Pi }_r)=O(\frac{n_r}{K_r}),n_{c,\min }=O(\frac{n_c}{K_c})$ and ${\theta }_{c,\max }=O(\rho ),{\theta }_{c,\min }=O(\rho )$, with a probability of at least $1-o({(n_r+n_c)}^{-\alpha })$,

• For row nodes, when $K_r=K_c=K$,

  $$\begin{array}{cc}\hfill & {\max }_{i_r\in [n_r]}{\parallel e_{i_r}^{\prime }({\widehat{\Pi }}_r-{\Pi }_r{\mathcal{P}}_r)\parallel }_1=O(\frac{K^2(\sqrt{\frac{C\max (n_r,n_c)}{\min (n_r,n_c)}}+\sqrt{\log (n_r+n_c)})}{{\sigma }_K(P)\sqrt{\rho n_c}}).\hfill \end{array}$$
• For column nodes, ${\widehat{f}}_c=O(\frac{K_r^2{K}_c^2\max (n_r,n_c)\log (n_r+n_c)}{{\sigma }_{K_r}^2(P)\rho {\delta }_c^2{m}_{V_c}^2{n}_r{n}_c})$. When $K_r=K_c=K$,

  $$\begin{array}{c}\hfill {\widehat{f}}_c=O(\frac{K^4\max (n_r,n_c)\log (n_r+n_c)}{{\sigma }_K^2(P)\rho n_r{n}_c}).\end{array}$$

Especially, when$n_r=O(n),n_c=O(n),K_r=O(1)$and$K_c=O(1)$,

• For row nodes, when $K_r=K_c$,

  $$\begin{array}{c}\hfill {\max }_{i_r\in [n_r]}{\parallel e_{i_r}^{\prime }({\widehat{\Pi }}_r-{\Pi }_r{\mathcal{P}}_r)\parallel }_1=O(\frac{\sqrt{\log (n)}}{{\sigma }_K(P)\sqrt{\rho n}}).\end{array}$$
• For column nodes,${\widehat{f}}_c=O(\frac{\log (n)}{{\sigma }_{K_r}^2(P)\rho {\delta }_c^2{m}_{V_c}^{2}n})$. When $K_r=K_c=K$,

  $$\begin{array}{c}\hfill {\widehat{f}}_c=O(\frac{\log (n)}{{\sigma }_K^2(P)\rho n}).\end{array}$$

By setting ${\Theta }_c=\rho I$, ODCNM degenerates to ONM. By comparing Corollaries 1 and 3, we see that theoretical results under ODCNM are consistent with those under ONM when ODCNM degenerates to ONM for the case where $K_r=K_c=K$.

4. Simulations

In this section, we present some simulations to investigate the performances of the two proposed algorithms. We measure their performances using the Mixed-Hamming error rate (MHamm for short) for row nodes, and the Hamming error rate (Hamm for short) for the column nodes defined below

$$\begin{array}{cc}\hfill & \mathrm{MHamm}=\frac{{\min }_{\pi \in S_{K_r}}{\parallel {\widehat{\Pi }}_r\pi -{\Pi }_r\parallel }_1}{n_r},\mathrm{Hamm}=\frac{{\min }_{\pi \in S_{K_c}}{\parallel {\widehat{\Pi }}_c\pi -{\Pi }_c\parallel }_0}{n_c},\hfill \end{array}$$

where $S_{K_r}$ is the set of all permutations of $\{1,2,\dots ,K_r\}$, $S_{K_c}$ is the set of all permutations of $\{1,2,\dots ,K_c\}$; ${\widehat{\Pi }}_c\in {\mathbb{R}}^{n_c\times K_c}$ is defined as ${\widehat{\Pi }}_c(i_c,k)=1$ if $\widehat{\ell }(i_c)=k$, and 0 otherwise for $i_c\in [n_c],k\in [K_c]$.

For all simulations in this section, the parameters $(n_r,n_c,K_r,K_c,P,\rho ,{\Pi }_r,{\Pi }_c,{\Theta }_c)$ are set as below. Unless specified, set $n_r=400,n_c=300,K_r=3,K_c=4$. For the column nodes, generate ${\Pi }_c$ by setting each column node belonging to one of the column communities with equal probability. Let each row community have 100 pure nodes, and let all the mixed row nodes have memberships $(0.6,0.3,0.1)$. $P=\rho \tilde{P}$ is set independently under ONM and ODCNM. Under ONM, $\rho$ is 0.5 in Experiment 1, and we study the influence of $\rho$ in Experiment 2. Under ODCNM, for $z_c\ge 1$, we generate the degree parameters for the column nodes as below: let ${\theta }_c\in {\mathbb{R}}^{n_c\times 1}$, such that $1/{\theta }_c(i_c)\stackrel{iid}{\sim }U(1,z_c)$ for $i_c\in [n_c]$, where $U(1,z_c)$ denotes the uniform distribution on $[1,z_c]$. We study the influences of $z_c$ and $\rho$ under ODCNM in Experiments 3 and 4, respectively. For all settings, we report the averaged MHamm and the averaged Hamm over 50 repetitions.

Experiment 1: Changing $n_c$ under ONM. Let $n_c$ range over $\{50,100,150,\dots ,300\}$. For this experiment, P is set as

$$P=\rho \left[\begin{array}{cccc}1& 0.3& 0.2& 0.3\\ 0.2& 0.9& 0.1& 0.2\\ 0.3& 0.2& 0.8& 0.3\end{array}\right].$$

Let $\rho =0.5$ for this experiment designed under ONM. The numerical results are shown in panels (a) and (b) of Figure 1. The results show that as $n_c$ increases, ONA and ODCNA perform better. For the row nodes, since both ONA and ODCNA apply the SP algorithm on $\widehat{U}$ to estimate ${\Pi }_r$, the estimated row membership matrices of ONA and ODCNA are same, and hence, MHamm for ONA is always equal to that of ODCNA.

Experiment 2: Changing $\rho$ under ONM.P is set the same as in Experiment 1, and we let the range of $\rho$ be $\{0.1,0.2,\dots ,1\}$ to study the influence of $\rho$ on the performances of ONA and ODCNA under ONM. The results are displayed in panels (c) and (d) of Figure 1. From the results, we can see that both methods perform better as $\rho$ increases, since a larger $\rho$ gives more edges generated in a directed network.

Experiment 3: Change $z_c$ under ODCNM.P is set to be the same as Experiment 1, and $\rho =0.5$. Let $z_c$ range in $\{1,2,\dots ,8\}$. Increasing $z_c$ decreases the edges generated under ODCNM. Panels (e) and (f) in Figure 1 display the simulation results of this experiment. The results show that generally, increasing the variability of the node degrees makes it harder to detect the node memberships for both ONA and ODCNA. Though ODCNA is designed under ODCNM, it holds similar performances as ONA for directed networks in which column nodes have various degrees in this experiment, and this is consistent with our theoretical findings in Corollaries 1 and 2.

Experiment 4: Change $\rho$ under ODCNM. Setting $z_c=3$, P is set to be the same as in Experiment 1, and let $\rho$ range in $\{0.1,0.2,\dots ,1\}$ under ODCNM. Panels (g) and (h) in Figure 1 display the simulation results of this experiment. The performances of the two proposed methods are similar as those of Experiment 2.

Remark 3.

For visuality, we plot A generated under ONM. Let $n_r=24,n_c=20,K_r=2,K_c=2$, and

$$P=\left[\begin{array}{ccc}1& 0.2& \\ 0.1& 0.9\end{array}\right].$$

For the row nodes, let ${\Pi }_r(i_r,1)=1$ for $1\le i_r\le 8$, ${\Pi }_r(i_r,2)=1$ for $9\le i_r\le 16$, and ${\Pi }_r(r_r,:)=[0.70.3]$ for $17\le i_r\le 24$. For the column nodes, let $\ell (i_c)=1$ for $1\le i_c\le 10$, and $\ell (i_c)=2$ for $11\le i_c\le 20$. For the above setting, we generate two random adjacency matrices in Figure 2, where we also report the error rates of ONA and ODCNA. Note that, since the adjacency matrices are shown in Figure 2, and as ${\Pi }_r,\ell ,K_r$, and $K_c$ are known here, readers can apply ONA and ODCNA to A in Figure 2 to check the effectiveness of ONA and ODCNA.

Remark 4.

For visuality, we also plot a directed network as well as its adjacency matrix generated under ONM. Let $n_r=30,n_c=30,K_r=2,K_c=3$, and

$$P=0.9\left[\begin{array}{ccc}1& 0.1& 0.1\\ 0.1& 0.9& 0.1\end{array}\right].$$

For row nodes, let ${\Pi }_r(i_r,1)=1$ for $1\le i_r\le 10$, ${\Pi }_r(i_r,2)=1$ for $11\le i_r\le 20$, and ${\Pi }_r(i_r,:)=[0.70.3]$ for $21\le i_r\le 30$. For column nodes, let $\ell (i_c)=1$ for $1\le i_c\le 10$, $\ell (i_c)=2$ for $11\le i_c\le 20$, and $\ell (i_c)=3$ for $21\le i_c\le 30$. For the above setting, we generate one adjacency matrix in panel (a) of Figure 3, where we also report the error rates of ONA and ODCNA. Furthermore, panels (b) and (c) of Figure 3 show the sending pattern and receiving pattern sides of this simulated directed network, respectively.

5. Real Data Analysis

For real-world directed networks, since nodes always have various degrees, we only apply ODCNA to deal with real-world datasets in this section. For the real-world directed networks analyzed in this section, the row nodes are always same as the column nodes, so we do not use subscript r and c to distinguish the row and column nodes here, and we let $n_r=n_c=n$. Meanwhile, the number of row communities is equal to that of the column communities; i.e, $K_r=K_c=K$ for real data, where we always set $K_r=K_c=K$, as analyzed in [18], since it is a challenge to determine the number of row (column) communities for real-world directed networks without prior knowledge. When the row nodes are the same as the column nodes, $A(i,j)=1$ means that a directed edge is sent from node i to node j. Thus, for any node, it has two patterns, the sending pattern and the receiving pattern. For the sending (receiving) pattern, we use the sending (receiving) cluster to denote the prior row (column) community, where we use the sending and receiving patterns to distinguish the behaviors of any node having the two patterns, as was performed in [18].

For ${\widehat{\Pi }}_r$ obtained from ODCNA, we call node i a highly mixed node if $0.8\ge {\max }_{1\le k\le K}{\widehat{\Pi }}_r(i,k)$, where 0.8 is a threshold. Here, 0.8 is a moderate value to define highly mixed nodes, and we can also choose 0.9, 0.95, or some other values in $(0,1)$. However, we choose 0.8 as the threshold, because setting the threshold to be larger (or lesser) than 0.8 may be too restrictive (loose) to define highly mixed nodes. The definition of highly mixed node is important, since it tells us whether a node only belongs to one community or belongs to multiple communities. Let $\tau$ be the proportion of highly mixed row nodes among all nodes, to measure the mixability of a directed network, i.e, $\tau =\frac{|i\in [n]:i\mathrm{is}\mathrm{a}\mathrm{highly}\mathrm{mixed}\mathrm{node}|}{n}$. Meanwhile, we let ${\widehat{\ell }}_r$ be an $n\times 1$ vector, such that ${\widehat{\ell }}_r(i)={\mathrm{argmax}}_{1\le k\le K}{\widehat{\Pi }}_r(i,k)$, where we use ${\widehat{\ell }}_r(i)$ to denote the home base sending pattern cluster of node i. Set

$$\begin{array}{c}\hfill {\mathrm{Hamm}}_{rc}=\frac{{\min }_{\pi \in S_K}{\parallel {\widehat{\Pi }}_c\pi -{\tilde{\widehat{\Pi }}}_r\parallel }_0}{n},\end{array}$$

where $S_K$ is the set of all permutations of $\{1,2,\dots ,K\}$; ${\tilde{\widehat{\Pi }}}_r\in {\mathbb{R}}^{n\times K}$ is defined as ${\tilde{\widehat{\Pi }}}_r(i,k)=1$ if ${\widehat{\ell }}_r(i)=k$, and 0 otherwise for $i\in [n],k\in [K]$. ${\mathrm{Hamm}}_{rc}$ is defined to measure the difference between the sending and receiving pattern clusters. After defining $\tau$ and ${\mathrm{Hamm}}_{rc}$, we see that a larger $\tau$ indicates a directed network in which a large proposition of nodes are highly mixed nodes with a sending pattern, and a larger ${\mathrm{Hamm}}_{rc}$ indicates that the sending pattern differs a lot with the receiving pattern. For $i\in [n]$, let $d_{sending}(i)={\sum }_{j=1}^{n}A(i,j)$ denote the total number of edges sent by node i, and let $d_{receiving}(i)={\sum }_{j=1}^{n}A(j,i)$ denote the total number of edges that are received by node i. Call $d_{sending}(i)$ and $d_{receiving}(i)$ the sending degree and receiving degree of node i, respectively. For real-world directed networks, we find that there are many nodes whose sending degree or receiving degree are zero, and so we need the following pre-processing steps before analyzing the real data:

(a)

Set $S_{sending,0}=\{i\in [n]:d_{sending}(i)=0\}$ and $S_{receiving,0}=\{i\in [n]:d_{receiving}(i)=0\}$.

(b)

Set $S_{degree,0}=S_{sending,0}\bigcup S_{receiving,0}$.

(c)

Update A by removing the nodes in $S_{degree,0}$.

(d)

Repeat (a)–(c) until $S_{degree,0}$ is a null set.

(e)

After obtaining A through the above four steps, obtain the largest connected component of A.

We now describe the real-world directed networks analyzed in this paper:

Metabolic: This is a directed network representing the metabolic reactions of E.coli bacteria. In this data, node means metabolite, and a directed edge from node i to node j means that there is a reaction where node i is an input and node j is a product [29]. These data can be downloaded from http://networksciencebook.com/translations/en/resources/data.html. The original dat has 1039 nodes; after preprocessing, $A\in {\{0,1\}}^{893\times 893}$. To estimate K, we plot the leading 20 singular values of A, and panel (a) of Figure 4 shows the result that suggests that $K=2$ for these data, where [18] also applies the idea of an eigengap to estimate K for real-world directed networks with an unknown number of communities.

Political blogs: this is a directed network of hyperlinks between weblogs on US politics [30], and it can be downloaded from http://www-personal.umich.edu/~mejn/netdata/. Political blogs send and receive hyperlinks to and from blogs for the same political persuasion [18], so node means blog and edge means hyperlink in these data. The original network has 1490 nodes. After removing nodes with zero degrees via pre-processing steps, there are 814 nodes left; i.e., $A\in {\{0,1\}}^{813\times 813}$ for these data. Since there are two parties, “liberal” and “conservative”, K is 2 for both the sending and receiving pattern communities for these data. [18] applies their DI-SIM algorithm to the Political blogs network, assuming that all nodes have non-overlapping property. In this paper, we apply our ODCNA algorithm on these data to study its asymmetric structure on the overlapping property.

Wikipedia links (crh): This directed network represents the wikilinks of the Wikipedia website in the Crimean Turkish language (crh). Node means article, and the directed edge between two articles is the wikilink [31]. These data can be downloaded from http://konect.cc/networks/wikipedia_link_crh/. The original data have 8286 nodes. After pro-processing, $A\in {\{0,1\}}^{3555\times 3555}$. Panel (c) of Figure 4 suggests $K=2$ for this data.

Wikipedia links (dv): These data represent the wikilinks of the Wikipedia website in the Divehi language (dv), where node means article and the directed edge is a wikilink [31]. These data can be downloaded from http://konect.cc/networks/wikipedia_link_dv/. The original data has 4266 nodes. After removing nodes with zero degrees, $A\in {\{0,1\}}^{2394\times 2394}$. Panel (d) of Figure 4 suggests $K=2$ for these data.

The proportion of highly mixed nodes and ${\mathrm{Hamm}}_{rc}$ for these directed networks are reported in

Table 1. The proportion of highly mixed nodes and the asymmetric structure measured by ${\mathrm{Hamm}}_{rc}$ for real-world directed networks considered in this paper when ODCNA’s input adjacency matrix is A; i.e., the case when assuming that nodes in a sending (receiving) pattern have overlapping (non-overlapping) property.

Data	$\mathit{\tau }$	${\mathbf{Hamm}}_{\mathbf{rc}}$
Metabolic	0.1209	0.2497
Political blogs	0.0246	0.0443
Wikipedia links (crh)	0.0444	0.0307
Wikipedia links (dv)	0.4089	0.1466

when assuming that nodes in sending (receiving) clusters having an overlapping (non-overlapping) property. For the Metabolic network, the results show that the sending pattern differs a lot with the receiving pattern, since ${\mathrm{Hamm}}_{rc}=0.2497$ is quite large, and there are $893\times 0.1209\approx 108$ highly mixed nodes in the sending pattern. For the Political blogs network, there is a slight asymmetric structure between the sending pattern and the receiving pattern, since ${\mathrm{Hamm}}_{rc}=0.0443$ is small. Meanwhile, for the sending pattern of Political blogs, there are $813\times 0.0246\approx 20$ highly mixed nodes. Thus, we may conclude that though there is a slight asymmetric structure in sending and receiving patterns for Political blogs, there are 20 highly mixed nodes in the sending pattern. For the Wikipedia links (crh), they have a slight asymmetric structure between sending and receiving patterns, and there are $3555\times 0.0444\approx 158$ highly mixed nodes in the sending pattern. For the Wikipedia links (dv) network, it has a large number of highly mixed nodes for its large $\tau$, and a heavy asymmetric structure in sending and receiving patterns for its large ${\mathrm{Hamm}}_{rc}$. Generally, Table 1 suggests that if there are a large number of highly mixed nodes in the sending pattern, there is a heavy asymmetric structure between the sending and receiving patterns, and vice versa.

For visualization, we plot the sending clusters and receiving clusters detected by ODCNA for these directed networks when assuming that nodes in the sending (receiving) clusters have an overlapping (non-overlapping) property, i.e., when the input adjacency matrix of the ODCNA approach is A. The results are shown in Figure 5, Figure 6, Figure 7 and Figure 8, where we also show the highly mixed nodes in sending clusters detected by ODCNA. We see that there exists a clear asymmetric structure between the sending and receiving patterns for Metabolic and Wikipedia links (dv), as shown in Figure 5 and Figure 8, while there is a slight asymmetric structure between the sending and receiving patterns for Political blogs and Wikipedia links (crh), as shown in Figure 6 and Figure 7. Furthermore, most nodes are in the same sending (receiving) cluster for Metabolic and Wikipedia links (crh), while the two sending (receiving) clusters for Political blogs and Wikipedia links (crh) have close sizes. The results also show that most highly mixed nodes have many edges, while some highly mixed nodes have only a few edges, where such a phenomenon can be explained easily, since nodes with many edges tend to have an overlapping property, while it is difficult to detect a community for nodes with only a few edges, and ODCNA tends to treat such nodes as highly mixed nodes.

Furthermore, for real-world directed networks, since we have no prior knowledge on whether nodes in the sending pattern side or the receiving pattern side or both sides have overlapping property, simply inputting A with K sending (receiving) pattern communities in our ODCNA algorithm is not enough. To solve this problem, we also apply ODCNA on $A^{\prime }$, and the numerical results are provided in

Table 2. The proportion of highly mixed nodes and the asymmetric structure measured by ${\mathrm{Hamm}}_{rc}$ for real-world directed networks considered in this paper when ODCNA’s input adjacency matrix is $A^{\prime }$, i.e., the case when assuming that nodes in sending (receiving) pattern have non-overlapping (overlapping) property.

Data	$\mathit{\tau }$	${\mathbf{Hamm}}_{\mathbf{rc}}$
Metabolic	0.0594	0.2945
Political blogs	0.1365	0.0443
Wikipedia links (crh)	0.1308	0.0543
Wikipedia links (dv)	0.3492	0.2059

, where the results show that there also exist highly mixed nodes in the receiving pattern for these directed networks, and there also exists a heavy asymmetric structure between the sending and receiving clusters for the Metabolic and Wikipedia links (dv), while there also exists a slight asymmetric structure between the sending and receiving clusters for the Political blogs and Wikipedia links (crh).

6. Discussion

In this paper, we introduced Overlapping and Non-overlapping models and their extension, by considering the degree heterogeneity. The models can model a directed network with $K_r$ row communities and $K_c$ column communities, in which the row node can belong to multiple sending clusters, while the column node only belongs to one of the receiving clusters. The proposed models are identifiable under the case when $K_r\le K_c$, and some other popular constraints on the connectivity matrix and membership matrices. For comparison, modeling a directed network in which the row nodes have overlapping property while column nodes do not, with $K_r>K_c$, is unidentifiable. Meanwhile, since previous works have found that modeling directed networks in which both row and column nodes have an overlapping property with $K_r\ne K_c$ is unidentifiable, our identifiable ONM and ODCNM supply a gap in modeling overlapping directed networks when $K_r\ne K_c$. These models provide exploratory tools for studying community structure in directed networks with one side overlapping while another side is non-overlapping. Two spectral algorithms are designed to fit ONM and ODCNM. We also showed an estimation consistency under mild conditions for our methods. Especially, when ODCNM reduces to ONM, our theoretical results under ODCNM are consistent with those under ONM. The numerical results for the simulated directed networks generated under ONM and ODCNM support our theoretical results, and the results for real-world directed networks reveal the existence of highly mixed nodes and an asymmetric structure between the sending and receiving clusters.

The models and algorithms introduced in this paper are useful tools for studying the asymmetric structure for directed networks, and we wish that they can be widely applied in network science. However, perhaps the main limitation of the models is that $K_r$ and $K_c$ in the directed network are assumed as givens, and such a limitation also holds for the spectral clustering algorithms developed under the ScBM and DCScBM studied in [18,19,20]. In most community problems, the number of row communities and the number of column communities are unknown; therefore, a complete calculation and theoretical study requires not only the algorithms and their theoretically consistent estimations described in this paper, but also a method for estimating $K_r$ and $K_c$. A possible solution to this problem may be a combination of algorithms developed in this paper and the modularity for the directed networks developed in [32]. Meanwhile, our idea can be extended in many ways. In this paper, we only consider modeling an un-weighted directed network, and it is possible to extend our work to a weighted directed network. Our algorithms are designed based on the adjacency matrix, and it is possible to design spectral algorithms to fit ONM and ODCNM by applying the regularized Laplace matrix used in [11,12]. When detecting large-scale directed networks, the random projection-based and the random sampling-based spectral clustering ideas in [33] may be applied to accelerate our algorithms. We leave the studies of these problems to our future work.