Extreme Value Statistics for Evolving Random Networks

Abstract

Our objective is to survey recent results concerning the evolution of random networks and related extreme value statistics, which are a subject of interest due to numerous applications. Our survey concerns the statistical methodology but not the structure of random networks. We focus on the problems arising in evolving networks mainly due to the heavy-tailed nature of node indices. Tail and extremal indices of the node influence characteristics like in-degrees, out-degrees, PageRanks, and Max-linear models arising in the evolving random networks are discussed. Related topics like preferential and clustering attachments, community detection, stationarity and dependence of graphs, information spreading, finding the most influential leading nodes and communities, and related methods are surveyed. This survey tries to propose possible solutions to unsolved problems, like testing the stationarity and dependence of random graphs using known results obtained for random sequences. We provide a discussion of unsolved or insufficiently developed problems like the distribution of triangle and circle counts in evolving networks, or the clustering attachment and the local dependence of the modularity, the impact of node or edge deletion at each step of evolution on extreme value statistics, among many others. Considering existing techniques of community detection, we pay attention to such related topics as coloring graphs and anomaly detection by machine learning algorithms based on extreme value theory. In order to understand how one can compute tail and extremal indices on random graphs, we provide a structured and comprehensive review of their estimators obtained for random sequences. Methods to calculate the PageRank and PageRank vector are shortly presented. This survey aims to provide a better understanding of the directions in which the study of random networks has been done and how extreme value analysis developed for random sequences can be applied to random networks.

1. Introduction

The study of power-law real-world random networks attracts significant attention due to the many fields of their application, e.g., citation, phone-call, cellular, urban transport, and economic trade networks, among others [1,2,3]. Several monographs [4,5,6,7] have certainly contributed to basic concepts of random network theory and practice. “Real networks differ from random graphs in that often their degree distribution follows a power-law” [1]. Heavy-tailed distributions are accepted as realistic models for many phenomena in random networks and graphs. Among the recent applications, we find graphical models that are used to model extremal dependence or causality between events [8,9], including a bond percolation; see [10] among others. The distributions of various extremal characteristics of random discrete structures, such as the maximum number of common neighbors of a set of vertices in the graph, the maximum codegree, and the maximum number of cliques sharing a given vertex in binomial random hypergraphs, are derived in [11,12].

In our survey, we focus on the standard configuration model, i.e., a model of a random graph with arbitrary degree distribution [13]. Let us denote sets of nodes (or vertices) and edges as $V\left(n\right)$ and $E\left(n\right)$, respectively, their cardinality as $\parallel V\left(n\right)\parallel$, $\parallel E\left(n\right)\parallel$, and the number of edges as n. A graph is a pair of unordered sets $G\left(n\right)=\left(V\right(n),E(n\left)\right)$. Our survey concerns random graphs evolved using preferential or clustering attachment of new nodes. Considering extremes in directed graphs, recent results in [14,15] concerning the tail and extremal indices of non-stationary sums and maxima of regularly varying random variables (r.v.s) are represented in the new context of evolving directed random graphs [16]. The relation between the tail and extremal indices of the latter sums and maxima and of PageRanks and the Max-linear models is discussed in [16]. This novel approach allows us to find the tail and extremal indices of PageRanks and the Max-linear models of random graphs for attachment methods providing that the distribution tails of the latter measures are regularly varying, e.g., the preferential attachment.

Special attention is devoted to the impact of the removal of vertices and/or edges on a graph topology and the tail and extremal indices of node influence measures during evolution. An important item of the latter modeling is a partitioning of graphs into communities that constitute some node clusters. The nodes in the communities have many mutual links and a few links with other communities. Such communities may be considered independent or weakly dependent sets of nodes. The concept can be helpful for the statistical analysis of real data.

Our objective is to survey results regarding extreme value statistics and the tail and extremal indices for the graph enlargement and network evolution obtained in the literature. The tail index shows the heaviness of the distribution tail of an underlying r.v. The extremal index measures a local dependence or a cluster structure of a random sequence. It is used to determine a limit distribution of the maximum of a stationary sequence of r.v.s [17,18]. To determine the extremal index of such node influence indices as PageRank and the Max-linear model, it is proposed in [16] to consider the nodes as roots of elementary trees with the nearest neighbor nodes. Then PageRanks and the Max-linear models of the roots are calculated using sums and maxima of PageRanks of their nearest neighbors that have in-coming edges to the roots. Using Theorem 1 in [16], the tail and extremal indices of PageRanks and the Max-linear models are determined by the latter indices of the neighbor nodes belonging to the most heavy-tailed communities of the network.

The attachment of new nodes usually begins from a seed network consisting of a single node or node communities. The node indices in the communities may be heterogeneous. The nodes of the graph cannot be definitely enumerated since $V\left(n\right)$ is an unordered set. This creates an obstacle to defining the stationarity in the graphs. In [19], it is determined that “a graph is stationary if, for all finite sets of vertices with the same adjacency matrices, the joint distributions of their in- and out-degrees are the same.” Tail and extremal indices of sets of injected nodes considered in time depend both on the choice of the seed network and the attachment policy. There are several open problems, namely, (1) how to define and test the stationarity on graphs; and (2) how to determine the dependence on graphs, among other problems. We survey results that may shed light on these problems.

An application related to network evolution is given with information spreading. The survey of spreading methods is also given in the paper.

We start with our methodology and contributions in Section 2. A power-law distribution and regularly varying distributions arising in random graphs, including evolving ones, are recalled in Section 3. Regularly varying distributions of PageRanks and the Max-linear models used as node influence indices are discussed, too. How to determine and test the stationarity and dependence on random graphs is discussed in Section 4. We review preferential and clustering attachment tools and distributions of triangle counts, the topics related to the clustering coefficient, and the attachment probability, as well as a short representation of other models of random networks in Section 5. Community detection and related topics like graph coloring and anomaly detection are discussed in Section 6. In Section 7, a spreading of information in evolving graphs and its relation to a preferential attachment are considered. Conclusions and a discussion are presented in Section 8. Auxiliary materials are represented in the Appendix A, Appendix B, Appendix C and Appendix D.

2. Our Methodology and Contributions

2.1. Methodology

This survey focuses on the statistical methodology based on extreme value theory and its application to evolving random networks rather than on the structure of random graphs. We do not aim to fit graph models to the observed networks. Our objective is to find heavy-tailed phenomena on random graphs. As the node influence indices such as the in-degree, PageRank, and Max-linear model are heavy-tailed distributed, particularly, if the preferential attachment is used for evolution, and each of the latter indices can be dependent due to links between nodes, the described methods are based on the statistical theory of extremes of random sequences, an asymptotic distribution of the maximum of a finite number of r.v.s (see the Fisher–Tippet–Gnedenko’s theorem [17]).

The tail and extremal indices are key measures of the extreme value theory. To obtain the latter indices of the PageRank and Max-linear model in the evolving graphs, we use the results regarding the same indices for sums and maxima of random length non-stationary random sequences derived in [14] and extended in [15]. To this end, communities of graphs may be used as a series of r.v.s. To partition graphs into communities, one needs specific algorithms. Considering existing techniques of community detection, we pay attention to such related topics as coloring graphs and anomaly detection using machine learning algorithms based on extreme value theory.

Sometimes, it is required that the node influence indices in the community have to be stationary distributed, or communities with the smallest tail index have to be independent or weakly dependent. Then tests of the stationarity and dependence are needed.

To estimate the tail index, we use bias-reduced estimators coupled with methods to select a number of the largest order statistics related to the distribution tail.

To estimate the extremal index in random graphs, we use the interval estimator [20] that is modified in [16,21]. Nonparametric estimators of the extremal index require, as a rule, the selection of a threshold and/or another declustering parameter, e.g., a block size [17]. A rather simple interval estimator needs only the choice of the threshold and demonstrates a good accuracy.

Preferential and clustering attachments are considered to model the graph evolution. The finding of leading communities for a fast information spreading presented in the survey is based on methods proposed in [22,23].

2.2. Contributions

This survey is an attempt to summarize the achievements of many authors regarding extremes in random graphs and to provide a structured overview of related topics like community detection and attachment methods for graph evolution that are needed for our approaches. Our achievements concern topics like the network classification and the choice of leading communities using tail indices of communities, the tail and extremal indices of PageRank, and the Max-linear model for evolving graphs with and without node and edge deletion.

The survey tries to propose possible solutions to unsolved problems, like a testing of stationarity and dependence of random graphs using known results obtained for random sequences. We provide a discussion of unsolved or not sufficiently developed problems like the distribution of triangle and circle counts in evolving networks, or the clustering attachment and the local dependence of the modularity, the impact of node or edge deletion at each step of the evolution on extreme value statistics, among many others. Tail index estimators derived for random sequences are proposed to be adapted to random graphs. We also suggest machine learning methods for anomaly detection as partition methods for community detection.

3. Heavy-Tailed Distributed Node Influence Indices

3.1. Definitions

Let us consider the in- and out-degrees as well as PageRanks and the Max-linear models as node influence measures.

Let us begin with definitions. We recall that a discrete r.v. X exhibits a power-law distribution if

$$\mathrm{P}\left(X=k\right)\sim C{k}^{-(1+\iota )},k\to \infty ,$$

(1)

holds for some positive constants C and $\iota$ ($a_n\sim b_n$, as usual, means that the sequences $a_n$ and $b_n$ are asymptotically equal, i.e., $a_n/b_n\to 1$, $n\to \infty$).

From (1) it follows that ${\sum }_{k=n+1}^{\infty }\mathrm{P}(X=k)\sim (C/\iota )n^{-\iota }$, as $n\to \infty$; see, e.g., Ref. [24] for details. This fact, together with (1), implies that the power-law distribution satisfies a von Mises-type condition

$$\underset{n\to \infty }{lim}\frac{n\mathrm{P}(X=n)}{{\sum }_{k=n+1}^{\infty }\mathrm{P}(X=k)}=\iota .$$

Thus, using Theorem 3 in [25], a discrete distribution satisfying (1) belongs to the class of distributions whose right tail $\overline{F}\left(x\right):=1-F\left(x\right)$, where $F\left(x\right)$ denotes the cumulative distribution function and is regularly varying at infinity with index $-\iota$ (the notation is $\overline{F}\in {\mathrm{RV}}_{-\iota }$), i.e.,

$$\underset{t\to \infty }{lim}\frac{\overline{F}\left(tx\right)}{\overline{F}\left(t\right)}=x^{-\iota }$$

for all $x>0$. If $\overline{F}\in {\mathrm{RV}}_{-\iota }$, then it is always possible to represent $\overline{F}\left(x\right)$ as $\ell \left(x\right)x^{-\iota }$, where $\ell \left(x\right)$ is a slowly varying function ($\ell \left(x\right)\in {\mathrm{RV}}_0$), i.e., by definition ${lim}_{x\to \infty }\ell \left(tx\right)/\ell \left(x\right)=1$ for any $t>0$, see e.g., p. 13 in [26].

Using [18], p. 67, a stationary sequence ${\left\{X_n\right\}}_{n\ge 1}$ has an extremal index $\theta \in [0,1]$ if for each $0<\tau <\infty$ there is a sequence of real numbers $u_n=u_n\left(\tau \right)$ such that

$$\begin{array}{ccc}\hfill & & \underset{n\to \infty }{lim}n(1-F\left(u_n\right))=\tau ,\underset{n\to \infty }{lim}P\{M_n\le u_n\}=e^{-\tau \theta }\hfill \end{array}$$

(2)

holds for $M_n=max\{X_1,\dots ,X_n\}$. Independent identically distributed (i.i.d.) r.v.s $\left\{X_n\right\}$ provide $\theta =1$. The converse may be incorrect. The reciprocal of $\theta$ approximates a mean cluster size, i.e., the mean number of exceedances over a threshold per cluster. Then, the cluster of exceedances may imply a block of data with at least one extreme observation (an exceedance) over a high threshold [17].

PageRank vector $R=(R_1,\dots ,R_n)$ used using search engine Google is determined as the unique solution to the system of linear equations [27]

$$\begin{array}{ccc}\hfill R_i& =& c\sum _{j:(j,i)\in E_n}\frac{R_j}{D_j}+\frac{c}{n}\sum _{j\in \mathfrak{D}}{R}_j+(1-c)q_i,i=1,\dots ,n.\hfill \end{array}$$

(3)

Here, the first sum is taken over pages j that are linked to page i (the in-degree). $D_j$ is the number of outgoing links of page j (the out-degree). $\mathfrak{D}$ is a set of dangling nodes, i.e., nodes without outgoing edges. $c\in (0,1)$ is a damping factor. $q=(q_1,q_2,\dots ,q_n)$ is a personalization probability vector (user’s preferences). It holds $q_i\ge 0$ and ${\sum }_{i=1}^n{q}_i=1$. n is the total number of pages. The World Wide Web (WWW) is a huge interconnected graph where nodes imply pages. The PageRank was designed to rank pages on the Web in such a way that a page is important if many important pages have a hyperlink to it [28].

A Max-linear model [8] is obtained using the following expression

$$\begin{array}{ccc}\hfill R_i& =& \underset{j:(j,i)\in E_n}{\bigvee }{A}_j{R}_j\vee Q_i,i=1,\dots ,n,\hfill \end{array}$$

(4)

where $\left\{Q_i\right\}$ are independent continuous r.v.s.

3.2. Power-Law of In- and Out-Degrees

Power-law distributions with different exponents of a node’s in- and out-degree in evolving networks are approved by numerous theoretical and empirical studies [1]. A preferential attachment (PA) (see details in Section 5.1) is suggested to explain conjectured power-law degree distributions in real networks [2]. Tail indices of in- and out-degrees and the power-law model of their asymptotic distributions were obtained in [29,30] depending on parameters of specific linear PA schemes named $\alpha -$, $\beta -$ and $\gamma -$ schemes that are used for growing networks. Namely, these tail indices are obtained as

$$\begin{array}{ccc}\hfill {\alpha }_{in}& =& \frac{1+{\delta }_{in}(\alpha +\gamma )}{\alpha +\beta },{\alpha }_{out}=\frac{1+{\delta }_{out}(\alpha +\gamma )}{\beta +\gamma },\hfill \end{array}$$

(5)

where nonnegative parameters of the PA schemes $(\alpha ,\beta ,\gamma ,{\delta }_{in},{\delta }_{out})$ are such that $\alpha +\beta +\gamma =1$, see Section 5.1 and formula (2.9) in [30]. A joint distribution of the in- and out-degrees is derived to have jointly regularly varying tail [29]. As far as we know, the tail index of the PageRank is not yet obtained in the literature. In [31], the existence of the limit of a graph in a local weak convergence sense was derived for directed graphs, but a power-law tail of the PageRank of the root in the limiting graph (that is not a branching tree) is not yet proven.

The distributions of in-degree and the PageRank are found to be similar [32]. Assuming the weak degree correlations in the Web graph, the PageRank vector may be approximated using the node in-degrees sufficiently accurate [32]. A statistical analysis of Web graphs has shown that distribution tails of the in-degree and the PageRank are a power-law (1) with the same exponent $\iota \approx 1.1$, Ref. [33]. The in-degree is available statistics that are gathered using a search engine. Then it is easier to use the in-degree rather than the PageRank in practice.

The node degree exponent $\iota$ of several networks was found to satisfy $\iota \in (2,3)$ [34]. Uniform random graphs with specified degree sequences are commonly used to model (power-law) real-world networks (see, Refs. [35,36] for a survey). In [37], a graph Hamiltonian is proposed as a method to model heterogeneous clustered graphs.

3.3. Regularly Varying Distributions of PageRanks and Max-Linear Models

3.3.1. PageRank and Max-Linear Model as Solutions of Fixed-Point Problems

Heavy-tailed regularly varying limit distributions of PageRank were derived in [38,39], for static networks without any evolution in time. A similar result has been derived in [40] for the Max-linear model.

The latter papers exploit a probabilistic approach to study (3). A Web page that is randomly chosen in a Web graph is then represented as a root of a Galton–Watson tree with random in- and out-degrees. Its PageRank can be modeled as an r.v. R. The latter is the solution of the following fixed-point problem [28,38,39,41]

$$R{=}^D\sum _{j=1}^N{A}_j{R}_j+Q.$$

(6)

In [39], $\left\{R_j\right\}$ are assumed to be i.i.d. copies of R. A personalization value of the vertex is assumed to have a bounded expectation $E\left(Q\right)<1$. N means the in-degree. All r.v.s in the triple $(Q,N,\left\{A_j\right\})$ are real-valued. ${=}^D$ denotes equality in distribution. The $A_i^{\prime }$s are random coefficients that equal $c/D_i$ in [39]. $D_i$ are the node out-degree such that $D_i\ge 1$ Let us recall the following Assumptions A. $\left\{R_j\right\}$ are i.i.d. and independent of $(Q,N,\left\{A_j\right\})$ with $\left\{A_j\right\}$ independent of $(N,Q)$. N and Q can be mutually dependent. Then it is stated in [38,39] that the stationary distribution of R in (6) is stated to be regularly varying, and its tail index is determined using the most heavy-tailed distributed term in the regularly varying distributed pair $(N,Q)$. Considering iterations of the PageRank, the initial distribution of $R^{\left(0\right)}$ impacts on the heaviness of the tail of the kth iteration $R^{\left(k\right)}$ of the PageRank (see, Ref. [39], Theorem 3.2).

The latter results were extended in [42]. The unique solution of (6) is proved to be intermediate regularly varying, if Q or N has an intermediate regularly varying distribution, or (Q, N) has a two-dimensional regularly varying distribution. The class of intermediate regularly varying distributions such that ${lim}_{\alpha \uparrow 1}{lim\; sup}_{x\to \infty }\overline{F}\left(\alpha x\right)/\overline{F}\left(x\right)=1$ includes regularly varying distributions. The multivariate version of (6)

$$\begin{array}{ccc}& & R\left(i\right){=}^D\sum _{k=1}^K\sum _{m=1}^{N^{\left(k\right)}\left(i\right)}{R}_m\left(k\right)+Q\left(i\right),\hfill \end{array}$$

where $R_m\left(k\right){=}^{D}R\left(k\right)$ holds, is considered subject to similar independence assumptions and regularly varying statements, assuming that $N^{\left(k\right)}\left(i\right)$ is a number of type-k children of a type-i ancestor and considering a multi-type Galton–Watson tree.

Under Assumptions A the power-law tail $P\left\{\right|R|>x\}\sim H{x}^{-\iota }$, $\iota >0$, $H>0$ as $x\to \infty$ of the so-called “minimal/endogeneous” solution of the equation

$$R{=}^D\left(\underset{j=1}{\overset{N}{\bigvee }}{A}_j{R}_j\right)\vee Q,$$

(7)

is derived in [40] without a specification of $\iota$.

3.3.2. PageRank and the Max-Linear Model as Sums and Maxima of Non-Stationary Sequences of Random Lengths

The reciprocal of the extremal index of random sequences is the limiting mean cluster size in the point process of exceedance times over a high threshold, i.e., the mean number of exceedances per cluster. The cluster may imply a block of data with at least one extreme observation (an exceedance) over a sufficiently high threshold [17].

The random lengths sum and maximum at the right-hand sides of (6) and (7) were considered in [16] omitting the independence Assumptions A. Based on results in [14,15], it is derived in Theorem 1 of Ref. [16] that the tail and extremal indices of PageRank and the Max-linear model of newly appended nodes in evolving networks are determined using the latter indices of the most heavy-tailed community(ies), i.e., the communities with a minimum tail index. This result also allows us to classify newly appended nodes. Ranking the communities in ascending order of their tail indices, a new node is assigned to class i if it has at least one link to nodes of the community i. If the new node has links to different communities, then the i is a minimum number. In terms of citation networks, it means, for instance, that PageRank and the Max-linear model of a set of books cited by papers from the “dominating” communities having the minimum tail index inherit the same minimum tail index if the communities are independent or weakly dependent [16]. It is reasonable that the random number of “dominating” communities be bounded. There exists a “top” community among the “dominating” ones such that its maximum PageRank is the largest. Then the Max-linear models of the cited papers published earlier have the extremal index of the “top” community. If the communities with the minimum tail index are independent or weakly dependent, then PageRanks of the cited papers have the same extremal index as the “top” community. If the “dominating” community is unique, then PageRanks and the Max-linear models of the cited papers have its tail and extremal indices.

In [15,16], $\{Y_{n,i}:n,i\ge 1\}$ is a double-indexed array of nonnegative r.v.s such that the “row index” n indicates the time, and the “column index” i enumerates the series. The length $N_n$ of “row” sequences $\{Y_{n,i}:i\ge 1\}$ for each n is random. For each i the “column” sequence $\{Y_{n,i}:n\ge 1\}$ is assumed to be strict-sense stationary with extremal index ${\theta }_i$ and having a regularly varying distribution tail

$$\begin{array}{ccc}\hfill P\{Y_{n,i}>x\}& =& {\ell }_i\left(x\right)x^{-k_i}\hfill \end{array}$$

with tail index $k_i>0$ and a slowly varying function ${\ell }_i\left(x\right)$. There are no assumptions about the dependence structure in i. For a better understanding, we represent the matrix of $\{Y_{n,i}:n,i\ge 1\}$ as

$$\left(\begin{array}{cccccc}c{Y}_{1,1}& c{Y}_{1,2}& c{Y}_{1,N_1}& \dots & 0& 0\\ c{Y}_{2,1}& 0& c{Y}_{2,3}& \dots & c{Y}_{n,N_2}& 0\\ \dots & \dots & \dots & \dots & \dots & \dots \\ 0& c{Y}_{n,2}& c{Y}_{n,3}& \dots & 0& c{Y}_{n,N_n}\end{array}\right),$$

(8)

$$\begin{array}{c}\hfill \left(\begin{array}{cccccc}{k}_1& k_2& k_3& \dots & \dots & k_{N_n}\\ {\theta }_1& {\theta }_2& {\theta }_3& \dots & \dots & {\theta }_{N_n}\end{array}\right),\end{array}$$

where $\left\{k_i\right\}$ and $\left\{{\theta }_i\right\}$ denote the tail and extremal indices of the “column” series in (8). Ref. [14] contains a simple result concerning the weighted sums and maxima over the “row” random-length sequences

$$\begin{array}{ccc}\hfill & & Y_n^{*}(z,N_n)=max(z_1{Y}_{n,1},\dots ,z_{N_n}{Y}_{n,N_n}),\hfill \\ & & Y_n(z,N_n)=z_1{Y}_{n,1}+\dots +z_{N_n}{Y}_{n,N_n},z_1,z_2,\dots >0.\hfill \end{array}$$

If the “column” series with a minimum tail index is unique, let us say, $k_1$, then the tail and extremal indices of both $Y_n^{*}(z,N_n)$ and $Y_n(z,N_n)$ are equal to $k_1$ and ${\theta }_1$.

A random number d of the “column” series with a minimum tail index, which is plausible in random graphs, is considered in [15], Theorem 4. If the d “column” series are mutually independent, and independent of the remaining “columns,” or weakly dependent (assumptions (A1) and (A2) in [15], respectively), and $N_n$ and $\left\{Y_{n,i}\right\}$ are independent, then $Y_n^{*}(z,N_n)$ and $Y_n(z,N_n)$ have the same tail index $k_1$. Let us recall the latter assumptions for a fixed $d>1$ proposed in [15].

(A1)

The stationary sequences ${\left\{Y_{n,i}\right\}}_{n\ge 1}$, $i\in \{1,\dots ,d\}$ are mutually independent, and independent of the sequences ${\left\{Y_{n,i}\right\}}_{n\ge 1}$, $i\in \{d+1,\dots ,l_n\}$.

(A2)

Assume ${\left\{Y_{n,i}\right\}}_{n\ge 1}$, $i\in \{1,\dots ,d\}$ satisfy the following conditions as $x\to \infty$

$$\begin{array}{ccc}\hfill \frac{P\{Y_{n,i}>x\}}{x^{-k_1}{\ell }_1\left(x\right)}& \to & c_i,i\in \{1,\dots ,d\},\hfill \end{array}$$

for some non-negative numbers $c_i$,

$$\begin{array}{ccc}\hfill \frac{P\{Y_{n,i}>x,Y_{n,j}>x\}}{x^{-k_1}{\ell }_1\left(x\right)}& \to & 0,i\ne j,i,j\in \{1,\dots ,d\}.\hfill \end{array}$$

By assumption (A4) in [15,16] there exists $i\in \{1,\dots ,d\}$ such that it holds

$$\begin{array}{ccc}\hfill & & P\{\underset{1\le j\le d,j\ne i}{max}\left(z_j{M}_n^{\left(j\right)}\right)>u_n,z_i{M}_n^{\left(i\right)}\le u_n\}=o\left(1\right),n\to \infty ,\hfill \end{array}$$

(9)

where

$$\begin{array}{ccc}\hfill M_n^{\left(i\right)}& =& max\{Y_{1,i},Y_{2,i},\dots ,Y_{n,i}\},i\in \{1,..,l_n\},n\ge 1\hfill \end{array}$$

denotes the maximum over the ith “column” of (8). If (A4) holds, then $Y_n^{*}(z,N_n)$ has the extremal index ${\theta }_i$. $Y_n(z,N_n)$ has the same extremal index if, in addition to (A4), (A1) or (A2) holds. The (A4) is valid for all d “column” series such that

$$\begin{array}{ccc}\hfill & & M_n^{\left(1\right)}\le M_n^{\left(2\right)}\le \dots \le M_n^{\left(d\right)}\hfill \end{array}$$

(10)

holds.

In [16], the results obtained in [14,15] are applied to evolving random networks. To this end, the evolution starts from a seed network that is represented by matrix (8). The following recursions

$$\begin{array}{ccc}\hfill Y_{i,j}^{\left(m\right)}& =& c\sum _{s=j}^{N_i}{Y}_{i,s}^{(m-1)}+Q_i,\hfill \end{array}$$

(11)

$$\begin{array}{ccc}\hfill X_{i,j}^{\left(m\right)}& =& \left(c\underset{s=j}{\overset{N_i}{\bigvee }}{X}_{i,s}^{(m-1)}\right)\vee Q_i,\left\{X_{i,j}^{\left(0\right)}\right\}\equiv \left\{Y_{i,j}^{\left(0\right)}\right\},\hfill \end{array}$$

(12)

$m,i,j\ge 1$, where m is connected with the time, are considered. $Y_{i,j}^{\left(m\right)}$ and $X_{i,j}^{\left(m\right)}$ build the jth “column” series of the next generation matrix $A^{\left(m\right)}$ using $A^{(m-1)}$ starting by $A^{\left(0\right)}$. It may be, for instance, the following

$$\begin{array}{ccc}\hfill & & A^{\left(0\right)}=\left(\begin{array}{cccccc}{Y}_{1,1}^{\left(0\right)}& Y_{1,2}^{\left(0\right)}& Y_{1,3}^{\left(0\right)}& \dots & 0& Q_1\\ Y_{2,1}^{\left(0\right)}& 0& Y_{2,3}^{\left(0\right)}& \dots & Y_{2,N_2}^{\left(0\right)}& Q_2\\ \dots & \dots & \dots & \dots & \dots & \dots \\ Y_{n,1}^{\left(0\right)}& Y_{n,2}^{\left(0\right)}& Y_{n,3}^{\left(0\right)}& \dots & Y_{n,N_n}^{\left(0\right)}& Q_n\end{array}\right),\hfill \end{array}$$

$$\begin{array}{ccc}\hfill & & \left(\begin{array}{cccccc}{k}_1^{\left(0\right)}& k_2^{\left(0\right)}& k_3^{\left(0\right)}& \dots & k_N^{\left(0\right)}& k_{N+1}^{\left(0\right)}\\ {\theta }_1^{\left(0\right)}& {\theta }_2^{\left(0\right)}& {\theta }_3^{\left(0\right)}& \dots & {\theta }_N^{\left(0\right)}& 1\end{array}\right).\hfill \end{array}$$

$\left\{Q_i\right\}$ is a sequence of i.i.d. r.v.s. Hence, its extremal index is equal to 1.

Network communities as “column” series of ${\mathit{A}}^{\left(\mathbf{0}\right)}$

Network communities may be treated as columns of $A^{\left(0\right)}$.

Since the communities have random lengths, the columns in matrix $A^{\left(0\right)}$ may be supplemented with zeros. A zero sth element in the ith row $Y_{i,s}^{\left(0\right)}$, $s\ge 1$ of $A^{\left(0\right)}$ implies that the ith root node has no followers in the sth community or there is no link between them; see Figure 1a. For example, if a row corresponds to a set of papers citing a book, then zero implies that the book is not cited by a paper from the corresponding community. Using Theorem 4 in [15], it is provedn in Theorem 1 in [16] that sums and maxima $Y_{i,j}^{\left(m\right)}$ and $X_{i,j}^{\left(m\right)}$ at the mth step of the evolution inherit the corresponding tail and extremal indices of the most heavy-tailed “column” series of $A^{\left(0\right)}$ for any $m\ge 1$ under specific conditions. The conditions are weaker for maxima over rows of $A^{\left(0\right)}$ than for sums. The maxima require the condition (A4) (see, Formula (9)) that is valid for all d the most heavy-tailed “column” series such that (10) holds. Considering the latter communities with the same minimum tail index (that are not necessarily distributed the same way) as the “columns” and taking into account that their order does not matter, one can conclude that (10) is always fulfilled for maxima over communities.

Sums over rows of $A^{\left(0\right)}$ require the independence or weak dependence of the most heavy-tailed “column” series (see, namely, conditions (A1) and (A2) in [15,16]) additionally to (10). The number of the most heavy-tailed “column” series may be random. Theorem 1 in [16] is valid assuming that each row of $A^{\left(0\right)}$ contains at least one nonzero element in the columns with the minimum tail index, i.e., the columns with the most heavy-tailed distribution tail. Otherwise, the sums and maxima over rows will be non-stationary distributed with different tail indices. Theorem 1 in [16] states that the limit distributions of the recursions (11) and (12) are determined using the distributions of columns $Y_{n,i}^{\left(0\right)}:n,i\ge 1$. The proposed approach is valid for any attachment method that leads to PageRanks with regularly varying distribution tails, particularly, for a preferential attachment.

Example 1. 

The approach in [16] may be interpreted in terms of citation networks. Namely, if newly published papers cite at least one paper from the most heavy-tailed communities published earlier, then the Max-linear models of the new papers have the same tail and extremal indices of such communities. If communities are weakly connected or disconnected, then the same result is valid for the PageRanks of the new papers.

Network communities as “row” series of ${\mathit{A}}^{\left(\mathbf{0}\right)}$

From another side, network communities may be interpreted as rows of $A^{\left(0\right)}$; see, Figure 1a in [19]. Such an approach has the advantage that sums and maxima over communities may be considered independent or weakly dependent r.v.s. Testing of the dependence on graphs is discussed in Section 4.3. The independent communities simplify a tail index estimation of sums and maxima over the communities. Let us recall that the communities are random length sub-graphs. The communities are allowed to be non-homogeneous. Namely, their node PageRanks may be non-stationary distributed in contrast to the previous case when the communities representing the most heavy-tailed “column” series of $A^{\left(0\right)}$ are to be stationary distributed.

Using a simulation study, it was found in [19] that the tail indices of the sums and maxima over communities are close to the minimum tail index of the representative series extracted from the communities. The representative series can be formed by taking one of the nodes of a community as a representative. To avoid numerous combinations, the ith representative series was chosen using the ith PageRank maxima within each community. Since communities have a random size, some nodes may fall into several representative series, leading to their dependence. In case of different pair-wise dependency among elements of the $d>1$ “column” series (the representative series) with the minimum tail index, the sums and maxima calculated over rows (communities) may be non-stationary distributed [15]. The different pair-wise dependence between the members of the most heavy-tailed communities is very restrictive and indefinite since nodes in the communities are unordered. In practice however, the representative series with a minimum tail index estimate is likely unique. Thus, the mentioned pair-wise dependency is not required. The results of the empirical study in [19] are in agreement with theoretical results in [14,15].

4. Stationarity and Dependence on Graphs

4.1. Interpretation of Stationarity on Graphs

In Section 3.3.2, the communities defined as the “column” sequences of matrices (8) (or $A^{\left(0\right)}$) have to be stationary distributed. Moreover, it is important to understand whether there are communities with the same tail index. The same issue may concern to representative series. The following questions arise. How can one interpret the stationarity on random graphs? How does one test a deviation in the tail indices?

One of the ideas to check non-stationarity is to use data blocks. This approach is usually applied to random sequences in different applications (see, e.g., Ref. [43]). To apply this idea to random graphs, one can partition communities into subcommunities (blocks) and estimate their tail indices, for instance, by methods recalled in Appendix A. Since the number of communities is random, one can apply a random graph coloring and stochastic decomposition methods of the symmetric adjacency matrix derived from the underlying undirected graph; see [44,45,46] among others (see, Section 6).

Another idea is to use stationary distributed Markov chains like the Metropolis algorithm (see, Refs. [47,48] among others) as sampling tools over networks and to test the stationarity of the obtained random sequences of node indices in a classical way.

For layered networks considered in [49], the homogeneity for networks implies that the replacing in the location and layer would lead to the same in-degree distribution.

4.2. Testing of a Change in the Tail Index among Communities

Here, we describe some tests applied to i.i.d. non-homogeneous random sequences containing observations with different tail indices. The latter tests can be related to random graphs within our vision.

In [43], an extreme value analysis for independent but non-identically distributed observations is considered. Assuming continuously changing extreme value indices (EVIs), authors provide a non-parametric estimator for the EVI functional $\gamma \left(s\right)$, $s\in [0,1]$, based on the observations $X_1,\dots ,X_n$. The (positive) EVI constitutes the reciprocal of the tail index. The main idea in [43] is the following. The sample $X_1,\dots ,X_n$ is divided into blocks

$$X_{\left[\right(i-1\left)nh\right]+1},\dots ,X_{\left[inh\right]},i=1,2,\dots ,[1/\left(2h\right)],$$

where each block contains $\left[2nh\right]$ observations. $[·]$ denotes the integer part and $h=h\left(n\right)$ is a bandwidth such that $h\to 0$, $nh\to \infty$ as $n\to \infty$. To estimate $\gamma \left(s\right)$, $s\in [0,1]$ locally, Hill’s estimator [50] is applied to each of the $[1/(2h\left)\right]$ blocks. Let ${\widehat{\gamma }}_i$ denote the Hill estimate associated with the i-th block. Then the function $\mathsf{\Gamma }\left(s\right)={\int }_0^s\gamma \left(u\right)\mathrm{d}u$ is estimated by aggregating the local estimators ${\widehat{\gamma }}_i$, $i=1,2,\dots ,[1/(2h\left)\right]$ as follows:

$$\widehat{\mathsf{\Gamma }}\left(s\right)=2h\sum _{1\le i\le 1/\left[2h\right]:i\left[2h\right]\le s}{\widehat{\gamma }}_i,s\in [0,1].$$

The global estimator $\widehat{\mathsf{\Gamma }}\left(s\right)$ can be used to test a pre-specified parametric trend in the EVIs, see [43] for details.

Let

$$\begin{array}{ccc}\hfill {\mathcal{I}}_n& =& \left\{I_n\left(v\right)\right\},{\mathcal{O}}_n=\left\{O_n\left(v\right)\right\},v\in V\left(n\right)\hfill \end{array}$$

(13)

denote the sets of the in- and out-degrees of a directed graph $\left(V\right(n),E(n\left)\right)$ at a time n. Due to the unorderedness of the elements of the sets ${\mathcal{I}}_n$ and ${\mathcal{O}}_n$, the estimation of the functional tail index $\alpha \left(s\right)=1/\gamma \left(s\right)$, $s\in [0,1]$ provided in [43] cannot be applied to random graphs directly. However, the idea can be useful in the statistic of directed graphs. Let $d(u,v)$ denote the distance in terms of the number of links between the nodes u and v. We define the h-neighborhood of node $v\in V\left(n\right)$ as follows

$${\mathcal{B}}_n\left(v\right)=\{u\in V\left(n\right):d(u,v)<h·N\left(n\right)\},h\in (0,1],$$

where $N\left(n\right)$ is the number of nodes in the network at time n. If the cardinality of a set ${\mathcal{B}}_n\left(v\right)$ is large enough (let us say, at least 500), we may obtain an estimate of the tail index $\alpha \left(v\right)$ based on $\left\{I_n\left(u\right),u\in {\mathcal{B}}_n\left(v\right)\right\}$. The set of estimates $\{\widehat{\alpha }\left(v\right),v\in V\left(n\right)\}$ built by disjoint sets ${\mathcal{B}}_n\left(v\right)$ can be used for a preliminary analysis of the tail index behavior.

Assuming that the breakpoint of the tail shape behavior is known, a test for the null hypothesis that the tail index is constant over time is constructed in [51]. Considering the case with a single known breakpoint m, we have ”split” observations

$$X_1,\dots ,X_m,X_{m+1},\dots ,X_n$$

where $X_1,\dots ,X_m$ are i.i.d. r.v.s from a distribution $F_1$, for which ${\overline{F}}_1\in {\mathrm{RV}}_{-{\alpha }_1}$, while $X_{m+1},\dots ,X_n$ are i.i.d. r.v.s from a distribution $F_2$, which satisfies ${\overline{F}}_2\in {\mathrm{RV}}_{-{\alpha }_2}$. Let us adopt below a test of [51] to directed random graphs. It is worth noting that this test can be adopted for undirected random graphs as well.

Let $(V{\left(n\right)}^{\prime },E{\left(n\right)}^{\prime })$ and $\left(V\left(n\right){,}^{″}E{\left(n\right)}^{″}\right)$ be two non-intersected communities of $\left(V\right(n),E(n\left)\right)$ with the corresponding sets of in-degrees ${\mathcal{I}}_n^{\prime }=\{I_n\left(v\right),v\in V{\left(n\right)}^{\prime }\}$ and ${\mathcal{I}}_n^{″}=\{I_n\left(v\right),v\in V{\left(n\right)}^{″}\}$. In particular, the null hypothesis is

$$H_0:{\alpha }_1={\alpha }_2=\alpha ,$$

where ${\alpha }_1$ and ${\alpha }_2$ are the tail indices of the in-degrees in ${\mathcal{I}}_n^{\prime }$ and ${\mathcal{I}}_n^{″}$, respectively. A modified statistic of Phillips and Loretan

$$\begin{array}{ccc}\hfill S& =& \frac{k_1^{*}{\left({\widehat{\alpha }}_2\right)}^2{\left({\widehat{\alpha }}_1/{\widehat{\alpha }}_2-1\right)}^2}{{\left({\widehat{\alpha }}_1\right)}^2+\left(k_1^{*}/k_2^{*}\right){\left({\widehat{\alpha }}_2\right)}^2}\hfill \end{array}$$

(14)

is represented in [52]. Here, ${\widehat{\alpha }}_1$ and ${\widehat{\alpha }}_2$ are estimates of the tail index computed using Hill’s estimator ${\widehat{\alpha }}_{in}^{\left(i\right)}\left(k\right)$, $i\in \{1,2\}$ from observations ${\mathcal{I}}_n^{\prime }$ (with the optimal choice $k=k_1^{*}$ of the k largest order statistics) and ${\mathcal{I}}_n^{″}$ (with the optimal choice $k=k_2^{*}$), respectively. The tail index estimation is described in Appendix A. Under the null hypothesis, statistic S converges in distribution to an r.v. that is chi-squared distributed with one degree of freedom. For example, at the $5\%$ level of significance, the rejection region would be the interval $[3.841,+\infty )$.

One can calculate nodes’ PageRanks (see, Appendix C) of communities for directed graphs or node degrees for undirected ones instead of the in-degrees, and then check the null hypothesis for pairs of the communities.

Note that the Hill estimator is weakly consistent for i.i.d. data; see [53]. The Hill and the ratio estimators may be applied to dependent data [54,55]. We recall that the ratio estimator introduced in [56] is a generalization of the Hill estimator (see Appendix A). The asymptotic behavior of the Hill estimator when independent observations are from non-identical distributions with comparable tails was discussed in [57]. The discussion of the tail index estimation in the case of dependent and non-stationary data can be found in [58]. There, in particular, a tail index estimator is proposed, whose construction is based on the block maxima method. The latter is weakly consistent when observations are from a non-stationary moving maxima model. One more application of the block maxima method for tail index estimation can be found in [59].

Although node indices within the communities are not i.i.d., the Hill estimation still works in practice; see [19,30]. Its consistency for undirected graphs is proven in [60], but the result regarding directed graphs remains, to the best of our knowledge, an unresolved problem.

4.3. Testing of Dependence on Graphs

Let us recall some methods which may be used to test the dependence of node influence indices of two communities.

The $\rho$-correlated Erdős–Rényi (ER) model was proposed to capture correlations between ER graphs in [61]. A random graph ER where an edge is sampled in i.i.d. fashion from a Bernoulli distribution with some parameter p is denoted as $ER\left(p\right)$. One can calculate a Pearson’s correlation coefficient of two r.v.s belonging to two graphs. $G_{ij}$ is a r.v. denoting whether there is an edge between nodes i and j in graph G. Two graphs G and H are called $\rho$-correlated $ER(p,q)$ if G is $ER\left(p\right)$, H is $ER\left(q\right)$, and the r.v.s $G_{ij}$ and $H_{ij}$ have Pearson’s correlation

$$\rho =\frac{P\{G_{ij}=1,H_{ij}=1\}-pq}{\sqrt{p(1-p)q(1-q)}}$$

for all $\{i,j\}\in \left(\genfrac{}{}{0pt}{}{\left[n\right]}{2}\right)$, where $\left[n\right]$ denotes an integer part, $\left(\genfrac{}{}{0pt}{}{\left[n\right]}{2}\right)$ denotes a binomial coefficient. The null hypothesis of the graph independence test is $\rho =0$, and the alternative is $\rho \ne 0$.

The Stochastic Block Model (SBM) generalizes ER graphs [4]. Using the SBM, which is parameterized by the block probability matrix $B\in {[0,1]}^{k\times k}$, where k is the number of blocks, one can check the dependence between two communities [62]. For $k=1$ a SBM is ER. A block refers to a submatrix in the adjacency matrix formed by the edges connecting every node in community i to every node in community j. The entry $B_{i,j}$ gives the probability of an edge from a node in community i to a node in community j, for all $i,j\in \{1,\dots ,k\}$. The prior estimate of the number of communities can be taken as k. The choice of k is a sensitive point of the approach. One can generalize the $\rho$-correlated ER to the $\rho$-correlated SBM.

In [63], it is shown that the sample Pearson correlation coefficient fails to capture linear dependence between two random variables when their variances are infinite, which is realistic for real-world networks. In such networks, it is important to measure degree–degree dependencies for neighboring vertices. Examples where the Pearson coefficient converges to zero in a network with strong negative degree-degree dependencies and an example where this coefficient converges in distribution to an r.v. are provided. It is also shown that Spearman’s $\rho$ coefficient is able to reveal strong dependencies in large graphs.

A distance correlation is an extension of Pearson’s correlation both to linear and nonlinear associations between two r.v.s or random vectors [64]. It takes values in $[0,1]$. The independence corresponds to the distance correlation equal to zero. The distance correlation is applied with a permutation test to check the dependence hypothesis because the numeration of nodes is undefined. The distance correlation is calculated first for an original pair of vectors. It is compared with those calculated with shuffles of these vectors. Under a $\rho$-correlated SBM, a naive permutation and Pearson’s test for a conditional dependency graph model is shown to be invalid in [64]. As an alternative, a block-permutation procedure is proposed. The procedure is proved to be valid and consistent—even when the two graphs have different marginal distributions.

One can measure dependencies in data of a power-law graph using statistical inference for multivariate regular variation. To this end, the polar coordinate transform to the examined random vectors $\left\{X_i\right\}$ and $\left\{Y_i\right\}$, $i=1,\dots ,n$ can be applied [30,55,65]. The empirical distribution function (edf) of the angular coordinates for the k largest values of the radial coordinate is calculated. The total dependence (or total independence) corresponds to the concentration of the edf to $\pi /4$ (or, to 0 or $\pi /2$). In the case of bivariate data, a Resnick–Starica plot built with radii can be used to find a suitable value of k.

To detect the dependence between in- and out-degrees a polar coordinate transform

$$\begin{array}{ccc}\hfill (I^a,O)& \to & \left(\sqrt{I^{2a}+O^2},arctan(O/I^a)\right)=(R,T)\hfill \end{array}$$

is applied in [30] if the limiting random vector $(I,O)\sim F$ is jointly regularly varying. Here, $a={\tau }_{in}/{\tau }_{out}$ is estimated in practice using the estimation of tail indices ${\tau }_{in}$ and ${\tau }_{out}$ of in- and out-degrees. The conditional distribution $P\{T\in ·|R>r\}\to S(·)$ converges weakly to the angular measure $S(·)$. The edf of $S(·)$ is calculated using the sample angles $T_n\left(v\right)=arctan(O_n\left(v\right)/I_n{\left(v\right)}^{\widehat{a}})$ for vertices $v=1,2,\dots ,N\left(n\right)$ of the graph $G\left(n\right)$ at time n for which radial coordinates $R_n\left(v\right)>r$ exceed some large threshold r. In [30], the tail indices are proposed to be estimated with the Hill estimator without a rigorous justification of the latter for non-i.i.d. data. For the case of dependent data consisting of node degrees from preferential attachment models, the consistency of the Hill estimator has been shown in [60,66] but asymptotic normality of the Hill estimator in this setting remains an open problem.

A conditional dependence is often considered in graphs. Suppose cycles of nodes $c_1$ and $c_2$ may have joint vertices, but they do not have any edge in common. Then for such $c_1$ and $c_2$ the r.v.s $Y_{c_1}$ and $Y_{c_2}$ are conditionally independent given the node weights $W_1$,...,$W_n$ [67]. Here, $Y_c$ denotes the indicator that $c\in I\left(k\right)$ occurs as a cycle in the graph, taking into account two orientations and k starting points, and $I\left(k\right)$ denotes the set of potential cycles of length k.

Conditional independence is tightly linked to graphical models [8,10,68,69,70]. Nodes in graphical models are associated with variables, and the set of edges encodes the conditional independence relations. Let $X={\left(X_i\right)}_{i\in I}$ be a random vector, and $X_A={\left(X_i\right)}_{i\in A\subset I}$ denotes the sub-vector. Then $X_A\perp \perp X_B|X_{I\setminus (A\cup B)}$ is denoted if $X_A$ and $X_B$ are conditionally independent given $X_{I\setminus (A\cup B)}$. Let us consider disjoint subsets $A,B,C$ of the set of vertices V of the undirected graph $G=(V,E)$. In terms of graphs, the conditional independence $A\perp \perp B|C$ implies that all paths from A to B pass through at least one vertex in C [68]. The conditional independence relates to max-stable random vectors. A random vector X is called max-stable if it satisfies the distributional equality $a_{n}X+b_n{=}^{D}max(X^{\left(1\right)},\dots ,X^{\left(n\right)})$ for independent copies $(X^{\left(1\right)},\dots ,X^{\left(n\right)})$ of X for some normalizing sequences $a_n>0$ and $b_n\in R$, where all operations are meant componentwise [70]. In [70], it is shown that for a max-stable random vector with positive continuous density, conditional independence implies unconditional independence.

Considering a sequence of random vectors $X\left(n\right)={(X_1\left(n\right),\dots ,X_d\left(n\right))}^T\in R^d$, where $d=d\left(n\right)$ is a sequence of positive integers, the distribution of the maximal component of X and the distribution of the maximum of their independent copies are proved to be asymptotically equivalent under certain weakly sufficient dependence conditions, i.e.,

$$\begin{array}{ccc}\hfill \left|P\{\underset{i\in \left[d\right]}{max}{X}_i\le x\}-\prod _{i\in \left[d\right]}P\{X_i\le x\}\right|& \to & 0\hfill \end{array}$$

for any fixed $x\in R$; see [11]. The notation $\left[d\right]$ is used for the set of indices $\{1,2,\dots ,d\}$. This property is called extremal independence. The latter means, in fact, the fulfillment of the max-stable property for the maximal component of X. As applications, distributions of various extremal characteristics of binomial random hypergraphs, such as a maximum codegree and a maximum number of cliques sharing a given vertex, are obtained in [11].

5. Network Evolution: Attachment Tools

5.1. Preferential Attachment

A PA, introduced by [71], is the most popular model of network evolution. It reflects a natural attachment of newly appended nodes likely to the most influential ones, i.e., to those with the largest node degrees. Usually, the attachment starts from a single node connected with itself (or a seed network) to evolve a network. The evolving network without node and edge deletion can be represented as a sequence of graphs $G\left(n\right)=\left(V\right(n),E(n\left)\right)$. Typically, the PA leads to the sudden appearance of a giant connected component at a certain critical point that is reflected in a heavy-tailed distribution of vertex degrees [30,72,73]. Using a linear PA, a newly appending node can be attached randomly to an existing node i with probability

$$P_{PA}\left(i\right)=d_i/\sum _{s=1}^{\left|\right|V\left(n\right)\left|\right|}{d}_s$$

proportional to the degree $d_i$ (or a number of nearest neighbors) of node i. Further generalization of the linear PA is given in [74], where a non-linear PA with the attachment probability proportional to the function of the node degree $P_{PA}\left(i\right)\propto f\left(d_i\right)$ is proposed. $A\propto B$ means that A is directly proportional to B, i.e. there exists some constant k such that $A=kB$. Particularly, the power function $f\left(d_i\right)=d_i^{\alpha }$, $\alpha >0$ is studied to generate graphs. Two models of the PA probability for non-directed graphs that depend on the preferential attachment function $f\left(j\right)$, $j\ge 1$ are proposed in [60]. Namely, a new node $v_{n+1}$ may connect to one of the existing nodes $v_i\in V\left(n\right)$ with probability

$$\begin{array}{ccc}& & \frac{f\left(D_i\left(n\right)\right)+\delta }{{\sum }_{i=1}^n(f\left(D_i\left(n\right)\right)+\delta )}\left(\mathrm{Model} \mathrm{A}\right),\hfill \end{array}$$

where $f\left(j\right)$ is assumed to be deterministic and non-decreasing, $D_i\left(n\right)$ is the degree of $v_i$ in the existing graph $G\left(n\right)$, or

$$\begin{array}{ccc}& & \frac{f\left(D_i\left(n\right)\right)+\delta }{{\sum }_{i=1}^n(f\left(D_i\left(n\right)\right)+\delta )+f\left(1\right)+\delta }\left(\mathrm{Model} \mathrm{B}\right),\hfill \end{array}$$

where $\delta >-f\left(1\right)$ is a parameter. A fruitful discussion of Model A can be found in [60]. In particular, three cases of the preferential attachment function f are distinguished. If $f\left(j\right)=j$, $j=1,2,\dots$, then Model A is called the linear preferential attachment model. Degree frequencies have a power-law distribution in this case. The function $f\left(j\right)=j^{\beta }$, $\beta >1$ corresponds to a so-called super-linear case. We refer to [75] for a comprehensive study of this model. Finally, by taking $f\left(j\right)=j^{\beta }$, $0<\beta <1$, we consider a so-called sub-linear case. Then the degree distribution is much light-tailed distributed in comparison with the linear case.

The PA provides the “rich-get-richer” mechanism since earlier appended nodes may increase the number of their edges further. The PA exhibits the scale-free and the small-world properties [76]. Spatial versions exhibit geometric clustering [77,78]. The scale-free property means that the degree distribution of such a network follows a power law for large node degrees. To model large-scale networks, a class of random graphs is defined in [79], which may show the small-world characteristics of “localized” clustering with a longer range of connectivity. Such a graph may be considered as the superposition of many subgraphs, each of whose subgraph contains only edges of a certain range k, which are presented with a power-law probability.

The small-world property [80] implies that the graph distance between different network nodes is typically short. The small-world phenomenon or “six degrees of separation” (i.e., every pair of individuals in the world is separated by only six links on average) is a fundamental issue in social networks where there is an abundance of short paths in a graph whose nodes are people with links joining pairs of people who know one another [80]. The small-world networks have two properties: small average path length and a high clustering coefficient [5]. An ER network displays a very small average path length as small-world networks, but fails in reproducing high clustering coefficients [5]. The Watts–Strogatz model is used in the small-world random network [81]. In this model, if n is equal to the number of nodes, each node is connected to the number of $k/2$ closest clockwise neighbors and also to $k/2$ closest counterclockwise neighbors. This produces a ring that is full of triangles for $k>2$ and, thus, has a high clustering coefficient. The geometric clustering affirms the property that a geometric vicinity of vertices results in a higher probability of establishing an edge between them.

Nodes i and j may be connected with probability $d_i{d}_j/{\sum }_{s=1}^{\left|\right|V\left(n\right)\left|\right|}{d}_s$,[73]. A kind of PA with a Poisson random number of new edges to the new vertex is proposed in [73]. The procedure works as follows. First, considering a sequence of graphs $G_n$ with possible multiple edges and self-loops, one calculates the mean degree ${\mathsf{\Lambda }}_i$ over $G_n$ of each node i which is called capacity, $i=1,\dots ,\left|\right|V\left(n\right)\left|\right|$. Then, each pair of nodes is connected, independently of the other pairs, with $E(i,j)$ edges, where $E(i,j)$ has Poisson distribution with parameter ${\mathsf{\Lambda }}_i{\mathsf{\Lambda }}_j/{\sum }_{k=1}^{\left|\right|V\left(n\right)\left|\right|}{\mathsf{\Lambda }}_k$.

The Spatial Preferential Attachment (SPA) model is introduced in [78]. This model combines the geometry and preferential attachment by introducing “spheres of influence,” which volume grows with the degree of a vertex. Then, the vertex degree distribution was shown to follow a power law.

The linear $\alpha -$, $\beta -$, and $\gamma -$ PA schemes for directed graphs are proposed in [2] and developed in [29,30]. The dynamics of this model depend not only on non-negative parameters $\alpha$, $\beta$, and $\gamma$, but on parameters ${\delta }_{in}>0$ and ${\delta }_{out}>0$ as well. A single directed edge is added to a directed graph at each step of evolution with an appended new node. A finite directed graph $G\left(n_0\right)$ is used as a seed network. It consists of at least one node $v_0$ and $n_0$ edges. A new node v is appended to the existing graph $G(n-1),n>n_0,$ by adding a single edge to $G(n-1)$. Thus, after n steps, $G\left(n\right)$ will be a graph with $n+n_0$ edges and a number of nodes $N\left(n\right)\ge n+1$. The edge creation is provided by flipping a three-sided coin with probabilities $\alpha$, $\beta$, and $\gamma$ such that $\alpha +\beta +\gamma =1$. An i.i.d. sequence of trinomial r.v.s with values 1, 2, and 3 and the corresponding probabilities $\alpha ,\beta ,\gamma \ge 0$ are generated to select one of the following schemes. Furthermore, $I_n\left(v\right)$ and $O_n\left(v\right)$ denote the in- and out-degree of v.

• 1 ($\mathbf{\alpha }-$scheme) The edge $v\to w\equiv (v,w)\in E\left(n\right)$ directed from the new node $v\in V\left(n\right)$ to an existing node $w\in V(n-1)$ is created with probability $\alpha$, while the existing node $w\in V(n-1)$ is chosen with probability

  $$\begin{array}{c}\hfill \frac{I_{n-1}\left(w\right)+{\delta }_{in}}{n-1+{\delta }_{in}N(n-1)}.\end{array}$$

  (15)
• 2 ($\mathbf{\beta }-$scheme) An edge $(v,w)$ is added to $E(n-1)$ with probability $\beta$ and the existing nodes $v,w\in V(n-1)=V\left(n\right)$ are chosen independently from $G(n-1)$ with probability

  $$\begin{array}{c}\hfill \left(\frac{I_{n-1}\left(w\right)+{\delta }_{in}}{n-1+{\delta }_{in}N(n-1)}\right)·\left(\frac{O_{n-1}\left(v\right)+{\delta }_{out}}{n-1+{\delta }_{out}N(n-1)}\right).\end{array}$$

  (16)
• 3 ($\mathbf{\gamma }-$scheme) An edge $(w,v)$ from the existing node $w\in V(n-1)$ to the new node v is created with probability $\gamma$. The existing node w is chosen with probability

  $$\begin{array}{c}\hfill \frac{O_{n-1}\left(w\right)+{\delta }_{out}}{n-1+{\delta }_{out}N(n-1)}.\end{array}$$

  (17)

It is remarkable that the latter schemes allow us to construct multiple edges and loops in the graph.

5.2. Tail Indices of Node Influence Characteristics for Preferential Attachment

The tail indices of the in- and out-degrees of graphs evolved by the linear $\alpha -$, $\beta -$, and $\gamma -$ PA schemes may be calculated using formula (2.9) in [30] (see also [29]):

$$\begin{array}{ccc}\hfill {\alpha }_{in}& =& \frac{1+{\delta }_{in}(\alpha +\gamma )}{\alpha +\beta },{\alpha }_{out}=\frac{1+{\delta }_{out}(\alpha +\gamma )}{\beta +\gamma },\hfill \end{array}$$

where marginals of the in- and out-degrees satisfy a power-law behavior

$$\begin{array}{ccc}\hfill p_i^{in}& \sim & C_{in}{i}^{-(1+{\alpha }_{in})}asi\to \infty ,\hfill \\ \hfill p_j^{out}& \sim & C_{out}{j}^{-(1+{\alpha }_{out})}asj\to \infty \hfill \end{array}$$

for some positive constants $C_{in}$ and $C_{out}$. The parameters ${\delta }_{in}$ and ${\delta }_{out}$ of the PA method (see, Section 5.1) can be estimated with the semi-parametric extreme value method (EV) based on the maximum-likelihood method; see [30].

In [82], a generalized class of random graphs with common dynamics is considered. At every time step, a new vertex appears in the graph, and it connects to $m\ge 1$ existing vertices with a probability proportional to a function of the degrees of the nodes $f\left(d_i(n-1)\right)$, where $d_i(n-1)$ denotes the degree of vertex i in the graph $P{A}_{n-1}$ obtained using the PA with $n-1$ vertices, and $f(·)$ is some PA function. Regarding the original Barabási–Albert model, which is the most popular random graph model for real-life networks, $f\left(k\right)=k$ holds. The function $f\left(k\right)=k+\delta$, for some constant $\delta >-m$, can also be found in applications. In this case, the power-law exponent of the degree distribution is given by $\tau =3+\delta /m$, [7]. In general, every vertex may join the existing network with a random number of edges that connects it to the existing vertices preferentially to their degrees. In [83], it is shown that such a system also exhibits the power-law degree distribution.

The formulae of the tail index of PageRank and the Max-linear model are not yet obtained to the best of our knowledge. In [19], tail indices of the PageRanks and the Max-linear models of superstar nodes in graphs evolved using the linear PA (15)–(17) are investigated with an empirical study. A superstar node within the community is assumed to have incoming links from all nodes of the community. Such nodes may be artificial, and they may not exist in the network. The sums and maxima of PageRanks over communities may serve as PageRanks and the Max-linear models of superstar nodes, respectively. It is shown that the tail indices of PageRanks and the Max-linear models of superstar nodes are close, and they may be approximated by the minimum tail index of PageRanks among representative series containing nodes taken within the communities as their representatives. The novelty of study [19] is that the evolution without node and edge deletion, with the uniform node deletion, and the uniform edge deletion has been studied. Namely, one node or one edge can be deleted at each evolution step when a new node is appended. The number of nodes in the graph is preserved if a node is deleted. Estimates in [84] suggest that the quantity of nodes in the Web graph is growing by a few percent a month. Thus, the mixed deletion policy can be plausible. It is found in [19] that the distribution tails of the PageRanks and Max-linear models of superstar nodes become heavier during the PA evolution without node and edge deletion. This means that the superstars become “richer” and rarer. In contrast, the distribution tails of the PageRanks and Max-linear models of superstar nodes become lighter by the evolution with a uniform node or edge deletion.

In a traditional directed PA, every new edge is added sequentially into the network. However, for real datasets, it can be realistic that several new edges are created at the same timestamp. Previous analyses on the evolution of social networks reveal that after reaching a stable phase, the growth of edge counts in a network follows a nonhomogeneous Poisson process with a constant rate across the day but varying rates from day to day. Taking this result into account, a new modification of the PA model with Poisson edge growth is proposed in [85], and its asymptotic behavior is studied.

5.3. Clustering Attachment

A network may grow through attachment of nodes proportionally to the clustering coefficients, or local densities of triangles of existing nodes. The clustering coefficient is proposed in [13] where the clustering is determined as “the propensity for two neighbors of the same vertex also to be neighbors of one another, forming a triangle of connections in the network.” The local clustering coefficient of vertices with degree k equals to

$$\begin{array}{ccc}\hfill c\left(k\right)& =& \frac{1}{N_k}\frac{2{\Delta }_k}{k(k-1)}\hfill \end{array}$$

(18)

for all k with $N_k\ge 1$, where $N_k$ denotes the number of vertices of degree k, and ${\Delta }_k$ is the number of triangles attached to vertices of degree k [35]. The local clustering coefficient $c\left(k\right)$ is not defined if $N_k=0$, $k\in \{0,1\}$. The asymptotic number of triangles for specific graphs was obtained in [35,86]. The asymptotic behavior of $c\left(k\right)$ for uniform graphs is derived in [35] and for the SPA model in [87], see also Section 5.5.

Usually, one can observe only node connections, but the presence of a network geometry is not always evident. High clustering and triangle counts are not enough to indicate the presence of geometry in the network since closely located nodes (like neighbors living in the same district or working in the same office) may intensively communicate and build triangles in the same way as nodes with high degrees, Ref. [86]. The extremal index cannot reflect the network geometry by indicating a concentration measure of giant nodes as well.

Based on the clustering coefficient of an individual node, a clustering attachment (CA) for undirected graphs is proposed in [88]. The attachment to an existing node i is proceeded with a probability proportional to its clustering coefficient

$$\begin{array}{ccc}\hfill P_{CA}(i,t)& \propto & c_{i,t}^{\alpha }+ϵ,\hfill \end{array}$$

(19)

where

$$c_{i,t}=\left\{\begin{array}{cc}0,\hfill & k_{i,t}=0\mathrm{or}{k}_{i,t}=1,\hfill \\ 2{\Delta }_{i,t}/\left(k_{i,t}(k_{i,t}-1)\right),\hfill & k_{i,t}\ge 2,\hfill \end{array}\right.$$

where $c_{i,t}$ is the clustering coefficient, $k_{i,t}$ is the degree of node i, ${\Delta }_{i,t}$ is the number of triangles involving node i at a time t, $ϵ$ is a constant probability for attachment, which may be zero, and $\alpha \ge 0$ is a parameter of the model. $x\propto y$ means there is a non-zero constant C such that $x=C·y$. Non-negative values of $\alpha$ are considered in [88].

For brevity, we omit further t in notations. Since $k_i(k_i-1)/2$ is the maximum number of triangles that may exist for node i, $0\le c_i\le 1$ holds. In fact, $c_i$ and hence, $P_{CA}\left(i\right)$ are r.v.s. Regarding the PA tool, the ith node degree may only increase, i.e., $k_i=k_i+1$ upon attaching to node i. For the CA one of two cases $(k_i\to k_i+1,{\Delta }_i\to {\Delta }_i)$ or $(k_i\to k_i+1,{\Delta }_i\to {\Delta }_i+1)$ is possible. With (19), one can see that even when a new triangle appears, the clustering coefficient increases only if $k_i>{(1+a)}^2/(1-a^2)$, where $a={\Delta }_i/({\Delta }_i+1)$. In contrast to PA, the CA does not feature a power-law distributed $k_i$ since attaching to a node i most likely drives down the i’s probability for further attachment.

Another novelty introduced in [88] is that each new node may attach not only to one of the existing nodes for the linear PA in [30] but to a fixed number $m_0\ge 2$ of existing nodes. This approach excludes the appearance of multiple edges. An existing node is (uniformly) removed at each step of evolution when a new node is added to preserve a constant number of nodes in the network in [88]. Removal of a node or an edge at each evolution step impacts both ${\Delta }_i$ and $k_i$ and influences the attachment probability to node i.

The CA causes clusters of consecutive exceedances of the evolving modularity over a sufficiently high threshold [88]. The modularity is a measure that allows us to divide graphs into communities. It shows the connectivity of nodes in the community. More precisely, regarding an undirected graph $G=(V,E)$, the modularity Q of a partition C shows how many edges exist within communities and between them [89]:

$$Q=\frac{1}{2s}\sum _{ij}\left[A_{ij}-\frac{k_i{k}_j}{2s}\right]\mathbf{1}(i,j)$$

(20)

Here, $A_{ij}$ is an element of an adjacency matrix ($A_{ij}=0$ if the edge between vertices i and j does not exist and $A_{ij}=1$ otherwise), $s=\frac{1}{2}{\sum }_{ij}{A}_{ij}=\parallel E\parallel /2$ is a half of the edge number in G, $\mathbf{1}(i,j)$ is equal to 1 when nodes i and j belong to the same community, $k_i={\sum }_j{A}_{ij}$ is the degree of node i, Refs. [90,91]. In fact, s increases linearly as $m_0\ge 2$ or it is constant as $m_0=1$ if one edge is removed at each evolution step, and m may remain constant, or it decreases if one node is removed together with its edges. This evolution impacts modularity.

The community detection using modularity maximization using the definition of (20) is unable to find the community structure in networks with many small communities [92]. To overcome the problem, a generalized modularity function is proposed in [93] that can be written in the form

$$Q=\frac{1}{2s}\sum _{ij}\left[A_{ij}-\gamma \frac{k_i{k}_j}{2s}\right]\mathbf{1}(i,j),$$

(21)

where $\gamma$ is the resolution parameter. A similar generalization was proposed previously on different grounds in [94]. What $\gamma$ one should use is discussed in [92].

Regarding directed graphs the modularity can be easily formulated by

$$\begin{array}{ccc}\hfill Q_d& =& \frac{1}{s}\sum _{ij}\left[A_{ij}-\frac{k_i^{in}{k}_j^{out}}{s}\right]\mathbf{1}(i,j),\hfill \end{array}$$

(22)

where $k_i^{in}$ and $k_j^{out}$ denote the in-degree of node i and the out-degree of node j, respectively [91].

Considering the change of the modularity at each evolution step, the extremal index of the modularity sequence indicates the consecutive large connectivity of nodes within the evolving communities, and thus, it reflects the evolution of the communities, [95]. Fluctuations of the modularity over time for $\alpha \ne 0$ in (19) are far larger than for $\alpha =0$; see Figure 1(b) in [88]. The modularity can become very large due to the sparsity in the network.

Denoting the graph at evolution step t as $G\left(t\right)=\left(V\right(t),E(t\left)\right)$, the probabilistic measure

$$P_{CA}(i,t)=\frac{c_i^{\alpha }\left(t\right)+ϵ}{{\sum }_{j\in V\left(t\right)}{c}_j^{\alpha }\left(t\right)+\parallel V\left(t\right)\parallel ϵ}$$

(23)

is used instead of (19). The case $ϵ=0$ is specific. If, in addition, node i does not belong to any triangle of nodes and $c_i=0$ holds not for all $i\in V\left(t\right)$, then $P_{CA}(i,t)=0$ follows by (23). It implies that new nodes cannot be attached to node i.

The simulation in Figure 2 shows that a cluster structure of the time series $\psi \left(t\right)=Q\left(t\right)/\langle Q\rangle -1$, where $\langle Q\rangle$ denotes the average over evolution steps over a time interval, against evolution steps, t is very sensitive to node removals. The creation of a new triangle leads to increasing modularity and, thus, to the appearance of its clusters of exceedances. The CA evolution without node and edge deletion causes a kind of a barbell graph (see, Figure 1 in [95]), i.e., a path of well-conducted cliques that are weakly connected with each other due to bottlenecks. A uniform node deletion at each step of the evolution leads to a large number of isolated nodes since the number of edges decreases due to node removal. The modularity is nearly constant for $\alpha =ϵ=0$ due to a constant attachment probability $P_{CA}(i,t)$ since per (23) it holds

$$P_{CA}(i,t)=\frac{\mathbf{1}\{c_{i,t}>0\}+ϵ}{{\sum }_{j\in V\left(t\right)}\mathbf{1}\{c_{j,t}>0\}+\parallel V\left(t\right)\parallel ϵ},i\in V\left(t\right),\alpha =0.$$

(24)

The extremal index of the evolving modularity and the tail index of node degrees were estimated for a variety of parameters $\alpha$ and $ϵ\in \{0,1\}$ in (23) [95]. The smaller the extremal index is, the larger the clustering (or local dependence) of a modularity sequence. Applying different parameters of the CA both to the cases without the node and edge deletion and with uniform node deletion, it is found that $ϵ=1$ in (23) leads to stable large values of the extremal and tail indices. This feature happens since the clustering coefficient does not impact the CA due to a dominating $ϵ$. It means a weak clustering of the modularity and a light-tailed node degree distribution. Without the node and edge deletion, $ϵ=0$ causes decreasing extremal indices that are close to zero and increasing of the tail indices of node degree as $\alpha >0$ increases [95]. With uniform node deletion, the extremal indices increase, and node degree tail indices increase slower than for the case without edge and node removal. It means that the node deletion leads to a weaker clustering of the modularity and a heavier tail of node degrees than without node and edge deletion.

5.4. Other Models of Random Networks

Some other models to generate random graphs that may fit observed networks are represented in [4,96,97].

In [96], the model is characterized by four stochastic discrete-time processes—the creation processes for node- and edge-creation, and the deletion processes for node- and edge-deletion. It is stated that random copying of edges is a simple stochastic mechanism for creating a Zipfian degree distribution. More exactly, one adds links to a node v by picking a random (other) node u in the graph, and copying some links from u to v. A node is created or deleted independently at each step with some probabilities. With probability $\beta$ one adds k edges from node v to nodes chosen independently and uniformly at random. With probability $1-\beta$, one copies k edges from a randomly chosen node to v. The value k is the parameter to be selected. If the out-degree of u is less than k, its edges are first copied to v, and then by picking another random node $u^{\prime }$ one can copy the required number of edges from $u^{\prime }$ to u until the number of edges of u will not be enough. The processes creating the graphs differ from traditional graph models. However, the process of copying generates complicated dependencies that make analysis very complex.

In [97] such models like a deterministic block model, the configuration random graph model, d-regular random graph, a geometric random graph model apart from the ER, and the SBM described in Section 4.3 are presented as a list of most common models. In [4], much attention is devoted to spatial networks, e.g., the Spatially Embedded Random Networks and the Waxman model.

Spectral density (i.e., the distribution of the graph adjacency matrix eigenvalues) and its kernel estimator are used in [97] to find the best graph model for the observed network in such a way that the divergence between the estimated spectral density and the known limiting spectral density (or one obtained using Monte Carlo) will be the smallest. A rich theory of the kernel estimators of probability density functions can be applied to spectral densities in a similar way. As the divergence measure, one can take the ${\ell }_1$ distance as in [97] or the Kullback–Leibler measure as proposed in [98].

5.5. Triangle Counts and Local Clustering Coefficients

The number of triangles of connected nodes has been studied for specific models of random graphs with known node degree distributions and in the context of different problems.

Network geometry is an important feature of real-world networks since it explains their scale-invariance, high clustering, and overlapping community structures. Indeed, triangles may overlap in real networks. Intuitively, node connections can either be formed between high-weight nodes, or between close-by nodes. When the node degree distribution has a heavy tail, the triangles are formed by high weighted vertices. At the same time, in geometric graphs similar vertices (which could represent people living in the same region and having similar social interests) are likely to connect and form triangles [86]. In [86], it is shown that the triangle counts $\left\{{\Delta }_i\right\}$ and an average clustering coefficient

$$\begin{array}{ccc}\hfill & & {\overline{C}}_t=\frac{1}{n}\sum _{i\in V\left(t\right)}{c}_{i,t}\hfill \end{array}$$

(25)

for $\parallel V\left(t\right)\parallel =n$ nodes, where the clustering coefficient $c_{i,t}$ of node i at the time t is determined per (19), are not enough to detect the presence of geometry in the graph if the node degree distribution has a sufficiently heavy tail. To show this, the inhomogeneous random graph (IRG) and geometric inhomogeneous random graph (GIRG) were compared as benchmark models. For the GIRG model, each node i was equipped with a weight $h_i$ and a uniformly sampled position $x_i$ on a d-dimensional torus ${[0,1]}^d$ endowed with the infinity norm. Weights were sampled independently from the Pareto distribution with the density

$$\rho \left(h\right)=K_1{h}^{-\tau },\tau \in (2,3),$$

for any $h>h_0>0$, where $K_1$ is the normalized constant. Two nodes i and j are then connected independently with probability

$$\begin{array}{ccc}\hfill p\left(h_i,h_j,x_i,x_j\right)& =& K_{2}min{\left(\frac{h_i{h}_j}{\mu n}{∥x_i-x_j∥}^{-d},1\right)}^{\gamma },\hfill \end{array}$$

(26)

for some $\gamma >1$, where $\mu$ is the average weight, $K_2$ is a correction factor, which ensures that the expected degree of any vertex i is proportional to its weight $h_i$. The IRG model is a non-geometric one and it differs from the GIRG in the sense that the nodes possess only weights $h_i$, sampled from the Pareto distribution. They are connected with probability

$$p\left(h_i,h_j\right)=min\left(\frac{h_i{h}_j}{\mu n},1\right).$$

Denoting the number of triangles in a graph G as $\Delta \left(G\right)$, the following asymptotic results are derived in [86]

$$\frac{\Delta \left(G\right)}{n^{3(3-\tau )/2}}\stackrel{\mathbb{P}}{⟶}{A}_{GIRG}$$

for the GIRG, when $\tau <7/3$, and

$$\frac{\Delta \left(G\right)}{n^{3(3-\tau )/2}}\stackrel{\mathbb{P}}{⟶}{A}_{IRG}$$

for the IRG as $n\to \infty$, where $A_{GIRG}$ and $A_{IRG}$ are explicit constants that depend on model parameters. Regarding the GIRG, $\Delta \left(G\right)$ scales as n, when $\tau \ge 7/3$ holds. Since the model parameters, and hence, the constants $A_{GIRG}$ and $A_{IRG}$, cannot be evaluated in practice, one cannot detect the geometry by distinguishing the constants. Similar conclusions concern the average clustering coefficient (25).

If the value of $\overline{C}$ does not vanish in n, then this indicates a geometry in the graph [86]. The average clustering in the GIRG is $\mathsf{\Omega }\left(1\right)$, i.e., it does not vanish as n increases [99]. The asymptotic decay of $\overline{C}$ for non-geometric graphs depends on the tail index $\iota$ in (1), and it may be very slow for some values of $\iota$ [86]. For example, $\overline{C}$ decays asymptotically as $n^{\iota -2}lnn$ for the IRG [7]. Then it is difficult to distinguish between the geometry and non-geometry of the graph in practice.

An important notice is that for $d=1$, $\gamma =\infty$ ($\gamma$ is the parameter in (26)) the GIRG is asymptotically equivalent to the hyperbolic random graph. Thus, the asymptotic results on the GIRG stay valid for the hyperbolic random graph.

As an alternative to the triangle counts and clustering coefficients, Ref. [86] proposes weighing triangles so that triangles with low evidence for geometry have a low weight. The statistic

$$\begin{array}{ccc}\hfill W& =& \sum _{i,j,k\in V,i<j<k}\frac{1}{d_i{d}_j{d}_k}\mathbf{1}\{(i,j,k)=\Delta \},\hfill \end{array}$$

where $\mathbf{1}\left\{\right(i,j,k)=\Delta \}$ is the indicator function of the event that the vertices $i,j,k$ form a triangle, is implemented in [86]. It is significantly greater for GIRGs ($W=O\left(n\right)$) than for IRGs ($W=O\left(1\right)$), and hence, it may serve to detect the network geometry.

Another natural model for real-world networks is given using the uniform random graph (URG). Given a positive integer n and a graphical degree sequence, a sequence of n positive integers $\mathit{d}=(d_1,d_2,\dots ,d_n)$, where ${\sum }_{i=1}^n{d}_i\equiv 0$$\left(mod2\right)$, the uniform random graph is a simple graph, uniformly sampled from the set of all simple graphs with degree sequence ${\left(d_i\right)}_{i\in \left[n\right]}$, Refs. [35,36]. It is assumed that $\mathit{d}$ is a realizable degree sequence, meaning that there exists a simple graph with degree sequence $\mathit{d}$. Let $G\left(\mathit{d}\right)$ denote the ensemble of all simple graphs on degree sequence $\mathit{d}$, and let $d_{max}={max}_{i\in \left[n\right]}{d}_i$, $\left[n\right]=\{1,2,\dots ,n\}$. In [35], ${\left({\mathit{d}}_i\right)}_n$ follows a power-law distribution with exponent $\tau \in (2,3)$. If, in addition, the empirical degree distribution $F_n\left(j\right)=1/n{\sum }_{i\in \left[n\right]}\mathbf{1}\{d_i\le j\}$ satisfies Assumption 1: (i) There exists a constant $K>0$ such that for every $n\ge 1$ and every $0\le j\le d_{max}$, $1-F_n\left(j\right)\le K{j}^{1-\tau }$; (ii) There exists a constant $C>0$ such that for all $j\in O\left(\sqrt{n}\right)$, $1-F_n\left(j\right)=C{j}^{1-\tau }(1+o\left(1\right))$ holds [35], then the number of triangles in the URG follows

$$\begin{array}{ccc}& & \frac{\Delta \left(G\right)}{n^{3(3-\tau )/2}}\stackrel{\mathbb{P}}{⟶}-\frac{1}{12}{\left(\frac{\pi C(\tau -1){\mu }^{-(\tau -1)/2}}{cos\left(\frac{\pi \tau }{2}\right)}\right)}^3,\hfill \end{array}$$

where $\mu =\mathbb{E}\left[\mathit{d}\right]$, and C is a constant. The result is similar to the number of triangles in the erased configuration model, where all multiple edges of the configuration model are merged, and all self-loops are removed

$$\begin{array}{ccc}& & \frac{\Delta \left(G\right)}{n^{3(3-\tau )/2}}\stackrel{\mathbb{P}}{⟶}-\frac{1}{12}{\left(C(\tau -1){\mu }^{-(\tau -1)/2}\mathsf{\Gamma }\left(\frac{1}{2}-\frac{\tau }{2}\right)\right)}^3.\hfill \end{array}$$

The local clustering coefficient $c\left(k\right)$ averaging over the vertices of degree k was intensively studied both theoretically and empirically. In real-world networks $c\left(k\right)\sim k^{-\varphi }$ for some $\varphi >0$ is observed. The asymptotic behavior of $c\left(k\right)$ in (18) is derived for the uniform random graph in Theorem 2 in [35]. For small values of k, $c\left(k\right)$ is independent of k. Then a range of slow decay in k follows. When $k\gg \sqrt{n}$, $c\left(k\right)$ decays as a power of k.

In [100], it is shown that $c\left(k\right)$ can be well approximated using $k^{-1}$ for four large networks. In [34], $\varphi =0.75$, and in [101], $\varphi =0.33$ were obtained. In [87], it was proven for the SPA model that both $c\left(k\right)$ and a local individual clustering coefficient $c_i$ of a vertex i (see, (19)) with a large enough degree k behave as $k^{-1}$.

The convergence with rate $O\left(n^{-1/2}\right)$ in total variation between the distribution of triangle counts and a Poisson distribution in generalized random graphs (GRG) is proved using the Stein–Chen method in [67]. Let $W_i>0$ be the weight of node i. The probability of the edge between any two nodes i and j, for $i\ne j$, is equal to

$$p_{ij}=\frac{W_i{W}_j}{W_i{W}_j+{\sum }_{i=1}^n{W}_i}.$$

Self-loops are prohibited. The node weights ${\left\{W_i\right\}}_n$ can be taken to be either deterministic or random. The ER random graph with $p_{ij}=\lambda /n$ is a special case of the GRG, if $W_i\equiv n\lambda /(n-\lambda )$ for some $0\le \lambda <n$. Let $\mathcal{L}\left(Y\right)$ be a distribution law of Y. For any integer-valued non-negative r.v.s Y and Z, we denote the total variation distance between their distributions $\mathcal{L}\left(Y\right)$ and $\mathcal{L}\left(Z\right)$ by

$$\begin{array}{ccc}\hfill \parallel \mathcal{L}\left(Y\right)-\mathcal{L}\left(Z\right)\parallel & \equiv & \underset{\parallel h\parallel =1}{sup}|Eh\left(Y\right)-Eh\left(Z\right)|,\hfill \end{array}$$

where h is any real function defined on $\{0,1,2,\dots \}$ and $\parallel h\parallel \equiv {sup}_{m\ge 0}\left|h\left(m\right)\right|$. Let $W_i$, $i=1,2,\dots ,n$ be i.i.d. r.v.s distributed as a r.v. W. Let $S_n\left(k\right)$ be the number of cycles of length $k\ge 3$ (cycles are triangles for $k=3$). The main result per [67] (Theorem 1) states that for $k\ge 3$, one has

$$\begin{array}{ccc}\hfill ∥\mathcal{L}\left(S_n\left(k\right)\right)-\mathcal{L}\left(Z_k\right)∥& =& O\left(n^{-1/2}\right),\hfill \end{array}$$

(27)

provided that

$$\mathbb{P}(W>x)=o\left(x^{-2k-1}\right),\mathrm{as}x\to +\infty .$$

$Z_k$ is a r.v. having a Poisson distribution with parameter $\lambda \left(k\right)={\left(\mathbb{E}{W}^2/\mathbb{E}W\right)}^k/\left(2k\right)$. Relation (27) is valid, if W has a power-law distribution or the moment $E{W}^{2k+1}$ is finite.

6. Community Detection Methods and Related Topics

6.1. Partition into Communities

Graph clustering attracts interest from many authors with a different understanding of what the cluster in the graph means [4]. The cluster is determined to be a smaller, denser ER graph planted within an ER graph in [102]. Generally, the community detection, or the graph clustering, consists of partitioning the vertices of a graph into clusters that are more densely connected [45]. Communities may more generally refer to groups of vertices that behave similarly [45]. Numerous existing methods for community detection relate mostly to conductance [22,23] or modularity measures, refs [90,91,92,94] as well as to information theory [103], and Bayesian generative models [104,105].

A Greedy Modularity Maximization Algorithm (GMMA) [90] is used to detect the community structure quickly. To maximize the modularity, Loivan’s algorithm is applied to non-directed graphs [106]. The directed Loivan’s algorithm with adapted modularity (20) is used for directed graphs [91,107].

The monitoring of the formation and evolution of communities in large social networks like Twitter is an important problem. One can study evolving subgraphs corresponding to a topical community. The conductance (see, Appendix D) quantifies how well or poorly connected a subgraph is to the rest of the graph relative to its internal connections. The paper [108] addresses tracking the conductance in real-time since the number of communities that are active at any time is very large, and the rate at which the communities evolve is very high. The minimum conductance might not be the best criterion for local graph clustering [102]. The mean field analysis is based on an aggregation of Web pages into classes according to pairs $k=(k_{in},k_{out})$ of their in- and out-degrees and using averages of PageRanks within each k-degree class to calculate the PageRank [109].

Most of the proposed methods to discover a community structure in networks are unsuitable for very large networks because of their computational cost. The GMMA is a hierarchical agglomeration algorithm for detecting communities, which is faster than many competing algorithms: its running time on a network with n vertices and m edges is $O(mdlog(n\left)\right)$, where d is a depth of the “dendrogram” describing the community structure [90,110]. Since many real-world networks are sparse and hierarchical, with $m\sim n$ and $d\sim logn$, the GMMA runs in essentially linear time, $O\left(n{log}^{2}n\right)$.

Some methods of community detection, including ones based on modularity, return multiple plausible partitions rather than just one. Many of them are similar to one another, differing only by a few nodes. In [111], it is proposed to cluster similar partitions into a small number of groups and then identify an archetypal partition as a representative of each group. This method allows the reduction of a number of partitions into communities.

Some cut-based methods and their relaxation in the form of spectral clustering for separating the graph into $K\ge 2$ groups—such that inside the group, the edge density is higher than between two different groups—are considered in [4]. The Laplacian is used instead of the adjacency matrix for spectral clustering. It is noted that the computation of eigenvectors required for the spectral analysis has complexity $O\left(n^3\right)$ (n is the number of nodes in the graph). In practice, when dealing with a sparse matrix whose eigenvalues are well separated, the complexity can be close to $O\left(Kn\right)$, where K is the number of eigenvectors needed.

6.2. Related Topics

6.2.1. Coloring Random Graphs

A vertex coloring of the graph relates to the community detection color problem. The related detection algorithms are based on the theory of random graph coloring and stochastic decomposition methods of the symmetric adjacency matrix derived from the underlying undirected graph; see [44,45,46,112] among others. Let us focus on directed graphs with possibly non-symmetric adjacency matrices. A survey of polynomial time algorithms that optimally color random k-colorable graphs (including the sparse ones) for a fixed $k\ge 3$ with high probability is given in [46]. The sparsity is governed by a parameter p that specifies the edge probability. The latter probability may be determined using the PA $\alpha -$, $\beta -$, $\gamma -$ schemes recalled in Section 5.1. In [46] a polynomial time algorithm that works for sparse random 3-colorable graphs for a specific p is proposed.

The determining or estimating of the chromatic number $\chi \left(G\right)$ of a graph G, which is the minimum number of colors in a proper vertex coloring, is also connected to the selection of the number of communities in the random graph. The probable value of $\chi \left(G\right)$ is roughly comparable with the average degree of the random graph, and it may be, therefore, much higher than a fixed number of expected colors [44].

6.2.2. Anomaly Detection Using Machine Learning

Community detection and clustering are central problems in machine learning [45]. A comprehensive review of an extreme learning machine is given in [113]. In [114,115], the machine learning algorithms based on extreme value theory are proposed to detect an abnormal class of anomaly observations. Since a training set may not include all possible classes, one may distinguish between known normal data and unknown abnormal test data. Considering a community to be a class may be used to test a new community not observable before.

Let $x_i\in R^p$ be the training data that are labeled as a class $y_i\in \{C_1,\dots ,C_J\}$, $i=1,\dots ,n$. $J\in N$ is the number of different classes in the training set. $p\in N$ is the dimension of the predictor space. The extreme value machine (EVM) introduced in [115] is based on the concept of margin distance of a training point $x_i$ as half of the minimum distance between $x_i$ and all the points belonging to a different class in the training data set:

$$\begin{array}{ccc}\hfill M^{\left(i\right)}& =& \underset{j:y_j\ne y_i}{min}{D}_j^{\left(i\right)}=\underset{j:y_j\ne y_i}{min}\frac{\parallel x_i-x_j\parallel }{2}.\hfill \end{array}$$

A new point is classified as normal if it is inside the marginal distribution of some point in the training set with high probability.

Using the equivalent representation ${\overline{M}}^{\left(i\right)}={max}_{j:y_j\ne y_i}(-D_j^{\left(i\right)})$, it is proposed to use the Fisher–Tippet–Gnedenko theorem to fit to the k largest observed $-D_j^{\left(i\right)}$ for each point $x_i$ a Generalized Extreme Value (GEV) distribution

$$\begin{array}{ccc}\hfill W^{\left(i\right)}\left(z\right)& =& \left\{\begin{array}{cc}exp\{-{\left(-\frac{z}{{\sigma }_i}\right)}^{{\alpha }_i}\},\hfill & ifz\le 0,\hfill \\ 1,\hfill & ifz\ge 0,\hfill \end{array}\right.\hfill \end{array}$$

assuming a zero upper endpoint. $b^{*}=sup\{x\in R:F\left(x\right)<1\}$ denotes the right endpoint of the distribution $F\left(x\right)$. $b^{*}=0$ since ${\overline{M}}^{\left(i\right)}$ is a negated distance. Here, ${\alpha }_i\in R$ and ${\sigma }_i\ge 0$ are the shape and scale parameters, respectively. The latter are to be estimated to obtain ${\widehat{W}}^{\left(i\right)}\left(z\right)$. A new point $x_0$ is assigned as normal, if

$$\begin{array}{ccc}\hfill {\widehat{W}}^{\left(i\right)}(-\parallel x_0-x_i\parallel )& \ge & \delta ,\hfill \end{array}$$

and as abnormal otherwise, where a threshold $\delta$ is chosen by a heuristic formula.

The drawbacks of the EVM are that it strongly relies on the distances between different classes in the training set, the endpoint is assumed to be zero, $\sigma$ is not rigorously selected, and the EVM gives a non-justified premium to normal classes far from the others. In contrast, in the generalized Pareto distribution classifier (GPDC) proposed in [114], the training data are assumed to be sampled from only one class, the class of the normal data points. This is a significant simplification. The upper endpoint is taken to be $b^{*}<0$. The Euclidean distances are used for simplicity. The evaluation of a new point $x_0$ requires computing $O(klogn)$ distances, where k is the biggest number of negated distances $-D_{\left(n\right)},\dots ,-D_{(n+1-k)}$, as far as the EVM needs $O\left(n\right)$ distances.

To apply the latter algorithms to irregular community detection, one can calculate the length of the shortest path between two nodes in links as the distance. To this end, Dijkstra’s algorithm may be applied [116].

6.2.3. Classification of Newly Appended Nodes in Evolving Graphs

An algorithm to classify newly appended nodes in evolving graphs is proposed in Algorithm 1 in [16]. It is based on results from [15], see also Section 3.3.2.

To this end, an initial directed graph with $n>1$ nodes used as a seed network is partitioned into communities. PageRanks of all nodes of the initial graph are calculated. The tail and extremal indices of each community are estimated by non-parametric estimators, e.g., the tail index using the Hill estimator [50] and the extremal index with an intervals estimator [20], see Appendix A and Appendix B. The communities are ranked in ascending order of their tail indices. A new node is assigned to Class 1 if it has a link to one of the most heavy-tailed (“dominating”) communities with the minimum tail index $k_1$. If the new node has a link to communities with the next minimum tail index $k_i$, $i\ge 2$, and it is not linked with communities with smaller tail indices, then it is assigned to Class i.

The minimum tail index $k_1$ and the extremal index ${\theta }_1$ of the “dominating” community are assigned to PageRanks and the Max-linear models of the new nodes from Class 1, if the “dominating” community is unique. If there is a random number of “dominating” communities, then the Max-linear models of the new nodes from Class 1 have the extremal index of the “dominating” community with the maximum PageRank. If arbitrary enumerated sequences of node PageRanks of the “dominating” communities are independent or weakly dependent, more precisely, they satisfy conditions (A1) or (A2) (see, Section 3.3.2), then the PageRanks of the new nodes from Class 1 have the same extremal index as Max-linear models. The conditions (A1) or (A2) for the “dominating” communities provide the same minimum tail index $k_1$ for the PageRanks and the Max-linear models of the new nodes from Class 1. Class 2 obtains the second minimum tail index corresponding to the next set of communities in the range and the respective extremal index as in the previous case, and in the same way, classes with numbers $i>2$ obtain their tail and extremal indices.

Since the tail index is to be estimated, it is plausible to assume that the community with the minimum tail index is unique. However, the tail indices may have close values. One can apply a test statistic (14) to show that the tail indices of the communities are likely different.

7. Leading Nodes for Information Spreading in Evolving Graphs

The influence maximization (IM)—the problem of finding a relatively small optimal set of nodes that have the most influence—is central for many applications, e.g., in social and computer networks for rapid information spreading [22,23] or for studying an epidemic spreading in heterogeneous complex networks [117]. The IM term was first introduced in [118], where a greedy optimization algorithm was proposed. At each stage of the algorithm, the best spreader is chosen sequentially outside the current set of optimal spreaders, which generates the largest increment in the influence of the set of spreaders. The algorithm is computationally costly and can be applied to relatively small networks.

The information spreading, as a a delivery model of messages in the whole network (the full spreading) [23] or in its part (the partial spreading) [22], may be applied to the parallel grid calculations in computation networks. Using a SPREAD algorithm proposed in [23] for undirected stationary graphs and assuming that all nodes have asynchronous or synchronous clocks, a node i that initiates communication at time t is first selected. The node i chooses the next node u randomly among nodes of the network by the global clock tick that works according to a Poisson process. The node u sends all of the messages it has to the node i. In [119], Algorithm 1, the SPREAD algorithm is modified for directed graphs. It is assumed that a single message that has been at the disposal of an initial graph or just a single node has to be spread to a fixed number of the rest nodes. The clocks of the latter nodes are assumed to be asynchronous. The message can be sent from node i to node j if there is a directed edge from i to j. Node j is selected uniformly with a node i among its neighbors without the message. This is proposed to provide with probability $P_{ij}=1/O_i$, where $O_i$ denotes the out-degree of node i and sends its single message to node j.

Let $G=(V,E)$ be an undirected connected graph of order n and $(X_1,\dots ,X_n)$ be a sample of node characteristics with a marginal cumulative distribution function $F\left(u\right)$. A standard graph index used for the leader election is the closeness centrality $C_x$ of node x

$$C_x=\frac{n-1}{{\sum }_{y,y\ne x}d(x,y)},0<C_x\le 1,$$

(28)

where d(x, y) is the shortest path $(x,\dots ,y)$ between nodes x and y [120]. If a node is closer to other nodes, then its value $C_x$ is closer to 1. Using the closeness centrality as a measure of a node’s leadership, the relation between its extremal index and the minimal spreading time is found using a simulation study in [21]. To determine the extremal index of the community S, assuming that the latter index exists, a high quantile of $F\left(x\right)$, the cumulative distribution function of $C_x$, is taken as the threshold $u^{*}$. The community S is assumed to be a strict-sense stationary set with extremal index $\theta$. The stationarity implies that the distribution of all nodes of S remains the same irrespective of the numeration of the nodes.

In [21], the selection of a leading community in a network and the comparison of the spreading time by the latter and other communities are observed using an example of homogeneous geometric undirected graphs. In geometric graphs, two nodes are connected if and only if the distance between them is less than a given radius r [4]. Such graphs cannot describe real-world networks, where there are many nodes of low degrees and a few nodes of extremely high degrees. It is found in [21] that a community with the same extremal index as the entire geometric graph is a leading one and determines the minimal spreading time of the entire graph.

In [119], the linear PA $\beta -$ and $\gamma -$ schemes proposed in [30] and recalled in Section 5.1 are applied to the information spreading purpose in directed graphs. The SPREAD algorithm is also reconsidered for directed graphs. Both the PA and the SPREAD are applied to non-homogeneous graphs. A message from one node is spreading to a given number of nodes in the network. The PA may be a better spreader than the SPREAD algorithm. This result is valid for the sets of the PA parameters with relatively small $\alpha$. The latter means dominating proportions of created new edges from existing nodes to newly appending ones or between the existing nodes only. The information spreading is investigated both for simulated and real non-homogeneous directed graphs generated by the PA with different sets of parameters. Such graphs may contain cycles and multiple edges. It is found in [119] that some nodes in the community with the smallest tail index of the out-degrees and PageRanks may spread the message faster than other nodes.

In [121], it is proposed to divide the network into sectors of influence and then to select the most influential nodes within these sectors.

8. Conclusions and Discussion

In our survey, we focus on the problems arising in evolving networks mostly due to the heavy-tailed nature of node indices. Much attention is devoted to one of the authors’ results devoted to the tail and extremal indices of PageRanks and the Max-linear models that are used as node influence indices in the evolving networks, as well as to the search for leading nodes and communities that may spread the information faster than other nodes in the network. The latter results are based on theoretical results regarding the tail and extremal indices of sums and maxima of random-length non-stationary stochastic sequences obtained in a series of the authors’ papers. Some results are obtained using simulation studies due to their complexity and application to real-world networks.

Related topics like preferential and clustering attachments, community detection, stationarity and dependence of graphs, information spreading, finding the most influential nodes and communities, and concerning methods are surveyed.

There are many unsolved or not sufficiently developed problems yet, like the distribution of triangle and circle counts in evolving networks or the tail index of the PageRank, among many others. The clustering attachment and asymptotic behavior of its characteristics like node degrees and triangle counts, the local dependence of the modularity are not yet well investigated. An application of well-known estimators of the extreme value index (or the tail index) created for random sequences is not rigorously supported for random graphs due to the non-homogeneity and dependence in real-world networks.

A future study may concern reconsideration of the same problems for evolving networks with node and edge deletion. A deeper study of the attachment models may lead not only to power-law distributed node in- and out-degrees and heavy-tailed PageRanks like preferential attachment tools but also to light-tailed distributed ones like the clustering attachment model. More attention may be devoted to new influence measures like the Max-linear model. A new challenge may appear due to the application of machine-learning methods to community detection.

Our future research will concern a deeper study of the CA. We aim to investigate the evolution of random undirected graphs using the CA without node and edge deletion and with uniform node or edge deletion, and to obtain an average clustering coefficient and an attachment probability for the CA.