Intrinsic Correlation with Betweenness Centrality and Distribution of Shortest Paths

Abstract

Betweenness centrality evaluates the importance of nodes and edges in networks and is one of the most pivotal indices in complex network analysis; for example, it is widely used in centrality ordering, failure cascading modeling, and path planning. Existing algorithms are based on single-source shortest paths technology, which cannot show the change of betweenness centrality with the growth of paths, and prevents deep analysis. We propose a novel algorithm that calculates betweenness centrality hierarchically and accelerates computing via GPUs. Based on the novel algorithm, we find that the distribution of shortest path has an intrinsic correlation with betweenness centrality. Furthermore, we find that the betweenness centrality indices of some nodes are 0, but these nodes are not edge nodes, and they characterize critical significance in real networks. Experimental evidence shows that betweenness centrality is closely related to the distribution of the shortest paths.

1. Introduction

Since the new millennium, the informatization tide has swept across the world rapidly [1], and networks now appear in almost all aspects of society as the representatives of informatization. Communication networks [2], transportation networks [3], social networks [4], industrial networks [5], and the Internet of Things [6] have been superimposed on each other to form the network named the “Internet of Everything,” and the prototype of intelligent life has even appeared. Cyberspace has gradually become large and complex, and researchers have paid attention to complex networks [7,8]. Network science has emerged as the times require [9], transforming into influencer science and shifting the focus to network modeling, centrality measures and global characteristics of networks, link prediction and recommendation algorithms based on networks, control and optimization of networks, and network propagation dynamics.

At present, the identification of key nodes and edges in the network has become a multidisciplinary research hotspot, involving areas such as medicine [10], biology [11], geography [12], systems science [13], physics [14], and mathematics [15], and network measure centrality highlights its role in evaluating the importance of nodes and edges [16]. Betweenness centrality is a core index for evaluating the importance of nodes and edges, which has recently been widely used in the networks related to information flow along the shortest path, including medicine [17], biology [18], social network analysis [19], IoT protocol formulation [20], environmental engineering [21], transportation network analysis [22], trade and global supply chain analysis [23], etc. Moreover, the calculation of betweenness centrality was incorporated into $DARPA$ $HPCS$ $SSCA$ in 2008; this index is a benchmark used extensively to evaluate the performance of emerging high-performance computing architectures for graph analysis [24].

Brandes’ algorithm, the best-known algorithm for computing betweenness centrality, is based on [25,26]; however, it cannot fully meet the demand for analysis of complex networks. There is a bottleneck in the existing algorithm, as Brandes’ algorithm is based on single-source shortest path traversal algorithms and the accumulation technique [4]. This bottleneck restricts the in-depth study of betweenness centrality to a certain extent, which masks the trend of betweenness centrality changing with path length, and does not support hierarchical analysis networks.

In this paper, we propose a Novel Algorithm for Hierarchical Analysis Networks (NAHAN), which is developed based on an all-pair shortest paths algorithm named DAWN [27]. We have mathematically demonstrated a strong correlation with betweenness and shortest path distributions, and confirmed the correlation through experiments on multiple networks. Further, we find that there are some special nodes and subgraphs in the network, whose significance cannot be evaluated only by betweenness. Each measure of centrality has its focus and shortcomings. Even nodes with 0 betweenness centrality may play an important role in the network, and evaluating node importance by a single index may no longer be suitable for increasingly complex networks. We discuss how to make up for the lack of betweenness centrality by other measures of centrality, in conjunction with the special cases found in this paper.

The main contributions of this work are as follows:

1.

We propose a novel algorithm that can analyze networks hierarchically and is convenient for conducting in-depth study on betweenness centrality;

2.

We discover the distribution of shortest paths has intrinsic correlation with the betweenness distribution, and experimental evidence confirms this relationship;

3.

We find that the betweenness centrality indices of some nodes are 0, but these nodes are not edge nodes and characterize critical significance in real networks.

In Section 2, we introduce the theoretical foundation of betweenness centrality and its typical algorithms. In Section 3, we elaborate the design of the NAHAN algorithm. In Section 4, we describe the intrinsic relationship with betweenness and the shortest paths distribution. In Section 5, we prove that some special nodes with betweenness centrality indices of 0 have critical significance in the network. In Section 6, we demonstrate that the distribution function of the shortest path and betweenness almost coincide. In Section 7, we conclude the work of this paper.

2. Related Work

In this section, we describe the related work of this paper in three aspects: the concept and properties of betweenness centrality, an introduction to Brandes’ algorithm, and an introduction to DAWN algorithm.

2.1. Betweenness Centrality

Bavelas et al. proposed betweenness centrality in an in-depth study of social networks [28]. Freeman described that the frequency of nodes’ occurrence on the shortest path directly affects their importance in the network, and proposed that evaluating the influence of nodes via betweenness centrality is rigorous [29,30]. Freeman expounded on the mathematical definition of betweenness centrality and the calculation method, which has complexities of $O\left(n^3\right)$ in time and $O\left(n^2\right)$ in space, where n is the number of nodes in the network.

Construct the connected network $G=(V,E)$, where V represents the set of nodes in the network, and E represents the set of edges. Betweenness centrality is usually divided into node betweenness centrality and edge betweenness centrality and has been defined as:    

$$\begin{array}{c}\hfill Bet\left(v\right)=\sum _{v_i,v_j\in V}\frac{\sigma (v_i,v_j\mid v)}{\sigma (v_i,v_j)},\\ \hfill Bet\left(e\right)=\sum _{v_i,v_j\in V}\frac{\sigma (v_i,v_j\mid e)}{\sigma (v_i,v_j)},\end{array}$$

(1)

where $\sigma (v_i,v_j)$ represents the number of shortest paths passing between $v_i$ and $v_j$, and $\sigma (v_i,v_j\mid v)$ and $\sigma (v_i,v_j\mid e)$ represents the number of shortest paths passing through v and e between $v_i$ and $v_j$, respectively.

2.2. Brandes’ Algorithm

At present, Brandes’ algorithm is the best-known algorithm that exactly computes the betweenness centrality, which requires $O(m+n)$ space and runs in $O\left(mn\right)$ and $O(mn+n^{2}lo{g}_{2}n)$ time on unweighted and weighted networks, respectively, where m is the number of links [4]. Erdős, Dóra et al. proposed its optimization [31,32,33]. Brandes’ algorithm has been integrated into a notable open-source project in the network analysis “NetworkX” [34].

Given the network $G=(V,E)$, the betweenness centrality of node v is defined as

$$Bet\left(v\right)=\sum _{s\ne t\ne v\in V}\frac{{\sigma }_{st}\left(v\right)}{{\sigma }_{st}}=\sum _{s\ne t\ne v\in V}Be{t}_{st}\left(v\right),$$

(2)

and Brandes proposed the following formula to calculate betweenness centrality:

$$Be{t}_{s·}\left(v\right)=\sum _{w:v\in P_s\left(w\right)}\frac{{\sigma }_{sv}}{{\sigma }_{sw}}[1+Be{t}_{s·}\left(w\right)],$$

(3)

where $Be{t}_{s·}\left(v\right)$ represents the betweenness centrality of node v, which has the shortest path to other nodes, with node s being the source node in the network.

2.3. DAWN Algorithm

DAWN is a more efficient algorithm based on adjacency matrix operations for solving the all-source shortest paths problem, which requires $O\left(n^2\right)$ space and $O(dim·n^{2.387})$ time, where $dim$ is the diameter of the graph. DAWN can accelerate computing via a multi-GPU system, and its time complexity mainly depends on the number of nodes and is insensitive to the density of the graphs [27].

Feng et al. computed special matrix multiplications and vector multiplications to generate the number and length of shortest paths between all pairs of nodes, and adopted a dual matrix mode to store the calculation results: one for the shortest path lengths and another for the amount of the shortest paths with all pairs of nodes. The DAWN algorithm is expressed as follows on unweighted and weighted graphs:

$$\left\{\begin{array}{c}\begin{array}{cc}{A}_{ij}=\sum _{1\le i\le n}\sum _{1\le j\le n}{a}_{ij}^{\left(k\right)},\hfill & \\ A_{ij}^w=\sum _{1\le i\le n}\sum _{1\le j\le n}{a}_{ij}^{\left(k\right)}+h_{ij}^{\left(k\right)},\hfill \end{array}\hfill \end{array}\right.$$

(4)

where $a_{ij}^{\left(k\right)}$ represents the number of shortest paths between nodes i and j, $h_{ij}^{\left(k\right)}$ represents the number of weighted edges, k represents the length of the path between nodes i and j in the graph, and Formula (4) holds under the condition $a_{ij}^{\left(k\right)}\ne 0\wedge i\ne j\wedge {\sum }_{1\le p\le k-1}{a}_{ij}^{\left(p\right)}=0$.

3. Design of the NAHAN Algorithm

Classic complex network models mainly include the Erdős–Rényi random network model [35], Barabási–Albert scale-free network model [36,37,38], and Watts–Strogatz small-world network model [39,40]. In this section, we introduce the design of the NAHAN algorithm on the unweighted networks and weighted networks.

3.1. Unweighted Networks

The unweighted network $G=(V,E)$ is conveniently described as an adjacency matrix:

$$A=\left[\begin{array}{ccc}{a}_{11}& \cdots & a_{1n}\\ ⋮& \ddots & ⋮\\ a_{n1}& \cdots & a_{nn}\end{array}\right].$$

(5)

The entry $a_{ij}=1$ in adjacency matrix A means that there is an edge with a distance of 1 between node i and node j. We strip the edges connecting v to other nodes from the network, which means setting all the values in row i and column j to 0 in the adjacency matrix A to obtain a matrix $B_v$. We define the iterative formulas of matrices $A^{\left(k\right)}$,$B_v^{\left(k\right)}$ and $C_v^{\left(k\right)}$ as

$$\left\{\begin{array}{c}{A}^{(k+1)}=A^{\left(k\right)}\times A^{\left(1\right)},\hfill \\ B_v^{(k+1)}=B_v^{\left(k\right)}\times B_v^{\left(1\right)},\hfill \\ C_v^{(k+1)}=A^{(k+1)}-B_v^{(k+1)},\hfill \end{array}\right.$$

(6)

where the equation holds under condition $1\le k<k+1\le n-1$. Although the definitions of the three matrices are similar, the slight gaps between them are the focus of the DAWN algorithm. We define them as follows:

1.

The element $a_{ij}^{\left(k\right)}$ in matrix $A^{\left(k\right)}=\left(a_{ij}^{\left(k\right)}\right)$ represents the number of paths with the length k between nodes i and j in the network.

2.

The element $b_{ij}^{\left(k\right)}$ in matrix $B_v^{\left(k\right)}=\left(b_{ij}^{\left(k\right)}\right)$ represents the number of paths with the length k that do not pass through node v when going between nodes i and j in the network.

3.

The element $c_{ij}^{\left(k\right)}$ in matrix $C_v^{\left(k\right)}=\left(c_{ij}^{\left(k\right)}\right)$ represents the number of paths with the length k that pass through node v when going between nodes i and j in the network.

Given the above definitions of the matrices, we propose a new betweenness centrality calculation formula:

$${Bet}_{AM}\left(v\right)=\sum _{1\le k\le n-1\wedge 1\le i,j\le n\wedge i\ne j}{F}_{ij}\left(k\right|v).$$

(7)

We define $F_{ij}\left(k\right)$ as

$$F_{ij}\left(k\right|v)=\left\{\begin{array}{c}\begin{array}{cc}\frac{c_{ij}^{\left(k\right)}}{a_{ij}^{\left(k\right)}},\hfill & a_{ij}^{\left(k\right)}\ne 0\wedge i\ne j\wedge \sum _{1\le p\le k-1}{a}_{ij}^{\left(p\right)}=0\hfill \\ 0,\hfill & \hfill \mathrm{Otherwise}\end{array}\hfill \end{array}\right.$$

(8)

where $i\ne j$ means not to calculate the circular path, which is consistent with the definition of betweenness described in Formula (2). ${\sum }_{1\le p\le k-1}{a}_{ij}^{\left(p\right)}=0$ represents that there is no path in the range $[1,k-1]$, and $a_{ij}^{\left(k\right)}\ne 0$ represents that there is the path from i to j. The paths that satisfy both conditions at the same time are the shortest paths from i to j. Formula (8) uses the matrix to describe Formula (2), which is the definition of betweenness.

The paths in the matrix $F_{ij}\left(k\right|v)$ are directed. The undirected edge can be represented by two directed edges, and the $F_{ij}\left(k\right|v)$ is a symmetric matrix on the undirected networks. To better analyze betweenness centrality, we suggest using the normalized Formula (9),

$$be{t}_{AM}\left(v\right)=\frac{1}{(n-1)\times (n-2)}\sum _{1\le k\le n-1\wedge 1\le i,j\le n\wedge i\ne j}{F}_{ij}\left(k\right|v).$$

(9)

We use pseudocode to describe the algorithm in Algorithm 1. We describe the mathematical foundation of the matrix and provide an example to facilitate the understanding of the algorithm in Appendix A and Appendix B.

Algorithm 1 Unweighted Networks

Input: $A^{\left(n\right)},B^{\left(n\right)}$

Output: $be{t}_{AM}\left(\nu \right)$

1: functionMAIN($A^{\left(n\right)},B^{\left(n\right)},be{t}_{AM}\left(\nu \right)$)  2:   $be{t}_{AM}\left(\nu \right)\leftarrow 0$  3:   for $i=1\to n-2$ do  4:    $A^{(i+1)}\leftarrow A^{\left(i\right)}\times A$  5:    $B^{(i+1)}\leftarrow B^{\left(i\right)}\times B$  6:    $C^{(i+1)}\leftarrow A^{(i+1)}-B^{(i+1)}$  7:   end for  8:   for $k=1\to n$ do  9:    $sum\leftarrow sum+F\left(k\right)$ 10:   end for 11:   $be{t}_{AM}\left(\nu \right)\leftarrow \frac{1}{(n-1)\times (n-2)}\times sum$ 12:   return $be{t}_{AM}\left(\nu \right)$ 13:  end function

3.2. Weighted Networks

The weighted network $G=(V,E,W)$ is conveniently described as an adjacency matrix:

$$A_w=\left[\begin{array}{ccc}{w}_{11}& \cdots & w_{1n}\\ ⋮& \ddots & ⋮\\ w_{n1}& \cdots & w_{nn}\end{array}\right].$$

(10)

Let w be the weight indices of the edges. We assume that $w\in \mathbb{N}$, where $\mathbb{N}$ is the set of natural numbers, and $w_{ij}$ represents the weight of the edge between nodes i and j.

First, we define matrices $A_w^{\left(k\right)}$, $B_{v|w}^{\left(k\right)}$, $C_{v|w}^{\left(k\right)}$ and $H_{A_w}^{\left(k\right)}$:

1.

Matrix $A_w^{\left(k\right)}$ represents the number of paths with the weight k between nodes i and j in the network. Matrix $A_w^{\left(1\right)}$ represents the adjacency matrix corresponding to the subnetwork formed by the edges with the weight 1 in the adjacency matrix $A_w$.

2.

Matrix $B_{v|w}^{\left(k\right)}$ represents the number of paths with weight k that do not pass through node v between nodes i and j in the network. Matrix $B_{v|w}^{\left(1\right)}$ represents the adjacency matrix corresponding to the subnetwork formed by the edges with a weight of 1 that do not pass through node v in the adjacency matrix $A_w$.

3.

Matrix $C_{v|w}^{\left(k\right)}$ represents the number of paths with the weight k that pass through node v between nodes i and j in the network. Matrix $C_{v|w}^{\left(k\right)}$ represents the adjacency matrix corresponding to the subnetwork formed by the edges with the weight 1 that pass through node v in the adjacency matrix $A_w$.

4.

Matrix $H_{A_w}^{\left(k\right)}$ represents the adjacency matrix corresponding to the subnetwork formed by the edges with the weight k in the adjacency matrix $A_w$. If there is no edge with the weight k in the network, then $H_{A_w}^{\left(k\right)}$ is a null matrix, and we also define $H_{A_w}^{\left(1\right)}$ as a null matrix.

Then, we define iterative formulas for matrix $A_w^{\left(k\right)}$ as follows:

$$\left\{\begin{array}{c}\begin{array}{ccc}\hfill A_w^{(k+1)}=& A_w^{\left(k\right)}\times A_w^{\left(1\right)}+H_{A_w}^{(k+1)},\hfill & \hfill 1\le k<k+1\le w_0\\ \hfill A_w^{\left(k\right)}=& A_w^{\left(r\right)}\times A_w^{\left(\lambda w_0\right)},\hfill & \hfill w_0<k<n\end{array}\hfill \end{array}\right.$$

(11)

where $r=k\mathrm{mod}{w}_0$, and $\lambda =\lfloor \frac{k-r}{w_0}\rfloor$. Similar to matrix $A_w^{\left(k\right)}$, we define iterative formulas for matrix $B_w^{\left(k\right)}$ as follows:

$$\left\{\begin{array}{c}\begin{array}{ccc}\hfill B_{v|w}^{(k+1)}=& B_{v|w}^{\left(k\right)}\times B_{v|w}^{\left(1\right)}+H_{B_w}^{(k+1)},\hfill & \hfill 1\le k<k+1\le w_v\\ \hfill B_{v|w}^{\left(k\right)}=& B_{v|w}^{\left(r\right)}\times B_{v|w}^{\left(\lambda w_v\right)},\hfill & \hfill w_v<k<n\end{array}\hfill \end{array}\right.$$

(12)

where $r=k\mathrm{mod}{w}_v$, and $\lambda =\lfloor \frac{k-r}{w_v}\rfloor$. $H_{Bw}^{\left(k\right)}$ represents the adjacency matrix corresponding to the subnetwork formed by the edges with the weight k in the adjacency matrix $B_{v|w}^{\left(k\right)}$, and $H_{Bw}^{\left(1\right)}$ is a null matrix. $w_0$ represents the upper limit of the weight of the weighted network represented by matrix $A_w^{\left(k\right)}$, and $w_v$ represents the upper limit of the weight of the weighted network represented by matrix $B_{v|w}^{\left(k\right)}$.

According to Formulas (11) and (12), we define the matrix $C_{v|w}^{\left(k\right)}$ as

$$C_{v|w}^{\left(k\right)}=A_w^{\left(k\right)}-B_{v|w}^{\left(k\right)}.$$

(13)

The calculation processes of the unweighted and weighted networks are the same except for the different matrix definitions and iterative formulas. Finally, we calculate the betweenness centrality of unweighted and weighted networks with Formula (7)–(9), respectively.

4. Distribution of Betweenness and Shortest Paths

Barabási–Albert scale-free networks and Erdős–Rényi random networks are the classic complex network models [35]. Betweenness centrality represents the frequency of nodes’ occurrence on the shortest path in the network. Previously, we studied the relationship of the betweenness and the average shortest path, which provided ideas for the research in this paper [41,42,43]. We obtain the following identity:

$$\begin{array}{cc}\hfill & \sum _{v\in V}bet\left(v\right)=l-1,\hfill \\ \hfill & \sum _{e\in E}bet\left(e\right)=l,\hfill \end{array}$$

(14)

where l represents the average shortest path of the networks.

The previous research proves the betweenness identities by exemplifying some special small-scale networks, due to being limited by the insufficiency of methods. In this section, we mathematically demonstrate the hierarchical correlation with betweenness and the distribution of shortest paths, which is an essential finding and makes the sum of betweenness equal to the shortest path average. We can calculate betweenness hierarchically by methodological advances and verify that correlation is ubiquitous and rigorous on real networks.

Katzav et al. studied the shortest paths distribution in the Erdős–Rényi network and proposed a distribution function [44]. Given the close connection between the shortest paths of betweenness centrality, we assume that there is a correlation between the distribution functions of the contribution value of betweenness centrality and the shortest paths on the general networks. First, we perform the sum operation on the definition of betweenness centrality:

$$\sum _{v\in V}Bet\left(v\right)=\sum _{i\ne j\in V-\left\{v\right\}}\frac{\sigma (i,j|v)}{\sigma (i,j)},$$

(15)

and we obtain the derivation according to the formula (18):

$$\sum _{v\in }Bet\left(v\right)=\sum _{v\in V}{F}_{ij}\left(k\right|v)=\sum _{v\in V}\sum _{1\le i\le n\wedge 1\le j\le n}\frac{c_{ij}^{\left(k\right)}}{a_{ij}^{\left(k\right)}}.$$

(16)

As the path length grows, the corresponding number of paths changes. To evaluate the effect of paths of different lengths on the betweenness distribution, we introduce the concept of k-order contribution value and use $D_{Bet}\left(k\right|v)$ to represent. We calculate the value within the current F matrix to obtain an intermediate value, and define this intermediate value as the k-order contribution values of betweenness centrality. The k-order contribution values represent the sum of the betweenness of all nodes under the condition that the length of shortest paths is k. We give the definition formula:

$$D_{Bet}\left(k\right|v)=\sum _{i\ne j\in V-\left\{v\right\}\wedge dist(i,j)=k}\frac{\sigma (i,j|v)}{\sigma (i,j)},$$

(17)

where $D_{Bet}\left(k\right|v)$ represents the k-order contribution values of node v, and $dist(i,j)$ represents the length of the shortest paths between i and j. We notice that the function $D_{Bet}\left(k\right|v)$ is a piecewise function that is only related to k after summing all nodes in the networks:

$$D_{Bet}\left(k\right)=\sum _{v\in V}{D}_{Bet}\left(k\right|v)=\sum _{v\in V}\sum _{i\ne j\in V-\left\{v\right\}\wedge dist(i,j)=k}\frac{\sigma (i,j|v)}{\sigma (i,j)}.$$

(18)

There are $k+1$ nodes in the paths of length k, and the head and tail nodes of the paths are not included in the computations of betweenness centrality, which makes the number of nodes included in the calculation in each path $k-1$. Based on the derivation, we simplify the above equation as

$$D_{Bet}\left(k\right)=\sum _{v\in V}\sum _{i\ne j\in V-\left\{v\right\}\wedge dist(i,j)=k}\frac{\sigma (i,j|v)}{\sigma (i,j)}=\sum _{i\ne j\in V-\left\{v\right\}\wedge dist(i,j)=k}dist(i,j)-1.$$

(19)

We sum the shortest paths that exist between all pairs of nodes in the network:

$$D_{Bet}\left(k\right)=\sum _{i\ne j\in V-\left\{v\right\}\wedge dist(i,j)=k}dist(i,j)-1=(k-1)f\left(k\right),$$

(20)

where $f\left(k\right)$ represents the number of shortest paths of the length k.

This is a novel finding that reveals the correlation between the k-order contribution values of betweenness centrality and the shortest paths. We accumulate the k-order contribution values to reveal the relationship of betweenness centrality and average shortest paths:

$$\sum _{v\in V}bet\left(v\right)=\sum _{k=1}^{dim}{D}_{bet}\left(k\right)=\sum _{k=1}^{dim}\frac{(k-1)}{\left(\genfrac{}{}{0pt}{}{n}{2}\right)}f\left(k\right),$$

(21)

where $\left(\genfrac{}{}{0pt}{}{n}{2}\right)$ is the normalization coefficient. We construct the shortest path distribution function:

$$F_{sp}\left(k\right)=\frac{f\left(k\right)}{{\sum }_{k=1}^{dim}f\left(k\right)}=\mu f\left(k\right),$$

(22)

where $\mu$ represents the constant coefficients of $f\left(k\right)$. We notice that there is a weighted relationship between the distribution functions of the shortest path and the k-order contribution values. We define the weighted shortest path distribution function as

$$F_{sp}^w\left(k\right)=\frac{(k-1)f\left(k\right)}{{\sum }_{k=1}^{dim}(k-1)f\left(k\right)}={\mu }^w(k-1)f\left(k\right),$$

(23)

where ${\mu }^w$ represents the constant coefficients of $f\left(k\right)$.

We have mathematically proved that there is a relationship between betweenness centrality and the shortest path distribution.

5. Betweenness Centrality of Special Nodes

Nodes with the betweenness centrality index 0 are ubiquitous in real networks. Researchers typically believe that these nodes appear at the edge of the networks. Combining the distribution of shortest paths, we find that such nodes appear even in the center of the networks. We propose three sufficient conditions for the betweenness centrality indices of nodes to be 0: Nodes have a degree of 1, specific nodes appear in the networks, and nodes appear within a specific connected component. It is generally accepted that the betweenness centrality indices of nodes with a degree of 1 are 0, and we discuss how to make up for the lack of single centrality by combined use of multiple centrality under special circumstances.

5.1. Specific Nodes in the Networks

Theorem 1.

There are specific nodes in undirected networks that form fully connected subgraphs with their first-order neighbor nodes, and the betweenness centrality indices of specific nodes are 0.

Proof of Theorem 1. 

We prove Theorem 1 in the following four steps:

1.

We assume that there is the node 1, which forms the undirected network $G_1$ with the first-order neighbor nodes 2, 3 ($G_1$ is shown in Figure 1);

2.

There is a shortest path of length 1 between nodes 2 and 3, so the betweenness centrality index of node 1 is 0;

3.

Step 2 illustrates that Theorem 1 holds in Figure 1, and we add the node 3, which is potentially connected to any of three nodes;

• When node 4 is connected to node 2 and 3, the betweenness centrality index of node 1 is 0;
• When node 4 is connected to node 1, the shortest path from node 4 to node 2 and 3 passes through node 2, and the betweenness centrality index of node 2 is not 0;
• It is necessary to add edges and nodes, which makes the betweenness centrality index of the node 1, and forms an undirected network ($G_2$), which is shown in Figure 2);
• The network shown in Figure 2 is a fully connected subgraph formed by node 1 and its first-order neighbor nodes.

4.

Arbitrary networks containing the special nodes can be formed by adding nodes and edges to the networks shown in Figure 1 and Figure 2.

In summary, we have proved Theorem 1. □

We have proved that Theorem 1 holds in arbitrary networks containing special nodes, and define the subgraph formed by nodes 1, 2, and 3, as the $B0-triangle$. The isolated nodes in the networks are the special $B0-triangle$, which forms a fully connected subgraph with itself. It is not guaranteed to find the $B0-triangle$ in arbitrary networks, which is why Theorem 1 is irreversible, such as the star networks (shown in Figure 2).

5.2. Fully Connected Component

Theorem 2.

There are fully connected components in undirected networks, and the betweenness centrality indices of nodes in the connected component are 0 except for the nodes connected to the other component.

Proof of Theorem 2. 

We prove Theorem 2 in the following four steps:

1.

We assume that there is a specific connected component, which is a fully connected component and is represented by an undirected network ($G_3$) (shown in Figure 3);

2.

Each pair of nodes in the connected component has the shortest paths of length 1, which makes the betweenness centrality indices of nodes 0;

3.

Step 2 illustrates that Theorem 2 is established in Figure 3; we add node $v_s$ to Figure 3, and it can reach arbitrary nodes under the condition that the shortest path length is 2;

4.

In the case of step 3, in the connected component, except for the nodes connected to the $v_s$, the betweenness centrality indices of nodes is 0.

In summary, we have proved Theorem 2. □

We have proved that Theorem 2 holds on the fully connected components. Theorems 1 and 2 can be associated and verified with each other, but they describe different objects. Theorem 1 guarantees that the betweenness centrality of the special node under certain conditions is 0, and Theorem 2 describes the universal phenomenon that all nodes in fully connected components obey. In real networks, the nodes described by Theorems 1 and 2 usually behave as redundant backup paths and aggregated communities, respectively.

5.3. Contribution of Special Nodes

Theorem 3.

The arbitrary path that is the shortest path with a length greater than 2 and passing through node v contains a shortest path with the length 2 that passes through node v.

Theorem 3 can be proved by the definition of the shortest path, which will not be repeated here. The sum of the frequencies of all nodes appears on the shortest path in the hierarchical network, which we call the k-order contribution value, and the network is layered according to the shortest path length. We rigorously formulate the k-order contribution values using mathematical formulas in Section 4. Combining Theorem 3, we can obtain two corollaries:

1.

For the nodes for which betweenness centrality is not 0, there is at least one shortest path of length 2 passing through the node;

2.

The betweenness centrality indices of the node with a second-order contribution value of 0 must be 0.

We propose a mathematical foundation that supports the optimization of the NAHAN algorithm. The nodes with the second-order contribution value of 0 do not need to continue the operation, which reduces the computation time. The optimization does not affect the accuracy and is only related to the structure of the networks. When the network is extremely sparse, there may be nodes with a degree of almost 1. In the study of real networks, some fully connected components also appear when they are relatively sparse. These nodes can be quickly searched by computing the 2-order betweenness contribution values via the NAHAN algorithm without having to compute the betweenness centrality of the entire network.

5.4. Discussion of Centrality Measures

Each centrality measure has its focus, and it is difficult to fully evaluate the importance of nodes with the single centrality measure. For example, the shortest path may not be selected in a routing network, since the processing power and line bandwidth of the router are limited. It is impossible to rigorously evaluate the importance of all nodes in the real networks through the single centrality measure. In order to evaluate the importance of the node, we need to choose the centrality measures according to the requirements in the scenarios. The importance of nodes can be objectively evaluated by using a combination of multiple centrality measures which considers the effects from multiple aspects.

Centrality measures are mainly defined based on the node neighbors, the paths, the eigenvectors, and the node set shrinkage, such as degree centrality measure [30], Katz centrality measure [45], eigenvector centrality measure [46], and residual closeness centrality measure [47]. The Katz centrality measure considers not only the shortest path between node pairs, but also other non-shortest paths between them, and assigns weights to the paths of different lengths. The Katz centrality measure takes into account the impact of other paths on node importance, and requires $O\left(n^3\right)$ time [48].

The importance of the special nodes we found can be correctly evaluated by using the eigenvector centrality measure and PageRank [49]. We can use the degree centrality measure to correctly evaluate the importance of nodes in the special subgraphs. Nodes in the special subgraphs have the same importance, except for nodes connected to other nodes out of the subgraph. The degree centrality measure can also effectively evaluate the importance of nodes in the global network. The number of links between nodes in the subgraph is fixed, which means that the number of links to nodes outside the subgraph determines the importance of the node.

6. Experimental Results

We verify the fit of the two distribution functions on the Erdős–Rényi networks, the Barabási–Albert networks, and the real networks. The Erdős–Rényi and Barabási–Albert network models were generated by NetworkX, which is widely used for the study of the complex networks [34]. The real network was derived from the Stanford University public dataset, which is a collaboration network covering scientific collaborations between authors of papers submitted to the general relativity and quantum cosmology categories on the e-print arXiv [50].

In order to intuitively reflect the relationship between the contribution value and the shortest path distribution, we normalized them as

$$F_{sp}\left(k\right)=\frac{f\left(k\right)}{{\sum }_{k=1}^{dim}f\left(k\right)},$$

(24)

$$F_{sp}^w\left(k\right)=\frac{(k-1)f\left(k\right)}{{\sum }_{k=1}^{dim}(k-1)f\left(k\right)},$$

(25)

$$F_d\left(k\right)=\frac{D_{bet}\left(k\right)}{{\sum }_{k=1}^{dim}{D}_{bet}\left(k\right)},$$

(26)

and we plotted the shortest path distribution, the weighted shortest path distribution, and the k-order contribution value distribution curve. We used NetworkX to generate an Erdős–Rényi network with 1000 nodes and the connection probability of 0.004; the results are shown in Figure 4.

For the Erdős–Rényi networks, the weighted shortest path distribution and the k-order contribution values distribution line almost completely coincided, showing a good degree of fit. We tested the fitting performance of the k-order contribution value distribution function on the Barabási–Albert network, and used NetworkX to generate a Barabási–Albert network with 1000 nodes and one edge to attach from a new node to existing nodes. The results are shown in Figure 5.

Figure 4 and Figure 5 show that there was a good fit between the distribution functions on the network model generated by calculations. The real networks were not strictly Barabási–Albert networks and Erdős–Rényi networks, so we tested the fit of the distribution function on the real network to verify the generality of predicting the distribution of shortest paths in the network via the k-order contribution values. The results are shown in Figure 6.

We studied the special nodes and connected components that exist in routing networks. A typical example of a special node is a backup route, whose main role is to balance network traffic and improve network survivability. A typical example of a special connected branch is the core of a routing network, consisting of multiple interconnected routers. The routes that are not connected to the outside are mainly responsible for balancing the load of the core network. Even nodes with 0 betweenness centrality may play an important role in the network, and evaluating node importance by a single index may no longer be suitable for increasingly complex networks.

7. Conclusions

In this paper, we propose an algorithm for hierarchical analysis networks, enabling us to study betweenness centrality in depth. We discover that the distribution of shortest path has intrinsic correlation with the betweenness distribution, and verify the fact both on the generated networks and real networks. The intrinsic correlation shows that the shortest paths with various lengths have different effects on the betweenness centrality. Furthermore, we identify that there are some special nodes with betweenness centrality indices of 0 which are not edge nodes. Even nodes with 0 betweenness centrality may play an important role in the real networks, such as backup routes and fixed academic teams. Thus, evaluating node importance by the single centrality measure may no longer be suitable for increasingly complex networks. We discuss that the combination of centrality measures to evaluate the importance of nodes would be more objective and rigorous.